jb… a weblog by Jonathan Buys

Agility

March 7, 2008

To create the perfect datacenter, what would you recommend? For me, the perfect datacenter would be based on agility. We would be able to add new capacity when needed, and reallocate resources whenever needed, quickly and easily. We would be able to backup everything, securely and easily, off-site. We would use, whenever possible, open source software so we would not be constricted by licensing schemes. Would we have a SAN? Yes, most likely something very simple to administrate, like a NetApp. We would boot from the SAN, have no moving parts in the servers themselves, so we would have very few hardware failures. Whenever possible we would keep to one style of hardware, ie: all blades, or all 1U rack mounts, etc…

We would purchase the servers as needed. Purchasing the equipment instead of leasing it gives us more flexibility, and decreases overall cost. We still abide by the server life cycle, but instead of returning older servers to the vendor, we re-purpose them by migrating them over to test and development, or management servers. Then, when the servers have truly reached the end of serviceable life, drop them on ebay to recoup a bit of the cost. 1 We would purchase racks with built in cooling, fans at top and bottom. We would have an ambient temperature sensor hooked up to Nagios to keep an eye on the environment. Nagios, of course, would keep an eye on everything else as well.

We would run our cabling under the floor in Snake Tray, gigabit Ethernet to the servers, maybe 10 Gig fiber backbone between the switches and routers? It may be expensive to implement, but it would last, and provide more than enough bandwidth. I would build a pair of OpenBSD firewalls with automatic failover and load balancing, one pair for each of two Internet connections. I suppose there would have to be two sets of firewalls on each Internet uplink, to provide a DMZ, a good place for a Snort system.

We would deploy a single operating system, possibly Ubuntu. Something with commercial support if needed, but enough freedom to keep things moving the was we want them to… no restrictions. The Ubuntu server is not bad, and with Canonical providing support its reliable enough to build a business off of. Keep everything at the same patch and kernel level.

Yes, this is a pipe dream. In reality datacenters are heterogeneous, organically grown, and often stuck together with duct tape and bubble-gum. What would we build though, if we had hundreds of thousands of dollars and a blank slate to work off of? If the task were given to me, this is what I would build.


  1. We may not be able to recoup much from eBay, but its better to sell them there than junk them altogether. The servers may end up in a hobbyist’s garage, building the next version of Linux!

The Little Things

March 4, 2008

Today I was out in the data center and decided to boot into Linux to get some work done on my Dell laptop. I was busy populating our internal wiki with hardware and OS data from our servers (how many dimms, what size, kernel level, etc…), which is a lot of work, lots of copy and paste, formating, grepping, going back and forth between the terminal and firefox. Lots of moving around, but not a lot of cpu or memory intensive tasks, just basic office tasks. I’m using Ubuntu, with Gnome and the desktop effects turned on, and I got so frustrated that I booted back into Windows. I hate to say it, but I was able to get more done in Windows today than I could in Linux.

Some of that is my fault, and the choice of applications, but much of it is not. I’m constantly getting the feeling that I don’t have X configured just right, the screen just doesn’t seem sharp enough. Also, I’m quite certain that the gnome interface is slower than Windows explorer. I don’t have any hard data to support this, just my gut feeling and a collection of separate annoyances that I’ve noticed. For instance, at times when dragging a window across the screen the window seems to stutter across the screen instead of flowing smoothly the way it should. I understand that Ubuntu comes with “Bulletproof X”, which is a big improvement in a lot of ways, but there is still no way that it can compare with Quartz.

With Windows, I use the excellent Putty SSH client, coupled with Launchy and exchanged SSH keys results in superfast access to my servers. Also with Putty, all I need to do is select text in the shell with the mouse and it is copied into the clipboard 1. Then it’s a quick Alt-Tab to Firefox and CTRL-P to past into the wiki, and I’m back. Maybe its because I’m more familiar with the interface, maybe its because I’ve spent the time to set up my windows environment, maybe its just because the interface is snappier. I was reading through one of my old Linux Magazine 2 issues when I came across a letter to the editor that said

Linux’s benefits are in its license, not in its interface.

I couldn’t agree more.

I want to use Linux on my laptop, its just that I want it to work the way I want it to work. I want it to be fast, smooth, and more than anything, intuitive. I know I may be asking too much in that last one, but things have been moving steadily towards a more usable interface for a long time. Each year since 2003 I’ve been waiting for the Year of the Linux Desktop, and I think its high time it arrived. I’ve been a fan of Gnome for a long time, I like the interface more than KDE (which I’ve always thought was a bad Windows clone), and Gnome is more fully featured than XFCE (or any of the other minimalist window managers out there). However, since Gnome seems to be running so sluggish on my laptop, it looks as though I’ll either be switching to XFCE, or seriously tweaking my X configuration. Ahh, and I thought those days were behind us.

Yes, Linux, you are getting there, but you’ve still got a ways to go. It’s not the big things anymore (winmodems, anyone?), it’s the little things that get us now. I’m seriously thinking that what we really need is an entirely new window manager, perhaps something not based on the thirty year old code base of X. Maybe its time we threw out everything we think we know about user interfaces and start from scratch. I wonder what awesomeness we could come up with.


  1. I realize that there is probably a configuration file somewhere I could edit (or a checkbox to check?) that would change this behavior. I just don’t know where it is.
  2. I remember when Linux magazine was a little more edgy, and a lot more geeky, back when it’s tagline was “Chronicling the Revolution.”

License Restrictions

February 17, 2008

Software licensing is one of the biggest expenses of high-end server systems. The vendors charge you not only to use the software, but they charge you for how efficiently you want to use the software as well. IBM, for example, charges a different license fee for AIX determined by how many cpus are in the system. So, to scale in response to load, weather its up or out, you have to pay for additional hardware, and then you have to pay for the ability to use that hardware. We are not talking small numbers here either, we are talking in the upwards of six figures 1, in addition to the cost of the hardware. In addition to that, if you are using proprietary applications on top of the OS, you are going to have to pay additional licensing fees for those as well. WebSphere in particular charges on a per cpu basis.

This is where open source solutions really shine. Companies are already going to pay for the superior performance of the hardware. Now with Linux and other open source solutions available to run on the software, and vendors like RedHat supporting them, companies have a whole new world of capability available to them. In a rather strange turn of events, IBM is even supporting Linux on it’s power architecture. The ability to break the chains of restrictive and unnecessary software licensing is well worth the (supposed) trade off in features. Being able to leave the software licensing and keep the hardware is worth its weight in gold. You still need to pay for the software, in a more roundabout way, and, in the end, a much more profitable way.

When a company chooses not to pay software licensing and instead to choose open source software for its needs, what it is choosing is to invest in itself instead of investing in another company. So, instead of paying for support from IBM or HP, a company can pay its employees to train and increase their knowledge, therefore increasing the capability of the company. When you pay an external source to be the technical knowledge base, you are limited to what they are willing to give you. When you invest in yourself, you are limited by your imagination.

For many older companies, ones that have been in the tech sector for the past thirty years or so, the idea of being self-supported is incredibly frightening. They have been used to simply purchasing the software and hardware from a vendor and going back to the vendor when something doesn’t work. For these companies, migrating to a self-supported open source infrastructure would take more than training, it would take a change in corporate culture. This will come easier as the newer companies that are proving the capability of open source become successful. If the culture is unable to change, there is a very good chance that the old companies will be left behind by more agile systems that can respond to the ever increasing rate of change in IT.


Oranges and Oranges

February 16, 2008

A couple of months ago Linux Magazine published an article written by Scott Granneman comparing Mac OS X Leopard and Ubuntu Gutsy Gibbon. I’ve been torn between these two systems, and their respective predecessors for years, so the article was of great interest to me. As a matter of fact, I was wondering how I missed it to begin with. The article is well written, but can be boiled down to this one paragraph:

Linux has come a long way when it comes to ease of use, and it’s definitely getting better all the time, but overall Leopard is still ahead of Ubuntu (and both are way ahead of Vista). Apple makes mistakes, but overall its system is more logical, simple, consistent, and unified than Ubuntu, which still has too many elements that are overly complex, inconsistent, and fractured.

The comment thread on this story is actually very interesting as well, as is the thread on slashdot, that is until the Linux zealots started commenting. One fanatic in particular going by the name of “progon” wrote a very lengthy comment explaining in detail why both Linux and Vista were superior to OS X. This comment stood out to me:

Apple seldom, if ever, listens to its customers and its designers don’t seem to be in sync with their customer base at all.

This is great, especially seeing how Apple did just that with the release of OS X 10.5.2. Its a real shame when people resort to bias when discussing the pros and cons of personal computers, but its really unsurprising. You simply can’t expect to spend a significant amount of time with any tool without developing some type of personal or emotional connection with it.

The conclusion of Scott’s article was that each operating system has its strengths and weaknesses, and that its really up to the user to decide what is most important to them. I agree with him in part. The part that I agree with is that it is up to the user to decide what is most important. The strongest feature that Linux has is its freedom, and with that freedom, it’s configurability. In a way, Linux is infinitely configurable, limited only by the users imagination and technical knowledge. If that freedom and configurability is that important, then Linux most certainly is worth a look. However, to compare the two operating systems solely on the basis of technical points, leaving out any marketing or cost concerns, Linux clearly has no leg to stand on when compared to the big cat.

The more important factor, and one that Linux seems to overlook more often than not, is what the user wants to do with the computer. Apple sells a total package, a computing utility. Linux was built by hobbyists and hackers, and piecemeal together. These design approaches are readily apparent in the gui, and the lengths that each takes to present a usable interface. The important point is not whither the user can perform a certain function, it’s if the user needs the function, and if so, how easy is it for the user to do. Compiz Fusion is undeniably cool eye candy, and about %90 useless for getting any real work done.

One other comment that I see popping up quite a bit when discussing Linux and Macs is that they target two separate markets, and that neither infringes much upon the others. This I think is total bull. There is only one target market for personal computers, the market comprised of everyone thinking about buying one. Which operating system that computer runs is not nearly as much of a concern to the user as being able to quickly and efficiently accomplish their tasks.


No Fail Diet Plan

January 12, 2008

Seriously, does anyone really need to know how to loose weight these days? No matter what super plan you choose, it always comes down to eat less an exercise more. There is no secret formula, and there is no magic pill that will help you achieve the body that you want.

Here’s another truth that no one wants to hear: your weight is your fault. You cannot blame genetics, how you were taught to eat, your job, your wife, your husband, your kids, your dog, the government, or any other outside influence other than yourself. It is true that some people are more prone to weight gain than others, but that does not mean that you can’t do anything about it. Blaming anything except your own habits is a cop-out. To start the 100% guaranteed program, you have to face reality, and you have to take responsibility for your own actions.

So, here’s my super simple, 100% guaranteed, no way you can not lose weight, weight loss plan.

Step one, watch what you eat. That means no McDonalds, Burger King, Hardie’s, or Sonic. Eating fast food makes you fat. I like a double-quarter pounder with fries as much as the next guy, but seriously, you never need that many calories in one setting. I’d say that’s like a once every six months meal, not once a week (or day). If you have to eat out, go to Subway and order a six-inch veggie delight on whole wheat. Seriously, try it, they are really good!

Next, stop right now and go look in your refrigerator and your pantry. OK, how much pre-prepared food did you see? Boxed, one-pan, no cooking skills necessary stuff, you know what I mean. If you are eating ingredients that you cannot pronounce, there is a problem. For example, I cannot think of the last time a recipe called for L-Cystein Hydrochloride. Use raw ingredients as much as possible. Even if the resulting recipe is high in fat and calories, at least you know what’s in it. That leads me to my next point.

Fill up half your plate with veggies. Make a salad that only contains vegetables. Lettuce, carrots, cabbage, broccoli, olives, onions, etc… Fill up a quarter of your plate with a slice of bread or something similar. This is your side dish. That doesn’t mean that it gets its own plate, its there to take up room on your plate, and therefor in your stomach. Now you should have only a quarter of your plate left to fill, that’s for the main course. Also, get a pint glass of ice cold water to go with your meal. Drink the water before drinking anything else.

Speaking of drinking, here’s another hard lesson to learn. Alcohol makes you fat. Alcohol add nothing of value to your diet. As in all things, moderation is OK, but drinking a six pack a night is a sure path to failure. Also, drinking a bottle of wine in one sitting is also very bad for your health. Remember the glass of water at dinner time? Drink that first, and you won’t be as tempted to drink so much beer or wine or whatever you like with your meal.

So, that covers eating and drinking, now for the fun part: exercise! If the sun is shining, turn off the TV, shutdown the computer, and go outside. It’s actually very difficult not to exercise when you are outside, because there is always something to do. At the very least, take a walk. Do some gardening, shovel snow, make a snowman, go out and do something. You can pick up a sport. I’m learning how to swim laps in a pool. It’s not as easy as it sounds, but learning something new is enthralling. I get bored weight lifting and running on the treadmill, but you may really enjoy it.

There are literally thousands of sports, and if you keep looking, you are bound to find one that’s just challenging enough without being too overbearing. However, you must be brave, because here is the final hard lesson to learn. When you try something new, you are going to look like a fool. This is universal, and unavoidable. No matter what it is, no one is good at something the first time they try it. No matter what your chosen exercise is, until you get good at it, you are going to look silly. Be brave, this will pass. Here’s the great thing, the more you do it, the better you get at it, and the less ridiculous you will look doing it. Right now I rather resemble an obese housecat thrown into the pool while I’m learning how to swim. That’s OK, it will pass, the more I swim, the better I get at it.

Nothing worth doing is easy.

Focusing on technique, and actually getting better at something, while watching what you eat, naturally leads to the goal: better health. Better health means you can climb those stairs without getting winded, it means you can go ahead and sign up for the corporate softball league, it means you can keep up with your son when he wants to play football with you in the back yard, and you can keep up with your daughter when she wants to play basketball in the driveway. Better health naturally means a smaller waistline, and less extra pounds to carry around.

Eat less, eat better, and exercise. 100% no fail, guaranteed to work, every time. Period. Now, as for me, I’m going out back to try out my new set of cross-country skis, and play in the snow with my kids. I’ll be back online, but not until after a good workout.


9/11

January 6, 2008

I slept. The world changed all around me, and I slept.

My wife woke me up in a shaking, excited voice, “Jon, get up! Terrorists attacked the World Trade Center!” I remember thinking that she must have been watching a movie of some sort, and I dozed off again. A few minutes later, I got up, put on some sweats and a flannel shirt, and went downstairs to see what she was talking about.Rhonda’s eyes were wet and red with shock. She sat on the edge of our recliner,her knees together, her elbows on her knees, and her chin in her hands. She was watching television, and again I thought that she was watching a movie. Attack on America, the headline said in bright red letters at the bottom of the screen. We were watching CNN.

Suddenly feeling the need not to stand, I took a seat on our couch. The picture on our screen showed the World Trade Centers moldering, smoking, and burning.The reporter spoke of chaos, of passenger airplanes crashing into the Pentagon and into both towers of the World Trade Center. As we watched, the structural integrity of the first tower failed, and the tower collapsed. Rhonda began to cry. I looked at the screen in disbelief. For some reason, I could not get the idea out of my head that we were watching a movie. I had the sensation that we were being fooled, like the Orson Welles 1938 radio broadcast of “War of the Worlds.” I knew, however, that this was not “War of the Worlds”; this was more like Pearl Harbor, this was War.

Not long after the first tower fell, the second collapsed in on itself. During the second tower collapse, America lost many of its bravest men and women; the firefighters and policemen we now herald as fallen heroes. I walked into the dining room to use the phone. I called into work to make sure that the watch was paying attention to the message traffic. Working in military communications, I knew that they would have a lot of correspondence to deal with. My overly excitable Chief answered the phone. I asked to talk to someone on watch, since I was obviously not going to get any answers out of him; he sounded as if he were going to have a heart attack.

“Well, we’re pretty busy, is this really important?’ He asked.

“I just wanna make sure that the watch is keeping a close eye” I responded.

“Oh, yea, we’re really jumpin’ here, everybody’s really busy, we got flash traffic coming all over the place!”

I found out later that he was the only one running around.

“Ok, thanks Chief.”

I got off the phone and returned to the living room.My wife was in tears, horrified and amazed that such a thing could happen. I sat back down on the couch and began to contemplate my own feelings. At first it seemed that I felt nothing. I remember wondering if I were so desensitized to violence that I was incapable of feeling anything. As I watched the firemen, policemen, and volunteers sift through the rubble, a desire came over me, a desire to help. I wanted to do something! I wanted to be in New York or Washington D.C. I wanted to help, to put hand to brick and begin the immediate repair of what the evil men had destroyed. I wanted to be surrounded by the smoke, ash, and debris. I wanted to put goggles over my eyes and a handkerchief over my mouth and make my way though the rubble the fallen towers had left. I wanted to find survivors. I wanted to hear the cries of victims, to smell the burning of the buildings to feel the dust collect on my clothes as I worked day and night to find survivors.

I could do none of those things. I was in my living room, watching television. We watched the news for the rest of the day, and when night came, I slept.


Change and Blessing

March 25, 2007

On Monday, the 23rd, at 9:17 AM, we were blessed with our fourth child, Jacob. Jacob and his Mom are home now, after a few days stay in the hospital at the University of Iowa. Jacob is strong and healthy, waking every few hours to eat and burp and have his diaper changed.

The day before, Sunday the 22nd, our van broke down. And, being the non- mechanic that I am, I ecided that it must be the starter. So, I went down to the auto parts store and picked up a new starter, brought it home and tried, and tried, and tried to install it. After several hours of swearing at the van, I resigned to my fate and packed up my tools. I would have to get the van fixed on Monday.

This caused a problem. My Mom is visiting to help us out while everything is so crazy. She was going to take the kids to school in the van while I took the wife to the hospital for her scheduled C section, and then meet us at the hospital later. With the van out of commission, that plan was no good. So, it was decided to keep the kids home with my Mom until lunch. Then, BJ got sick and threw-up all over our bed, soaking through almost all the way to the mattress. So, since we didn’t have a spare set of clean sheets or covers that day, we decided that Rhonda would sleep with the kids in their room, and I’d pull out the hide-a-way in the couch and stay with BJ just in case. Needless to say, neither of us got much sleep that night.

Monday morning came, my wife and I let the kids sleep in and we drove to thehospital. It was an incredible morning, and I was so glad to see that Jacobwas healthy and strong. Around noon or so, I left the hospital after I wassure that both mother and child were OK and that they simply needed to rest. Ialso had other kids that needed me, and some other issues to take care of.

My wife was not doing as well as I had hoped. The anesthesia that she wasgiven made her very sick, and with the stitches from the C section she wasjust miserable. I stayed with her and Jacob for the evening and then headedback home.

As if this wasn’t enough, I’ve also come to the end of the semester forcollege. I had three term papers to write, two for one class and one foranother. I finished two of them over the weekend, but the last one I didn’tfinish till yesterday.

On Tuesday I dropped off the kids at school and went down to the hospital. Ivisited Jacob and my wife for an hour or so (she was feeling much better),then jumped in my car and headed to Des Moines for an interview. Yes, aninterview. That’s why I needed my dry cleaning, I was picking up my suit. Oncein Des Moines I wanted to print out a fresh copy of my resume, so I went allover town looking for a Kinkos. I finally found one, but their system was notworking at the time, and I had run out of time. I jumped in my car and startedchanging into my suit. I tied my tie in my rear view mirror, and really had noidea how I looked when I stepped out of my car in my suit at Communications Data Services.

The interview went way better than I could have imagined. We talked for almostfour hours and I felt like we really hit it off. I met with the entire teamand we talked Linux and AIX for a while. They must have felt as good about thematch as I did, because by Friday morning I had an offer that I could not turndown. I accepted the offer, and start May 14th.

My wife and Jacob came home on Thursday, and I went back to work for theafternoon. Friday, after I received and accepted the offer from CDS, I gave myboss my two weeks notice. That did not go quite as well as I was hoping itwould, but its for the best.

We are not getting much sleep right now, and probably won’t for the next fewweeks. My wife is going to need our prayers, she is going to be alone with allfour kids for a few weeks while I start work in Des Moines and get everythingready for the move. Never a dull moment in our lives!

So, to sum it all up:

  1. New Baby!! :-)
  2. New Job!! :-)
  3. Broken Van
  4. Sick Kid
  5. College Papers

What a week! I can’t wait to see what next week brings. God has really blessed us and answered our prayers.