OS X Lion is a big step forward in personal computing, and, over the next few years, we are going to see a lot of our preconceptions about how computers work begin to melt away. Apple is setting a high bar for themselves and their developers. Lion is an ambitious release with ambitious goals that are going to take some time to actually come to fruition. However, as futuristic as Lion is, Mission Control feels like a step back.

One of the best features of the Mac since 10.3 been Exposé. Exposé became a part of my workflow so easily that I came to rely on it just by muscle memory. Exposé was always one of the gems of OS X that highlighted how different the mental model of using the computer was when compared to Windows. Unfortunately, with Lion, Apple decided to cut way back on the features that were available in Exposé and mash it together with Spaces. Spaces are Apple’s implementation of multiple desktops, a feature that UNIX and Linux desktops of had for decades. Spaces before 10.7 was fairly good, but I rarely used it.

One situation when I would use Spaces is during server patching. When patching servers it would not be unusual to have 20 or even 30 different terminal windows open at the same time, as well as a chat window, Mail, and Safari. With that many windows open at the same time I would use Exposé to keep track of them all, and I would use Spaces to keep all of the different terminal groups in their own space. On Snow Leopard and before you could activate Spaces, and then drag windows back and forth between Spaces even if you weren’t actually on the desktop that was active at the time. It didn’t matter, they were all equal.

Then, using Spaces’ 10,000 ft. view, you could see all of your Spaces, and all of your overlapping windows. It is interesting to note that this core functionality has not changed in Mission Control. Mission Control does still let you see all Spaces and all overlapping windows in them. In Exposé, the magic was that you could then hit the key command for Exposé and inside of each one of the Spaces, in real time, the windows would all zoom out into Exposé in each of their Spaces to give you a full view of every window in every space that you had opened. Exposé allowed you to drag-and-drop your individual windows between Spaces and watch as the windows in the new space automatically resized and rearranged to make room for the new arrival.

Mission control is almost Exposé; it does give you the ability to see different application windows, but what it takes away is the elegance of Exposé. Instead of a set of miniaturized windows, each a small, live updating thumbnail, mission control overlaps windows from the same application. Worse, applications windows overlap in the order they were used, which is wrong. If you think about it, when you are going into Exposé, you don’t want to see the application window you just used, you want to see the window that was active before you began using the window that you are currently in. I have always used Exposé to switch between tasks, not to switch between applications.

Also gone from Mission Control is the live nature of Spaces. You can only interact with the space you are currently in, you cannot interact with any other open space, except to switch to that space. That is a step back, no matter how you look at.

I can understand why some things needed to change and Spaces. In Lion, the three finger swipe gesture between Spaces would not have worked with Snow Leopard’s implementation of Spaces. With Snow Leopard, Spaces occupied a two-dimensional graph, both horizontal and vertical. Spaces could be either up or down, or left or right. In Lion, Spaces are only horizontal. This works because the best way to move between Spaces is the three finger swipe from left to right, or from right to left. On Snow Leopard Spaces, to be able to use gestures, you would also have to be able to recognize a three finger swipe up for the space above, and a three finger swipe down for the space below. In Lion, those gestures are already accounted for. Not to say that it could not work, but I could see users becoming easily disoriented as to which space they were in, and which way they should be swiping. With Mission Control, when you reach the end of available Spaces, the desktop bounces slightly, a move clearly reminiscent of iOS. However, what does not make sense to me why Apple decided to change Exposé in a way that is clearly inferior.

With that being said, I love the three-finger swipe gesture to activate Mission Control, but I do wish I could see the full Exposé. I find myself using Spaces a more now than I ever did before. I do not use them as additional desktops but I do always have a space open for iCal, another for iTunes, and maybe another for mail. It’s good to see improvements in Spaces, but it is unfortunate the improvements had to come the cost of the most sophisticated window management system on any platform.

Since writing Keyboard Driven Safari I’ve updated my list of extensions that make Safari my browser of choice.

• Shut Up
• YouTube5
• uTube
• Incognito
• Beautifier
• [Optimize Legibility][7]

  update: Optimize Legibility seems to have some performance issues. My Thanks to Dominic Evans for pointing this out.

• Beautipedia
• Type-To-Navigate
• Invisible Status Bar

The combination of shutup.css, YouTube5, and uTube makes for a great experience viewing videos on YouTube. No comments, a very nicely centered layout, and high-quality 720p h.264 video. Beautifier and Optimize Legibility are both very small, and insert some simple css into each page to enhance typography and font rendering for a better reading experience on sites that may not think of these things on their own.

When reading a Wikipedia article on my MacBook, the experience is similar to using Instapaper on my iPad, thanks to Beautipedia. Absolutely gorgeous. I’m not entirely sure I need Incognito, but the idea of being tracked by some gigantic, faceless corporate entity just feels wrong. Incognito blocks Google Adsense and Google Analytics, as well as Facebook tracking.

The next two extensions are new to the list: Type-To-Navigate and Invisible Status Bar, both by Daniel Bergey. Type-To-Navigate lets me type the characters in a link, and hit the return key to “click” the link. Previous to installing this extension, I would use the Safari “find” feature to quickly find a specific link on a page, and then click on the link with the mouse. Type-To-Navigate removes two steps from the process, and the results are fantastic.

Invisible Status Bar borrows from Chrome, and allows me to hide the status bar, while still being able to hover over a link to see the full URL. Very nice to have when reading through long text, and deciding if a link is worth following or not.

The last extension I use is the fantastic 1Password. 1Password is the one of the first three apps I install on a new Mac, and the first I install on a new iOS device.

Dropped from the list is Google Search Cleanup. I’ve switched most of my search to DuckDuckGo, thanks to outstanding keyboard navigation (it supports vi keyboard bindings for navigation). DuckDuckGo is a very small operation taking on giants, I can’t help but root for the underdog.

I have some issues with the design of Lotus Notes. I’m a relatively new user, I started using Notes in 2006, and at the time we were using 6.5 on Windows. I’ve since upgraded to 8.5.2 on Mac OS X.

The very first thing that strikes me is the default “new mail sound” on the Mac:

Now, imagine hearing that sound several times a day. I eventually got tired of having my Mac muted, and copied the new mail sound out of Mail.app and into Lotus Notes. Here is how it sounds now:

Much better. The original sounds like a carousel at a carnival or something. I would love to hear the rational behind that sound.

Secondly, there are preferences. Lots and lots of preferences, spread all over the place. Each day I start Notes at work, and someone will send me a link to a web site. I’ll click on that link, and each day an error will pop up saying “Unable to launch program”. I click “OK”, and then remember that I need to set my default browser in the Notes preferences. That is not a design issue, that is just buggy software. The design issue here is where I need to go to find the preference to change the default web browser.

On a Mac, all well designed applications’ preferences can be opened by pressing the key combination “Command + ,”. Notes also recognizes that key combo, but only opens a subset of the available preferences called “User Preferences”. The User Preferences are also available under a submenu named “Preferences” under the main Lotus Notes menu. In addition to the User Preferences, there are also menu options for Toolbar Preferences and Status Bar Preferences. Each menu option opens up a similar looking window with a plethora of options for changing different aspects of Notes’ appearance or functionality. None of the options change the default browser.

Under the “File” menu, where only functions relevant to the currently opened document should appear, there are four more menus leading to submenus for additional preferences: Application, Replication, Locations, and Instant Messaging. Under the Locations menu, there is a submenu option to “Manage Locations”. This opens a new tab in front of the mail applications tab in the main window of Lotus Notes. Double-clicking on “Online” opens up a third tab with options defining how my Lotus Notes client connects to the Lotus Notes server. This tab has sub-tabs. The fourth sub-tab over is labeled “Internet Browser”. Clicking on this tab shows a screen with one option for choosing the web browser. Double clicking on the name of the browser, or clicking on the edit button towards the top left hand corner of the locations tab will show a drop down menu. Clicking on the drop down menu does not drop down a menu, but opens up another window where I can then choose the default browser. Then, I need to click “Save & Close”, close the Locations tab, and then I’ll be able to click on the link.

This is completely unnecessary. Mac OS X keeps the default browser settings, so all an external application needs to do is pass a URL to OS X (via the NSFileManager class, if you are into Obj-C), and the OS will launch the appropriate application to handle the protocol.

This doesn’t even begin to get into other standard Mac practices like pressing Command-N to start a new email (Notes tries to create a new application?), the lack of spotlight integration, or the fact that none of the controls are standard controls. No syncing with Address Book or iCal, and the text fields do not inherit the standard NSTextField or NSTextView functionality. The UI is literally surrounded by buttons. Lotus Notes is a very powerful application, but all I want to do is check my email and calendar. Every now and again I use it to send a time-off request.

I understand that Notes has a long history, and I understand that it is a cross-platform application. I’ve also read the paper(pdf) on the redesign of Notes, and commend the team on bringing Notes this far. However, I believe the application has a long way to go, assuming a goal of Lotus Notes is to be a viable mail application on Mac OS X, and not just what the company forces the user to use.

I’m willing to help in any way I can.

Apple makes great products, and Brooks Brothers makes great clothes, but neither make the man. It is pure foolishness to judge another person at all, much less using a measure as trifling as a choice of computer.

I woke this morning at 5:20, got dressed, and went outside for my morning workout. Today’s weather was beautiful, perfect temperature, and the smell of fresh rain. Lately I’ve been riding my bike, a Schwinn that is neither strong enough to be a mountain bike, nor sleek enough to be a road bike. I would call it a “small town bike”, as it gets me around all four corners of our small town.

I rode down the street, fast down the slight decline, feeling the wind on my cheeks, blowing the last of the sleep from my mind. I slowed down a little going up a slight incline, starting the muscles in my legs working for the day. Across the highway, down a few cross roads, until I came to the city park, my favorite part of the ride. The park has a small pond with a path around it and a bridge over it. Sometimes I see frogs jump into the pond, and twice now I’ve seen a large bird I believe to be a stork on the far side of the pond.

As I rode around the pond I was distracted by something in the path, something fairly large. Large enough that I was sure it was not supposed to be here. This was unusual. As I rode closer I saw that it was a blanket. My first thought was “why would someone leave a blanket out here”. As I rode closer I began to think that it was shaped oddly like a man.

My thoughts raced back to New Mexico, where I found a man, drunk, lying in the street. He was unconscious, and looked as if he had fallen face-first into a busy city street off the curb at a bus stop. His companions at the bus stop simply looked at him, and continued to wait for the bus. I was driving by on the way to work, but stopped to help.

Gliding by the oddly man shaped blanket I glanced down at it and thought I caught a glimpse of skin under the blanket. I stopped my bike, got off a few feet past it, and could see a bare foot uncovered. I started to feel anxious. There was someone there. I took a small step towards the blanket, thinking about what I would do if it were a homeless man sleeping. I had decided to step up to the blanket and ask if whoever was inside needed help. If it were a hobo, he’d probably say no and yell at me to leave him alone. I did not want to get yelled at, but I also did not want to leave someone if they needed help.

I looked again at the foot, and something about it struck me as odd. Even a hobo would probably have shoes, and this foot looked oddly clean, well kept, and small. A horrible thought ran through my head, something worthy of the opening scene from a prime time crime drama on TV. All of these thoughts raced through my head in the course of a few seconds as I took another step towards the blanket. Then, the blanket was thrown back, and a voice greeted me.

“What’s up? Did you think we were dead?”

Of course. Under the blanket were a pair of teenagers. A boy with a bright, fake diamond earring, and a girl with pink hair. I have a feeling I know what they were up to under there.

“Wasn’t sure.”

I replied.

“Naw, just chilling.”

Said the boy.

The girl giggled. Feeling slightly embarrassed at interrupting their ill placed chilling, I looked at the pond, then looked at the sky, then said:

“Nice place.”

Then got back on my bike and continued on the path.

Another mile down the road I saw two rabbits.

The Experiment

Ideas, Stress, and Time

I remember someone saying once that ideas were worthless, and only implementation mattered. I disagree. Ideas have power, and depending on who you are, that power can be used to either create, or destroy. Or, in my case, both. When I get an idea, the drive to make that idea become reality can consume me. I may do nothing about it for days, weeks, or even months, and then, seemingly suddenly, put all of my time and energy into it in one big explosion. During the time leading up to the creation phase, I’m thinking. At night while I drift off to sleep, during the day when there is a lull in my work flow, during all the little times that something else is not occupying my cognitive functions, I’m thinking about my idea. I consider the idea, I poke it and prod it from as many different angles as I can until I get to a point where I know how to make it real. Or, at least as real as my limited knowledge base can facilitate.

My problem with the thinking/creating loop is that what I’m thinking about and what I’m creating may not be what’s best for me at the time. The offset between what I should be thinking about and what I actually am thinking about creates unnecessary stress in my life, and makes it harder to concentrate on things that I need to be concentrating on, when I need to be concentrating on them. Having an idea that may lead to the accomplishment of a life-long goal is a wonderful thing, if the idea comes at a time when a person can actually act on it without neglecting other commitments. Unfortunately, that is not the situation that I find myself in now.

Masters

Last Fall I enrolled in the Masters of Human-Computer Interaction program at Iowa State University. When I finish this semester I’m in the middle of right now, I’ll be at the half-way point. The program is entirely online, so I can continue to work at my day job as a systems administrator while I work at night on my assignments. The course load is more than I was expecting, and the course work takes far more time than I thought I would need. I spend several hours on the weekends, and several nights each week studying. Some classes have been a lot of fun, others have felt like pushing a bolder up a hill. I’m lucky enough right now to have two classes that I’m enjoying, one on programming in Python, and another on the fundamentals of design.

My hope for the Masters degree is that it will lead to exciting new opportunities in the future. Something that my experience as a systems administrator alone would not. While I enjoy my work, I feel like I could do more, and I do not feel like it is ultimately going to fulfill that life-long goal I was referring to above.

So, I work during the day, and I go to school at night. In between times I try my best to be a good husband and father to my family. We have four kids, and they all need time with Dad. So, we do stuff, fishing, roller-coasters, bike rides, cooking… lots of stuff. Stuff that, really, is more important than anything else. The kind of stuff that gives me a reason to get up in the morning.

My obligations to my family are clear, as are my obligations to my employer, as well as my obligations to Iowa State. However, grad school was not the only thing I started last Fall. I also started Farmdog Software.

Farmdog

Everyone is busy, everyone has obligations, and no one is going to think that I’ve taken on too much, especially those who have started successful software companies. What I’ve told myself for the past nine months is to just suck it up and keep working. Long after everyone else is asleep, I’ve stayed up to keep working on Go2 and Farmdog. Unfortunately, as Dan Benjamin recently said on Back to Work, you can’t start a successful business part time. It needs your full attention, you need to be committed to it.

Farmdog Software has been my dream for a long time. Since first working with my mentor back in England, and learning how he started a successful consulting business, I’ve wanted to work for myself. I’ve been dreaming of working out of my home for twelve years, and my hope with Farmdog was that it would be the catalyst that would finally help me achieve that dream. It has not. It is entirely my fault. I see where I’ve made mistakes, and how my timing was completely off. Underestimating how much time it would take to run the business, and how much time it would take to go to grad school has left me me stressed, unhappy, and drained. My family, my boss, and I all deserve better than that.

Being stretched this thin caused the quality of my work to go down. The most recent build of Go2, 1.3, has glaring bugs that make it unusable for me. While I love my beta testers, and seriously can not thank them enough for finding the faults in Go2, I can not, in good conscience, release 1.3 as it is. It needs a major reworking, and some serious thought into its direction and what it does. I still use it every day at work to launch SSH sessions, which is what I built it for, but I need to think through what the best direction of it as a product is. It needs the kind of thought that I just don’t have the ability to give it right now.

So, Go2 1.3 is not going to be submitted to the App Store. Farmdog Software is going on hiatus until I finish Grad School next year.

Farmdog has been an experiment of sorts, I wanted to see if I could do it, if I could become an “indie developer”. I accomplished what I set out to do, but not well, and not with the kind of dedication the endeavor deserved. As an experiment, we will call Farmdog a success, and a proof of concept that, given the appropriate time and attention, Farmdog can lead to accomplishing my goal. However, right now is not the time. Right now is the time to concentrate on finishing my Masters degree.

The Future

I have many ideas for apps, many that I’d love to start building right now, but they are going to have to wait. While Farmdog is going on hiatus, it is not being abandoned. When I finish my Masters degree, I am going to return to Cocoa with everything I can muster. I am going to leave Go2 in the App Store, it generates one or two sales a week, and if people find it useful I see no reason not to let them have it. So far support has not been an issue, but if it becomes one I’ll pull it. Farmdog is going to stay alive, simply in a holding pattern until I return.

I want to make clear how much I appreciate everyone who’s helped beta test Go2, and how much I appreciate the (very few) customers I’ve been lucky enough to have. If you are one of the awesome few who’ve purchased Go2, I seriously can not thank you enough. You’ve helped make the experiment a success, and given me a direction for the future. If Go2 had not sold at all, or if it had been given a bunch of negative reviews, I probably would have decided my future lay elsewhere.

For the CocoaHeads in Des Moines (and in Cupertino), thank you as well. It’s great to know that there’s a group of people in the area who are willing to help, and to share what they’ve learned.

As I write this, I feel relieved, but the real burden was lifted as soon as I made the decision two days ago. As much as I like to think that I’m superman, I really have only a limited set of abilities, and something had to go.

Daring Fireball linked to Paul Thurrott today, citing Paul’s comment that Lion is simply an evolutionary, rather than revolutionary update. John says:

With Mac OS X in particular, to my recollection, Thurrott has had this same take for every single major (i.e., new cat name) release. Every single one, he’s considered to be “minor” and “evolutionary”.

Sounds like fun, let’s take a look.

Apple Mac OS X 10.1 Review

Microsoft and Apple developed XP and OS X in parallel, and although the first version of OS X (10.0) shipped months ago (in March), the OS wasn’t ready for consumers and casual users until 10.1’s release in late September.

Although Thurrott does not specifically say 10.1 was a minor update in this review, he does address it in the 10.3 review.

Apple Mac OS X 10.2 “Jaguar” Review

Essentially a minor upgrade to previous Mac OS X versions, Jaguar includes numerous small updates and refinements but few major new innovations.

Apple Mac OS X 10.3 “Panther” Review

Over the course of three minor updates (10.1 in late 2001, 10.2 in late 2002, and now 10.3) Apple has done much to strengthen Mac OS X’s underpinnings with a usable fascia that’s bursting with functionality.

Fascia? and…

Panther costs a whopping $129 for most Mac OS X users, although customers who have purchased a new Mac since October 8 can get it for free. That’s a lot of money to pay for mostly subtle refinements that, arguably, should have been in the system to begin with.

Apple Mac OS X 10.4 “Tiger” Review

Mac OS X 10.4 “Tiger” is, in fact, a minor upgrade to an already well-designed and rock-solid operating system. It will not change the way you use your computer at all, and instead uses the exact same mouse and windows interface we’ve had since the first Mac debuted in 1984.

Apple Mac OS X 10.5 ‘Leopard’ Review

While the Apple hype machine and its fanatical followers would have you believe that Mac OS X 10.5 “Leopard” is a major upgrade to the company’s venerable operating system, nothing could be further from the truth. Instead, Leopard is yet another evolutionary upgrade in a long line of evolutionary OS X upgrades, all of which date back to the original OS X release in 2001.

and…

Meanwhile, Leopard is an incremental, evolutionary update over the previous release with no major architectural changes, which makes me wonder why Apple is even charging for it: In the Windows world, such releases are called service packs.

Quick Take: Mac OS X 10.6 “Snow Leopard”

Bottom line: Mac OS X 10.6 “Snow Leopard” is a nice refinement to an already solid OS offering. But it’s almost too evolutionary to get excited about.

And finally, the article in question.

Apple IOS And Apple Mac OS X “Lion”

All in all, Lion doesn’t change much: People buy Macs for the beautiful hardware, not the lackluster OS X user experience. That’s still very much the case.

You’ve really just got to love this guy.

A few years ago, there was a distinction between the “world wide web” and the Internet. The distinction has decreased, but in the wake of Apple’s landmark WWDC keynote, I think the technical distinction between the terms is an important point to make. The difference between the Internet and the web that rides on top of the Internet forms the basis for two very different points of view for the future of personal computing.

Forgive me as I drastically oversimplify a highly complex system.

The Internet

I recently had the privilege of speaking at career day at my kids school. When asked if any of them could explain the Internet, the kids gave a few entertaining answers like “that’s where the Google is”, and “it’s where you can find stuff”. Although most kids knew what a computer was, none knew that the Internet was a collection of computers of varying sizes. Calling the Internet the “Cloud” further abstracts the reality of what the Internet is from what it is perceived as being. The Internet is an interconnecting of devices, both large and small. Many other systems ride on top of the Internet, like email, file transfer, name mapping, and, most popular of all, the world wide web.

The World Wide Web

The World Wide Web uses the hypertext transfer protocol to send hypertext markup language between computers. The markup language is compiled into web pages viewed in a web browser. Web pages contain hypertext links to other web pages, links that branch out like a spider’s web, hence the name, World Wide Web, the “www” part of the naming scheme for web sites.

The Web rides on top of the Internet. Companies like Google make money by selling ads on web sites. So, in their view of the Internet, the web is not only the biggest thing, it’s the only thing, and the more they can draw you into the web browser, the more money they can make.

Who Profits From This

Apple’s recent announcement of iCloud centers around native applications that run on their devices. This is because Apple makes money from selling Macs, iPhones, iPods, and iPads. Apple makes software to make their hardware more appealing to potential buyers. They are a hardware company. On the other hand, Google is an advertising company. They make software to sell ads. They give away the software for free so they can build a large user base to make their advertising space more appealing to corporations.

In Google’s view of the Internet, the “Cloud” is centered on their ability to present the user with ads, therefore the Google view of the Cloud is inside the web browser. Apple, being a hardware company, sees the Cloud as an interconnect between their devices, not a restricted window seen through the web browser. Apple is willing to use the entire Internet to move things around, Google wants to keep you in the browser as much as possible. So much so that they are building an entire operating system that is nothing but a web browser.

I should note that I do not know what file transfer mechanisms iCloud uses. It is very possible that they are using protocols that are built on top of http, and are technically higher up the stack, but what I’m trying to address is the conceptual model. Apple’s iCloud is device centric, Google’s “cloud” is web centric.

In Apple’s view of the Cloud, it is entirely possible that eventually a person could forget they were using the Internet at all. The promise of iCloud is that news, messages, emails, documents, pictures, music, and entertainment could be delivered to all of your devices without thinking about it. Eventually, as this technology is refined, it will be more common to live outside of the web browser than inside, a concept I could see ad companies like Google having a problem with.

I’ve been administering Unix machines for many years now, and frankly, it kinda sucks. It makes me wonder, when sitting in front of a crisp, 2.3 million pixel display (i.e. a laptop) why I’m telling those pixels to draw me a computer terminal from the 80s.

via: [On TermKit

Steven Wittens - Acko.net]1

I too have been administering Unix and Unix-like machines for many years. I admire Steven’s ambition, and his obvious programming and design expertise, but I believe his architecture with TermKit is a bit misguided. TermKit is a combination of Cocoa, Node.js, and WebKit, and while it works, there are a lot of moving parts to get it to work.

Steven seems to have missed the point of the command line, and why we are still use it after all these years. I like using a carpenter’s analogy. Sometimes, you just need a hammer. Sure, there are framing nailers and powder-actuated guns, but sometimes, the only way to get the job done is to hit something with something else hard and heavy. That’s the command line, that’s Unix.

Neal Stephenson uses a better tool analogy. Popping open the Terminal is akin to bringing out the Hole Hawg, it’s ugly, it’s powerful, and it gets the job done.

The most basic point for why Terminal.app still exists, and why the command line is still a preferred tool for systems administration is that Unix is a text-based operating system. All of the configuration files are text, much of the system information is available as text, and all input and output from the commands are text. So, when you are operating on the lowest level of the system, reaching for optimum efficiency, the command line gives you direct access. Anything built on top of that is an unnecessary layer of abstraction.

Being able to “cat” a PDF file is a neat trick, but I have a hard time imagining how it could help revolutionize systems administration.

High Availability, HA for short, refers to an applications ability to continue operating after a hardware failure. HA comes in many different shapes and sizes, but two methods in production today are the presence of multiple machines performing the same task, and pairs of machines in a master-slave setup. Sometimes the master-slave setup is extended to include several slaves, but the main idea is that if the master should go away, the slave will pick up where the master left off, with no interruption in service.

In it’s conception, HA was meant to protect against the physical failure of a machine. With the rise of virtualization technologies in the data center, the concept of protecting against machine failure has been extended to include virtual machines as well as physical. If a physical machine fails, software like VMware and Xen have the ability to seamlessly transfer the virtual machines to another host in the cluster. Virtualization then delivers on its promise of guaranteeing HA by solving a lower problem of protecting the machine, independent of the application running on the machine.

However, virtualization technology has its trade-offs, especially in environments that support web applications. The first is disk space. Since each virtual machine is a unique and self contained operating system, minimum disk requirements must be met per virtual machine. This creates large files that hold virtual file systems, most of which is unrelated to the operation of the application that the virtual machine supports. It is not uncommon for a single virtual machine to be 20GB in size.

There are ways to mitigate this problem. A virtual machine could be created with a disk that automatically grows in size as needed, known as thin-provisioning. This is a dangerous practice though, because it allows the sysadmin to over-allocate his available disk space. If one virtual machine suddenly grows too big, and inadvertently fills the available disk, all of the virtual machines could freeze until more disk is available.

Another problem with virtualization is again related to the point that each virtual machine is an independent operating system. Each operating system must be managed to ensure compliance with standards, logging, user accounts, and all of the myriad other tasks associated with running a production operating system. Again, there are ways to mitigate this problem. Systems like CF engine and Puppet are good examples of system automation, and are a necessity after the data center grows beyond a certain point (where that point is seems unique to the each datacenter).

Each shortcoming of virtualization has been addressed with a solution of one type or another, but they each address the problem of managing operating systems. Remember, the purpose of HA is to ensure that the application remains available, not the operating system. The operating system is an auxiliary requirement, needed of course for the application to run, but receiving an undue amount of attention unrelated to the original purpose of HA. HA is not meant to ensure that the operating system remains available, it is meant for the application residing on the operating system.

VMware and it’s ilk are the current darlings of the datacenter, due in no small part because they do in fact deliver on promises of HA. However, they deliver at the cost of misdirection away from the actual problem.

It is my opinion that virtualization technology solves the problem of HA at the wrong layer. HA should be implemented higher in the stack, at the application layer. Consider a much older technology implemented in FreeBSD, jails: FreeBSD jails pair a filesystem, and IP address, and an application into a tightly controlled bundle. Jails allow you to add multiple applications to a single operating system, while keeping them independent of one another. This allows you to fully utilize the hardware, and removes a layer of abstraction that exists in virtualization. Jails do not provide HA. Building an architecture using jails forces the designer to implement HA farther up the stack. Designers still need to provide for hardware failure, the same as an environment built using VMware does, but they do so using existing mature technologies.

For example, a web cluster uses multiple, nearly identical load balanced web servers. In this scenario, HA for the web site is ensured by providing the same data on all web servers. If the web cluster is a database driven application, the database could ensure HA using a master-slave replication setup. This simple setup ensures HA without the overhead and additional complication of virtualization.

Where jails really shine is in the management of each physical server. Each jail mounts the operating system as a read-only filesystem, with symbolic links to unique read-write directories for each jail. That means that there is only a single operating system to manage for each jail, one OS to patch, one OS to configure. Coupling jails with zfs provides an amazingly simple method of backups, and application distribution. A typical jailed application residing in a zfs filesystem will require less than 500MB of storage space, compared to a VMware image of 20GB. Multiplied by several hundred virtual machines or jails, the difference in size becomes a significant difference in cost.

When building a new environment from scratch, it may be tempting to agree with the vendors and shell out tens of thousands of dollars for VMware licensing and associated hardware. When you do, know that you are solving the problem, but at a much higher cost than necessary.