I’m in the process, a very long process, of switching from Vim to BBEdit as my primary editor. The reasons are long and varied, but boil down to me being tired of screwing around with Vim’s configuration. I do a lot of work in Python now, and I’m using the experience of building and maintaining cloudchain to learn how to navigate BBEdit. Hopefully, someday I’ll be as good here as I was with Vim.

Today I learned that BBEdit ships with support for ctags, best defined by the documentation:

Ctags generates an index (or tag) file of language objects found in source files that allows these items to be quickly and easily located by a text editor or other utility. A tag signifies a language object for which an index entry is available (or, alternatively, the index entry created for that object).

The tag file serves two purposes. First, BBEdit will use the tags to allow you to jump to the point in your project where the selected function was defined. Second, if you copy the tags file to a specific spot, BBEdit will use that file for code autocompletion.

• ⌘- -> Find the definition of the selected function.
• ⌘⎇[ -> Jump back to the point you were at in the previous file (if the function was defined elsewhere).

To generate the tags file, open your project directory in Terminal and run bbedit --maketags. Then copy the resulting tags file to ~/Application Support/BBEdit/Completion Sources/Python/tags. Quit and restart BBEdit and autocompletion and function definition should both work.

This looks fantastic.

Today, the team I’m a part of at TargetSmart is releasing our first open source project, a bit of Python I like to call “cloudchain”. cloudchain is designed to make it easy to store and retrieve secrets using AWS. cloudchain relies on the AWS Identity and Access Management (IAM) Key Management Service (KMS) to securely store and manage access to encryption keys, and stores the encrypted secret in a DynamoDB table.

Part of the reason, if not the biggest reason, we are open sourcing this project is to request feedback from the community. cloudchain itself is only a few lines of glue plugging together a few AWS services, but its the idea itself that I’d like vetted. We are using this in a few projects internally, and so far it’s worked out. However, I know that there are things I haven’t thought of, and ways to improve the process, so I’m hoping others will be able to look at the project with fresh eyes and see things we haven’t.

There are three steps in the process. First, cloudchain retrieves an encryption key from KMS and uses it to encrypt the plain text secret. The boto3 library used returns a dictionary with a “Ciphertext” entry containing the encrypted key. cloudchain then base64 encodes the encrypted key into a string, and saves that string to a DynamoDB table named, by default, “safedb”.

Setup

pip install cloudchain

A new encryption key should be created in KMS. Using the console makes this easy, and sets up permissions to the key using IAM users or Roles. IAM users should be given permission individually, while instances launching in AWS should be identified by a role.

A new DynamoDB table should be created as well. Run this command using the AWS CLI tools:

aws dynamodb create-table \
--table-name safedb \
--attribute-definitions \
AttributeName=Service,AttributeType=S \
AttributeName=Username,AttributeType=S \
--key-schema \
AttributeName=Service,KeyType=HASH \
AttributeName=Username,KeyType=RANGE \
--provisioned-throughput \
ReadCapacityUnits=1,WriteCapacityUnits=1 

This will create the DynamoDB table with two attributes: Service and Username. cloudchain assumes that the combination of a service and a username will require a unique secret. The first time a secret is written to the table the third “Secret” attribute is created.

Configuration

The cloudchain cli, cchain, looks for a configuration file at ~/.cchainrc. This should be a standard Python ConfigParser compatible file with the following format:

[dynamo]
region_name = us-east-1
endpoint_url = https://dynamodb.us-east-1.amazonaws.com
tablename = safedb

[IAMKMS]
keyalias = alias/key

The “keyalias” should be the name of the KMS encryption key created during the setup, prefixed by “alias/”. The “endpoint_url” should point at the closest HTTPS endpoint, or at localhost if using a local development environment.

Import cloudchain as a Module

Both the test.py unit tests and the cchain cli import cloudchain.py. After importing, cloudchain expects four variables to be set:

• region_name
• endpoint_url
• tablename
• keyalias

Reasonable defaults are mentioned in the configuration section above, but the keyalias must be unique.

After importing, cloudchain can be called on to encrypt and decrypt secrets:

To Encrypt:

cloudchain.savecreds(args['service'], args['user'], args['save'])

To Decrypt: cloudchain.readcreds(args['service'], args['user'])

Where:

• service = The service name the username and secret are associated with
• user = The username
• save = The unencrypted secret to encrypt

Command Line Use

The command line script supports five arguments:

  -h, --help            show this help message and exit
  -u USER, --user USER  User name
  -e SERVICE, --service SERVICE
						Service or application
  -s SAVE, --save SAVE  Save password to the safe
  -r, --read            Read password from the safe

• The --save and --read arguments are mutually exclusive, and cannot be used at the same time.
• --save expects the unencrypted secret as an argument, and requires both --user and --service flags.
• --user expects the username as an argument.
• --service expects the service name as an argument.
• --read requires no arguments, and requires both --user and --service flags.

Examples

To save a secret:

./cchain -u testuser --service testservice --save testsecreet

To retrieve a secret:

./cchain -u testuser --service testservice --read

We hope this is useful, and that we can continue to make cloudchain better, easier to use, and more secure as development continues.

For the past two months I’ve been working, on and off, with a Rocelco Height Adjustable Standing Desk Riser, a less expensive choice for working at a standing desk than the popular VARIDESK. The Rocelco is a solid alternative for budget conscious workers, but as with most products, the drop in price comes with a set of trade-offs.

Having worked for several months with a VARIDESK, and the past two with the Rocelco, my opinion is that the VARIDESK is simply a better product, and will probably stand up better over the course of several years. The Rocelco does what it advertises, it raises the monitor and keyboard tray up to a reasonable level that feel appropriate to my height. However, the pistons it uses to raise the desk are so strong that you can’t simply pull on the top to raise it and stand away while the desk raises itself. If you do the top shoots up with enough force that when it reaches it’s full height it stops suddenly and shakes.

The first time this happened I was a bit afraid for my monitor. It even managed to shake the desk out of position slightly. If I had a cup of coffee on the desk I’m sure it would have splashed out. The pistons are strong. Once I realized this I remembered from then on to guide the desk to the standing position.

There are no alternative desk heights with the Rocelco, not without engaging the desk locks on each side of the structure. Where the VARIDESK has set points along the path of the raise, the Rocelco has one smooth transition from collapsed to fully expanded, although at any point along the way the locks could, theoretically, be engaged to lock the desk at a specific height, with the mechanics of how the desk raises it would be awkward at best. I’ve not bothered to try.

I’m a bit worried about the long-term prospects of the keyboard tray. The tray seems to be sitting an eighth of an inch lower than it was when I first unpacked the desk, and pulling up on the tray shows that it’s developed a bit of play to it. After two months of on and off use I would expect it to remain solid, I’m not sure what shape it will be in after a year or two. Also, neither the tray nor the desk seem solid enough to support me leaning on it, which, honestly, is a good thing. I shouldn’t be leaning on the desk while working anyway.

Since switching to a sanding desk last year I’ve become accustomed to long periods of standing, and walking around my office to think and work through problems. While I think the Rocelco is a fine starter desk, neither the aesthetics nor the mechanics of it make me happy enough not to start planning it’s replacement. For the next version I’m leaning heavily towards The Wirecutter’s recommendation of the Jarvis Bamboo, but I’m also considering a drafting desk like Dr. Bunsen’s.

Starting a new job working from home gave me the opportunity to evaluate exactly what I wanted from my work environment. I knew I was getting a new Mac (13” Retina MBP, as you do), but I also knew that I needed a new monitor and standing desk.

I worked for a few weeks using nothing but a 15” rMBP on my desk at home, and found that my right wrist was starting to hurt after only a few hours. I’ve had run-ins with RSI before, and had to wear a brace and adjust my posture while typing to relieve it. While my desk works well for the occasional work day at home, spending a few weeks sitting at it proved to me that it’s insufficient for serious extended periods of concentrated work.

However, I was ruined after looking at the beautiful retina screen for as long as I did. At my previous job I used an Apple Thunderbolt Display, which was beautiful and functional, but had lower DPI than the MacBook display, and was not very adjustable. The Thunderbolt could power the MacBook, and acted as a hub for anything that I needed to plug in, like network cable or external hard drive. The Thunderbolt is also $1,000. Given that I look at text all day, and sometimes into the night, I wanted a display that equaled my MacBook, had adjustable height, and wouldn’t break the bank. After researching a few alternatives, I settled on the Dell Ultra HD 4K Monitor P2415Q 24-Inch Screen LED-Lit Monitorat less than half the price.

I’ve read about Dell’s Ultrasharp line before, and always came away with the impression that they were very high quality. The 24” screen has a resolution of 3840 x 2160, the same as the next step 27”, which means the pixels are at a much higher density. After using this one for the past week, I can say that the screen itself is superb. There are no dead pixels, text is crisp and clear, colors are sharp, and the matte finish means that I’m not inadvertently staring at my reflection when working in a dark color scheme.

The screen adjusts to a height level with my eyes, so I’m not hunched over or looking down while working. It also rotates 90°, which I tried briefly and discarded. Just too weird.

While I’m perfectly satisfied with the screen, I do miss quite a few of the niceties of the Thunderbolt Display. There are no integrated speakers, no FaceTime camera, no ethernet port, and no power adaptor for the MacBook. It’s just a monitor. It does have a USB hub, and apparently the display port is capable of carrying audio, but neither feature is integrated into the Mac enough to be useful. For example, if my Time Machine drive was plugged into the monitor when the monitor went to sleep, the drive would be lost to OS X and I’d get that annoying alert about not ejecting a drive. If I need to do video or audio conferencing I’m either going to have to take the Mac out of it’s Twelve South BookArc or I’m going to need to plug in a USB camera, microphone, and speakers like it’s 2003.

I mentioned that the display port can carry audio. The monitor has a speaker jack on it, and I at first plugged in my desk speakers into that, but when I did the Mac lost all capability to control the volume output to the speakers. Hitting F11 or F12 showed an image on the screen indicating that the Mac was just shrugging it’s shoulders. Nothing it could do. Plugging the speakers back into the Mac and setting the headphone jack as the default audio out port solved this issue. I had to remind myself… it’s just a monitor.

The last part of the puzzle is still being shipped. I ordered an adjustable standing desk similar to a VariDesk, but made by a Canadian company for, again, about half the cost. I have the 32” Rodelco ADR on the way, and I imagine I’ll once again be standing for about three-quarters of the day. I spent several months standing while working at my previous job, and I miss that more than the Thunderbolt display. I’ll post again after I’ve worked with it for a week or so with my first impressions.

The 13” is a big change from the 15”, but I don’t feel like I’m missing anything. The 15” seemed an odd size to me. Too big to comfortably use apps full screen, but too small to be able to see more than two windows at the same time. The 13” is perfect for concentrating on a single task, is super light and easy to carry, and has enough power to push the incredible number of pixels on the Dell screen. So far I’m pretty happy with my new setup.

I was listening to Core Intuition a few weeks back and Manton said that if anyone was interested in his project to let him know, and he would send a couple of stickers. I was interested, so I emailed him a quick note, and quickly forgot about it.

Yesterday an envelope arrived in the mail with two stickers and a handwritten note. I’m looking forward to his new project, that Manton Reece seems like a stand-up guy.

Monday I was offered a new position, yesterday I accepted it. I’m hoping that this is the last time I’ll have to look for a job for a very, very long time. Having an unexpected change in your career and having to search for a new job is one of the most stressful things a person can do. It was hard, I didn’t sleep well.

I did get to talk to a lot of interesting and smart people though, and got a view of what their challenges were and what their company was like. Most of the people I spoke with were at a company that was growing fast, and they were looking for help scaling and automating their infrastructure, something I’ve come to specialize in. I applied for thirty-four positions, got back twenty-one responses, spoke with twelve companies, and found three positions where I was a near perfect fit.

Of the companies I spoke with, one of the common themes I heard again and again was “I read your blog.” Through my writing here they were able to take a look back through my history, understand how I convey information, and generally get to know me better. One of the first things I did after I started the search was write two blog posts explaining how I came to be where I am in my career, and what I believe DevOps to be.

Writing these posts served two functions. Most importantly it forced me to think through what I was trying to say, to understand what I think about the topic, and convey that in a way that’s concise and understandable. After writing the posts, when the questions inevitably came up during an interview, I was better able to answer without rambling.

I’ve often wondered if keeping this site up was worth my time, my experience during the past month prove, at least to me, that time spent blogging is time well spent. It’s an investment in my future, a calling card to the world. This site is my little corner of the Internet.

Everyone’s professional path follows a slightly different trajectory. We are each a unique recipe of skills, experience, and interests, which shape who we are and how we come to be in the careers that we have. My experience in moving from a systems administrator to a devops role is unique, because, well, we are all unique.

Somewhere around January of 2010 I saw a shift in the future of the systems administrator role. Automation systems and cloud services were going to change things, and I knew that to stay relevant I was going to have to change too. At the time I felt a bit of an entrepreneurial pull, and decided I was going to be an independent Mac developer. I bought the book, set my alarm for an hour earlier each morning, and spent that hour digging through the book learning Cocoa and Objective-C. I did this for months, reading every page, every chapter, and finishing every programming challenge. At the end of the book I started working on my first app, an adoption of a shell script I called “go”.

Go was basically a bookmark manager, but it let you bookmark anything on your Mac. I used it to quickly SSH to servers, mount NFS shares, launch internal management web apps, and other administrative tasks. I was a bit enamored by the delicious generation at the time, and made some rather, well, regrettable UI choices. When the Mac App Store was announced, I knew that I wanted to get my app in there on day one, so I started working right away on version two, or, Go2. Go2 launched in the MAS in 2011, but never sold enough to make it worthwhile. So, after taking a break from development, I started working on the next app, a static blogging app I called “Paragraphs”. I worked on this app for months, launched it, watched it fizzle out, and decided I needed to rethink my professional goals.

During this same period I went to grad school for Human-Computer Interaction. I still saw that I needed to change my professional career path, I was fascinated by design and the science of how we use these machines, and felt it a personal goal of mine to have a graduate degree. So, nights and weekends for two years I was neck deep in a mix of psychology, Adobe Illustrator, OmniGraffle, Silverback, and Python. I graduated with a Masters degree in HCI in 2012, spent a few months at a start up in Des Moines, went back to CDS, and the following year shut down my development company. All while holding down a full time job and raising a family. To say I was stressed doesn’t even come close.

At CDS I was a Linux sysadmin, but I was also very deeply involved in the design and deployment of the architecture. After my boss quit I applied for and was appointed to a new position titled “Systems Architect Consultant”, which was kind of a fancy term for special projects and development. I had been studying automation, cloud architecture, high availability, and the myriad of other topics surrounding devops, but as the systems architect I was solely devoted to researching those topics and designing solutions based on my research. Around the end of my time there CDS sent me to AWS re:Connect, where I was kickstarted into learning just how big AWS was, and the reality of what I foresaw years before hit me with full force.

I could tell that things were not going well at the company though, and two months before a big round of layoffs I made the move to work for a small software shop named Future Health Software as a systems automation engineer. Unfortunately, between the time I interviewed for the position and the time I was hired, the company was acquired by an investment firm that also acquired Future Health’s biggest competitor, Chirotouch.

Not wanting to concern myself with any of the internal politics or rivalries involved in merging two previously competing companies, I dug into my work and learned everything I could about running production workloads on AWS. I kept an open mind, and learned that the AWS systems were meant to be used like a development framework, bootstrapping an application in the most efficient way possible. Of course, most places hadn’t used the service like this, especially the ones that had been using AWS since all that was available was EC2 and S3. So, I set about rebuilding the entire architecture, six years of it, in CloudFormation code, and developing a deployment system based on CodeDeploy.

After several months of setbacks, wrong turns, and distractions of managing the legacy environment, I had a system that could rebuild the entire infrastructure from scratch. I’m quite proud of that work, and hope to someday put it under a production workload. Unfortunately, before I got the chance, the company closed our office and announced that we were going to be let go. Which brings us to today.

Over the years as I’ve worked out where I want to be in my professional life I’ve learned many different technologies. I’ve spent time digging through the Linux kernel source and reading RFCs, I’ve learned to think pragmatically, read and understand code, and build systems from the ground up. I’ve also learned how the people who use the systems I build actually use them, and how the design choices I make affect the usability and reliability of the system.

For me, the path to get from a sysadmin to a devops engineer travelled from Unix to Linux, through Cocoa and Objective-C, past Python and PHP, stopping in HCI, design, and UX, back to Linux, Docker, and now on to the cloud. And you know, what? It’ll never stop. The tech industry moves fast, you’ve got to keep learning your entire life to stay on the right side of that wave. You’ll either ride it, or be crushed by it, but it’ll never stop.

So, be open to learning new things, try something out of the ordinary, push yourself to try something you haven’t done before, and always do your absolute best. I promise it’ll be a heck of a ride.

The phrase “DevOps” gets thrown around quite a bit, so I thought it might be helpful for me to write down exactly what it means to me. DevOps is the evolution of systems administration. A few years ago, I noticed that the SysAdmin field was finally starting to change, after years of being relatively static. For decades, A sysadmin would set up the hardware, install the operating system, setup SSH (or, telnet in the bad old days), install your application, and get it running. Even when virtualization became more mainstream and worked its way into production workloads, it didn’t change the core tasks of a sysadmin. There were simply more boxes to manage, and without appropriate configuration management, each virtual machine became a unique little snow flake. A few tools became more commonplace like CFEngine, Puppet, or Chef to ease the burden of virtual machine sprawl, but it wasn’t until cloud computing came along that the role of a sysadmin really started to change.  

I realized that the traditional role of a sysadmin was going to change over next few years, and I needed to be on the right side of the this wave of change. Today’s sysadmin needs to know code – because the entire infrastructure is based on code. DevOps as a concept means to me that the mindset of a traditional sysadmin in a data center wasn’t going to cut it for the kind of skills needed for the new environment. Hardware doesn’t matter, operating systems don’t matter, the only thing that matters is the ability to run the application, and run it in a way that is stable, scaleable, and secure. 

One has to completely change the way they think about infrastructure. A common flaw that I’ve seen is trying to treat AWS like your data center. AWS will let you do that, gladly, but it will be expensive. It’s utility computing, like water. You get your water from the local water company, and you pay for what you use. The water company will let you turn on your faucet and just run water all day, but it will charge you for it and you will pay for far more than what you actually need. That’s why you have to have tooling, in this example a faucet to turn the water off and on. In a similar manner, you need automation systems in your infrastructure to scale up and down as needed.  

The other part of the evolution is that you have to stop thinking of the infrastructure as a systems administration task, and instead think of AWS and other cloud providers more like an application programming framework. Tying into building blocks like legos, and picking the appropriate parts to run your application. 

Systems administration is evolving in the same way programming languages have, building up to ever higher layers of abstraction. DevOps is simply working with the developers to build the best possible system to run the application.

And everything is changing for me again. The CTO of the company I work for spoke with me yesterday, our office is being shut down and they are laying off the staff. I’ve got till March 1st to find something new.

If anyone is looking for a good devops guy, let me know!