QA at DreamHost

I don’t want to brag, but I happen to have one of the coolest jobs in the world. I’m a QA engineer here at DreamHost, but what does that mean? I still get questions about it from time to time, and it’s easy to forget that the entire world doesn’t know all about software development and the roles that different groups play, so I’m going to take a step back and explain a bit about how things work here.

What exactly would you say it is you do here?

Office-Space-homies4-1024x576

QA stands for “Quality Assurance”. It can mean a lot of different things, but the way I like to think of it is “breaking things before our customers”. A gentler way of looking at it: testing changes to our software before we roll them out to production machines, to try to catch things that could go wrong. And there are a LOT of things that can go wrong. Let me walk you through a quick overview!

Ch-ch-changes

david-bowie-changes-resized-600

DreamHost employs a team of developers, working on products like DreamObjects and DreamCompute and all kinds of other projects you may be aware of, such as integration with external services like CloudFlare.

Our developers use git to manage the source code for the projects they work on. Most of the repositores for open-source projects we contribute to (or create) are available on GitHub, but we also have an internal server running Gerrit for code review. Whenever changes are pushed to GitHub or Gerrit, Jenkins runs automated tests written by the developers and QA against the new code. Jenkins integrates with Gerrit via a couple of handy plugins, and automatically votes against changes that break tests.

Happily ever after?

The only problem with this approach is that it only works when there are automated tests for Jenkins to run. Sometimes, the code being changed doesn’t have any automated tests. Before these changes can go live, they need to be looked at by someone other than the developers. As a QA engineer, I need to know a few things, like:

How did this work before? How should it work now? Are there any special setup instructions? Where can I get a good burrito around here?

(Burritos are essential to the QA process.)

Armed with this knowledge, I try to exercise the new feature as much as I can. For example, one of the changes that we’ve made to the panel since I started here is automatic (free!) DNS hosting when a domain is added to an account. Testing this involved registering new domains, as well as transferring domains in from an outside registrar, and making sure that DNS hosting was automatically enabled on the new domain. Usually, things aren’t perfect on the first attempt, so I file bugs against the developers responsible for the new code and work with them to make sure we’re in agreement about what is happening, what should be happening, and what I did that got it to break.

And then…

That’s not quite the entire story, but release engineering is worth a whole post on its own. Hopefully, this post provided a useful high-level overview of QA at DreamHost, and in future entries I’ll get to go into more detail about what kinds of tools we use for automated testing.