Imagine you could only compile once a week. Once every two weeks. Once a month. Scary?
All that code that you write blindly, not knowing whether it even compiles. All those hidden errors lurking, accumulating in the dark. Then, on that special day of compilation, you get to finally find out if you made it. And of course, you most likely did not. Unfortunately, that special compilation day is followed by an even worse day. Demo day.
You are supposed to sit down with your customer, or product owner, and show them all the progress you have made to your application. Most times, compilation day goes sour, and you are stuck fixing last minute, last hour, and last night-Compilation issues only to find out that now, the application isn’t working. So there goes demo day.
You secretly start wondering if there was a way to ‘break the rules’. A way that you could secretly compile your code — maybe on a daily basis?— and see if you left something out. Hell, why not do it every time you check in your code? or even save it? What if you could _continuously_ compile your code? Your life would be so much easier. Not to mention your customer’s life. You could find those compilation errors quickly, and thus fix them quickly while they are still fresh in memory. You could start focusing on that demo now that the code compiles, and see if the app is actually working!
Aren’t we lucky that we all already live with that kind of continuous compilation? At least in static languages, we can see if our code ‘checks out’ by the compiler almost instantly. We can write a single line, or even a single code keyword, and see if it works when the reality of the compiler hits. If we are working in a dynamically typed language, like Ruby or Python, we write tests, and we run those tests continuously , so that we get feedback that the code works. Well, some of us do, anyway. A lot of us just _say_ we do it, but deep down we know it is lip service.
Sometimes we get a ‘fluke’ - we write a function and it fully compiles and runs with no problems with no compilation and debugging errors. That kind of magic doesn’t happen very often though.
Things aren’t perfect. Now that we live in this magical world where compilation is continuous, we start to realize we still have problems with our customer’s demos.
The demos aren’t working. Well, they are working, but not really. they break and they work only partially. And when we try to fix the functionality problems, we break other functionality. We need testing, or, more precisely, _automated_ testing to tell us if our code functions right.
Why does it need to be automated though? Because manual testing takes so much time. It’s really a good waste of money to repeat the same regression tests by a human. We should be using humans for tasks with creativity built in, like exploration testing, but we need ‘automated checks’ for regression testing, to keep telling us we didn’t break anything, continuously.
So we write some tests, and we run them locally every time we are about to check in the code. things feel better for a while, but then we realize the customer demos are still not really working. “Why would the customer tests not be working _now_?” you ask yourself. Turns out, the demos are supposed to be shown on the customer’s machine. The customer’s machine is very different from a developer’s machine. It has weird firewall requirements, it needs active directory permissions to run your application, and the database is shared on a different machine. For you, demo day is also, it turns out, deployment day.
Because you deploy before you demo (usually the day before if you are lucky), it means you have one day every week you deploy. Or maybe demo is once every two or three weeks, so you deploy just once every two or three weeks. If you’re lucky.
Deployment once a week? Now you spend your week compiling and testing code, but you’re still as blind as a bat. You do not know whether all this code will actually work within the constraints of the customer’s machine. Sure, it works on _your machine_, but we all know that doesn’t mean much.You spend all this time writing and polishing your beautiful code, all the while you have no idea whether it will _fit in_ with the hard reality of living on a customer’s machine.
Demo day comes, and you deploy, and your perfectly working code is shamingly crashing, lagging, stalling and basically acting like an angry child in the middle of a supermarket: Badly. Your demo sucks. And now you can’t get any feedback from the customer. Instead, you have wasted your customer’s time, and some of their hard earned trust.
This keeps happening.
Some weeks you get things right, some weeks the new pieces of code just don’t fit the configuration. You have to wait a week between demos to find out if the changes you made to your code or configuration files actually work, because, well, you deploy once a week.
You secretly start wondering what happened if you broke the rules and got your own little machine that looks and acts just like a customer’s machine. If you had that kind of machine locally, you could try and deploy to it on a daily basis, and find out if your code actually works after deployment. If it didn’t you could fix it way before demo day. And then you can use the demos to actually get feedback on the functionality of your application. On second thought, what if you went all out and _automated_ the deployment to the fake customer machine? then you could test your deployment continuously! On every check in.
Now that would be quite beautiful.
Deployment testing is like a compilation step for your release cycle. It verifies that your product, when deployed, can face the harsh realities of a production machine. It should happen continuously, so that you do not find out at the last minute that your code or configuration, is not able to handle the current reality.
For deployment testing, you need to create a _staging_ environment. This environment mimics the production environment as closely as possible, and it should not be contaminated with compilers, editors or development tools.
How do you test your deployment with a staging environment? You run acceptance tests ,and hopefully also all the other automated tests you’ve written, against the application deployed on staging. For example, you could run browser tests (using selenium or capybara — google those) against a web application installed on staging. What happens if you _do_ need to debug something that only seems to happen in the staging environment? That’s why you have a ‘test’ environment. think of ‘test’ as ‘staging + debugging tools’. It can be ‘dirty’ in that you can use it to examine things more closely, but in an environment that can simulate real world difficulties.
The environment right before ‘test’ is ‘dev-test’. that can either be a developer local machine, or a continuous integration build agent, that usually just compiles and runs automated tests.
To me, a beautiful build, is one that encompasses all the levels of confidence we just discussed: Compilation, automated testing, automated deployment, and deployment testing.
If possible,I take the next logical step - I also deploy to a production machine in an automated manner - the same code that has passed all the other stages.
I, and many others, call this chain ‘Continuous Delivery’. (will be part of my builds book)