It’s 3:52 a.m. on a Saturday and various members of Aria Systems’ Production Operations, Engineering and Customer Service teams are staring anxiously at the clocks on their computer screen. In just eight short minutes the sands will begin to pour through the hourglass: we have only two hours to bring down the site, migrate the latest code version, bring it back up, test it, and release it to the world… or face the dreaded “roll back” and explain to our customers why we couldn’t deliver. Every step is meticulously rehearsed weeks in advance (and timed to the minute) to avoid this.
It’s 5:35 a.m., and we need to make the call – roll out or roll back. Slow-loading pages, hung API calls, mistyped commands – these are the time-consuming mortal enemies we face at this ungodly hour when minds are not as sharp as we’d like them to be. We are testing brand new code on a live environment – if anything goes wrong, the world will see it – and we only have 25 minutes left in our window of announced downtime. Once we hit 6:00 a.m., our customers will start hitting buttons.
When I first started at Aria, we certainly had our ups and downs with rolling out new code. With every new release, there are expected fixes, added features, streamlined dashboards… our customers look forward to the unveiling of a better product. But in order to deploy new versions of Aria, we had to take a complete outage for two hours and pray nothing took longer than expected. This was a weak spot for our company, and it needed to be remedied.
Fast forward, two years later. These days, we migrate new code through a process called LiveRelease, and it is very near and dear to me. I have managed these migrations since the dark days of “4:00 am Outages,” and I can tell you the road was not easy. But it certainly was worth it. Now, we are able to release new code on a steady schedule with zero downtime. We now have the ability to migrate and test new versions of Aria with no customer interruption. And since we have this ability, this also means we can execute a LiveRelease during normal business hours, when all resources are available – and awake!
Of course, we always migrate to our testing environments first. But it is the nature of the beast in IT that no two environments are ever truly identical. We must be prepared for unexpected behavior when moving code to a customer-facing environment. With 4:00 a.m. Outages, we would not know if issues existed until the code was brought live – visible to customers. With LiveRelease, we can now move the code directly to a customer-facing environment, while keeping the new version hidden from the customers until we are 100% satisfied with the quality. We are no longer restricted to a window of time, and no longer performing delicate work at odd hours.
This is just one of many examples of how the dedicated minds at Aria are constantly striving to raise the bar when it comes to quality products, quality service and quality delivery. We hope our customers enjoy the reduction in scheduled downtime and improved quality as much as we’ve enjoyed sleeping in on Saturdays!
Erin Byrnes, Aria Systems
Just learning about subscription billing? Download the new Subscription Billing for Dummies book and learn more. Are you concerned about buying the right subscription commerce solution for your company? Consider downloading the Six Key Buying Considerations e-Paper and make an educated decision on the future of your recurring revenue offering.