Two Techniques for low risk roll out of your micro service
“When are we going live ?”
“Well! We don’t know for sure. We are agile. ”
The go-live question often pops up every other week in the development life cycle of a new micro service. Deadlines are a reality and corporate budgets are often fixed on a periodic basis. Business sponsors can get edgy if they continuously see demos of work in progress code for a long period of time without knowing when it will all become real.
It is easy to confuse going live as simply going to production. Traditionally going live was a big-bang publicized affair where release managers, project managers, engineers and the business planned months in advance for installing new software in production and rolling out to all its end users. The dates were sacrosanct and any failure or rollback sometimes led to consequences. It was an intense affair and managers of development teams often dreaded going to production, often pushing it to the last possible date. With the advent of distributed architectures, services became more granular and it was easier to scale hardware on-demand with the cloud. This meant that deployments became less interesting — in fact they became rather staid and with the right set of strategies, it was now possible to think of deployments as almost a zero risk activity. It is hence important to distinguish the deployment of a service from the process of going live — which is effectively the time the service starts receiving traffic. Decoupling the go-live effectively from the deployment has many advantages — including but not limited to de-risking, learning from active users, the ability to go live in small steps instead of a Big Bang and so on.
Here are two techniques that I have seen work very well to effectively decoupling a deployment of a service from its go-live.
Shadowing
Shadowing is a technique using which a micro service can start taking shadow requests — i.e ones that it can process but where the response is not used by the consumer. This is particularly useful when a new service is being rolled out and a existing service or application provides similar features which are being replaced, modernized, augmented or improved upon.
Shadowing can be a very effective technique for both product owners and technology owners as it can be used to facilitate a wide range of uses including :
- collecting data about a new feature or a version like accuracy of business rules
- test the resiliency, performance and availability of a new service
- fine tuning and hardening of a new service continuously
There are many ways to implement shadowing depending on the use case and existing landscape. In case of older legacy systems that are monolithic in nature, one way is to invoke the new service using a message or an event while it may be useful to spawn the shadow request to the new service from a part of an existing web service in other use cases. Based on the infrastructure it may be possible to do this using invocation methods based on dedicated environments or specific network level components.
Throttling
Throttling is the process of regulating the traffic requests to a service. A slow and controlled ramp up of users can be extremely useful in getting feedback from a small set of users before making it more generally available, especially when the volume of requests expected for the service is in millions.
There are many ways to throttle a service. For example, if the service is a user on boarding service, the users can be segmented across various lines and volumes estimated using some of the following :
Dog fooding : If you release a new functionality to a set of users who are internal like employees, any surprises related to the customer experience will be localized. It should be possible to roll it to a few employees first before the larger rollout.
Sampling : This is a way of releasing to a random sample of users. The sample can be derived using a mathematical formula, segregating using starting or ending alphabets and so on. The sample can be increased over time.
Profiling : A geography or other profile can be used to segment users so that the service can be released only to a specific set of users
Throttling is often synonymous also with canary releases.
Implementation
There are various ways to implement both shadowing and throttling. The exact tools used are often dependent on the enterprise tools that are approved within an organization and also the governance, infrastructure and best practices laid down. These techniques can also complement deployment patterns like blue-green and micro service patterns like Strangler. A very popular way to have fine grained control for shadowing and throttling is by using feature toggles or feature flags which provide product and engineering teams to change system behavior on the fly without deploying code. Automated canary releases that can facilitate throttling are also maturing in most publicly available cloud platforms.
