Feature flags and A/B testing
Hypertune lets you use feature flags and A/B testing to develop, test and release new features faster and more safely.
Problem
Before using feature flags, engineering teams tend to suffer the following problems:
If a new feature starts causing errors in production, it's slow to roll back the feature as this requires a new code deployment
There's a high risk of a large negative impact when releasing a new feature, as it's released to all users, all at once
New features aren't tested in the production environment before being released to users
"Feature branches" result in large pull requests that are hard to review, hard to merge due to many conflicts with the main branch, and more likely to sneak bugs into production
It's difficult to test a new feature with a small group of alpha or beta users
It's difficult to control which users have access to which features
It's not possible to A/B test new features to measure their impact on key business metrics and prevent regressions
Nontechnical product managers, customer success, marketing and sales teams can't toggle features for users or set up A/B tests themselves
It's difficult to attribute error spikes with feature releases
Solution
Once engineering teams adopt Hypertune, they create a feature flag before starting work on a new feature. Work-in-progress, unfinished code for the feature is put behind the flag and merged straight to production in small, incremental pull requests.
Initially, the flag is off for everyone except the developers working on the new feature. This way, the feature is hidden for everyone by default, but the developers can see it and test it directly in production, as they develop it.
The developers can then add colleagues they're collaborating on the new feature with, like designers and product managers.
Once the initial version of the feature is ready, they can add the QA team, and then a small group of alpha or beta users.
Once the feature is ready to be released to everyone, they can roll it out to a small percentage of users and gradually ramp up this percentage over time, as they gain confidence that the feature works.
If the feature is only intended for certain users, they can be added directly.
To measure the impact of the new feature on key metrics, they can run an A/B test, where half of users see the new feature and the other half don't.
If at any point the feature starts causing errors in production, it can be turned off immediately.
Benefits
This feature flag-driven development workflow results in the following benefits:
If a new feature starts causing errors in production, it can be turned off immediately
Risk can be mitigated when releasing a new feature via staged, percentage-based roll outs
New features can be tested directly in production before being released to users
Pull requests can be small and incremental, making them easier to review and merge and less likely to sneak bugs into production
New features can be tested with a small group of alpha or beta users
It's easy to control which users have access to which features
It's easy to A/B test new features to measure their impact on key business metrics and prevent regressions
Nontechnical product managers, customer success, marketing and sales teams can toggle features for users and set up A/B tests themselves
Error spikes can be attributed to feature releases
ROI
These benefits mean engineering teams can:
Develop, test and release new features faster with the same number of developers
Improve product reliability for users
Prevent regressions in key business metrics
Get started
Follow the quickstart to get started.
Last updated