Feature flags and A/B testing
Hypertune lets you use feature flags and A/B testing to develop, test and release new features faster and more safely.
Problem
Before using feature flags, feature releases are typically tied to code deployments and so engineering teams tend to suffer the following problems:
If a new feature is broken, it's slow to roll back the feature as this requires a new code deployment
The entire bundled release needs to be rolled back, dramatically slowing down the engineering team
New features are released to all users at the same time, increasing their blast radius and risk
Code deployments are infrequent due to their risk and so feature releases are infrequent too
New features aren't tested in the production environment before being released
"Feature branches" result in large pull requests that are hard to review, hard to merge due to many conflicts with the main branch, and more likely to sneak bugs into production
It's difficult to collaborate on new features with other stakeholders like product managers, designers, customer success, marketing, sales, etc
It's difficult to test a new feature with a small group of beta users to get early feedback before it's ready for a full release
It's difficult to control which users have access to which features
Other stakeholders like product managers, designers, customer success, marketing, sales, etc, need to ask and wait for engineering to toggle features for different users
It's not possible to A/B test new features to measure their impact on key business metrics and prevent regressions
It's difficult to attribute error spikes with feature releases
Solution
Once engineering teams adopt Hypertune, they create a feature flag before starting work on a new feature. Work-in-progress, unfinished code for the feature is put behind the flag and merged straight to production in small, incremental pull requests.
Initially, the flag is off for everyone except the developers working on the new feature. This way, the feature is hidden for everyone by default, but the developers can see it and test it directly in production, as they develop it.
The developers can then add colleagues they're collaborating on the feature with, like product managers and designers, to get feedback from them.
Once the initial version of the feature is ready, they can add the QA team, and then a small group of alpha or beta users, to get more feedback.
Once the feature is ready to be released to everyone, they can roll it out to a small percentage of users and gradually ramp up this percentage over time, as they gain confidence that the feature works.
If the feature is only intended for certain users, they can be added directly.
To measure the impact of the new feature on key metrics, they can run an A/B test, where half of users see the new feature and the other half don't.
If at any point the feature starts causing errors in production, it can be turned off immediately, without requiring a code deployment.
Benefits
This feature flag-driven development workflow results in the following benefits:
If a new feature starts causing errors in production, it can be turned off immediately without requiring a code deployment
Risk can be mitigated when releasing a new feature via staged, percentage-based roll outs
Code deployments can happen more frequently due to lower risk and so feature releases can be more frequent too
New features can be tested directly in production before being released to users
Pull requests can be small and incremental, making them easier to review and merge and less likely to sneak bugs into production
It's easy to collaborate on new features with other stakeholders like product managers, designers, customer success, marketing, sales, etc
A new feature can be tested with a small group of beta users to get early feedback before it's ready for a full release
It's easy to control which users have access to which features
Other stakeholders like product managers, designers, customer success, marketing, sales, etc, can toggle features for different users themselves without needing to ask and wait for engineering
It's easy to A/B test new features to measure their impact on key business metrics and prevent regressions
Error spikes can be attributed to feature releases
ROI
These benefits mean engineering teams can:
Develop, test and release new features faster with the same number of developers
Improve product reliability for users
Prevent regressions in key business metrics
Get started
Follow the quickstart to get started.
Last updated