Splits

A split is the underlying abstraction that powers:

• A/B tests

• Staged, percentage-based rollouts

• Multivariate tests

• Machine learning loops

You can create splits in the Hypertune UI and then embed them anywhere in your flag targeting logic.

This means you can have a single feature flag with rules to enable the feature for specific users, e.g. employees, QA, beta users, etc, and a final default rule to A/B test the feature on everyone else.

It also means you can reuse a single A/B test across the logic of different feature flags to roll out and test related features in sync.

This power and flexibility is unique to Hypertune.

Structure

Each split has a:

• Type

  • Test — for A/B tests, staged rollouts and multivariate tests
  • ML loop — for machine learning loops
• Name

• Set of dimensions

• Payload event type
• Goal function (for ML loop splits only)

Each dimension has a:

• Name

• Set of arms

Each arm has a:

• Name

• Traffic allocation percentage (for Test splits only)

Using splits in flag targeting logic

Once created, you can embed splits anywhere in your flag targeting logic with a Split Control expression.

If the split has more than one dimension, you can select the one which is relevant for the flag.

Then for each arm in the dimension, you can set the flag values, or nest more flag targeting logic.

You can also set the Unit ID for the split, typically context.user.id. A hash of the Unit ID is used to determine the arm for Test splits so the same user will always end up in the same arm.

If the split has a payload event type, you can also set the value for the payload, or nest more logic for it.

When you evaluate a flag that uses a split, an "exposure" will be logged with the Unit ID and the arm the unit was assigned for each dimension. If the split has a payload event, this will be logged too.

A/B tests

An A/B test is a Test split with a single dimension with two arms.

Staged, percentage-based rollouts

A staged, percentage-based rollout is a Test split with a single dimension with one arm.

Multivariate tests

A multivariate test (or multidimensional test) is a Test split with multiple dimensions.

This lets you test all combinations of the arms of each dimension. For example, you can have one dimension called Button color with three arms: Red, Blue, Green. And another dimension called Button text with three arms: Sign up, Get access, Request access.

You can embed the test into a String flag, buttonColor, that controls the color of your call-to-action button, select the Button color dimension, then set flag values for each arm.

And you can embed the same test into another String flag, buttonText, that controls the text of your call-to-action button, but this time select the Button text dimension, then set flag values for each arm.

This will test all 9 combinations of button color and button text.

Machine learning loops

You can convert any Test split into an ML loop split.

A machine learning loop requires a goal function, expressed as a formula of your event types.

And instead of setting fixed traffic allocation percentages for each arm, Hypertune automatically and continuously learns the best arm for each dimension. If the split has a payload event type, the payload event fields are used as "features" for the machine learning model.

For example, you can set up a machine learning loop to personalize the headline on your landing page to each unique visitor, to maximize sign ups.

Viewing results

To view the results of your splits, ensure you've set up the event types you want to track, then build a funnel to compare conversion rates across different arms.