Data model
In Amplitude, your organization is the top-most hierarchical level. Within an organization, Experiment follows the project structure that Amplitude Analytics defines. Associate all Experiment data with an Analytics project.
An Amplitude project contains flags, experiments, and deployments.
Projects
Experiment uses the same projects as Amplitude Analytics. As a best practice, create a project for each product and for each environment. Because flags, experiments, and deployments only exist within a single project, duplicate these objects across projects for the same product.
Copy a flag to another project
When developing a new feature with an experiment, create the experiment in the dev environment project to develop and test the implementation. Then, copy the experiment into the prod project to run the experiment in prod.
A deployment serves a group of flags or experiments for use in an application. Each project has a deployment that uses the project API key as the deployment key, available by default. Amplitude randomly generates deployment keys. On creation, Amplitude assigns each experiment deployment an associated deployment key. Experiment uses the deployment key to identify the deployment and authorize requests to the evaluation servers.
Client and server deployments
Deployments are either client or server deployments. Use client-side deployments to initialize client-side SDKs and server-side deployments to initialize server-side SDKs or authorize requests to the Evaluation API.
Deployments
Deployments belong to Analytics projects, and a project can have multiple deployments. Name deployments after the platform (client-side) or service (server-side) to which Experiment serves variants (for example: android, ios, web). The default project API key deployment is useful for getting started. However, explicit deployments for each platform or service work better for larger organizations or teams that share the same Amplitude project across multiple platforms for the same application.
Add deployments to Flags and Experiments in the same project. When Experiment's evaluation servers receive a request to fetch variants for a user, Experiment uses the deployment key to look up all associated flags and experiments for evaluation.
Flags and experiments
Experiment uses feature flags and experiments to serve a variable experience to a user. The flag key identifies flags and experiments. Flags and experiments associate with 0-n deployments and contain 1-k variants. The evaluation mode (local or remote) determines whether the flag or experiment supports local evaluation, and local mode can limit the flag's targeting capabilities.
Feature flags and experiments share the same underlying data model, and you can migrate from one to the other retroactively. The most visible difference appears in the product interface: experiments guide you through an experiment lifecycle and let you define success metrics and perform analysis. Flags contain more basic functionality, and don't include special planning and analysis sections.
Flags
Use flags for standard feature flagging without user analysis. New flags come with a default variant, on.
Flag use cases
- Rolling out a feature to a subset of users (for example, beta customers).
- Different experience for a behavioral cohort (for example, power users).
Experiments
Use experiments for feature experimentation on users. New experiments come with two default variants, control and treatment.
Experiment use cases
- Run an A/B test for a new feature in your application.
- Experiment on multiple recommendation algorithms on your server.
Variants
A variant is the part of the experiment that you change for the customer. For example, if you experiment with the text on a call to action (CTA) button, each version of the text is a variant. Variants can exist within a flag or an experiment. A single variant can associate with a single flag, or multiple variants can associate with a larger experiment. An experiment requires variants so that you can test the experiment's hypothesis.
SDK use
Only the value and payload are available when accessing a variant from an SDK or the Evaluation API.
| Property | Requirement | Description |
|---|---|---|
value | Required | A string that identifies the variant in the instrumentation. Amplitude checks the value string for equality when an SDK or Evaluation API accesses the variant. Format must be lowercase, kebab-case, or snake_case. |
payload | Optional | Dynamic JSON payload for sending arbitrary data with the variant. For example, you can send a hex code to change the color of a component in your application. |
name | Optional | Name for the variant. The name is like value, but doesn't have formatting limitations, and you can change it without breaking the instrumentation in your code base. |
description | Optional | A more detailed description of the variant. Use the description to explain what the user experiences when viewing the variable experience in more detail. |
Users
Experiment users map to a user within Amplitude Analytics. Alongside flag configurations, users are an input to evaluation. Flag and experiment targeting rules access user properties.
Pass users to evaluation through fetch requests for remote evaluation, or directly to the evaluate function for local evaluation.
Include either a user ID or device ID in the user object for evaluation to succeed. For example, remote evaluation returns a 400 error if both the User ID and Device ID are null, empty, or missing.
| Property | Type | Description |
|---|---|---|
user_id | string | The User ID is the primary identifier for the user. This value is typically the user's ID within your system. Experiment uses the User ID when resolving the Amplitude ID on enrichment before remote evaluation, where the Amplitude ID is the default bucketing key. |
device_id | string | The Device ID is the secondary identifier for the user. An analytics SDK usually generates the Device ID randomly on the client side, or sets it in a cookie on the server side. Experiment also uses the Device ID when resolving the Amplitude ID on enrichment before remote evaluation, where the Amplitude ID is the default bucketing key. |
user_properties | object | Optional object of custom properties used when evaluating the user during local or remote evaluation. |
groups | object | Beta. Optional object that lists groups associated with this user. Format is an object where the key is the group type, and the value is an array of group value strings (for example, {"org name":["Amplitude"]}). |
group_properties | object | Beta. Optional object that lists group properties associated with this user. Format is a nested object where the key is the group type, and the value is an object where the key is the group value, and the value is an object of properties (for example, {"org name":{"Amplitude":{"plan":"enterprise"}}}). |
Beta
This feature is in Beta. This feature may continue to evolve. This documentation may not yet reflect the latest updates.
If your organization has purchased the Accounts add-on, you can perform bucketing and analysis on groups rather than users. Contact your representative to gain access.
Include groups with the user when sent with the fetch request (recommended). Or, identify groups with the user through a group identify call from the Group Identify API or with setGroup() from an analytics SDK.
All Experiment SDKs support groups. The following table lists minimum versions:
| SDK | Minimum version |
|---|---|
| Android | 1.9.0 |
| iOS | 1.10.0 |
| React Native | 1.1.0 |
| JavaScript | 1.5.6 |
| Ruby | 1.4.0 |
| Go | 1.7.0 |
| Python | 1.3.0 |
| JVM | 1.3.0 |
| Node | 1.5.0 |
| PHP | 1.0.0 |
Full user definition
Was this helpful?