feat(experiments): holdout groups UI #25772

jurajmajerik · 2024-10-23T19:05:30Z

Changes

Add UI for managing holdout groups
Add the holdout to test variants in the results calculation

1. List holdouts

2. Create/edit holdout

3. Edit holdout during experiment creation

4. Edit holdout in the Draft mode

5. Show holdout in the Launched mode (no longer editable)

6. Show results for the holdout

How did you test this code?

👀

github-actions · 2024-10-23T19:16:38Z

Size Change: +44 B (0%)

Total Size: 1.15 MB

ℹ️ View Unchanged

Filename	Size	Change
`frontend/dist/toolbar.js`	1.15 MB	+44 B (0%)

_{compressed-size-action}

posthog-bot · 2024-10-23T21:07:07Z

📸 UI snapshots have been updated

8 snapshot changes in total. 0 added, 8 modified, 0 deleted:

chromium: 0 added, 8 modified, 0 deleted (diff for shard 1, diff for shard 2)
webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2024-10-23T21:25:35Z

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

chromium: 0 added, 2 modified, 0 deleted (diff for shard 1)
webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

…osthog into experiment-holdouts-ui

…riment-holdouts-ui

posthog-bot · 2024-10-25T08:46:46Z

📸 UI snapshots have been updated

4 snapshot changes in total. 0 added, 4 modified, 0 deleted:

chromium: 0 added, 4 modified, 0 deleted (diff for shard 1)
webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2024-10-25T09:08:55Z

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

chromium: 0 added, 2 modified, 0 deleted (diff for shard 1)
webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

…osthog into experiment-holdouts-ui

posthog-bot · 2024-10-25T09:27:46Z

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

chromium: 0 added, 2 modified, 0 deleted (diff for shard 1)
webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

neilkakkar

haven't tried running locally yet, not sure if you've incorporated changes with my latest holdout_id pr.

neilkakkar · 2024-10-25T13:07:50Z

frontend/src/scenes/experiments/ExperimentView/HoldoutSelector.tsx

+                    }
+                    size="xsmall"
+                    options={holdoutOptions}
+                    value={experiment.holdout || null}


wouldn't this be the entire holdout object?

neilkakkar · 2024-10-25T13:08:32Z

frontend/src/scenes/experiments/ExperimentView/components.tsx

@@ -52,8 +52,23 @@ export function VariantTag({
 }): JSX.Element {
    const { experimentResults, getIndexForVariant } = useValues(experimentLogic({ experimentId }))

+    if (variantKey.startsWith('holdout-')) {


i guess we should disable user created variants starting with holdout- 🙈

I think this check should be good enough for now: https://github.com/PostHog/posthog/pull/25772/files#diff-ac20870bd5fba7ae95bc9fde57e9108900e26f210382221cc1466059172748f3R55

Can add validation later if we run into issues

neilkakkar · 2024-10-25T13:12:00Z

frontend/src/scenes/experiments/Holdouts.tsx

+                                onChange={(rollout_percentage) =>
+                                    setHoldout({
+                                        filters: {
+                                            groups: [{ properties: [], rollout_percentage }],


groups seem extra

…riment-holdouts-ui

jurajmajerik · 2024-10-25T14:28:28Z

Feedback addressed :)

posthog-bot · 2024-10-25T15:59:10Z

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

chromium: 0 added, 2 modified, 0 deleted (diff for shard 1)
webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot · 2024-10-25T16:18:08Z

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

chromium: 0 added, 2 modified, 0 deleted (diff for shard 1)
webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

jurajmajerik added 2 commits October 23, 2024 19:49

init

af17ad1

wip

c003b75

jurajmajerik marked this pull request as draft October 23, 2024 19:05

jurajmajerik and others added 4 commits October 23, 2024 22:06

wip

90626b9

wip

3fd49d1

Update UI snapshots for chromium (1)

27b0e90

Update UI snapshots for chromium (2)

5a5850d

Update UI snapshots for chromium (1)

ccbe969

jurajmajerik and others added 5 commits October 24, 2024 13:24

wip

8828df8

Merge branch 'experiment-holdouts-ui' of https://github.com/PostHog/p…

3ce742e

…osthog into experiment-holdouts-ui

Merge branch 'master' of https://github.com/PostHog/posthog into expe…

e6b7b88

…riment-holdouts-ui

wip

fc5d0ce

Update UI snapshots for chromium (1)

adeba07

Update UI snapshots for chromium (1)

67b14a1

jurajmajerik and others added 4 commits October 25, 2024 11:18

wip

336b1b6

Merge branch 'experiment-holdouts-ui' of https://github.com/PostHog/p…

1850c15

…osthog into experiment-holdouts-ui

wip

3b37e8b

Update UI snapshots for chromium (1)

94508b7

jurajmajerik marked this pull request as ready for review October 25, 2024 09:37

jurajmajerik requested a review from neilkakkar October 25, 2024 09:37

neilkakkar reviewed Oct 25, 2024

View reviewed changes

jurajmajerik added 2 commits October 25, 2024 15:18

Merge branch 'master' of https://github.com/PostHog/posthog into expe…

020d24f

…riment-holdouts-ui

address feedback

e5bf641

jurajmajerik requested a review from neilkakkar October 25, 2024 14:28

github-actions bot and others added 5 commits October 25, 2024 14:40

Update query snapshots

81d2600

Update query snapshots

23f406c

Merge branch 'master' into experiment-holdouts-ui

145915f

Update query snapshots

8b0ca4a

Update UI snapshots for chromium (1)

a2802a9

Update UI snapshots for chromium (1)

fb86847

neilkakkar approved these changes Oct 28, 2024

View reviewed changes

jurajmajerik merged commit c61b3a9 into master Oct 28, 2024
94 checks passed

jurajmajerik deleted the experiment-holdouts-ui branch October 28, 2024 10:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(experiments): holdout groups UI #25772

feat(experiments): holdout groups UI #25772

jurajmajerik commented Oct 23, 2024 •

edited

Loading

github-actions bot commented Oct 23, 2024 •

edited

Loading

posthog-bot commented Oct 23, 2024

posthog-bot commented Oct 23, 2024

posthog-bot commented Oct 25, 2024

posthog-bot commented Oct 25, 2024

posthog-bot commented Oct 25, 2024

neilkakkar left a comment

neilkakkar Oct 25, 2024

neilkakkar Oct 25, 2024

jurajmajerik Oct 25, 2024

neilkakkar Oct 25, 2024

jurajmajerik commented Oct 25, 2024

posthog-bot commented Oct 25, 2024

posthog-bot commented Oct 25, 2024

feat(experiments): holdout groups UI #25772

feat(experiments): holdout groups UI #25772

Conversation

jurajmajerik commented Oct 23, 2024 • edited Loading

Changes

1. List holdouts

2. Create/edit holdout

3. Edit holdout during experiment creation

4. Edit holdout in the Draft mode

5. Show holdout in the Launched mode (no longer editable)

6. Show results for the holdout

How did you test this code?

github-actions bot commented Oct 23, 2024 • edited Loading

posthog-bot commented Oct 23, 2024

📸 UI snapshots have been updated

posthog-bot commented Oct 23, 2024

📸 UI snapshots have been updated

posthog-bot commented Oct 25, 2024

📸 UI snapshots have been updated

posthog-bot commented Oct 25, 2024

📸 UI snapshots have been updated

posthog-bot commented Oct 25, 2024

📸 UI snapshots have been updated

neilkakkar left a comment

Choose a reason for hiding this comment

neilkakkar Oct 25, 2024

Choose a reason for hiding this comment

neilkakkar Oct 25, 2024

Choose a reason for hiding this comment

jurajmajerik Oct 25, 2024

Choose a reason for hiding this comment

neilkakkar Oct 25, 2024

Choose a reason for hiding this comment

jurajmajerik commented Oct 25, 2024

posthog-bot commented Oct 25, 2024

📸 UI snapshots have been updated

posthog-bot commented Oct 25, 2024

📸 UI snapshots have been updated

jurajmajerik commented Oct 23, 2024 •

edited

Loading

github-actions bot commented Oct 23, 2024 •

edited

Loading