feat: Add Cypress Tests for Onboarding Flow (#20676)

* wip * iteration 1 * some small fixes * update * Update UI snapshots for `chromium` (1) * fix up product intro page test * Update UI snapshots for `webkit` (2) * Update UI snapshots for `webkit` (2) * rebase on main, fix a type issue * Update UI snapshots for `chromium` (2) * Update UI snapshots for `chromium` (2) * remove console log * remove extraneous file * fix tests again... * Update UI snapshots for `chromium` (2) * more test tweaks * Update UI snapshots for `chromium` (2) * yet another set of fixeds * Update UI snapshots for `chromium` (2) * Update UI snapshots for `webkit` (2) * Update UI snapshots for `chromium` (2) * Update UI snapshots for `webkit` (2) * one more test tweak * fix(admin): follow up for org page performance improvements (#20912) * feat: Add clickhouse migration to create a dictionary (#20665) * feat: Add clickhouse migration to create a dictionary * refactor: Use dictionary created in migration for squashing * fix: Test file drop references to dictionary * fix: Update comment * chore: Master merge * fix: Typo in dictionary Co-authored-by: James Greenhill <[email protected]> --------- Co-authored-by: James Greenhill <[email protected]> * fix: picking embedding input samples (#20913) * fix: Split healthcheck to separate nginx unit app (#20917) Split healthcheck to separate nginx unit app When posthog is under load the healthcheck can stop responding. Split it off into a separate app so it has a dedicated thread to handle it * chore: BillingProduct cleanup (#20910) * fix: show names on frontend (#20918) * fix(experiment): revert default goal back to Trends (#20921) * Add missing healthcheck routes to app (#20923) * Add missing healthcheck routes to app * Switch healthz to _health * chore: Add context to deployment messages (#20911) Add context to deployment messages Co-authored-by: xneyder <[email protected]> * chore(environments): Add migration to backfill projects (#20887) * chore(environments): Add migration to backfill projects * Fix `noop` * Add `project_id` to plugin server test setup * Fix `project_id` * Also add `posthog_project` to plugin server tests * Update `createTeam` * Fix func tests * refactor(style): do not use star imports for locals (#20915) * feat(sentry): enable sampling via http header (#20916) * fix: Add missing underscore to healthcheck routes (#20926) Add missing underscore to healthcheck routes * chore(deps): Update posthog-js to 1.115.0 (#20924) * fix(toolbar): Keep flags in sync (#20927) * fix(toolbar): Keep flags in sync * add test * fix: some typos in summary settings (#20925) * fix: some typos in summary settings * Update UI snapshots for `chromium` (2) * don't compare instance equality you fool * feels wrong for reset to change opt in status... * Fix --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * fix: custom player inspector events should respect mini filters (#20929) * fix: custom player inspector events should respect mini filters * all the alls * fix: don't show billing step on self-hosted (#20931) don't show billing step on self-hosted * chore: improve onboarding spacing consistency (#20838) * Make the space between sections consistent * add more space above team section * Improve the topbar spacing and positioning * Move the breadcrumbs above and center the title * Make spacing consistent across all SDKs * Add chain op to plans for when they aren't available * Make sdk margin top consistent with other steps * Update the invite onboarding step title * Move the breadcrumbs to be left aligned * Set top bar title to onboarding in onboarding * Fix cypress tests * Update cypress tests * Remove stale snapshots * remove billing changes --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * fix: flappy snapshot (#20930) * attempt fix for flappy snapshot * Update UI snapshots for `chromium` (2) * fix another one * Update UI snapshots for `chromium` (2) --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * fix storybook * Update UI snapshots for `webkit` (2) * Update UI snapshots for `chromium` (1) * Update UI snapshots for `chromium` (2) * fix some tests * another round of fixes --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Thomas Obermüller <[email protected]> Co-authored-by: Tomás Farías Santana <[email protected]> Co-authored-by: James Greenhill <[email protected]> Co-authored-by: David Newell <[email protected]> Co-authored-by: Frank Hamand <[email protected]> Co-authored-by: Juraj Majerik <[email protected]> Co-authored-by: danielxnj <[email protected]> Co-authored-by: xneyder <[email protected]> Co-authored-by: Michael Matloka <[email protected]> Co-authored-by: PostHog Bot <[email protected]> Co-authored-by: Neil Kakkar <[email protected]> Co-authored-by: Paul D'Ambra <[email protected]> Co-authored-by: Raquel Smith <[email protected]> Co-authored-by: Zach Waterfield <[email protected]>
PostHog · Mar 19, 2024 · d8e7d1a · d8e7d1a
1 parent fefc8d2
commit d8e7d1a
Show file tree

Hide file tree

Showing 21 changed files with 3,435 additions and 1,003 deletions.
diff --git a/cypress/e2e/billingUpgradeCTA.cy.ts b/cypress/e2e/billingUpgradeCTA.cy.ts
@@ -37,7 +37,7 @@ describe('Billing Upgrade CTA', () => {
             const events = (win as any)._cypress_posthog_captures
 
             const matchingEvents = events.filter((event) => event.event === 'billing CTA shown')
-            expect(matchingEvents.length).to.equal(4)
+            expect(matchingEvents.length).to.equal(3)
         })
 
         cy.intercept('/api/billing-v2/', { fixture: 'api/billing-v2/billing-v2-unsubscribed.json' })

diff --git a/cypress/e2e/billingv2.cy.ts b/cypress/e2e/billingv2.cy.ts
@@ -14,7 +14,7 @@ describe('Billing', () => {
 
         cy.get('[data-attr=more-button]').first().click()
         cy.contains('.LemonButton', 'Unsubscribe').click()
-        cy.get('.LemonModal h3').should('contain', 'Why are you unsubscribing from Product analytics + data stack?')
+        cy.get('.LemonModal h3').should('contain', 'Why are you unsubscribing from Product analytics?')
         cy.get('[data-attr=unsubscribe-reason-survey-textarea]').type('Product analytics')
         cy.contains('.LemonModal .LemonButton', 'Unsubscribe').click()
 
@@ -35,7 +35,7 @@ describe('Billing', () => {
     it('Unsubscribe survey text area maintains unique state between product types', () => {
         cy.get('[data-attr=more-button]').first().click()
         cy.contains('.LemonButton', 'Unsubscribe').click()
-        cy.get('.LemonModal h3').should('contain', 'Why are you unsubscribing from Product analytics + data stack?')
+        cy.get('.LemonModal h3').should('contain', 'Why are you unsubscribing from Product analytics?')
 
         cy.get('[data-attr=unsubscribe-reason-survey-textarea]').type('Product analytics')
         cy.contains('.LemonModal .LemonButton', 'Cancel').click()

diff --git a/cypress/e2e/onboarding.cy.ts b/cypress/e2e/onboarding.cy.ts
@@ -3,6 +3,8 @@ import { decideResponse } from '../fixtures/api/decide'
 
 describe('Onboarding', () => {
     beforeEach(() => {
+        cy.intercept('/api/billing-v2/', { fixture: 'api/billing-v2/billing-v2-unsubscribed.json' })
+
         cy.intercept('**/decide/*', (req) =>
             req.reply(
                 decideResponse({
@@ -21,24 +23,158 @@ describe('Onboarding', () => {
         // Confirm product intro is not included as the first step in the upper right breadcrumbs
         cy.get('[data-attr=onboarding-breadcrumbs] > :first-child > * span').should('not.contain', 'Product intro')
 
-        // Navigate to the product intro page by clicking the left side bar
-        cy.get('[data-attr=menu-item-replay').click()
+        cy.window().then((win) => {
+            win.POSTHOG_APP_CONTEXT.current_team.has_completed_onboarding_for = {}
+        })
+
+        cy.get('[data-attr=menu-item-savedinsights]').click()
 
         // Confirm we're on the product_intro page
         cy.get('[data-attr=top-bar-name] > span').contains('Onboarding')
-        cy.get('[data-attr=product-intro-title]').contains('Watch how users experience your app')
+        cy.get('[data-attr=product-intro-title]').contains('Product analytics with autocapture')
 
-        // Go back to /products
-        cy.visit('/products')
+        cy.get('[data-attr=start-onboarding]').should('be.visible')
+        cy.get('[data-attr=skip-onboarding]').should('not.exist')
+    })
 
-        // Again get started on product analytics onboarding
-        cy.get('[data-attr=product_analytics-onboarding-card]').click()
+    // it('Step through PA onboarding', () => {
+    //     cy.visit('/products')
 
-        // Navigate to the product intro page by changing the url
-        cy.visit(urls.onboarding('session_replay', 'product_intro'))
+    //     // Get started on product analytics onboarding
+    //     cy.get('[data-attr=product_analytics-onboarding-card]').click()
 
-        // Confirm we're on the product intro page
-        cy.get('[data-attr=top-bar-name] > span').contains('Onboarding')
-        cy.get('[data-attr=product-intro-title]').contains('Watch how users experience your app')
-    })
+    //     // Installation should be complete
+    //     cy.get('svg.LemonIcon.text-success').should('exist')
+    //     cy.get('svg.LemonIcon.text-success').parent().should('contain', 'Installation complete')
+
+    //     // Continue to configuration step
+    //     cy.get('[data-attr=sdk-continue]').click()
+
+    //     // Confirm the appropriate breadcrumb is highlighted
+    //     cy.get('[data-attr=onboarding-breadcrumbs] > :nth-child(3) > * span').should('contain', 'Configure')
+    //     cy.get('[data-attr=onboarding-breadcrumbs] > :nth-child(3) > * span').should('not.have.css', 'text-muted')
+
+    //     // Continue to plans
+    //     cy.get('[data-attr=onboarding-continue]').click()
+
+    //     // Verify pricing table visible
+    //     cy.get('.BillingHero').should('be.visible')
+    //     cy.get('table.PlanComparison').should('be.visible')
+
+    //     // Confirm buttons on pricing comparison
+    //     cy.get('[data-attr=upgrade-Paid] .LemonButton__content').should('have.text', 'Upgrade')
+    //     cy.get('[data-attr=upgrade-Free] .LemonButton__content').should('have.text', 'Current plan')
+
+    //     // Continue
+    //     cy.get('[data-attr=onboarding-skip-button]').click()
+
+    //     // Click back to Install step
+    //     cy.get('[data-attr=onboarding-breadcrumbs] > :first-child > * span').click()
+
+    //     // Continue through to finish
+    //     cy.get('[data-attr=sdk-continue]').click()
+    //     cy.get('[data-attr=onboarding-continue]').click()
+    //     cy.get('[data-attr=onboarding-skip-button]').click()
+    //     cy.get('[data-attr=onboarding-continue]').click()
+
+    //     // Confirm we're on the insights list page
+    //     cy.url().should('contain', 'project/1/insights')
+
+    //     cy.visit('/onboarding/product_analytics?step=product_intro')
+
+    //     // Should see both an option to skip onboarding and an option to see the sdk instructions
+    //     cy.get('[data-attr=skip-onboarding]').should('be.visible')
+    //     cy.get('[data-attr=start-onboarding-sdk]').should('be.visible')
+
+    //     cy.get('[data-attr=skip-onboarding]').first().click()
+    //     cy.url().should('contain', 'project/1/insights')
+
+    //     cy.visit('/onboarding/product_analytics?step=product_intro')
+    //     cy.get('[data-attr=start-onboarding-sdk]').first().click()
+    //     cy.url().should('contain', 'project/1/onboarding/product_analytics?step=install')
+
+    //     cy.visit('/products')
+    //     cy.get('[data-attr=return-to-product_analytics] > svg').click()
+    //     cy.url().should('contain', 'project/1/insights')
+    // })
+
+    // it('Step through SR onboarding', () => {
+    //     cy.visit('/products')
+    //     cy.get('[data-attr=session_replay-onboarding-card]').click()
+
+    //     // Installation should be complete
+    //     cy.get('svg.LemonIcon.text-success').should('exist')
+    //     cy.get('svg.LemonIcon.text-success').parent().should('contain', 'Installation complete')
+    //     // Continue to configuration step
+    //     cy.get('[data-attr=sdk-continue]').click()
+    //     // Continue to plans
+    //     cy.get('[data-attr=onboarding-continue]').click()
+    //     // Verify pricing table visible
+    //     cy.get('.BillingHero').should('be.visible')
+    //     cy.get('table.PlanComparison').should('be.visible')
+    //     // Confirm buttons on pricing comparison
+    //     cy.get('[data-attr=upgrade-Paid] .LemonButton__content').should('have.text', 'Upgrade')
+    //     cy.get('[data-attr=upgrade-Free] .LemonButton__content').should('have.text', 'Current plan')
+    //     // Continue through to finish
+    //     cy.get('[data-attr=onboarding-skip-button]').click()
+    //     cy.get('[data-attr=onboarding-continue]').click()
+    //     // Confirm we're on the recordings list page
+    //     cy.url().should('contain', 'project/1/replay/recent')
+    //     cy.visit('/onboarding/session_replay?step=product_intro')
+    //     cy.get('[data-attr=skip-onboarding]').should('be.visible')
+    //     cy.get('[data-attr=start-onboarding-sdk]').should('not.exist')
+    // })
+
+    // it('Step through FF onboarding', () => {
+    //     cy.visit('/onboarding/feature_flags?step=product_intro')
+    //     cy.get('[data-attr=start-onboarding-sdk]').first().click()
+    //     cy.get('[data-attr=sdk-continue]').click()
+
+    //     // Confirm the appropriate breadcrumb is highlighted
+    //     cy.get('[data-attr=onboarding-breadcrumbs] > :nth-child(5) > * span').should('contain', 'Plans')
+    //     cy.get('[data-attr=onboarding-breadcrumbs] > :nth-child(3) > * span').should('not.have.css', 'text-muted')
+
+    //     cy.get('[data-attr=onboarding-skip-button]').click()
+    //     cy.get('[data-attr=onboarding-continue]').click()
+
+    //     cy.url().should('contain', '/feature_flags')
+
+    //     cy.visit('/onboarding/feature_flags?step=product_intro')
+
+    //     cy.get('[data-attr=skip-onboarding]').should('be.visible')
+    //     cy.get('[data-attr=start-onboarding-sdk]').should('be.visible')
+
+    //     cy.get('[data-attr=skip-onboarding]').first().click()
+    // })
+
+    // it('Step through Surveys onboarding', () => {
+    //     cy.visit('/onboarding/surveys?step=product_intro')
+    //     cy.get('[data-attr=skip-onboarding]').should('be.visible')
+    //     cy.get('[data-attr=start-onboarding-sdk]').should('not.exist')
+    //     cy.get('[data-attr=skip-onboarding]').first().click()
+    //     cy.url().should('contain', 'survey_templates')
+
+    //     cy.visit('/products')
+    //     cy.get('[data-attr=surveys-onboarding-card]').click()
+    //     // Installation should be complete
+    //     cy.get('svg.LemonIcon.text-success').should('exist')
+    //     cy.get('svg.LemonIcon.text-success').parent().should('contain', 'Installation complete')
+
+    //     // Continue to configuration step
+    //     cy.get('[data-attr=sdk-continue]').click()
+
+    //     // Verify pricing table visible
+    //     cy.get('.BillingHero').should('be.visible')
+    //     cy.get('table.PlanComparison').should('be.visible')
+
+    //     // Confirm buttons on pricing comparison
+    //     cy.get('[data-attr=upgrade-Paid] .LemonButton__content').should('have.text', 'Upgrade')
+    //     cy.get('[data-attr=upgrade-Free] .LemonButton__content').should('have.text', 'Current plan')
+
+    //     // Continue
+    //     cy.get('[data-attr=onboarding-skip-button]').click()
+    //     cy.get('[data-attr=onboarding-continue]').click()
+
+    //     cy.url().should('contain', '/survey_templates')
+    // })
 })
diff --git a/cypress/fixtures/api/billing-v2/billing-v2-unsubscribed.json b/cypress/fixtures/api/billing-v2/billing-v2-unsubscribed.json
@@ -14,13 +14,18 @@
         "discord_integration",
         "apps",
         "boolean_flags",
+        "multivariate_flags",
         "persist_flags_cross_authentication",
         "feature_flag_payloads",
         "multiple_release_conditions",
         "release_condition_overrides",
         "targeting_by_group",
         "local_evaluation_and_bootstrapping",
         "flag_usage_stats",
+        "experimentation",
+        "funnel_experiments",
+        "secondary_metrics",
+        "statistical_analysis",
         "feature_flags_data_retention",
         "console_logs",
         "recordings_performance",
@@ -173,6 +178,14 @@
             "limit": null,
             "note": null
         },
+        {
+            "key": "multivariate_flags",
+            "name": "Multivariate feature flags & experiments",
+            "description": "Create three or more variants of a feature flag to test or release different versions of a feature.",
+            "unit": null,
+            "limit": null,
+            "note": null
+        },
         {
             "key": "persist_flags_cross_authentication",
             "name": "Persist flags across authentication",
@@ -229,6 +242,38 @@
             "limit": null,
             "note": null
         },
+        {
+            "key": "experimentation",
+            "name": "A/B testing",
+            "description": "Test changes to your product and evaluate the impacts those changes make.",
+            "unit": null,
+            "limit": null,
+            "note": null
+        },
+        {
+            "key": "funnel_experiments",
+            "name": "Funnel & trend experiments",
+            "description": "Measure the impact of a change on a aggregate values or a series of events, like a signup flow.",
+            "unit": null,
+            "limit": null,
+            "note": null
+        },
+        {
+            "key": "secondary_metrics",
+            "name": "Secondary experiment metrics",
+            "description": "Track additional metrics to see how your experiment affects other parts of your app or different flows.",
+            "unit": null,
+            "limit": null,
+            "note": null
+        },
+        {
+            "key": "statistical_analysis",
+            "name": "Statistical analysis",
+            "description": "Get a statistical analysis of your experiment results to see if the results are significant, or if they're likely just due to chance.",
+            "unit": null,
+            "limit": null,
+            "note": null
+        },
         {
             "key": "feature_flags_data_retention",
             "name": "Data retention",
@@ -1717,6 +1762,14 @@
                             "limit": null,
                             "note": null
                         },
+                        {
+                            "key": "multivariate_flags",
+                            "name": "Multivariate feature flags & experiments",
+                            "description": "Create three or more variants of a feature flag to test or release different versions of a feature.",
+                            "unit": null,
+                            "limit": null,
+                            "note": null
+                        },
                         {
                             "key": "persist_flags_cross_authentication",
                             "name": "Persist flags across authentication",
@@ -1773,6 +1826,38 @@
                             "limit": null,
                             "note": null
                         },
+                        {
+                            "key": "experimentation",
+                            "name": "A/B testing",
+                            "description": "Test changes to your product and evaluate the impacts those changes make.",
+                            "unit": null,
+                            "limit": null,
+                            "note": null
+                        },
+                        {
+                            "key": "funnel_experiments",
+                            "name": "Funnel & trend experiments",
+                            "description": "Measure the impact of a change on a aggregate values or a series of events, like a signup flow.",
+                            "unit": null,
+                            "limit": null,
+                            "note": null
+                        },
+                        {
+                            "key": "secondary_metrics",
+                            "name": "Secondary experiment metrics",
+                            "description": "Track additional metrics to see how your experiment affects other parts of your app or different flows.",
+                            "unit": null,
+                            "limit": null,
+                            "note": null
+                        },
+                        {
+                            "key": "statistical_analysis",
+                            "name": "Statistical analysis",
+                            "description": "Get a statistical analysis of your experiment results to see if the results are significant, or if they're likely just due to chance.",
+                            "unit": null,
+                            "limit": null,
+                            "note": null
+                        },
                         {
                             "key": "feature_flags_data_retention",
                             "name": "Data retention",
@@ -1879,14 +1964,6 @@
                             "limit": null,
                             "note": null
                         },
-                        {
-                            "key": "group_experiments",
-                            "name": "Group experiments",
-                            "description": "Target experiments to specific groups of users so everyone in the same group gets the same variant.",
-                            "unit": null,
-                            "limit": null,
-                            "note": null
-                        },
                         {
                             "key": "funnel_experiments",
                             "name": "Funnel & trend experiments",
@@ -1911,6 +1988,22 @@
                             "limit": null,
                             "note": null
                         },
+                        {
+                            "key": "group_experiments",
+                            "name": "Group experiments",
+                            "description": "Target experiments to specific groups of users so everyone in the same group gets the same variant.",
+                            "unit": null,
+                            "limit": null,
+                            "note": null
+                        },
+                        {
+                            "key": "multiple_environments",
+                            "name": "Multi-environment support",
+                            "description": "Test flags in local development or staging by using the same flag key across PostHog projects.",
+                            "unit": null,
+                            "limit": null,
+                            "note": null
+                        },
                         {
                             "key": "feature_flags_data_retention",
                             "name": "Data retention",
@@ -3637,5 +3730,11 @@
     "discount_amount_usd": null,
     "amount_off_expires_at": null,
     "never_drop_data": null,
-    "stripe_portal_url": "https://billing.stripe.com/p/session/test_YWNjdF8xSElNRERFdUlhdFJYU2R6LF9QZzdwUjRPMXBobnRsdHdKaDVpbEVzbkREcE9RQnFT0100S0djyn8G"
+    "customer_trust_scores": {
+        "surveys": 0,
+        "feature_flags": 0,
+        "session_replay": 3,
+        "product_analytics": 3
+    },
+    "stripe_portal_url": "https://billing.stripe.com/p/session/test_YWNjdF8xSElNRERFdUlhdFJYU2R6LF9QaEVaQ0hCTUE0aE8wUFhlVWVqd29MaElGd3lwRjFa010044U4IxJp"
 }