Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix NPS Survey Gating #5356

Merged
merged 19 commits into from
Apr 6, 2024
Merged

Fix NPS Survey Gating #5356

merged 19 commits into from
Apr 6, 2024

Conversation

adhiamboperes
Copy link
Collaborator

@adhiamboperes adhiamboperes commented Mar 11, 2024

Explanation

I found out during testing that there is an underlying gating DataProvider update behavior that occurs because of the way I initially designed the survey popup behaviour:

  • when the survey popup is displayed, we update the user profile with the current survey shown timestamp, to be used as gating so that the survey isn't triggered again before the grace period ends.
  • Updating the profile notifies upstream observers, including the gating DataProvider.
  • Due to this update, the gating provider is triggered to recompute.
  • This time, the gating result will be false for triggering the survey, since the grace period condition has changed.
  • The UI behavior is to navigate away from the exploration screen if the survey isn't to be shown. The activity is destroyed, and the popup along with it.

I mitigated this by removing the observer once the initial gating result has been received and processed.

Tests

I added tests that test survey gating based on changes in the time criterion, while other gating criteria remains constant/fulfilled. Exhaustive gating tests were added in the SurveyGatingController.

I also made a change in the StateFragmentTest that unregisters the idling resource after initializing profiles instead of at the end of the test runs. Previously, StateFragmentTestwould not run locally and fail with the error:

Error performing 'single click - At Coordinates: 173, 329 and precision: 16, 16' on view 'with id: org.oppia.android:id/play_test_exploration_button'.
at androidx.test.espresso.PerformException$Builder.build(PerformException.java:86)
at androidx.test.espresso.base.DefaultFailureHandler.getUserFriendlyError(DefaultFailureHandler.java:87)
at androidx.test.espresso.base.DefaultFailureHandler.handle(DefaultFailureHandler.java:59)
at androidx.test.espresso.ViewInteraction.waitForAndHandleInteractionResults(ViewInteraction.java:322)
at androidx.test.espresso.ViewInteraction.desugaredPerform(ViewInteraction.java:178)
at androidx.test.espresso.ViewInteraction.perform(ViewInteraction.java:119)
at org.oppia.android.app.player.state.StateFragmentTest.startPlayingExploration(StateFragmentTest.kt:475)org.oppia.android.app.player.state.StateFragmentTest.testFinishChapter_lateNight_isPastGracePeriod_minimumAggregateTimeMet_noSurveyPopup(StateFragmentTest.kt:264)
Caused by: androidx.test.espresso.IdlingResourceTimeoutException: Wait for [TestCoroutineDispatcherIdlingResource] to become idle timed out
at androidx.test.espresso.IdlingPolicy.handleTimeout(IdlingPolicy.java:61)
at androidx.test.espresso.base.UiControllerImpl$5.resourcesHaveTimedOut(UiControllerImpl.java:434)
at androidx.test.espresso.base.IdlingResourceRegistry$Dispatcher.handleTimeout(IdlingResourceRegistry.java:481)

This was caused by registering an idling resource before test begins, then when the activity starts, the main thread becomes busy. Generally, the test will continue when the main thread becomes idle, but the idling resource now blocks it and does not call onTransitionToIdle(). So it will always time out. Unregistering the idling resource before the activity starts frees up the main thread.

Disabled running on Robolectric due to known issue with Drag and Drop interaction.

Espresso Run Screenshot
Screenshot 2024-03-13 at 19 35 12

Essential Checklist

  • The PR title and explanation each start with "Fix #bugnum: " (If this PR fixes part of an issue, prefix the title with "Fix part of #bugnum: ...".)
  • Any changes to scripts/assets files have their rationale included in the PR explanation.
  • The PR follows the style guide.
  • The PR does not contain any unnecessary code changes from Android Studio (reference).
  • The PR is made from a branch that's not called "develop" and is up-to-date with "develop".
  • The PR is assigned to the appropriate reviewers (reference).

For UI-specific PRs only

If your PR includes UI-related changes, then:

  • Add screenshots for portrait/landscape for both a tablet & phone of the before & after UI changes
  • For the screenshots above, include both English and pseudo-localized (RTL) screenshots (see RTL guide)
  • Add a video showing the full UX flow with a screen reader enabled (see accessibility guide)
  • For PRs introducing new UI elements or color changes, both light and dark mode screenshots must be included
  • Add a screenshot demonstrating that you ran affected Espresso tests locally & that they're passing

@adhiamboperes adhiamboperes self-assigned this Mar 11, 2024
@adhiamboperes adhiamboperes marked this pull request as ready for review March 12, 2024 22:34
@adhiamboperes adhiamboperes requested review from a team as code owners March 12, 2024 22:34
@adhiamboperes
Copy link
Collaborator Author

@BenHenning, PTAL.

@BenHenning
Copy link
Member

@adhiamboperes is it possible to add tests for any of the discovered user-facing problems?

@adhiamboperes
Copy link
Collaborator Author

@adhiamboperes is it possible to add tests for any of the discovered user-facing problems?

@BenHenning, I have added some UI tests and updated the PR description.

Copy link
Member

@BenHenning BenHenning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @adhiamboperes! Left a few comments--PTAL.

@adhiamboperes
Copy link
Collaborator Author

@BenHenning, PTAL.

Copy link
Member

@BenHenning BenHenning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @adhiamboperes! Just a couple of follow-ups. Feel free to merge once they're addressed since the PR otherwise LGTM.

@oppiabot oppiabot bot added the PR: LGTM label Apr 4, 2024
Copy link

oppiabot bot commented Apr 4, 2024

Hi @adhiamboperes, this PR is ready to be merged. Please address any remaining comments prior to merging, and feel free to merge this PR once the CI checks pass and you're happy with it. Thanks!

@adhiamboperes adhiamboperes enabled auto-merge (squash) April 5, 2024 20:44
@adhiamboperes
Copy link
Collaborator Author

Resolved all comments as fixed and enabling auto merge.

@adhiamboperes adhiamboperes merged commit 9e8553c into develop Apr 6, 2024
37 checks passed
@adhiamboperes adhiamboperes deleted the fix-survey-gating branch April 6, 2024 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants