Upgrade Prod Superset instance to version 4.0.1

[himeshr]

Acceptance Criteria

• Upgrade our current prod superset to version 4.0.1. https://github.com/apache/superset/releases/tag/4.0.1
  • The metadata is currently stored on the ec2 instance itself, therefore, we'll need to ensure that we take a manual backup before we start with the upgradation work.
• Also, setup automatic storage backups for the prod superset ec2 instance for recovery purposes
• Configure the system to use required features
  • TAGGING_SYSTEM
  • Drill-By
  • Full CSV export
    Discuss with implementation team and update this section.

Fixing issue with existing report after upgradation

1. Analyse report.
2. If report broken then Categorise into type of issues .
3. Document recovery steps for each issue.
4. Apply fix in all broken reports.

Validation

• All existing reports should not be broken, make changes in reports via sql commands or so to fix large scale modifications required

Tech details

• Create a replica of prod superset ec2 instance
• Upgrade to latest 4.1 version of superset
• No Containers switch
• No DB upgrade
• Configure the system to use all new features
  • TAGGING_SYSTEM
  • Drill-By
  • Full CSV export
• Compare the old and new systems and give it out for UAT to implementation team
• After successful UAT, perform the similar upgradation on Prod Superset
• Bring down the temp infra which is no longer required (Routes and EC2 instances)

Reference doc

https://preset.io/blog/apache-superset-4-0-release-notes/
https://preset.io/blog/superset-3-0-release-notes/

[vedfordev]

Created Documentation For Update File

Issues and Workaround Sheet

[vedfordev]

blocked by apache/superset#28020

Moving card to on hold

[vedfordev]

move card to qa ready. implementation team will do sanity test. @himeshr @pkundu

[himeshr]

Prod Superset: https://reporting-superset.avniproject.org/login/
Test Superset: https://test-reporting-superset.avniproject.org/login/

[dinesh2096]

QA Notes :

Charts

• Mandatory field should be displayed - In the old env in aggregrate we showing the mandarory field but in the new env we didnt
• Jinja Template is missing in the reports - In the old report we have the jinja added in the report but in the new env it is misisng
• Under data we missed Include time - Under the data in the old env we have the option named as include time but missing in the new env
• Total is missing in the result - In the charts under result the count is not displayed

Users :

• Getting unexpected error - when we change the role for the user the report breaks

[vedfordev]

QA Notes :

Charts

• Mandatory field should be displayed - In the old env in aggregrate we showing the mandarory field but in the new env we didnt
• Jinja Template is missing in the reports - In the old report we have the jinja added in the report but in the new env it is misisng
• Under data we missed Include time - Under the data in the old env we have the option named as include time but missing in the new env
• Total is missing in the result - In the charts under result the count is not displayed

Users :

• Getting unexpected error - when we change the role for the user the report breaks

last one was the issue that resolved. others are mostly ui issue that we can't resolve.

[dinesh2096]

• when we login to the user we are getting a warning message i have tested in diff org and diff user (Unable to see recent)

Click here to watch video

• When we have role added to the user we are getting the error

Click here to watch video

[himeshr]

@vedfordev Include Dinesh, me and a representative from Implementation team tech leads, do discuss the issues reported by Dinesh and the solutions found / issues disregarded.

After this, we'll determine the future course of action related to migration of superset.
We might have to present this to rest of the product team and get go ahead before asking for prod migration date from Implementation team.