[DMP 2024]: Integration with Metabase

[vinayvenu]

Ticket Contents

Description

Many organisations using Avni also use Metabase for analytics. The Avni hosted instance already has an instance of Metabase.

Each time a new organisation is onboarded, somebody manually creates database connection, user groups and basic visualisations for it on Metabase. This is both time-consuming and error-prone.

Metabase already has a good API that can be used to do all of this. If we can automate this mechanism, then users can directly use a bunch of predefined reports

Goals & Mid-Point Milestone

Goals

• Set up local instance of Avni and Metabase
• Create an API in avni-server to set up organisation, database, user group and collection for an organisation in Metabase through APIs from Avni
• Set up a dashboard with stats on Metabase
• [Mid-point Milestone]
• Allow call of this API from the administrator section of avni-webapp
• Define and implement all standard dashboards that can be created on Metabase

Setup/Installation

• Setup documentation here
• Ensure you set up avni-server and avni-webapp for this project. Others are not necessary.

Expected Outcome

There has to be some basic dashboards that are created on Metabase automatically.
The scope of the number of dashboards can be limited to the time available, but the solution should be extensible for newer standard visualisations
The solution should be able to recreate these dashboards as an when changes happen to Avni, and also be able to retain manual visualisations built on top of these

Acceptance Criteria

No response

Implementation Details

Metabase API - https://www.metabase.com/learn/administration/metabase-api

Required knowledge - Java, Spring Boot

Mockups/Wireframes

No response

Product Name

Avni

Organisation Name

Samanvay Foundation

Domain

Other

Tech Skills Needed

Java

Mentor(s)

@vinayvenu

Category

API, Backend, Database

Related cards

• Integration with Metabase [DMP 2024] avni-server#733

Status

https://github.com/vinayvenu/avni-c4gt-diary/blob/main/batman.md

[Harish-2003]

What else tech stack do i need to know other than Java.... Are there any beginner friendly issues to do so that i can able to get familiar with the codebase

[bhatia828]

Hi @vinayvenu , I want to contribute to this project but the installation guide is only for MAC and Ubuntu. Please let me know what can be done for windows. Thanks

[vinayvenu]

hi @bhatia828 since all developers on Avni either use a Mac or Ubuntu, nobody has yet been able to create an installation guide for windows. This will include creating scripts etc, but it doesn't exist today.

[vinayvenu]

What else tech stack do i need to know other than Java.... Are there any beginner friendly issues to do so that i can able to get familiar with the codebase

Basic knowledge of databases and APIs will be required. You should be able to look up good-first-issues on avni-server - https://github.com/avniproject/avni-server/labels/good%20first%20issue

[iamdhanasekhar]

Hi @vinayvenu I would like to contribute to this project my tech stack is spring boot, hibernate,Jpa will this tech stack enough for this project

[vinayvenu]

@iamdhanasekhar yes, that should be enough

[Shamantak12]

hi @vinayvenu ,
I want to contribute to this project would you like to assign me on this?

[vinayvenu]

@Shamantak12 you can try starting off with a smaller issue to see how things go first - https://github.com/avniproject/avni-server/labels/good%20first%20issue

[pradnya809]

@vinayvenu hi can i work on this on windows?

[AieshaShetty]

Hi @vinayvenu, I would like to contribute to this project. My tech stack includes core java, spring boot, JPA, authentication through java-jwt. Can we work on how to setup this application on Windows too?

[vinayvenu]

@AieshaShetty you can try, although you might not get help from the core contributors because none of them are on Windows. The build/run scripts on all repos will need some tweaking to work fine on Windows.

[Abhishekyaduvansh]

Hi @vinayvenu , I would also Like to contribute on this project. I have good understanding on Java with Spring boot and JPA. This would be My First time contribution to an open source project. I have the same Question how to setup this application on windows? Please let me know.

[MiirzaBaig]

Wanna contribute here, i shall go with the "good first issues" in order to understand the process. Is that correct? @AieshaShetty

[AbhimanyuSamagra]

Do not ask process related questions about how to apply and who to contact in the above ticket. The only questions allowed are about technical aspects of the project itself. If you want help with the process, you can refer instructions listed on Unstop and any further queries can be taken up on our Discord channel titled DMP queries. Here's a Video Tutorial on how to submit a proposal for a project.

[merciajeno]

Hello @vinayvenu , Regarding to database, I am a novice in MySQL. Is that sufficient for this open souce project?

[vinayvenu]

@merciajeno this will require you to set up two databases on your local machine. It requires intermediate skills in understanding databases

[abhinavkumar1122]

Hi @vinayvenu I have experience of around 3 years in java and database , so I want to contribute in this project .

[UTKARSHJHA-2005]

Hi @vinayvenu I am a 2nd year student and I want to contribute in this project.

[Kgayathri03]

Hi @vinayvenu I am a 3rd year student and I want to contribute in this project.

[Varun-Sethi-Dev]

Hey I would like to work on this I have knowledge of core java, php for backend and databases, and majorly android development could you guide me where to start and what needed to learn and how to contribute to it.

[Omsharnma]

hi @vinayvenu ,
I want to contribute to this project would you like to assign me on this?

[pinki639]

Hi @vinayvenu , I want to contribute to this project i know basic and advanced java

[ombhardwajj]

Weekly Learnings & Updates

Week 1

• DRY
• Code Smell
• Primitive Obsession
• Generic Method
• Note : A working API for Metabase was built in the week before this.

Issue:

• POST API to automate Metabase Setup. #733

Commit:

Followed DRY , Primitive Obsession and Code smell removed

Week 2

• Got to know about debugging at different levels
• Got a feel of OOPs like how we break down into smaller components

Issue:

• POST API to automate Metabase Setup. #733

Commit:

• Some Refactoring Changes

Week 3

• Working of XRays
• Visualizations in Metabase

PR:

• PR Merged:POST API to automate Metabase Setup

Week 4

-This week was about building sample dashboards manually via UI!
-Created a dashboard and it was reviewed,the stats which was not very useful/which would bloat up the screen was removed

Week 5

• Learnt about schemas in Databases

Week 6

• Learnt about Object Mapper class in Jackson

Issue:

• POST API to automate questions Setup in metabase. #733

Week 7

• TimeUnit

Issue:

• POST API to automate questions Setup in metabase. #733

Commits:

• A working API to automate the creation of questions

• Metabase package created , API to get status of DB sync created and waitforsync eliminated

Week 8

• PostConstruct

Issue:

• POST API to automate questions Setup in metabase. #733

Commits:

• Question creation for Programs and Encounters with Address table automated

Week 9

• Used the exisiting functions and created new ones!

Issue:

• POST API to automate questions Setup in metabase. #733

Commits:

• Some Refactoring changes in order to follow good coding practises
• Automated questions for Address,Media,Sync Telemetrymated

Week 10

• Tried to analyze the request body which would turn questions into model.
• It was really large couldnt complete the analysis!

Issue:

• POST API to automate questions Setup in metabase. #733

Week 11

• Fixing the code according the review comments !!

Issue:

• POST API to automate questions Setup in metabase. #733

Week 12

• Fixing the code according the review comments !!

Issue:

• POST API to automate questions Setup in metabase. #733

[ombhardwajj]

Weekly Goals

Week 1

• Follow good coding practices in the code written down in the commit made to this PR

Week 2

• Fix the permissions group code , as it broke down after changes made last week!
• Fix the primitive obsession in code!

Week 3

• Figure out how XRays work in Metabase
• Try to see how Visualizations are made in Metabase

Week 4

• Create sample dashboard(s) in Metabase For Avni Users (manually)

Week 5

• Setup the user schema for OpenCHS DB i.e. install Avni ETL

Week 6

• Write down the basic working code to automate the creation of questions!

Week 7

• Commit the working code to automate the creation of questions for subject types and location!

Week 8

• Commit the working code to automate the creation of questions for Programs ,encounters with location!

Week 9

• Commit the working code to automate the creation of questions for Address,Media and Sync Telemetry!

Week 10

• Analyze the request body to turn questions into models!

Week 11

• Fix the code committed in the last 3 weeks according to the review comments made!

Week 12

• Continue to Fix the code according to the review comments made!

[samr874]

@abhinavkumar1122 are the contributions still active for this project