Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Other DB backends than Redis (support charts, better data storage) #203

Open
PetrDlouhy opened this issue Aug 12, 2024 · 1 comment
Open

Comments

@PetrDlouhy
Copy link
Collaborator

PetrDlouhy commented Aug 12, 2024

I was using this package with mixed results in the past.

I always found very difficult to set up the experiments correctly.
If the test should have statistical significance, I should first calculate correct sample size, then run the test and then don't look at the results until the sample size is achieved: https://www.invespcro.com/ab-testing/results-analysis/

Now I have a case when we made a change on the download page which changed the ratio of logged and unlogged users downloading our add-on like this:
Snímek obrazovky_2024-08-12_11-05-40
On this chart I can clearly see, that there is significant change far outside regular fluctuation. If I am thinking about it, I see this as a much clearer proof that the change had influence than few numbers on the A/B testing dashboard. I can see this change correspond in time with the UI change and also if there are some other influences that could have influenced the singular numbers.

What I am trying to say is, that I would like to be able to make charts from the usages monitored by django-experiments.
I am using and developing django-admin-charts which could be used very well for this purpose. The only problem is, that it requires the data to have in PostgreSQL/MySQL database, but django-experiments uses Redis.

I always felt, that having the data in Redis is a bit wanky. I would like to store data from my experiments in the main DB to have them available forever.

@mixcloud-buildkite @millar @matclayton Do you have any idea why was Redis chosen for storing the data? What do you say to possibility of using various DB backends for this purpose?

@matclayton
Copy link
Member

Hi,

Sadly we no longer use this package internally and have started using Growthbook, we originally went with redis as it supports counters and given the volume of data we're talking about storing the event log wasn't sensible in MySQL/PostgreSQL

Hope that helps, by all means open a PR and merge etc as you need!
Mat

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants