You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was using this package with mixed results in the past.
I always found very difficult to set up the experiments correctly.
If the test should have statistical significance, I should first calculate correct sample size, then run the test and then don't look at the results until the sample size is achieved: https://www.invespcro.com/ab-testing/results-analysis/
Now I have a case when we made a change on the download page which changed the ratio of logged and unlogged users downloading our add-on like this:
On this chart I can clearly see, that there is significant change far outside regular fluctuation. If I am thinking about it, I see this as a much clearer proof that the change had influence than few numbers on the A/B testing dashboard. I can see this change correspond in time with the UI change and also if there are some other influences that could have influenced the singular numbers.
What I am trying to say is, that I would like to be able to make charts from the usages monitored by django-experiments.
I am using and developing django-admin-charts which could be used very well for this purpose. The only problem is, that it requires the data to have in PostgreSQL/MySQL database, but django-experiments uses Redis.
I always felt, that having the data in Redis is a bit wanky. I would like to store data from my experiments in the main DB to have them available forever.
@mixcloud-buildkite@millar@matclayton Do you have any idea why was Redis chosen for storing the data? What do you say to possibility of using various DB backends for this purpose?
The text was updated successfully, but these errors were encountered:
Sadly we no longer use this package internally and have started using Growthbook, we originally went with redis as it supports counters and given the volume of data we're talking about storing the event log wasn't sensible in MySQL/PostgreSQL
Hope that helps, by all means open a PR and merge etc as you need!
Mat
I was using this package with mixed results in the past.
I always found very difficult to set up the experiments correctly.
If the test should have statistical significance, I should first calculate correct sample size, then run the test and then don't look at the results until the sample size is achieved: https://www.invespcro.com/ab-testing/results-analysis/
Now I have a case when we made a change on the download page which changed the ratio of logged and unlogged users downloading our add-on like this:
On this chart I can clearly see, that there is significant change far outside regular fluctuation. If I am thinking about it, I see this as a much clearer proof that the change had influence than few numbers on the A/B testing dashboard. I can see this change correspond in time with the UI change and also if there are some other influences that could have influenced the singular numbers.
What I am trying to say is, that I would like to be able to make charts from the usages monitored by
django-experiments
.I am using and developing
django-admin-charts
which could be used very well for this purpose. The only problem is, that it requires the data to have in PostgreSQL/MySQL database, butdjango-experiments
uses Redis.I always felt, that having the data in Redis is a bit wanky. I would like to store data from my experiments in the main DB to have them available forever.
@mixcloud-buildkite @millar @matclayton Do you have any idea why was Redis chosen for storing the data? What do you say to possibility of using various DB backends for this purpose?
The text was updated successfully, but these errors were encountered: