Other DB backends than Redis (support charts, better data storage) #203
Description
I was using this package with mixed results in the past.
I always found very difficult to set up the experiments correctly.
If the test should have statistical significance, I should first calculate correct sample size, then run the test and then don't look at the results until the sample size is achieved: https://www.invespcro.com/ab-testing/results-analysis/
Now I have a case when we made a change on the download page which changed the ratio of logged and unlogged users downloading our add-on like this:
On this chart I can clearly see, that there is significant change far outside regular fluctuation. If I am thinking about it, I see this as a much clearer proof that the change had influence than few numbers on the A/B testing dashboard. I can see this change correspond in time with the UI change and also if there are some other influences that could have influenced the singular numbers.
What I am trying to say is, that I would like to be able to make charts from the usages monitored by django-experiments.
I am using and developing django-admin-charts which could be used very well for this purpose. The only problem is, that it requires the data to have in PostgreSQL/MySQL database, but django-experiments uses Redis.
I always felt, that having the data in Redis is a bit wanky. I would like to store data from my experiments in the main DB to have them available forever.
@mixcloud-buildkite @millar @matclayton Do you have any idea why was Redis chosen for storing the data? What do you say to possibility of using various DB backends for this purpose?