Sociogram Performance degrades substantially as redux store gets larger

[jthrilly]

The performance of the sociogram dragging degrades substantially as the redux store grows.

To reproduce:

• install the development protocol
• generate 10 sample sessions
• Dragging on the sociogram (which emits a debounced action) becomes unusable. A variant of this issue has already been identified (Drag and Drop Performance issues #1031)

[wwqrd]

@jthrilly Is this task still outstanding? I see that there was a merge against changing to a different storage backend, does that resolve this issue?

[jthrilly]

No, the data storage engine is one piece of this. This issue might be better re-phrased as "app performance generally degrades as the redux store grows".

The underlying speed of the storage system only papers over the cracks in this case (and actually, I'm not certain specifically how the performance of sqlite through a cordova plugin compares with the web view's local storage implementation - we just needed to do that for other reasons).

As I pointed out in the OP, there is another issue relevant to this here: #1031 (drag and drop performance hitches, which worsened as redux store grew).

I was able to resolve the above issue satisfactorily by correctly throttling edge update events, but once again the underlying issue is that interacting with large (and perhaps specifically deeply nested) objects in javascript is quite slow, and grows slower as the objects increase in size/complexity. Our state objects can end up being really quite large and complex, given we have (1) some number of protocols installed on the device with each containing a codebook and stage definitions, and (2) some number of sessions, each containing a full network data model, with those members themselves being collections of objects. I'm sure redux has a sophisticated diffing approach for fast updates, but I think this sort of thing is just slow in a single threaded context. Using local storage also makes this worse, because it requires serialisation.

You may recall that we discussed this on slack a while back. Our idea was to looking having separate stores. One for the app and its settings, one for protocols, and one for each session. I think looking into this is the only real solution to these issues.

[wwqrd]

So essentially every state change is causing a JSON.stringify() because it's persisted. I can see why that would slow things down! One immediate improvement that could be made to that is to not persist most of the stage interaction events, maybe "change stage" only.

In addition to separate redux stores, anything but the active protocol/session could be kept on disk and out of state.

[jthrilly]

One immediate improvement that could be made to that is to not persist most of the stage interaction events, maybe "change stage" only.

Unfortunately, this would lead to a lot of data loss potential, so I don't think we can do it. The best solution is to increase the performance of the app following the example in #1031, and to separate out the stores so that each interview session always starts from a baseline performance, and only degrades according to its own contents.