Excessive develop memory consumption comparing to 0.4.0

[qrilka]

Maybe it mostly comes from reflex-dom but I'm not quite sure how to trace it down properly.
Here is a PR porting minimalistic example to 0.5.0 - qrilka/reflex-huge-table#1
I was using mostly Chrome task manager to check RAM consumption and running make open and then clicking "go" button showed the following result:

• in master version (using reflex-0.4.0) initially page consumed around 28M and after button click consumed RAM jumped to 60M then after some time it decreased to 43M and was almost stable showing numbers from 43M to 44M

• in reflex-0.5.0 branch initially page was about 37M. After button click it jumped to 181M and didn't stop there and kept increasing up to 227M. Then probably some GC happened and memory shrunk to 152M. After that it wasn't quite stable and was showing numbers from that 152M up to 180M

Also I have used Chrome developer tools and took heap snapshots on initial page state and after table rendering.

branch	inital snapshot size	initial "Object"	snapshot after click	"Object" after click
master	9.4M	4113 (4%)	21.5M	153495 (58%)
reflex-0.5.0	12.7M	5662(4%)	115M	544847(76%)

Probably the way I build table in that example is not quite optimal (any advice on better way to do it?) but the main point is different run-time characteristics of the same code built with the same GHCJS running in the same Chrome but using different reflex and reflex-dom versions.

Please let me know how could I help debugging this issue further.

[qrilka]

@ryantrinkle could we have your comment on this one?

[eskimor]

I just started with reflex and ghcjs and was also quite buffled by it's high amount of memory consumption, compared to the purescript version I already have. I am glad to know that this used to be better and that it is not an inherent problem with ghcjs.

[ryantrinkle]

@qrilka I'll definitely take a look at this. Thanks for supplying the code to repro it! It can be a bit tough to keep an eye on this sort of thing, but I'm hoping we can get a CI-based system set up soon, including your code as a test case.

[qrilka]

That's great @ryantrinkle
If I could be of any help in this - please let me know

[ryantrinkle]

Well, I think what we really need is some way to know when and where performance regressions are occurring. It might be a bit of a challenge, but if you can think of some ways to work towards narrowing down which commit might have caused this increased memory usage, that would be very helpful.

[mightybyte]

Also perhaps an automated way of measuring memory consumption. Firing up Chrome developer tools manually takes time and effort.

[george-steel]

I also just ported an app to 0.5 and have also had a massive performance (both speed and memory) problems with creating tables.

[ryantrinkle]

@george-steel Sorry to hear you've run into issues, although it's useful to have another point of comparison. Sorting out this issue is a top priority for me and my team.

[george-steel]

@ryantrinkle Here are some details of how I encountered the bug:

I am displaying a table (about a dozen rows and 2 dozen columns) that has cells that get highlighted in different ways in response to user input. Using elDynAttr for the individual cells was very slow, taking over a second to update. I tried replacing that by wrapping the entire table in a dingle dyn and rebuilding it on each change, but that was almost as slow. Since your last response, I managed to get a fast solution by using some css magic and static cell attributes to only need elDynAttr on tr and col elements, which works quickly.

My app is now working, and hopefully this gives you some more info on the bug.

[ryantrinkle]

@qrilka, @george-steel: We've put in place some changes that bring down the memory consumption - not quite to reflex-dom-0.3 levels, but much less than previously. Work will continue.

[eskimor]

That is awesome! I just noticed that my current reflex version takes far longer to build up the UI than my PureScript version, (instant versus, ok this loaded, now that, wait, ok now it is complete). I don't really know what's taking so long, but any performance improvements are good I guess :-) Compare:

https://dev.gonimo.com (PureScript)
https://app.alpha.gonimo.com (Reflex, Haskell)

The difference is big on mobile, on desktop not so much.

[mightybyte]

@eskimor Thanks for that comparison. It's certainly helpful. I suspect that the majority of the difference you are noticing is simply the time it takes to download the Javascript. This would also explain the bigger difference on mobile. I would suggest compressing your JS with gzip -9 or zopfli and setting your server up to serve the compressed files. That will get you a huge improvement. If you are really aggressive, you can also run the JS through the Google Closure Compiler before compressing it.

[eskimor]

gzip -9 is used and also closure compiler. I wrote "build up the UI" on purpose, because I meant the time it takes to fully build up the UI and load data from server (Websocket) when everything is cached.

Also the compressed JS with closure is only about two times as big as the PS version.

[qrilka]

I still haven't found a way to automate measurements (couldn't spend noticeable time on it :( ) so I've rerun "manual" tests and the results look quite good. At least overhead of 0.5 version seem to scale almost linearly with number of data object used.

branch	inital snapshot size	initial "Object"	snapshot after click	"Object" after click
master	9.4M	4113 (4%)	21.5M	153495 (58%)
master(new)	10.1M	4038 (4%)	21.1M	154736 (60%)
reflex-0.5.0(old)	12.7M	5662(4%)	115M	544847(76%)
reflex-0.5.0(new)	12.8M	5230(4%)	28.9M	220991(57%)

@ryantrinkle feel free to close the ticket but probably it would be good to have another one about reflex memory profiling

[qrilka]

correction: memory consumption of 0.5 seems to grow almost proportional to the one of 0.4

[ryantrinkle]

@qrilka Oh! I should have mentioned that I automated the testing. See prof-huge-table-with, which you can invoke with a reflex-platform git hash to get a profile with it.

[eskimor]

I just updated reflex-platform again, starting times improved noticeable on my mobile! Still not great but a few seconds faster! :-)

[ryantrinkle]

@qrilka I think this is sorted out now. Of course, there will be more improvements in the future, and it's now possible to compile to native ARM, but this specific issue is fixed. Thanks for reporting!