Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Database #2110

Merged
merged 14 commits into from
Sep 24, 2020
Merged

Database #2110

merged 14 commits into from
Sep 24, 2020

Conversation

andrevmatos
Copy link
Contributor

@andrevmatos andrevmatos commented Aug 26, 2020

Fixes #2044
Fixes #1933
Fixes #1831
Fixes #2025
Fixes #2056
Fixes #2077
Fixes #2085
Fixes #1618
Fixes #1615

Short description
Use PouchDB (backed by default by IndexedDB on browsers, LevelDown on NodeJS) as a persistent storage to store state.
Only pending transfers are kept in Redux (in memory) state, and synced to the database as soon as they change, by a persister middleware. This gives us the speed of in-memory access, the backends/adapters/sync power of Pouch, and memory-safety of not keeping everything in memory all the time and instead accessing historic data only if needed.
RaidenState/Channels is kept in memory (Redux) and just synced asynchronously to the db (per individual top level keys, and individual channels).
Transfers state, as they can grow boundless (#1618), is kept on database and retrieved on demand if needed using secondary indexes. This allows us to keep sub-0.2s transfers (cli-to-cli, nodejs, webrtc) while saving everything at real time to avoid losing state.
Legacy (once) and new async migration schemas are also implemented.

Definition of Done

  • Steps to manually test the change have been documented
  • Acceptance criteria are met
  • Change has been manually tested by the reviewer (dApp)

Steps to manually test the change (dApp)

@andrevmatos andrevmatos added enhancement New feature or request sdk 🖥 dApp 📱 refactor breaking 💔 For breaking changes in the protocol or services optimization ⚡ Optimizations for the implementation or protocol security Pull requests that address a security vulnerability cli 🔤 Command-line SDK-based Node.js Client issues labels Aug 26, 2020
@github-actions
Copy link

github-actions bot commented Aug 26, 2020

You modified raiden-dapp/src,
Please remember to add a change log entry at raiden-dapp/CHANGELOG.md if necessary.

@andrevmatos andrevmatos force-pushed the feature/db branch 2 times, most recently from da442ed to 113f648 Compare August 27, 2020 16:12
@andrevmatos andrevmatos self-assigned this Aug 27, 2020
@andrevmatos andrevmatos force-pushed the feature/db branch 3 times, most recently from 6f384b2 to 8899408 Compare August 28, 2020 19:12
@andrevmatos andrevmatos force-pushed the feature/db branch 4 times, most recently from 66aea7f to 6d96db4 Compare September 5, 2020 01:41
@andrevmatos andrevmatos force-pushed the feature/db branch 2 times, most recently from 6a6502d to a524699 Compare September 13, 2020 00:57
@codecov
Copy link

codecov bot commented Sep 13, 2020

Codecov Report

Merging #2110 into master will increase coverage by 6.98%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2110      +/-   ##
==========================================
+ Coverage   87.69%   94.68%   +6.98%     
==========================================
  Files         159      154       -5     
  Lines        5999     6093      +94     
  Branches     1137     1064      -73     
==========================================
+ Hits         5261     5769     +508     
+ Misses        676      261     -415     
- Partials       62       63       +1     
Flag Coverage Δ
#dapp 91.73% <ø> (ø)
#sdk 95.89% <ø> (+9.89%) ⬆️
#sdk_e2e 67.85% <ø> (?)
#sdk_unit 85.14% <ø> (-0.86%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
raiden-dapp/src/services/raiden-service.ts 87.50% <ø> (ø)
raiden-ts/src/channels/epics.ts 97.36% <ø> (+0.55%) ⬆️
raiden-ts/src/channels/reducer.ts 98.11% <ø> (+1.11%) ⬆️
raiden-ts/src/channels/state.ts 100.00% <ø> (ø)
raiden-ts/src/channels/utils.ts 96.80% <ø> (+0.03%) ⬆️
raiden-ts/src/config.ts 100.00% <ø> (ø)
raiden-ts/src/db/epics.ts 100.00% <ø> (ø)
raiden-ts/src/db/migrations.ts 100.00% <ø> (ø)
raiden-ts/src/db/utils.ts 94.25% <ø> (ø)
raiden-ts/src/epics.ts 96.61% <ø> (+0.11%) ⬆️
... and 55 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 655e1e3...fa87efe. Read the comment docs.

@andrevmatos andrevmatos force-pushed the feature/db branch 2 times, most recently from 0d5ef5d to d5c62d4 Compare September 13, 2020 01:53
@andrevmatos andrevmatos marked this pull request as ready for review September 13, 2020 02:04
@andrevmatos andrevmatos force-pushed the feature/db branch 5 times, most recently from 98684a7 to 289c595 Compare September 18, 2020 23:09
@andrevmatos andrevmatos force-pushed the feature/db branch 5 times, most recently from c361935 to 4647e0e Compare September 23, 2020 22:45
@andrevmatos
Copy link
Contributor Author

This is too big to get reviewed. I'll merge as is, and we can test and narrow down any issues on our development branch (master) ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking 💔 For breaking changes in the protocol or services cli 🔤 Command-line SDK-based Node.js Client issues dApp 📱 enhancement New feature or request optimization ⚡ Optimizations for the implementation or protocol refactor sdk 🖥 security Pull requests that address a security vulnerability
Projects
None yet
1 participant