Replay Database

[NikhilNarayana]

Description

Add a database that caches replay data so we don't have to parse replay files all the time. Currently the work being done in this space can be seen in #400. It addresses most of the issues raised with previous implementation attempts, such as:

• Database interactions do not block main, since they run in a worker
• Migrations are automatically handled by the kysely package
• We have typed databases and typed database interactions thanks to the kysely package
• It's fairly well tested

Roll-out plan, and future improvements

1. Merge Add DatabaseReplayProvider #400
2. Once well tested, remove the old FileSystemReplayProvider
3. Add pagination and continuation support such that we don't return all replays in a folder
4. Remove virtualization in the replay browser
5. Add search/sort/filter capabilities by utilising the database
6. [Stretch] Add match/set/session groupings

Show old issue notes

There are some issues with the current implementation in the rewrite/db branch from PR #126.

1. It maxes out CPU during stats processing/db insertion.

   • so like imagine you were playing some netplay tournament, and then cuz you're launching dolphin through the launcher, and now launcher detects a bunch of new slp files so it starts indexing them by calculating stats and inserting them into the DB, your cpu is at 100% and you're dropping dolphin frames or something as a result. That's really bad. What we need is some sort of background indexer which is allowed some amount of cpu, and then just pause when dolphin is open.

2. it has some weird event listener issues cuz it kept complaining that the event listeners hit the max of 11.

3. figure out how to best manage typings between the sqlite and the rest of the app.

   • currently all the sqlite schema stuff is using strings and it's not very typesafe. i personally want to use some sort of ORM to manage database changes and migrations.

4. figure out a good workflow for database migrations without losing data and having to re-index on major schema changes.