LiteFS Recommendation

[zwhitchcox]

I'm wondering why LiteFS would be recommended. It doesn't really seem to have a use case to me.

It maxes out at 100 transactions per second. If you have a large application, you are going to need more throughput than that. If you have a smaller application, you don't need replication anyway, and you can get away with volume snapshots or Litestream.

If you're trying to make your application scalable, Postgres seems like a more prudent option to me (although, I'm personally just using a SQLite server, as I only have one node atm). If you're just trying to get as much mileage as possible out of a single node application, it seems like Litestream would be a better solution.

Am I missing something?

[kentcdodds]

I don't have time to answer properly right now but I have considered this and I do have an answer. Before I give it, can you tell me what you're building that requires more than 100 writes per second?

[zwhitchcox]

Sure!

Well, we've got 70+ technicians in the fields who will be pinging their location in a background, and we'll have an API writing that to the database. Then we also have ETL's syncing the database with external services.

On top of that, we've got all the regular database writes as well as logs of who has written what, around 30 in-office people in addition to the technicians. We are working on the CRM right now, but will also have an ERP, dispatch board, inventory, customer signatures, approvals, phone call meta data, CSR statuses, customer intake information, and all kinds of stuff like that.

This is just for our first customer too, hopefully we will be growing rapidly, which will make these numbers compound.

Maybe my use case is odd though in having that much data.

[kentcdodds]

Maybe my use case is odd though in having that much data.

Honestly, it doesn't sound like what you've described will amount to more than 100 writes per second. But I suppose it could eventually as you get more customers.

Most people aren't ingesting that much data per user. You should probably use Postgres. You can still do read replicas with Postgres on Fly if you want.

I talked to Ben Johnson about the status of moving on from Fuse and ditching this limitation and he told me they got LiteFS to a solid spot and then priorities shifted so he's not certain when this work would happen.

I would welcome an example of the Epic Stack with Postgres if you want to put that together! My commit where I moved from Postgres to SQLite might be helpful! kentcdodds/kentcdodds.com@5f19e7b

Cheers :)

[zwhitchcox]

Awesome! The shift is really not that difficult..the only catch is running the tests in isolation. I think we’re just going to have one db per worker. Still working on this though. I’ll see if what we come up with is worth PRing.