Need more than 5Mb cache size

[gilescope]

There's a hard limit on chrome of 5Mb which is utf16 so typically you'll have 2.5mb to play with.

IndexedDb seems to have a minimum of 10x the size by default but looks to be much bigger if you have a large storage size.

It would be great if there was a feature to use indexeddb.

[gilescope]

Looks like https://github.com/devashishdxt/rexie is probably a nice way to talk to it.

[johanhelsing]

Ouch, that's good to know.

I agree it would probably be better to use indexed db (PRs welcome).

Rexie looks promising, thanks for the tip.

At the very least, it would be good to put a warning about this in the readme.

[johanhelsing]

So I looked briefly into it. It's not straightforward, since rexie has an async API and this crate presents a blocking one not sure if it's possible to hack around that or not...

One option would be to keep it all in memory, and save after every update in a future that could be polled by bevys IoTaskPool. But then we'd be restrained by browsers memory limitations :/

[Dimchikkk]

So I looked briefly into it. It's not straightforward, since rexie has an async API and this crate presents a blocking one not sure if it's possible to hack around that or not...

One option would be to keep it all in memory, and save after every update in a future that could be polled by bevys IoTaskPool. But then we'd be restrained by browsers memory limitations :/

Compared to the limited size of localStorage, being restricted by browser memory is still a significant improvement.

[johanhelsing]

Not a pure improvement, though. Browser memory is a serious limitation when doing gamedev, usually around 100 mb if i remember correctly. Also, needs a more complicated async implementation, perhaps making the bevy dependency required again...

[johanhelsing]

Not saying we shouldn't do it, just pointing out the downsides.

One option is to implement it as an alternative backend.

[Dimchikkk]

@johanhelsing from what I read on internet it's 4Gb per tab for Chrome... let's say only 3Gb is used for db, so it's 600 times bigger than 5Mb of localStorage.

Yeah, alternative backend sounds good.

[johanhelsing]

Yeah, my experience comes from dealing with unity apps at work: we had to support 32-bit browsers, which have much tighter constraints. From what I read, that means we should still have 512mb, but in practice we found that it was much lower, perhaps unity just allocates memory in a weird way or has huge spikes in heap allocation or something, idk. Regardless, luckily it's irrelevant for this project :)

I'd be happy to accept a PR implementing it with indexeddb, perhaps even making it the default.

[Dimchikkk]

So, ideally to have separate crate with indexeddb wrapper that has bevy dependency and sync API so it can easily be integrated with bevy_pkv. Did I get it right @johanhelsing ?

[johanhelsing]

I was just thinking of a feature in this crate, similar to how both rocksdb and sled is supported on native.

I guess maybe it depends on how complicated such an implementation gets, and how reusable it is whether it makes sense to split into a separate crate.

If i understand what rexie is, it may be a pretty thin wrapper: we just need to await it on the bevy io task pool.

[Dimchikkk]

Makes sense, so you are fine with making bevy dependency to be required again.

[johanhelsing]

yeah, but keep it optional for the local storage db. Perhaps there is even away to abstract away what async executor it uses so it's agnostic about it being the bevy io task pool, but not a huge priority.