-
|
Hi all, I've been testing using S3 as a backend for ImmuDB. But what I'm noting is that even when I set the options for S3 storage, the local storage is still used (right now I'm using the env variables as described in the docs), and the data is only written to S3 when the immudb server is shutdown. Is the S3 storage feature only meant to backup the database, rather than to be used as the main storage medium? Is there a way to get it to write to S3 after , say, every few minutes, or perhaps after every commit? Any info on this will be greatly appreciated. Thanks in anticipation. Can S3 storage be used with the embedded version? |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 1 reply
-
|
Hello @tutiplain. The S3 storage backend is not meant solely as a backup feature and this should work. Can you please share more information or some logs perhaps? Do you have cloudwatch enabled on the bucket in S3? What you are trying to do should definitely work so I am wondering is its just a configuration issue somewhere. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, Thanks for replying. Right now I am using immudb server on my local dev machine, and using MinIO as my S3 backend. The MInIO instance is also running locally on my machine. I have set up the following environment variables on my machine:
I then run immudb server with the following command: The terminal output shows the following: `
|
Beta Was this translation helpful? Give feedback.
-
|
Hi @tutiplain, immudb uses a multi-file approach. Files are created with exactly the same size as they are filled up. Once a file is fully written and a new file is created for the next writes, the previous one will be automatically uploaded to the S3 server. This upload also takes place when gracefully closing immudb server |
Beta Was this translation helpful? Give feedback.
-
|
Hi @tutiplain if you want to force immudb to write all local data to S3, you need to gracefully shutdown immudb. As @jeroiraz mentioned, files are automatically synced once they reach a certain size and are filled up (currently 1MB) or when immudb is shutting down. |
Beta Was this translation helpful? Give feedback.
Hi @tutiplain, immudb uses a multi-file approach. Files are created with exactly the same size as they are filled up. Once a file is fully written and a new file is created for the next writes, the previous one will be automatically uploaded to the S3 server. This upload also takes place when gracefully closing immudb server