AWS S3 File Availability across SDKs

[Daniel-K-Ivanov]

Hello everyone,
I am running a Java application that performs frequent ListObjectV2 requests to an S3 bucket. This application has been running smoothly, consistently retrieving file listings from S3 with expected latency.

I have the same logic written in Rust. The Rust implementation also uses ListObjectV2 requests, mirroring the Java application's logic and timing. Both applications are running on the same virtual machine, ensuring identical network conditions and bandwidth.

To my surprise, the S3 bucket updated contents are available (updated) approximately 0.5 seconds earlier in the Java application than in the Rust application. This half-second delay in the Rust version is puzzling, given that both applications should theoretically have similar access times to the updated S3 bucket content.

I am reproducing the same strange results with the official AWS Rust SDK and Rusoto.
To be clear:

• the latency of the responses and the interval between ListObjectV2 requests are identical in both the Java and Rust implementations.
• If at a time X the ListObjectV2 Java SDK receives information on the new content of the Bucket, the Rust implementations do not receive the update for at least X + ~500ms even though ListObjectV2 requests are executed multiple times between X and X+500ms yielding no "updates"

Is there something inherent to the Java SDK that leverages the S3 API better (setting specific headers for example) to achieve superior latency on Bucket updates (have the updated contents of the Bucket 0.5 seconds earlier)?

Can it be that Rust SDKs hit the external AWS Gateway API and do not utilise an internal routing which the Java SDK f.e does?

[rcoh]

Seems like you marked this as closed so I assume you figured it out. The Rust and Java SDKs are sending the exact same requests so I'm not sure why this discrepancy would exist. We'd need logs, reproducer, etc. to debug

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.