S3 Bucket endpoints issue

[jwoertink]

I'm not sure if this is an issue on the s3 shard or not, but I get this error:

<Code>PermanentRedirect</Code>
<Message>The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message>
<Endpoint>s3.amazonaws.com</Endpoint>

My code looks like this:

client = Awscr::S3::Client.new("us-west-1", aws_key_id, aws_secret)
Shrine.configure do |config|
  config.storages["store"] = Shrine::Storage::S3.new(bucket: "my-app/avatars", client: client, public: true)
end

# other code...
result = Shrine.upload(upload_io, "store", metadata: { "filename" => "face.jpg" })
result.url

The file fails to upload with an error similar to above.

One thing I noticed was that on this bucket, all of the file URLs actually have bucket.region.aws.com/subfolder instead of region.aws.com/bucket/subfolder.... So what I did was added the endpoint option to my client

client = Awscr::S3::Client.new("us-west-1", aws_key_id, aws_secret, endpoint: "https://my-app.s3-us-west-1.amazonaws.com/")

Now the file actually uploads (though it sets the content-type to octetstream... 🤔 ), but the URL I get back is

https://s3-us-west-1.amazonaws.com/my-app/avatars/.....lotsmorestuff

which sends me to the S3 error page above.

[igor-alexandrov]

@jwoertink as far as I remember endpoint param should not include your bucket:

client = Awscr::S3::Client.new("us-west-1", aws_key_id, aws_secret, endpoint: "https://s3-us-west-1.amazonaws.com/")

Can you please check?

[jwoertink]

Awesome! so I updated to this:

client = Awscr::S3::Client.new("us-west-1", aws_key_id, aws_secret, endpoint: "https://s3-us-west-1.amazonaws.com")
Shrine.configure do |config|
  config.storages["avatars"] = Shrine::Storage::S3.new(bucket: "my-bucket/avatars", client: client, public: true)
end

This works 🎉 The files upload, and the result.url I get back now returns the image. It still sets the content-type to binary/octet-stream, but there's probably a setting I'm missing on that. I'll look in to the content-type plugin you have.

I guess the only other question is the URL returned is signed with all these query string params. Is there a way to not have that? Right now I'm doing this:

uri = URI.parse(result.url)
String.build do |str|
    str << "https://"
    str << "#{BUCKET}."
    str << uri.host.to_s << uri.path
end

This matches up with the object url in the S3 console

https://my-bucket.s3-us-west-1.amazonaws.com/avatars/08a58fc7ba749ea3e3b4da15b8b0597d.jpg