Model Analyzer support for s3 repositories

[Talavig]

We are using tritonserver over k8s, and we are trying to use model analyzer to auto-config our models. As it currently stands and for security reasons, we are using an s3 external storage as our model repository, and when running our tritonserver pods we are loading our models with their config.pbtxt files from there.
Because of this, we are forced to use model analyzer in remote mode for analyzing our models (our research notebooks also sit on the same kubernetes cluster).
We would like model analyzer to overwrite the config.pbtxt file in our s3 storage, but as far as we know this is not currently supported. Are there any plans to include such a feature in the near future?
Thanks in advance:)

[tgerdesnv]

Hi @Talavig, yes we will be looking into this and other features around remote/s3/cloud profiling.

Can you clarify what your ask is? Just the ability to write results directly to s3? Is this a blocker? Or just inconvenient because you have to take the remote results and manually copy them back into s3?

[Talavig]

I would like to have the ability to put an s3 path as the output path, and upload the new config.pbtxt automatically to our remote model repository, because I want to introduce automatic model configuration as part of my model deployment pipeline.
From my understanding, as it currently stands, the only way to do this now is to parse the ckpt file, reformat it as an config.pbtxt file, and upload the result outside of model analyzer.

[tgerdesnv]

From my understanding, as it currently stands, the only way to do this now is to parse the ckpt file, reformat it as an config.pbtxt file, and upload the result outside of model analyzer.

Yikes! That is messy, and I'm sorry you had to do that. The good news is that r23.11 was just released, and it includes a commit that sounds like it would be useful to you. MA will now write the config.pbtxt file when running in remote mode:
#784

Hopefully that would make your life a bit easier -- you would just need to copy the file from local to S3. However, I hear that you still want to be able to have that output go directly into an S3 directory instead of local?

[tgerdesnv]

Closing due to inactivity