look into improving startup time related to tonic tls-roots initialization

[cprice404]

While doing some benchmarking for a lambda environment I discovered that it was taking a very long time to initialize the CacheClient in cases where I wanted to create multiple gRPC channels. On my laptop, instantiating 10 of the grpc clients to support 10 channels was taking about 3 seconds.

@kvcache helped me profile and investigate and it turns out that all of the time is being spent by tonic in loading the tls-roots certificates:

client-sdk-rust/sdk/Cargo.toml

Line 38 in 223be86

tonic = { version = "0.10", features = ["tls", "tls-roots", "tls-webpki-roots"] }

If we remove the tls-roots feature from the tonic dependency specification, the time required to instantiate the clients drops to ~3 milliseconds.

However, it doesn't seem safe to disable that feature if it is only configurable at build time, it seems like it would have a high risk of breaking the TLS support in some environments.

We should probably try upgrading to a newer version of tonic and seeing if the TLS support is more configurable now. e.g., if we could manually parse the certs once on our own and then pass the result of that in explicitly so that it could be re-used across multiple clients, that would probably help a great deal. It's also possible that there are optimizations that could be made to the parsing itself.