Indexing: limit shard parallelism when index concurrency is set

[jtibshirani]

By default, zoekt-sourcegraph-indexserver builds one repo at a time. For each
repo, shards are built in parallel using a number of threads equal to available
CPUs. There are two ways to adjust the indexing concurrency:

1. Passing cpu_fraction, which limits the available CPUs for parallel shard
   building
2. Passing index_concurrency (or setting the SRC_INDEX_CONCURRENCY
   environment variable), to index more than one repo at once

If you set index_concurrency to some number greater than 1, then indexing
will use more threads than available CPUs. This seems undesirable, especially
if you set cpu_fraction, since you'd expect that to put an upper bound on CPU
usage.

This changes the shard-level parallelism to available CPUs / index_concurrency
(rounded down), to bound the CPU usage as expected.

[keegancsmith]

FYI in practice indexConcurrency value is never configured. I think I know of one customer who uses this setting and we don't document it anywhere.

[keegancsmith]

minor: do you think we should round up rather than down? I would rather slightly oversubscribe and let the OS take care of it rather than under utilize. I suppose the issue is RAM? But even then my gut says obsersubscribe.

[jtibshirani]

Indeed I think that's a better trade-off, will update this.

I was thinking about it like... if you set CPU fraction then you want that to be an upper-bound on CPU use... but in practice it's more like a hint?

[jtibshirani]

FYI in practice indexConcurrency value is never configured

Got it! I just saw it was configured on dot com and wanted to avoid spawning too many ctags processes there (when we move towards parallel symbol building for shards).