Performance and scalability benchmarks for MACE

[hatemhelal]

It would be useful to have some repeatable performance benchmarks showing the expected runtime and scalability of the MACE architecture. The idea would be to use the PyTorch implementation available in this repo to run some common use cases for molecular dynamics simulations (e.g. inference with a single system). Ideally the time per evaluation would be converted into an upper bound for the simulation time per day (e.g. ns / day units). Some possibly interesting axes of investigation:

• Hardware variants: biased toward recent GPUs but still would be interesting to have some coverage from systems available to the MACE community.
• Data Types: initially comparing float64 to float32
• compilation with PyTorch compared to normal eager mode

It seems reasonable to measure the performance directly from PyTorch as real production MD codes would ideally have minimal overhead over the evaluation through PyTorch.

[ilyes319]

Indeed, I think we do have some prototype of that on the branch (https://github.com/ACEsuit/mace/blob/high_perf/mace/tools/cuda_tools.py) with the cuda kernels. It is important for the benchmark to include a wide variety of sizes/density to really get an idea (this one is not doing).