PyTorch release benchmark suite Improvements

[atalman]

For PyTorch Releases we execute following benchmarks:
https://github.com/pytorch/benchmark/tree/main/userbenchmark/release-test

These are the tests that we run:

# run mnist
mkdir -p "${RESULT_DIR}/mnist"
pushd "${EXAMPLES_DIR}/mnist"
export LOG_FILE=${RESULT_DIR}/mnist/result.log
export MEM_FILE=${RESULT_DIR}/mnist/result_mem.log
${PREFIX} bash "${CURRENT_DIR}/monitor_proc.sh" python main.py --epochs 10
# run mnist-hogwild
mkdir -p ${RESULT_DIR}/mnist_hogwild
pushd "${EXAMPLES_DIR}/mnist_hogwild"
export LOG_FILE=${RESULT_DIR}/mnist_hogwild/result.log
export MEM_FILE=${RESULT_DIR}/mnist_hogwild/result_mem.log
${PREFIX} bash "${CURRENT_DIR}/monitor_proc.sh" python main.py --epochs 10
# run CPU WLM LSTM
mkdir -p ${RESULT_DIR}/wlm_cpu_lstm
pushd "${EXAMPLES_DIR}/word_language_model"
export LOG_FILE=${RESULT_DIR}/wlm_cpu_lstm/result.log
export MEM_FILE=${RESULT_DIR}/wlm_cpu_lstm/result_mem.log
${PREFIX} bash "${CURRENT_DIR}/monitor_proc.sh" python main.py --epochs 10 --model LSTM
# run GPU WLM LSTM
mkdir -p ${RESULT_DIR}/wlm_gpu_lstm
pushd "${EXAMPLES_DIR}/word_language_model"
export LOG_FILE=${RESULT_DIR}/wlm_gpu_lstm/result.log
export MEM_FILE=${RESULT_DIR}/wlm_gpu_lstm/result_mem.log
${PREFIX} bash "${CURRENT_DIR}/monitor_proc.sh" python main.py --epochs 10 --model LSTM --cuda
# run CPU WLM Transformer
mkdir -p ${RESULT_DIR}/wlm_cpu_trans
pushd "${EXAMPLES_DIR}/word_language_model"
export LOG_FILE=${RESULT_DIR}/wlm_cpu_trans/result.log
export MEM_FILE=${RESULT_DIR}/wlm_cpu_trans/result_mem.log
${PREFIX} bash "${CURRENT_DIR}/monitor_proc.sh" python main.py --epochs 10 --model Transformer
# run GPU WLM Transformer
mkdir -p ${RESULT_DIR}/wlm_gpu_trans
pushd "${EXAMPLES_DIR}/word_language_model"
export LOG_FILE=${RESULT_DIR}/wlm_gpu_trans/result.log
export MEM_FILE=${RESULT_DIR}/wlm_gpu_trans/result_mem.log
${PREFIX} bash "${CURRENT_DIR}/monitor_proc.sh" python main.py --epochs 10 --model Transformer --cuda

Models are taken from here: https://github.com/pytorch/examples

Improvement suggestions:

• Lower the epoch to 2-3 to make it run faster. Run the results and compare.
• Introduce Additional models. Something to look into: https://github.com/mlcommons/algorithmic-efficiency/tree/main/algorithmic_efficiency/workloads
• Add an option test test torch only without the domains by running models that does not require torchvision/torchaudio