Pytorchures is a simple model profiler intended for any pytorch model. It measures execution time of model layers individually. Every layer of a model is wrapped with timing class which measures latency when called.
Install
pip install pytorchures
Run
from pytorchures import TimedModule
model = TimedModule(model)
_output_ = model(inputs)
profiling_data = model.get_timings()
with open(profiling_filename, "w") as f:
json.dump(profiling_data, f, indent=4)
One layer extract of sample output .json
{
"module_name": "InvertedResidual",
"device_type": "cuda",
"execution_times_ms": [
5.021333694458008,
2.427816390991211,
2.4025440216064453
],
"mean_time_ms": 3.283898035685221,
"median_time_ms": 2.427816390991211,
"sub_modules": [
{
"module_name": "Sequential",
"device_type": "cuda",
"execution_times_ms": [
4.198789596557617,
1.9135475158691406,
1.9412040710449219
],
"mean_time_ms": 2.684513727823893,
"median_time_ms": 1.9412040710449219,
"sub_modules": [
{
"module_name": "Conv2dNormActivation",
"device_type": "cuda",
"execution_times_ms": [
2.0263195037841797,
0.7545948028564453,
0.9317398071289062
],
"mean_time_ms": 1.2375513712565105,
"median_time_ms": 0.9317398071289062,
"sub_modules": [
...
This repo was developed under WSL 2 running Ubuntu 20.04 LTS, and Ubuntu 22.04 LTS. The editor of choice is VS Code.
The code was tested for Python 3.11, if you want to run other release please subsitute the python version in commands below which install python and virtual environment.
sudo apt-get update
sudo apt-get install python3.11
sudo apt-get install python3.11-venv
Install for PIL image.show() to work on WSL
sudo apt install imagemagick
If you choose to use the recommended VS Code as editor please install the extensions from extensions.json.
Create venv
python3.11 -m venv .venv
To activate venv type - VS Code should automatically detect your new venv, so select it as your default interpreter.
source venv/bin/activate
In order to be able to develop and run the code install this repo in editable mode.
pip install -e .
To install in editable mode with additional dependencies for development use the command below.
pip install -e .[dev]
The entry point to profiling the sample object detection models is
run_profiling.py file.
Running on CPU
python pytorchures/run_profiling.py --device 'cpu' --nr_images 3
Running on GPU
python pytorchures/run_profiling.py --device 'cuda' --nr_images 3
The script will print CPU wall time of every layer encountered in the model. Values are printed in a nested manner.
from pytorchures import TimedModule
model = TimedModule(model)
_output = model(inputs)
profiling_data = model.get_timings()
with open(profiling_filename, "w") as f:
json.dump(profiling_data, f, indent=4)
In the code above the model and all it's sublayers are recursively wrapped with TimedModule class which measures execution times when a layers are called and stores them for every time the model is called.
Execution times of every wrapped layer are retrieved as hierarchical dictionary using model.get_timings().
This dictionary can be saved to json file.
If for some reason there is a need to clear recorded timings call model.clear_timings(). This may useful in only some of the measurements should be included in the final results. It is often the case that first inference run takes much more time due to resource initialization, so clearing the measurements is a way to exclude this first run.
All tests are located in 'tests' folder. Please follow Arange-Act-Assert pattern for all tests. The tests should load in the test explorer.
This repo uses 'Black' code formatter.
Build the package. This command will create a dist folder with pytorchures package as .whl and tar.gz.
python -m build
Check if the build pacakge were build correctly.
twine check dist/*
Optionally upload the new package to testpypi server.
twine upload -r testpypi dist/*
To test the package from use testpypi the command:
pip install --index-url https://test.pypi.org/simple/ pytorchures
Upload the new package to production pypi server.
twine upload dist/*