v0.7.5: Patch release

What's Changed

• Fix Bug #2319 by @jomayeri in #2438
• update pytorch pool operator function signiture by @cli99 in #2443
• Fix build issues on Windows by @eltonzheng in #2428
• rollback ds config changes by @cli99 in #2395
• Use CUDA events for inference model profiling by @mrwyattii in #2371
• Fixing a config mismatch in unit test. by @jomayeri in #2447
• Reduction Kernel Utility by @cmikeh2 in #2436
• deepspeed/launcher/launch.py: add option enable_each_rank_log by @guoyejun in #2409
• Fixes for various CI problems by @mrwyattii in #2457
• Cache Allocation and Softmax Fixes by @cmikeh2 in #2433
• Fix checkpoint loading at inference-engine by @RezaYazdaniAminabadi in #2429
• Create a new folder structure to isolate model-specific code in DS by @awan-10 in #2464
• don't gather partitioned activations for mp size 1 by @guoyejun in #2454
• Updating autotune json default in docs. by @jomayeri in #2476
• Added MLFLOW environment variables for logging metrics within trainig… by @savitamittal1 in #2477
• fix accelerate link in README by @kyoto7250 in #2481
• Fix Stable-Diffusion: Add correct memory-allocation at DeepSpeed-Attention by @RezaYazdaniAminabadi in #2474
• Fix CI issues related to cupy install by @mrwyattii in #2483
• Add scale_attn_by_inverse_layer_idx feature by @hyunwoongko in #2486
• Stable Diffusion Enhancements by @cmikeh2 in #2491
• stage_1_and_2.py: no allreduce needed when mp size is 1 by @guoyejun in #2494
• Make bf16_optimizer work for non pipeline parallelism by @tjruwase in #2470
• Fix nightly CI tests by @mrwyattii in #2493
• Make data contiguous before the inplace reshape-copy_ function. by @lokoppakmsft in #2489
• Fix typos: deepseed -> deepspeed by @jinyouzhi in #2499

New Contributors

• @guoyejun made their first contribution in #2409
• @savitamittal1 made their first contribution in #2477
• @kyoto7250 made their first contribution in #2481
• @lokoppakmsft made their first contribution in #2489
• @jinyouzhi made their first contribution in #2499

Full Changelog: v0.7.4...v0.7.5

v0.7.4: Patch release

What's Changed

• MOE residual matmult unit test by @samadejacobs in #2323
• MOE matmult with memaccess by @samadejacobs in #2336
• Refactor residual add kernels by @arashb in #2333
• mem access for quantize kernel by @GuanhuaWang in #2331
• increase min pre-commit versions by @jeffra in #2346
• Extend scratch buffer for long prompts by @cmikeh2 in #2212
• [docs] fix zero docs by @jeffra in #2350
• Staging profile inference v1 (#2348) by @awan-10 in #2349
• Kernel Data Conversion Utility by @cmikeh2 in #2327
• Add Onebit Optimizers in init by @l4d2boomer in #2340
• docs(mixture-of-experts-inference): fix typo in tuto by @jqueguiner in #2345
• Use blob storage for datasets in unit tests by @mrwyattii in #2342
• Refactor gptj_residual_add kernels for better readability by @arashb in #2358
• Updated issue templates by @jeffra in #2363
• fix cuda invalid config error in dequant kernel by @GuanhuaWang in #2362
• Add missing pytest fixture scope by @arashb in #2353
• Extend residual_add kernel tests to cover pre_attn_norm by @arashb in #2354
• Refactor fused_bias_residual kernels for better readability by @arashb in #2356
• Capture error message during sweep tests by @molly-smith in #2351
• Fix an exception when auto-casting dicts to fp16 by @mjksmith in #2370
• Refactor remaining distributed tests by @mrwyattii in #2216
• Fix the MLP output tensor's shape by @arashb in #2380
• add 11.8 to cuda_minor_mismatch_ok to allow building with current CUDA by @Thomas-MMJ in #2390
• Pin Transformers test version by @mrwyattii in #2402
• Change type to tuple in replace_wo_policy isinstance check by @lekurile in #2387
• Checkpoint backwards-compatbility workaround by @tjruwase in #2384
• Add Predicated Global Load to Memory Access Utils by @cmikeh2 in #2373
• MII blog post by @jeffra in #2418
• Fix figure reference by @awan-10 in #2419
• Add SLURM Multinode Runner by @dashstander in #2404
• Fix issue with corrupted output on long generation for GPT by @andrewchernyh in #2359
• Fix GPT Neo-X multi-gpu inference by @andrewchernyh in #2401
• CI fixes related to triton by @jeffra in #2422
• [docs] update mii blog title by @jeffra in #2423
• add SD injection policy by @jeffra in #2381
• Fix checkpoint loading when it is a dictionary by @RezaYazdaniAminabadi in #2425
• Make error regex more generic in collect_results.py by @molly-smith in #2415
• fixes #2389 by @clumsy in #2411
• Fix for inference gpt-j test by @mrwyattii in #2430
• Fixing bug 2361 by @jomayeri in #2410
• Universal checkpoint for zero stage 1 by @tjruwase in #2284
• only add deps if extra is explicitly called by @jeffra in #2432
• Add TestInjectionPolicy inference unittest class for testing custom injection policies by @lekurile in #2426
• [memory estimators] new config args sync by @stas00 in #2431
• parallelize writing of layer checkpoint files across data parallel instances by @adammoody in #1419
• Fix broken link to DeepSpeed Megatron fork by @lekurile in #2440

New Contributors

• @l4d2boomer made their first contribution in #2340
• @jqueguiner made their first contribution in #2345
• @mjksmith made their first contribution in #2370
• @Thomas-MMJ made their first contribution in #2390
• @lekurile made their first contribution in #2387
• @dashstander made their first contribution in #2404
• @andrewchernyh made their first contribution in #2359
• @clumsy made their first contribution in #2411
• @jomayeri made their first contribution in #2410

Full Changelog: v0.7.3...v0.7.4

v0.7.3: Patch release

What's Changed

• Add blob storage to CI runners by @mrwyattii in #2260
• Update replace_module.py, test-gptj.py related fix by @molly-smith in #2269
• Fix OrderedDict import for python3.6 by @Dipet in #2267
• Ds inference/fix mp2 by @RezaYazdaniAminabadi in #2270
• Trajepl: nebula load fix by @trajepl in #2182
• Prevent torch ext folder mkdir at tmp by @jeffra in #2274
• Ds-inference Int8 support through ZeroQuant technology by @RezaYazdaniAminabadi in #2217
• add a new unit test for cuda ops by @awan-10 in #2278
• Addition to code owners file by @cmikeh2 in #2279
• Memory Access Utility by @cmikeh2 in #2276
• Fp32 accuracy bug fix by @RezaYazdaniAminabadi in #2285
• Refactor universal checkpointing and tensor fragments by @tjruwase in #2253
• [ds-inference] fix progress bar by @stas00 in #2286
• Offload all gradients to nvme by @tjruwase in #2282
• fused bias relu unittest by @molly-smith in #2297
• Fix for pytest picking up wrong deepspeed by @mrwyattii in #2299
• Fix for Zero3 when MP>1 by @Quentin-Anthony in #2289
• Unit test for bias add kernel by @mrwyattii in #2298
• Update relu.cu with mem_access_utils by @molly-smith in #2306
• Add tensor parallel inference unit tests by @mrwyattii in #2232
• Fix the residual add mp scaling for GPTNeoX by @arashb in #2310
• Add unit tests for residual_add kernel by @arashb in #2307
• add inference eval scripts by @jeffra in #2303
• Upgrade P40 tests to torch 1.8 by @mrwyattii in #2316
• ZeRO-Inference blog by @tjruwase in #2271
• ZeRO-Inference blog - wrap up by @tjruwase in #2321
• ZeRO-Inference blog - Update README by @tjruwase in #2322
• Refactor relu bias add with mem_access utils by @mrwyattii in #2317
• add quant unit test by @GuanhuaWang in #2315
• only override forward if using cuda-graph by @jeffra in #2291
• Add more options to inference benchmark by @mrwyattii in #2325

New Contributors

• @molly-smith made their first contribution in #2269

Full Changelog: v0.7.2...v0.7.3

v0.7.2: Patch release

What's Changed

• Enable contiguous gradients with Z1+MoE by @siddharth9820 in #2250
• Correctly detect CPU optimizer usage by @tjruwase in #2257
• Update Half Precision Kernel Compatibility by @cmikeh2 in #2261
• fix #2240: wrong time unit in flops_profiler by @yzs981130 in #2241

New Contributors

• @cmikeh2 made their first contribution in #2261
• @yzs981130 made their first contribution in #2241

Full Changelog: v0.7.1...v0.7.2

v0.7.1: Patch release

What's Changed

• Fix for distributed tests on pytorch>=1.12 by @mrwyattii in #2141
• delay torch import for inference compatability check by @jeffra in #2167
• Fix wrong unit of latency in flops-profiler (#2090) by @zionwu in #2095
• [docs] adoption updates by @jeffra in #2173
• Update for AMD CI workflow by @mrwyattii in #2172
• [docs] update offload docs to include stage 1 by @jeffra in #2178
• Fixing model partitioning without injection by @RezaYazdaniAminabadi in #2179
• Match compute and reduce dtype by @tjruwase in #2145
• Enable fused_lamb_cuda_kernel on ROCm by @rraminen in #2148
• Update README to latest Composer version by @hanlint in #2177
• [deepspeed/autotuner] Missing hjson import by @rahilbathwal5 in #2175
• [docs] add more models to adoption by @jeffra in #2189
• [CI] fix lightning tests by @jeffra in #2190
• Fix typos on README.md by @gasparitiago in #2192
• Fix the layer-past for GPT based models by @RezaYazdaniAminabadi in #2196
• Add gradient_average flag support for sparse grads by @Dipet in #2188
• Adding the compression tutorial on GPT distillation and quantization by @minjiaz in #2197
• Log user config exactly by @tjruwase in #2201
• Fix the tensor-slicing copy for qkv parameters by @RezaYazdaniAminabadi in #2198
• Refactor Distributed Tests by @mrwyattii in #2180
• fix table syntax by @kamalkraj in #2204
• Correctly detect offload configuration by @tjruwase in #2208
• add cuda 11.7 by @jeffra in #2211
• use torch 1.9 in accelerate tests by @jeffra in #2215
• [zero-3] print warning once and support torch parameter by @awan-10 in #2127
• Add support of OPT models by @arashb in #2205
• fix typos in readme. by @zhjohnchan in #2218
• Fix regression w. dist_init_required by @jeffra in #2225
• add doc for new bert example by @conglongli in #2224
• Remove the random-generator from context during inference by @RezaYazdaniAminabadi in #2228
• allow saving ckpt w/o ckpt json + bloom copy fix by @jeffra in #2237
• Correctly detect zero_offload by @tjruwase in #2213
• [docs] update community videos by @jeffra in #2249
• Refactor dist tests: Checkpointing by @tjruwase in #2202
• Make OPT policy backward compatible with pre-OPT transformers versions by @arashb in #2254
• fix ds-inference without policy by @RezaYazdaniAminabadi in #2247

New Contributors

• @zionwu made their first contribution in #2095
• @hanlint made their first contribution in #2177
• @rahilbathwal5 made their first contribution in #2175
• @gasparitiago made their first contribution in #2192
• @arashb made their first contribution in #2205
• @zhjohnchan made their first contribution in #2218

Full Changelog: v0.7.0...v0.7.1

DeepSpeed v0.7.0

New features

• DeepSpeed Compression: https://www.microsoft.com/en-us/research/blog/deepspeed-compression-a-composable-library-for-extreme-compression-and-zero-cost-quantization/

What's Changed

• Adding DeepSpeed Compression Composer by @yaozhewei in #2105
• Remove hardcoded ROCm install path by @mrwyattii in #2093
• Fix softmax dim of Residual MoE implementation in moe/layer.py by @hero007feng in #2110
• reduce ds-inference log verbosity by @jeffra in #2111
• DeepSpeed Compression announcement by @conglongli in #2114
• Checkpoint reshaping by @tjruwase in #1953
• Fix init_process_group by @Quentin-Anthony in #2121
• DS Benchmarks QoL Improvements by @Quentin-Anthony in #2120
• [ROCm] Wrong command broke ROCm build. by @jpvillam-amd in #2118
• DeepSpeed Communication Profiling and Logging by @Quentin-Anthony in #2012
• Add flake8 to pre-commit checks by @aphedges in #2051
• Fix conflict between Tutel and top-2 gate in MoE layer by @yetiansh in #2053
• adding HF Accelerate+DS tests workflow by @pacman100 in #2134
• [inference tests] turn off time check for now by @jeffra in #2142
• Allow turning off loss scaling wrt GAS + update tput calculator by @jeffra in #2140
• Refactor ZeRO configs to use Pydantic by @mrwyattii in #2004
• Add purely-local sliding window sparse attention config by @Quentin-Anthony in #1962
• Trajepl/nebula ckpt engine by @trajepl in #2085
• Graceful exit on failures for multi-node runs by @jerrymannil in #2008
• fix: fix BF16_Optimizer compatibility issue by @shjwudp in #2152
• Fix random token-generation issue + MP-checkpoint loading/saving by @RezaYazdaniAminabadi in #2132
• Added retain_graph as a kwarg to the main engine backward function by @ncilfone in #1149
• Elastic Training support in DeepSpeed by @aj-prime in #2156
• prevent cuda 10 builds of inference kernels on ampere by @jeffra in #2157
• [zero-3] shutdown zero.Init from within ds.init by @jeffra in #2150
• enable fp16 input autocasting by @jeffra in #2158
• Release swap buffers for persisted params by @tjruwase in #2089
• Tensor parallelism for Mixture of Experts by @siddharth9820 in #2074

New Contributors

• @hero007feng made their first contribution in #2110
• @jpvillam-amd made their first contribution in #2118
• @yetiansh made their first contribution in #2053
• @pacman100 made their first contribution in #2134
• @jimwu6 made their first contribution in #2144
• @trajepl made their first contribution in #2085
• @ncilfone made their first contribution in #1149

Full Changelog: v0.6.7...v0.7.0

v0.6.7: Patch release

What's Changed

• Add Inference support for running the BigScience-BLOOM Architecture by @RezaYazdaniAminabadi in #2083
• [ds-inference] checkpoint loading => tqdm by @stas00 in #2107
• Dont overwrite hook handles in flop profiler by @Sanger2000 in #2106
• Support HuggingFace NeoX injection policy by @mrwyattii in #2087

Full Changelog: v0.6.6...v0.6.7

v0.6.6: Patch release

What's Changed

• [docs] add 530b paper by @jeffra in #1979
• small fix for the HF Bert models by @RezaYazdaniAminabadi in #1984
• Add unit test for various model families and inference tasks by @mrwyattii in #1981
• Fix for lightning tests by @mrwyattii in #1988
• fix typo when getting kernel dim in conv calculation by @cli99 in #1989
• Add torch-latest and torch-nightly CI workflows by @mrwyattii in #1990
• [bug] Add user-defined launcher args for MPI launcher by @mrwyattii in #1933
• Propagate max errorcode to deepspeed when using PDSH launcher by @jerrymannil in #1994
• [docs] add new build badges to landing page by @jeffra in #1998
• DeepSpeed Comm. Backend v1 by @awan-10 in #1985
• Relax DeepSpeed MoE ZeRO-1 Assertion by @Quentin-Anthony in #2007
• update CODEOWNERS by @conglongli in #2017
• [CI] force upgrade HF dependencies & output py env by @jeffra in #2015
• [inference] test suite for ds-kernels (bert, roberta, gpt2, gpt-neo, gpt-j) by @jeffra in #1992
• DeepSpeed examples refresh by @jeffra in #2021
• Fix transformer API for training-evaluation pipeline by @RezaYazdaniAminabadi in #2018
• DataLoader Length Fix by @Sanger2000 in #1718
• DeepSpeed Monitor Module (Master) by @Quentin-Anthony in #2013
• Use partition numel by @tjruwase in #2011
• fix import errors by @KMFODA in #2026
• Fix inference unit test import error catching by @mrwyattii in #2024
• Retain available params until last use by @tjruwase in #2016
• Split parameter offload from z3 by @tjruwase in #2009
• Fix flops profiler print statements by @mrwyattii in #2038
• Add compression papers by @conglongli in #2042
• Fix the half-precision version of CPU-Adam by @RezaYazdaniAminabadi in #2032
• Fix for AMD unit tests by @mrwyattii in #2047
• Wrong partition_id while copying fp32_params -> fp16 params in Z2 for MoE by @siddharth9820 in #2058
• Fix missing import in replace_module.py by @aphedges in #2050
• Comms Benchmarks by @Quentin-Anthony in #2040
• add ds inference paper by @jeffra in #2072
• Comments for better understanding of zero stage1_2 by @kisseternity in #2027
• [docs] fix broken read-the-docs build by @jeffra in #2075
• Fix building package without a GPU by @aphedges in #2049
• Fix partition id in the fp32->fp16 param copying step for z2+cpu-offload by @siddharth9820 in #2059
• Codeowner addendum and fix to small model debugging script by @samadejacobs in #2076
• remove require grad in params count by @cli99 in #2065
• Add missing newline for ZeroOneAdam parameter table by @manuelciosici in #2088
• fixed "None type has no len()" by @xiazeyu in #2091
• Improving memory utilization of Z2+MoE by @siddharth9820 in #2079

New Contributors

• @jerrymannil made their first contribution in #1994
• @Sanger2000 made their first contribution in #1718
• @KMFODA made their first contribution in #2026
• @siddharth9820 made their first contribution in #2058
• @samadejacobs made their first contribution in #2076
• @xiazeyu made their first contribution in #2091

Full Changelog: v0.6.5...v0.6.6

v0.6.5: Patch release

What's Changed

• GatheredParameters - accept a tuple of params by @stas00 in #1941
• Update partition_parameters.py by @manuelciosici in #1943
• fix step in adam by @szhengac in #1823
• [pipe] prevent deadlock with multiple evals sequence by @stas00 in #1944
• Fairseq support by @jeffra in #1915
• DeepSpeed needs to start cleaning up by @tjruwase in #1947
• trivial fix by @kisseternity in #1954
• Enabling CUDA-graph for the bert-type models by @RezaYazdaniAminabadi in #1952
• Add loss scale guard to avoid inf loop by @Quentin-Anthony in #1958
• [launcher] add option to bypass ssh check by @liamcli in #1957
• Bump nokogiri from 1.13.4 to 1.13.6 in /docs by @dependabot in #1965
• Fix typo in timer.py by @Quentin-Anthony in #1964
• [docs] fix dependabot version issue by @jeffra in #1966
• Don't add curand on rocm by @jeffra in #1968
• Add Unidirectional Sparse Attention Type to BigBird and BSLongformer by @Quentin-Anthony in #1959
• Fix: Sparse tensors not updating by @Dipet in #1914
• Fixing several bugs in the inference-api and the kernels by @RezaYazdaniAminabadi in #1951

New Contributors

• @Quentin-Anthony made their first contribution in #1958

Full Changelog: v0.6.4...v0.6.5

v0.6.4: Patch release

What's Changed

• [fix] Windows installs cannot import fcntl by @mrwyattii in #1921
• [build] explicitly add op_builder to manifest by @jeffra in #1920
• Enable DeepSpeed inference on ROCm by @rraminen in #1922
• bf16 inference by @tjruwase in #1917
• spell err by @kisseternity in #1929
• [ZeRO-3] Rename confusing log message by @jeffra in #1932
• [bug] Fix time log error in PipelineEngine by @Codle in #1934
• Improve z3 trace management by @tjruwase in #1916

New Contributors

• @kisseternity made their first contribution in #1929
• @Codle made their first contribution in #1934

Full Changelog: v0.6.3...v0.6.4