From dab51f799f77aa72c18774faffaedf8d0ee2c977 Mon Sep 17 00:00:00 2001 From: Yuge Zhang Date: Wed, 7 Sep 2022 19:06:29 +0800 Subject: [PATCH] Release note (v2.9) (#5114) --- README.md | 2 +- docs/source/conf.py | 2 +- docs/source/release.rst | 105 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 107 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index e3bf4eb92f..04c8cdd2b9 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ NNI automates feature engineering, neural architecture search, hyperparameter tu ## What's NEW!   -* **New release**: [v2.8 is available](https://github.com/microsoft/nni/releases/tag/v2.8) - _released on June-22-2022_ +* **New release**: [v2.9 is available](https://github.com/microsoft/nni/releases/tag/v2.9) - _released on Sept-8-2022_ * **New demo available**: [Youtube entry](https://www.youtube.com/channel/UCKcafm6861B2mnYhPbZHavw) | [Bilibili 入口](https://space.bilibili.com/1649051673) - _last updated on June-22-2022_ * **New research paper**: [SparTA: Deep-Learning Model Sparsity via Tensor-with-Sparsity-Attribute](https://www.usenix.org/system/files/osdi22-zheng-ningxin.pdf) - _published in OSDI 2022_ * **New research paper**: [Privacy-preserving Online AutoML for Domain-Specific Face Detection](https://openaccess.thecvf.com/content/CVPR2022/papers/Yan_Privacy-Preserving_Online_AutoML_for_Domain-Specific_Face_Detection_CVPR_2022_paper.pdf) - _published in CVPR 2022_ diff --git a/docs/source/conf.py b/docs/source/conf.py index 384c56359b..0c4d4467da 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -31,7 +31,7 @@ version = '' # The full version, including alpha/beta/rc tags # FIXME: this should be written somewhere globally -release = 'v2.8' +release = 'v2.9' # -- General configuration --------------------------------------------------- diff --git a/docs/source/release.rst b/docs/source/release.rst index c27575257c..4c3dd5ef7b 100644 --- a/docs/source/release.rst +++ b/docs/source/release.rst @@ -5,6 +5,111 @@ Change Log ========== +Release 2.9 - 9/8/2022 +---------------------- + +Neural Architecture Search +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +* New tutorial of model space hub and one-shot strategy. + (`tutorial `__) +* Add pretrained checkpoints to AutoFormer. + (`doc `__) +* Support loading checkpoint of a trained supernet in a subnet. + (`doc `__) +* Support view and resume of NAS experiment. + (`doc `__) + +Enhancements +"""""""""""" + +* Support ``fit_kwargs`` in lightning evaluator. + (`doc `__) +* Support ``drop_path`` and ``auxiliary_loss`` in NASNet. + (`doc `__) +* Support gradient clipping in DARTS. + (`doc `__) +* Add ``export_probs`` to monitor the architecture weights. +* Rewrite configure_optimizers, functions to step optimizers / + schedulers, along with other hooks for simplicity, and to be + compatible with latest lightning (v1.7). +* Align implementation of DifferentiableCell with DARTS official repo. +* Re-implementation of ProxylessNAS. +* Move ``nni.retiarii`` code-base to ``nni.nas``. + +Bug fixes +""""""""" + +* Fix a performance issue caused by tensor formatting in ``weighted_sum``. +* Fix a misuse of lambda expression in NAS-Bench-201 search space. +* Fix the gumbel temperature schedule in Gumbel DARTS. +* Fix the architecture weight sharing when sharing labels in differentiable strategies. +* Fix the memo reusing in exporting differentiable cell. + +Compression +^^^^^^^^^^^ + +* New tutorial of pruning transformer model. + (`tutorial `__) +* Add ``TorchEvaluator``, ``LightningEvaluator``, ``TransformersEvaluator`` + to ease the expression of training logic in pruner. + (`doc `__, + `API `__) + +Enhancements +"""""""""""" + +* Promote all pruner API using ``Evaluator``, the old API is deprecated and will be removed in v3.0. + (`doc `__) +* Greatly enlarge the set of supported operators in pruning speedup via automatic operator conversion. +* Support ``lr_scheduler`` in pruning by using ``Evaluator``. +* Support pruning NLP task in ``ActivationAPoZRankPruner`` and ``ActivationMeanRankPruner``. +* Add ``training_steps``, ``regular_scale``, ``movement_mode``, ``sparse_granularity`` for ``MovementPruner``. + (`doc `__) +* Add ``GroupNorm`` replacement in pruning speedup. Thanks external contributor + `@cin-xing `__. +* Optimize ``balance`` mode performance in ``LevelPruner``. + +Bug fixes +""""""""" + +* Fix the invalid ``dependency_aware`` mode in scheduled pruners. +* Fix the bug where ``bias`` mask cannot be generated. +* Fix the bug where ``max_sparsity_per_layer`` has no effect. +* Fix ``Linear`` and ``LayerNorm`` speedup replacement in NLP task. +* Fix tracing ``LightningModule`` failed in ``pytorch_lightning >= 1.7.0``. + +Hyper-parameter optimization +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +* Fix the bug that weights are not defined correctly in ``adaptive_parzen_normal`` of TPE. + +Training service +^^^^^^^^^^^^^^^^ + +* Fix trialConcurrency bug in K8S training service: use``${envId}_run.sh`` to replace ``run.sh``. +* Fix upload dir bug in K8S training service: use a separate working + directory for each experiment. Thanks external contributor + `@amznero `__. + +Web portal +^^^^^^^^^^ + +* Support dict keys in Default metric chart in the detail page. +* Show experiment error message with small popup windows in the bottom right of the page. +* Upgrade React router to v6 to fix index router issue. +* Fix the issue of details page crashing due to choices containing ``None``. +* Fix the issue of missing dict intermediate dropdown in comparing trials dialog. + +Known issues +^^^^^^^^^^^^ + +* Activation based pruner can not support ``[batch, seq, hidden]``. +* Failed trials are NOT auto-submitted when experiment is resumed + (`[FEAT]: resume waiting/running, dedup on tuner side + (TPE-only) #4931 `__ is + reverted due to its pitfalls). + Release 2.8 - 6/22/2022 -----------------------