* [Feature] Support torch ZeRORedundancyOptimizer

Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>

* lint

* Fix saving optimizer state_dict

* Fix handling import error

* Add test case

* fix UT

* Revert "fix UT"

This reverts commit dd64538960ff7440c6020f533d43945ffc23f2d2.

* fix handling import in UT

* Fix saving zero checkpoint and delete redundant master_only

* lint

* test unittest

* Fix handling impor error

* Fix UT condition

* Edit docstrings

* Fix typo

* Skip redundant procudure in checkpoint hook

* fix typo again

* Update mmengine/optim/optimizer/zero_optimizer.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Add api info

* lint

* Fix lint

* Handling AmpOptimWrapper case

* handling overlap_with_ddp

* Fix error

Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>
Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* [Docs] Add mmengine logo in docs

* Remove generated changelog.md

* [Enhancement] Add a deprecated function

* Refine and add ut

* tmp save

* minor refine

* Fix unit test

* Fix unit test

* fix docstring

* Update utils.rst

* init npu

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/dist/dist.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* change to is_hccl_backend

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* add comment with AmpOptimWrapper

* Update mmengine/runner/amp.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/runner/amp.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* add npu fn in base_model

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* clean lint

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/model/base_model/base_model.py

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

* add is_npu_available

* try to fix

* Add comments

* Refine grammar

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>
Co-authored-by: HAOCHENYE <21724054@zju.edu.cn>

* refine chinese doc

* remove  in title

* Update docs/zh_cn/tutorials/param_scheduler.md

* fix link

* remove files from building docs

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: zhouzaida <zhouzaida@163.com>

* [Enhance] add documents for , and support clip grad by value

* refine docstring

* fix as comment

* Fix as comment

* minor refine

* minor refine

* remove error comment for clip grad

* refine docstring

* change pre-commit-config

* modify docs with pre-commit hook

* change pre-commit-config-zh-cn

* fallback readme.md

Co-authored-by: HAOCHENYE <21724054@zju.edu.cn>

* [Fix] Fix error path of changelog.md in docs/zh_cn/index.rsts

* Fix stupid commit

* run bash script during build docs

* run bash script during build docs

* run bash script during build docs

* translate notes to 说明

* update auth

* Bump version to v0.2.0

* Add changelog

* minor refine

* minor refine

* update date

* clean the code

* minor refine

* update release date

* Fix grammar

* Fix grammar

* Fix grammar

* Fix grammar

* Minor refine

* Fix error format in readme

* update date

* minor refine

* minor refine

* minor refine

* minor refine

* refine grammar

* First commit

* add BaseTestTimeAugModel

* Add unit test

* move loop logic to test_step

* fix ddp test

* rename model to module

* optim __init__

* Fix as comment

* Fix as comment

* make val_step should not be called

* make tta do not inherit base model

* Fix unit test

* Enhance docstring

* Fix as comment

* Fix as comment

* minor refine

* minor refine

* minor refine

* fix unit test

* minor refine

* minor refine

* minor refine

* minor refine

* minor refine

* minor refine

* fix unit test

* Update config.md

* Update config.md

* [Refactor] Refactor fileio but without breaking bc

* handle compatibility

* fix format

* modify io functions

* fix ut

* fix ut

* rename method names

* refine

* refine docstring

* fix ut in windows

* update ut

* minor fix

* ensure client is not None when closing it

* add more examples for list_dir_or_file interface

* refine docstring

* refine deprecated info

* fix ut

* add a description for lmdb docstring

* [Feature] Support convert BN to SyncBN by config

* make unit test compatible with cpu

* Fix as comment

* fix unit test

* change signature of convert_sync_batchnorm: rename sync_bn to implemention

* fix unit test

* fix unit test

* [Docs] Fix api files of en docs

* minor fix

* [Docs] Fix outdated index

* update index.rst

* fix readme url

* [Docs] Add readthedocs.yml

* include torch in readthedocs.yml

* add torch torchvision in docs.txt

* [WIP] Refactor-docs-structure

* [Docs] Check link and fix rst  (#498)

* check link

* fix rst

* Fix conf.py

* Fix lint

* minor refine

* minor refine

* modify zh_readme and index.rst

* add migration exampls get_started design to en

* add tutorials to en

* correct chinese doc link

* Apply suggestions from code review

* Update docs/en/advanced_tutorials/distributed.md

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

* structure tutorials

* refine data element docs

* modify introduce

* fix comment

* fix comment

* fix comment

* migrate model

* fix title error

* refine as comment

* fix as comment

* minor refine

* Fix as comment

* Update docs/zh_cn/migration/migrate_model_from_mmcv.md

* Update docs/zh_cn/migration/migrate_model_from_mmcv.md

Co-authored-by: Qian Zhao <112053249+C1rN09@users.noreply.github.com>

* [Fix] fix the typo

* [Fix] add the missing argument

* add an example for gan training

* remove data element

* fix as comments

* add type_as

* fix as comment

* refine

* fix as comment

* add more descriptions

* fix as comment

* fix a api

* Add the distributed training script

* fix md format

* tmp commit

* refine docstring

* minor refine

* fix as comment

* add tutorial for merge_from_dict

* minor refine

* minor refine

* export md by jupyter

* temp add ipnb

* refine the docs

* remote ipynb

* fix as comment

* add change _base_.variable

* Fix as comment

* minor refine

* Fix as comment

* Fix as comment

* minor refine

* first commit

* tmp commit

* tmp commit

* tmp

* finish base module

* tmp commit

* tmp commit

* first finish

* minor refine

* clean the code

* delete design

* refine

* minor refine

* fix as comment

* fix as comment

* refine from jupyter notebook

* minor refine

* split model.md to model.md and initialize.md

* minor refine

* minor refine

* make link availiable

* fix as chengqi comment

* fix as chengqi comment

* fix link

* fix as wenwei comment

* Fix as comment

* Fix as comment

* Fix lint

* [Docs] Add data transform tutorials

* Rename `RandomMultiscaleResize` to `RandomChocieResize`.

* Imporve according to comments

* Add general data pipeline description.

* [Docs] Make API of docs clearer

* minor fix

* fix md format

* simpify logging tutorials

* minor refine

* minor refine

* refine index.rst

* fix as comment

* add description of by_epoch

* add log processor description

* minor refine

* minor refine

* Fix as comment

* Fix as comment

* Add a doc to explain how to migrate hook from mmcv

* update style

* update format

* Add functional changes of hooks

* fix table style

* refine

* fix typo