* init npu

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/dist/dist.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* change to is_hccl_backend

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* add comment with AmpOptimWrapper

* Update mmengine/runner/amp.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/runner/amp.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* add npu fn in base_model

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* clean lint

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/model/base_model/base_model.py

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

* add is_npu_available

* try to fix

* Add comments

* Refine grammar

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>
Co-authored-by: HAOCHENYE <21724054@zju.edu.cn>

* Support multiple optimizers

* minor refinement

* improve unit tests

* minor fix

* Update unit tests for resuming or saving ckpt for multiple optimizers

* refine docstring

* refine docstring

* fix typo

* update docstring

* refactor the logic to build multiple optimizers

* resolve comments

* ParamSchedulers spports multiple optimizers

* add optimizer_wrapper

* fix comment and docstirng

* fix unit test

* add unit test

* refine docstring

* RuntimeInfoHook supports printing multi learning rates

* resolve comments

* add optimizer_wrapper

* fix mypy

* fix lint

* fix OptimizerWrapperDict docstring and add unit test

* rename OptimizerWrapper to OptimWrapper, OptimWrapperDict inherit OptimWrapper, and fix as comment

* Fix AmpOptimizerWrapper

* rename build_optmizer_wrapper to build_optim_wrapper

* refine optimizer wrapper

* fix AmpOptimWrapper.step, docstring

* resolve confict

* rename DefaultOptimConstructor

* fix as comment

* rename clig grad auguments

* refactor optim_wrapper config

* fix docstring of DefaultOptimWrapperConstructor

fix docstring of DefaultOptimWrapperConstructor

* add get_lr method to OptimWrapper and OptimWrapperDict

* skip some amp unit test

* fix unit test

* fix get_lr, get_momentum docstring

* refactor get_lr, get_momentum, fix as comment

* fix error message

Co-authored-by: zhouzaida <zhouzaida@163.com>

* Support default_scope when building optimizer and evaluator.

* add docstring

* fix

* fix

* [Feature]: Add optimzier and constructor.

* refactor unit tests

* add optimizer doc

* add parrots wrapper

* add parrots wrapper

* solve comments

* resolve comments