* fix build optimizer wrapper without type

* refine logic

* fix as comment

* fix optim_wrapper config error in docstring and unit test

* refine docstring of build_optim_wrapper

* Support multiple optimizers

* minor refinement

* improve unit tests

* minor fix

* Update unit tests for resuming or saving ckpt for multiple optimizers

* refine docstring

* refine docstring

* fix typo

* update docstring

* refactor the logic to build multiple optimizers

* resolve comments

* ParamSchedulers spports multiple optimizers

* add optimizer_wrapper

* fix comment and docstirng

* fix unit test

* add unit test

* refine docstring

* RuntimeInfoHook supports printing multi learning rates

* resolve comments

* add optimizer_wrapper

* fix mypy

* fix lint

* fix OptimizerWrapperDict docstring and add unit test

* rename OptimizerWrapper to OptimWrapper, OptimWrapperDict inherit OptimWrapper, and fix as comment

* Fix AmpOptimizerWrapper

* rename build_optmizer_wrapper to build_optim_wrapper

* refine optimizer wrapper

* fix AmpOptimWrapper.step, docstring

* resolve confict

* rename DefaultOptimConstructor

* fix as comment

* rename clig grad auguments

* refactor optim_wrapper config

* fix docstring of DefaultOptimWrapperConstructor

fix docstring of DefaultOptimWrapperConstructor

* add get_lr method to OptimWrapper and OptimWrapperDict

* skip some amp unit test

* fix unit test

* fix get_lr, get_momentum docstring

* refactor get_lr, get_momentum, fix as comment

* fix error message

Co-authored-by: zhouzaida <zhouzaida@163.com>

* [feat]:support display paramwise result in constructing optimizer

* [fix]:fix format issue

* delete unnecessary rank logic and fix format

* Support multiple optimizers

* minor refinement

* improve unit tests

* minor fix

* Update unit tests for resuming or saving ckpt for multiple optimizers

* refine docstring

* refine docstring

* fix typo

* update docstring

* refactor the logic to build multiple optimizers

* resolve comments

* ParamSchedulers spports multiple optimizers

* refine docstring

* RuntimeInfoHook supports printing multi learning rates

* resolve comments

* fix typo

* improve exception in call_hook

* refine unit test

* add test_call_hook

* refine

* update docstring and ut

* Update

* Add unittest

* [Fix] Fix ema state dict swapping in EMAHook.

* fix pt1.5 ut

* add more comments

* add the epoch/iter that begins validating

* fix lint

* add property and fix unit test

* minor changes

* fix typos and add unit test

* add unit test about begin

* fix docstring

* fix 'Runner' object has no attribute 'log_buffer'

* update

* add train

* [Enhance] Add RuntimeInfoHook to update runtime information.

* move lr to runtime info

* docstring

* resolve comments

* update ut and doc

* refine sync random seed

* cancel seed param in batch-sampler

* Add profiling tools

* fix docstr

* fix docstr

* update

* fix bug

* update

* update

* fix error

* fix mypy

* uodate

* merge main

* fix UT

* move get_max_cuda_memory and set_multi_processing to a public function

* fix lint

* fix lint

* fix lint

* delete _set_multi_processing

* fix error

* rename

* support custom log processor

* supplementary docs

* format code

* [Fix] Fix some bugs in hooks and runner.

* fix markdown

* fix latex formula

* resolve comments

* [Feature] Support EMA and SWA.

* add ema hook

* add avg model ut

* add more unit tests

* resolve comments

* fix warmup ema

* rename

* fix comments

* add assert

* fix typehint

* add comments

* Add pyupgrade pre-commit hook

* fix ut

* remove comments

* [Fix] Replace torch distributed with mmengine dist module

* minor refinement

* move all_reduce_params to dist.py

* add unit tests

* update unit tests

* fix test_logger.py

* add examples

* [Enhance] Make loop related attributes to be runner's properties.

* move iter and epoch to loop

* resolve comments

* implement parse_data_info

* add unit test

* fix join prefix of ann_file

* fix docstring

* Replace markdownlint with mdformat to avoid installing ruby

* fix circleci

* fix runner dump cfg

* convert dict cfg to Config

* pull flake8 from github

* pull flake8 from github

* [Refactor] Change some methods to static methods

* only change build_dataloader to static method

* [Feature] Support convert epoch-based schedulers to iter-based.

* Support convert and refactor LR and Momentum to mixin.

* Add unit tests

* fix args and add runner ut

* resolve comments

* add docs for custom dataset wrapper

* Update basedataset.md

* add import error information

* Update config with newest mmcv

* add empty line to test config

* fix logger hook

* fix to record multi-level logs, log train/a/b to a/b

* fix loggerhook json path

* fix uunit test

* change runner.train_dataloader to runner.train_loop.dataloader

* clean debug code

* refie comments and docstring

* fix unit test

* refactor instancedata

* fix docs

* fix comment

* [Fix] Fix RecursionError when key is invalid

* refine comments

* change CheckPointHook before_run to before train

* using tmp_path in each checkpointhook test case

* [Enhance] Refine registry logging info.

* update

* fix

* [Feature] Support collect all registered module info.

* update

* update

* add unit tests

* add to runner

* resolve comments