* support save best based on multi metrics

* add unit test

* resolve bugs after rebasing

* revise docstring

* revise docstring

* fix as comment

* revise as comment

* Support changing value of dict in config(without base config)

* minor refine

* fix build multiple scheduler

* add new unit test

* fix comment and error message

* fix comment and error message

* extract _parse_scheduler_cfg

* always call build_param_scheduler during train and resume. If there is only one optimizer, the defaut value for sheduler will be a list, otherwise there is multiple optimizer, the default value of sheduler will be a dict

* minor refine

* rename runner test exp name

* fix as comment

* minor refine

* fix ut

* only check parameter scheduler

* minor refine

* make scheduler default to None

* fix bc breaking

* refine warning message

* fix as comment

* fix as comment

* fix lint

* Fix resume message hub and save metainfo in messagehub

* fix as comment

* Support use 'global var' in config function

* upload test file

* [Fix]: Fix resume bug

* [Fix]: Change last_checkpoint check logic

* [Fix]: Fix lint

* [Fix]: Change warning to print_log

* [Fix] Fix gpu tensors in results list are not on the same device.

* cast all tensor to cpu

* remove blink info debug msg

* minor refine

* minor refine

* fix save scheduler state dict with optim wrapper

* remove for loop and inherit TestParameterScheduler

* remove for loop and inherit TestParameterScheduler

* minor refine

* [Enhance] Allow Compose(None)

* add typehint

* fix

* [Fix] Fix weight initializing and registry logging.

* sync params

* resolve comments

* [Enhance] Auto set the `end` of param schedulers.

* Add log output and unit test

* Update docstring

* Update unit tests of `CosineAnnealingParamScheduler`.

* fix resume message_hub

* add unit test

* support resume from messagehub

* minor refine

* add comment

* fix typo

* update docstring

* first commit

* Support modify base config and add unit test

* remove import mmengine in config

* add unit test

* fix lint

* add unit test

* move RemoveAssignFromAST to config utils

* git add utils

* fix format issue in test file

* refine unit test

* refine unit test

* clean code

* fix as comment

* fix as comment

* add get_registry_by_scope method

* add unit test and docstring example

* rename get_registry_by_scope to switch_scope_and_registry

* move build function to registry/builder

* fix docstring

* rename builder->registry_builder, move build_from_cfg to registry_builder

rename builder->registry_builder, move build_from_cfg to registry_builder

* rename registry_builder to build_function

rename registry_builder to build_function

* fix docstring and type hint

* rename build_function to build_functions

* fix abs criterion error

* update

* fix missing device ids in wrap_model

* clean the code

* use default broadcast_buffers

* refine MMSeparateDistributedDataParallel

* rename tmp variable

* refine docstring

* add type hints

* refactor docstring of ddp model

* add  arg in docstring

* minor refine

* better ddp link

* fix cbds

* fix

* fix optimizer wrapper counts

* fix ut

* fix amp

* fix amp

* make logic consistent with torch.autocast

* support multiple device

* fix as comment

* fix as comment

* avoid circle import

* [Enhance] Support scheduling betas with MomentumScheduler.

* enhance ut

* test adam betas

* enhance ut

* enhance ut

* fix build multiple runner error

* fix comments

* fix cpu ci

* Fix ema hook and add unit test

* save state_dict of ema.module

save state_dict of ema.module

* replace warning.warn with MMLogger.warn

* fix as comment

* fix bug

* fix bug

* support dynamic interval in iterbasedtrainloop

* update typehint

* update typehint

* add dynamic interval in epochbasedtrainloop

* update

* fix

Co-authored-by: luochunhua.vendor <luochunhua@pjlab.org.cn>

* fix bug of not save-best in iteration-based training

* revise the unit test

* fix unit test of autocast

* fix compatiblity of unit test of optimizerwrapper

* clean code

* fix as comment

* fix docstring

* assert image shape before forward

* add unit test

* enhance error message

* allow gray image input

* fix as comment

* fix unit test

* fix unit test