* fix bug of not save-best in iteration-based training

* revise the unit test

* fix unit test of autocast

* fix compatiblity of unit test of optimizerwrapper

* clean code

* fix as comment

* fix docstring

* assert image shape before forward

* add unit test

* enhance error message

* allow gray image input

* fix as comment

* fix unit test

* fix unit test

* impl lr and momentum visualizer

* provide fakerun

* refine data_root and data_prefix params

* modify unittest

* [Feature]: Set different seed for diff rank

* [Feature]: Add log

* [Fix]: Fix lint

* [Fix]: Fix docstring

* [Fix]: Fix sampler seed

* [Fix]: Fix log bug

* [Fix]: Change diff_seed to diff_rank_seed

* [Fix]: Fix lint

* [Feat] Support mps

* fix docstring

* Fix docstring format of BaseDataElement

* fix docstring

* fix RuntimeError of SyncBuffersHook

* add UT

* [Feat] Support revert syncbn

* use logger.info but not warning

* fix info string

* add autocast wrapper

* fix docstring

* fix docstring

* fix compare version

* fix unit test

* fix incompatible arguments

* fix as comment

* fix unit test

* rename auto_cast to autocast

* [Feat] Support save best ckpt

* reformat code

* rename function and reformat code

* fix logging info

* LogProcessor support custom significant digit

* rename to num_digits

* modify cuda() to to()

* rollback load_checkpoint

* refine runner

* add TODO

* [Refactor]:Refactor `after_val_epoch` to make it output metric by epoch

* add an option for user to choose the way of outputing metric

* rename variable

* reformat docstring

* add type alias

* reformat code

* add test function

* add comment and test code

* add comment and test code

* [Feat] Support FSDP Training

* fix version comparison

* change param format and move `FSDP_WRAP_POLICY` to wrapper file

* add docstring and type hint,reformat code

* fix type hint

* fix typo, reformat code

* move import resource

* move import resource

* [Enhance] dump messagehub in runner.resume

* delete unnecessary code

* delete debugging code

Co-authored-by: imabackstabber <312276423@qq.com>

* support resume from ceph

* move func and refine

* delete symlink

* fix unittest

* perserve _allow_symlink and symlink

* support mlu

* add ut and refine docstring

* Fix setLevel of MMLogger

Fix setLevel of MMLogger

* add docstring and comment

* [Feature] Dump predictions to pickle file for offline evaluation.

* print_log

* merge context

* update unit test

* add docstring

* fix bug in AmpOptimWrapper

* add docstring for backward

* add warning and docstring for accumuate gradient

* fix docstring

* fix docstring

* add params_group method

* fix as comment

* fix as comment

* make default_value of loss_scale to dynamic

* Fix docstring

* decouple should update and should no sync

* rename attribute in OptimWrapper

* fix docstring

* fix comment

* fix comment

* fix as comment

* fix as comment and add unit test

* [Features]Add OnecycleLR

* [Features]Add OnecycleLR

* yapf disable

* build_iter_from_epoch

* add epoch

* fix args

* fix according to comments;

* lr-param

* fix according to comments

* defaults -> default to

* remove epoch and steps per step

* variabel names

* fix build train_loop during test

* fix build train_loop during test

* fix build train_loop during test

* fix build train_loop during test

* Fix as comment

* [Fix] Fix image dtype when enable_normalize=False.

* update ut

* move to collate

* update ut