diff --git a/docs/zh_cn/migration/migrate_param_scheduler_from_mmcv.md b/docs/zh_cn/migration/migrate_param_scheduler_from_mmcv.md new file mode 100644 index 0000000000000000000000000000000000000000..7e47e86496a5b7a5f1587f4367c5a9016efe6461 --- /dev/null +++ b/docs/zh_cn/migration/migrate_param_scheduler_from_mmcv.md @@ -0,0 +1,564 @@ +# è¿ç§» MMCV å‚数调度器到 MMEngine + +MMCV 1.x 版本使用 [LrUpdaterHook](https://mmcv.readthedocs.io/zh_CN/v1.6.0/api.html#mmcv.runner.LrUpdaterHook) å’Œ [MomentumUpdaterHook](https://mmcv.readthedocs.io/zh_CN/v1.6.0/api.html#mmcv.runner.MomentumUpdaterHook) æ¥è°ƒæ•´å¦ä¹ 率和动é‡ã€‚ +但éšç€æ·±åº¦å¦ä¹ 算法è®ç»ƒæ–¹å¼çš„ä¸æ–å‘展,使用 Hook 修改å¦ä¹ 率已ç»éš¾ä»¥æ»¡è¶³æ›´åŠ ä¸°å¯Œçš„è‡ªå®šä¹‰éœ€æ±‚ï¼Œå› æ¤ MMEngine æ供了å‚数调度器(ParamScheduler)。 +一方é¢ï¼Œå‚数调度器的接å£ä¸Ž PyTroch çš„å¦ä¹ 率调度器(LRScheduler)对é½ï¼Œå¦ä¸€æ–¹é¢ï¼Œå‚数调度器æ供了更丰富的功能,详细请å‚考[å‚数调度器使用指å—](../tutorials/param_scheduler.md)。 + +## å¦ä¹ 率调度器(LrUpdater)è¿ç§» + +MMEngine ä¸ä½¿ç”¨ LRScheduler 替代 LrUpdaterHook,é…置文件ä¸çš„å—段从原本的 `lr_config` 修改为 `param_scheduler`。 +MMCV ä¸çš„å¦ä¹ 率é…置与 MMEngine ä¸çš„å‚数调度器é…置对应关系如下: + +### å¦ä¹ 率预çƒï¼ˆWarmup)è¿ç§» + +由于 MMEngine ä¸çš„å¦ä¹ çŽ‡è°ƒåº¦å™¨åœ¨å®žçŽ°æ—¶å¢žåŠ äº† begin å’Œ end å‚数,指定了调度器的生效区间,所以å¯ä»¥é€šè¿‡è°ƒåº¦å™¨ç»„åˆçš„æ–¹å¼å®žçŽ°å¦ä¹ 率预çƒã€‚MMCV ä¸æœ‰ 3 ç§å¦ä¹ 率预çƒæ–¹å¼ï¼Œåˆ†åˆ«æ˜¯ 'constant', 'linear', 'exp',在 MMEngine ä¸å¯¹åº”çš„é…置应修改为: + +#### 常数预çƒ('constant') + +<table class="docutils"> + <thead> + <tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> + <tbody> + <tr> + <td> + +```python +lr_config = dict( + warmup='constant', + warmup_ratio=0.1, + warmup_iters=500, + warmup_by_epoch=False +) +``` + +</td> + <td> + +```python +param_scheduler = [ + dict(type='ConstantLR', + factor=0.1, + begin=0, + end=500, + by_epoch=False), + dict(...) # 主å¦ä¹ 率调度器é…ç½® +] +``` + +</td> + </tr> + </thead> + </table> + +#### 线性预çƒ('linear') + +<table class="docutils"> + <thead> + <tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> + <tbody> + <tr> + <td> + +```python +lr_config = dict( + warmup='linear', + warmup_ratio=0.1, + warmup_iters=500, + warmup_by_epoch=False +) +``` + +</td> + <td> + +```python +param_scheduler = [ + dict(type='LinearLR', + start_factor=0.1, + begin=0, + end=500, + by_epoch=False), + dict(...) # 主å¦ä¹ 率调度器é…ç½® +] +``` + +</td> + </tr> + </thead> + </table> + +#### 指数预çƒ('exp') + +<table class="docutils"> + <thead> + <tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> + <tbody> + <tr> + <td> + +```python +lr_config = dict( + warmup='exp', + warmup_ratio=0.1, + warmup_iters=500, + warmup_by_epoch=False +) +``` + +</td> + <td> + +```python +param_scheduler = [ + dict(type='ExponentialLR', + gamma=0.1, + begin=0, + end=500, + by_epoch=False), + dict(...) # 主å¦ä¹ 率调度器é…ç½® +] +``` + +</td> + </tr> + </thead> + </table> + +### 'fixed' å¦ä¹ 率(FixedLrUpdaterHook)è¿ç§» + +<table class="docutils"> +<thead> +<tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> +<tbody> +<tr> +<td> + +```python +lr_config = dict(policy='fixed') +``` + +</td> +<td> + +```python +param_scheduler = [ + dict(type='ConstantLR', factor=1) +] +``` + +</td> +</tr> +</thead> +</table> + +### 'step' å¦ä¹ 率(StepLrUpdaterHook)è¿ç§» + +<table class="docutils"> +<thead> +<tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> +<tbody> +<tr> +<td> + +```python +lr_config = dict( + policy='step', + step=[8, 11], + gamma=0.1, + by_epoch=True +) +``` + +</td> +<td> + +```python +param_scheduler = [ + dict(type='MultiStepLR', + milestone=[8, 11], + gamma=0.1, + by_epoch=True) +] +``` + +</td> +</tr> +</thead> +</table> + +### 'poly' å¦ä¹ 率(PolyLrUpdaterHook)è¿ç§» + +<table class="docutils"> +<thead> +<tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> +<tbody> +<tr> +<td> + +```python +lr_config = dict( + policy='poly', + power=0.7, + min_lr=0.001, + by_epoch=True +) +``` + +</td> +<td> + +```python +param_scheduler = [ + dict(type='PolyLR', + power=0.7, + eta_min=0.001, + begin=0, + end=num_epochs, + by_epoch=True) +] +``` + +</td> +</tr> +</thead> +</table> + +### 'exp' å¦ä¹ 率(ExpLrUpdaterHook)è¿ç§» + +<table class="docutils"> +<thead> +<tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> +<tbody> +<tr> +<td> + +```python +lr_config = dict( + policy='exp', + power=0.5, + by_epoch=True +) +``` + +</td> +<td> + +```python +param_scheduler = [ + dict(type='ExponentialLR', + gamma=0.5, + begin=0, + end=num_epochs, + by_epoch=True) +] +``` + +</td> +</tr> +</thead> +</table> + +### 'CosineAnnealing' å¦ä¹ 率(CosineAnnealingLrUpdaterHook)è¿ç§» + +<table class="docutils"> +<thead> +<tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> +<tbody> +<tr> +<td> + +```python +lr_config = dict( + policy='CosineAnnealing', + min_lr=0.5, + by_epoch=True +) +``` + +</td> +<td> + +```python +param_scheduler = [ + dict(type='CosineAnnealingLR', + eta_min=0.5, + T_max=num_epochs, + begin=0, + end=num_epochs, + by_epoch=True) +] +``` + +</td> +</tr> +</thead> +</table> + +### 'FlatCosineAnnealing' å¦ä¹ 率(FlatCosineAnnealingLrUpdaterHook)è¿ç§» + +åƒ FlatCosineAnnealing è¿™ç§ç”±å¤šä¸ªå¦ä¹ 率ç–略拼接而æˆçš„å¦ä¹ 率,原本需è¦é‡å†™ Hook æ¥å®žçŽ°ï¼Œè€Œåœ¨ MMEngine ä¸åªéœ€å°†ä¸¤ä¸ªå‚数调度器组åˆå³å¯ + +<table class="docutils"> +<thead> +<tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> +<tbody> +<tr> +<td> + +```python +lr_config = dict( + policy='FlatCosineAnnealing', + start_percent=0.5, + min_lr=0.005, + by_epoch=True +) +``` + +</td> +<td> + +```python +param_scheduler = [ + dict(type='ConstantLR', factor=1, begin=0, end=num_epochs * 0.75) + dict(type='CosineAnnealingLR', + eta_min=0.005, + begin=num_epochs * 0.75, + end=num_epochs, + T_max=num_epochs * 0.25, + by_epoch=True) +] +``` + +</td> +</tr> +</thead> +</table> + +### 'CosineRestart' å¦ä¹ 率(CosineRestartLrUpdaterHook)è¿ç§» + +<table class="docutils"> +<thead> +<tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> +<tbody> +<tr> +<td> + +```python +lr_config = dict(policy='CosineRestart', + periods=[5, 10, 15], + restart_weights=[1, 0.7, 0.3], + min_lr=0.001, + by_epoch=True) +``` + +</td> +<td> + +```python +param_scheduler = [ + dict(type='CosineRestartLR', + periods=[5, 10, 15], + restart_weights=[1, 0.7, 0.3], + eta_min=0.001, + by_epoch=True) +] +``` + +</td> +</tr> +</thead> +</table> + +### 'OneCycle' å¦ä¹ 率(OneCycleLrUpdaterHook)è¿ç§» + +<table class="docutils"> +<thead> +<tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> +<tbody> +<tr> +<td> + +```python +lr_config = dict(policy='OneCycle', + max_lr=0.02, + total_steps=90000, + pct_start=0.3, + anneal_strategy='cos', + div_factor=25, + final_div_factor=1e4, + three_phase=True, + by_epoch=False) +``` + +</td> +<td> + +```python +param_scheduler = [ + dict(type='OneCycleLR', + eta_max=0.02, + total_steps=90000, + pct_start=0.3, + anneal_strategy='cos', + div_factor=25, + final_div_factor=1e4, + three_phase=True, + by_epoch=False) +] +``` + +</td> +</tr> +</thead> +</table> + +需è¦æ³¨æ„的是 `by_epoch` å‚æ•° MMCV 默认是 `False`, MMEngine 默认是 `True` + +### 'LinearAnnealing' å¦ä¹ 率(LinearAnnealingLrUpdaterHook)è¿ç§» + +<table class="docutils"> +<thead> +<tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> +<tbody> +<tr> +<td> + +```python +lr_config = dict( + policy='LinearAnnealing', + min_lr_ratio=0.01, + by_epoch=True +) +``` + +</td> +<td> + +```python +param_scheduler = [ + dict(type='LinearLR', + start_factor=1, + end_factor=0.01, + begin=0, + end=num_epochs, + by_epoch=True) +] +``` + +</td> +</tr> +</thead> +</table> + +## 动é‡è°ƒåº¦å™¨ï¼ˆMomentumUpdater)è¿ç§» + +MMCV 使用 `momentum_config` å—段和 MomentumUpdateHook 调整动é‡ã€‚ MMEngine ä¸åŠ¨é‡åŒæ ·ç”±å‚数调度器控制。用户å¯ä»¥ç®€å•å°†å¦ä¹ 率调度器åŽçš„ `LR` 修改为 `Momentum`,å³å¯ä½¿ç”¨åŒæ ·çš„ç–ç•¥æ¥è°ƒæ•´åŠ¨é‡ã€‚动é‡è°ƒåº¦å™¨åªéœ€è¦å’Œå¦ä¹ çŽ‡è°ƒåº¦å™¨ä¸€æ ·æ·»åŠ è¿› `param_scheduler` 列表ä¸å³å¯ã€‚举一个简å•çš„例å: + +<table class="docutils"> +<thead> +<tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> +<tbody> +<tr> +<td> + +```python +lr_config = dict(...) +momentum_config = dict( + policy='CosineAnnealing', + min_momentum=0.1, + by_epoch=True +) +``` + +</td> +<td> + +```python +param_scheduler = [ + # å¦ä¹ 率调度器é…ç½® + dict(...), + # 动é‡è°ƒåº¦å™¨é…ç½® + dict(type='CosineAnnealingMomentum', + eta_min=0.1, + T_max=num_epochs, + begin=0, + end=num_epochs, + by_epoch=True) +] +``` + +</td> +</tr> +</thead> +</table> + +## å‚数更新频率相关é…ç½®è¿ç§» + +如果在使用 epoch-based è®ç»ƒå¾ªçŽ¯ä¸”é…置文件ä¸æŒ‰ epoch 设置生效区间(`begin`,`end`)或周期(`T_max`)ç‰å˜é‡çš„åŒæ—¶å¸Œæœ›å‚数率按 iteration 更新,在 MMCV ä¸éœ€è¦å°† `by_epoch` 设置为 False。而在 MMEngine ä¸éœ€è¦æ³¨æ„,é…ç½®ä¸çš„ `by_epoch` ä»éœ€è®¾ç½®ä¸º True,通过在é…ç½®ä¸æ·»åŠ `convert_to_iter_based=True` æ¥æž„建按 iteration æ›´æ–°çš„å‚数调度器, +关于æ¤é…置详è§[å‚数调度器教程](../tutorials/param_scheduler.md)。 +以è¿ç§»CosineAnnealing为例: + +<table class="docutils"> +<thead> +<tr> + <th>MMCV-1.x</th> + <th>MMEngine</th> +<tbody> +<tr> +<td> + +```python +lr_config = dict( + policy='CosineAnnealing', + min_lr=0.5, + by_epoch=False +) +``` + +</td> +<td> + +```python +param_scheduler = [ + dict( + type='CosineAnnealingLR', + eta_min=0.5, + T_max=num_epochs, + by_epoch=True, # 注æ„,by_epoch 需è¦è®¾ç½®ä¸º True + convert_to_iter_based=True # 转æ¢ä¸ºæŒ‰ iter æ›´æ–°å‚æ•° + ) +] +``` + +</td> +</tr> +</thead> +</table> + +ä½ å¯èƒ½è¿˜æƒ³é˜…读[å‚数调度器的教程](../tutorials/param_scheduler.md)或者[å‚数调度器的 API 文档](mmengine.optim.scheduler)。