align_grad_clip

[zty-king]

PR Category

Auto Parallel

PR Types

Others

Description

• 当前pp不属于本rank的model_chunk不会进行计算（前向和反向），相应参数的梯度为None，在使用ClipGradByGlobalNorm计算全局梯度裁剪系数，每个pp_stage只计算属于本stage上的model_chunk对应的参数梯度的_squared_l2_norm，并求和（即对所有梯度的每个元素进行平方，再求和），而此时计算的这个global_norm_var，其实是一个局部和，用此进行梯度裁剪，逻辑上有问题，因此需要进行对其修复。
• 修复方法默认与动手对齐，即每个pp_stage算出局部和之后，将其all gather到每个stage上，再将局部和相加，即得到一个真正的global_norm_var。这样做的好处是：更适合pp，通信量更小，效率更高

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 33.33333% with 14 lines in your changes missing coverage. Please review.

Please upload report for BASE (develop@159fd7c). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
python/paddle/nn/clip.py	33.33%	14 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             develop   #74080   +/-   ##
==========================================
  Coverage           ?   33.33%           
==========================================
  Files              ?        1           
  Lines              ?       21           
  Branches           ?        0           
==========================================
  Hits               ?        7           
  Misses             ?       14           
  Partials           ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[zty-king]

/re-run all-failed

[zty-king]

补充本地coverage测试如下：

[xuxinyi389]

LGTM