[Big tensor] fix nan in cross_entropy

[hxzd5568]

PR Category

Operator Mechanism

PR Types

Bug fixes

Description

Pcard-67164
修复cross_entropy(reduction="sum") float16下出现nan。

原因

由于参与sum的数字过大，结果出现inf， inf参与运算后结果为nan

修复方法

由于问题出现在sum的结果溢出，似乎无法在kernel层面修改（输出类型一般和输入一致，即便中间提升精度，最终也会转化回原始类型而导致溢出）。而在上层分配的时候修改其数据类型更加方便。

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[hxzd5568]

/re-run coverage test

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.

Please upload report for BASE (develop@1107fe4). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
python/paddle/nn/functional/loss.py	0.00%	4 Missing ⚠️

❌ Your patch status has failed because the patch coverage (0.00%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             develop   #74070   +/-   ##
==========================================
  Coverage           ?    0.00%           
==========================================
  Files              ?        1           
  Lines              ?        4           
  Branches           ?        0           
==========================================
  Hits               ?        0           
  Misses             ?        4           
  Partials           ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[wanghuancoder]

这种情况出nan我觉得可以接受。就是fp16撑不住了。不需要cast成32.否则在一些场景下会出现性能问题。可以再看书豪评估一下。

[lshpku]

不能在sum这里改吗，改成这样sum(out, [], paddle.float32, False)，这样会把cast和sum融合成一个kernel
你对比一下下面两个代码的 nsys trace：

x = paddle.randn([32, 32], dtype='bfloat16')
y = paddle.sum(x, axis=1, dtype='float32')
print(y.numpy().dtype)

x = paddle.randn([32, 32], dtype='bfloat16')
y = paddle.sum(x.cast('float32'), axis=1)
print(y.numpy().dtype)

[hxzd5568]

done

[hxzd5568]

/re-run all-failed