Fix paddle.vision.ops.deform_conv2d API big Tensor

[xingmingyyj]

PR Category

Operator Mechanism

PR Types

Bug fixes

Description

• 修复int溢出导致的访存越界问题
• 修复 CUDA error(9)问题
• batch较大时存在精度diff，暂未查明原因

paddle.vision.ops.deform_conv2d(Tensor([5070448, 3, 5, 5],"float32"), Tensor([5070448, 18, 5, 5],"float32"), Tensor([6, 3, 3, 3],"float32"), None, list[1,1,], list[1,1,], list[1,1,], 1, 1, Tensor([5070448, 9, 5, 5],"float32"), )
Not equal to tolerance rtol=0.01, atol=0.01
Tensor-likes are not close!

Mismatched elements: 3 / 2281701600 (0.0%)
Greatest absolute difference: 0.04030437394976616 at index (1852051, 0, 0, 2) (up to 0.01 allowed)
Greatest relative difference: 1.0 at index (910808, 0, 0, 3) (up to 0.01 allowed)
ACTUAL: (shape=torch.Size([5070448, 18, 5, 5]), dtype=torch.float32)
tensor([[[[ 0.0000e+00,  0.0000e+00, -3.4506e-02, -8.3038e-04,  0.0000e+00],
          [-8.7802e-04,  3.7523e-03,  1.5622e-02, -8.7787e-04,  5.6141e-03],

Pcard-73263

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[xingmingyyj]

/re-run all-failed