Skip to content

Commit 6ca0eed

Browse files
authored
[0.9.1][bugfix] V0.9.1 fix rope accruracy bug for deepseek model (#1887)
### What this PR does / why we need it? Fix the accuracy problem of deepseek model with eager mode introduced by an interface change of rope in #1719. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? End to end testing and CI passed. Signed-off-by: linfeng-yuan <[email protected]>
1 parent 9bcf007 commit 6ca0eed

File tree

1 file changed

+6
-2
lines changed

1 file changed

+6
-2
lines changed

vllm_ascend/ops/rotary_embedding.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -102,8 +102,12 @@ def native_rope_deepseek_forward(self,
102102
2).reshape(b, h_q, d)
103103
b, h_k, d = key.shape
104104
key = key.view(b, h_k, d // 2, 2).transpose(3, 2).reshape(b, h_k, d)
105-
q_pe, k_pe = rope_forward_oot(self, positions, query, key, offsets,
106-
neox_style)
105+
q_pe, k_pe = rope_forward_oot(self,
106+
positions,
107+
query,
108+
key,
109+
offsets=offsets,
110+
is_neox_style_override=neox_style)
107111
return q_pe, k_pe
108112

109113

0 commit comments

Comments
 (0)