Skip to content

Commit 77d26be

Browse files
author
赵江江
committed
feat: add mtp ut case
Signed-off-by: 赵江江 <[email protected]>
1 parent 9259b2c commit 77d26be

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

tests/e2e/singlecard/spec_decode_v1/test_v1_mtp_correctness.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ def test_mtp_correctness(
6060
should be the same when using mtp speculative decoding.
6161
'''
6262
ref_llm = LLM(model=model_name,
63-
gpu_memory_utilization=0.7,
63+
gpu_memory_utilization=0.5,
6464
max_model_len=256,
6565
enforce_eager=True)
6666
ref_outputs = ref_llm.chat(test_prompts, sampling_config)
@@ -72,7 +72,7 @@ def test_mtp_correctness(
7272
"method": "deepseek_mtp",
7373
"num_speculative_tokens": 1,
7474
},
75-
gpu_memory_utilization=0.7,
75+
gpu_memory_utilization=0.5,
7676
max_model_len=256,
7777
enforce_eager=True)
7878
spec_outputs = spec_llm.chat(test_prompts, sampling_config)

0 commit comments

Comments
 (0)