Skip to content

[Intel][Gaudi] Remove HPU's dependency on vllm #6061

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 11 commits into
base: main
Choose a base branch
from
Draft

[Intel][Gaudi] Remove HPU's dependency on vllm #6061

wants to merge 11 commits into from

Conversation

yangw1234
Copy link

Motivation

This PR removes HPU's dependency on vllm.

This PR depends on #5252 and #5923

Modifications

Checklist

yangw1234 added 11 commits April 28, 2025 03:40
@yangw1234
support hpu
f66f429 
fix import error and improve contiguous pa

clean up

clean up

clean up

style

pre-commit

clean up

enable warmup

add moe layer

fix accuracy

disable radix cache automatically

remove sort

clean up

fix

fix model type

refine

style

style

add hpu test

add change

try directly install vllm-fork

clean up unnecessary changes

fix awq

address comments

fix acc

address comments

fix warmup

fix

cpu scheduler

fix style

fix device

remove block scales

refactor

refactor

fix

fix

remove change

fix attn mask

fix attn bias

add to device

fix style

address comments

address comments

Update python/sglang/srt/model_executor/forward_batch_info.py

Co-authored-by: JieXin Liang <[email protected]>

Update python/sglang/srt/model_executor/model_runner.py

Co-authored-by: JieXin Liang <[email protected]>

address comments

optimize allocator and rope

add profile

add heap based allocator
@yangw1234
fix style
50a8428 
pin numpy

try running scheduler in hpu

disable overlap schedule

fix style

remove schedule func
@yangw1234
add type hint
b689283
@yangw1234
remove rope
4668342
@yangw1234
remove unnecessary changes
e9de267
@yangw1234
change to hpu_attn_backend
6ff2e12
@yangw1234
fix reference
5c9efd2
@yangw1234
fix reference
cd2bcae
@yangw1234
fix style
f09d5e2
@yangw1234
initial support for prefix caching
940b52c 
update xgrammar

fix tests

fix warmup

add fix

optimize perf a bit

optimize some

remove
@yangw1234
remove vllm as a dependency
450fb96 
revert registry

fix rope
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@merrymercy merrymercy Awaiting requested review from merrymercy merrymercy will be requested when the pull request is marked ready for review merrymercy is a code owner

@Ying1123 Ying1123 Awaiting requested review from Ying1123 Ying1123 will be requested when the pull request is marked ready for review Ying1123 is a code owner

@hnyls2002 hnyls2002 Awaiting requested review from hnyls2002 hnyls2002 will be requested when the pull request is marked ready for review hnyls2002 is a code owner

@zhyncs zhyncs Awaiting requested review from zhyncs zhyncs will be requested when the pull request is marked ready for review zhyncs is a code owner

@ispobock ispobock Awaiting requested review from ispobock ispobock will be requested when the pull request is marked ready for review ispobock is a code owner

@ByronHsu ByronHsu Awaiting requested review from ByronHsu ByronHsu will be requested when the pull request is marked ready for review ByronHsu is a code owner

@xiezhq-hermann xiezhq-hermann Awaiting requested review from xiezhq-hermann xiezhq-hermann will be requested when the pull request is marked ready for review xiezhq-hermann is a code owner

@HaiShaw HaiShaw Awaiting requested review from HaiShaw HaiShaw will be requested when the pull request is marked ready for review HaiShaw is a code owner

@ch-wan ch-wan Awaiting requested review from ch-wan ch-wan will be requested when the pull request is marked ready for review ch-wan is a code owner

@zhaochenyang20 zhaochenyang20 Awaiting requested review from zhaochenyang20 zhaochenyang20 will be requested when the pull request is marked ready for review zhaochenyang20 is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@yangw1234