From 5fbbca6d2bf9ed0c1f483741b917265a95f37040 Mon Sep 17 00:00:00 2001 From: Abhishek koserwal Date: Tue, 3 Jun 2025 19:47:07 +0530 Subject: [PATCH] Update 02_inf-simulator.md Signed-off-by: Abhishek koserwal --- docs/architecture/Components/02_inf-simulator.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/docs/architecture/Components/02_inf-simulator.md b/docs/architecture/Components/02_inf-simulator.md index 18c56f6..f87db66 100644 --- a/docs/architecture/Components/02_inf-simulator.md +++ b/docs/architecture/Components/02_inf-simulator.md @@ -90,16 +90,21 @@ For more details see the [vLLM documentation](https://docs.vllm.ai/en/stable/get ## Working with docker image +### Clone the llm-d-inference-sim repository +```bash +git clone https://github.com/llm-d/llm-d-inference-sim.git +``` + ### Building To build a Docker image of the vLLM Simulator, run: ```bash -make build-llm-d-inference-sim-image +make image-build ``` ### Running To run the vLLM Simulator image under Docker, run: ```bash -docker run --rm --publish 8000:8000 ai-aware-router/llm-d-inference-sim:0.0.1 /ai-aware-router/llm-d-inference-sim --port 8000 --model "Qwen/Qwen2.5-1.5B-Instruct" --lora "tweet-summary-0,tweet-summary-1" +docker run --rm --publish 8000:8000 ghcr.io/llm-d/llm-d-inference-sim:dev --port 8000 --model "Qwen/Qwen2.5-1.5B-Instruct" --lora "tweet-summary-0,tweet-summary-1" ``` **Note:** The above command exposes the simulator on port 8000, and serves the Qwen/Qwen2.5-1.5B-Instruct model. @@ -108,7 +113,7 @@ docker run --rm --publish 8000:8000 ai-aware-router/llm-d-inference-sim:0.0.1 /a ### Building To build the vLLM simulator, run: ```bash -make build-llm-d-inference-sim +make build ``` ### Running