-
Notifications
You must be signed in to change notification settings - Fork 53
add pluggable loadFormat support for model loaders #318
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
- Add missing chart dependencies (common, redis) - Fix YAML formatting in Chart.yaml (document start, line length) - Bump chart version to 1.0.19 (required by chart-testing) - Configure typos checker to handle base64 SVG content - All GitHub Actions lint checks now pass
- Add debug output to test connection pod with service lookups and timing - Add cluster state debugging before and after chart-testing - Add verbose curl output and error handling - Add pod logs collection for failed pods - Add timeout to service wait loop to prevent infinite hangs
The chart-testing install was failing because the Gateway resource requires Istio validation webhook which is not available in the CI environment. Disabling gateway creation in CI values resolves the connection refused error.
- Enhanced GitHub Actions workflow with verbose debugging for ct install failures - Disabled test pods in CI values to isolate chart installation issues - Added helm releases and cluster state checking on ct install failure
- Moved epp, prefill, and decode properties from gateway to modelservice section - Merged duplicate modelservice sections in runai-streamer-values.yaml - Both values files now pass helm template schema validation Resolves: "Additional property epp/decode/prefill is not allowed" errors
@jeremyeder dug around on why this was failing in a deployment. Since the controller only exposes the variables that are present in its TemplateVars struct from modelservice when it renders the preset ConfigMap. Since This PR gets things working for me llm-d/llm-d-model-service#221 I didn't test it with the load-format validation yet, just tested that modelservice doesn't blow up on the unknown var which * I think * is all we need to get unblocked. Can test it with swapping this out in values.yaml which is # -- Model Service controller image registry
registry: ghcr.io
# -- Model Service controller image repository
repository: nerdalert/llm-d-model-service
# -- Model Service controller image tag
tag: "0.0.1" Will look into CI as well. Not sure how this passes with modelservice crashing in a deployment. cc/ @sriumcp PTAL and sanity check me here if you get a sec 🙏 |
Model service is disabled in |
fixes #317
Also adds runai_streamer example and tests.