Skip to content

Fix enum validation error in Google GenAI provider #1762

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

DaveOkpare
Copy link
Contributor

@DaveOkpare DaveOkpare commented Aug 2, 2025

Changes


Important

Fixes enum validation error in Google GenAI provider by changing model_validate() to model_validate_json() in parse_genai_tools().

This description was created by Ellipsis for d28e76d. You can customize this summary. It will automatically update as commits are pushed.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Looks good to me! 👍

Reviewed everything up to d28e76d in 48 seconds. Click for details.
  • Reviewed 16 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 draft comments. View those below.
  • Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. instructor/processing/function_calls.py:286
  • Draft comment:
    Using json.dumps to wrap function_call.args and then calling model_validate_json is a clear fix for string-to-enum conversion (#1756). Please ensure that function_call.args is always JSON serializable to prevent runtime issues. Additionally, consider adding a test case that verifies enum conversion behavior.
  • Reason this comment was not posted:
    Comment looked like it was already resolved.

Workflow ID: wflow_WILmjoF4w4JcHmG7

You can customize Ellipsis by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.

@jxnl
Copy link
Collaborator

jxnl commented Aug 7, 2025

this does not feel right / feels like would slow things randomly @claude can you figure out if we can add some tests or fix this another way

Copy link
Contributor

claude bot commented Aug 7, 2025

Claude encountered an error —— View job

Failed with exit code 128

I'll analyze this and get back to you.

@DaveOkpare
Copy link
Contributor Author

DaveOkpare commented Aug 8, 2025

@jxnl Gemini’s structured output docs state:

“Passing JSON Schema directly is not yet supported when using the SDK.”

The SDK turns enum fields into strings with "enum": [...] constraints, so while using strict=True with Pydantic we have these options:

  • model_validate() rejects these strings and add a fallback to model_validate_json() to parse valid enum strings into Enum members while preserving strictness.
  • use model_validate_json() directly.

Performance test

Case n=5 n=500 Winner
Needs coercion (strings) 11 µs vs 17 µs 1,090 µs vs 1,674 µs model_validate_json
Already valid (Enums) 5 µs vs 13 µs 603 µs vs 1,294 µs model_validate

Test compared using model_validate() with a fallback to model_validate_json() against using model_validate_json() for all the Gemini cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants