Add Hugging Face as a provider #1911

hanouticelina · 2025-06-04T16:29:57Z

closes #1085.
Hi there, maintainer of huggingface_hub library 🤗 here,
This PR introduces support for Hugging Face's Inference Providers (documentation here) as a Model Provider.

Our API is fully compatible with the OpenAI REST API spec, and the implementation closely mirrors the existing OpenAIProvider / OpenAIModel pair. Under the hood, we use the huggingface_hub.AsyncInferenceClient client, which is a drop-in replacement of the async OpenAI client but includes provider-specific (de)serialization logic that cannot be reproduced reliably with the OpenAI client alone, see @Wauplin’s detailed explanation here).

Note that huggingface_hub is a stable and widely used library that was already listed as a dependency in the lockfile.

TODO:

Add tests.
Add documentation.

Kludex · 2025-06-05T12:26:41Z

Great! :)

How can I help here?

hanouticelina · 2025-06-05T12:29:58Z

Hi @Kludex,
i'm currently finishing adding tests and documentation and then the PR will be ready for review! :)

Kludex · 2025-06-05T12:36:36Z

Hi @Kludex, i'm currently finishing adding tests and documentation and then the PR will be ready for review! :)

Amazing! :)

hyperlint-ai · 2025-06-09T12:50:29Z

PR Change Summary

Added support for Hugging Face as a model provider in the API, enabling users to utilize Hugging Face's Inference Providers.

Introduced Hugging Face as a new model provider in the API.
Added installation and configuration instructions for Hugging Face support.
Implemented usage examples for Hugging Face models and providers.

Added Files

docs/models/huggingface.md

How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

…nference-providers

hanouticelina · 2025-06-09T13:58:14Z

Hi @Kludex,
Sorry for the delay, the PR is finally ready for review! I'll have limited availability over the next two weeks, but I'll do my best to be responsive in case you have any feedback.
At 🤗, we're super excited to get this merged!

hanouticelina · 2025-06-24T12:37:22Z

Hi @Kludex,
Just wanted to follow up now that I'm back and have a bit more time, let me know if you get a chance to take a look at the PR! 😊

Copilot

Pull Request Overview

Adds first-class support for Hugging Face Inference Providers by plugging in a new HuggingFaceProvider/HuggingFaceModel pair, updating tests, CLI, docs, and optional dependencies.

Introduce HuggingFaceProvider and HuggingFaceModel in pydantic_ai_slim
Extend tests and CLI to recognize huggingface as a provider
Add documentation and update pyproject.toml for the huggingface optional group

Reviewed Changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
pydantic_ai_slim/pydantic_ai/providers/huggingface.py	New provider implementation; wiring environment API key and client setup
pydantic_ai_slim/pydantic_ai/models/huggingface.py	New model implementation handling sync/async and streaming
docs/models/huggingface.md	Documentation for installing and configuring HF provider
pyproject.toml	Added `huggingface` to optional dependencies
tests/providers/test_huggingface.py	New unit tests for provider initialization/errors
tests/models/test_huggingface.py	Extensive tests covering completions, streaming, error handling

Comments suppressed due to low confidence (2)

pydantic_ai_slim/pydantic_ai/providers/huggingface.py:41

The docstring states the default provider is "auto", but the provider parameter defaults to None. Please update the doc to match the code or adjust the default value accordingly.

        provider: str | None = None,

docs/models/huggingface.md:9

[nitpick] The install command looks like a typo. Consider replacing pip/uv-add with pip install to avoid confusion.

pip/uv-add "pydantic-ai-slim[huggingface]"

Copilot · 2025-06-25T10:42:51Z

pydantic_ai_slim/pydantic_ai/providers/huggingface.py

+
+    @property
+    def base_url(self) -> str:
+        return self.client.model  # type: ignore


The base_url property is returning self.client.model instead of the actual base URL. It should return something like self._client.base_url (or store base_url on init) so that provider.base_url reflects the configured endpoint.

Suggested change

return self.client.model # type: ignore

return self._base_url

Kludex · 2025-06-25T10:54:59Z

pydantic_ai_slim/pydantic_ai/models/__init__.py

+        'huggingface:Qwen/QwQ-32B',
+        'huggingface:Qwen/Qwen2.5-72B-Instruct',
+        'huggingface:Qwen/Qwen3-235B-A22B',
+        'huggingface:Qwen/Qwen3-32B',
+        'huggingface:deepseek-ai/DeepSeek-R1',
+        'huggingface:meta-llama/Llama-3.3-70B-Instruct',
+        'huggingface:meta-llama/Llama-4-Maverick-17B-128E-Instruct',
+        'huggingface:meta-llama/Llama-4-Scout-17B-16E-Instruct',


How can we keep the list of those models up-to-date? Do you folks have an endpoint that we can call to list a lot of them, or something?

Kludex

I wrote some comments here.

We need to create ThinkingParts (does the HuggingFace client handles them?) and we need to add code coverage.

I prefer tests strictly with VCR, if possible.

Kludex · 2025-06-25T10:56:25Z

pydantic_ai_slim/pydantic_ai/providers/huggingface.py

+        api_key: str | None = None,
+        hf_client: AsyncInferenceClient | None = None,
+        http_client: AsyncClient | None = None,
+        provider: str | None = None,


It's a bit weird that inside the class that is called provider, you can set a provider as well. Is there an alternative name here?

Kludex · 2025-06-25T10:56:51Z

pydantic_ai_slim/pydantic_ai/providers/huggingface.py

+            )
+
+        if http_client is not None:
+            raise ValueError('`http_client` is ignored for HuggingFace provider, please use `hf_client` instead')


Suggested change

raise ValueError('`http_client` is ignored for HuggingFace provider, please use `hf_client` instead')

raise ValueError('`http_client` is ignored for HuggingFace provider, please use `hf_client` instead.')

Kludex · 2025-06-25T10:57:00Z

pydantic_ai_slim/pydantic_ai/providers/huggingface.py

+            raise ValueError('`http_client` is ignored for HuggingFace provider, please use `hf_client` instead')
+
+        if base_url is not None and provider is not None:
+            raise ValueError('Cannot provide both `base_url` and `provider`')


Suggested change

raise ValueError('Cannot provide both `base_url` and `provider`')

raise ValueError('Cannot provide both `base_url` and `provider`.')

Kludex · 2025-06-25T10:57:10Z

pydantic_ai_slim/pydantic_ai/providers/huggingface.py

+            raise ValueError('Cannot provide both `base_url` and `provider`')
+
+        if hf_client is None:
+            self._client = AsyncInferenceClient(api_key=api_key, provider=provider, base_url=base_url)  # type: ignore


What's the type issue here?

Kludex · 2025-06-25T10:57:46Z

pydantic_ai_slim/pydantic_ai/providers/huggingface.py

+    def __init__(
+        self,
+        base_url: str | None = None,
+        api_key: str | None = None,
+        hf_client: AsyncInferenceClient | None = None,
+        http_client: AsyncClient | None = None,
+        provider: str | None = None,


For a better user experience, it would be nice to create some overloads to reflect the ValueErrors we have below. But not a blocker.

Kludex · 2025-06-25T10:58:26Z

tests/conftest.py

@@ -291,6 +291,11 @@ def openrouter_api_key() -> str:
    return os.getenv('OPENROUTER_API_KEY', 'mock-api-key')


+@pytest.fixture(scope='session')
+def huggingface_api_key() -> str:
+    return os.getenv('HF_TOKEN', 'hf_token') or os.getenv('HUGGINGFACE_API_KEY', 'hf_token')


What's this HUGGINGFACE_API_KEY? It's not mentioned or used in the code.

Kludex · 2025-06-26T08:12:35Z

@hanouticelina If I can help to speed things up here, please let me know.

hanouticelina · 2025-06-26T08:15:05Z

@Kludex thank you! I'll be working on this PR in the next few days and get back to you if i need any help :)

hanouticelina added 4 commits June 4, 2025 17:03

add hf inference providers support

2c3b9cb

update dependencies

537a657

nit

af602a5

update docstring

1f3f7a2

Kludex self-assigned this Jun 5, 2025

hanouticelina added 2 commits June 9, 2025 13:16

add tests

bea050c

add docs and known models for hf

40aef2e

Merge branch 'main' of github.com:hanouticelina/pydantic-ai into hf-i…

af6fa42

…nference-providers

hanouticelina marked this pull request as ready for review June 9, 2025 13:05

hanouticelina added 7 commits June 9, 2025 14:10

fix imports in test

7a4b9a4

fix tests

a153081

fix provider test

2f0ec51

adapt cli test

69aee55

re-record vcr cassettes

f68dace

fix token name

cc982e5

fix examples test

00da46e

Kludex requested a review from Copilot June 25, 2025 10:41

Copilot AI reviewed Jun 25, 2025

View reviewed changes

Kludex added 2 commits June 25, 2025 12:45

Merge remote-tracking branch 'origin/main' into hf-inference-providers

bb20a34

Add API docs and refactor a bit the wording

922fd13

Kludex reviewed Jun 25, 2025

View reviewed changes

Kludex requested changes Jun 25, 2025

View reviewed changes

Kludex assigned hanouticelina and unassigned Kludex Jun 25, 2025

	return self.client.model # type: ignore
	return self._base_url

	raise ValueError('`http_client` is ignored for HuggingFace provider, please use `hf_client` instead')
	raise ValueError('`http_client` is ignored for HuggingFace provider, please use `hf_client` instead.')

	raise ValueError('Cannot provide both `base_url` and `provider`')
	raise ValueError('Cannot provide both `base_url` and `provider`.')

Add Hugging Face as a provider #1911

Are you sure you want to change the base?

Add Hugging Face as a provider #1911

Uh oh!

Conversation

hanouticelina commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kludex commented Jun 5, 2025

Uh oh!

hanouticelina commented Jun 5, 2025

Uh oh!

Kludex commented Jun 5, 2025

Uh oh!

hyperlint-ai bot commented Jun 9, 2025

PR Change Summary

Uh oh!

hanouticelina commented Jun 9, 2025

Uh oh!

hanouticelina commented Jun 24, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

Kludex Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

Kludex left a comment

Choose a reason for hiding this comment

Uh oh!

Kludex Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

Kludex Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

Kludex Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

Kludex Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

Kludex Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

Kludex Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

Kludex commented Jun 26, 2025

Uh oh!

hanouticelina commented Jun 26, 2025

Uh oh!

Uh oh!

hanouticelina commented Jun 4, 2025 •

edited

Loading