Skip to content

Add Prediction field #970

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

danielpeng2
Copy link

Describe the change

Adding the prediction field which allows for using Predicted Outputs.

Provide OpenAI documentation link
https://platform.openai.com/docs/api-reference/chat/create#chat-create-prediction

Describe your solution
Adds the optional parameter and modifies the response to include how many prediction tokens it used

Tests
Briefly describe how you have tested these changes. If possible — please add integration tests.
Tested locally and verified that I received predicted tokens accepted in the response
image

Issue: #954

@danielpeng2 danielpeng2 changed the title Daniel/add prediction field Add Prediction Field Apr 23, 2025
@danielpeng2 danielpeng2 changed the title Add Prediction Field Add Prediction field Apr 23, 2025
@sashabaranov sashabaranov requested a review from Copilot April 23, 2025 22:03
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for predicted outputs by introducing a new prediction field in the chat completion request and updating usage metrics to track predicted token counts.

  • Adds AcceptedPredictionTokens and RejectedPredictionTokens to the Usage metrics in common.go.
  • Introduces a Prediction configuration and corresponding struct in chat.go for predicted outputs.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
common.go Adds new fields to track accepted and rejected predicted tokens.
chat.go Introduces Prediction field in ChatCompletionRequest and defines Prediction struct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant