Skip to content

Conversation

jpvajda
Copy link
Contributor

@jpvajda jpvajda commented Jul 23, 2025

🎉 PR Summary: Agent Function Call Context / History Feature

TL;DR

Successfully implemented the new Agent Function Call Context / History feature for the Deepgram JS SDK. This feature allows developers to provide conversation context and function call history to agents, enabling more contextual and intelligent conversations. The implementation includes schema updates, new client methods, comprehensive testing, and maintains full backward compatibility.

What Changed

Core Schema Updates

  • flags.history: Added optional boolean flag to AgentLiveSchema to enable/disable history message reporting (default: true)
  • agent.context.messages: Added optional array property to accept conversation history during agent configuration
  • New Type Definitions: Created HistoryConversationText and HistoryFunctionCall interfaces matching the API specification

Enhanced Client Functionality

  • sendHistoryConversationText(message): Send individual conversation history messages
  • sendHistoryFunctionCall(message): Send function call history messages
  • sendHistoryMessages(messages): Send multiple history messages at once (bulk operation)

API Specification Compliance

Full adherence to the provided API specification:

  • ✅ History conversation text with type, role, content properties
  • ✅ History function calls with type, function_calls array containing id, name, client_side, arguments, response
  • ✅ Agent configuration flags for history control
  • ✅ Context messages array integration

Testing

Unit Tests (New)

  • tests/unit/agent-history.test.ts: Comprehensive test suite with 11 test cases covering:
    • Schema configuration with history flags (enabled/disabled)
    • Schema configuration with context messages
    • New client method functionality
    • Type safety and validation
    • Backward compatibility verification
    • Edge cases and error handling

Integration Testing

  • Schema Testing Demo: Created and successfully tested a comprehensive demo application that validated:
    • Configuration acceptance with new properties
    • Client method functionality
    • Message sending with proper formatting
    • Backward compatibility with existing configurations

Regression Testing

  • All existing tests pass: 243/243 tests passing (including 11 new history tests)
  • No breaking changes: Existing functionality unaffected
  • Build verification: Clean TypeScript compilation with no errors

API Specification Compliance

Feature Status Implementation
flags.history property Added to AgentLiveSchema with boolean type and default documentation
agent.context.messages Array of HistoryConversationText | HistoryFunctionCall
History conversation text type: "History", role: "user" | "assistant", content: string
History function calls type: "History", function_calls array with all required properties
Function call objects id, name, client_side, arguments, response all implemented
Client methods Three new methods for sending history messages
Backward compatibility All existing configurations continue to work unchanged

Usage Example

import { createClient } from "@deepgram/sdk";

const client = createClient(API_KEY);
const connection = client.agent();

// Configure agent with history enabled and context
connection.configure({
  flags: {
    history: true  // Enable history reporting
  },
  audio: {
    input: { encoding: "linear16", sample_rate: 16000 }
  },
  agent: {
    language: "en",
    context: {
      messages: [
        // Conversation history
        {
          type: "History",
          role: "user",
          content: "What's my order status?"
        },
        // Function call history
        {
          type: "History", 
          function_calls: [{
            id: "fc_order_001",
            name: "lookup_order",
            client_side: true,
            arguments: '{"order_id": "12345"}',
            response: '{"status": "shipped", "tracking": "ABC123"}'
          }]
        },
        {
          type: "History",
          role: "assistant", 
          content: "Your order #12345 has shipped with tracking ABC123."
        }
      ]
    },
    speak: {
      provider: { type: "deepgram", model: "aura-2-zeus-en" }
    }
  }
});

// Send additional history during conversation
connection.sendHistoryConversationText({
  type: "History",
  role: "user", 
  content: "When will it arrive?"
});

// Send multiple messages at once
connection.sendHistoryMessages([
  { type: "History", role: "user", content: "Thank you!" },
  { type: "History", role: "assistant", content: "You're welcome!" }
]);

Files Modified

New Files

  • src/lib/types/HistoryConversationText.ts - Conversation history message type
  • src/lib/types/HistoryFunctionCall.ts - Function call history message type
  • tests/unit/agent-history.test.ts - Comprehensive test suite

Modified Files

  • src/lib/types/AgentLiveSchema.ts - Added flags and agent.context properties
  • src/lib/types/index.ts - Exported new history types
  • src/packages/AgentLiveClient.ts - Added three new history methods

Quality Assurance

Code Quality

  • TypeScript Compliance: Full type safety with proper interfaces and generics
  • Industry Best Practices: Following existing SDK patterns and conventions
  • Documentation: Comprehensive JSDoc comments with examples and links
  • Error Handling: Proper error management and user-friendly messages

Testing Coverage

  • Schema Validation: All new properties accept correct types and reject invalid ones
  • Method Functionality: All client methods work as expected
  • Edge Cases: Empty arrays, optional properties, and backward compatibility
  • Integration: Real-world usage scenarios tested successfully

Performance & Compatibility

  • No Performance Impact: New features are opt-in and don't affect existing functionality
  • Backward Compatible: 100% compatibility with existing agent configurations
  • Memory Efficient: History messages are processed without unnecessary overhead
  • Build Optimization: Clean compilation with no warnings or errors

Types of changes

What types of changes does your code introduce to the community JavaScript SDK?
Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update or tests (if none of the other choices apply)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

  • I have read the CONTRIBUTING doc
  • I have lint'ed all of my code using repo standards
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

Summary by CodeRabbit

  • New Features
    • Agents can accept and receive conversation and function-call history; agent config includes a history on/off flag and optional context messages.
  • Events
    • New "History" event for delivering history payloads.
  • Examples
    • Added a Node.js example demonstrating history-enabled agent workflows, function-calling, and live audio streaming.
  • Tests
    • Added tests validating history configuration, message sending, type safety, and backward compatibility.

@jpvajda jpvajda requested a review from lukeocodes July 23, 2025 22:43
Copy link
Contributor

coderabbitai bot commented Jul 23, 2025

Walkthrough

Adds two history message types and exports; extends AgentLiveSchema with optional flags.history and agent.context.messages; adds AgentLiveClient methods to send history messages; introduces an AgentEvents History enum member; adds unit tests; and provides a Node example demonstrating history + function calling.

Changes

Cohort / File(s) Change Summary
Schema extension
src/lib/types/AgentLiveSchema.ts
Imported HistoryConversationText and HistoryFunctionCall; added flags?: { history?: boolean }; added agent.context?: { messages?: (HistoryConversationText | HistoryFunctionCall)[] }.
History message types
src/lib/types/HistoryConversationText.ts, src/lib/types/HistoryFunctionCall.ts
Added HistoryConversationText (type: "History", role: `"user"
Type exports
src/lib/types/index.ts
Exported new history types (export * from "./HistoryConversationText", export * from "./HistoryFunctionCall").
Client API
src/packages/AgentLiveClient.ts
Added sendHistoryConversationText, sendHistoryFunctionCall, and sendHistoryMessages to serialize and send history messages over the WebSocket.
Events enum
src/lib/enums/AgentEvents.ts
Added History = "History" enum member with JSDoc describing the payload.
Tests
tests/unit/agent-history.test.ts
New unit tests covering flags.history handling, optional context.messages, single and batch history sends, type safety, and edge cases.
Examples
examples/node-agent-live-history/index.js, examples/node-agent-live-history/package.json
Added a Node.js example demonstrating history + function calling and its package manifest.
Whitespace/formatting
src/packages/AbstractLiveClient.ts
Minor formatting/whitespace adjustments only (no behavioral changes).

Sequence Diagram(s)

sequenceDiagram
    participant Client as AgentLiveClient
    participant WS as WebSocket

    Client->>WS: sendHistoryConversationText(historyMessage)
    Client->>WS: sendHistoryFunctionCall(historyMessage)
    Client->>WS: sendHistoryMessages([historyMessage...])
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested reviewers

  • lukeocodes
  • naomi-lgbt

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 67d81e9 and accd41b.

📒 Files selected for processing (1)
  • examples/node-agent-live-history/package.json (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • examples/node-agent-live-history/package.json
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Test Browser Builds
✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/agent-context-history

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@codecov-commenter
Copy link

codecov-commenter commented Jul 23, 2025

Codecov Report

❌ Patch coverage is 64.28571% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.34%. Comparing base (6634e40) to head (accd41b).

Files with missing lines Patch % Lines
src/packages/AbstractLiveClient.ts 16.66% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #424      +/-   ##
==========================================
+ Coverage   75.17%   75.34%   +0.17%     
==========================================
  Files          26       26              
  Lines        1152     1160       +8     
  Branches      292      292              
==========================================
+ Hits          866      874       +8     
  Misses        286      286              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jpvajda
Copy link
Contributor Author

jpvajda commented Jul 29, 2025

I need to double checkout the approach after chatting with @lukeocodes

@jpvajda jpvajda force-pushed the feat/agent-context-history branch from ab6a2f3 to 9557a5e Compare August 1, 2025 23:55
@jpvajda
Copy link
Contributor Author

jpvajda commented Aug 1, 2025

@lukeocodes I double checked the approach and it's good, this is ready for review.

@jpvajda
Copy link
Contributor Author

jpvajda commented Aug 11, 2025

@lukeocodes I moved the imports, back to you.

@jpvajda jpvajda requested a review from lukeocodes August 11, 2025 18:20
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
src/lib/enums/AgentEvents.ts (1)

70-75: Doc: Expand History event payload to include function_calls variant

Current JSDoc only documents the conversation text shape. Update to document both the conversation and function call history payloads for clarity.

   /**
-   * Triggered when history events are received from the agent.
-   * { type: "History", role: string, content: string }
+   * Triggered when history events are received from the agent.
+   * Payload can be one of:
+   * - { type: "History", role: "user" | "assistant", content: string }
+   * - { type: "History", function_calls: { id: string; name: string; client_side: boolean; arguments: string; response?: string }[] }
+   * See: HistoryConversationText and HistoryFunctionCall types.
    */
   History = "History",
examples/node-agent-live-history/index.js (2)

8-8: Fix misleading API key log placement

"API Key loaded successfully" is logged before the presence check, which can mislead. Move the success log to after the env var validation.

-console.log("🔑 API Key loaded successfully");
+// API key success log moved to after validation
@@
   if (!process.env.DEEPGRAM_API_KEY) {
     console.error("❌ Error: DEEPGRAM_API_KEY environment variable is required");
     console.log("💡 Run with: DEEPGRAM_API_KEY=your_api_key_here npm start");
     process.exit(1);
   }
+  console.log("🔑 API Key loaded successfully");

Also applies to: 35-41


193-206: Guard: handle only client-side function calls

Process only client-side function requests to avoid executing unintended server-side calls.

   // Process each function call in the request
   for (const func of data.functions) {
     console.log(`🔄 Processing function: ${func.name} with args: ${func.arguments}`);
+    if (!func.client_side) {
+      console.warn(`⏭️  Skipping non client-side function: ${func.name}`);
+      continue;
+    }
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ecf9106 and 67d81e9.

📒 Files selected for processing (4)
  • examples/node-agent-live-history/index.js (1 hunks)
  • examples/node-agent-live-history/package.json (1 hunks)
  • src/lib/enums/AgentEvents.ts (1 hunks)
  • src/packages/AbstractLiveClient.ts (3 hunks)
✅ Files skipped from review due to trivial changes (1)
  • src/packages/AbstractLiveClient.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Test Browser Builds
🔇 Additional comments (1)
examples/node-agent-live-history/index.js (1)

110-113: Verify and align the audio encoding with the BBC stream source

The snippet in examples/node-agent-live-history/index.js (around lines 110–113) currently declares:

input: {
  encoding: "linear16",
  sample_rate: 16000
}

However, the BBC World Service stream is typically served as MP3 (audio/mpeg), so sending raw PCM may result in errors or garbled audio.

Please verify:

  • The actual Content-Type returned by your BBC stream endpoint, for example:
    curl -sI "<STREAM_URL>" | grep -i Content-Type
  • Which input encodings the Agent Live service supports (see your Agent Live docs).

Then apply one of the following:

Option A: If MP3 is supported, update to:

 audio: {
   input: {
-    encoding: "linear16",
-    sample_rate: 16000
+    encoding: "mp3"
   }
 },

Option B: If you must use PCM (linear16), switch your source to a 16 kHz PCM stream or insert a decode/transcode step before sending.

@jpvajda
Copy link
Contributor Author

jpvajda commented Aug 11, 2025

@lukeocodes This now includes a working example of Context History with Agent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants