feat(node): Instrument stream responses for openai #17110
Conversation
size-limit report 📦
|
RulaKhaled
changed the title
This reverts commit 1607304.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| return; | ||
| } | ||
| if (streamEvent instanceof Error) { | ||
| captureException(streamEvent); |
There was a problem hiding this comment.
m: when we capture an exception here, should we also set span status to errored?
also, I think we need to mark this as unhandled.
There was a problem hiding this comment.
Potentially yeah. It's a series of events we're capturing in a single span, which makes things a bit vague. I guess if one fails it's fair to assume the span is in an error state, will update
| setTokenUsageAttributes, | ||
| } from './utils'; | ||
|
|
||
| interface StreamingState { |
There was a problem hiding this comment.
m: It would be nice to have some doc strings about this interface and it's fields.
l: I'm not the biggest fan of having a big interface with a bunch of nullable fields. Can we type this a bit stronger? For example if chunk.usage we should always have promptTokens, completionTokens, and totalTokens defined. Having some helpers that construct state from input (builder pattern?) might also be a useful abstraction.
| span.setAttributes({ | ||
| [OPENAI_RESPONSE_TIMESTAMP_ATTRIBUTE]: new Date(timestamp * 1000).toISOString(), | ||
| }); | ||
| } |
There was a problem hiding this comment.
Bug: Function Change Introduces Invalid Default Attributes
The setCommonResponseAttributes function was modified to use required parameters, removing the original conditional checks that only set attributes if values were truthy. This change causes the function to always set span attributes, even when provided with empty strings or zero values from uninitialized streaming state. Consequently, spans will now include meaningless attributes such as empty strings for IDs/models and '1970-01-01T00:00:00.000Z' for timestamps, altering telemetry by setting invalid defaults instead of omitting these attributes.
Locations (1)
| @@ -195,6 +142,9 @@ function addRequestAttributes(span: Span, params: Record<string, unknown>): void | |||
| if ('input' in params) { | |||
| span.setAttributes({ [GEN_AI_REQUEST_MESSAGES_ATTRIBUTE]: JSON.stringify(params.input) }); | |||
| } | |||
| if ('stream' in params) { | |||
| span.setAttributes({ [GEN_AI_REQUEST_STREAM_ATTRIBUTE]: Boolean(params.stream) }); | |||
| } | |||
There was a problem hiding this comment.
Bug: Attribute Duplication Causes Type Inconsistency
The GEN_AI_REQUEST_STREAM_ATTRIBUTE is set twice: once in extractRequestAttributes with the original params.stream value, and again in addRequestAttributes as Boolean(params.stream). This duplication can lead to the attribute being overwritten with an inconsistent type (original vs. boolean).
Locations (1)
| description: 'chat error-model stream-response', | ||
| op: 'gen_ai.chat', | ||
| origin: 'manual', | ||
| status: 'ok', |
There was a problem hiding this comment.
Bug: Streaming Tests Fail on Error Handling
The tests for OpenAI streaming chat completions incorrectly expect status: 'ok' for error handling spans. When model: 'error-model' is used, an error is thrown before the stream is created, and the span's status should be unknown_error. This is consistent with non-streaming error handling and the instrumentation's behavior of setting an error status. This applies to both PII and non-PII test cases.
Locations (2)
This adds support for OpenAI streaming responses in the Node.js SDK
What's new