[BUG]: Memory Leak for temporal worker processes

[mnahkies]

Tracer Version(s)

5.51.0

Node.js Version(s)

22.13.1

Bug Report

Related: #5554

We've been experiencing memory leaks for our temporal worker processes. The last known "good" version of dd-trace for us is v5.28.0 - though we haven't exhaustively tried every patch version since then.

We're using the https://www.npmjs.com/package/@temporalio/interceptors-opentelemetry package to expose metrics from temporal, example annotation:

  podAnnotations:
    ad.datadoghq.com/energy-device-service.checks: |
      {
        "openmetrics": {
          "init_configs": {},
          "instances": [
            {
              "openmetrics_endpoint": "http://%%host%%:9464/metrics",
              "metrics": [
                "temporal_workflow_failed",
                "temporal_workflow_completed",
                "temporal_workflow_endtoend_latency"
              ]
            }
          ]
        }
      }

The process also runs a koa server exposing a couple of HTTP health check endpoints, and uses the prisma ORM with the normal node-postgres driver.

We don't observe memory leaks on our pure REST (koa) deployments or GCP PubSub deployments - leading me to believe that the leak is specifically related to the use of the temporal SDK.

I've said bundled using webpack, because I believe that temporal does bundle the workflow code using webpack. We don't configure/invoke any bundling ourselves.

Reproduction Code

No response

Error Logs

No response

Tracer Config

No response

Operating System

No response

Bundling

Webpack

[BridgeAR]

@mnahkies thank you for your report! We are going to look into this with high priority!