Skip to content

Expose metrics of engine startup permit state #7072

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

LennonChin
Copy link
Contributor

Why are the changes needed?

The metrics kyuubi_operation_state_LaunchEngine_* cannot reflect the state of Semaphore after configuring the maximum engine startup limit through kyuubi.server.limit.engine.startup, add some metrics to show the relevant permit state.

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

@LennonChin
Copy link
Contributor Author

@wForget cc

@codecov-commenter
Copy link

codecov-commenter commented May 22, 2025

Codecov Report

Attention: Patch coverage is 0% with 11 lines in your changes missing coverage. Please review.

Project coverage is 0.00%. Comparing base (6d99b20) to head (d6bf369).

Files with missing lines Patch % Lines
...g/apache/kyuubi/session/KyuubiSessionManager.scala 0.00% 7 Missing ⚠️
...a/org/apache/kyuubi/metrics/MetricsConstants.scala 0.00% 4 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff           @@
##           master   #7072   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files         697     697           
  Lines       43192   43203   +11     
  Branches     5854    5854           
======================================
- Misses      43192   43203   +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@@ -65,6 +65,9 @@ These metrics include:
| `kyuubi.engine.timeout` | | counter | 1.2.0 | <div style='width: 150pt;word-wrap: break-word;white-space: normal'> cumulative timeout engines</div> |
| `kyuubi.engine.failed` | `${user}` | counter | 1.2.0 | <div style='width: 150pt;word-wrap: break-word;white-space: normal'> cumulative explicitly failed engine count for a `${user}`</div> |
| `kyuubi.engine.failed` | `${errorType}` | counter | 1.2.0 | <div style='width: 150pt;word-wrap: break-word;white-space: normal'> cumulative explicitly failed engine count for a particular `${errorType}`, e.g. `ClassNotFoundException`</div> |
| `kyuubi.engine.startup.permit.limit.total` | | meter | 1.10.1 | <div style='width: 150pt;word-wrap: break-word;white-space: normal'> concurrently startup engines permit limit </div> |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1.11.0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pan3793 Updated doc version, added grafana pannel json and resolved conflicts, please review again.

@pan3793
Copy link
Member

pan3793 commented May 26, 2025

It's also encouraged to update the Grafana Dashboard when adding new metrics https://github.com/apache/kyuubi/tree/master/grafana

@LennonChin LennonChin force-pushed the engine_startup_metrics branch from 483f44e to d6bf369 Compare May 26, 2025 06:41
@LennonChin LennonChin requested a review from pan3793 May 26, 2025 06:50
Copy link
Member

@pan3793 pan3793 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, waiting for CI.

@pan3793 pan3793 added this to the v1.11.0 milestone May 29, 2025
@pan3793 pan3793 closed this in cad5a39 May 29, 2025
@pan3793
Copy link
Member

pan3793 commented May 29, 2025

Thanks, merged to master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants