-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Added new page for automated analysis, as well as screenshots #29406
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
6f49879
added new page for automated analysis, as well as screenshots
1ec19bb
Crop PII and Preview text.
brett0000FF d83113d
Update content/en/profiler/automated_analysis.md
brantigua 91a1b8c
Update content/en/profiler/automated_analysis.md
brantigua ef0b317
Update content/en/profiler/automated_analysis.md
brantigua a0876a4
Update content/en/profiler/automated_analysis.md
brantigua 322f3c4
Update content/en/profiler/automated_analysis.md
brantigua 267edf5
Update content/en/profiler/automated_analysis.md
brantigua e73bdd3
Update content/en/profiler/connect_traces_and_profiles.md
brantigua e7d25fd
Update content/en/profiler/automated_analysis.md
brantigua 546b7be
Update content/en/profiler/automated_analysis.md
brantigua 28a93ff
updated based on teams comments
2e2473e
updated image for details view
69ea622
Clean up missing commits and remaining typos.
brett0000FF f163dc7
Change to use normal table.
brett0000FF File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,114 @@ | ||
--- | ||
title: Automated Analysis | ||
description: Automatically surfacing critical issues with contextual insights and recommended next steps | ||
aliases: | ||
- /profiler/automated_analysis/ | ||
further_reading: | ||
- link: 'profiler/enabling' | ||
tag: 'Documentation' | ||
text: 'Enable continuous profiler for your application' | ||
- link: 'getting_started/profiler' | ||
tag: 'Documentation' | ||
text: 'Getting Started with Profiler' | ||
- link: 'https://www.datadoghq.com/blog/introducing-datadog-profiling/' | ||
tag: 'Blog' | ||
text: 'Introducing always-on production profiling in Datadog' | ||
- link: 'https://www.datadoghq.com/blog/continuous-profiler-timeline-view/' | ||
tag: 'Blog' | ||
text: "Diagnose runtime and code inefficiencies using Continuous Profiler's timeline view" | ||
multifiltersearch: | ||
# "id" must match the corresponding key in the "data" object | ||
headers: | ||
- name: Name | ||
id: name | ||
filter_by: true | ||
- name: Severity | ||
id: severity | ||
filter_by: true | ||
- name: Description | ||
id: description | ||
filter_by: false | ||
|
||
data: | ||
- name: GC Setup | ||
severity: Info | ||
description: Triggers when one of the following is detected - serial GC used on a multi-core machine, parallel GC on a single-core machine, more GC threads were configured than available cores, or a parallel GC was configured to run in 1 thread | ||
- name: Duplicated Flags | ||
severity: Info | ||
description: Triggers if duplicate flags were provided to the runtime (e.g. `-Xmx2g -Xmx5g`). This is a problem as it may lead to changes not having the expected effect. | ||
- name: Options | ||
severity: Warn | ||
description: Triggers if undocumented, deprecated or non-recommended option flags were detected. | ||
- name: Stackdepth Setting | ||
severity: Warn | ||
description: Triggers if events were found with truncated stacktraces which may make it hard to understand profiling data. | ||
- name: VMOperation Peak Duration | ||
severity: Warn | ||
description: Triggers if a blocking VM operation (or combination of ops close in time) taking more than 2 seconds. Reports details about the operation with the highest duration. | ||
- name: GC Pauses | ||
severity: Warn | ||
description: Triggers if more than 10% of time was spent in GC pauses. | ||
- name: GC Pause Peak Duration | ||
severity: Info | ||
description: Triggers if at least one GC pause took more than 1 second. | ||
- name: Deadlocked Threads | ||
severity: Warn | ||
description: Triggers if we see a custom datadog event capturing deadlocked threads during the profile. | ||
- name: Deadlocked Threads Detected | ||
severity: Warn | ||
description: Triggers if max number of deadlocked threads over query context is bigger than 0. | ||
- name: Thrown Exceptions | ||
severity: Warn | ||
description: Triggers when the rate of thrown (caught and uncaught) exceptions per minute goes above a threshold (defauls to 10K) | ||
- name: Primitive Value Boxing | ||
severity: Info | ||
description: Triggers if more than 5% of CPU time was spent doing primitive<>object value conversions. | ||
- name: Explicit GC | ||
severity: Info | ||
description: Triggers if there are System.gc() calls. | ||
- name: Head of line blocking | ||
severity: Info | ||
description: Triggers if a queue event gets stuck behind the given activity. | ||
--- | ||
|
||
## Overview | ||
Automated Analysis helps teams detect and troubleshoot problems faster, without requiring deep expertise in code profiling. Powered by Continuous Profiler, Automated Analysis continuously monitors your applications and surfaces critical issues in real time, along with actionable insights to guide resolution. When a problem is detected, Automated Analysis provides: | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
- A high-level summary explaining the issue and why it matters | ||
- Contextual insights from profiling data (e.g., affected methods, packages, or processes) | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- Recommended next steps to help you resolve the issue efficiently | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
By reducing the expertise required to take action, Automated Analysis enables faster resolution of performance issues, increases product adoption, and helps teams avoid costly blind spots. It also lays the foundation for powering performance and cost insights across the Datadog platform. | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
{{< img src="profiler/profiling_automated_analysis.png" alt="The Profiler Thread Time line showing a Thrown Exception insight" style="width:100%;" >}} | ||
|
||
## Explore insights | ||
Automated Analysis will be surfaced within the [Profile explorer][1] | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- As a banner at the top of the page when you're scoped to a specific service | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
{{< img src="profiler/profiling_automated_analysis_banner.png" alt="The Automated Analysis banner displaying insights detected for a given service" style="width:100%;">}} | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- As a column within the service list | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
{{< img src="profiler/profiling_automated_analysis_column.png" alt="The Automated Analysis column displaying insights detected for a given service within the service list" style="width:100%;">}} | ||
|
||
Clicking an an insight in either area will display a high-level summary explaining the issue, contextual insights from profiling data, and recommended next steps. | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
{{< img src="profiler/profiling_automated_analysis_details.png" alt="Expanded Profiling Insights showing the details of a detected Issue" style="width:100%;">}} | ||
|
||
## Aggregated vs Individual Insights | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Individual insights are tied to a specific profile and show up directly in the flamegraph and timeline views. They highlight problems detected within a single recording, giving detailed context about a specific performance issue observed in that exact moment. | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
{{< img src="profiler/profiling_automated_analysis_individual.png" alt="Example of an individual profile insight within the trace explorer" style="width:100%;">}} | ||
|
||
Aggregated insights, on the other hand, are surfaced at the service level and represent patterns detected across many profiles. They help you identify recurring or systemic issues affecting a service over time, making them ideal for quickly spotting problematic trends without digging into individual profiles. | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
{{< img src="profiler/profiling_automated_analysis_aggregated.png" alt="Example of an aggregated insight within the profiler explorer" style="width:100%;">}} | ||
|
||
|
||
## Supported Insights | ||
brantigua marked this conversation as resolved.
Show resolved
Hide resolved
|
||
<!-- The table below is auto-generated. Add new entries in multifiltersearch with new insights as they become available. --> | ||
{{< multifilter-search >}} | ||
|
||
|
||
## Further reading | ||
|
||
{{< partial name="whats-next/whats-next.html" >}} | ||
|
||
[1]: https://app.datadoghq.com/profiling/explorer | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
High-level question, but what's the reason we call this Automated Analysis instead of Recommendations like we have for APM, CCM, DBM?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've pushed for that in the past but many of these insights are missing a vital piece to make then a Recommendation which is: always having a clear impact statement.
We can say that the findings here can help with a given issue, but it's hard to definitely tie that to an impact (example: fixing GC issues will reduce CPU load but we can't concretely say by how much due to many other factors). However, when we do have that info, we promote that insight and it will show up in APM recommendations