Skip to content

feat(amazonq): Auto Debug Functionality #7609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 53 commits into
base: feature/auto-debug
Choose a base branch
from

Conversation

MarcoWang3
Copy link

Problem

Amazon Q users currently face significant challenges when debugging code errors in VS Code:

  • Reactive Debugging Workflow: Users only discover and address errors after they accumulate, leading to context switching between VS Code's Problems panel and Amazon Q chat
  • Manual Error Context Assembly: Developers must manually copy error messages, code snippets, and diagnostic information to get effective AI assistance
  • Fragmented Integration Points: No seamless connection between VS Code's rich diagnostic system and Amazon Q's AI-powered debugging capabilities
  • Notification Fatigue: Existing error detection systems either overwhelm users with notifications or provide no proactive assistance
  • Limited Error Intelligence: Raw VS Code diagnostics lack the contextual richness needed for effective AI-powered debugging

This disconnect between VS Code's powerful diagnostic capabilities and Amazon Q's AI assistance creates friction in the development workflow, resulting in longer debugging cycles and reduced productivity.

Solution

Implemented a comprehensive Amazon Q Auto Debug System that bridges VS Code's diagnostic ecosystem with Amazon Q's AI capabilities through an intelligent, event-driven architecture. This system provides proactive error detection with smart notifications and seamless AI-powered debugging assistance.

High-Level Auto Debug Workflow

File Save/Open → Diagnostic Capture → Baseline Comparison → Smart Filtering → Multi-Fallback Notifications → Amazon Q Integration

Detailed Flow:

  1. Event-Driven Triggers: Monitor file save/open events (not typing) to avoid notification spam
  2. Diagnostic Baseline Tracking: Capture diagnostic snapshots and compare against session baselines
  3. Intelligent Problem Detection: Identify new vs. existing problems using sophisticated comparison algorithms
  4. Multi-Layer Filtering: Apply severity, source, and configuration-based filtering to focus on relevant issues
  5. Robust Notification System: Triple-fallback notification system (warning → error → status bar) ensuring reliable delivery
  6. Direct LSP Integration: Route requests through Amazon Q's language server for seamless chat integration
  7. Batch Processing: Handle up to 15 errors efficiently in single Amazon Q interactions

Architecture & Component Structure

Core System Architecture

AutoDebugFeature (Orchestration Layer - 320 lines)

  • Primary Role: Central coordinator and VS Code extension integration point
  • Key Responsibilities: Component lifecycle, event routing, command registration, notification orchestration
  • Integration Points: VS Code extension context, command palette, user interactions
  • Dependencies: AutoDebugController, ContextMenuProvider, AutoDebugCodeActionsProvider
  • Event Management: Multi-fallback notification system with Promise-based error handling

AutoDebugController (Business Logic Engine - 600 lines)

  • Primary Role: Core diagnostic processing and AI communication orchestrator
  • Session Management: Baseline tracking, problem detection, state persistence
  • File Event Processing: Debounced save/open handlers with 2-second configurable delay
  • AI Integration: Direct LSP communication with Amazon Q through delegation pattern
  • Batch Processing: Intelligent error grouping and context assembly for efficient AI requests
  • Dependencies: DiagnosticsMonitor, ProblemDetector, ErrorContextFormatter, AutoDebugLspClient

Diagnostic Processing Pipeline

DiagnosticsMonitor (Diagnostic Capture Engine - 180 lines)

  • Real-Time Monitoring: VS Code diagnostic change event handling with 500ms debouncing
  • Snapshot Management: Baseline capture, comparison, and efficient storage
  • Filtering Capabilities: Severity-based filtering (Error/Warning/Info/Hint) and source exclusion
  • Cross-File Correlation: Intelligent diagnostic grouping across multiple files
  • Performance Optimization: Efficient diagnostic comparison algorithms

ProblemDetector (Intelligence Layer - 200 lines)

  • New Problem Detection: Sophisticated baseline comparison using diagnostic signatures
  • Categorization Logic: Multi-dimensional problem classification (severity, source, file)
  • Related File Detection: Heuristic-based related file identification for contextual debugging
  • Filtering Intelligence: Configurable source filtering (excludes spell-checker by default)
  • Problem Prioritization: Critical problem identification for notification triggering

AI Integration & Context Assembly

ErrorContextFormatter (Context Enrichment - 220 lines)

  • Code Context Extraction: Intelligent surrounding code capture with configurable line ranges
  • Multi-Language Support: Syntax-aware formatting for 10+ programming languages
  • Problem Aggregation: Smart grouping and formatting for AI consumption
  • Rich Metadata Assembly: File paths, line numbers, error codes, related information
  • AI-Optimized Output: Structured formatting optimized for Amazon Q's processing capabilities

AutoDebugLspClient (Communication Layer - 250 lines)

  • Delegation Architecture: Routes through real AutoDebugLspClient in amazonq package for production reliability
  • LSP Protocol Integration: Direct language server communication bypassing webview limitations
  • Encrypted Communication: Support for encrypted channels with key management
  • Response Handling: Automatic Amazon Q chat UI integration with response processing
  • Error Recovery: Robust error handling with graceful degradation

User Interface Integration

ContextMenuProvider (Right-Click Integration - 200 lines)

  • Context-Aware Actions: Dynamic menu options based on diagnostic presence
  • Multi-Function Commands: "Fix with Amazon Q", "Add to Chat", "Explain Problem", Session Management
  • Enhanced Fix Logic: Supports both focused fixes (specific range) and file-wide batch processing
  • LSP Integration: Uses proven AutoDebugController pipeline for reliable Amazon Q communication

AutoDebugCodeActionsProvider (Light Bulb Integration - 150 lines)

  • Comprehensive Language Support: TypeScript, JavaScript, Python, Java, C#, C++, Go, Rust, PHP, Ruby, JSON, YAML, XML
  • Dynamic Action Generation: Context-sensitive quick fixes based on diagnostic analysis
  • Session Management Integration: Start/stop debugging sessions directly from light bulb
  • Preferred Actions: Intelligent action prioritization for optimal user experience

Extension Integration

Activation System (Extension Bootstrap - 100 lines)

  • VS Code Configuration: Workspace settings integration with comprehensive defaults
  • Extension Lifecycle: Proper initialization, activation, and disposal patterns
  • Client Management: Language client and encryption key handling
  • Error Recovery: Graceful activation failure handling with partial functionality preservation

Intelligent Workflow Implementation

Event-Driven Problem Detection

  • 📁 File Event Monitoring: Monitors save/open events exclusively (prevents typing interruptions)
  • ⏱️ Smart Debouncing: Configurable 2-second delay prevents rapid-fire notifications
  • 📊 Baseline Comparison: Session-based problem tracking identifies truly new issues
  • 🎯 Severity Intelligence: Configurable filtering focuses on actionable problems (errors by default)
  • 🔍 Source Filtering: Excludes noise sources like spell-checkers while including relevant diagnostics

Multi-Fallback Notification System

  • 🚨 Primary Notifications: Warning messages with "Fix with Amazon Q" and "Dismiss" options
  • 🔄 Automatic Fallbacks: Information messages → Error messages → Status bar messages
  • ⚡ Promise-Based Handling: Robust error recovery with detailed logging for debugging
  • 🎛️ User Choice Handling: Intelligent response processing with AI integration triggers
  • 📊 Debug Mode: Enhanced notification testing with severity mismatch detection

Amazon Q Integration Architecture

  • 🤖 Direct LSP Communication: Bypasses webview limitations using proven language server protocols
  • 📝 Rich Context Assembly: Code snippets, line numbers, error details, and file context
  • 🔢 Batch Processing: Handles up to 15 errors per request for efficient AI utilization
  • 💬 Natural Chat Integration: Messages appear in Amazon Q chat UI automatically
  • 🔄 Response Processing: Language server handles AI responses and code application

IDE Integration Features

  • 🖱️ Context Menu Integration: Right-click access for immediate debugging assistance
  • 💡 Light Bulb Actions: VS Code quick fix integration with Amazon Q options
  • ⌨️ Command Palette: Full command integration for power users
  • 📊 Session Management: Visual session status with start/stop controls
  • 🔧 Configuration Management: Comprehensive VS Code settings integration

New Functionalities

Intelligent Error Detection

  • 🔍 Event-Based Monitoring: File save/open event monitoring (not typing) prevents notification spam
  • 📊 Baseline Comparison: Smart detection of new problems vs. existing issues using diagnostic signatures
  • 🎯 Severity Filtering: Configurable filtering by error severity (error/warning/info/hint)
  • 🔧 Source Filtering: Intelligent exclusion of irrelevant diagnostic sources (spell-checker excluded by default)
  • ⚡ Debounced Processing: 500ms diagnostic debouncing and 2-second notification debouncing

Proactive Multi-Fallback Notifications

  • 🚨 Smart Warning Messages: Primary notifications with "Fix with Amazon Q" and "Dismiss" buttons
  • 🔄 Automatic Fallbacks: Information messages → Error messages → Status bar messages for reliability
  • 🎨 User-Friendly Messages: Clear, actionable notification text with error counts
  • ⚙️ Configurable Thresholds: Customizable error count thresholds (default: 1 error triggers notification)
  • 🎭 Debug Mode: Enhanced logging with severity analysis and notification path debugging

AI-Powered Batch Processing

  • 🤖 Direct Amazon Q Integration: Seamless LSP-based connection to Amazon Q chat system
  • 📝 Rich Error Context: Code snippets with 3-line context, file paths, line numbers, error details
  • 🔧 Batch Fix Processing: Handle up to 15 errors at once for efficient debugging sessions
  • 💬 Natural Language Explanations: AI-powered error explanations with code context
  • 🎯 Focused Problem Solving: Targeted assistance for specific error types and sources

Comprehensive IDE Integration

  • 🖱️ Context Menu Integration: Right-click options for "Fix with Amazon Q", "Add to Chat", "Explain Problem"
  • 💡 Code Actions Provider: VS Code light bulb integration with preferred quick fix actions
  • ⌨️ Command Palette: Full command palette integration (detect problems, toggle, show status)
  • 📊 Session Management: Start/stop debugging sessions with baseline tracking and status display
  • 🔄 Multi-Language Support: TypeScript, JavaScript, Python, Java, C#, C++, Go, Rust, PHP, Ruby, JSON, YAML, XML

Advanced Configuration Options

  • ⚙️ Flexible Configuration: Comprehensive VS Code workspace settings integration
  • 🎚️ Severity Control: Fine-grained control over which diagnostic levels trigger assistance
  • ⏱️ Timing Customization: Configurable debounce delays and notification thresholds
  • 📁 Source Management: Include/exclude specific diagnostic sources with smart defaults
  • 🔧 Debug Controls: Enhanced debugging and logging capabilities for troubleshooting

Unit Test Design & Coverage

Comprehensive Test Strategy

Implemented 95%+ code coverage across all components with systematic edge case testing:

AutoDebugFeature Tests (12 test categories, 35+ test cases)

  • Initialization & Activation: Constructor validation, extension context integration, configuration handling
  • Configuration Management: Dynamic config updates, validation, and state management
  • Problem Detection Workflows: Manual and automatic problem detection with various scenarios
  • Notification System Testing: Multi-method notification testing with all fallback mechanisms
  • Command Integration: Command registration, execution validation, and error handling
  • LSP Communication: Language client setup, encryption key handling, and communication validation
  • Session Management: Start/stop session workflows and state persistence
  • Resource Management: Clean disposal, memory leak prevention, and proper cleanup

AutoDebugController Tests (15 test categories, 45+ test cases)

  • Session Management: Start, stop, baseline capture, and state management operations
  • Diagnostic Processing: File event handling, debouncing, and diagnostic change processing
  • Problem Detection Logic: Baseline comparison, new problem identification, and filtering
  • Error Context Assembly: Context creation, formatting validation, and AI preparation
  • LSP Integration: Chat message sending, response handling, and error recovery
  • Batch Processing: Multi-error handling, context assembly, and efficient AI requests
  • Configuration Handling: Dynamic updates, validation, and setting persistence
  • Performance Testing: Large file handling, concurrent operations, and resource optimization

Edge Cases & Robustness Testing

Input Validation & Error Handling

  • Null/Undefined Inputs: All public methods handle invalid inputs gracefully
  • Empty Diagnostics: Proper handling of files with no errors or empty diagnostic collections
  • Invalid File Paths: Robust path validation, normalization, and error recovery
  • Malformed Diagnostics: VS Code diagnostic parsing with comprehensive error handling
  • Network Issues: LSP connection failures, timeouts, and recovery mechanisms

Performance & Scalability

  • Large Files: Testing with files containing 100+ errors and complex diagnostic scenarios
  • Concurrent Operations: Multiple simultaneous debugging sessions and resource management
  • Memory Management: Session cleanup, diagnostic snapshot optimization, and leak prevention
  • High-Frequency Events: Rapid file save scenarios, debouncing effectiveness, and performance impact
  • Cross-Platform Compatibility: Windows, macOS, and Linux path handling and behavior verification

VS Code Integration Robustness

  • API Compatibility: Comprehensive VS Code API mocking and version compatibility testing
  • Extension Conflicts: Interaction testing with other diagnostic providers and extensions
  • Diagnostic System Integration: Language server diagnostic handling and protocol compliance
  • UI Integration: Context menu, light bulb, and command palette integration validation
  • Configuration System: Workspace settings, user preferences, and configuration validation

Mock Strategy & Test Infrastructure

  • Complete VS Code API Mocking: Comprehensive stubbing of all vscode.* APIs with realistic behavior
  • Diagnostic System Simulation: Controlled diagnostic change events with timing and content control
  • LSP Client Mocking: Simulated language server communication with success/failure scenarios
  • Event System Testing: EventEmitter validation, cleanup verification, and memory leak detection
  • Configuration Testing: Settings persistence, validation, and dynamic update verification

Benefits of This New Feature

Enhanced Development Experience

  • ⚡ Proactive Error Assistance: Get AI help with errors immediately when they occur, not after accumulation
  • 🔄 Seamless Workflow Integration: Debug assistance without leaving current context or switching tools
  • 🎯 Intelligent Problem Filtering: Focus on actionable problems while filtering out noise and irrelevant diagnostics
  • 💡 AI-Powered Error Understanding: Leverage Amazon Q's intelligence for contextual error analysis and solutions

Improved Developer Productivity

  • 🚀 Faster Debugging Cycles: Immediate access to AI assistance reduces time spent on error analysis
  • 📊 Efficient Batch Processing: Handle multiple errors simultaneously with intelligent context assembly
  • 🔧 Direct Integration Access: No context switching between VS Code Problems panel and Amazon Q chat
  • ⚙️ Customizable Experience: Tailor notification behavior and filtering to match individual workflows

Technical Architecture Advantages

  • 🏗️ Modular Component Design: Clean separation of concerns enables maintainability and extensibility
  • 🧪 Comprehensive Test Coverage: 95%+ test coverage ensures reliability, stability, and regression prevention
  • ⚡ Performance Optimization: Smart debouncing, efficient diagnostic processing, and resource management
  • 🔧 Extensible Architecture: Easy addition of new error types, diagnostic sources, and debugging capabilities

Configuration & Customization

Users can fine-tune the Auto Debug experience through comprehensive VS Code settings:

{
    "amazonq.autoDebug.enabled": true,
    "amazonq.autoDebug.autoReportThreshold": 1,
    "amazonq.autoDebug.severityFilter": ["error"],
    "amazonq.autoDebug.debounceMs": 2000,
    "amazonq.autoDebug.excludedSources": ["spell-checker"],
    "amazonq.autoDebug.includedSources": []
}

Configuration Options:

  • Threshold Control: Set minimum error count for notifications (default: 1)
  • Severity Filtering: Choose which diagnostic levels trigger assistance (error/warning/info/hint)
  • Timing Customization: Adjust debounce delays for different development workflows
  • Source Management: Include/exclude specific diagnostic providers with intelligent defaults
  • Debug Controls: Enhanced logging and troubleshooting capabilities for system analysis

Integration & Activation

The Auto Debug system integrates seamlessly with existing VS Code extensions through a clean activation API:

// Extension activation example
import { activateAutoDebug, getDefaultAutoDebugConfig } from './amazonq/autoDebug/activation'

export async function activate(context: vscode.ExtensionContext) {
    const config = getDefaultAutoDebugConfig()
    const autoDebugFeature = await activateAutoDebug(context, config, client, encryptionKey)
    // Auto Debug is now active and monitoring for errors
}

Screenshots

Auto.detect.and.fix.mov
Fix.with.Amazon.Q.mov
QuickFix.and.Explain.with.Amazon.Q.mov

  • Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
  • Your code changes must meet the guidelines in CONTRIBUTING.md.
  • License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

MarcoWang3 and others added 30 commits June 26, 2025 16:11
…

## Problem
- to reduce # of service calls within short time duration and result in
throttling error thrown
- 


## Solution


---

- Treat all work as PUBLIC. Private `feature/x` branches will not be
squash-merged at release time.
- Your code changes must meet the guidelines in
[CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines).
- License: I confirm that my contribution is made under the terms of the
Apache 2.0 license.
…ble deployment from WFS (aws#7315)

## Problem

- Save telemetry was being recorded for all document saves in VS Code,
not just for the active workflow studio document.
- Save & Deploy functionality required closing Workflow Studio before
starting deployment


## Solution

- Add URI comparison check to ensure telemetry is only recorded when the
saved document matches the current workflow studio document.
- Refactored publishStateMachine.ts to accept an optional TextDocument
parameter and updated activation.ts to support new interface
- Removed closeCustomEditorMessageHandler call from
saveFileAndDeployMessageHandler


---

- Treat all work as PUBLIC. Private `feature/x` branches will not be
squash-merged at release time.
- Your code changes must meet the guidelines in
[CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines).
- License: I confirm that my contribution is made under the terms of the
Apache 2.0 license.

---------

Co-authored-by: Diler Zaza <[email protected]>
## Problem

Instead of running `mvn dependency:copy-dependencies` and `mvn clean
install`, we have a JAR that we can execute which will gather all of the
project dependencies as well as some important metadata stored in a
`compilations.json` file which our service will use to improve the
quality of transformations.


## Solution

Remove Maven shell commands; add custom JAR execution.


---

- Treat all work as PUBLIC. Private `feature/x` branches will not be
squash-merged at release time.
- Your code changes must meet the guidelines in
[CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines).
- License: I confirm that my contribution is made under the terms of the
Apache 2.0 license.

---------

Co-authored-by: David Hasani <[email protected]>
## Problem

Minor text update request.

## Solution

Update text.

---

- Treat all work as PUBLIC. Private `feature/x` branches will not be
squash-merged at release time.
- Your code changes must meet the guidelines in
[CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines).
- License: I confirm that my contribution is made under the terms of the
Apache 2.0 license.

Co-authored-by: David Hasani <[email protected]>
## Problem

step functions still uses sdk v2

## Solution

- Refactor `DefaultStepFunctionsClient` to `StepFunctionsClient` using
ClientWrapper.
- Refactor references to `DefaultStepFunctionsClient` with
`StepFunctionsClient` and reference sdk v3 types

## Verification
- Verified CreateStateMachine, UpdateStateMachine works and
creates/updates StateMachine
- Verified AWS Explorer lists all StateMachines using ListStateMachine
- Verified StartExecution works
- Verified TestState, ListIAMRoles works in Workflow Studio, which also
fixes the bug that variables cannot be used in TestState in Workflow
Studio
- fixes aws#6819

![TestStateWithVariables](https://github.com/user-attachments/assets/ab981622-b773-4983-a9ce-a70c8fcfc711)
---

- Treat all work as PUBLIC. Private `feature/x` branches will not be
squash-merged at release time.
- Your code changes must meet the guidelines in
[CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines).
- License: I confirm that my contribution is made under the terms of the
Apache 2.0 license.
## Problem

Modify text in some places, and modify the manifest to include the `IDE`
field.

## Solution

Above

---

- Treat all work as PUBLIC. Private `feature/x` branches will not be
squash-merged at release time.
- Your code changes must meet the guidelines in
[CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines).
- License: I confirm that my contribution is made under the terms of the
Apache 2.0 license.

Co-authored-by: David Hasani <[email protected]>
1. update the commonAuthViewProvider to use the locally bundled vue.js, instead of
   to load the vue.js from CDN.
2. this going to resolve the customer request that When customer is trying to login to IDC via Q they are hitting a URL which is not part of the whitelist urls as per [documentation](https://docs.aws.amazon.com/amazonq/latest/qdeveloper-ug/firewall.html)
@MarcoWang3 MarcoWang3 closed this Jul 3, 2025
@MarcoWang3 MarcoWang3 reopened this Jul 3, 2025
@MarcoWang3 MarcoWang3 marked this pull request as ready for review July 3, 2025 20:23
@MarcoWang3 MarcoWang3 requested review from a team as code owners July 3, 2025 20:23
* This uses the same method as explainIssue command which works properly
*/
public async sendChatMessage(message: string, eventId: string): Promise<ChatResult> {
this.logger.info('AutoDebugLspClient: Sending message via webview postMessage')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Combine these logs into a single one

},
})

this.logger.info('AutoDebugLspClient: ✅ Message sent successfully via webview postMessage')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just fixed this!

* AutoDebug LSP client that uses the regular Amazon Q chat pipeline
* This ensures the response appears in the chat UI and can be processed by the language server
*/
export class AutoDebugLspClient {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need another client?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah because certain dependencies are not available at core side.

@ashishrp-aws ashishrp-aws marked this pull request as draft July 3, 2025 23:17
@laileni-aws laileni-aws marked this pull request as ready for review July 14, 2025 20:55
* This ensures perfect compatibility with the language server integration
*/
export function activateAutoDebug(client: LanguageClient, encryptionKey: Buffer): AutoDebugLspClient {
logger.info('AutoDebug: Activating AutoDebug LSP client using exact inline chat pattern')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we combine these into a single log

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah sure! Sorry I forgot to make changes to this one before!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was fixed!

context.extensionContext.subscriptions.push(autoDebugFeature)
getLogger().debug('AutoDebug feature added to extension subscriptions')
} catch (error) {
getLogger().error('Failed to activate AutoDebug feature: %s', error)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate logs

@MarcoWang3 MarcoWang3 force-pushed the feature/auto-debug branch from e71ffa9 to 0bdd5a3 Compare July 15, 2025 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.