Skip to content

feat: better test reports #32526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 6, 2025
Merged

feat: better test reports #32526

merged 7 commits into from
May 6, 2025

Conversation

itsyoboieltr
Copy link
Contributor

@itsyoboieltr itsyoboieltr commented May 5, 2025

Description

Open in GitHub Codespaces

This PR improves the current test reporting capabilities.

Related issues

Fixes: #31994

Manual testing steps

  1. Test reporting works (and is less flaky)

Screenshots/Recordings

Not applicable

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

@itsyoboieltr itsyoboieltr self-assigned this May 5, 2025
Copy link
Contributor

github-actions bot commented May 5, 2025

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@metamaskbot metamaskbot added the team-extension-platform Extension Platform team label May 5, 2025
@github-project-automation github-project-automation bot moved this to Needs dev review in PR review queue May 5, 2025
@itsyoboieltr itsyoboieltr moved this from Needs dev review to Needs more work from the author in PR review queue May 5, 2025
@metamaskbot
Copy link
Collaborator

Builds ready [6b2a3ca]
UI Startup Metrics (1207 ± 65 ms)
PlatformBuildTypePageMetricMean (ms)Min (ms)Max (ms)Std Dev (ms)P 75 (ms)P 95 (ms)
ChromeBrowserifyHomeuiStartup1207109915426512451318
load105495312495710871172
domContentLoaded104893812445810821165
domInteractive16135151628
firstPaint75679125240710651151
backgroundConnect74274720
firstReactRender18153531824
getState1243471826
initialActions002001
loadScripts812706100255845916
setupStore75172710
WebpackHomeuiStartup22201756267618223562460
load17421320226116218361940
domContentLoaded17351311225716118281934
domInteractive171158111448
firstPaint192741661160211312
backgroundConnect3710278464261
firstReactRender17154367107278346
getState1344381534
initialActions318145
loadScripts17301308225516218231927
setupStore286391472753
FirefoxBrowserifyHomeuiStartup13401142161710714071574
load11981024147810612711403
domContentLoaded11981023147810612711402
domInteractive1033735339117165
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect201292102028
firstReactRender23185672549
getState9420821726
initialActions001001
loadScripts11811010145910512591365
setupStore5413167
WebpackHomeuiStartup16151431215814217021874
load13871226190413114681636
domContentLoaded13871225190413114661636
domInteractive79331321888117
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect23156772437
firstReactRender36296053848
getState1154881031
initialActions102111
loadScripts13661204183712814501614
setupStore85364912
cc: @HowardBraham
Benchmark value 20 exceeds gate value 18 for chrome browserify home p95 backgroundConnect
Benchmark value 2220 exceeds gate value 2192 for chrome webpack home mean uiStartup
Benchmark value 1742 exceeds gate value 1711 for chrome webpack home mean load
Benchmark value 1736 exceeds gate value 1704 for chrome webpack home mean domContentLoaded
Benchmark value 1731 exceeds gate value 1699 for chrome webpack home mean loadScripts
Benchmark value 2461 exceeds gate value 2454 for chrome webpack home p95 uiStartup
Benchmark value 26 exceeds gate value 24 for firefox browserify home p95 getState
Benchmark value 1616 exceeds gate value 1615 for firefox webpack home mean uiStartup
Benchmark value 1388 exceeds gate value 1380 for firefox webpack home mean load
Benchmark value 1388 exceeds gate value 1380 for firefox webpack home mean domContentLoaded
Benchmark value 1367 exceeds gate value 1360 for firefox webpack home mean loadScripts
Sum of mean exceeds: 147ms | Sum of p95 exceeds: 11ms
Sum of all benchmark exceeds: 158ms

Bundle size diffs
  • background: 0 Bytes (0%)
  • ui: 0 Bytes (0%)
  • common: 0 Bytes (0%)

@HowardBraham HowardBraham added the no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed label May 5, 2025
@itsyoboieltr itsyoboieltr moved this from Needs more work from the author to Needs dev review in PR review queue May 5, 2025
@itsyoboieltr itsyoboieltr requested review from davidmurdoch and HowardBraham and removed request for davidmurdoch May 5, 2025 22:14
HowardBraham
HowardBraham previously approved these changes May 6, 2025
@github-project-automation github-project-automation bot moved this from Needs dev review to Review finalised - Ready to be merged in PR review queue May 6, 2025
@HowardBraham HowardBraham enabled auto-merge May 6, 2025 17:17
@Gudahtt Gudahtt disabled auto-merge May 6, 2025 17:42
@metamaskbot
Copy link
Collaborator

Builds ready [8be50c6]
UI Startup Metrics (1193 ± 56 ms)
PlatformBuildTypePageMetricMean (ms)Min (ms)Max (ms)Std Dev (ms)P 75 (ms)P 95 (ms)
ChromeBrowserifyHomeuiStartup1193107713655612271300
load103990411875110671128
domContentLoaded103389511815110621124
domInteractive17136271728
firstPaint68584117941010411114
backgroundConnect73314711
firstReactRender231579102439
getState1363571728
initialActions003001
loadScripts79266192449824876
setupStore84303813
WebpackHomeuiStartup22531711264519423972506
load17481285212615718462051
domContentLoaded17401282211715418411958
domInteractive161169111446
firstPaint1706751577201304
backgroundConnect3210172243877
firstReactRender21455391111315359
getState1544291935
initialActions327146
loadScripts17351280209215118391933
setupStore216171182540
FirefoxBrowserifyHomeuiStartup13641200171010414391593
load1216105315519912891411
domContentLoaded1216105215519912891411
domInteractive1013821029115155
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect2214128142038
firstReactRender23205352330
getState84344811
initialActions001001
loadScripts1196104015309912691391
setupStore8418318612
WebpackHomeuiStartup15321333209512916061801
load13171173189112013711552
domContentLoaded13171173189012013701552
domInteractive79383363083114
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect22157892337
firstReactRender35285753645
getState105558931
initialActions002111
loadScripts12981158181311813551526
setupStore85263815
cc: @HowardBraham
Benchmark value 2253 exceeds gate value 2192 for chrome webpack home mean uiStartup
Benchmark value 1749 exceeds gate value 1711 for chrome webpack home mean load
Benchmark value 1740 exceeds gate value 1704 for chrome webpack home mean domContentLoaded
Benchmark value 1735 exceeds gate value 1699 for chrome webpack home mean loadScripts
Benchmark value 2506 exceeds gate value 2454 for chrome webpack home p95 uiStartup
Benchmark value 2051 exceeds gate value 2030 for chrome webpack home p95 load
Sum of mean exceeds: 171ms | Sum of p95 exceeds: 73ms
Sum of all benchmark exceeds: 244ms

Bundle size diffs [🚀 Bundle size reduced!]
  • background: -6.85 KiB (-0.13%)
  • ui: 57.51 KiB (0.82%)
  • common: -5.94 KiB (-0.06%)

@davidmurdoch davidmurdoch merged commit 24c39b1 into main May 6, 2025
162 checks passed
@github-project-automation github-project-automation bot moved this from Review finalised - Ready to be merged to Merged, Closed or Archived in PR review queue May 6, 2025
@davidmurdoch davidmurdoch deleted the feat/better-test-reports branch May 6, 2025 18:02
@github-actions github-actions bot locked and limited conversation to collaborators May 6, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed team-extension-platform Extension Platform team
Projects
Archived in project
5 participants