[L1T] Restructure handling of Calorimeter input to Correlator Layer-1 + new TkElectrons in barrel (Phase-2) #48002

cerminar · 2025-05-02T16:05:20Z

PR description:

This PR rewrites the Correlator Layer-1 emulator code handling the Calorimeter input objects, enabling more realistic treatment of barrel and endcap data.

In particular:

drop the conversion of the input calorimeter objects to PFCluster, moving all needed manipulations (deserialization, calibrations and ID models to the Layer-1 emulator.
- "decoded" calorimeter objects are now built starting from digitized words via unpackers according to most up-to-date interface documents
write PFCluster collections directly from the CTL1 emulator AFTER the decoding storing bit-wise accurate information across the whole calorimenter. This allows acessing data in the final precision for performance studies and ML model training
- modify the PFCluster data-format to store digitized objects in packed format. This allows retrieving quantized quantities in the ntuplization steps
adds more realistic HGCal decoding (update to more recent interface document)
- add emulation of new multiclass ID within the HGCal decoding step
PFCandidate ref to source becomes polymorphic, allowing uniform interface across barrel and endcap
GCT EM and Hadronic emulated objects are now decoded and calibrated within the emulator
updated emulator of the barrel "TDR" regionizer
flexible packed binary format of calorimeter and track objects across barrel and edncap to optimize Firmware resource usage

e-gamma L1T:

adds new composite ID model for the barrel cluster to track matching for building TkElectrons

PR validation:

We will soon post some comparison plots.

cmsbuild · 2025-05-02T16:05:56Z

cms-bot internal usage

cmsbuild · 2025-05-02T16:07:23Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48002/44675

ERROR: Unable to merge PR.

See log https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48002/44675/cms-checkout-topic.log

cmsbuild · 2025-05-05T16:09:34Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48002/44696

ERROR: Unable to merge PR.

See log https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48002/44696/cms-checkout-topic.log

cmsbuild · 2025-05-05T16:46:50Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48002/44698

Found files with invalid states:
- L1Trigger/Phase2L1ParticleFlow/data/compositeID.json:
  - Added: bba28e5
  - Deleted: 205ac24
- L1Trigger/Phase2L1ParticleFlow/interface/HGC3DClusterID.h:
  - Added: 2faf40a
  - Modified: bfd76ba, 205ac24
  - Deleted: 41b5f24
- L1Trigger/Phase2L1ParticleFlow/interface/conifer_cpp.h:
  - Added: 2faf40a
  - Deleted: 205ac24
- L1Trigger/Phase2L1ParticleFlow/data/endcap.json:
  - Added: bba28e5
  - Deleted: 205ac24
- L1Trigger/Phase2L1ParticleFlow/src/HGC3DClusterID.cc:
  - Added: 2faf40a
  - Modified: bfd76ba, 45a5419, 5ba67e6
  - Deleted: 41b5f24
- L1Trigger/Phase2L1ParticleFlow/data/my_prj.json:
  - Added: bba28e5
  - Deleted: 205ac24
There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/L1TParticleFlow/interface/datatypes.h modified in PR(s): [L1T] PR containing simulation and emulation related to jet mass reconstruction (Phase-2) #47792
- File DataFormats/L1TParticleFlow/src/classes_def.xml modified in PR(s): [L1T] PR containing simulation and emulation related to jet mass reconstruction (Phase-2) #47792
- File L1Trigger/Phase2L1ParticleFlow/test/make_l1ctLayer1_dumpFiles_fromRAW_cfg.py modified in PR(s): L1 tracking update #47067

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-checks:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48002/44698/code-checks.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48002/44698/code-checks.patch | patch -p1
You can also run scram build code-checks to apply code checks directly
code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48002/44698/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48002/44698/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2025-05-05T16:55:50Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48002/44699

Found files with invalid states:
- L1Trigger/Phase2L1ParticleFlow/data/compositeID.json:
  - Added: bba28e5
  - Deleted: 205ac24
- L1Trigger/Phase2L1ParticleFlow/interface/HGC3DClusterID.h:
  - Added: 2faf40a
  - Modified: bfd76ba, 205ac24
  - Deleted: 41b5f24
- L1Trigger/Phase2L1ParticleFlow/interface/conifer_cpp.h:
  - Added: 2faf40a
  - Deleted: 205ac24
- L1Trigger/Phase2L1ParticleFlow/data/endcap.json:
  - Added: bba28e5
  - Deleted: 205ac24
- L1Trigger/Phase2L1ParticleFlow/src/HGC3DClusterID.cc:
  - Added: 2faf40a
  - Modified: bfd76ba, 45a5419, 5ba67e6
  - Deleted: 41b5f24
- L1Trigger/Phase2L1ParticleFlow/data/my_prj.json:
  - Added: bba28e5
  - Deleted: 205ac24
There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/L1TParticleFlow/interface/datatypes.h modified in PR(s): [L1T] PR containing simulation and emulation related to jet mass reconstruction (Phase-2) #47792
- File DataFormats/L1TParticleFlow/src/classes_def.xml modified in PR(s): [L1T] PR containing simulation and emulation related to jet mass reconstruction (Phase-2) #47792
- File L1Trigger/Phase2L1ParticleFlow/test/make_l1ctLayer1_dumpFiles_fromRAW_cfg.py modified in PR(s): L1 tracking update #47067

cmsbuild · 2025-05-05T16:56:17Z

A new Pull Request was created by @cerminar for master.

It involves the following packages:

DataFormats/L1TCorrelator (upgrade, l1)
DataFormats/L1TParticleFlow (upgrade, l1)
L1Trigger/DemonstratorTools (l1)
L1Trigger/L1CaloTrigger (upgrade, l1)
L1Trigger/Phase2L1ParticleFlow (upgrade, l1)

@BenjaminRS, @Moanwar, @cmsbuild, @quinnanm, @srimanob, @subirsarkar can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @mmusich, @rovere this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2025-05-09T11:49:04Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48002/44769

Found files with invalid states:
- L1Trigger/Phase2L1ParticleFlow/data/compositeID.json:
  - Added: 00f4a29
  - Deleted: b970a90
- L1Trigger/Phase2L1ParticleFlow/interface/HGC3DClusterID.h:
  - Added: 18b68be
  - Modified: da6d5e8, b970a90
  - Deleted: 72b2a8d
- L1Trigger/Phase2L1ParticleFlow/interface/conifer_cpp.h:
  - Added: 18b68be
  - Deleted: b970a90
- L1Trigger/Phase2L1ParticleFlow/data/endcap.json:
  - Added: 00f4a29
  - Deleted: b970a90
- L1Trigger/Phase2L1ParticleFlow/src/HGC3DClusterID.cc:
  - Added: 18b68be
  - Modified: da6d5e8, 2f2e58c, d8ed941
  - Deleted: 72b2a8d
- L1Trigger/Phase2L1ParticleFlow/data/my_prj.json:
  - Added: 00f4a29
  - Deleted: b970a90
There are other open Pull requests which might conflict with changes you have proposed:
- File L1Trigger/Phase2L1ParticleFlow/test/make_l1ctLayer1_dumpFiles_fromRAW_cfg.py modified in PR(s): L1 tracking update #47067

cmsbuild · 2025-05-09T11:49:31Z

Pull request #48002 was updated. @BenjaminRS, @Moanwar, @cmsbuild, @quinnanm, @srimanob, @subirsarkar can you please check and sign again.

cmsbuild · 2025-05-13T16:39:03Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48002/44799

Found files with invalid states:
- L1Trigger/Phase2L1ParticleFlow/data/compositeID.json:
  - Added: 00f4a29
  - Deleted: b970a90
- L1Trigger/Phase2L1ParticleFlow/interface/HGC3DClusterID.h:
  - Added: 18b68be
  - Modified: da6d5e8, b970a90
  - Deleted: 72b2a8d
- L1Trigger/Phase2L1ParticleFlow/interface/conifer_cpp.h:
  - Added: 18b68be
  - Deleted: b970a90
- L1Trigger/Phase2L1ParticleFlow/data/endcap.json:
  - Added: 00f4a29
  - Deleted: b970a90
- L1Trigger/Phase2L1ParticleFlow/src/HGC3DClusterID.cc:
  - Added: 18b68be
  - Modified: da6d5e8, 2f2e58c, d8ed941
  - Deleted: 72b2a8d
- L1Trigger/Phase2L1ParticleFlow/data/my_prj.json:
  - Added: 00f4a29
  - Deleted: b970a90
There are other open Pull requests which might conflict with changes you have proposed:
- File L1Trigger/Phase2L1ParticleFlow/test/make_l1ctLayer1_dumpFiles_fromRAW_cfg.py modified in PR(s): L1 tracking update #47067

cmsbuild · 2025-05-13T16:39:29Z

Pull request #48002 was updated. @BenjaminRS, @Moanwar, @cmsbuild, @quinnanm, @srimanob, @subirsarkar can you please check and sign again.

cerminar · 2025-05-14T16:05:29Z

Following up on the menu MET (thanks @RobertJWard ) rates at the DPG we discovered a missing commit in the rebase to 151X. The issue is now addressed.
Validation plots are now available here comparing:
AR 2024 on 131X samples
151X vanilla on 140X samples
151X+PR on 140X samples
In particular:
MET rates: https://cerminar.web.cern.ch/cerminar/plots/152X_validation_140Xv0D0/jetmet_rate
e/g rates: https://cerminar.web.cern.ch/cerminar/plots/152X_validation_140Xv0D0/ctl2_eg_rate
e/g efficiencies: https://cerminar.web.cern.ch/cerminar/plots/152X_validation_140Xv0D0/ctl2_eg_eff
e/g scale and resolution: https://cerminar.web.cern.ch/cerminar/plots/152X_validation_140Xv0D0/ctl2_eg_reso

Short summary: no showstoppers.

More detailed summary: big increase in efficiency for most of e/g objects at the price of some increase in rate (some of the menu WPs will need to be changed). Some increase in rate of PUPPI-MET. Seems related to a change in scale of the barrel. This change is related to the new GCT emulated egamma clusters and it was already seen in previous studies:
https://cerminar.web.cern.ch/cerminar/plots/fp131Xv3-CMSSW14.0.X-GCTEmu/ctl2_eg_reso/

BenjaminRS · 2025-05-14T19:26:44Z

please test

cmsbuild · 2025-05-14T21:52:32Z

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-693570/46141/summary.html
COMMIT: b371880
CMSSW: CMSSW_15_1_X_2025-05-14-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48002/46141/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 6 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 64 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 4167455
DQMHistoTests: Total failures: 751
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 4166684
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 220 log files, 188 edm output root files, 51 DQM output files
TriggerResults: found differences in 8 / 49 workflows

cmsbuild added this to the CMSSW_15_1_X milestone May 2, 2025

cmsbuild added l1-pending pending-signatures tests-pending orp-pending upgrade-pending code-checks-pending changes-dataformats labels May 2, 2025

cmsbuild added code-checks-rejected and removed code-checks-pending labels May 2, 2025

cerminar force-pushed the 151X_calodev_v2 branch from 4cde659 to 810fc40 Compare May 5, 2025 16:08

cmsbuild added code-checks-pending and removed code-checks-rejected labels May 5, 2025

cmsbuild added code-checks-rejected and removed code-checks-pending labels May 5, 2025

cerminar force-pushed the 151X_calodev_v2 branch from 810fc40 to 5fc398b Compare May 5, 2025 16:44

cmsbuild added code-checks-pending and removed code-checks-rejected labels May 5, 2025

cmsbuild added code-checks-rejected code-checks-pending and removed code-checks-pending code-checks-rejected labels May 5, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels May 5, 2025

cerminar added 6 commits May 9, 2025 13:40

Introduce tight WP for HGC ID

c675a7c

New WPs for endcap Composite TkEle

fcfd62a

code checks

9edd940

Fix clang warnings

286ba3f

Fix bit ordering in HGC hwqual

aa24ec6

fix Phase-2 L1T configuration for GCT emulated objects

df7c30d

cerminar force-pushed the 151X_calodev_v2 branch from 006e2f0 to df7c30d Compare May 9, 2025 11:46

cmsbuild added tests-pending code-checks-pending and removed tests-approved code-checks-approved labels May 9, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels May 9, 2025

cmsbuild mentioned this pull request May 13, 2025

L1 tracking update #47067

Open

remove GCT duplicates when using Ideal regionizer (#23)

b371880

cmsbuild added code-checks-pending and removed code-checks-approved labels May 13, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels May 13, 2025

cmsbuild added tests-started and removed tests-pending labels May 14, 2025

cmsbuild added tests-approved and removed tests-started labels May 14, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[L1T] Restructure handling of Calorimeter input to Correlator Layer-1 + new TkElectrons in barrel (Phase-2) #48002

[L1T] Restructure handling of Calorimeter input to Correlator Layer-1 + new TkElectrons in barrel (Phase-2) #48002

cerminar commented May 2, 2025

cmsbuild commented May 2, 2025 •

edited

Loading

cmsbuild commented May 2, 2025

cmsbuild commented May 5, 2025

cmsbuild commented May 5, 2025

cmsbuild commented May 5, 2025

cmsbuild commented May 5, 2025

cmsbuild commented May 9, 2025

cmsbuild commented May 9, 2025

cmsbuild commented May 13, 2025

cmsbuild commented May 13, 2025

cerminar commented May 14, 2025

BenjaminRS commented May 14, 2025

cmsbuild commented May 14, 2025

[L1T] Restructure handling of Calorimeter input to Correlator Layer-1 + new TkElectrons in barrel (Phase-2) #48002

Are you sure you want to change the base?

[L1T] Restructure handling of Calorimeter input to Correlator Layer-1 + new TkElectrons in barrel (Phase-2) #48002

Conversation

cerminar commented May 2, 2025

PR description:

PR validation:

cmsbuild commented May 2, 2025 • edited Loading

cmsbuild commented May 2, 2025

cmsbuild commented May 5, 2025

cmsbuild commented May 5, 2025

cmsbuild commented May 5, 2025

cmsbuild commented May 5, 2025

cmsbuild commented May 9, 2025

cmsbuild commented May 9, 2025

cmsbuild commented May 13, 2025

cmsbuild commented May 13, 2025

cerminar commented May 14, 2025

BenjaminRS commented May 14, 2025

cmsbuild commented May 14, 2025

Comparison Summary

cmsbuild commented May 2, 2025 •

edited

Loading