Merge FV3 changes for water vapor photochemistry diagnostics #2707

DWesl · 2025-04-18T17:08:37Z

Commit Queue Requirements:

Fill out all sections of this template.
All sub component pull requests have been reviewed by their code managers.
Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules
Commit 'test_changes.list' from previous step

Description:

Add water vapor photochemistry diagnostics, with changes in FV3 and CCPP-physics to support this and add the results to dtend_qv_phys.

Add parameters to water vapor photochemistry scheme to return the terms of the water vapor photochemistry scheme, then stick the descriptions in the metadata files next to the ozone photochemistry diagnostics, and add the variable declarations to allocate storage for the diagnostics.

Commit Message:

* UFSWM - Add water vapor photochemistry diagnostics
  * FV3 - Add water vapor photochemistry diagnostics
    * ccpp-physics - Add water vapor photochemistry diagnostics

Priority:

Normal

Git Tracking

UFSWM:

None

Sub component Pull Requests:

FV3: Add qv photochemistry diagnostics NOAA-EMC/fv3atm#949
- ccpp-physics: ENH: Add qv photochemistry diagnostics ccpp-physics#265

UFSWM Blocking Dependencies:

None

Changes

Regression Test Changes (Please commit test_changes.list):

PR Updates/Changes Baselines.

Input data Changes:

None.

Library Changes/Upgrades:

No Updates

Testing Log:

DWesl · 2025-04-22T13:19:05Z

I worked out how to run the regression tests. How do I get more information from the failures than "this broke stuff"?

DWesl · 2025-04-22T14:13:10Z

Found the logs, and the results with more detail than the logs. There are changes to precipitation and radiation, mostly,

grantfirl · 2025-04-22T14:16:02Z

Found the logs, and the results with more detail than the logs. There are changes to precipitation and radiation, mostly,

Ya, I was going to suggest looking in tests/logs/RegressionTests_your_machine.log for a bit more info, then to look in the run directory at logs for individual tests. Looking at the your code changes, I would not expect any results to change unless the new diagnostics that you're adding are added to the output files, so it's somewhat surprising that you're seeing different results.

grantfirl · 2025-04-22T14:21:43Z

@DWesl Are you sure that you've updated your PR branches to the latest develop branches? The fact that the PR says that there are conflicts that must be resolved makes me think that perhaps you're not testing with your changes on top of the latest code (which the RT results are based on).

…-photochemistry-diagnostics

DWesl · 2025-04-22T14:50:37Z

@DWesl Are you sure that you've updated your PR branches to the latest develop branches? The fact that the PR says that there are conflicts that must be resolved makes me think that perhaps you're not testing with your changes on top of the latest code (which the RT results are based on).

Just finished updating that, and re-running tests. The FV3 update looks like just my changes now, which is nice.

DWesl · 2025-04-23T13:59:52Z

Ya, I was going to suggest looking in tests/logs/RegressionTests_your_machine.log for a bit more info, then to look in the run directory at logs for individual tests.

tests/logs/RegressionTests_hera.log seems to be missing, after running the regression tests for five hours yesterday. These tests show neither errors nor failures:

logs/log_hera/rt_cpld_control_gefs_intel.log
logs/log_hera/rt_cpld_control_gfsv17_iau_intel.log
logs/log_hera/rt_cpld_control_gfsv17_intel.log
logs/log_hera/rt_cpld_control_p8_mixedmode_intel.log
logs/log_hera/rt_cpld_control_sfs_intel.log
logs/log_hera/rt_cpld_dcp_gefs_intel.log
logs/log_hera/rt_cpld_mpi_gfsv17_intel.log
logs/log_hera/rt_cpld_restart_gefs_intel.log
logs/log_hera/rt_cpld_restart_gfsv17_intel.log

and these directories have {atm,sfc}f???.tile?.nc_nccmp.log files, but those files are empty:

run_dir/cpld_control_gefs_intel
run_dir/cpld_control_gfsv17_iau_intel
run_dir/cpld_control_gfsv17_intel
run_dir/cpld_control_p8_mixedmode_intel
run_dir/cpld_control_sfs_intel
run_dir/cpld_dcp_gefs_intel
run_dir/cpld_mpi_gfsv17_intel
run_dir/cpld_restart_gefs_intel
run_dir/cpld_restart_gfsv17_intel

the cpld_debug_gfsv17_intel test case had only gotten three hours in before I logged out for the day.

grantfirl · 2025-04-30T16:42:24Z

@DWesl I'm running the full rt.conf on Hera for this PR. If you add my user (@grantfirl) as a collaborator on your branch, I can push the logs. If not, I can paste them in a comment.

DWesl · 2025-04-30T16:55:10Z

I think I just added you as a collaborator on my fork.

Is rt.sh meant to be run as a batch job? It took long enough to run when I tried that I left before it finished

DeniseWorthen · 2025-04-30T17:02:53Z

@DWesl Did you run it using ecflow? rt.sh -e ..... You can also run via rocoto, see rt.sh -r. That will run multiple jobs at one time.

rt.sh -help

Usage: ./rt.sh -a <account> | -b <file> | -c | -d | -e | -h | -k | -l <file> | -m | -n <name> | -o | -r | -v | -w

  -a  <account> to use on for HPC queue
  -b  create new baselines only for tests listed in <file>
  -c  create new baseline results
  -d  delete run directories that are not used by other tests
  -e  use ecFlow workflow manager
  -h  display this help
  -k  keep run directory after rt.sh is completed
  -l  runs test specified in <file>
  -m  compare against new baseline results
  -n  run single test <name>
  -o  compile only, skip tests
  -r  use Rocoto workflow manager
  -v  verbose output
  -w  for weekly_test, skip comparing baseline results

DWesl · 2025-04-30T17:06:53Z

I specified -a, but neither -e nor -r. I suppose that would do it.

Should I look into adding a note to rt.sh to remind users passing -a that they probably want to pass either -e or -r as well?

grantfirl · 2025-04-30T17:09:05Z

@DWesl Agree with @DeniseWorthen's response. I use -e for ecflow. Here is the command that I use to run the entire rt.conf in the background:

./rt.sh -e -k -l rt.conf > rt_full.log 2>&1 &

I do this within a screen session so that I can leave it and come back later.

grantfirl · 2025-04-30T17:10:34Z

@DWesl Also, there is no need for you to re-run the tests if mine complete successfully.

DWesl · 2025-04-30T17:48:59Z

@DWesl Also, there is no need for you to re-run the tests if mine complete successfully.

Yeah, but there's a decent chance I'll wind up with a third PR at some point, and knowing how to run the tests will help with that

./rt.sh -e -k -l rt.conf > rt_full.log 2>&1 &

I do this within a screen session so that I can leave it and come back later.

I'd likely do nohup ./rt.sh -e -k -l rt.conf > rt_full.log 2>&1 & instead of using screen if that's the only command, since I don't currently take note of which node I'm on

grantfirl · 2025-04-30T19:16:41Z

@DWesl I'm running into runtime failures for the following tests:
control_diag_debug intel
rap_diag_debug intel
control_wam_debug intel
control_diag_debug gnu
rap_diag_debug gnu

There seems to be a division by zero which is caught when the debug compilation flags are being used. If you have access to Hera, my RT run directories are here: /scratch1/BMC/gmtb/Grant.Firl/stmp2/Grant.Firl/FV3_RT/rt_2254886

DWesl · 2025-04-30T20:38:24Z

As mentioned on the other PR, I'm running regression tests on a variant that should avoid this. The run directory is /scratch1/NCEPDEV/global/Daniel.Wesloh/global-workflow-20250418/sorc/ufs_model.fd/tests on Hera.

DWesl · 2025-05-01T13:21:08Z

It's telling me the GRBPRS.F024 files don't match in a number of places. Working out how to get more detailed information.

DWesl · 2025-05-01T14:15:08Z

Ah. Including my UPP updates for stratopause humidity in the tree will cause differences. I undid those changes and started the tests again.

EDIT: then killed the old test process, since it'll note stratopause humidity changes in every test

Four failures, all because the tests couldn't find the files to compare to.

DWesl · 2025-05-01T16:32:48Z

The debug tests failed, indicating dtend_qv_phys changed. This would be because that variable now includes the tendencies from photochemistry. Is there a quick way to check that is the only thing that changed?

DWesl · 2025-05-01T16:50:27Z

The four failures are {control,rap}_diag_debug_{intel,gnu}, and each of the two failures in those cases only changes dtend_qv_phys, on the order of $10^{-13}$ (kg/kg/s?):

$ bash <(grep -B5  -e 'NOT IDENTICAL' logs/log_hera/run_{control,rap}_diag_debug_{intel,gnu}.log | grep nccmp | cut -d+ -f2)
Variable      Group   Count         Sum      AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210240 1.45142e-07 1.84607e-07 -6.10962e-13 2.31178e-12 2.92274e-12 4.52122e-14 1.88223e-13
Variable      Group   Count         Sum     AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210240 1.44722e-07 1.8447e-07 -6.13018e-13 2.30345e-12 2.91647e-12 4.50814e-14 1.88045e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210240 1.45142e-07 1.84607e-07 -6.10962e-13 2.31178e-12 2.92274e-12 4.52122e-14 1.88223e-13
Variable      Group   Count         Sum     AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210240 1.44722e-07 1.8447e-07 -6.13017e-13 2.30343e-12 2.91645e-12 4.50813e-14 1.88045e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210232 1.45916e-07 1.84811e-07 -6.12138e-13 2.31538e-12 2.92752e-12 4.54535e-14 1.88498e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range       Mean      StdDev
dtend_qv_phys /     3210240 1.45751e-07 1.84795e-07 -6.14611e-13 2.30602e-12 2.92064e-12 4.5402e-14 1.88471e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210234 1.45916e-07 1.84811e-07 -6.12138e-13 2.31538e-12 2.92752e-12 4.54534e-14 1.88498e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range       Mean      StdDev
dtend_qv_phys /     3210240 1.45751e-07 1.84795e-07 -6.14611e-13 2.30602e-12 2.92063e-12 4.5402e-14 1.88471e-13

I think this would be due to the inclusion of dtend_qv_photochem in this value now.

RegressionTests_hera.log suggests ./rt.sh -c -b -test_changes.list to create new baselines. Should I do that, do something with the result, and run the tests again?

…-photochemistry-diagnostics

DWesl · 2025-05-02T02:02:21Z

And with the new changes, still two failed comparisons for each of four failed test cases:

$ bash <(grep -B5  -e 'NOT IDENTICAL' logs/log_hera/run_{control,rap}_diag_debug_{intel,gnu}.log | grep nccmp | cut -d+ -f2)
Variable      Group   Count         Sum      AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210240 1.45142e-07 1.84607e-07 -6.10962e-13 2.31178e-12 2.92274e-12 4.52122e-14 1.88223e-13
Variable      Group   Count         Sum     AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210240 1.44722e-07 1.8447e-07 -6.13018e-13 2.30345e-12 2.91647e-12 4.50814e-14 1.88045e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210240 1.45142e-07 1.84607e-07 -6.10962e-13 2.31178e-12 2.92274e-12 4.52122e-14 1.88223e-13
Variable      Group   Count         Sum     AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210240 1.44722e-07 1.8447e-07 -6.13017e-13 2.30343e-12 2.91645e-12 4.50813e-14 1.88045e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210232 1.45916e-07 1.84811e-07 -6.12138e-13 2.31538e-12 2.92752e-12 4.54535e-14 1.88498e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range       Mean      StdDev
dtend_qv_phys /     3210240 1.45751e-07 1.84795e-07 -6.14611e-13 2.30602e-12 2.92064e-12 4.5402e-14 1.88471e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210234 1.45916e-07 1.84811e-07 -6.12138e-13 2.31538e-12 2.92752e-12 4.54534e-14 1.88498e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range       Mean      StdDev
dtend_qv_phys /     3210240 1.45751e-07 1.84795e-07 -6.14611e-13 2.30602e-12 2.92063e-12 4.5402e-14 1.88471e-13

Still of order $10^{-13}$ kg/kg/s, or $10^{-2}$ mg/kg/day

grantfirl · 2025-05-02T15:00:54Z

@DWesl Thanks for your testing. The result changes look reasonable given the code changes. Please mark the Regression Test Changes section in the description that it changes baselines. That should be all you need to do since you've pushed the test_changes.list and the RegressionTests_*.log.

grantfirl · 2025-05-02T15:01:58Z

@DWesl There is no reason to create baselines on your part unless you suspect reproducibility issues (which you shouldn't for these changes). When a PR is marked that it changes baselines, the UFS code managers will create the new baselines during the merge process.

DWesl · 2025-05-02T15:56:18Z

Edited top comment to note change in baselines.

…-photochemistry-diagnostics

$ bash <(grep -B5 -e 'NOT IDENTICAL' logs/log_hera/run_{control,rap}_diag_debug_{intel,gnu}.log | grep nccmp | cut -d+ -f2) Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210240 1.45142e-07 1.84607e-07 -6.10962e-13 2.31178e-12 2.92274e-12 4.52122e-14 1.88223e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210240 1.44722e-07 1.8447e-07 -6.13018e-13 2.30345e-12 2.91647e-12 4.50814e-14 1.88045e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210240 1.45142e-07 1.84607e-07 -6.10962e-13 2.31178e-12 2.92274e-12 4.52122e-14 1.88223e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210240 1.44722e-07 1.8447e-07 -6.13017e-13 2.30343e-12 2.91645e-12 4.50813e-14 1.88045e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210232 1.45916e-07 1.84811e-07 -6.12138e-13 2.31538e-12 2.92752e-12 4.54535e-14 1.88498e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210240 1.45751e-07 1.84795e-07 -6.14611e-13 2.30602e-12 2.92064e-12 4.5402e-14 1.88471e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210234 1.45916e-07 1.84811e-07 -6.12138e-13 2.31538e-12 2.92752e-12 4.54534e-14 1.88498e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210240 1.45751e-07 1.84795e-07 -6.14611e-13 2.30602e-12 2.92063e-12 4.5402e-14 1.88471e-13

…-photochemistry-diagnostics

In particular, allow the ozone scheme to have two extra sets of parameters during the ozone hole without crashing the model.

…hemistry-index-fix

With a separate index for each photochemistry scheme, we can move the index-finding calls in the conditional for the photochemistry scheme.

…hemistry-index-fix

…-photochemistry-diagnostics

jkbk2004 · 2025-05-27T19:54:41Z

@DWesl can you sync up branches?

DWesl · 2025-05-27T20:00:27Z

I thought I just did: the "merge status" widget at the end of the commit thread doesn't mention conflicting changes to FV3, RegressionTests_hera.log, and test_changes.list the way it did an hour or two ago. The regression test re-run I started after doing that still has an hour or two to go, but I'll run git push to be sure the merged version made it here.

DWesl · 2025-05-28T13:19:42Z

Tests run with upstream updates. Changes still small:

$ bash <(grep -B5  -e 'NOT IDENTICAL' logs/log_hera/run_{control,rap}_diag_debug_{intel,gnu}.log | grep nccmp | cut -d+ -f2)
Variable      Group   Count         Sum      AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210240 1.45142e-07 1.84607e-07 -6.10962e-13 2.31178e-12 2.92274e-12 4.52122e-14 1.88223e-13
Variable      Group   Count         Sum     AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210240 1.44722e-07 1.8447e-07 -6.13018e-13 2.30345e-12 2.91647e-12 4.50814e-14 1.88045e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210240 1.45142e-07 1.84607e-07 -6.10962e-13 2.31178e-12 2.92274e-12 4.52122e-14 1.88223e-13
Variable      Group   Count         Sum     AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210240 1.44722e-07 1.8447e-07 -6.13017e-13 2.30343e-12 2.91645e-12 4.50813e-14 1.88045e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210232 1.45916e-07 1.84811e-07 -6.12138e-13 2.31538e-12 2.92752e-12 4.54535e-14 1.88498e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range       Mean      StdDev
dtend_qv_phys /     3210240 1.45751e-07 1.84795e-07 -6.14611e-13 2.30602e-12 2.92064e-12 4.5402e-14 1.88471e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range        Mean      StdDev
dtend_qv_phys /     3210234 1.45916e-07 1.84811e-07 -6.12138e-13 2.31538e-12 2.92752e-12 4.54534e-14 1.88498e-13
Variable      Group   Count         Sum      AbsSum          Min         Max       Range       Mean      StdDev
dtend_qv_phys /     3210240 1.45751e-07 1.84795e-07 -6.14611e-13 2.30602e-12 2.92063e-12 4.5402e-14 1.88471e-13

…dition. That was ufs-community/ccpp-physics#286 and NOAA-EMC/fv3atm#971, with corresponding PR ufs-community#2750 here.

…agnostics

Mismatches: Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210240 1.45142e-07 1.84607e-07 -6.10962e-13 2.31178e-12 2.92274e-12 4.52122e-14 1.88223e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210240 1.44722e-07 1.8447e-07 -6.13018e-13 2.30345e-12 2.91647e-12 4.50814e-14 1.88045e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210240 1.45142e-07 1.84607e-07 -6.10962e-13 2.31178e-12 2.92274e-12 4.52122e-14 1.88223e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210240 1.44722e-07 1.8447e-07 -6.13017e-13 2.30343e-12 2.91645e-12 4.50813e-14 1.88045e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210232 1.45916e-07 1.84811e-07 -6.12138e-13 2.31538e-12 2.92752e-12 4.54535e-14 1.88498e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210240 1.45751e-07 1.84795e-07 -6.14611e-13 2.30602e-12 2.92064e-12 4.5402e-14 1.88471e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210234 1.45916e-07 1.84811e-07 -6.12138e-13 2.31538e-12 2.92752e-12 4.54534e-14 1.88498e-13 Variable Group Count Sum AbsSum Min Max Range Mean StdDev dtend_qv_phys / 3210240 1.45751e-07 1.84795e-07 -6.14611e-13 2.30602e-12 2.92063e-12 4.5402e-14 1.88471e-13

Merge FV3 changes for water vapor photochemistry diagnostics

2844c86

DWesl mentioned this pull request Apr 18, 2025

Add qv photochemistry diagnostics NOAA-EMC/fv3atm#949

Open

Merge branch 'develop' of ufs-community/ufs-weather-model into add-qv…

8380391

…-photochemistry-diagnostics

Merge upstream into FV3

b7849ae

BUG: Avoid divide-by-zero in CCPP-Physics ozone diagnostic calculations

a98519e

TST: Commit tests/logs/RegressionTests_hera.log

7a750e7

Four failures, all because the tests couldn't find the files to compare to.

DWesl added 3 commits May 1, 2025 19:42

Generate new baselines.

ba66d92

Merge branch 'develop' of ufs-community/ufs-weather-model into add-qv…

2054cbb

…-photochemistry-diagnostics

Update RegressionTests_hera.log and test_changes.list

a5188bf

DWesl added 13 commits May 7, 2025 19:12

Merge branch 'develop' of ufs-community/ufs-weather-model into add-qv…

55cf998

…-photochemistry-diagnostics

Merge branch 'develop' of ufs-community/ufs-weather-model into add-qv…

c853b1f

…-photochemistry-diagnostics

Merge branch 'develop' of ufs-community/ufs-weather-model into add-qv…

da6f017

…-photochemistry-diagnostics

BUG: Allow photochemistry schemes to have parameters at different times.

210bcc4

In particular, allow the ozone scheme to have two extra sets of parameters during the ozone hole without crashing the model.

Merge branch 'develop' of ufs-community/ufs-weather-model into photoc…

59e042a

…hemistry-index-fix

STY: Move the find_photochemistry_index calls inside the conditionals.

81fba08

With a separate index for each photochemistry scheme, we can move the index-finding calls in the conditional for the photochemistry scheme.

Update test logs with results from this morning's run

eedcbb3

TST: Update test log for updates from GitHub.

e5a9a27

Merge upstream FV3 changes into photochemistry-index-fix

d211f8c

TST: Update test results

34f93b3

Merge branch 'develop' of ufs-community/ufs-weather-model into photoc…

b0121f7

…hemistry-index-fix

Merge branch 'develop' of ufs-community/ufs-weather-model into add-qv…

16b8c5c

…-photochemistry-diagnostics

grantfirl mentioned this pull request May 27, 2025

Bug fix for surface roughness calculation over lakes and sea ice points when wave model is turned on #2739

Open

13 tasks

Add test results from newest upstream updates.

f97d206

Merge photochemistry index fix into the photochemistry diagnostics ad…

e6d48ab

…dition. That was ufs-community/ccpp-physics#286 and NOAA-EMC/fv3atm#971, with corresponding PR ufs-community#2750 here.

DWesl mentioned this pull request May 28, 2025

Allow photochemistry schemes to have parameters at different times. #2750

Closed

13 tasks

DWesl added 2 commits May 28, 2025 16:08

Merge branch 'photochemistry-index-fix' into add-qv-photochemistry-di…

08f8306

…agnostics

Merge FV3 changes for water vapor photochemistry diagnostics #2707

Are you sure you want to change the base?

Merge FV3 changes for water vapor photochemistry diagnostics #2707

Uh oh!

Conversation

DWesl commented Apr 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Commit Queue Requirements:

Description:

Commit Message:

Priority:

Git Tracking

UFSWM:

Sub component Pull Requests:

UFSWM Blocking Dependencies:

Changes

Regression Test Changes (Please commit test_changes.list):

Input data Changes:

Library Changes/Upgrades:

Testing Log:

Uh oh!

DWesl commented Apr 22, 2025

Uh oh!

DWesl commented Apr 22, 2025

Uh oh!

grantfirl commented Apr 22, 2025

Uh oh!

grantfirl commented Apr 22, 2025

Uh oh!

DWesl commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DWesl commented Apr 23, 2025

Uh oh!

grantfirl commented Apr 30, 2025

Uh oh!

DWesl commented Apr 30, 2025

Uh oh!

DeniseWorthen commented Apr 30, 2025

Uh oh!

DWesl commented Apr 30, 2025

Uh oh!

grantfirl commented Apr 30, 2025

Uh oh!

grantfirl commented Apr 30, 2025

Uh oh!

DWesl commented Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

grantfirl commented Apr 30, 2025

Uh oh!

DWesl commented Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DWesl commented May 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DWesl commented May 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DWesl commented May 1, 2025

Uh oh!

DWesl commented May 1, 2025

Uh oh!

DWesl commented May 2, 2025

Uh oh!

grantfirl commented May 2, 2025

Uh oh!

grantfirl commented May 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DWesl commented May 2, 2025

Uh oh!

jkbk2004 commented May 27, 2025

Uh oh!

DWesl commented May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DWesl commented May 28, 2025

Uh oh!

DWesl commented Apr 18, 2025 •

edited

Loading

DWesl commented Apr 22, 2025 •

edited

Loading

DWesl commented Apr 30, 2025 •

edited

Loading

DWesl commented Apr 30, 2025 •

edited

Loading

DWesl commented May 1, 2025 •

edited

Loading

DWesl commented May 1, 2025 •

edited

Loading

grantfirl commented May 2, 2025 •

edited

Loading

DWesl commented May 27, 2025 •

edited

Loading