Upgrade UFS-WM to spack-stack/1.9.1 #2619
Comments
|
@BrianCurtis-NOAA @AlexanderRichert-NOAA @jkbk2004 do any of you know the status of nco approvals for the ops. machines for the new lib versions to be introduced w/ spack-stack/1.9.1? |
Not offhand-- @Hang-Lei-NOAA can you comment? |
|
@Hang-Lei-NOAA might know |
|
I don’t know. As I know so far, there is no clear plan released to us.
…On Tue, Feb 25, 2025 at 3:31 PM Brian Curtis ***@***.***> wrote:
@Hang-Lei-NOAA <https://github.com/Hang-Lei-NOAA> might know
—
Reply to this email directly, view it on GitHub
<#2619 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKWSMFFQA4CNPT2GJY4RFDD2RTHKTAVCNFSM6AAAAABX3IUU46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMOBTGIYTOOJZGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
[image: BrianCurtis-NOAA]*BrianCurtis-NOAA* left a comment
(ufs-community/ufs-weather-model#2619)
<#2619 (comment)>
@Hang-Lei-NOAA <https://github.com/Hang-Lei-NOAA> might know
—
Reply to this email directly, view it on GitHub
<#2619 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKWSMFFQA4CNPT2GJY4RFDD2RTHKTAVCNFSM6AAAAABX3IUU46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMOBTGIYTOOJZGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@NickSzapiro-NOAA tested hera/gnu but some issues. @NickSzapiro-NOAA What is the error message? |
|
Like in #2263 |
|
I was also testing spack-stack/1.9.0 installation on Hera/intel last week (/contrib/spack-stack/spack-stack-1.9.0/envs/ue-oneapi-2024.2.1/install/modulefiles/Core) and few tests failed with this error: |
|
Which version of SCOTCH is being used? |
7.0.4 same as currently used |
|
Can you test with current spack-stack-1.9.1 branch: |
|
@RatkoVasic-NOAA Can you check the err log for your cpld_debug_gfsv17_intel test? |
|
@DeniseWorthen here: |
@RatkoVasic-NOAA I was curious if you have a branch available on your fork you used for this testing? |
|
Branch: ss-191 Awesome, thank you! |
|
@RatkoVasic-NOAA could I ask if you're using |
It uses modulefiles/ufs_hera.intel.lua modulefile. |
Perfect. Thank you for that info. |
|
Good news from @GeorgeVandenberghe-NOAA who tested scotch v7.0.7 on
|
|
@GeorgeVandenberghe-NOAA if this scotch fix will need to make it into WCOSS2, please get that process started ASAP. |
|
great news. @MatthewMasarik-NOAA @JessicaMeixner-NOAA @junwang-noaa will the plan be to go to scotch 7.07? we'd have to get this into spack-stack/1.9.1 to move this forward, so just want to see if there is a consensus. |
|
If we can confirm this fixes all the issues, then yes we'd want to move to Scotch 7.07. We'll need to get a request in for WCOSS2 ASAP. |
|
Just noting for awareness that requests for WCOSS2 library installations should be opened here: https://github.com/NOAA-EMC/WCOSS2-requests. |
I made several attempts running the RTs with George's scotch build, but am getting compile errors in the WW3 source. I'm working on troubleshooting. |
@rickgrubin-noaa Can you provide what |
|
@mathomp4 this is a bit long, but hopefully provides appropriate context and info. |
|
@RatkoVasic-NOAA If you can, can you try the MAPL branch: in your tests? That brings over some SSI related changes (essentially disabling it) from @atrayano ETA: Note: This cannot run with the hybrid MPI+OpenMP GOCART you all run. It should run as pure MPI |
|
@mathomp4 Great news! I ran one test on Gaea-C6 (cpld_control_p8_lnd_intel) using new MAPL branch (feature/v2.53.0-disable-ssi) it passed successfully. |
@RatkoVasic-NOAA Encouraging news! I'll talk with my team and see if we can make this a run-time switch (right now we sort of "bulk replaced" some code others might need). Once we can figure that out, we can look to release a MAPL v2.53.3. |
|
@RatkoVasic-NOAA I hate to ask it, but we might have another branch for you to try: This one is a one line change that if it works for you, would be much easier for us to put in a run-time option for your situation. |
|
@mathomp4 sure, I'll try it today |
|
CC @atrayano so he can monitor |
|
@mathomp4 , @atrayano , @ulmononian I'm going to test new MAPL (using @mathomp4 do you plan on tagging this version (feature/v2.53.0-hardcode-pet)? |
|
@RatkoVasic-NOAA We'll need to make it flexible with a run time option. This branch hardcoded something that is non-default (for us) so we need to make it an option that can be set in an RC file. We should have something tomorrow I hope and we can make a v2.53.3 tag |
|
@mathomp4 @atrayano @ulmononian |
this is great news. thanks @RatkoVasic-NOAA, @mathomp4, and @atrayano! |
|
@ulmononian @RatkoVasic-NOAA We are working on our mods now. The hope is we can get a tag made soon (later today we hope, Monday if we find weird issues) and submitted to spack. We are targeting that you activate the ESMF_PIN you'll want by add an option to |
|
@RatkoVasic-NOAA One more test please. Can you build this branch: and in your top-level configuration file, e.g. then add: That should run-time select the same option that worked in Once you confirm it works, I will issue a v2.53.3 for you. |
|
@mathomp4 - bad news. It doesn't work with And error message is: When I run with |
|
@RatkoVasic-NOAA Dang. Okay. That seems to say somehow the flag isn't being set. We'll take a look... |
|
@RatkoVasic-NOAA I just decided to put in some prints and, it should be working. Everything I see shows MAPL uses I can also confirm the flag does do something. I tried setting it to Can you now try: and see what happens. I just want to see what it prints out in your log. NOTE: This will be one print per process, so, well, it'll be voluminous! |
|
@RatkoVasic-NOAA We think we found the issue. The prints one will probably fail much the same. Hopefully soon I can push a fix once we test. ETA: Yes. We think we have it. @RatkoVasic-NOAA Please pull |
|
SHOOT this was my message from yesterday, but I didn't press button to submit :-) Sure, I'm going to try with new branch. |
|
@RatkoVasic-NOAA Yeah, that pretty much shows you were never calling the setter in your workflow (as |
|
@mathomp4 - good news. First test passed. I'm now running all tests that call MAPL. Fingers crossed. |
Huzzah! If all pass, we'll get out a v2.53.3 for you ASAP. |
|
It passed all tests on gaea-c6!!! |
|
@mathomp4 I can confirm, gaea-C6 passed all tests with |
|
@RatkoVasic-NOAA The tag has been issued: https://github.com/GEOS-ESM/MAPL/releases/tag/v2.53.3 and I have a PR into spack: We'll also get this into |
|
@mathomp4 thank you! |
Description
spack-stack/1.9.1 will be released in the near-term. the ufs-wm should upgrade to this stack.
Solution
make necessary changes to module files, test scripts, etc. to utilize the spack-stack/1.9.1 release.
@RatkoVasic-NOAA @rickgrubin-tomorrow @jkbk2004 @AlexanderRichert-NOAA @BrianCurtis-NOAA
The text was updated successfully, but these errors were encountered: