Adding additional functionality namelists to ww3_outp.

[EJRainville-NOAA]

Pull Request Summary

This PR adds additional functionality to the ww3_outp program to allow for input namelist files

Description

This PR addresses issue #619 and adds a new feature by adding the option to use a namelist file as an input to the ww3_outp program. The program ww3_outp.F90 was modified, an additional module file is added (w3nmloutpmd), and it new namelist files are added to the nml directory and to the ufs1.1 test case. Answer changes are not expected in this PR.

Issue(s) addressed

fixes #619

Commit Message

Added namelist input file functionality feature for the ww3_outp program.

Check list

• Branch is up to date with the authoritative repository (NOAA-EMC) develop branch.
• Checked the checklist for a developer submitting to develop.
• If a version number update is required, checked the updating version number checklist.
• If a new feature was added, a regression test for testing the new feature is added.

Testing

• How were these changes tested?
  This PR is tested with the full regression test matrix on Orion.

• Are the changes covered by regression tests? (If not, why? Do new tests need to be added?)
  The changes are covered in the ufs1.1 work_unstr_a_nml test case that was fixed as a previous PR. No additional new tests are needed to be added to cover the changes in this update.

• Have the matrix regression tests been run (if yes, please note HPC and compiler)?
  Yes, the regression tests have been run on Orion with intel compilers.

• Please indicate the expected changes in the regression test output, (Note the list of known non-identical tests.)
  No changes are expected to output but additional nml logs are added when using the new nml feature following the structure of previous nml feature additions.

• Please provide the summary output of matrix.comp (matrix.Diff.txt, matrixCompFull.txt and matrixCompSummary.txt):
  matrixDiff.txt
  matrixCompFull.txt
  matrixCompSummary.txt

[mickaelaccensi]

I would highly recommend to create a bash script to automatically convert .inp files to .nml for all the regtests and keep them all up-to-date when a modification is done in the inp or nml file.
See examples in WW3/model/tools/bash/
Once it's done, you can test it on a inp file and then run it on all inp files of regtests directory with loop_nml.sh

[EJRainville-NOAA]

@mickaelaccensi Thanks for the suggestion! I agree that automating the conversion would be useful for keeping the .inp and .nml files synced across the regtests. For this PR, I’ve focused on just adding the nml functionality for the ww3_outp program. I’ll take a look at the existing scripts in WW3/model/tools/bash/ and add the conversion as a follow-up once this is in place.

[mingchen-NOAA]

@EJRainville-NOAA The matrix tests on Ursa GNU fails. Multiple errors were reported in w3nmloutpmd.F90.
You can find my tests in Ursa /scratch4/NCEPDEV/marine/Ming.Chen/ww3/ursa/ww3_pr/pr1629_gnu
You can find error reports in build.log for each case. For example: /scratch4/NCEPDEV/marine/Ming.Chen/ww3/ursa/ww3_pr/pr1629_gnu/regtests/ww3_tp2.2/work_PR1_MPI_BIN2NC/build.log

[EJRainville-NOAA]

@mingchen-NOAA Thanks for reviewing this! I will look at your build.log file to see what the issue with the GNU compilers are and start to fix those issues.

[mingchen-NOAA]

@EJRainville-NOAA I ran your branch using Ursa GNU matrix08 and was able to reproduce the error.

free(): invalid pointer
Program received signal SIGABRT: Process abort signal.

I then ran the same test using the develop branch with Ursa GNU (/scratch4/NCEPDEV/marine/Ming.Chen/ww3/ursa/ww3_pr/ww3_baseline_gnu/regtests), and no error was observed. So it looks like the error is likely related to the changes in this PR.

Based on the error message and some additional testing, the most likely cause appears to be an OpenMP race condition or thread-safety issue. I added export OMP_NUM_THREADS=1 to matrix08, and the case ran successfully.

I will continue investigating to identify which part of changes is causing the issue.

[mingchen-NOAA]

Info:
Failure case: mww3_test_05/work_ST6_PR3_UQ_OMP in matrix08 using the GNU compiler.

./bin/run_cmake_test -b slurm -o all -S -T -s ST6_PR3_UQ_OMP  -w work_ST6_PR3_UQ_OMP  -m grdset_a -f -O -n 24 ../model mww3_test_05

Failure diagnosis: An intermittent OpenMPI race condition or thread-safety issue was identified.

Investigation update:
Ran the develop branch 10 times using the GNU compiler — all runs completed successfully.
Ran this PR using the GNU compiler — the 4th run failed.
Ran this PR 10 times with w3iopomd.F90 reverted to the original version from develop — all runs completed successfully.

Based on these results, the issue is most likely related to the changes in w3iopomd.F90.

Suggest next steps:
Remove the non-essential changes in w3iopomd.F90 and run the test 10 times again.
If all 10 runs are successful, run the full GNU regression test and review the results.

[EJRainville-NOAA]

Info: Failure case: mww3_test_05/work_ST6_PR3_UQ_OMP in matrix08 using the GNU compiler.

./bin/run_cmake_test -b slurm -o all -S -T -s ST6_PR3_UQ_OMP  -w work_ST6_PR3_UQ_OMP  -m grdset_a -f -O -n 24 ../model mww3_test_05

Failure diagnosis: An intermittent OpenMPI race condition or thread-safety issue was identified.

Investigation update: Ran the develop branch 10 times using the GNU compiler — all runs completed successfully. Ran this PR using the GNU compiler — the 4th run failed. Ran this PR 10 times with w3iopomd.F90 reverted to the original version from develop — all runs completed successfully.

Based on these results, the issue is most likely related to the changes in w3iopomd.F90.

Suggest next steps: Remove the non-essential changes in w3iopomd.F90 and run the test 10 times again. If all 10 runs are successful, run the full GNU regression test and review the results.

@mingchen-NOAA I reverted the changes made in w3iopomd.F90 and ran the test case again 10 times and all tests ran without errors. I also removed the unused variables in w3_outp.F90. All regression tests pass now with the GNU compilers and I haven't seen evidence of any thread safety issues. Thanks for the help in debugging that!