Use of climatology files when processing from wind information (in GridStat)

[robdarvell]

We have been looking into the use of climatology files within some of our ensemble work using grid_stat and we have come across a question, which we have been unable to answer ourselves, as we are not sure whether it is something we have done, or it is something from within the code base.

Here we are looking at u and v components (separate) and also the wind speed field (which has been created as a separate field rather than from the u and v components). These have been created on different model levels.
For the climatology files we have separate files for u, v and wind_speed. (which is what we need)

Simplifying the issue down could be seen as follows:
If I run a job for just the wind speed fields and reference the wind speed climatology files and the relevant field information within those then everything is fine. VAR1=Wind_Speed_850; VAR2=Wind_Speed_500

If I then add, for example, a single u-component level, and it’s relevant climatology file then the job runs and produces output for all of the fields, but when it is trying to do the wind_speed climatologies it starts questioning as to there not being any v component information to match up with the u component to generate the wind speed information (even though there is a specific wind_speed climate file (which I think it does then use)). VAR1=Wind_Speed_850; VAR2=Wind_Speed_500, VAR3=U-Wind_850

If we then add a v-component level it then it appears to starts calculating things for wind_speed from u and v rather than using the actual wind_speed file. (which is not what we want)

This is the case if we use both a single climatology file (with everything in) or when we list all of the fields independently.

Would the same occur if there was a case where you could reference a climo file to a specific field (as in using the VAR terminology), which I think doesn’t exist as it stands ?

We are also finding that the code is slowing down using the single climatology file option given the number of files we have been putting into the file compared to listing everything out individually. (we have scalar and vector in the same file, so may need to switch out) and how it reads in the files for each field. (the files now contain 4 times per day each day of the year)

Any thoughts as to whether we have an incorrect setup or not ?

[j-opatz]

Hi Rob,

And thank you for your question. Before I can answer, could you tell me what file type(s) you are using for your forecast/observation/climatology files? MET has different methods of dealing with various file types, and U/V components is a particularly tricky one.

[robdarvell]

Hi,

The job is using ECMWF GRIB1 climatology files. The forecast/obs files have been generated through the GenEnsProd pre-processing step with the data having originated from netcdf files (that have been through the RegridDataPlane to put onto a 1.5deg grid).
The extra preprocessing using RegridDataPlane is done separately as we do that for time improvements (which may not be necessary here but we are trying to keep consistent where we can with other things)

[JohnHalleyGotway]

@robdarvell, I took a look at this situation, and I think I was able to replicate it here.

I ran some GFS data through pcp_combine to get NetCDF files to mimic your input. And I used ERA climo data where U, V, and WS data are in separate files.

Here's a tar file with my sample script/config/data:
3290_example.tar.gz

Comparing 3290/log/grid_stat_combined.log to 3290/log/grid_stat_single.log, you'll see the difference in behavior.

• For "combined", when U/V/WIND are all in the same GRIB file (ERA/all500hPa_mean.grib), MET finds the WIND record in that file and happily uses it.
• For "single", it goes searching for the WIND data. It looks first in ERA/u500hPa_mean.grib but doesn't find it there... BUT it does find UGRD... and so it attempts to derive wind speed from it... which results in this log warning:

WARNING: MetGrib1DataFile::data_plane_array() -> when deriving winds, the number of U-wind records (1) does not match the number of V-wind records (0) for GRIB file "ERA/u500hPa_mean.grib".

• It checks ERA/v500hPa_mean.grib next, which results in this warning message:

WARNING: MetGrib1DataFile::data_plane_array() -> when deriving winds, the number of U-wind records (0) does not match the number of V-wind records (1) for GRIB file "ERA/v500hPa_mean.grib".

• It checks ERA/ws500hPa_mean.grib next, and does actually find it:

DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for VarInfo "WIND/P500" in GRIB record 1 of GRIB file "ERA/ws500hPa_mean.grib".

You can find the relevant section of logic in data2d_grib.cc.

So that's what's going on here. It may actually be working as you like, but printing a log of warning messages, and likely not as efficiently as it could.

I am also very concerned about wind rotation. When reading U/V data, MET tries to rotate grid-relative winds to earth-relative. It assumes that U and V are both in the same input file. With this ERA data, that's not the case. So we need to think about how to handle this.

So this isn't an "answer" but an acknowledgement that there's more investigation and work to be done here.