I believe there is a bug in the FourByteInt read path in netcdf-fortran v4.6.*.
In fortran/netcdf_expanded.F90, reading into FourByteInt uses a different implementation than TwoByteInt:
TwoByteInt reads directly into the destination array
FourByteInt reads into a flat temporary array and then reshapes:
defaultIntArray(size(values))
values = reshape(defaultIntArray, shape(values))
The core problem is that the Fortran intrinsic size(values) returns a default integer. On typical 32-bit default integer builds, that means the maximum representable size is only:
So for very large arrays, the FourByteInt path overflows in the wrapper logic, while the TwoByteInt path does not hit the same problem.
Observed behavior
- reading into a 16-bit integer array works
- reading into a 32-bit integer array segfaults
- small arrays work
- large arrays fail
Example:
- works:
36000 x 14000
- fails:
216000 x 84000
The failing case has:
216000 * 84000 = 18,144,000,000 elements
which is far above the limit of a default 32-bit integer.
Why this looks like a library bug
The problem is not just memory size. The important point is that FourByteInt and TwoByteInt are handled differently in the wrapper, and only the FourByteInt path relies on size(values) for a flat temporary array.
This makes the FourByteInt implementation fail for large arrays due to integer overflow in the wrapper itself.
Expected behavior
At minimum, this should not segfault.
Preferably, the FourByteInt path should avoid this temporary-array approach entirely, or it should detect unsupported sizes and return an error.
Suggestion
Please consider changing the FourByteInt wrapper to avoid using:
defaultIntArray(size(values))
because size(values) is a default integer and therefore not safe for arrays with more than 2,147,483,647 elements.
I believe there is a bug in the
FourByteIntread path innetcdf-fortranv4.6.*.In
fortran/netcdf_expanded.F90, reading intoFourByteIntuses a different implementation thanTwoByteInt:TwoByteIntreads directly into the destination arrayFourByteIntreads into a flat temporary array and then reshapes:defaultIntArray(size(values))values = reshape(defaultIntArray, shape(values))The core problem is that the Fortran intrinsic
size(values)returns a default integer. On typical 32-bit default integer builds, that means the maximum representable size is only:2,147,483,647So for very large arrays, the
FourByteIntpath overflows in the wrapper logic, while theTwoByteIntpath does not hit the same problem.Observed behavior
Example:
36000 x 14000216000 x 84000The failing case has:
216000 * 84000 = 18,144,000,000elementswhich is far above the limit of a default 32-bit integer.
Why this looks like a library bug
The problem is not just memory size. The important point is that
FourByteIntandTwoByteIntare handled differently in the wrapper, and only theFourByteIntpath relies onsize(values)for a flat temporary array.This makes the
FourByteIntimplementation fail for large arrays due to integer overflow in the wrapper itself.Expected behavior
At minimum, this should not segfault.
Preferably, the
FourByteIntpath should avoid this temporary-array approach entirely, or it should detect unsupported sizes and return an error.Suggestion
Please consider changing the
FourByteIntwrapper to avoid using:defaultIntArray(size(values))because
size(values)is a default integer and therefore not safe for arrays with more than2,147,483,647elements.