Testing

Running netCDF-Java tests

NetCDF-Java uses JUnit for testing. While there are a few ways to run JUnit tests, the most common ways for netCDF-Java are using gradle or using an IDE, such as IntelliJ. Here we will focus on the use of gradle.

Gradle and tests

In general, tests are executed on the command line using the following command:

./gradlew :subproject:test

where subproject is the name of the gradle subproject you wish to test, and test is the name of the gradle task that runs test. Executing the test task will make sure all other tasks it depends on are up-to-date or executed prior to running the tests. Some tests require a local server, and, thanks to the gretty plugin, gradle will take care of spinning up a local server for the tests to use before running those tests. A list of subprojects associated with the netCDF-Java codebase can be obtained from gradle using the following command at the root of the repository:

./gradlew projects

The file settings.gradle, which is located at the root of the repository, contains a list of these names as well, and also shows how some projects have been renamed (as the name controls the artifacts produced by the subproject).

As a concrete example, to run all of the tests within cdm/core/src/test, use

./gradlew :cdm:core:test

If one or more tests fail, gradle will generate a test report in html, and will tell you where you can find the report in the output on the command like. For example,

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':<project>:test'.
> There were failing tests. See the report at: file://<path-to-repository>/<gradle-project>/build/reports/tests/test/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

The html report will contain a list of failed tests, as well as their full stack traces, and any associated messages generated by the test itself.

Running specific tests

If you would like to run all of the tests associated with a package for a given project, you can use the --tests option:

./gradlew :cdm:cdm-core:test --tests ucar.nc2.ncml.*

The --tests option can also be used to run all tests in a particular test class:

./gradlew :cdm:cdm-core:test --tests ucar.nc2.ncml.TestAggExisting.testNcmlDirect

and even a single method within a test class:

./gradlew :cdm:cdm-core:test --tests ucar.nc2.ncml.TestAggExisting.testNcmlDirect.testNcmlDirect

Running (almost) all tests

To run all the tests from all gradle subprojects in netCDF-Java, execute the testAll task:

./gradlew testAll

If this is your first time running the netCDF-Java tests, you might be surprised to learn that you didn't actually run all of the tests! If you inspect the output on the command line, you will see something like:

"unidata.testdata.path" system property not defined.
Skipping all NeedsCdmUnitTest tests.

This is because some of our tests require access to the THREDDS Test Data (roughly 100 GB of data).

Obtaining and specifying the location of THREDDS Test Data

We have configured gradle to run as many tests as it can, based on what resources are available in the environment the tests are running. Some tests require access to THREDDS Test Data. This set of test data is also used by the THREDDS Data Server. The test data can be obtained using the instructions at https://github.com/unidata/thredds-test-data.

Once you have the test data, you need to tell gradle where it can find them. This is done through the use of a system property, which can be set using:

./gradlew -Dunidata.testdata.path=<path-to-test-data> :cdm:core:test

Note that if you are running on Windows using powershell, be sure to enclose the system property name in quotes, e.g. -D"unidata.testdata.path"=....

This can also be set within a special gradle configuration file, so that you don't need to supply it each time you run a command. On most systems, this file is located in the home directory under .gradle and is called gradle.properties. To add the unidata.testdata.path property, simply edit (or create) ~/.gradle/gradle.properties like so:

# For tests annotated with NeedsCdmUnitTest
systemProp.unidata.testdata.path=<path-to-threds-test-data-repo>/local/thredds-test-data

where <path-to-threds-test-data-repo> is the location to which thredds-test-data repository is checked out locally.

What else is needed to run all of the tests? Well...

Test related system properties

As previously mentioned, we have configured gradle to run as many tests as it can, based on what resources are available in the environment the tests are running. Under certain circumstances, tests are disabled by gradle using the JUnit @Category annotation. For example, if gradle determines that the system property unidata.testdata.path has not been set, or does not point to a valid directory, it will exclude any test annotated with @Category(NeedsCdmUnitTest.class). Below is a table of user controllable JUnit tests categories used by netCDF-Java, and the system properties that control them. These are disabled by default. To enable them, set the listed system property to a valid directory (unidata.testdata.path) or true (all others).

Test Category	Description	System Property
NeedsCdmUnitTest	Tests that need access to THREDDS Test Data	unidata.testdata.path
NeedsUcarNetwork	Tests that need access to the UCAR network	ucarNetworkAvailable
Slow	Test that have a long execution time	runSlowTests
NeedsRdaData	Requires direct access to RDA datasets	isRdaDataAvailable

Note - It's not clear which RDA datasets are needed, or how to access them, so anything marked NeedsRdaData never runs.

There are a few JUnit Categories that are used to exclude tests from running on certain systems (for various reasons). These are enabled by default, and only disabled if gradle detects it is running on a specific system.

Test Category	Description	System
NeedsExternalResource	Tests that need network access to publicly available resources	GitHub Actions
NotPullRequest	Tests that should not be run against pull requests	GitHub Actions
NotJenkins	Tests that should not be run on Jenkins	Jenkins

If you would like to run the smaller set of tests run on these systems on your local machine, set the appropriate environmental variable (the value does not matter - it just needs to be set):

System	Environmental Variable
GitHub Actions	GITHUB_ACTIONS
Jenkins	JENKINS_URL

Finally, to bypass all of these checks and attempt to run every test possible for a given subproject, set the following system property:

-DrunAllTestExceptIgnored

This will result in failures for the cdm-test subproject due to NeedsRdaData becoming enabled (see above). Note that any test annotated with @Ignore will still be ignored. The annotation must be removed for JUnit to recognize and run the test.

netCDF-C library

Finally, some netCDF-Java tests require the use of the netCDF-C library. Most specifically, the netCDF-C library (built with HDF5 support) is used by netCDF-Java when writing netCDF-4 files, and tests related to writing netCDF-4 files cannot run without it. First, you need to obtain a copy of the netCDF-C library (however you'd like, threads safety in the C library isn't an issue anymore as we manage threads locking on the java side when writing). Next, specify the location of the netCDF-C library using the system property jna.library.path=<path-to-lib> using -D on the command line or systemProp. in ~/.gradle/gradle.properties, where <path-to-lib> is the path to the directory containing netcdf.dll (Windows), netcdf.dylib (Mac), or libnetcdf.so (*nix). Any dependencies of netCDF-C must either be on the system library path, or in the same directory as the netCDF-C library.

Running (almost) all tests - bottom line

To run every test possible (except those using unknown data sources), make sure you do the following:

1. Obtain a local copy of THREDDS Test Data
2. Set the following system properties:
   • runSlowTests=true details
   • unidata.testdata.path=<path-to-test-data> (details)
   • jna.library.path=<path-to-netcdf-c-lib> (details)