Define test matrix coverage

[mattkram]

We currently test across a range of conda and python versions. This means every time CI runs we run something like 80 parallel cases. We are also testing very old versions of conda.

I recommend we clearly defined what our supported versions are, and update our test matrix accordingly.

I recommend we test against:

• Supported versions of conda
• Supported versions of Python

And that we update this as those versions reach end of life and new versions become available.

[mattkram]

I think I'm searching for a different term here. I'm thinking of "what is our promised level of support"?

[mattkram]

Just looked and we are testing

• 9 versions of conda
• 3 operating systems
• 3 versions of Python

os: ["ubuntu-latest", "macos-latest", "windows-latest"] 
python-version: [3.8, 3.9, "3.10"] 
conda-version: ["4.9", "4.10", "4.11", "4.12", "4.13", "4.14", "22.9", "22.11", "23.1"]

I recommend we keep the Python and OS as is. For conda I recommend we remove old versions and support the last 1-3 versions, up for discussion. I believe conda now releases quarterly.

[mattkram]

Another option is to only test the oldest and newest versions of conda and Python supported before PRs are merged, and test the rest after merge into main. Theoretically we could have some errors and have to revert or change some merged behavior, but it may be rare enough to make it worthwhile?

[AlbertDeFusco]

That sounds reasonable. I've not had to implement any version specific code so far anyway.