[ENH] Sort values naturally when reading files

[PrimozGodec]

Issue

Fixes #4778

Description of changes

When reading files sort values naturally (before alphabetical sort was used).
Natural sort sorts numbers inside string as numbers: "sth2" is before "sth12"

Discussion

#4778 also said that Ordered is not used it is partially true. It sets the ordered flag in the DiscreteVariable. This flag tells whether values are ordered in the variable so that other widgets would respect that. I think values are in a current situation always ordered variable and all widgets respect this order (at least none expect FeatureStatistics care about this flag). So I think this flag can be removed. It is used only in the future statistics widget where if this flag is set the min and the max values are shown for the discrete variable.

In case we decide to leave this variable as it is we need to write better documentation for this checkbox.

Includes

• Code changes
• Tests
• Documentation

[codecov]

Codecov Report

Merging #4793 into master will increase coverage by 0.12%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #4793      +/-   ##
==========================================
+ Coverage   83.86%   83.99%   +0.12%     
==========================================
  Files         281      276       -5     
  Lines       56804    56083     -721     
==========================================
- Hits        47638    47106     -532     
+ Misses       9166     8977     -189     

[janezd]

This flag tells whether values are ordered in the variable so that other widgets would respect that. I think values are in a current situation always ordered variable and all widgets respect this order (at least none expect FeatureStatistics care about this flag). So I think this flag can be removed.

Not all widgets. Heat map doesn't. #4529! If the flag is set, it would have to be observed!

The flag was meant, in ancient times, to change behaviour of some methods like subsetting in binary trees. If variable's values are ordered, tree induction wouldn't try all possible subsets (as it does now), but only all possible cuts.

As it is, I don't think it is observed anywhere, and the order of values is "not ignored" by chance. It seems we can remove it, but let's discuss.

[ajdapretnar]

I would just state that Heat Map does consider the order. Split by uses the order set by the user. #4529 is about labels, which have nothing to do with DiscreteVariable as such. The behaviour is the same as in heat map plots in other libraries.

[janezd]

Just a thought - does natural_sorted indeed belong to io_util or is it more general? (Our "more general functions" are not well-organized and packed, though, so this may not really matter.)

[PrimozGodec]

Just a thought - does natural_sorted indeed belong to io_util or is it more general? (Our "more general functions" are not well-organized and packed, though, so this may not really matter.)

It is a more general function and can be placed somewhere else. Maybe in Orange.utils?

[janezd]

You mean Orange.util? I'd prefer to avoid it, it could become a kitchen sink of functions.

Orange.misc has similar stuff, but organized into modules. What about putting it there? Perhaps into Orange.misc.collections? As I said, it was just a though. These "handy functions" were never organized well in Orange.

[PrimozGodec]

@janezd I moved the function to misc