Neural Network widget #2553

ajdapretnar · 2017-08-29T13:43:00Z

Issue

Neural Networks only in scripting part, not in a widget.

Description of changes

Added a simple, cumbersome NN widget. Should work for basic tasks. Documentation pending until layout consolidated.

Includes

Code changes
Tests
Documentation

kernc · 2017-08-29T14:02:19Z

Orange/widgets/model/owneuralnetwork.py

+
+class OWNNLearner(OWBaseLearner):
+    name = "Neural Network"
+    description = "Predict according to the nearest training instances."


Update description.

kernc · 2017-08-29T14:08:44Z

Orange/widgets/model/owneuralnetwork.py

+                                               callback=self.settings_changed)
+        self.activation_combo = gui.comboBox(
+            box, self, "activation_index", orientation=Qt.Horizontal,
+            label="Activation:", items=[i.capitalize() for i in self.activation],


The correct human-friendly labels would be: Identity, Logistic, tanh and ReLU. Imho .upper() would server better.

kernc · 2017-08-29T14:09:50Z

Orange/widgets/model/owneuralnetwork.py

+            callback=self.settings_changed)
+        self.solver_combo = gui.comboBox(
+            box, self, "solver_index", orientation=Qt.Horizontal,
+            label="Solver:", items=[i.capitalize() for i in self.solver],


.upper() would server better. Those are acronyms.

kernc · 2017-08-29T14:12:10Z

Orange/widgets/model/owneuralnetwork.py

+                                               "hidden_layers_input",
+                                               label="Hidden layers:",
+                                               orientation=Qt.Horizontal,
+                                               callback=self.settings_changed)


Add tooltip= describing its format.

kernc · 2017-08-29T14:12:32Z

Orange/widgets/model/owneuralnetwork.py

+        box = gui.vBox(self.controlArea, "Network")
+        self.hidden_layers_edit = gui.lineEdit(box, self,
+                                               "hidden_layers_input",
+                                               label="Hidden layers:",


Neurons per hidden layer?

Neurons and no of layers. This was a big pain and this is a sad sad cheat to accommodate for n layers of m neurons. It is a workaround to avoid complex GUI with +/- for layers.

Yeah, I'll update the label.

ajdapretnar · 2017-08-29T14:17:51Z

@kernc Would you care to check whether spin box maxval and steps make sense or not? To adjust this as well, while I'm at it.

kernc · 2017-08-29T14:23:47Z

I'd never wait to scroll through [1e-5, 1] in 1e-5 increments. Perhaps 1e-2 stepsize makes more sense in general, but the majority of users will just edit the number manually.

codecov-io · 2017-08-30T07:49:21Z

Codecov Report

Merging #2553 into master will increase coverage by <.01%.
The diff coverage is 93.75%.

@@            Coverage Diff             @@
##           master    #2553      +/-   ##
==========================================
+ Coverage   75.08%   75.09%   +<.01%     
==========================================
  Files         327      329       +2     
  Lines       57725    57773      +48     
==========================================
+ Hits        43341    43382      +41     
- Misses      14384    14391       +7

ajdapretnar · 2017-08-30T07:49:35Z

All updated.

kernc · 2017-08-30T15:46:40Z

Orange/widgets/model/owneuralnetwork.py

+                                               callback=self.settings_changed,
+                                               tooltip="A list of integers "
+                                                       "defining neurons. "
+                                                       "Lenght of list "


typo. Also might be better to indent these params a single level, like the rest below.

janezd · 2017-09-01T11:58:23Z

Orange/tests/test_neuralnetwork.py

+from Orange.evaluation import CA, CrossValidation, MSE
+
+
+class TestKNNLearner(unittest.TestCase):


Why TestKNNLearner?

Busted!

janezd · 2017-09-01T12:02:43Z

Orange/widgets/model/icons/NN.svg

+	L33.717,27.185z"/>
+<path fill="#989898" d="M28.403,24.722l4.067-0.016l0.084,1.769l3.396-2.431l-3.612-2.101l0.084,1.762l-3.985,0.016
+	C28.464,24.05,28.462,24.383,28.403,24.722z"/>
+</svg>


Icon is kind of crowded, lines are too thin... Maybe remove some arrows, and reduce the right-most layer to two neurons.

Icon is taken from Orange2 and will be replaced with something nicer from Lara. Soon. :)

janezd · 2017-09-01T12:03:36Z

Orange/widgets/model/owneuralnetwork.py

+                                               callback=self.settings_changed,
+                                               tooltip="A list of integers "
+                                                       "defining neurons. "
+                                                       "Lenght of list "


janezd · 2017-09-01T12:16:32Z

Orange/widgets/model/owneuralnetwork.py

+                                                       "layers. E.g. 4, 2, 2, 3.")
+        self.activation_combo = gui.comboBox(
+            box, self, "activation_index", orientation=Qt.Horizontal,
+            label="Activation:", items=[i.upper() for i in self.activation],


Don't use upper. It looks ugly. If you'd like some -- but not all -- letters in upper case, add a list of labels in parallel with the method names you have now.

janezd · 2017-09-01T12:17:08Z

Orange/widgets/model/owneuralnetwork.py

+            box, self, "alpha", 1e-5, 1.0, 1e-2,
+            label="Alpha:", decimals=5, alignment=Qt.AlignRight,
+            callback=self.settings_changed, controlWidth=80)
+        self.max_iter_spin = gui.spin(


Right-align.

Do you mean Qt.AlignRight as parameter?

Umm, yes add, alignment=Qt.AlignRight to the last spinbox. Numbers usually look better if aligned to the right. (Neurons per hidden layer is different.)

janezd · 2017-09-01T12:21:02Z

Orange/widgets/model/owneuralnetwork.py

+    def get_learner_parameters(self):
+        return (("Hidden layers", self.get_hidden_layers()),
+                ("Activation", self.activation[self.activation_index].capitalize()),
+                ("Solver", self.solver[self.solver_index].capitalize()),


capitalize or upper or a list of user-readable names.

janezd · 2017-09-01T12:22:44Z

Orange/widgets/model/owneuralnetwork.py

+            preprocessors=self.preprocessors)
+
+    def get_learner_parameters(self):
+        return (("Hidden layers", self.get_hidden_layers()),


This prints out as: Hidden layers: (100,)

Yes. It should. I think.

Oh, yes, I see what's the issue.

Oh no, I don't.

Something like hidden_layer_sizes=", ".join(map(str, self.get_hidden_layers()))

janezd

Perhaps you can set the default layer size to empty string, hidden_layers_input = Setting(""), and add
placeholderText="e.g. 100, 200, 100" to the arguments of hidden_layers_edit. This will show this "tooltip" to the user.

Then, of course, remove self.hidden_layers_edit.setText("100") in get_hidden_layers.

The problem with this is that the user doesn't know then that the default is 100. Maybe

        self.hidden_layers_edit = gui.lineEdit(
            box, self, "hidden_layers_input",
            label="Neurons per hidden layers:",
            placeholderText="e.g. 100, 200, 100; default is 100",
            orientation=Qt.Vertical,
            callback=self.settings_changed,
            addSpace=24)

janezd · 2017-09-01T19:59:13Z

Orange/widgets/model/owneuralnetwork.py

+            box, self, "alpha", 1e-5, 1.0, 1e-2,
+            label="Alpha:", decimals=5, alignment=Qt.AlignRight,
+            callback=self.settings_changed, controlWidth=80)
+        self.max_iter_spin = gui.spin(


Umm, yes add, alignment=Qt.AlignRight to the last spinbox. Numbers usually look better if aligned to the right. (Neurons per hidden layer is different.)

janezd · 2017-09-01T20:00:35Z

Orange/widgets/model/owneuralnetwork.py

+        self.hidden_layers_edit = gui.lineEdit(box, self,
+                                               "hidden_layers_input",
+                                               label="Neurons per hidden "
+                                                     "layer:",


Perhaps layer -> layers? It is not obvious the user can input a list. Not many users will find the tooltip.

janezd · 2017-09-01T20:04:08Z

Orange/widgets/model/owneuralnetwork.py

+            preprocessors=self.preprocessors)
+
+    def get_learner_parameters(self):
+        return (("Hidden layers", self.get_hidden_layers()),


Something like hidden_layer_sizes=", ".join(map(str, self.get_hidden_layers()))

ajdapretnar · 2017-09-04T09:52:36Z

I don't think the widget should have an empty string as a default, especially since one can't use it on the fly. All other widgets have sklearn's default values and so should this one.
Moreover, the user can see the default is "100". Not sure, perhaps I can set the default to "100,". Would this make more sense?

Everything else is fixed, once agreed I can add docs and call it a day. ;)

kernc · 2017-09-04T15:25:13Z

An empty string results in no hidden layers. Such a perceptron is not forbidden by sklearn.

Set the default to "100,", but also set the placeholder?

astaric · 2017-09-09T19:52:50Z

Orange/widgets/model/owneuralnetwork.py

+                ("Max iterations", self.max_iterations))
+
+    def get_hidden_layers(self):
+        layers = tuple(map(int, re.findall('\d+', self.hidden_layers_input)))


Lint suggests you use r'\d+'. I agree

ajdapretnar · 2017-09-11T06:59:09Z

Fixed and rebased.

ajdapretnar · 2017-09-14T11:00:07Z

I don't understand why tests don't pass. They pass locally.

astaric · 2017-09-14T11:35:25Z

It's not the tests that are failing. If you click on details, you can see that the pylint and documentation builds have failed.

Pylint has two complaints:

When you do MSE(), you get a scoring function, when the scoring function is given results, it returns a score. The way you should call it is MSE()(results) (or scorer = MSE(); scorer(results))
It wants QApplication import at the top of the file.

Documentation complains about the images in the documentation not being as small as they could be.

ajdapretnar · 2017-09-14T11:52:13Z

I guess it's fixed now.

ajdapretnar · 2017-09-19T10:56:52Z

I'm eager to merge this. Please advise what to do with the rouge import line in __main__?

kernc approved these changes Aug 29, 2017

View reviewed changes

ajdapretnar force-pushed the nn-widget branch from f4e09e6 to 8998523 Compare August 30, 2017 07:49

kernc reviewed Aug 30, 2017

View reviewed changes

janezd reviewed Sep 1, 2017

View reviewed changes

ajdapretnar force-pushed the nn-widget branch 2 times, most recently from 4a45b0d to a66422d Compare September 4, 2017 09:15

astaric reviewed Sep 9, 2017

View reviewed changes

ajdapretnar force-pushed the nn-widget branch from 2cc2fe0 to 596fb9f Compare September 11, 2017 06:58

ajdapretnar added 2 commits September 14, 2017 13:51

Neural Network widget

5610ee0

NN documentation

ba4e730

ajdapretnar force-pushed the nn-widget branch from bd12db6 to ba4e730 Compare September 14, 2017 11:51

fix import order: python stdlib imports come first

327e7a0

kernc force-pushed the nn-widget branch from 229767e to 327e7a0 Compare September 19, 2017 11:32

kernc merged commit ccc49b8 into biolab:master Sep 19, 2017

ajdapretnar mentioned this pull request Sep 21, 2017

Port or implement widget for training neural networks #28

Closed

ajdapretnar deleted the nn-widget branch November 30, 2018 09:56

		from Orange.evaluation import CA, CrossValidation, MSE


		class TestKNNLearner(unittest.TestCase):

Neural Network widget #2553

Neural Network widget #2553

Conversation

ajdapretnar commented Aug 29, 2017 • edited Loading

Issue

Description of changes

Includes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ajdapretnar commented Aug 29, 2017

kernc commented Aug 29, 2017

codecov-io commented Aug 30, 2017 • edited Loading

Codecov Report

ajdapretnar commented Aug 30, 2017

kernc Aug 30, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

janezd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ajdapretnar commented Sep 4, 2017

kernc commented Sep 4, 2017

Choose a reason for hiding this comment

ajdapretnar commented Sep 11, 2017

ajdapretnar commented Sep 14, 2017

astaric commented Sep 14, 2017

ajdapretnar commented Sep 14, 2017

ajdapretnar commented Sep 19, 2017

ajdapretnar commented Aug 29, 2017 •

edited

Loading

codecov-io commented Aug 30, 2017 •

edited

Loading

kernc Aug 30, 2017 •

edited

Loading