Cloning Hidden Nodes

Cloning Hidden Nodes
Cloning can help when there are problems finding a neural network configuration that will learn a complex dataset. Neural networks can be built with cloning enabled or cloning can be enabled while the network is learning. Hidden nodes are cloned during the learning process when the average error has stopped reducing. The hidden node with the greatest net input is frozen to determine if it is contributing to the learning process. If the node is contributing to the learning process it is cloned. Nodes that are not contributing are skipped and the node with the next greatest net input is tested. If no suitable node can be found a new node is created and fully connected with random low weights. It is then cloned producing an exact copy. The node that has been cloned is frozen again and its clone takes over in the learning process. The freeze level of the node that has been cloned is reduced until both the node and its clone are being used in the learning process. The learning process does not need to restart when a node is cloned.
About Clones & Cloning
1.     Nodes will not be cloned if the neural network is learning and the average error is decreasing.
2.     Clones may be produced during learning if cloning is enabled and the average error is not decreasing.
3.     Nodes that are cloned are initially frozen.
4.     The freeze level may reduce to zero very quickly and never be seen on the network display.
5.     Clones are first produced in hidden layer 1, then in hidden layer 2 and then in hidden layer 3.
6.     Clones produced in one hidden layer may move to other hidden layers.
7.     Networks that have been created with no hidden nodes or connections can use cloning. 
8.     Clones can be cloned.

Created with help of DrExplain