Structure Evolution of a Neural Network
by means of the EVASION-Method
.
generation
0
. generation 2000 . . . parity problem |
generation
500
. generation 3000 |
.
. . . generation 1000 . compressed |
The parity problem is often used to test cleverly deviced learning algorithms. For the above example this is the EVASON-method. The Neural Network of generation 0 is oversized. The aim is to find the minimum structure. EVASION means EVAcuation out of the dimenSION. Valleys have to be formed at the edges of the optimization space (the zero weight axes), so that the gradient path is leading from the hyper-space to the adjacent hyper-subspace. Following the gradient-path evolution descends into the smallest possible subspace. Evolution-strategic learning will eliminate the superfluous weights. In the figures the thickness of the connections represents the weight strength of the Neural Network.
Ingo Rechenberg: Evolutionsstrategie '94. Stuttgart: Frommann-Holzboog 1994.