Chapter 2. The Simulations
fixation position has a slightly different frequency.
Training was done using the recurrent back-propagation algorithm supplied by the PDP++ software. The software allows the user to specify the exact timesteps that the simulation goes through when an event is presented. The total number of timesteps per presented event was set to seven. At timestep one, only the input is presented to the network. Activation is then allowed to spread through the network until at timestep four, the correct output target is also presented to the network. This is used for error- backpropagation. The network essentially compares its output to the target output and changes the weights in such a way as to reduce the error (see e.g. Dawson et al. (2001)). Training was ended after 2750 epochs, corresponding to about 4 million words pre- sented to the network. An important thing to note is that both networks started with the exactly same initial small random weights. This is important in so far as that it can be excluded that any observed effects can be explained by different random weights of the two networks at the start of the training process. The only differences between the two training regimes are that different events are presented at random for each epoch according to their frequency.
After training finished, the networks were tested to see what they have learned
and which words they were unable to learn. Testing was done in the following way. PDP++ produces a text file containing the activation of the output units that it produced for any given event. That is, it produces a string of numbers between -1 and 1. To be able to analise the networks behaviour, this string of numbers has to be converted into the phonetic form of the word. To do this, each individual feature was converted into the corresponding phonetic feature by calculating to which of the possible 11 features
in figure 2.2 it had the shortest distance.
After doing this for each of the 6 different
slots, the phoneme that the network calculated emerged.