Source Code for Module Bio.NeuralNetwork.StopTraining

 1  """Classes to help deal with stopping training a neural network. 
 2   
 3  One of the key issues with training a neural network is knowning when to 
 4  stop the training of the network. This is tricky since you want to keep 
 5  training until the neural network has 'learned' the data, but want to 
 6  stop before starting to learn the noise in the data. 
 7   
 8  This module contains classes and functions which are different ways to 
 9  know when to stop training. Remember that the neural network classifier 
10  takes a function to call to know when to stop training, so the classes 
11  in this module should be instaniated, and then the stop_training function 
12  of the classes passed to the network. 
13  """ 
14   
15   
16 -class ValidationIncreaseStop(object): 
17      """Class to stop training on a network when the validation error increases. 
18   
19      Normally, during training of a network, the error will always decrease 
20      on the set of data used in the training. However, if an independent 
21      set of data is used for validation, the error will decrease to a point, 
22      and then start to increase. This increase normally occurs due to the 
23      fact that the network is starting to learn noise in the training data 
24      set. This stopping criterion function will stop when the validation 
25      error increases. 
26      """ 
27 -    def __init__(self, max_iterations=None, min_iterations=0, 
28                   verbose=0): 
29          """Initialize the stopping criterion class. 
30   
31          Arguments: 
32   
33          o max_iterations - The maximum number of iterations that 
34          should be performed, regardless of error. 
35   
36          o min_iterations - The minimum number of iterations to perform, 
37          to prevent premature stoppage of training. 
38   
39          o verbose - Whether or not the error should be printed during 
40          training. 
41          """ 
42          self.verbose = verbose 
43          self.max_iterations = max_iterations 
44          self.min_iterations = min_iterations 
45   
46          self.last_error = None 
47   
48 -    def stopping_criteria(self, num_iterations, training_error, 
49                            validation_error): 
50          """Define when to stop iterating. 
51          """ 
52          if num_iterations % 10 == 0: 
53              if self.verbose: 
54                  print "%s; Training Error:%s; Validation Error:%s"\ 
55                        % (num_iterations, training_error, validation_error) 
56   
57          if num_iterations > self.min_iterations: 
58              if self.last_error is not None: 
59                  if validation_error > self.last_error: 
60                      if self.verbose: 
61                          print "Validation Error increasing -- Stop" 
62                      return 1 
63   
64          if self.max_iterations is not None: 
65              if num_iterations > self.max_iterations: 
66                  if self.verbose: 
67                      print "Reached maximum number of iterations -- Stop" 
68                  return 1 
69   
70          self.last_error = validation_error 
71          return 0 
72