Package Bio :: Package NeuralNetwork :: Module StopTraining
[hide private]
[frames] | no frames]

Source Code for Module Bio.NeuralNetwork.StopTraining

 1  # This code is part of the Biopython distribution and governed by its 
 2  # license.  Please see the LICENSE file that should have been included 
 3  # as part of this package. 
 4  # 
 5   
 6  """Classes to help deal with stopping training a neural network. 
 7   
 8  One of the key issues with training a neural network is knowning when to 
 9  stop the training of the network. This is tricky since you want to keep 
10  training until the neural network has 'learned' the data, but want to 
11  stop before starting to learn the noise in the data. 
12   
13  This module contains classes and functions which are different ways to 
14  know when to stop training. Remember that the neural network classifier 
15  takes a function to call to know when to stop training, so the classes 
16  in this module should be instaniated, and then the stop_training function 
17  of the classes passed to the network. 
18  """ 
19   
20  from __future__ import print_function 
21   
22   
23 -class ValidationIncreaseStop(object):
24 """Class to stop training on a network when the validation error increases. 25 26 Normally, during training of a network, the error will always decrease 27 on the set of data used in the training. However, if an independent 28 set of data is used for validation, the error will decrease to a point, 29 and then start to increase. This increase normally occurs due to the 30 fact that the network is starting to learn noise in the training data 31 set. This stopping criterion function will stop when the validation 32 error increases. 33 """
34 - def __init__(self, max_iterations=None, min_iterations=0, 35 verbose=0):
36 """Initialize the stopping criterion class. 37 38 Arguments: 39 40 o max_iterations - The maximum number of iterations that 41 should be performed, regardless of error. 42 43 o min_iterations - The minimum number of iterations to perform, 44 to prevent premature stoppage of training. 45 46 o verbose - Whether or not the error should be printed during 47 training. 48 """ 49 self.verbose = verbose 50 self.max_iterations = max_iterations 51 self.min_iterations = min_iterations 52 53 self.last_error = None
54
55 - def stopping_criteria(self, num_iterations, training_error, 56 validation_error):
57 """Define when to stop iterating. 58 """ 59 if num_iterations % 10 == 0: 60 if self.verbose: 61 print("%s; Training Error:%s; Validation Error:%s" 62 % (num_iterations, training_error, validation_error)) 63 64 if num_iterations > self.min_iterations: 65 if self.last_error is not None: 66 if validation_error > self.last_error: 67 if self.verbose: 68 print("Validation Error increasing -- Stop") 69 return 1 70 71 if self.max_iterations is not None: 72 if num_iterations > self.max_iterations: 73 if self.verbose: 74 print("Reached maximum number of iterations -- Stop") 75 return 1 76 77 self.last_error = validation_error 78 return 0
79