This class describes a neural network.
There are facilities to train the network and use the output.
The input layer is made of inactive neurons (returning the optionally normalized input) and output neurons are linear. The type of hidden neurons is free, the default being sigmoids. (One should still try to pass normalized inputs, e.g. between [0.,1])
The basic input is a TTree and two (training and test) TEventLists. Input and output neurons are assigned a value computed for each event with the same possibilities as for TTree::Draw(). Events may be weighted individually or via TTree::SetWeight(). 6 learning methods are available: kStochastic, kBatch, kSteepestDescent, kRibierePolak, kFletcherReeves and kBFGS.
This implementation, written by C. Delaere, is inspired from the mlpfit package from J.Schwindling et al. with some extensions:
Neural Networks are more and more used in various fields for data analysis and classification, both for research and commercial institutions. Some randomly chosen examples are:
More than 50% of neural networks are multilayer perceptrons. This implementation of multilayer perceptrons is inspired from the MLPfit package originally written by Jerome Schwindling. MLPfit remains one of the fastest tool for neural networks studies, and this ROOT add-on will not try to compete on that. A clear and flexible Object Oriented implementation has been chosen over a faster but more difficult to maintain code. Nevertheless, the time penalty does not exceed a factor 2.
The multilayer perceptron is a simple feed-forward network with the following structure:
It is made of neurons characterized by a bias and weighted links between them (let's call those links synapses). The input neurons receive the inputs, normalize them and forward them to the first hidden layer.
Each neuron in any subsequent layer first computes a linear combination of the outputs of the previous layer. The output of the neuron is then function of that combination with f being linear for output neurons or a sigmoid for hidden layers. This is useful because of two theorems:
The aim of all learning methods is to minimize the total error on a set of weighted examples. The error is defined as the sum in quadrature, divided by two, of the error on each individual output neuron. In all methods implemented, one needs to compute the first derivative of that error with respect to the weights. Exploiting the well-known properties of the derivative, especially the derivative of compound functions, one can write:
This computation is called back-propagation of the errors. A loop over all examples is called an epoch. Six learning methods are implemented.
is the most trivial learning method. This is the Robbins-Monro stochastic approximation applied to multilayer perceptrons. The weights are updated after each example according to the formula: \(w_{ij}(t+1) = w_{ij}(t) + \Delta w_{ij}(t)\)
with
\(\Delta w_{ij}(t) = - \eta(d e_p / d w_{ij} + \delta) + \epsilon \Delta w_{ij}(t-1)\)
The parameters for this method are Eta, EtaDecay, Delta and Epsilon.
It is the same as the stochastic minimization, but the weights are updated after considering all the examples, with the total derivative dEdw. The parameters for this method are Eta, EtaDecay, Delta and Epsilon.
Weights are set to the minimum along the line defined by the gradient. The only parameter for this method is Tau. Lower tau = higher precision = slower search. A value Tau = 3 seems reasonable.
Weights are set to the minimum along the line defined by the conjugate gradient. Parameters are Tau and Reset, which defines the epochs where the direction is reset to the steepest descent.
Weights are set to the minimum along the line defined by the conjugate gradient. Parameters are Tau and Reset, which defines the epochs where the direction is reset to the steepest descent.
Implies the computation of a NxN matrix computation, but seems more powerful at least for less than 300 weights. Parameters are Tau and Reset, which defines the epochs where the direction is reset to the steepest descent.
TMLP is build from 3 classes: TNeuron, TSynapse and TMultiLayerPerceptron. Only TMultiLayerPerceptron should be used explicitly by the user.
TMultiLayerPerceptron will take examples from a TTree given in the constructor. The network is described by a simple string: The input/output layers are defined by giving the expression for each neuron, separated by comas. Hidden layers are just described by the number of neurons. The layers are separated by colons. In addition, input/output layer formulas can be preceded by '@' (e.g "@out") if one wants to also normalize the data from the TTree. Input and outputs are taken from the TTree given as second argument. Expressions are evaluated as for TTree::Draw(), arrays are expended in distinct neurons, one for each index. This can only be done for fixed-size arrays. If the formula ends with "!", softmax functions are used for the output layer. One defines the training and test datasets by TEventLists.
Example:
Both the TTree and the TEventLists can be defined in the constructor, or later with the suited setter method. The lists used for training and test can be defined either explicitly, or via a string containing the formula to be used to define them, exactly as for a TCut.
The learning method is defined using the TMultiLayerPerceptron::SetLearningMethod() . Learning methods are :
A weight can be assigned to events, either in the constructor, either with TMultiLayerPerceptron::SetEventWeight(). In addition, the TTree weight is taken into account.
Finally, one starts the training with TMultiLayerPerceptron::Train(Int_t nepoch, Option_t* options). The first argument is the number of epochs while option is a string that can contain: "text" (simple text output) , "graph" (evoluting graphical training curves), "update=X" (step for the text/graph output update) or "+" (will skip the randomisation and start from the previous values). All combinations are available.
Example:
When the neural net is trained, it can be used directly ( TMultiLayerPerceptron::Evaluate() ) or exported to a standalone C++ code ( TMultiLayerPerceptron::Export() ).
Finally, note that even if this implementation is inspired from the mlpfit code, the feature lists are not exactly matching:
In addition, the paw version of mlpfit had additional limitations on the number of neurons, hidden layers and inputs/outputs that does not apply to TMultiLayerPerceptron.
Definition at line 26 of file TMultiLayerPerceptron.h.
Public Types | |
enum | EDataSet { kTraining , kTest } |
enum | ELearningMethod { kStochastic , kBatch , kSteepestDescent , kRibierePolak , kFletcherReeves , kBFGS } |
Public Types inherited from TObject | |
enum | { kIsOnHeap = 0x01000000 , kNotDeleted = 0x02000000 , kZombie = 0x04000000 , kInconsistent = 0x08000000 , kBitMask = 0x00ffffff } |
enum | { kSingleKey = BIT(0) , kOverwrite = BIT(1) , kWriteDelete = BIT(2) } |
enum | EDeprecatedStatusBits { kObjInCanvas = BIT(3) } |
enum | EStatusBits { kCanDelete = BIT(0) , kMustCleanup = BIT(3) , kIsReferenced = BIT(4) , kHasUUID = BIT(5) , kCannotPick = BIT(6) , kNoContextMenu = BIT(8) , kInvalidObject = BIT(13) } |
Public Member Functions | |
TMultiLayerPerceptron () | |
Default constructor. | |
TMultiLayerPerceptron (const char *layout, const char *weight, TTree *data, TEventList *training, TEventList *test, TNeuron::ENeuronType type=TNeuron::kSigmoid, const char *extF="", const char *extD="") | |
The network is described by a simple string: The input/output layers are defined by giving the branch names separated by comas. | |
TMultiLayerPerceptron (const char *layout, const char *weight, TTree *data=0, const char *training="Entry$%2==0", const char *test="", TNeuron::ENeuronType type=TNeuron::kSigmoid, const char *extF="", const char *extD="") | |
The network is described by a simple string: The input/output layers are defined by giving the branch names separated by comas. | |
TMultiLayerPerceptron (const char *layout, TTree *data, TEventList *training, TEventList *test, TNeuron::ENeuronType type=TNeuron::kSigmoid, const char *extF="", const char *extD="") | |
The network is described by a simple string: The input/output layers are defined by giving the branch names separated by comas. | |
TMultiLayerPerceptron (const char *layout, TTree *data=0, const char *training="Entry$%2==0", const char *test="", TNeuron::ENeuronType type=TNeuron::kSigmoid, const char *extF="", const char *extD="") | |
The network is described by a simple string: The input/output layers are defined by giving the branch names separated by comas. | |
virtual | ~TMultiLayerPerceptron () |
Destructor. | |
void | ComputeDEDw () const |
Compute the DEDw = sum on all training events of dedw for each weight normalized by the number of events. | |
virtual void | Draw (Option_t *option="") |
Draws the network structure. | |
void | DrawResult (Int_t index=0, Option_t *option="test") const |
Draws the neural net output It produces an histogram with the output for the two datasets. | |
Bool_t | DumpWeights (Option_t *filename="-") const |
Dumps the weights to a text file. | |
Double_t | Evaluate (Int_t index, Double_t *params) const |
Returns the Neural Net for a given set of input parameters #parameters must equal #input neurons. | |
void | Export (Option_t *filename="NNfunction", Option_t *language="C++") const |
Exports the NN as a function for any non-ROOT-dependant code Supported languages are: only C++ , FORTRAN and Python (yet) This feature is also useful if you want to plot the NN as a function (TF1 or TF2). | |
Double_t | GetDelta () const |
Double_t | GetEpsilon () const |
Double_t | GetError (Int_t event) const |
Error on the output for a given event. | |
Double_t | GetError (TMultiLayerPerceptron::EDataSet set) const |
Error on the whole dataset. | |
Double_t | GetEta () const |
Double_t | GetEtaDecay () const |
TMultiLayerPerceptron::ELearningMethod | GetLearningMethod () const |
Int_t | GetReset () const |
TString | GetStructure () const |
Double_t | GetTau () const |
TNeuron::ENeuronType | GetType () const |
Bool_t | LoadWeights (Option_t *filename="") |
Loads the weights from a text file conforming to the format defined by DumpWeights. | |
void | Randomize () const |
Randomize the weights. | |
Double_t | Result (Int_t event, Int_t index=0) const |
Computes the output for a given event. | |
void | SetData (TTree *) |
Set the data source. | |
void | SetDelta (Double_t delta) |
Sets Delta - used in stochastic minimisation (look at the constructor for the complete description of learning methods and parameters) | |
void | SetEpsilon (Double_t eps) |
Sets Epsilon - used in stochastic minimisation (look at the constructor for the complete description of learning methods and parameters) | |
void | SetEta (Double_t eta) |
Sets Eta - used in stochastic minimisation (look at the constructor for the complete description of learning methods and parameters) | |
void | SetEtaDecay (Double_t ed) |
Sets EtaDecay - Eta *= EtaDecay at each epoch (look at the constructor for the complete description of learning methods and parameters) | |
void | SetEventWeight (const char *) |
Set the event weight. | |
void | SetLearningMethod (TMultiLayerPerceptron::ELearningMethod method) |
Sets the learning method. | |
void | SetReset (Int_t reset) |
Sets number of epochs between two resets of the search direction to the steepest descent. | |
void | SetTau (Double_t tau) |
Sets Tau - used in line search (look at the constructor for the complete description of learning methods and parameters) | |
void | SetTestDataSet (const char *test) |
Sets the Test dataset. | |
void | SetTestDataSet (TEventList *test) |
Sets the Test dataset. | |
void | SetTrainingDataSet (const char *train) |
Sets the Training dataset. | |
void | SetTrainingDataSet (TEventList *train) |
Sets the Training dataset. | |
void | Train (Int_t nEpoch, Option_t *option="text", Double_t minE=0) |
Train the network. | |
Public Member Functions inherited from TObject | |
TObject () | |
TObject constructor. | |
TObject (const TObject &object) | |
TObject copy ctor. | |
virtual | ~TObject () |
TObject destructor. | |
void | AbstractMethod (const char *method) const |
Use this method to implement an "abstract" method that you don't want to leave purely abstract. | |
virtual void | AppendPad (Option_t *option="") |
Append graphics object to current pad. | |
virtual void | Browse (TBrowser *b) |
Browse object. May be overridden for another default action. | |
ULong_t | CheckedHash () |
Check and record whether this class has a consistent Hash/RecursiveRemove setup (*) and then return the regular Hash value for this object. | |
virtual const char * | ClassName () const |
Returns name of class to which the object belongs. | |
virtual void | Clear (Option_t *="") |
virtual TObject * | Clone (const char *newname="") const |
Make a clone of an object using the Streamer facility. | |
virtual Int_t | Compare (const TObject *obj) const |
Compare abstract method. | |
virtual void | Copy (TObject &object) const |
Copy this to obj. | |
virtual void | Delete (Option_t *option="") |
Delete this object. | |
virtual Int_t | DistancetoPrimitive (Int_t px, Int_t py) |
Computes distance from point (px,py) to the object. | |
virtual void | DrawClass () const |
Draw class inheritance tree of the class to which this object belongs. | |
virtual TObject * | DrawClone (Option_t *option="") const |
Draw a clone of this object in the current selected pad for instance with: gROOT->SetSelectedPad(gPad) . | |
virtual void | Dump () const |
Dump contents of object on stdout. | |
virtual void | Error (const char *method, const char *msgfmt,...) const |
Issue error message. | |
virtual void | Execute (const char *method, const char *params, Int_t *error=0) |
Execute method on this object with the given parameter string, e.g. | |
virtual void | Execute (TMethod *method, TObjArray *params, Int_t *error=0) |
Execute method on this object with parameters stored in the TObjArray. | |
virtual void | ExecuteEvent (Int_t event, Int_t px, Int_t py) |
Execute action corresponding to an event at (px,py). | |
virtual void | Fatal (const char *method, const char *msgfmt,...) const |
Issue fatal error message. | |
virtual TObject * | FindObject (const char *name) const |
Must be redefined in derived classes. | |
virtual TObject * | FindObject (const TObject *obj) const |
Must be redefined in derived classes. | |
virtual Option_t * | GetDrawOption () const |
Get option used by the graphics system to draw this object. | |
virtual const char * | GetIconName () const |
Returns mime type name of object. | |
virtual const char * | GetName () const |
Returns name of object. | |
virtual char * | GetObjectInfo (Int_t px, Int_t py) const |
Returns string containing info about the object at position (px,py). | |
virtual Option_t * | GetOption () const |
virtual const char * | GetTitle () const |
Returns title of object. | |
virtual UInt_t | GetUniqueID () const |
Return the unique object id. | |
virtual Bool_t | HandleTimer (TTimer *timer) |
Execute action in response of a timer timing out. | |
virtual ULong_t | Hash () const |
Return hash value for this object. | |
Bool_t | HasInconsistentHash () const |
Return true is the type of this object is known to have an inconsistent setup for Hash and RecursiveRemove (i.e. | |
virtual void | Info (const char *method, const char *msgfmt,...) const |
Issue info message. | |
virtual Bool_t | InheritsFrom (const char *classname) const |
Returns kTRUE if object inherits from class "classname". | |
virtual Bool_t | InheritsFrom (const TClass *cl) const |
Returns kTRUE if object inherits from TClass cl. | |
virtual void | Inspect () const |
Dump contents of this object in a graphics canvas. | |
void | InvertBit (UInt_t f) |
Bool_t | IsDestructed () const |
IsDestructed. | |
virtual Bool_t | IsEqual (const TObject *obj) const |
Default equal comparison (objects are equal if they have the same address in memory). | |
virtual Bool_t | IsFolder () const |
Returns kTRUE in case object contains browsable objects (like containers or lists of other objects). | |
R__ALWAYS_INLINE Bool_t | IsOnHeap () const |
virtual Bool_t | IsSortable () const |
R__ALWAYS_INLINE Bool_t | IsZombie () const |
virtual void | ls (Option_t *option="") const |
The ls function lists the contents of a class on stdout. | |
void | MayNotUse (const char *method) const |
Use this method to signal that a method (defined in a base class) may not be called in a derived class (in principle against good design since a child class should not provide less functionality than its parent, however, sometimes it is necessary). | |
virtual Bool_t | Notify () |
This method must be overridden to handle object notification. | |
void | Obsolete (const char *method, const char *asOfVers, const char *removedFromVers) const |
Use this method to declare a method obsolete. | |
void | operator delete (void *ptr) |
Operator delete. | |
void | operator delete[] (void *ptr) |
Operator delete []. | |
void * | operator new (size_t sz) |
void * | operator new (size_t sz, void *vp) |
void * | operator new[] (size_t sz) |
void * | operator new[] (size_t sz, void *vp) |
TObject & | operator= (const TObject &rhs) |
TObject assignment operator. | |
virtual void | Paint (Option_t *option="") |
This method must be overridden if a class wants to paint itself. | |
virtual void | Pop () |
Pop on object drawn in a pad to the top of the display list. | |
virtual void | Print (Option_t *option="") const |
This method must be overridden when a class wants to print itself. | |
virtual Int_t | Read (const char *name) |
Read contents of object with specified name from the current directory. | |
virtual void | RecursiveRemove (TObject *obj) |
Recursively remove this object from a list. | |
void | ResetBit (UInt_t f) |
virtual void | SaveAs (const char *filename="", Option_t *option="") const |
Save this object in the file specified by filename. | |
virtual void | SavePrimitive (std::ostream &out, Option_t *option="") |
Save a primitive as a C++ statement(s) on output stream "out". | |
void | SetBit (UInt_t f) |
void | SetBit (UInt_t f, Bool_t set) |
Set or unset the user status bits as specified in f. | |
virtual void | SetDrawOption (Option_t *option="") |
Set drawing option for object. | |
virtual void | SetUniqueID (UInt_t uid) |
Set the unique object id. | |
virtual void | SysError (const char *method, const char *msgfmt,...) const |
Issue system error message. | |
R__ALWAYS_INLINE Bool_t | TestBit (UInt_t f) const |
Int_t | TestBits (UInt_t f) const |
virtual void | UseCurrentStyle () |
Set current style settings in this object This function is called when either TCanvas::UseCurrentStyle or TROOT::ForceStyle have been invoked. | |
virtual void | Warning (const char *method, const char *msgfmt,...) const |
Issue warning message. | |
virtual Int_t | Write (const char *name=0, Int_t option=0, Int_t bufsize=0) |
Write this object to the current directory. | |
virtual Int_t | Write (const char *name=0, Int_t option=0, Int_t bufsize=0) const |
Write this object to the current directory. | |
Protected Member Functions | |
void | AttachData () |
Connects the TTree to Neurons in input and output layers. | |
void | BFGSDir (TMatrixD &, Double_t *) |
Computes the direction for the BFGS algorithm as the product between the Hessian estimate (bfgsh) and the dir. | |
void | BuildNetwork () |
Instantiates the network from the description. | |
void | ConjugateGradientsDir (Double_t *, Double_t) |
Sets the search direction to conjugate gradient direction beta should be: | |
Double_t | DerivDir (Double_t *) |
scalar product between gradient and direction = derivative along direction | |
bool | GetBFGSH (TMatrixD &, TMatrixD &, TMatrixD &) |
Computes the hessian matrix using the BFGS update algorithm. | |
Double_t | GetCrossEntropy () const |
Cross entropy error for a softmax output neuron, for a given event. | |
Double_t | GetCrossEntropyBinary () const |
Cross entropy error for sigmoid output neurons, for a given event. | |
void | GetEntry (Int_t) const |
Load an entry into the network. | |
Double_t | GetSumSquareError () const |
Error on the output for a given event. | |
Bool_t | LineSearch (Double_t *, Double_t *) |
Search along the line defined by direction. | |
void | MLP_Batch (Double_t *) |
One step for the batch (stochastic) method. | |
void | MLP_Stochastic (Double_t *) |
One step for the stochastic method buffer should contain the previous dw vector and will be updated. | |
void | SetGammaDelta (TMatrixD &, TMatrixD &, Double_t *) |
Sets the gamma \((g_{(t+1)}-g_{(t)})\) and delta \((w_{(t+1)}-w_{(t)})\) vectors Gamma is computed here, so ComputeDEDw cannot have been called before, and delta is a direct translation of buffer into a TMatrixD. | |
void | SteepestDir (Double_t *) |
Sets the search direction to steepest descent. | |
Protected Member Functions inherited from TObject | |
virtual void | DoError (int level, const char *location, const char *fmt, va_list va) const |
Interface to ErrorHandler (protected). | |
void | MakeZombie () |
Private Member Functions | |
TMultiLayerPerceptron (const TMultiLayerPerceptron &) | |
void | BuildFirstLayer (TString &) |
Instantiates the neurons in input Inputs are normalised and the type is set to kOff (simple forward of the formula value) | |
void | BuildHiddenLayers (TString &) |
Builds hidden layers. | |
void | BuildLastLayer (TString &, Int_t) |
Builds the output layer Neurons are linear combinations of input, by default. | |
void | BuildOneHiddenLayer (const TString &sNumNodes, Int_t &layer, Int_t &prevStart, Int_t &prevStop, Bool_t lastLayer) |
Builds a hidden layer, updates the number of layers. | |
void | ExpandStructure () |
Expand the structure of the first layer. | |
void | MLP_Line (Double_t *, Double_t *, Double_t) |
Sets the weights to a point along a line Weights are set to [origin + (dist * dir)]. | |
TMultiLayerPerceptron & | operator= (const TMultiLayerPerceptron &) |
void | Shuffle (Int_t *, Int_t) const |
Shuffle the Int_t index[n] in input. | |
Private Attributes | |
Int_t | fCurrentTree |
! index of the current tree in a chain | |
Double_t | fCurrentTreeWeight |
! weight of the current tree in a chain | |
TTree * | fData |
! pointer to the tree used as datasource | |
Double_t | fDelta |
! Delta - used in stochastic minimisation - Default=0. | |
Double_t | fEpsilon |
! Epsilon - used in stochastic minimisation - Default=0. | |
Double_t | fEta |
! Eta - used in stochastic minimisation - Default=0.1 | |
Double_t | fEtaDecay |
! EtaDecay - Eta *= EtaDecay at each epoch - Default=1. | |
TTreeFormula * | fEventWeight |
! formula representing the event weight | |
TString | fextD |
String containing the derivative name. | |
TString | fextF |
String containing the function name. | |
TObjArray | fFirstLayer |
Collection of the input neurons; subset of fNetwork. | |
Double_t | fLastAlpha |
! internal parameter used in line search | |
TObjArray | fLastLayer |
Collection of the output neurons; subset of fNetwork. | |
ELearningMethod | fLearningMethod |
! The Learning Method | |
TTreeFormulaManager * | fManager |
! TTreeFormulaManager for the weight and neurons | |
TObjArray | fNetwork |
Collection of all the neurons in the network. | |
TNeuron::ENeuronType | fOutType |
Type of output neurons. | |
Int_t | fReset |
! number of epochs between two resets of the search direction to the steepest descent - Default=50 | |
TString | fStructure |
String containing the network structure. | |
TObjArray | fSynapses |
Collection of all the synapses in the network. | |
Double_t | fTau |
! Tau - used in line search - Default=3. | |
TEventList * | fTest |
! EventList defining the events in the test dataset | |
Bool_t | fTestOwner |
! internal flag whether one has to delete fTest or not | |
TEventList * | fTraining |
! EventList defining the events in the training dataset | |
Bool_t | fTrainingOwner |
! internal flag whether one has to delete fTraining or not | |
TNeuron::ENeuronType | fType |
Type of hidden neurons. | |
TString | fWeight |
String containing the event weight. | |
Friends | |
class | TMLPAnalyzer |
Additional Inherited Members | |
Static Public Member Functions inherited from TObject | |
static Longptr_t | GetDtorOnly () |
Return destructor only flag. | |
static Bool_t | GetObjectStat () |
Get status of object stat flag. | |
static void | SetDtorOnly (void *obj) |
Set destructor only flag. | |
static void | SetObjectStat (Bool_t stat) |
Turn on/off tracking of objects in the TObjectTable. | |
Protected Types inherited from TObject | |
enum | { kOnlyPrepStep = BIT(3) } |
#include <TMultiLayerPerceptron.h>
Enumerator | |
---|---|
kTraining | |
kTest |
Definition at line 32 of file TMultiLayerPerceptron.h.
Enumerator | |
---|---|
kStochastic | |
kBatch | |
kSteepestDescent | |
kRibierePolak | |
kFletcherReeves | |
kBFGS |
Definition at line 30 of file TMultiLayerPerceptron.h.
TMultiLayerPerceptron::TMultiLayerPerceptron | ( | ) |
Default constructor.
Definition at line 264 of file TMultiLayerPerceptron.cxx.
TMultiLayerPerceptron::TMultiLayerPerceptron | ( | const char * | layout, |
TTree * | data = 0 , |
||
const char * | training = "Entry$%2==0" , |
||
const char * | test = "" , |
||
TNeuron::ENeuronType | type = TNeuron::kSigmoid , |
||
const char * | extF = "" , |
||
const char * | extD = "" |
||
) |
The network is described by a simple string: The input/output layers are defined by giving the branch names separated by comas.
Hidden layers are just described by the number of neurons. The layers are separated by colons.
Ex: "x,y:10:5:f"
The output can be prepended by '@' if the variable has to be normalized. The output can be followed by '!' to use Softmax neurons for the output layer only.
Ex: "x,y:10:5:c1,c2,c3!"
Input and outputs are taken from the TTree given as second argument. training and test are two cuts (see TTreeFormula) defining events to be used during the neural net training and testing.
Example: "Entry$%2", "(Entry$+1)%2".
Both the TTree and the cut can be defined in the constructor, or later with the suited setter method.
Definition at line 446 of file TMultiLayerPerceptron.cxx.
TMultiLayerPerceptron::TMultiLayerPerceptron | ( | const char * | layout, |
const char * | weight, | ||
TTree * | data = 0 , |
||
const char * | training = "Entry$%2==0" , |
||
const char * | test = "" , |
||
TNeuron::ENeuronType | type = TNeuron::kSigmoid , |
||
const char * | extF = "" , |
||
const char * | extD = "" |
||
) |
The network is described by a simple string: The input/output layers are defined by giving the branch names separated by comas.
Hidden layers are just described by the number of neurons. The layers are separated by colons.
Ex: "x,y:10:5:f"
The output can be prepended by '@' if the variable has to be normalized. The output can be followed by '!' to use Softmax neurons for the output layer only.
Ex: "x,y:10:5:c1,c2,c3!"
Input and outputs are taken from the TTree given as second argument. training and test are two cuts (see TTreeFormula) defining events to be used during the neural net training and testing.
Example: "Entry$%2", "(Entry$+1)%2".
Both the TTree and the cut can be defined in the constructor, or later with the suited setter method.
Definition at line 518 of file TMultiLayerPerceptron.cxx.
TMultiLayerPerceptron::TMultiLayerPerceptron | ( | const char * | layout, |
TTree * | data, | ||
TEventList * | training, | ||
TEventList * | test, | ||
TNeuron::ENeuronType | type = TNeuron::kSigmoid , |
||
const char * | extF = "" , |
||
const char * | extD = "" |
||
) |
The network is described by a simple string: The input/output layers are defined by giving the branch names separated by comas.
Hidden layers are just described by the number of neurons. The layers are separated by colons.
Ex: "x,y:10:5:f"
The output can be prepended by '@' if the variable has to be normalized. The output can be followed by '!' to use Softmax neurons for the output layer only.
Ex: "x,y:10:5:c1,c2,c3!"
Input and outputs are taken from the TTree given as second argument. training and test are the two TEventLists defining events to be used during the neural net training. Both the TTree and the TEventLists can be defined in the constructor, or later with the suited setter method.
Definition at line 318 of file TMultiLayerPerceptron.cxx.
TMultiLayerPerceptron::TMultiLayerPerceptron | ( | const char * | layout, |
const char * | weight, | ||
TTree * | data, | ||
TEventList * | training, | ||
TEventList * | test, | ||
TNeuron::ENeuronType | type = TNeuron::kSigmoid , |
||
const char * | extF = "" , |
||
const char * | extD = "" |
||
) |
The network is described by a simple string: The input/output layers are defined by giving the branch names separated by comas.
Hidden layers are just described by the number of neurons. The layers are separated by colons.
Ex: "x,y:10:5:f"
The output can be prepended by '@' if the variable has to be normalized. The output can be followed by '!' to use Softmax neurons for the output layer only.
Ex: "x,y:10:5:c1,c2,c3!"
Input and outputs are taken from the TTree given as second argument. training and test are the two TEventLists defining events to be used during the neural net training. Both the TTree and the TEventLists can be defined in the constructor, or later with the suited setter method.
Definition at line 380 of file TMultiLayerPerceptron.cxx.
|
virtual |
Destructor.
Definition at line 569 of file TMultiLayerPerceptron.cxx.
|
private |
|
protected |
Connects the TTree to Neurons in input and output layers.
The formulas associated to each neuron are created and reported to the network formula manager. By default, the branch is not normalised since this would degrade performance for classification jobs. Normalisation can be requested by putting '@' in front of the formula.
Definition at line 1248 of file TMultiLayerPerceptron.cxx.
Computes the direction for the BFGS algorithm as the product between the Hessian estimate (bfgsh) and the dir.
Definition at line 2476 of file TMultiLayerPerceptron.cxx.
Instantiates the neurons in input Inputs are normalised and the type is set to kOff (simple forward of the formula value)
Definition at line 1383 of file TMultiLayerPerceptron.cxx.
Builds hidden layers.
Definition at line 1401 of file TMultiLayerPerceptron.cxx.
Builds the output layer Neurons are linear combinations of input, by default.
If the structure ends with "!", neurons are set up for classification, ie. with a sigmoid (1 neuron) or softmax (more neurons) activation function.
Definition at line 1465 of file TMultiLayerPerceptron.cxx.
|
protected |
Instantiates the network from the description.
Definition at line 1352 of file TMultiLayerPerceptron.cxx.
|
private |
Builds a hidden layer, updates the number of layers.
Definition at line 1420 of file TMultiLayerPerceptron.cxx.
void TMultiLayerPerceptron::ComputeDEDw | ( | ) | const |
Compute the DEDw = sum on all training events of dedw for each weight normalized by the number of events.
Definition at line 1145 of file TMultiLayerPerceptron.cxx.
Sets the search direction to conjugate gradient direction beta should be:
\(||g_{(t+1)}||^2 / ||g_{(t)}||^2\) (Fletcher-Reeves)
\(g_{(t+1)} (g_{(t+1)}-g_{(t)}) / ||g_{(t)}||^2\) (Ribiere-Polak)
Definition at line 2361 of file TMultiLayerPerceptron.cxx.
scalar product between gradient and direction = derivative along direction
Definition at line 2452 of file TMultiLayerPerceptron.cxx.
Draws the network structure.
Neurons are depicted by a blue disk, and synapses by lines connecting neurons. The line width is proportional to the weight.
Reimplemented from TObject.
Definition at line 2506 of file TMultiLayerPerceptron.cxx.
Draws the neural net output It produces an histogram with the output for the two datasets.
Index is the number of the desired output neuron. "option" can contain:
Definition at line 1515 of file TMultiLayerPerceptron.cxx.
Dumps the weights to a text file.
Set filename to "-" (default) to dump to the standard output
Definition at line 1589 of file TMultiLayerPerceptron.cxx.
Returns the Neural Net for a given set of input parameters #parameters must equal #input neurons.
Definition at line 1695 of file TMultiLayerPerceptron.cxx.
|
private |
Expand the structure of the first layer.
Definition at line 1306 of file TMultiLayerPerceptron.cxx.
void TMultiLayerPerceptron::Export | ( | Option_t * | filename = "NNfunction" , |
Option_t * | language = "C++" |
||
) | const |
Exports the NN as a function for any non-ROOT-dependant code Supported languages are: only C++ , FORTRAN and Python (yet) This feature is also useful if you want to plot the NN as a function (TF1 or TF2).
Definition at line 1720 of file TMultiLayerPerceptron.cxx.
|
protected |
Computes the hessian matrix using the BFGS update algorithm.
from gamma (g_{(t+1)}-g_{(t)}) and delta (w_{(t+1)}-w_{(t)}). It returns true if such a direction could not be found (if gamma and delta are orthogonal).
Definition at line 2387 of file TMultiLayerPerceptron.cxx.
|
protected |
Cross entropy error for a softmax output neuron, for a given event.
Definition at line 1124 of file TMultiLayerPerceptron.cxx.
|
protected |
Cross entropy error for sigmoid output neurons, for a given event.
Definition at line 1093 of file TMultiLayerPerceptron.cxx.
|
inline |
Definition at line 78 of file TMultiLayerPerceptron.h.
Load an entry into the network.
Definition at line 741 of file TMultiLayerPerceptron.cxx.
|
inline |
Definition at line 77 of file TMultiLayerPerceptron.h.
Error on the output for a given event.
Definition at line 1028 of file TMultiLayerPerceptron.cxx.
Double_t TMultiLayerPerceptron::GetError | ( | TMultiLayerPerceptron::EDataSet | set | ) | const |
Error on the whole dataset.
Definition at line 1057 of file TMultiLayerPerceptron.cxx.
|
inline |
Definition at line 76 of file TMultiLayerPerceptron.h.
|
inline |
Definition at line 79 of file TMultiLayerPerceptron.h.
|
inline |
Definition at line 80 of file TMultiLayerPerceptron.h.
|
inline |
Definition at line 82 of file TMultiLayerPerceptron.h.
|
inline |
Definition at line 83 of file TMultiLayerPerceptron.h.
|
protected |
Error on the output for a given event.
Definition at line 1080 of file TMultiLayerPerceptron.cxx.
|
inline |
Definition at line 81 of file TMultiLayerPerceptron.h.
|
inline |
Definition at line 84 of file TMultiLayerPerceptron.h.
Search along the line defined by direction.
buffer is not used but is updated with the new dw so that it can be used by a later stochastic step. It returns true if the line search fails.
Definition at line 2256 of file TMultiLayerPerceptron.cxx.
Loads the weights from a text file conforming to the format defined by DumpWeights.
Definition at line 1639 of file TMultiLayerPerceptron.cxx.
One step for the batch (stochastic) method.
DEDw should have been updated before calling this.
Definition at line 2185 of file TMultiLayerPerceptron.cxx.
Sets the weights to a point along a line Weights are set to [origin + (dist * dir)].
Definition at line 2213 of file TMultiLayerPerceptron.cxx.
One step for the stochastic method buffer should contain the previous dw vector and will be updated.
Definition at line 2140 of file TMultiLayerPerceptron.cxx.
|
private |
void TMultiLayerPerceptron::Randomize | ( | ) | const |
Randomize the weights.
Definition at line 1221 of file TMultiLayerPerceptron.cxx.
Computes the output for a given event.
Look at the output neuron designed by index.
Definition at line 1015 of file TMultiLayerPerceptron.cxx.
Set the data source.
Definition at line 578 of file TMultiLayerPerceptron.cxx.
Sets Delta - used in stochastic minimisation (look at the constructor for the complete description of learning methods and parameters)
Definition at line 702 of file TMultiLayerPerceptron.cxx.
Sets Epsilon - used in stochastic minimisation (look at the constructor for the complete description of learning methods and parameters)
Definition at line 692 of file TMultiLayerPerceptron.cxx.
Sets Eta - used in stochastic minimisation (look at the constructor for the complete description of learning methods and parameters)
Definition at line 682 of file TMultiLayerPerceptron.cxx.
Sets EtaDecay - Eta *= EtaDecay at each epoch (look at the constructor for the complete description of learning methods and parameters)
Definition at line 712 of file TMultiLayerPerceptron.cxx.
void TMultiLayerPerceptron::SetEventWeight | ( | const char * | branch | ) |
Set the event weight.
Definition at line 594 of file TMultiLayerPerceptron.cxx.
|
protected |
Sets the gamma \((g_{(t+1)}-g_{(t)})\) and delta \((w_{(t+1)}-w_{(t)})\) vectors Gamma is computed here, so ComputeDEDw cannot have been called before, and delta is a direct translation of buffer into a TMatrixD.
Definition at line 2413 of file TMultiLayerPerceptron.cxx.
void TMultiLayerPerceptron::SetLearningMethod | ( | TMultiLayerPerceptron::ELearningMethod | method | ) |
Sets the learning method.
Available methods are: kStochastic, kBatch, kSteepestDescent, kRibierePolak, kFletcherReeves and kBFGS. (look at the constructor for the complete description of learning methods and parameters)
Definition at line 672 of file TMultiLayerPerceptron.cxx.
Sets number of epochs between two resets of the search direction to the steepest descent.
(look at the constructor for the complete description of learning methods and parameters)
Definition at line 733 of file TMultiLayerPerceptron.cxx.
Sets Tau - used in line search (look at the constructor for the complete description of learning methods and parameters)
Definition at line 722 of file TMultiLayerPerceptron.cxx.
void TMultiLayerPerceptron::SetTestDataSet | ( | const char * | test | ) |
Sets the Test dataset.
Those events will not be used for the minimization but for control. Note that the tree must be already defined.
Definition at line 651 of file TMultiLayerPerceptron.cxx.
void TMultiLayerPerceptron::SetTestDataSet | ( | TEventList * | test | ) |
Sets the Test dataset.
Those events will not be used for the minimization but for control
Definition at line 621 of file TMultiLayerPerceptron.cxx.
void TMultiLayerPerceptron::SetTrainingDataSet | ( | const char * | train | ) |
Sets the Training dataset.
Those events will be used for the minimization. Note that the tree must be already defined.
Definition at line 633 of file TMultiLayerPerceptron.cxx.
void TMultiLayerPerceptron::SetTrainingDataSet | ( | TEventList * | train | ) |
Sets the Training dataset.
Those events will be used for the minimization
Definition at line 610 of file TMultiLayerPerceptron.cxx.
Shuffle the Int_t index[n] in input.
Input:
Output:
This method is used for stochastic training
Definition at line 2121 of file TMultiLayerPerceptron.cxx.
Sets the search direction to steepest descent.
Definition at line 2235 of file TMultiLayerPerceptron.cxx.
Train the network.
nEpoch is the number of iterations. option can contain:
Definition at line 770 of file TMultiLayerPerceptron.cxx.
|
friend |
Definition at line 27 of file TMultiLayerPerceptron.h.
|
private |
! index of the current tree in a chain
Definition at line 124 of file TMultiLayerPerceptron.h.
|
private |
! weight of the current tree in a chain
Definition at line 125 of file TMultiLayerPerceptron.h.
|
private |
! pointer to the tree used as datasource
Definition at line 123 of file TMultiLayerPerceptron.h.
|
private |
! Delta - used in stochastic minimisation - Default=0.
Definition at line 143 of file TMultiLayerPerceptron.h.
|
private |
! Epsilon - used in stochastic minimisation - Default=0.
Definition at line 142 of file TMultiLayerPerceptron.h.
|
private |
! Eta - used in stochastic minimisation - Default=0.1
Definition at line 141 of file TMultiLayerPerceptron.h.
|
private |
! EtaDecay - Eta *= EtaDecay at each epoch - Default=1.
Definition at line 144 of file TMultiLayerPerceptron.h.
|
private |
! formula representing the event weight
Definition at line 139 of file TMultiLayerPerceptron.h.
|
private |
String containing the derivative name.
Definition at line 135 of file TMultiLayerPerceptron.h.
|
private |
String containing the function name.
Definition at line 134 of file TMultiLayerPerceptron.h.
|
private |
Collection of the input neurons; subset of fNetwork.
Definition at line 127 of file TMultiLayerPerceptron.h.
|
private |
! internal parameter used in line search
Definition at line 146 of file TMultiLayerPerceptron.h.
|
private |
Collection of the output neurons; subset of fNetwork.
Definition at line 128 of file TMultiLayerPerceptron.h.
|
private |
! The Learning Method
Definition at line 138 of file TMultiLayerPerceptron.h.
|
private |
! TTreeFormulaManager for the weight and neurons
Definition at line 140 of file TMultiLayerPerceptron.h.
|
private |
Collection of all the neurons in the network.
Definition at line 126 of file TMultiLayerPerceptron.h.
|
private |
Type of output neurons.
Definition at line 133 of file TMultiLayerPerceptron.h.
|
private |
! number of epochs between two resets of the search direction to the steepest descent - Default=50
Definition at line 147 of file TMultiLayerPerceptron.h.
|
private |
String containing the network structure.
Definition at line 130 of file TMultiLayerPerceptron.h.
|
private |
Collection of all the synapses in the network.
Definition at line 129 of file TMultiLayerPerceptron.h.
|
private |
! Tau - used in line search - Default=3.
Definition at line 145 of file TMultiLayerPerceptron.h.
|
private |
! EventList defining the events in the test dataset
Definition at line 137 of file TMultiLayerPerceptron.h.
|
private |
! internal flag whether one has to delete fTest or not
Definition at line 149 of file TMultiLayerPerceptron.h.
|
private |
! EventList defining the events in the training dataset
Definition at line 136 of file TMultiLayerPerceptron.h.
|
private |
! internal flag whether one has to delete fTraining or not
Definition at line 148 of file TMultiLayerPerceptron.h.
|
private |
Type of hidden neurons.
Definition at line 132 of file TMultiLayerPerceptron.h.
|
private |
String containing the event weight.
Definition at line 131 of file TMultiLayerPerceptron.h.