Analysis of Boosted Decision Trees.

Boosted decision trees have been successfully used in High Energy Physics analysis for example by the MiniBooNE experiment (Yang-Roe-Zhu, physics/0508045). In Boosted Decision Trees, the selection is done on a majority vote on the result of several decision trees, which are all derived from the same training sample by supplying different event weights during the training.

Decision trees:

successive decision nodes are used to categorize the events out of the sample as either signal or background. Each node uses only a single discriminating variable to decide if the event is signal-like ("goes right") or background-like ("goes left"). This forms a tree like structure with "baskets" at the end (leave nodes), and an event is classified as either signal or background according to whether the basket where it ends up has been classified signal or background during the training. Training of a decision tree is the process to define the "cut criteria" for each node. The training starts with the root node. Here one takes the full training event sample and selects the variable and corresponding cut value that gives the best separation between signal and background at this stage. Using this cut criterion, the sample is then divided into two subsamples, a signal-like (right) and a background-like (left) sample. Two new nodes are then created for each of the two sub-samples and they are constructed using the same mechanism as described for the root node. The devision is stopped once a certain node has reached either a minimum number of events, or a minimum or maximum signal purity. These leave nodes are then called "signal" or "background" if they contain more signal respective background events from the training sample.

Boosting:

the idea behind the boosting is, that signal events from the training sample, that *end up in a background node (and vice versa) are given a larger weight than events that are in the correct leave node. This results in a re-weighed training event sample, with which then a new decision tree can be developed. The boosting can be applied several times (typically 100-500 times) and one ends up with a set of decision trees (a forest).

Bagging:

In this particular variant of the Boosted Decision Trees the boosting is not done on the basis of previous training results, but by a simple stochastic re-sampling of the initial training event sample.

Analysis:

applying an individual decision tree to a test event results in a classification of the event as either signal or background. For the boosted decision tree selection, an event is successively subjected to the whole set of decision trees and depending on how often it is classified as signal, a "likelihood" estimator is constructed for the event being signal or background. The value of this estimator is the one which is then used to select the events from an event sample, and the cut value on this estimator defines the efficiency and purity of the selection.

Definition at line 49 of file MethodDT.h.

Public Member Functions
	MethodDT (const TString &jobName, const TString &methodTitle, DataSetInfo &theData, const TString &theOption="")
	the standard constructor for just an ordinar "decision trees"

	MethodDT (DataSetInfo &dsi, const TString &theWeightFile)
	constructor from Reader

virtual	~MethodDT (void)
	destructor

void	AddWeightsXMLTo (void *parent) const

const Ranking *	CreateRanking ()

void	DeclareCompatibilityOptions ()
	options that are used ONLY for the READER to ensure backward compatibility

void	DeclareOptions ()
	Define the options (their key words) that can be set in the option string.

void	GetHelpMessage () const

Double_t	GetMvaValue (Double_t err=nullptr, Double_t errUpper=nullptr)
	returns MVA value

Int_t	GetNNodes ()

Int_t	GetNNodesBeforePruning ()

Double_t	GetPruneStrength ()

virtual Bool_t	HasAnalysisType (Types::EAnalysisType type, UInt_t numberClasses, UInt_t numberTargets)
	FDA can handle classification with 2 classes and regression with one regression-target.

virtual TClass *	IsA () const

void	ProcessOptions ()
	the option string is decoded, for available options see "DeclareOptions"

Double_t	PruneTree ()
	prune the decision tree if requested (good for individual trees that are best grown out, and then pruned back, while boosted decision trees are best 'small' trees to start with.

virtual void	ReadWeightsFromStream (std::istream &)=0

void	ReadWeightsFromStream (std::istream &istr)

virtual void	ReadWeightsFromStream (TFile &)

void	ReadWeightsFromXML (void *wghtnode)

void	SetMinNodeSize (Double_t sizeInPercent)

void	SetMinNodeSize (TString sizeInPercent)

virtual void	Streamer (TBuffer &)

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

Double_t	TestTreeQuality (DecisionTree *dt)

void	Train (void)

Public Member Functions inherited from TMVA::MethodBase
	MethodBase (const TString &jobName, Types::EMVA methodType, const TString &methodTitle, DataSetInfo &dsi, const TString &theOption="")
	standard constructor

	MethodBase (Types::EMVA methodType, DataSetInfo &dsi, const TString &weightFile)
	constructor used for Testing + Application of the MVA, only (no training), using given WeightFiles

virtual	~MethodBase ()
	destructor

void	AddOutput (Types::ETreeType type, Types::EAnalysisType analysisType)

TDirectory *	BaseDir () const
	returns the ROOT directory where info/histograms etc of the corresponding MVA method instance are stored

virtual void	CheckSetup ()
	check may be overridden by derived class (sometimes, eg, fitters are used which can only be implemented during training phase)

DataSet *	Data () const

DataSetInfo &	DataInfo () const

void	DisableWriting (Bool_t setter)

Bool_t	DoMulticlass () const

Bool_t	DoRegression () const

void	ExitFromTraining ()

Types::EAnalysisType	GetAnalysisType () const

UInt_t	GetCurrentIter ()

virtual Double_t	GetEfficiency (const TString &, Types::ETreeType, Double_t &err)
	fill background efficiency (resp.

const Event *	GetEvent () const

const Event *	GetEvent (const TMVA::Event *ev) const

const Event *	GetEvent (Long64_t ievt) const

const Event *	GetEvent (Long64_t ievt, Types::ETreeType type) const

const std::vector< TMVA::Event * > &	GetEventCollection (Types::ETreeType type)
	returns the event collection (i.e.

TFile *	GetFile () const

const TString &	GetInputLabel (Int_t i) const

const char *	GetInputTitle (Int_t i) const

const TString &	GetInputVar (Int_t i) const

TMultiGraph *	GetInteractiveTrainingError ()

const TString &	GetJobName () const

virtual Double_t	GetKSTrainingVsTest (Char_t SorB, TString opt="X")

virtual Double_t	GetMaximumSignificance (Double_t SignalEvents, Double_t BackgroundEvents, Double_t &optimal_significance_value) const
	plot significance, \( \frac{S}{\sqrt{S^2 + B^2}} \), curve for given number of signal and background events; returns cut for maximum significance also returned via reference is the maximum significance

UInt_t	GetMaxIter ()

Double_t	GetMean (Int_t ivar) const

const TString &	GetMethodName () const

Types::EMVA	GetMethodType () const

TString	GetMethodTypeName () const

virtual TMatrixD	GetMulticlassConfusionMatrix (Double_t effB, Types::ETreeType type)
	Construct a confusion matrix for a multiclass classifier.

virtual std::vector< Float_t >	GetMulticlassEfficiency (std::vector< std::vector< Float_t > > &purity)

virtual std::vector< Float_t >	GetMulticlassTrainingEfficiency (std::vector< std::vector< Float_t > > &purity)

virtual const std::vector< Float_t > &	GetMulticlassValues ()

Double_t	GetMvaValue (const TMVA::Event const ev, Double_t err=nullptr, Double_t *errUpper=nullptr)

const char *	GetName () const

UInt_t	GetNEvents () const

UInt_t	GetNTargets () const

UInt_t	GetNvar () const

UInt_t	GetNVariables () const

virtual Double_t	GetProba (const Event *ev)

virtual Double_t	GetProba (Double_t mvaVal, Double_t ap_sig)
	compute likelihood ratio

const TString	GetProbaName () const

virtual Double_t	GetRarity (Double_t mvaVal, Types::ESBType reftype=Types::kBackground) const
	compute rarity:

virtual void	GetRegressionDeviation (UInt_t tgtNum, Types::ETreeType type, Double_t &stddev, Double_t &stddev90Percent) const

virtual const std::vector< Float_t > &	GetRegressionValues ()

const std::vector< Float_t > &	GetRegressionValues (const TMVA::Event *const ev)

Double_t	GetRMS (Int_t ivar) const

virtual Double_t	GetROCIntegral (PDF pdfS=nullptr, PDF pdfB=nullptr) const
	calculate the area (integral) under the ROC curve as a overall quality measure of the classification

virtual Double_t	GetROCIntegral (TH1D histS, TH1D histB) const
	calculate the area (integral) under the ROC curve as a overall quality measure of the classification

virtual Double_t	GetSeparation (PDF pdfS=nullptr, PDF pdfB=nullptr) const
	compute "separation" defined as

virtual Double_t	GetSeparation (TH1 , TH1 ) const
	compute "separation" defined as

Double_t	GetSignalReferenceCut () const

Double_t	GetSignalReferenceCutOrientation () const

virtual Double_t	GetSignificance () const
	compute significance of mean difference

const Event *	GetTestingEvent (Long64_t ievt) const

Double_t	GetTestTime () const

const TString &	GetTestvarName () const

virtual Double_t	GetTrainingEfficiency (const TString &)

const Event *	GetTrainingEvent (Long64_t ievt) const

virtual const std::vector< Float_t > &	GetTrainingHistory (const char *)

UInt_t	GetTrainingROOTVersionCode () const

TString	GetTrainingROOTVersionString () const
	calculates the ROOT version string from the training version code on the fly

UInt_t	GetTrainingTMVAVersionCode () const

TString	GetTrainingTMVAVersionString () const
	calculates the TMVA version string from the training version code on the fly

Double_t	GetTrainTime () const

TransformationHandler &	GetTransformationHandler (Bool_t takeReroutedIfAvailable=true)

const TransformationHandler &	GetTransformationHandler (Bool_t takeReroutedIfAvailable=true) const

TString	GetWeightFileName () const
	retrieve weight file name

Double_t	GetXmax (Int_t ivar) const

Double_t	GetXmin (Int_t ivar) const

Bool_t	HasMVAPdfs () const

void	InitIPythonInteractive ()

Bool_t	IsModelPersistence () const

virtual Bool_t	IsSignalLike ()
	uses a pre-set cut on the MVA output (SetSignalReferenceCut and SetSignalReferenceCutOrientation) for a quick determination if an event would be selected as signal or background

virtual Bool_t	IsSignalLike (Double_t mvaVal)
	uses a pre-set cut on the MVA output (SetSignalReferenceCut and SetSignalReferenceCutOrientation) for a quick determination if an event with this mva output value would be selected as signal or background

Bool_t	IsSilentFile () const

virtual void	MakeClass (const TString &classFileName=TString("")) const
	create reader class for method (classification only at present)

TDirectory *	MethodBaseDir () const
	returns the ROOT directory where all instances of the corresponding MVA method are stored

virtual std::map< TString, Double_t >	OptimizeTuningParameters (TString fomType="ROCIntegral", TString fitType="FitGA")
	call the Optimizer with the set of parameters and ranges that are meant to be tuned.

void	PrintHelpMessage () const
	prints out method-specific help method

void	ProcessSetup ()
	process all options the "CheckForUnusedOptions" is done in an independent call, since it may be overridden by derived class (sometimes, eg, fitters are used which can only be implemented during training phase)

void	ReadStateFromFile ()
	Function to write options and weights to file.

void	ReadStateFromStream (std::istream &tf)
	read the header from the weight files of the different MVA methods

void	ReadStateFromStream (TFile &rf)
	write reference MVA distributions (and other information) to a ROOT type weight file

void	ReadStateFromXMLString (const char *xmlstr)
	for reading from memory

void	RerouteTransformationHandler (TransformationHandler *fTargetTransformation)

virtual void	Reset ()

virtual void	SetAnalysisType (Types::EAnalysisType type)

void	SetBaseDir (TDirectory *methodDir)

void	SetFile (TFile *file)

void	SetMethodBaseDir (TDirectory *methodDir)

void	SetMethodDir (TDirectory *methodDir)

void	SetModelPersistence (Bool_t status)

void	SetSignalReferenceCut (Double_t cut)

void	SetSignalReferenceCutOrientation (Double_t cutOrientation)

void	SetSilentFile (Bool_t status)

void	SetTestTime (Double_t testTime)

void	SetTestvarName (const TString &v="")

void	SetTrainTime (Double_t trainTime)

virtual void	SetTuneParameters (std::map< TString, Double_t > tuneParameters)
	set the tuning parameters according to the argument This is just a dummy .

void	SetupMethod ()
	setup of methods

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

virtual void	TestClassification ()
	initialization

virtual void	TestMulticlass ()
	test multiclass classification

virtual void	TestRegression (Double_t &bias, Double_t &biasT, Double_t &dev, Double_t &devT, Double_t &rms, Double_t &rmsT, Double_t &mInf, Double_t &mInfT, Double_t &corr, Types::ETreeType type)
	calculate <sum-of-deviation-squared> of regression output versus "true" value from test sample

bool	TrainingEnded ()

void	TrainMethod ()

virtual void	WriteEvaluationHistosToFile (Types::ETreeType treetype)
	writes all MVA evaluation histograms to file

virtual void	WriteMonitoringHistosToFile () const
	write special monitoring histograms to file dummy implementation here --------------—

void	WriteStateToFile () const
	write options and weights to file note that each one text file for the main configuration information and one ROOT file for ROOT objects are created

Public Member Functions inherited from TMVA::IMethod
	IMethod ()

virtual	~IMethod ()

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

Public Member Functions inherited from TMVA::Configurable
	Configurable (const TString &theOption="")
	constructor

virtual	~Configurable ()
	default destructor

void	AddOptionsXMLTo (void *parent) const
	write options to XML file

template<class T >
void	AddPreDefVal (const T &)

template<class T >
void	AddPreDefVal (const TString &optname, const T &)

void	CheckForUnusedOptions () const
	checks for unused options in option string

template<class T >
TMVA::OptionBase *	DeclareOptionRef (T &ref, const TString &name, const TString &desc)

template<class T >
OptionBase *	DeclareOptionRef (T &ref, const TString &name, const TString &desc="")

template<class T >
TMVA::OptionBase *	DeclareOptionRef (T *&ref, Int_t size, const TString &name, const TString &desc)

template<class T >
OptionBase *	DeclareOptionRef (T *&ref, Int_t size, const TString &name, const TString &desc="")

const char *	GetConfigDescription () const

const char *	GetConfigName () const

const TString &	GetOptions () const

MsgLogger &	Log () const

virtual void	ParseOptions ()
	options parser

void	PrintOptions () const
	prints out the options set in the options string and the defaults

void	ReadOptionsFromStream (std::istream &istr)
	read option back from the weight file

void	ReadOptionsFromXML (void *node)

void	SetConfigDescription (const char *d)

void	SetConfigName (const char *n)

void	SetMsgType (EMsgType t)

void	SetOptions (const TString &s)

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

void	WriteOptionsToStream (std::ostream &o, const TString &prefix) const
	write options to output stream (e.g. in writing the MVA weight files

Public Member Functions inherited from TNamed
	TNamed ()

	TNamed (const char name, const char title)

	TNamed (const TNamed &named)
	TNamed copy ctor.

	TNamed (const TString &name, const TString &title)

virtual	~TNamed ()
	TNamed destructor.

void	Clear (Option_t *option="") override
	Set name and title to empty strings ("").

TObject *	Clone (const char *newname="") const override
	Make a clone of an object using the Streamer facility.

Int_t	Compare (const TObject *obj) const override
	Compare two TNamed objects.

void	Copy (TObject &named) const override
	Copy this to obj.

virtual void	FillBuffer (char *&buffer)
	Encode TNamed into output buffer.

const char *	GetName () const override
	Returns name of object.

const char *	GetTitle () const override
	Returns title of object.

ULong_t	Hash () const override
	Return hash value for this object.

TClass *	IsA () const override

Bool_t	IsSortable () const override

void	ls (Option_t *option="") const override
	List TNamed name and title.

TNamed &	operator= (const TNamed &rhs)
	TNamed assignment operator.

void	Print (Option_t *option="") const override
	Print TNamed name and title.

virtual void	SetName (const char *name)
	Set the name of the TNamed.

virtual void	SetNameTitle (const char name, const char title)
	Set all the TNamed parameters (name and title).

virtual void	SetTitle (const char *title="")
	Set the title of the TNamed.

virtual Int_t	Sizeof () const
	Return size of the TNamed part of the TObject.

void	Streamer (TBuffer &) override
	Stream an object of class TObject.

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

Public Member Functions inherited from TObject
	TObject ()
	TObject constructor.

	TObject (const TObject &object)
	TObject copy ctor.

virtual	~TObject ()
	TObject destructor.

void	AbstractMethod (const char *method) const
	Use this method to implement an "abstract" method that you don't want to leave purely abstract.

virtual void	AppendPad (Option_t *option="")
	Append graphics object to current pad.

virtual void	Browse (TBrowser *b)
	Browse object. May be overridden for another default action.

ULong_t	CheckedHash ()
	Check and record whether this class has a consistent Hash/RecursiveRemove setup (*) and then return the regular Hash value for this object.

virtual const char *	ClassName () const
	Returns name of class to which the object belongs.

virtual void	Delete (Option_t *option="")
	Delete this object.

virtual Int_t	DistancetoPrimitive (Int_t px, Int_t py)
	Computes distance from point (px,py) to the object.

virtual void	Draw (Option_t *option="")
	Default Draw method for all objects.

virtual void	DrawClass () const
	Draw class inheritance tree of the class to which this object belongs.

virtual TObject *	DrawClone (Option_t *option="") const
	Draw a clone of this object in the current selected pad with: `gROOT->SetSelectedPad(c1)`.

virtual void	Dump () const
	Dump contents of object on stdout.

virtual void	Error (const char method, const char msgfmt,...) const
	Issue error message.

virtual void	Execute (const char method, const char params, Int_t *error=nullptr)
	Execute method on this object with the given parameter string, e.g.

virtual void	Execute (TMethod method, TObjArray params, Int_t *error=nullptr)
	Execute method on this object with parameters stored in the TObjArray.

virtual void	ExecuteEvent (Int_t event, Int_t px, Int_t py)
	Execute action corresponding to an event at (px,py).

virtual void	Fatal (const char method, const char msgfmt,...) const
	Issue fatal error message.

virtual TObject *	FindObject (const char *name) const
	Must be redefined in derived classes.

virtual TObject *	FindObject (const TObject *obj) const
	Must be redefined in derived classes.

virtual Option_t *	GetDrawOption () const
	Get option used by the graphics system to draw this object.

virtual const char *	GetIconName () const
	Returns mime type name of object.

virtual char *	GetObjectInfo (Int_t px, Int_t py) const
	Returns string containing info about the object at position (px,py).

virtual Option_t *	GetOption () const

virtual UInt_t	GetUniqueID () const
	Return the unique object id.

virtual Bool_t	HandleTimer (TTimer *timer)
	Execute action in response of a timer timing out.

Bool_t	HasInconsistentHash () const
	Return true is the type of this object is known to have an inconsistent setup for Hash and RecursiveRemove (i.e.

virtual void	Info (const char method, const char msgfmt,...) const
	Issue info message.

virtual Bool_t	InheritsFrom (const char *classname) const
	Returns kTRUE if object inherits from class "classname".

virtual Bool_t	InheritsFrom (const TClass *cl) const
	Returns kTRUE if object inherits from TClass cl.

virtual void	Inspect () const
	Dump contents of this object in a graphics canvas.

void	InvertBit (UInt_t f)

Bool_t	IsDestructed () const
	IsDestructed.

virtual Bool_t	IsEqual (const TObject *obj) const
	Default equal comparison (objects are equal if they have the same address in memory).

virtual Bool_t	IsFolder () const
	Returns kTRUE in case object contains browsable objects (like containers or lists of other objects).

R__ALWAYS_INLINE Bool_t	IsOnHeap () const

R__ALWAYS_INLINE Bool_t	IsZombie () const

void	MayNotUse (const char *method) const
	Use this method to signal that a method (defined in a base class) may not be called in a derived class (in principle against good design since a child class should not provide less functionality than its parent, however, sometimes it is necessary).

virtual Bool_t	Notify ()
	This method must be overridden to handle object notification (the base implementation is no-op).

void	Obsolete (const char method, const char asOfVers, const char *removedFromVers) const
	Use this method to declare a method obsolete.

void	operator delete (void *ptr)
	Operator delete.

void	operator delete (void ptr, void vp)
	Only called by placement new when throwing an exception.

void	operator delete[] (void *ptr)
	Operator delete [].

void	operator delete[] (void ptr, void vp)
	Only called by placement new[] when throwing an exception.

void *	operator new (size_t sz)

void *	operator new (size_t sz, void *vp)

void *	operator new[] (size_t sz)

void *	operator new[] (size_t sz, void *vp)

TObject &	operator= (const TObject &rhs)
	TObject assignment operator.

virtual void	Paint (Option_t *option="")
	This method must be overridden if a class wants to paint itself.

virtual void	Pop ()
	Pop on object drawn in a pad to the top of the display list.

virtual Int_t	Read (const char *name)
	Read contents of object with specified name from the current directory.

virtual void	RecursiveRemove (TObject *obj)
	Recursively remove this object from a list.

void	ResetBit (UInt_t f)

virtual void	SaveAs (const char filename="", Option_t option="") const
	Save this object in the file specified by filename.

virtual void	SavePrimitive (std::ostream &out, Option_t *option="")
	Save a primitive as a C++ statement(s) on output stream "out".

void	SetBit (UInt_t f)

void	SetBit (UInt_t f, Bool_t set)
	Set or unset the user status bits as specified in f.

virtual void	SetDrawOption (Option_t *option="")
	Set drawing option for object.

virtual void	SetUniqueID (UInt_t uid)
	Set the unique object id.

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

virtual void	SysError (const char method, const char msgfmt,...) const
	Issue system error message.

R__ALWAYS_INLINE Bool_t	TestBit (UInt_t f) const

Int_t	TestBits (UInt_t f) const

virtual void	UseCurrentStyle ()
	Set current style settings in this object This function is called when either TCanvas::UseCurrentStyle or TROOT::ForceStyle have been invoked.

virtual void	Warning (const char method, const char msgfmt,...) const
	Issue warning message.

virtual Int_t	Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0)
	Write this object to the current directory.

virtual Int_t	Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0) const
	Write this object to the current directory.

Static Public Member Functions
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

Static Public Member Functions inherited from TMVA::MethodBase
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

Static Public Member Functions inherited from TMVA::IMethod
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

Static Public Member Functions inherited from TMVA::Configurable
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

Static Public Member Functions inherited from TNamed
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

Static Public Member Functions inherited from TObject
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

static Longptr_t	GetDtorOnly ()
	Return destructor only flag.

static Bool_t	GetObjectStat ()
	Get status of object stat flag.

static void	SetDtorOnly (void *obj)
	Set destructor only flag.

static void	SetObjectStat (Bool_t stat)
	Turn on/off tracking of objects in the TObjectTable.

Private Member Functions
void	Init (void)
	common initialisation with defaults for the DT-Method

Private Attributes
Bool_t	fAutomatic
	use user given prune strength or automatically determined one using a validation sample

Double_t	fDeltaPruneStrength
	step size in pruning, is adjusted according to experience of previous trees

Double_t	fErrorFraction
	ntuple var: misclassification error fraction

std::vector< Event * >	fEventSample
	the training events

UInt_t	fMaxDepth
	max depth

Int_t	fMinNodeEvents
	min number of events in node

Float_t	fMinNodeSize
	min percentage of training events in node

TString	fMinNodeSizeS
	string containing min percentage of training events in node

Int_t	fNCuts
	grid used in cut applied in node splitting

Double_t	fNodePurityLimit
	purity limit for sig/bkg nodes

Bool_t	fPruneBeforeBoost
	ancient variable, only needed for "CompatibilityOptions"

DecisionTree::EPruneMethod	fPruneMethod
	method used for pruning

TString	fPruneMethodS
	prune method option String

Double_t	fPruneStrength
	a parameter to set the "amount" of pruning..needs to be adjusted

Bool_t	fRandomisedTrees
	choose a random subset of possible cut variables at each node during training

SeparationBase *	fSepType
	the separation used in node splitting

TString	fSepTypeS
	the separation (option string) used in node splitting

DecisionTree *	fTree
	the decision tree

Int_t	fUseNvars
	the number of variables used in the randomised tree splitting

Bool_t	fUsePoissonNvars
	fUseNvars is used as a poisson mean, and the actual value of useNvars is at each step drawn form that distribution

Bool_t	fUseYesNoLeaf
	use sig or bkg classification in leave nodes or sig/bkg

std::vector< Double_t >	fVariableImportance
	the relative importance of the different variables

Static Private Attributes
static const Int_t	fgDebugLevel = 0
	debug level determining some printout/control plots etc.

Additional Inherited Members
Public Types inherited from TMVA::MethodBase
enum	EWeightFileType { kROOT =0 , kTEXT }

Public Types inherited from TObject
enum	{ kIsOnHeap = 0x01000000 , kNotDeleted = 0x02000000 , kZombie = 0x04000000 , kInconsistent = 0x08000000 , kBitMask = 0x00ffffff }

enum	{ kSingleKey = (1ULL << ( 0 )) , kOverwrite = (1ULL << ( 1 )) , kWriteDelete = (1ULL << ( 2 )) }

enum	EDeprecatedStatusBits { kObjInCanvas = (1ULL << ( 3 )) }

enum	EStatusBits { kCanDelete = (1ULL << ( 0 )) , kMustCleanup = (1ULL << ( 3 )) , kIsReferenced = (1ULL << ( 4 )) , kHasUUID = (1ULL << ( 5 )) , kCannotPick = (1ULL << ( 6 )) , kNoContextMenu = (1ULL << ( 8 )) , kInvalidObject = (1ULL << ( 13 )) }

Public Attributes inherited from TMVA::MethodBase
Bool_t	fSetupCompleted

TrainingHistory	fTrainHistory

Protected Types inherited from TObject
enum	{ kOnlyPrepStep = (1ULL << ( 3 )) }

Protected Member Functions inherited from TMVA::MethodBase
virtual std::vector< Double_t >	GetDataMvaValues (DataSet *data=nullptr, Long64_t firstEvt=0, Long64_t lastEvt=-1, Bool_t logProgress=false)
	get all the MVA values for the events of the given Data type

const TString &	GetInternalVarName (Int_t ivar) const

virtual std::vector< Double_t >	GetMvaValues (Long64_t firstEvt=0, Long64_t lastEvt=-1, Bool_t logProgress=false)
	get all the MVA values for the events of the current Data type

const TString &	GetOriginalVarName (Int_t ivar) const

const TString &	GetWeightFileDir () const

Bool_t	HasTrainingTree () const

Bool_t	Help () const

Bool_t	IgnoreEventsWithNegWeightsInTraining () const

Bool_t	IsConstructedFromWeightFile () const

Bool_t	IsNormalised () const

virtual void	MakeClassSpecific (std::ostream &, const TString &="") const

virtual void	MakeClassSpecificHeader (std::ostream &, const TString &="") const

void	NoErrorCalc (Double_t const err, Double_t const errUpper)

void	SetNormalised (Bool_t norm)

void	SetWeightFileDir (TString fileDir)
	set directory of weight file

void	SetWeightFileName (TString)
	set the weight file name (depreciated)

void	Statistics (Types::ETreeType treeType, const TString &theVarName, Double_t &, Double_t &, Double_t &, Double_t &, Double_t &, Double_t &)
	calculates rms,mean, xmin, xmax of the event variable this can be either done for the variables as they are or for normalised variables (in the range of 0-1) if "norm" is set to kTRUE

Bool_t	TxtWeightsOnly () const

Bool_t	Verbose () const

Protected Member Functions inherited from TMVA::Configurable
void	EnableLooseOptions (Bool_t b=kTRUE)

const TString &	GetReferenceFile () const

Bool_t	LooseOptionCheckingEnabled () const

void	ResetSetFlag ()
	resets the IsSet flag for all declare options to be called before options are read from stream

void	WriteOptionsReferenceToFile ()
	write complete options to output stream

Protected Member Functions inherited from TObject
virtual void	DoError (int level, const char location, const char fmt, va_list va) const
	Interface to ErrorHandler (protected).

void	MakeZombie ()

Protected Attributes inherited from TMVA::MethodBase
Types::EAnalysisType	fAnalysisType

UInt_t	fBackgroundClass

bool	fExitFromTraining = false

std::vector< TString > *	fInputVars

IPythonInteractive *	fInteractive = nullptr
	temporary dataset used when evaluating on a different data (used by MethodCategory::GetMvaValues)

UInt_t	fIPyCurrentIter = 0

UInt_t	fIPyMaxIter = 0

std::vector< Float_t > *	fMulticlassReturnVal

Int_t	fNbins

Int_t	fNbinsH

Int_t	fNbinsMVAoutput

Ranking *	fRanking

std::vector< Float_t > *	fRegressionReturnVal

Results *	fResults

UInt_t	fSignalClass

DataSet *	fTmpData = nullptr
	temporary event when testing on a different DataSet than the own one

const Event *	fTmpEvent

Protected Attributes inherited from TMVA::Configurable
MsgLogger *	fLogger
	! message logger

Protected Attributes inherited from TNamed
TString	fName

TString	fTitle

#include <TMVA/MethodDT.h>

Inheritance diagram for TMVA::MethodDT:

[legend]

Constructor & Destructor Documentation

◆ MethodDT() [1/2]

TMVA::MethodDT::MethodDT	(	const TString &	jobName,
		const TString &	methodTitle,
		DataSetInfo &	theData,
		const TString &	theOption = `""`
	)

the standard constructor for just an ordinar "decision trees"

Definition at line 127 of file MethodDT.cxx.

◆ MethodDT() [2/2]

TMVA::MethodDT::MethodDT	(	DataSetInfo &	dsi,
		const TString &	theWeightFile
	)

constructor from Reader

Definition at line 155 of file MethodDT.cxx.

◆ ~MethodDT()

TMVA::MethodDT::~MethodDT ( void )

virtual

destructor

Definition at line 368 of file MethodDT.cxx.

Member Function Documentation

◆ AddWeightsXMLTo()

void TMVA::MethodDT::AddWeightsXMLTo ( void * parent ) const

virtual

Implements TMVA::MethodBase.

Definition at line 523 of file MethodDT.cxx.

◆ Class()

static TClass * TMVA::MethodDT::Class ( )

static

Returns: TClass describing this class

◆ Class_Name()

static const char * TMVA::MethodDT::Class_Name ( )

static

Returns: Name of this class

◆ Class_Version()

static constexpr Version_t TMVA::MethodDT::Class_Version ( )

inlinestaticconstexpr

Returns: Version of this class

Definition at line 139 of file MethodDT.h.

◆ CreateRanking()

const TMVA::Ranking * TMVA::MethodDT::CreateRanking ( )

virtual

Implements TMVA::MethodBase.

Definition at line 566 of file MethodDT.cxx.

◆ DeclareCompatibilityOptions()

void TMVA::MethodDT::DeclareCompatibilityOptions ( )

virtual

options that are used ONLY for the READER to ensure backward compatibility

Reimplemented from TMVA::MethodBase.

Definition at line 245 of file MethodDT.cxx.

◆ DeclareOptions()

void TMVA::MethodDT::DeclareOptions ( )

virtual

Define the options (their key words) that can be set in the option string.

UseRandomisedTrees choose at each node splitting a random set of variables
UseNvars use UseNvars variables in randomised trees
SeparationType the separation criterion applied in the node splitting. known:
- GiniIndex
- MisClassificationError
- CrossEntropy
- SDivSqrtSPlusB
nEventsMin: the minimum number of events in a node (leaf criteria, stop splitting)
nCuts: the number of steps in the optimisation of the cut for a node (if < 0, then step size is determined by the events)
UseYesNoLeaf decide if the classification is done simply by the node type, or the S/B (from the training) in the leaf node
NodePurityLimit the minimum purity to classify a node as a signal node (used in pruning and boosting to determine misclassification error rate)
PruneMethod The Pruning method: known:
- NoPruning // switch off pruning completely
- ExpectedError
- CostComplexity
PruneStrength a parameter to adjust the amount of pruning. Should be large enough such that overtraining is avoided");

Implements TMVA::MethodBase.

Definition at line 212 of file MethodDT.cxx.

◆ DeclFileName()

static const char * TMVA::MethodDT::DeclFileName ( )

inlinestatic

Returns: Name of the file containing the class declaration

Definition at line 139 of file MethodDT.h.

◆ GetHelpMessage()

void TMVA::MethodDT::GetHelpMessage ( ) const

virtual

Implements TMVA::IMethod.

Definition at line 561 of file MethodDT.cxx.

◆ GetMvaValue()

Double_t TMVA::MethodDT::GetMvaValue	(	Double_t *	err = `nullptr`,
		Double_t *	errUpper = `nullptr`
	)

virtual

returns MVA value

Implements TMVA::MethodBase.

Definition at line 551 of file MethodDT.cxx.

◆ GetNNodes()

Int_t TMVA::MethodDT::GetNNodes ( )

inline

Definition at line 97 of file MethodDT.h.

◆ GetNNodesBeforePruning()

Int_t TMVA::MethodDT::GetNNodesBeforePruning ( )

inline

Definition at line 96 of file MethodDT.h.

◆ GetPruneStrength()

Double_t TMVA::MethodDT::GetPruneStrength ( )

inline

Definition at line 91 of file MethodDT.h.

◆ HasAnalysisType()

Bool_t TMVA::MethodDT::HasAnalysisType	(	Types::EAnalysisType	type,
		UInt_t	numberClasses,
		UInt_t	numberTargets
	)

virtual

FDA can handle classification with 2 classes and regression with one regression-target.

Implements TMVA::IMethod.

Definition at line 180 of file MethodDT.cxx.

◆ Init()

void TMVA::MethodDT::Init ( void )

privatevirtual

common initialisation with defaults for the DT-Method

Implements TMVA::MethodBase.

Definition at line 343 of file MethodDT.cxx.

◆ IsA()

virtual TClass * TMVA::MethodDT::IsA ( ) const

inlinevirtual

Returns: TClass describing current object

Reimplemented from TMVA::MethodBase.

Definition at line 139 of file MethodDT.h.

◆ ProcessOptions()

void TMVA::MethodDT::ProcessOptions ( )

virtual

the option string is decoded, for available options see "DeclareOptions"

Implements TMVA::MethodBase.

Definition at line 256 of file MethodDT.cxx.

◆ PruneTree()

Double_t TMVA::MethodDT::PruneTree ( )

prune the decision tree if requested (good for individual trees that are best grown out, and then pruned back, while boosted decision trees are best 'small' trees to start with.

Well, at least the standard "optimal pruning algorithms" don't result in 'weak enough' classifiers !!

Definition at line 406 of file MethodDT.cxx.

◆ ReadWeightsFromStream() [1/3]

virtual void TMVA::MethodBase::ReadWeightsFromStream ( std::istream & )

virtual

Implements TMVA::MethodBase.

◆ ReadWeightsFromStream() [2/3]

void TMVA::MethodDT::ReadWeightsFromStream ( std::istream & istr )

virtual

Implements TMVA::MethodBase.

Definition at line 541 of file MethodDT.cxx.

◆ ReadWeightsFromStream() [3/3]

virtual void TMVA::MethodBase::ReadWeightsFromStream ( TFile & )

inlinevirtual

Reimplemented from TMVA::MethodBase.

Definition at line 266 of file MethodBase.h.

◆ ReadWeightsFromXML()

void TMVA::MethodDT::ReadWeightsFromXML ( void * wghtnode )

virtual

Implements TMVA::MethodBase.

Definition at line 531 of file MethodDT.cxx.

◆ SetMinNodeSize() [1/2]

void TMVA::MethodDT::SetMinNodeSize ( Double_t sizeInPercent )

Definition at line 320 of file MethodDT.cxx.

◆ SetMinNodeSize() [2/2]

void TMVA::MethodDT::SetMinNodeSize ( TString sizeInPercent )

Definition at line 331 of file MethodDT.cxx.

◆ Streamer()

virtual void TMVA::MethodDT::Streamer ( TBuffer & )

virtual

Reimplemented from TMVA::MethodBase.

◆ StreamerNVirtual()

void TMVA::MethodDT::StreamerNVirtual ( TBuffer & ClassDef_StreamerNVirtual_b )

inline

Definition at line 139 of file MethodDT.h.

◆ TestTreeQuality()

Double_t TMVA::MethodDT::TestTreeQuality ( DecisionTree * dt )

Definition at line 506 of file MethodDT.cxx.

◆ Train()

void TMVA::MethodDT::Train ( void )

virtual

Implements TMVA::MethodBase.

Definition at line 375 of file MethodDT.cxx.

Member Data Documentation

◆ fAutomatic

Bool_t TMVA::MethodDT::fAutomatic

private

use user given prune strength or automatically determined one using a validation sample

Definition at line 126 of file MethodDT.h.

◆ fDeltaPruneStrength

Double_t TMVA::MethodDT::fDeltaPruneStrength

private

step size in pruning, is adjusted according to experience of previous trees

Definition at line 132 of file MethodDT.h.

◆ fErrorFraction

Double_t TMVA::MethodDT::fErrorFraction

private

ntuple var: misclassification error fraction

Definition at line 122 of file MethodDT.h.

◆ fEventSample

std::vector<Event*> TMVA::MethodDT::fEventSample

private

the training events

Definition at line 106 of file MethodDT.h.

◆ fgDebugLevel

const Int_t TMVA::MethodDT::fgDebugLevel = 0

staticprivate

debug level determining some printout/control plots etc.

Definition at line 134 of file MethodDT.h.

◆ fMaxDepth

UInt_t TMVA::MethodDT::fMaxDepth

private

max depth

Definition at line 119 of file MethodDT.h.

◆ fMinNodeEvents

Int_t TMVA::MethodDT::fMinNodeEvents

private

min number of events in node

Definition at line 112 of file MethodDT.h.

◆ fMinNodeSize

Float_t TMVA::MethodDT::fMinNodeSize

private

min percentage of training events in node

Definition at line 113 of file MethodDT.h.

◆ fMinNodeSizeS

TString TMVA::MethodDT::fMinNodeSizeS

private

string containing min percentage of training events in node

Definition at line 114 of file MethodDT.h.

◆ fNCuts

Int_t TMVA::MethodDT::fNCuts

private

grid used in cut applied in node splitting

Definition at line 116 of file MethodDT.h.

◆ fNodePurityLimit

Double_t TMVA::MethodDT::fNodePurityLimit

private

purity limit for sig/bkg nodes

Definition at line 118 of file MethodDT.h.

◆ fPruneBeforeBoost

Bool_t TMVA::MethodDT::fPruneBeforeBoost

private

ancient variable, only needed for "CompatibilityOptions"

Definition at line 137 of file MethodDT.h.

◆ fPruneMethod

DecisionTree::EPruneMethod TMVA::MethodDT::fPruneMethod

private

method used for pruning

Definition at line 124 of file MethodDT.h.

◆ fPruneMethodS

TString TMVA::MethodDT::fPruneMethodS

private

prune method option String

Definition at line 125 of file MethodDT.h.

◆ fPruneStrength

Double_t TMVA::MethodDT::fPruneStrength

private

a parameter to set the "amount" of pruning..needs to be adjusted

Definition at line 123 of file MethodDT.h.

◆ fRandomisedTrees

Bool_t TMVA::MethodDT::fRandomisedTrees

private

choose a random subset of possible cut variables at each node during training

Definition at line 127 of file MethodDT.h.

◆ fSepType

SeparationBase* TMVA::MethodDT::fSepType

private

the separation used in node splitting

Definition at line 110 of file MethodDT.h.

◆ fSepTypeS

TString TMVA::MethodDT::fSepTypeS

private

the separation (option string) used in node splitting

Definition at line 111 of file MethodDT.h.

◆ fTree

DecisionTree* TMVA::MethodDT::fTree

private

the decision tree

Definition at line 108 of file MethodDT.h.

◆ fUseNvars

Int_t TMVA::MethodDT::fUseNvars

private

the number of variables used in the randomised tree splitting

Definition at line 128 of file MethodDT.h.

◆ fUsePoissonNvars

Bool_t TMVA::MethodDT::fUsePoissonNvars

private

fUseNvars is used as a poisson mean, and the actual value of useNvars is at each step drawn form that distribution

Definition at line 129 of file MethodDT.h.

◆ fUseYesNoLeaf

Bool_t TMVA::MethodDT::fUseYesNoLeaf

private

use sig or bkg classification in leave nodes or sig/bkg

Definition at line 117 of file MethodDT.h.

◆ fVariableImportance

std::vector<Double_t> TMVA::MethodDT::fVariableImportance

private

the relative importance of the different variables

Definition at line 130 of file MethodDT.h.

Libraries for TMVA::MethodDT:

[legend]

The documentation for this class was generated from the following files:

tmva/tmva/inc/TMVA/MethodDT.h
tmva/tmva/src/MethodDT.cxx

Decision trees:

Boosting:

Bagging:

Analysis:

Public Member Functions

Static Public Member Functions

Private Member Functions

Private Attributes

Static Private Attributes

Additional Inherited Members

Constructor & Destructor Documentation

◆ MethodDT() [1/2]

◆ MethodDT() [2/2]

◆ ~MethodDT()

Member Function Documentation

◆ AddWeightsXMLTo()

◆ Class()

◆ Class_Name()

◆ Class_Version()

◆ CreateRanking()

◆ DeclareCompatibilityOptions()

◆ DeclareOptions()

◆ DeclFileName()

◆ GetHelpMessage()

◆ GetMvaValue()

◆ GetNNodes()

◆ GetNNodesBeforePruning()

◆ GetPruneStrength()

◆ HasAnalysisType()

◆ Init()

◆ IsA()

◆ ProcessOptions()

◆ PruneTree()

◆ ReadWeightsFromStream() [1/3]

◆ ReadWeightsFromStream() [2/3]

◆ ReadWeightsFromStream() [3/3]

◆ ReadWeightsFromXML()

◆ SetMinNodeSize() [1/2]

◆ SetMinNodeSize() [2/2]

◆ Streamer()

◆ StreamerNVirtual()

◆ TestTreeQuality()

◆ Train()

Member Data Documentation

◆ fAutomatic

◆ fDeltaPruneStrength

◆ fErrorFraction

◆ fEventSample

◆ fgDebugLevel

◆ fMaxDepth

◆ fMinNodeEvents

◆ fMinNodeSize

◆ fMinNodeSizeS

◆ fNCuts

◆ fNodePurityLimit

◆ fPruneBeforeBoost

◆ fPruneMethod

◆ fPruneMethodS

◆ fPruneStrength

◆ fRandomisedTrees

◆ fSepType

◆ fSepTypeS

◆ fTree

◆ fUseNvars

◆ fUsePoissonNvars

◆ fUseYesNoLeaf

◆ fVariableImportance