==> Start TMVAMulticlass
Creating testdata....
... event: 0 (2000)
... event: 1000 (2000)
======> EVENT:0
var1 = -1.14361
var2 = -0.822373
var3 = -0.395426
var4 = -0.529427
created tree: TreeS
... event: 0 (2000)
... event: 1000 (2000)
======> EVENT:0
var1 = -1.54361
var2 = -1.42237
var3 = -1.39543
var4 = -2.02943
created tree: TreeB0
... event: 0 (2000)
... event: 1000 (2000)
======> EVENT:0
var1 = -1.54361
var2 = -0.822373
var3 = -0.395426
var4 = -2.02943
created tree: TreeB1
======> EVENT:0
var1 = 0.463304
var2 = 1.37192
var3 = -1.16769
var4 = -1.77551
created tree: TreeB2
created data file: tmva_example_multiple_background.root
created tmva_example_multiple_background.root for tests of the multiclass features
DataSetInfo : [dataset] : Added class "Signal"
: Add Tree TreeS of type Signal with 2000 events
DataSetInfo : [dataset] : Added class "bg0"
: Add Tree TreeB0 of type bg0 with 2000 events
DataSetInfo : [dataset] : Added class "bg1"
: Add Tree TreeB1 of type bg1 with 2000 events
DataSetInfo : [dataset] : Added class "bg2"
: Add Tree TreeB2 of type bg2 with 2000 events
: Dataset[dataset] : Class index : 0 name : Signal
: Dataset[dataset] : Class index : 1 name : bg0
: Dataset[dataset] : Class index : 2 name : bg1
: Dataset[dataset] : Class index : 3 name : bg2
Factory : Booking method: ␛[1mBDTG␛[0m
:
: the option NegWeightTreatment=InverseBoostNegWeights does not exist for BoostType=Grad
: --> change to new default NegWeightTreatment=Pray
DataSetFactory : [dataset] : Number of events in input trees
:
:
:
:
: Number of training and testing events
: ---------------------------------------------------------------------------
: Signal -- training events : 1000
: Signal -- testing events : 1000
: Signal -- training and testing events: 2000
: bg0 -- training events : 1000
: bg0 -- testing events : 1000
: bg0 -- training and testing events: 2000
: bg1 -- training events : 1000
: bg1 -- testing events : 1000
: bg1 -- training and testing events: 2000
: bg2 -- training events : 1000
: bg2 -- testing events : 1000
: bg2 -- training and testing events: 2000
:
DataSetInfo : Correlation matrix (Signal):
: ----------------------------------------
: var1 var2 var3 var4
: var1: +1.000 +0.397 +0.623 +0.832
: var2: +0.397 +1.000 +0.716 +0.737
: var3: +0.623 +0.716 +1.000 +0.859
: var4: +0.832 +0.737 +0.859 +1.000
: ----------------------------------------
DataSetInfo : Correlation matrix (bg0):
: ----------------------------------------
: var1 var2 var3 var4
: var1: +1.000 +0.365 +0.592 +0.811
: var2: +0.365 +1.000 +0.708 +0.740
: var3: +0.592 +0.708 +1.000 +0.859
: var4: +0.811 +0.740 +0.859 +1.000
: ----------------------------------------
DataSetInfo : Correlation matrix (bg1):
: ----------------------------------------
: var1 var2 var3 var4
: var1: +1.000 +0.407 +0.610 +0.834
: var2: +0.407 +1.000 +0.710 +0.741
: var3: +0.610 +0.710 +1.000 +0.851
: var4: +0.834 +0.741 +0.851 +1.000
: ----------------------------------------
DataSetInfo : Correlation matrix (bg2):
: ----------------------------------------
: var1 var2 var3 var4
: var1: +1.000 -0.647 -0.016 -0.013
: var2: -0.647 +1.000 +0.015 +0.002
: var3: -0.016 +0.015 +1.000 -0.024
: var4: -0.013 +0.002 -0.024 +1.000
: ----------------------------------------
DataSetFactory : [dataset] :
:
Factory : Booking method: ␛[1mMLP␛[0m
:
MLP : Building Network.
: Initializing weights
Factory : ␛[1mTrain all methods␛[0m
Factory : [dataset] : Create Transformation "I" with events from all classes.
:
: Transformation, Variable selection :
: Input : variable 'var1' <---> Output : variable 'var1'
: Input : variable 'var2' <---> Output : variable 'var2'
: Input : variable 'var3' <---> Output : variable 'var3'
: Input : variable 'var4' <---> Output : variable 'var4'
Factory : [dataset] : Create Transformation "D" with events from all classes.
:
: Transformation, Variable selection :
: Input : variable 'var1' <---> Output : variable 'var1'
: Input : variable 'var2' <---> Output : variable 'var2'
: Input : variable 'var3' <---> Output : variable 'var3'
: Input : variable 'var4' <---> Output : variable 'var4'
Factory : [dataset] : Create Transformation "P" with events from all classes.
:
: Transformation, Variable selection :
: Input : variable 'var1' <---> Output : variable 'var1'
: Input : variable 'var2' <---> Output : variable 'var2'
: Input : variable 'var3' <---> Output : variable 'var3'
: Input : variable 'var4' <---> Output : variable 'var4'
Factory : [dataset] : Create Transformation "G" with events from all classes.
:
: Transformation, Variable selection :
: Input : variable 'var1' <---> Output : variable 'var1'
: Input : variable 'var2' <---> Output : variable 'var2'
: Input : variable 'var3' <---> Output : variable 'var3'
: Input : variable 'var4' <---> Output : variable 'var4'
Factory : [dataset] : Create Transformation "D" with events from all classes.
:
: Transformation, Variable selection :
: Input : variable 'var1' <---> Output : variable 'var1'
: Input : variable 'var2' <---> Output : variable 'var2'
: Input : variable 'var3' <---> Output : variable 'var3'
: Input : variable 'var4' <---> Output : variable 'var4'
TFHandler_Factory : Variable Mean RMS [ Min Max ]
: -----------------------------------------------------------
: var1: 0.047647 1.0025 [ -3.6592 3.2645 ]
: var2: 0.32647 1.0646 [ -3.6891 3.7877 ]
: var3: 0.11493 1.1230 [ -4.5727 4.5640 ]
: var4: -0.076531 1.2652 [ -4.8486 5.0412 ]
: -----------------------------------------------------------
: Preparing the Decorrelation transformation...
TFHandler_Factory : Variable Mean RMS [ Min Max ]
: -----------------------------------------------------------
: var1: 0.082544 1.0000 [ -3.6274 3.1017 ]
: var2: 0.36715 1.0000 [ -3.3020 3.4950 ]
: var3: 0.066865 1.0000 [ -2.9882 3.3086 ]
: var4: -0.20593 1.0000 [ -3.3088 2.8423 ]
: -----------------------------------------------------------
: Preparing the Principle Component (PCA) transformation...
TFHandler_Factory : Variable Mean RMS [ Min Max ]
: -----------------------------------------------------------
: var1: 5.7502e-10 1.8064 [ -8.0344 7.8312 ]
: var2:-1.6078e-11 0.90130 [ -2.6765 2.7523 ]
: var3: 3.0841e-10 0.73386 [ -2.6572 2.2255 ]
: var4:-2.6886e-10 0.62168 [ -1.7384 2.2297 ]
: -----------------------------------------------------------
: Preparing the Gaussian transformation...
: Preparing the Decorrelation transformation...
TFHandler_Factory : Variable Mean RMS [ Min Max ]
: -----------------------------------------------------------
: var1: 0.013510 1.0000 [ -2.6520 6.2074 ]
: var2: 0.0096839 1.0000 [ -2.8402 6.3073 ]
: var3: 0.010397 1.0000 [ -3.0251 5.8860 ]
: var4: 0.0053980 1.0000 [ -3.0998 5.7078 ]
: -----------------------------------------------------------
: Ranking input variables (method unspecific)...
Factory : Train method: BDTG for Multiclass classification
:
: Training 1000 Decision Trees ... patience please
: Elapsed time for training with 4000 events: 5.57 sec
: Dataset[dataset] : Create results for training
: Dataset[dataset] : Multiclass evaluation of BDTG on training sample
: Dataset[dataset] : Elapsed time for evaluation of 4000 events: 1.67 sec
: Creating multiclass response histograms...
: Creating multiclass performance histograms...
: Creating xml weight file: ␛[0;36mdataset/weights/TMVAMulticlass_BDTG.weights.xml␛[0m
: Creating standalone class: ␛[0;36mdataset/weights/TMVAMulticlass_BDTG.class.C␛[0m
: TMVAMulticlass.root:/dataset/Method_BDT/BDTG
Factory : Training finished
:
Factory : Train method: MLP for Multiclass classification
:
: Training Network
:
: Elapsed time for training with 4000 events: 24.1 sec
: Dataset[dataset] : Create results for training
: Dataset[dataset] : Multiclass evaluation of MLP on training sample
: Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.0147 sec
: Creating multiclass response histograms...
: Creating multiclass performance histograms...
: Creating xml weight file: ␛[0;36mdataset/weights/TMVAMulticlass_MLP.weights.xml␛[0m
: Creating standalone class: ␛[0;36mdataset/weights/TMVAMulticlass_MLP.class.C␛[0m
: Write special histos to file: TMVAMulticlass.root:/dataset/Method_MLP/MLP
Factory : Training finished
:
: Ranking input variables (method specific)...
BDTG : Ranking result (top variable is best ranked)
: --------------------------------------
: Rank : Variable : Variable Importance
: --------------------------------------
: 1 : var4 : 3.117e-01
: 2 : var1 : 2.504e-01
: 3 : var2 : 2.430e-01
: 4 : var3 : 1.949e-01
: --------------------------------------
MLP : Ranking result (top variable is best ranked)
: -----------------------------
: Rank : Variable : Importance
: -----------------------------
: 1 : var4 : 6.076e+01
: 2 : var2 : 4.824e+01
: 3 : var1 : 2.116e+01
: 4 : var3 : 1.692e+01
: -----------------------------
Factory : === Destroy and recreate all methods via weight files for testing ===
:
: Reading weight file: ␛[0;36mdataset/weights/TMVAMulticlass_BDTG.weights.xml␛[0m
: Reading weight file: ␛[0;36mdataset/weights/TMVAMulticlass_MLP.weights.xml␛[0m
MLP : Building Network.
: Initializing weights
Factory : ␛[1mTest all methods␛[0m
Factory : Test method: BDTG for Multiclass classification performance
:
: Dataset[dataset] : Create results for testing
: Dataset[dataset] : Multiclass evaluation of BDTG on testing sample
: Dataset[dataset] : Elapsed time for evaluation of 4000 events: 1.03 sec
: Creating multiclass response histograms...
: Creating multiclass performance histograms...
Factory : Test method: MLP for Multiclass classification performance
:
: Dataset[dataset] : Create results for testing
: Dataset[dataset] : Multiclass evaluation of MLP on testing sample
: Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.0146 sec
: Creating multiclass response histograms...
: Creating multiclass performance histograms...
Factory : ␛[1mEvaluate all methods␛[0m
: Evaluate multiclass classification method: BDTG
: Creating multiclass response histograms...
: Creating multiclass performance histograms...
: Creating multiclass response histograms...
: Creating multiclass performance histograms...
TFHandler_BDTG : Variable Mean RMS [ Min Max ]
: -----------------------------------------------------------
: var1: 0.070153 1.0224 [ -4.0592 3.5808 ]
: var2: 0.30372 1.0460 [ -3.6952 3.7877 ]
: var3: 0.12152 1.1222 [ -3.6800 3.9200 ]
: var4: -0.072602 1.2766 [ -4.8486 4.2221 ]
: -----------------------------------------------------------
: Evaluate multiclass classification method: MLP
: Creating multiclass response histograms...
: Creating multiclass performance histograms...
: Creating multiclass response histograms...
: Creating multiclass performance histograms...
TFHandler_MLP : Variable Mean RMS [ Min Max ]
: -----------------------------------------------------------
: var1: 0.070153 1.0224 [ -4.0592 3.5808 ]
: var2: 0.30372 1.0460 [ -3.6952 3.7877 ]
: var3: 0.12152 1.1222 [ -3.6800 3.9200 ]
: var4: -0.072602 1.2766 [ -4.8486 4.2221 ]
: -----------------------------------------------------------
:
: 1-vs-rest performance metrics per class
: -------------------------------------------------------------------------------------------------------
:
: Considers the listed class as signal and the other classes
: as background, reporting the resulting binary performance.
: A score of 0.820 (0.850) means 0.820 was acheived on the
: test set and 0.850 on the training set.
:
: Dataset MVA Method ROC AUC Sig eff@B=0.01 Sig eff@B=0.10 Sig eff@B=0.30
: Name: / Class: test (train) test (train) test (train) test (train)
:
: dataset BDTG
: ------------------------------
: Signal 0.968 (0.978) 0.508 (0.605) 0.914 (0.945) 0.990 (0.996)
: bg0 0.910 (0.931) 0.256 (0.288) 0.737 (0.791) 0.922 (0.956)
: bg1 0.947 (0.954) 0.437 (0.511) 0.833 (0.856) 0.971 (0.971)
: bg2 0.978 (0.982) 0.585 (0.678) 0.951 (0.956) 0.999 (0.996)
:
: dataset MLP
: ------------------------------
: Signal 0.970 (0.975) 0.596 (0.632) 0.933 (0.938) 0.988 (0.993)
: bg0 0.929 (0.934) 0.303 (0.298) 0.787 (0.793) 0.949 (0.961)
: bg1 0.962 (0.967) 0.467 (0.553) 0.881 (0.906) 0.985 (0.992)
: bg2 0.975 (0.979) 0.629 (0.699) 0.929 (0.940) 0.998 (0.998)
:
: -------------------------------------------------------------------------------------------------------
:
:
: Confusion matrices for all methods
: -------------------------------------------------------------------------------------------------------
:
: Does a binary comparison between the two classes given by a
: particular row-column combination. In each case, the class
: given by the row is considered signal while the class given
: by the column index is considered background.
:
: === Showing confusion matrix for method : BDTG
: (Signal Efficiency for Background Efficiency 0.01%)
: ---------------------------------------------------
: Signal bg0 bg1 bg2
: test (train) test (train) test (train) test (train)
: Signal - 0.497 (0.373) 0.710 (0.693) 0.680 (0.574)
: bg0 0.271 (0.184) - 0.239 (0.145) 0.705 (0.667)
: bg1 0.855 (0.766) 0.369 (0.222) - 0.587 (0.578)
: bg2 0.714 (0.585) 0.705 (0.581) 0.648 (0.601) -
:
: (Signal Efficiency for Background Efficiency 0.10%)
: ---------------------------------------------------
: Signal bg0 bg1 bg2
: test (train) test (train) test (train) test (train)
: Signal - 0.911 (0.853) 0.991 (0.981) 0.945 (0.913)
: bg0 0.833 (0.774) - 0.654 (0.582) 0.930 (0.901)
: bg1 0.971 (0.980) 0.716 (0.681) - 0.871 (0.862)
: bg2 0.976 (0.951) 0.971 (0.973) 0.936 (0.941) -
:
: (Signal Efficiency for Background Efficiency 0.30%)
: ---------------------------------------------------
: Signal bg0 bg1 bg2
: test (train) test (train) test (train) test (train)
: Signal - 0.978 (0.957) 0.999 (1.000) 0.998 (0.997)
: bg0 0.965 (0.926) - 0.874 (0.835) 0.991 (0.976)
: bg1 1.000 (0.999) 0.916 (0.894) - 0.988 (0.985)
: bg2 0.999 (0.999) 0.997 (0.999) 0.996 (0.997) -
:
: === Showing confusion matrix for method : MLP
: (Signal Efficiency for Background Efficiency 0.01%)
: ---------------------------------------------------
: Signal bg0 bg1 bg2
: test (train) test (train) test (train) test (train)
: Signal - 0.465 (0.490) 0.974 (0.953) 0.632 (0.498)
: bg0 0.320 (0.269) - 0.224 (0.250) 0.655 (0.627)
: bg1 0.943 (0.920) 0.341 (0.275) - 0.632 (0.687)
: bg2 0.665 (0.642) 0.697 (0.680) 0.706 (0.598) -
:
: (Signal Efficiency for Background Efficiency 0.10%)
: ---------------------------------------------------
: Signal bg0 bg1 bg2
: test (train) test (train) test (train) test (train)
: Signal - 0.865 (0.854) 0.996 (0.994) 0.908 (0.907)
: bg0 0.784 (0.776) - 0.666 (0.655) 0.919 (0.895)
: bg1 0.998 (0.998) 0.791 (0.785) - 0.912 (0.902)
: bg2 0.943 (0.903) 0.946 (0.939) 0.924 (0.928) -
:
: (Signal Efficiency for Background Efficiency 0.30%)
: ---------------------------------------------------
: Signal bg0 bg1 bg2
: test (train) test (train) test (train) test (train)
: Signal - 0.978 (0.964) 0.997 (0.997) 0.993 (0.986)
: bg0 0.952 (0.924) - 0.936 (0.928) 0.992 (0.990)
: bg1 1.000 (1.000) 0.945 (0.936) - 0.998 (0.995)
: bg2 0.994 (0.985) 0.998 (0.998) 0.998 (0.998) -
:
: -------------------------------------------------------------------------------------------------------
:
Dataset:dataset : Created tree 'TestTree' with 4000 events
:
Dataset:dataset : Created tree 'TrainTree' with 4000 events
:
Factory : ␛[1mThank you for using TMVA!␛[0m
: ␛[1mFor citation information, please visit: http://tmva.sf.net/citeTMVA.html␛[0m
==> Wrote root file: TMVAMulticlass.root
==> TMVAMulticlass is done!