==> Start TMVAMulticlass
--- TMVAMulticlass: Using input file: /github/home/ROOT-CI/build/tutorials/machine_learning/data/tmva_multiclass_example.root
DataSetInfo              : [dataset] : Added class "Signal"
                         : Add Tree TreeS of type Signal with 2000 events
DataSetInfo              : [dataset] : Added class "bg0"
                         : Add Tree TreeB0 of type bg0 with 2000 events
DataSetInfo              : [dataset] : Added class "bg1"
                         : Add Tree TreeB1 of type bg1 with 2000 events
DataSetInfo              : [dataset] : Added class "bg2"
                         : Add Tree TreeB2 of type bg2 with 2000 events
                         : Dataset[dataset] : Class index : 0  name : Signal
                         : Dataset[dataset] : Class index : 1  name : bg0
                         : Dataset[dataset] : Class index : 2  name : bg1
                         : Dataset[dataset] : Class index : 3  name : bg2
Factory                  : Booking method: ␛[1mBDTG␛[0m
                         : 
                         : the option NegWeightTreatment=InverseBoostNegWeights does not exist for BoostType=Grad
                         : --> change to new default NegWeightTreatment=Pray
                         : Rebuilding Dataset dataset
                         : Building event vectors for type 2 Signal
                         : Dataset[dataset] :  create input formulas for tree TreeS
                         : Building event vectors for type 2 bg0
                         : Dataset[dataset] :  create input formulas for tree TreeB0
                         : Building event vectors for type 2 bg1
                         : Dataset[dataset] :  create input formulas for tree TreeB1
                         : Building event vectors for type 2 bg2
                         : Dataset[dataset] :  create input formulas for tree TreeB2
DataSetFactory           : [dataset] : Number of events in input trees
                         : 
                         : 
                         : 
                         : 
                         : Number of training and testing events
                         : ---------------------------------------------------------------------------
                         : Signal -- training events            : 1000
                         : Signal -- testing events             : 1000
                         : Signal -- training and testing events: 2000
                         : bg0    -- training events            : 1000
                         : bg0    -- testing events             : 1000
                         : bg0    -- training and testing events: 2000
                         : bg1    -- training events            : 1000
                         : bg1    -- testing events             : 1000
                         : bg1    -- training and testing events: 2000
                         : bg2    -- training events            : 1000
                         : bg2    -- testing events             : 1000
                         : bg2    -- training and testing events: 2000
                         : 
DataSetInfo              : Correlation matrix (Signal):
                         : ----------------------------------------
                         :             var1    var2    var3    var4
                         :    var1:  +1.000  +0.385  +0.621  +0.838
                         :    var2:  +0.385  +1.000  +0.698  +0.723
                         :    var3:  +0.621  +0.698  +1.000  +0.849
                         :    var4:  +0.838  +0.723  +0.849  +1.000
                         : ----------------------------------------
DataSetInfo              : Correlation matrix (bg0):
                         : ----------------------------------------
                         :             var1    var2    var3    var4
                         :    var1:  +1.000  +0.413  +0.612  +0.833
                         :    var2:  +0.413  +1.000  +0.728  +0.753
                         :    var3:  +0.612  +0.728  +1.000  +0.855
                         :    var4:  +0.833  +0.753  +0.855  +1.000
                         : ----------------------------------------
DataSetInfo              : Correlation matrix (bg1):
                         : ----------------------------------------
                         :             var1    var2    var3    var4
                         :    var1:  +1.000  +0.423  +0.619  +0.846
                         :    var2:  +0.423  +1.000  +0.705  +0.730
                         :    var3:  +0.619  +0.705  +1.000  +0.855
                         :    var4:  +0.846  +0.730  +0.855  +1.000
                         : ----------------------------------------
DataSetInfo              : Correlation matrix (bg2):
                         : ----------------------------------------
                         :             var1    var2    var3    var4
                         :    var1:  +1.000  -0.658  +0.032  -0.004
                         :    var2:  -0.658  +1.000  -0.000  +0.014
                         :    var3:  +0.032  -0.000  +1.000  -0.048
                         :    var4:  -0.004  +0.014  -0.048  +1.000
                         : ----------------------------------------
DataSetFactory           : [dataset] :  
                         : 
Factory                  : Booking method: ␛[1mMLP␛[0m
                         : 
MLP                      : Building Network. 
                         : Initializing weights
Factory                  : Booking method: ␛[1mPDEFoam␛[0m
                         : 
Factory                  : Booking method: ␛[1mDL_CPU␛[0m
                         : 
                         : Parsing option string: 
                         : ... "!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=N:WeightInitialization=XAVIERUNIFORM:Architecture=GPU:Layout=TANH|100,TANH|50,TANH|10,LINEAR:TrainingStrategy=Optimizer=ADAM,LearningRate=1e-3,TestRepetitions=1,ConvergenceSteps=10,BatchSize=100,MaxEpochs=20"
                         : The following options are set:
                         : - By User:
                         :     <none>
                         : - Default:
                         :     Boost_num: "0" [Number of times the classifier will be boosted]
                         : Parsing option string: 
                         : ... "!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=N:WeightInitialization=XAVIERUNIFORM:Architecture=GPU:Layout=TANH|100,TANH|50,TANH|10,LINEAR:TrainingStrategy=Optimizer=ADAM,LearningRate=1e-3,TestRepetitions=1,ConvergenceSteps=10,BatchSize=100,MaxEpochs=20"
                         : The following options are set:
                         : - By User:
                         :     V: "True" [Verbose output (short form of "VerbosityLevel" below - overrides the latter one)]
                         :     VarTransform: "N" [List of variable transformations performed before training, e.g., "D_Background,P_Signal,G,N_AllClasses" for: "Decorrelation, PCA-transformation, Gaussianisation, Normalisation, each for the given class of events ('AllClasses' denotes all events of all classes, if no class indication is given, 'All' is assumed)"]
                         :     H: "False" [Print method-specific help message]
                         :     Layout: "TANH|100,TANH|50,TANH|10,LINEAR" [Layout of the network.]
                         :     ErrorStrategy: "CROSSENTROPY" [Loss function: Mean squared error (regression) or cross entropy (binary classification).]
                         :     WeightInitialization: "XAVIERUNIFORM" [Weight initialization strategy]
                         :     Architecture: "GPU" [Which architecture to perform the training on.]
                         :     TrainingStrategy: "Optimizer=ADAM,LearningRate=1e-3,TestRepetitions=1,ConvergenceSteps=10,BatchSize=100,MaxEpochs=20" [Defines the training strategies.]
                         : - Default:
                         :     VerbosityLevel: "Default" [Verbosity level]
                         :     CreateMVAPdfs: "False" [Create PDFs for classifier outputs (signal and background)]
                         :     IgnoreNegWeightsInTraining: "False" [Events with negative weights are ignored in the training (but are included for testing and performance evaluation)]
                         :     InputLayout: "0|0|0" [The Layout of the input]
                         :     BatchLayout: "0|0|0" [The Layout of the batch]
                         :     RandomSeed: "0" [Random seed used for weight initialization and batch shuffling]
                         :     ValidationSize: "20%" [Part of the training data to use for validation. Specify as 0.2 or 20% to use a fifth of the data set as validation set. Specify as 100 to use exactly 100 events. (Default: 20%)]
DL_CPU                   : [dataset] : Create Transformation "N" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'var1' <---> Output : variable 'var1'
                         : Input : variable 'var2' <---> Output : variable 'var2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
␛[31m<ERROR>                         : CUDA backend not enabled. Please make sure you have CUDA installed and it was successfully detected by CMAKE by using -Dtmva-gpu=On  ␛[0m
                         : Will now use instead the CPU architecture !
                         : Will now use the CPU architecture with BLAS and IMT support !
Factory                  : ␛[1mTrain all methods␛[0m
Factory                  : [dataset] : Create Transformation "I" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'var1' <---> Output : variable 'var1'
                         : Input : variable 'var2' <---> Output : variable 'var2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
Factory                  : [dataset] : Create Transformation "D" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'var1' <---> Output : variable 'var1'
                         : Input : variable 'var2' <---> Output : variable 'var2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
Factory                  : [dataset] : Create Transformation "P" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'var1' <---> Output : variable 'var1'
                         : Input : variable 'var2' <---> Output : variable 'var2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
Factory                  : [dataset] : Create Transformation "G" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'var1' <---> Output : variable 'var1'
                         : Input : variable 'var2' <---> Output : variable 'var2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
Factory                  : [dataset] : Create Transformation "D" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'var1' <---> Output : variable 'var1'
                         : Input : variable 'var2' <---> Output : variable 'var2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
TFHandler_Factory        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:   0.052185     1.0190   [    -4.0592     3.2645 ]
                         :     var2:    0.33312     1.0446   [    -3.6891     3.7877 ]
                         :     var3:    0.10463     1.1205   [    -3.6296     3.9200 ]
                         :     var4:  -0.078123     1.2764   [    -4.8486     4.3625 ]
                         : -----------------------------------------------------------
                         : Preparing the Decorrelation transformation...
TFHandler_Factory        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:   0.089502     1.0000   [    -3.4349     2.7570 ]
                         :     var2:    0.38543     1.0000   [    -3.3765     3.1055 ]
                         :     var3:   0.052636     1.0000   [    -2.8007     3.1004 ]
                         :     var4:   -0.20867     1.0000   [    -3.0012     2.5822 ]
                         : -----------------------------------------------------------
                         : Preparing the Principle Component (PCA) transformation...
TFHandler_Factory        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:-2.3297e-10     1.8127   [    -7.2691     6.3617 ]
                         :     var2:-3.1381e-10    0.89464   [    -2.7283     2.6323 ]
                         :     var3:-2.2463e-10    0.73955   [    -2.6363     2.4256 ]
                         :     var4:-9.8869e-11    0.61727   [    -1.7822     2.2327 ]
                         : -----------------------------------------------------------
                         : Preparing the Gaussian transformation...
                         : Preparing the Decorrelation transformation...
TFHandler_Factory        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:  0.0071986     1.0000   [    -2.5427     5.8540 ]
                         :     var2:  0.0087421     1.0000   [    -2.8611     4.9796 ]
                         :     var3:  0.0090897     1.0000   [    -2.9572     5.6365 ]
                         :     var4:  0.0084612     1.0000   [    -3.0233     5.7479 ]
                         : -----------------------------------------------------------
                         : Ranking input variables (method unspecific)...
Factory                  : Train method: BDTG for Multiclass classification
                         : 
                         : Training 1000 Decision Trees ... patience please
                         : Elapsed time for training with 4000 events: 4.14 sec         
                         : Dataset[dataset] : Create results for training
                         : Dataset[dataset] : Multiclass evaluation of BDTG on training sample
                         : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 1.06 sec       
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
                         : Creating xml weight file: ␛[0;36mdataset/weights/TMVAMulticlass_BDTG.weights.xml␛[0m
                         : Creating standalone class: ␛[0;36mdataset/weights/TMVAMulticlass_BDTG.class.C␛[0m
                         : TMVAMulticlass.root:/dataset/Method_BDT/BDTG
Factory                  : Training finished
                         : 
Factory                  : Train method: MLP for Multiclass classification
                         : 
                         : Training Network
                         : 
                         : Elapsed time for training with 4000 events: 17.1 sec         
                         : Dataset[dataset] : Create results for training
                         : Dataset[dataset] : Multiclass evaluation of MLP on training sample
                         : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.00569 sec       
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
                         : Creating xml weight file: ␛[0;36mdataset/weights/TMVAMulticlass_MLP.weights.xml␛[0m
                         : Creating standalone class: ␛[0;36mdataset/weights/TMVAMulticlass_MLP.class.C␛[0m
                         : Write special histos to file: TMVAMulticlass.root:/dataset/Method_MLP/MLP
Factory                  : Training finished
                         : 
Factory                  : Train method: PDEFoam for Multiclass classification
                         : 
                         : Build up multiclass foam 0
                         : Elapsed time: 0.414 sec                                 
                         : Build up multiclass foam 1
                         : Elapsed time: 0.397 sec                                 
                         : Build up multiclass foam 2
                         : Elapsed time: 0.391 sec                                 
                         : Build up multiclass foam 3
                         : Elapsed time: 0.278 sec                                 
                         : Elapsed time for training with 4000 events: 1.58 sec         
                         : Dataset[dataset] : Create results for training
                         : Dataset[dataset] : Multiclass evaluation of PDEFoam on training sample
                         : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.0673 sec       
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
                         : Creating xml weight file: ␛[0;36mdataset/weights/TMVAMulticlass_PDEFoam.weights.xml␛[0m
                         : writing foam MultiClassFoam0 to file
                         : writing foam MultiClassFoam1 to file
                         : writing foam MultiClassFoam2 to file
                         : writing foam MultiClassFoam3 to file
                         : Foams written to file: ␛[0;36mdataset/weights/TMVAMulticlass_PDEFoam.weights_foams.root␛[0m
                         : Creating standalone class: ␛[0;36mdataset/weights/TMVAMulticlass_PDEFoam.class.C␛[0m
Factory                  : Training finished
                         : 
Factory                  : Train method: DL_CPU for Multiclass classification
                         : 
TFHandler_DL_CPU         : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:    0.12276    0.27828   [    -1.0000     1.0000 ]
                         :     var2:   0.075909    0.27943   [    -1.0000     1.0000 ]
                         :     var3:  -0.010745    0.29684   [    -1.0000     1.0000 ]
                         :     var4:   0.035804    0.27714   [    -1.0000     1.0000 ]
                         : -----------------------------------------------------------
                         : Start of deep neural network training on CPU using MT,  nthreads = 1
                         : 
TFHandler_DL_CPU         : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:    0.12276    0.27828   [    -1.0000     1.0000 ]
                         :     var2:   0.075909    0.27943   [    -1.0000     1.0000 ]
                         :     var3:  -0.010745    0.29684   [    -1.0000     1.0000 ]
                         :     var4:   0.035804    0.27714   [    -1.0000     1.0000 ]
                         : -----------------------------------------------------------
                         : *****   Deep Learning Network *****
DEEP NEURAL NETWORK:   Depth = 4  Input = ( 1, 1, 4 )  Batch size = 100  Loss function = C
   Layer 0   DENSE Layer:   ( Input =     4 , Width =   100 )  Output = (  1 ,   100 ,   100 )   Activation Function = Tanh
   Layer 1   DENSE Layer:   ( Input =   100 , Width =    50 )  Output = (  1 ,   100 ,    50 )   Activation Function = Tanh
   Layer 2   DENSE Layer:   ( Input =    50 , Width =    10 )  Output = (  1 ,   100 ,    10 )   Activation Function = Tanh
   Layer 3   DENSE Layer:   ( Input =    10 , Width =     4 )  Output = (  1 ,   100 ,     4 )   Activation Function = Identity
                         : Using 3200 events for training and 800 for testing
                         : Compute initial loss  on the validation data 
                         : Training phase 1 of 1:  Optimizer ADAM (beta1=0.9,beta2=0.999,eps=1e-07) Learning rate = 0.001 regularization 0 minimum error = 0.69969
                         : --------------------------------------------------------------
                         :      Epoch |   Train Err.   Val. Err.  t(s)/epoch   t(s)/Loss   nEvents/s Conv. Steps
                         : --------------------------------------------------------------
                         :    Start epoch iteration ...
                         :          1 Minimum Test error found - save the configuration 
                         :          1 |     0.624981    0.557798   0.0472384  0.00461113     75069.2           0
                         :          2 Minimum Test error found - save the configuration 
                         :          2 |     0.518793    0.493072   0.0522871  0.00477046     67344.8           0
                         :          3 Minimum Test error found - save the configuration 
                         :          3 |     0.475562    0.461253   0.0501356  0.00498738     70877.7           0
                         :          4 Minimum Test error found - save the configuration 
                         :          4 |     0.445689    0.434406   0.0489295  0.00439576     71855.6           0
                         :          5 Minimum Test error found - save the configuration 
                         :          5 |     0.417892    0.409434   0.0493713  0.00455992     71410.5           0
                         :          6 Minimum Test error found - save the configuration 
                         :          6 |     0.394896    0.390723   0.0509508  0.00490231     69491.9           0
                         :          7 Minimum Test error found - save the configuration 
                         :          7 |     0.377869    0.375392   0.0507089  0.00523193     70365.2           0
                         :          8 Minimum Test error found - save the configuration 
                         :          8 |     0.365002     0.36459   0.0554342  0.00532182     63856.4           0
                         :          9 Minimum Test error found - save the configuration 
                         :          9 |     0.355477    0.356372    0.060496   0.0063365     59084.8           0
                         :         10 Minimum Test error found - save the configuration 
                         :         10 |     0.347519    0.348964   0.0574262  0.00523883     61317.5           0
                         :         11 Minimum Test error found - save the configuration 
                         :         11 |     0.341158    0.343259   0.0527341  0.00539666     67599.8           0
                         :         12 Minimum Test error found - save the configuration 
                         :         12 |      0.33473    0.338827    0.054837  0.00490053     64081.4           0
                         :         13 Minimum Test error found - save the configuration 
                         :         13 |     0.329316    0.334146   0.0549971  0.00564283     64837.4           0
                         :         14 Minimum Test error found - save the configuration 
                         :         14 |     0.324826    0.328272   0.0529676  0.00524288     67051.3           0
                         :         15 Minimum Test error found - save the configuration 
                         :         15 |     0.319833    0.324801   0.0537172  0.00548995     66352.6           0
                         :         16 Minimum Test error found - save the configuration 
                         :         16 |     0.316248    0.320548    0.055267  0.00501047     63673.3           0
                         :         17 Minimum Test error found - save the configuration 
                         :         17 |      0.31321    0.317457   0.0540819  0.00563532     66052.2           0
                         :         18 Minimum Test error found - save the configuration 
                         :         18 |     0.308665    0.314784   0.0606508    0.006589     59191.5           0
                         :         19 Minimum Test error found - save the configuration 
                         :         19 |     0.305357     0.30805   0.0668718   0.0080295     54382.7           0
                         :         20 Minimum Test error found - save the configuration 
                         :         20 |     0.301412    0.306036   0.0760324  0.00807989     47091.7           0
                         : 
                         : Elapsed time for training with 4000 events: 1.13 sec         
                         : Dataset[dataset] : Create results for training
                         : Dataset[dataset] : Multiclass evaluation of DL_CPU on training sample
                         : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.0768 sec       
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
                         : Creating xml weight file: ␛[0;36mdataset/weights/TMVAMulticlass_DL_CPU.weights.xml␛[0m
                         : Creating standalone class: ␛[0;36mdataset/weights/TMVAMulticlass_DL_CPU.class.C␛[0m
Factory                  : Training finished
                         : 
                         : Ranking input variables (method specific)...
BDTG                     : Ranking result (top variable is best ranked)
                         : --------------------------------------
                         : Rank : Variable  : Variable Importance
                         : --------------------------------------
                         :    1 : var4      : 3.060e-01
                         :    2 : var1      : 2.473e-01
                         :    3 : var2      : 2.400e-01
                         :    4 : var3      : 2.067e-01
                         : --------------------------------------
MLP                      : Ranking result (top variable is best ranked)
                         : -----------------------------
                         : Rank : Variable  : Importance
                         : -----------------------------
                         :    1 : var4      : 5.440e+01
                         :    2 : var1      : 2.568e+01
                         :    3 : var2      : 2.223e+01
                         :    4 : var3      : 7.204e+00
                         : -----------------------------
PDEFoam                  : Ranking result (top variable is best ranked)
                         : --------------------------------------
                         : Rank : Variable  : Variable Importance
                         : --------------------------------------
                         :    1 : var4      : 2.766e-01
                         :    2 : var1      : 2.682e-01
                         :    3 : var2      : 2.510e-01
                         :    4 : var3      : 2.042e-01
                         : --------------------------------------
                         : No variable ranking supplied by classifier: DL_CPU
TH1.Print Name  = TrainingHistory_DL_CPU_trainingError, Entries= 0, Total sum= 7.51844
TH1.Print Name  = TrainingHistory_DL_CPU_valError, Entries= 0, Total sum= 7.42818
Factory                  : === Destroy and recreate all methods via weight files for testing ===
                         : 
                         : Reading weight file: ␛[0;36mdataset/weights/TMVAMulticlass_BDTG.weights.xml␛[0m
                         : Reading weight file: ␛[0;36mdataset/weights/TMVAMulticlass_MLP.weights.xml␛[0m
MLP                      : Building Network. 
                         : Initializing weights
                         : Reading weight file: ␛[0;36mdataset/weights/TMVAMulticlass_PDEFoam.weights.xml␛[0m
                         : Read foams from file: ␛[0;36mdataset/weights/TMVAMulticlass_PDEFoam.weights_foams.root␛[0m
                         : Reading weight file: ␛[0;36mdataset/weights/TMVAMulticlass_DL_CPU.weights.xml␛[0m
Factory                  : ␛[1mTest all methods␛[0m
Factory                  : Test method: BDTG for Multiclass classification performance
                         : 
                         : Dataset[dataset] : Create results for testing
                         : Dataset[dataset] : Multiclass evaluation of BDTG on testing sample
                         : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.556 sec       
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
Factory                  : Test method: MLP for Multiclass classification performance
                         : 
                         : Dataset[dataset] : Create results for testing
                         : Dataset[dataset] : Multiclass evaluation of MLP on testing sample
                         : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.00815 sec       
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
Factory                  : Test method: PDEFoam for Multiclass classification performance
                         : 
                         : Dataset[dataset] : Create results for testing
                         : Dataset[dataset] : Multiclass evaluation of PDEFoam on testing sample
                         : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.0942 sec       
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
Factory                  : Test method: DL_CPU for Multiclass classification performance
                         : 
                         : Dataset[dataset] : Create results for testing
                         : Dataset[dataset] : Multiclass evaluation of DL_CPU on testing sample
                         : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.0767 sec       
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
Factory                  : ␛[1mEvaluate all methods␛[0m
                         : Evaluate multiclass classification method: BDTG
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
TFHandler_BDTG           : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:   0.065615     1.0061   [    -4.0592     3.5808 ]
                         :     var2:    0.29707     1.0658   [    -3.6952     3.7877 ]
                         :     var3:    0.13183     1.1245   [    -4.5727     4.5640 ]
                         :     var4:  -0.071010     1.2654   [    -4.8486     5.0412 ]
                         : -----------------------------------------------------------
                         : Evaluate multiclass classification method: MLP
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
TFHandler_MLP            : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:   0.065615     1.0061   [    -4.0592     3.5808 ]
                         :     var2:    0.29707     1.0658   [    -3.6952     3.7877 ]
                         :     var3:    0.13183     1.1245   [    -4.5727     4.5640 ]
                         :     var4:  -0.071010     1.2654   [    -4.8486     5.0412 ]
                         : -----------------------------------------------------------
                         : Evaluate multiclass classification method: PDEFoam
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
TFHandler_PDEFoam        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:   0.065615     1.0061   [    -4.0592     3.5808 ]
                         :     var2:    0.29707     1.0658   [    -3.6952     3.7877 ]
                         :     var3:    0.13183     1.1245   [    -4.5727     4.5640 ]
                         :     var4:  -0.071010     1.2654   [    -4.8486     5.0412 ]
                         : -----------------------------------------------------------
                         : Evaluate multiclass classification method: DL_CPU
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
TFHandler_DL_CPU         : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:    0.12643    0.27476   [    -1.0000     1.0864 ]
                         :     var2:   0.066267    0.28510   [    -1.0016     1.0000 ]
                         :     var3: -0.0035395    0.29791   [    -1.2498     1.1706 ]
                         :     var4:   0.037349    0.27475   [    -1.0000     1.1474 ]
                         : -----------------------------------------------------------
TFHandler_DL_CPU         : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:    0.12643    0.27476   [    -1.0000     1.0864 ]
                         :     var2:   0.066267    0.28510   [    -1.0016     1.0000 ]
                         :     var3: -0.0035395    0.29791   [    -1.2498     1.1706 ]
                         :     var4:   0.037349    0.27475   [    -1.0000     1.1474 ]
                         : -----------------------------------------------------------
                         : 
                         : 1-vs-rest performance metrics per class
                         : -------------------------------------------------------------------------------------------------------
                         : 
                         : Considers the listed class as signal and the other classes
                         : as background, reporting the resulting binary performance.
                         : A score of 0.820 (0.850) means 0.820 was acheived on the
                         : test set and 0.850 on the training set.
                         : 
                         : Dataset        MVA Method     ROC AUC        Sig eff@B=0.01 Sig eff@B=0.10 Sig eff@B=0.30 
                         : Name:          / Class:       test  (train)  test  (train)  test  (train)  test  (train)  
                         : 
                         : dataset        BDTG           
                         : ------------------------------
                         :                Signal         0.967 (0.980)  0.496 (0.616)  0.910 (0.953)  0.994 (0.997)  
                         :                bg0            0.908 (0.927)  0.201 (0.331)  0.729 (0.777)  0.924 (0.944)  
                         :                bg1            0.945 (0.955)  0.413 (0.429)  0.833 (0.860)  0.970 (0.973)  
                         :                bg2            0.974 (0.984)  0.600 (0.677)  0.926 (0.973)  0.995 (0.998)  
                         : 
                         : dataset        MLP            
                         : ------------------------------
                         :                Signal         0.975 (0.976)  0.591 (0.609)  0.931 (0.940)  0.997 (0.994)  
                         :                bg0            0.930 (0.934)  0.279 (0.389)  0.781 (0.789)  0.960 (0.951)  
                         :                bg1            0.963 (0.964)  0.494 (0.462)  0.889 (0.900)  0.990 (0.994)  
                         :                bg2            0.971 (0.977)  0.653 (0.697)  0.901 (0.900)  0.994 (1.000)  
                         : 
                         : dataset        PDEFoam        
                         : ------------------------------
                         :                Signal         0.923 (0.931)  0.270 (0.365)  0.760 (0.774)  0.957 (0.966)  
                         :                bg0            0.843 (0.855)  0.113 (0.166)  0.604 (0.615)  0.830 (0.841)  
                         :                bg1            0.903 (0.912)  0.290 (0.293)  0.686 (0.733)  0.922 (0.927)  
                         :                bg2            0.972 (0.972)  0.481 (0.477)  0.921 (0.923)  1.000 (0.999)  
                         : 
                         : dataset        DL_CPU         
                         : ------------------------------
                         :                Signal         0.959 (0.963)  0.249 (0.343)  0.909 (0.910)  0.992 (0.993)  
                         :                bg0            0.918 (0.909)  0.274 (0.334)  0.750 (0.722)  0.941 (0.925)  
                         :                bg1            0.948 (0.945)  0.304 (0.305)  0.851 (0.837)  0.982 (0.985)  
                         :                bg2            0.899 (0.915)  0.511 (0.516)  0.734 (0.737)  0.866 (0.902)  
                         : 
                         : -------------------------------------------------------------------------------------------------------
                         : 
                         : 
                         : Confusion matrices for all methods
                         : -------------------------------------------------------------------------------------------------------
                         : 
                         : Does a binary comparison between the two classes given by a 
                         : particular row-column combination. In each case, the class 
                         : given by the row is considered signal while the class given 
                         : by the column index is considered background.
                         : 
                         : === Showing confusion matrix for method : BDTG           
                         : (Signal Efficiency for Background Efficiency 0.01%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.489 (0.430)  0.864 (0.764)  0.784 (0.472) 
                         :  bg0            0.311 (0.181)  -              0.207 (0.132)  0.694 (0.611) 
                         :  bg1            0.830 (0.834)  0.288 (0.339)  -              0.668 (0.511) 
                         :  bg2            0.708 (0.593)  0.684 (0.593)  0.625 (0.600)  -             
                         : 
                         : (Signal Efficiency for Background Efficiency 0.10%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.901 (0.852)  0.996 (0.993)  0.956 (0.892) 
                         :  bg0            0.810 (0.763)  -              0.643 (0.601)  0.924 (0.904) 
                         :  bg1            0.984 (0.984)  0.716 (0.677)  -              0.898 (0.843) 
                         :  bg2            0.983 (0.928)  0.982 (0.953)  0.948 (0.897)  -             
                         : 
                         : (Signal Efficiency for Background Efficiency 0.30%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.981 (0.960)  1.000 (0.999)  0.999 (0.998) 
                         :  bg0            0.963 (0.927)  -              0.852 (0.814)  0.990 (0.986) 
                         :  bg1            0.999 (0.998)  0.915 (0.888)  -              0.984 (0.984) 
                         :  bg2            0.999 (0.996)  0.998 (0.995)  0.998 (0.993)  -             
                         : 
                         : === Showing confusion matrix for method : MLP            
                         : (Signal Efficiency for Background Efficiency 0.01%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.456 (0.481)  0.936 (0.943)  0.645 (0.548) 
                         :  bg0            0.421 (0.278)  -              0.302 (0.229)  0.604 (0.477) 
                         :  bg1            0.913 (0.925)  0.261 (0.400)  -              0.566 (0.602) 
                         :  bg2            0.675 (0.662)  0.710 (0.669)  0.696 (0.641)  -             
                         : 
                         : (Signal Efficiency for Background Efficiency 0.10%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.875 (0.891)  0.999 (1.000)  0.920 (0.894) 
                         :  bg0            0.766 (0.755)  -              0.696 (0.707)  0.909 (0.905) 
                         :  bg1            0.997 (0.992)  0.780 (0.790)  -              0.901 (0.867) 
                         :  bg2            0.880 (0.890)  0.944 (0.933)  0.891 (0.886)  -             
                         : 
                         : (Signal Efficiency for Background Efficiency 0.30%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.974 (0.972)  1.000 (1.000)  0.995 (0.998) 
                         :  bg0            0.954 (0.960)  -              0.914 (0.914)  0.995 (0.992) 
                         :  bg1            0.999 (0.999)  0.958 (0.944)  -              0.998 (0.996) 
                         :  bg2            0.999 (0.990)  1.000 (0.996)  0.999 (0.989)  -             
                         : 
                         : === Showing confusion matrix for method : PDEFoam        
                         : (Signal Efficiency for Background Efficiency 0.01%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.276 (0.123)  0.429 (0.447)  0.415 (0.458) 
                         :  bg0            0.156 (0.076)  -              0.126 (0.081)  0.522 (0.492) 
                         :  bg1            0.481 (0.407)  0.200 (0.222)  -              0.420 (0.322) 
                         :  bg2            0.478 (0.479)  0.462 (0.490)  0.477 (0.461)  -             
                         : 
                         : (Signal Efficiency for Background Efficiency 0.10%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.678 (0.661)  0.829 (0.854)  0.839 (0.825) 
                         :  bg0            0.569 (0.563)  -              0.521 (0.475)  0.838 (0.816) 
                         :  bg1            0.824 (0.802)  0.541 (0.521)  -              0.825 (0.790) 
                         :  bg2            0.937 (0.921)  0.959 (0.948)  0.885 (0.884)  -             
                         : 
                         : (Signal Efficiency for Background Efficiency 0.30%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.904 (0.888)  0.974 (0.966)  0.972 (0.964) 
                         :  bg0            0.783 (0.792)  -              0.723 (0.735)  0.944 (0.925) 
                         :  bg1            0.963 (0.949)  0.839 (0.843)  -              0.955 (0.944) 
                         :  bg2            0.999 (0.998)  0.999 (1.000)  0.999 (1.000)  -             
                         : 
                         : === Showing confusion matrix for method : DL_CPU         
                         : (Signal Efficiency for Background Efficiency 0.01%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.477 (0.466)  0.924 (0.957)  0.186 (0.162) 
                         :  bg0            0.337 (0.271)  -              0.227 (0.275)  0.430 (0.264) 
                         :  bg1            0.841 (0.924)  0.145 (0.217)  -              0.348 (0.305) 
                         :  bg2            0.565 (0.547)  0.541 (0.518)  0.422 (0.442)  -             
                         : 
                         : (Signal Efficiency for Background Efficiency 0.10%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.882 (0.900)  0.993 (0.995)  0.788 (0.715) 
                         :  bg0            0.723 (0.735)  -              0.712 (0.753)  0.724 (0.782) 
                         :  bg1            0.992 (0.991)  0.681 (0.717)  -              0.811 (0.810) 
                         :  bg2            0.812 (0.777)  0.748 (0.742)  0.681 (0.687)  -             
                         : 
                         : (Signal Efficiency for Background Efficiency 0.30%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.985 (0.978)  0.999 (1.000)  0.979 (0.974) 
                         :  bg0            0.916 (0.934)  -              0.916 (0.929)  0.948 (0.957) 
                         :  bg1            0.999 (1.000)  0.933 (0.948)  -              0.968 (0.967) 
                         :  bg2            0.929 (0.891)  0.927 (0.901)  0.860 (0.833)  -             
                         : 
                         : -------------------------------------------------------------------------------------------------------
                         : 
Dataset:dataset          : Created tree 'TestTree' with 4000 events
                         : 
Dataset:dataset          : Created tree 'TrainTree' with 4000 events
                         : 
Factory                  : ␛[1mThank you for using TMVA!␛[0m
                         : ␛[1mFor citation information, please visit: http://tmva.sf.net/citeTMVA.html␛[0m
==> Wrote root file: TMVAMulticlass.root
==> TMVAMulticlass is done!