ROOT Version 6.28 Release Notes

2023-02-01

Introduction

ROOT version 6.28/00 was released on February 03, 2023.

For more information, see:

http://root.cern

The following people have contributed to this new version:

Guilherme Amadio, CERN/SFT,
Rahul Balasubramanian, NIKHEF/ATLAS,
Bertrand Bellenot, CERN/SFT,
Jakob Blomer, CERN/SFT,
Patrick Bos, Netherlands eScience Center,
Rene Brun, CERN/SFT,
Carsten D. Burgard, TU Dortmund University/ATLAS,
Artem Busorgin,
Will Buttinger, RAL/ATLAS,
Philippe Canal, FNAL,
Olivier Couet, CERN/SFT,
Michel De Cian, EPFL/LHCb,
Hans Dembinski, TU Dortmund,
Simeon Ehrig, HZDR,
Mattias Ellert, Uppsala University,
Matthew Feickert, UW Madison/ATLAS,
Markus Frank, CERN/LHCb,
Oliver Freyermuth, U Bonn,
Gerri Ganis, CERN/SFT,
Andrei Gheata, CERN/SFT,
Sayandeep Ghosh, Jadavpur U Calcutta,
Konstantin Gizdov, University of Edinburgh/LHCb,
Max Goblirsch, CERN/ATLAS,
Stefan Gränitz,
Enrico Guiraud, CERN/SFT,
Stephan Hageboeck, CERN/IT,
Jonas Hahnfeld, CERN/SFT,
Ahmat Mahamat Hamdan, CERN/SFT,
Fernando Hueso-González, University of Valencia,
Pawan Johnson, Indian Institute of Technology Kharagpur,
Subham Jyoti, ITER Bhubaneswar,
Ivan Kabadzhov, CERN/SFT,
Baidyanath Kundu, Princeton/SFT,
Giovanna Lazzari Miotto, CERN/SFT,
Yueh-Shun Li, National Central U Taiwan/CMS,
Sergey Linev, GSI,
Jerry Ling, Harvard University,
Javier Lopez-Gomez, CERN/SFT,
Enrico Lusiani, INFN/CMS,
Julia Mathe, CERN/SFT,
Pere Mato, CERN/SFT,
Lorenzo Moneta, CERN/SFT,
Nicolas Morange, CNRS/ATLAS,
Alja Mrak Tadel, UCSD/CMS,
Axel Naumann, CERN/SFT,
Duncan Ogilvie,
Hanna Olvhammar, CERN/SFT,
Vincenzo Eduardo Padulano, CERN/SFT and UPV,
Tapasweni Pathak,
Danilo Piparo, CERN/SFT,
David Poulton, Wits/SFT,
Fons Rademakers, CERN/SFT,
Jonas Rembser, CERN/SFT,
Mathias Schmitt,
Neel Shah, VJTI,
Petr Stepanov, Catholic University of America,
Enric Tejedor Saavedra, CERN/SFT,
Neel Shah, GSOC,
Sanjiban Sengupta, CERN/SFT,
Harshal Shende, GSOC,
Garima Singh, Princeton/SFT,
Surya Somayyajula, UW Madison,
Matevz Tadel, UCSD/CMS,
Vassil Vassilev, Princeton/CMS,
Andrii Verbytskyi, MPP Munich/ATLAS,
Wouter Verkerke, NIKHEF/ATLAS,
Frank Winklmeier, U Oregon/ATLAS,
Zef Wolffs, NIKHEF/ATLAS,
Jun Zhang

Deprecation and Removal

rootreadspeed

This version adds the new rootreadspeed CLI tool. This tool can be used to help identify bottlenecks in analysis runtimes, by providing time and throughput measurements when reading ROOT files via file systems or XRootD. More detailed information can be found in the tool’s help information.

To see help information, install and source a recent enough version of ROOT, and run the command rootreadspeed --help in your terminal.

Example usage of the tool

$ rootreadspeed --files <local-folder>/File1.root xrootd://<url-folder>/File2.root --trees Events --all-branches --threads 8

Core Libraries

Interpreter

llvm and clang have been upgraded to version 13. C++20 support will be provided as part of this release (not yet available for 6.28/00). cling has been upgraded to use llvm’s new just-in-time compilation engive ORCv2.

Cling checks pointer validity now only in interactive mode, improving performance for jitted code in batch systems.

Support for profiling/debugging interpreted/JITted code

This version of ROOT adds an LLVM JIT event listener to create perf map files during runtime. This allows profiling of interpreted/JITted code generated by cling. Instead of function addresses, the perf data will contain full function names. In addition, stack frame pointers are enabled in JITted code, so full stack traces can be generated. Debugging is aided by switching off optimisations and adding frame pointers for better stack traces. However, since both have a runtime cost, they are disabled by default. Similar to LD_DEBUG and LD_PROFILE for ld.so, the environment variables CLING_DEBUG=1 and/or CLING_PROFILE=1 can be set to enable debugging and/or profiling.

Other changes

root [] int data;
ROOT_prompt_0:1:1: warning: 'data' shadows a declaration with the same name in the 'std' namespace; use '::data' to reference this declaration
int data;
^
root [] .help edit

I/O Libraries

Faster reading from EOS

A new cross-protocol redirection has been added to allow files on EOS mounts to be opened by TFile::Open via XRootD protocol rather than via FUSE when that is possible. The redirection uses the eos.url.xroot extended file attribute that is present on files in EOS. The attribute can be viewed with getfattr -n eos.url.xroot [file] on the command line. When the URL passed into TFile::Open is a for a file on an EOS mount, the extended attribute is used to attempt the redirection to XRootD protocol. If the redirection fails, the file is opened using the plain file path as before. This feature is controlled by the pre-existing configuration option TFile.CrossProtocolRedirects and is enabled by default. It can be disabled by setting TFile.CrossProtocolRedirects to 0 in rootrc.

RNTuple

ROOT’s experimental successor of TTree has seen many updates during the last few months. Specifically, v6.28 includes the following changes:

auto f = TFile::Open("data.root");
auto ntpl = f->Get<ROOT::Experimental::RNTuple>("Events");

auto reader = ROOT::Experimental::RNTupleReader::Open(ntpl);
// or for RDataFrame
auto rdf = ROOT::Experimental::MakeNTupleDataFrame(ntpl);

Please, report any issues regarding the abovementioned features should you encounter them. RNTuple is still experimental and is scheduled to become production grade in 2024. Thus, we appreciate feedback and suggestions for improvement.

RDataFrame

New features

Notable bug fixes and improvements

Distributed RDataFrame

Histogram Libraries

Math Libraries

Fitter class

Some improvements and small fixes to the internal object memory management have been applied to the ROOT::Fit::Fitter class.

Support for providing the second derivatives (Hessian matrix) from the model function is added to the Fitter class and the corresponding function interfaces. The functionality it is then propagated in the implementation of the FitMethod classes and it is also added to the Minimizer classes for providing a user computed Hessian of the objective functions to the minimizers. Only Minuit2 (see below) has the capabilities of using this external Hessian.

The GradFunctor class has been improved by providing a new constructor taking an std::function implementing the full gradient calculations instead of the single partial derivative.

The specialized methods for least-square/likelihood functions such as Fumili, Fumili2 and GSLMultiFit have been improved in case of binned likelihood fits, where a better approximation is used than before. This makes these method work better (conerging with less number of function calls) for these types of fits.

Minuit2

The support for using an External Hessian calculator has been added. The external Hessian can be used for both the initial seeding, using only the diagonal part, if the strategy is equal to 1 (the default value) and in MnHesse, after the minimization, to compute the covariance and correlation matrices.

The print log of Minuit2 has been improved, especially when printing vector and matrices with large number of parameters (when the print level = 3).

KahanSum updates

The ROOT::Math::KahanSum class was slightly modified:

Foam

The usage of TRef in the TFoamCell class has ben replaced with array indices. This avoids, when generating a large number of toys requiring a re-initialization of TFoam an increase in the memory usage caused by TRef.

RVec

RooFit Libraries

Consistent definition of the default minimizer type for all of RooFit/RooStats

In previous releases, the default minimizer type that RooFit used was hardcoded to be the original Minuit, while RooStats used the default minimizer specified by ROOT::Math::MinimizerOptions::DefaultMinimizerType(). Now it is possible to centrally define the global minimizer for all RooFit libraries via ROOT::Math::MinimizerOptions::SetDefaultMinimizer(), or alternatively in the .rootrc file by adding for example Root.Fitter: Minuit2 to select Minuit2.

Code modernization by using std::string in RooFit interfaces

The following lesser-used RooFit functions now return a std::string instead of a const char*, potentially requiring the update of your code:

Before v6.28, it was ensured that no RooArgSet and RooDataSet objects on the heap were located at an address that had already been used for an instance of the same class before. With v6.28, this is not guaranteed anymore. Hence, if your code uses pointer comparisons to uniquely identify RooArgSet or RooDataSet instances, please consider using the new RooArgSet::uniqueId() or RooAbsData::uniqueId().

Introducing binned likelihood fit optimization in HistFactory

In a binned likelihood fit, it is possible to skip the PDF normalization when the unnormalized binned PDF can be interpreted directly in terms of event yields. This is now done by default for HistFactory models, which results in great speedups for binned fits with many channels. Some RooFit users like ATLAS were already using this for a long time.

To disable this optimization when using the hist2workspace executable, add the -disable_binned_fit_optimization command line argument. Directly in C++, you can also set the binnedFitOptimization to false in the HistFactory configuration as follows:

RooStats::HistFactory::MakeModelAndMeasurementFast(measurement, {.binnedFitOptimization=false});

If your compiler doesn’t support aggregate initialization with designators, you need to create and edit the configuration struct explicitely:

RooStats::HistFactory::HistoToWorkspaceFactoryFast::Configuration hfCfg;
hfCfg.binnedFitOptimization = false;
RooStats::HistFactory::MakeModelAndMeasurementFast(measurement, hfCfg);

Disable copy assignment for RooAbsArg and derived types

Copy assignment for RooAbsArgs was implemented in an unexpected and inconsistent way. While one would expect that the copy assignment is copying the object, it said in the documentation of RooAbsArg::operator= that it will “assign all boolean and string properties of the original bject. Transient properties and client-server links are not assigned.” This contradicted with the implementation, where the server links were actually copied too. Furthermore, in RooAbsRealLValue, the assigment operator was overloaded by a function that only assigns the value of another RooAbsReal.

With all these inconsistencies, it was deemed safer to disable copy assignment of RooAbsArgs from now on.

RooBrowser: a graphical user interface for workspace exploration, visualization, and analysis

This experimental new feature utilises the technology from ROOT’s familiar TBrowser in order to create an interface for graphically exploring and visualizing the content of a workspace, as well as perform basic fitting operations with the models and datasets.

Demonstration of RooBrowser using json workspace from the roofit tutorials directory
Demonstration of RooBrowser using json workspace from the roofit tutorials directory

Removal of deprecated HistFactory functionality

Removal of HistoToWorkspaceFactory (non-Fast version)

The original HistoToWorkspaceFactory produced models that consisted of a Poisson term for each bin. In this “number counting form” the dataset has one row and the collumns corresponded to the number of events for each bin. This led to severe performance problems in statistical tools that generated pseudo-experiments and evaluated likelihood ratio test statistics.

Nowadays, everyone uses the faster HistoToWorkspaceFactoryFast implementation that produces a model in the “standard form” where the dataset has one row for each event, and the column corresponds to the value of the observable in the histogram.

Therefore, the original HistoToWorkspaceFactory is now removed to avoid confusion and maintainance burden.

Removing constant parameter flag from RooStats:HistFactory::NormFactor

As printed out by the HistFactory in a warning message for a long time already, setting the Const attribute to the <NormFactor> tag is deprecated and it will be ignored. Instead, add <ParamSetting Const="True"> myparam </ParamSetting> to your top-level XML’s <Measurement> entry.

This deprecation implied that the constant parameter flag in the RooStats:HistFactory::NormFactor class had no effect as well. To avoid ambiguity in the future, the possibility to set and retrieve this flag with NormFactor::SetConst() and NormFactor::GetConst() was removed, as well as the Sample::AddNormFactor(std::string Name, double Val, double Low, double High, bool Const) overload. Also, the aforementioned deprecation warning is not printed anymore.

Removal of RooAbsMinimizerFcn and RooMinimizerFcn from the public interface

The RooAbsMinimizerFcn class and its implementation RooMinimizerFcn were removed from the public interface. These classes are implementation details of the RooMinimizer and should not be used in your code. In the unlikely case that this causes any problem for you, please open a GitHub issue requesting to extend the RooMinimizer by the needed functionality.

Vectorize RooAbsBinning interface for bin index lookups

The RooAbsBinning interface for bin index lookups was changed to enable vectorized implementations. Instead of having the override RooAbsBinning::binNumber(), the binning implementations now have to override the RooAbsBinning::binNumbers() function to evaluate the bin indices of multiple values in one function call.

Disable relative and absolute epsilon in RooAbsRealLValue::inRange()

So far, the RooAbsRealLValue::inRange() function used the following undocumented convention to check whether a value x is in the range with limits a and b: test if [x - eps * x, x + eps * x] overlaps with [a, b], where the parameter eps is defined as max(epsRel * x, epsAbs).

The values of the relative and absolute epsilons were inconsistent among the overloads:

With this release, the default absolute and relative epsilon is zero to avoid confusion. You can change them with RooNumber::setRangeEpsRel(epsRel) and RooNumber::setRangeEpsAbs(epsAbs).

TMVA

SOFIE : Code generation for fast inference of Deep Learning models

A large number of new features have been added in the TMVA SOFIE library. The list of all operators supported in the RModel class is the one provided below for the ONNX parser.

The interface of RModel::Generate has been changed to

RModel::Generate(Options options = Options::kDefault, int batchsize = 1)`

where Options is a new enumeration having 3 different values:

In addition, the RModel::Generate function takes as an additional optional argument the batch size (default is = 1) and the inference code can then be generated for the desired batch size.

SOFIE ONNX Parser

The ONNX parser supports now several new ONNX operators. The list of the current supported ONNX operators is the following:

In addition a Custom (user defined) operator is supported. An example of using a Custom operator is the program tmva/pymva/test/EmitCustomModel.cxx.

The ONNX parser supports also the fusing of the operators MatMul + Add in a Gemm operator and fusing Conv + Add and ConvTranspose + Add.

SOFIE Keras Parser

The Keras parser supports now model with input batch size not defined (e.g bathsize=-1), and by default the model is generated with batchsize=1. The Keras parser supports now in addition to the Dense layer the Conv2D layer, several activation functions (Relu, Selu, Sigmoid, Softmax, Tanh, LeakyRelu) and these other layers: BatchNormalization, Reshape, Convatenate, Add, Subtract, Multiply. Models with Dropout layers are supported in case the Dropout is used only during training and not inference.

For model having operators not yet supported in the Keras parser it is then reccomended to convert the Keras model to ONNX using the python tf2onnx tool.

SOFIE PyTorch Parser

If using PyTorch it is recommended to save the model directly in ONNX format instad of the native .pt format by using the torch.onnx.export function of PyTorch. The support for parsing directly .pt files is limited to the Gemm, Conv, Relu, Selu, Sigmoid and Transpose operators.

SOFIE RDataFrame Integration

The SOFIE inference is now integrated with RDataFrame, where a model can be evaluated on the columns of an input TTree with RDataFrame using the adapter functor class SofieFunctor. Examples of using SOFIE with RDataFrame are the new tutorials (in the tutorials/tmva directory) TMVA_SOFIE_RDataFrame.C or TMVA_SOFIE_RDataFrame.py. TMVA_SOFIE_RDataFrame_JIT.C is an example where the SOFIE model is generated and compiled at runtime using ROOT Cling and evaluated using RDataFrame.

RSofieReader

RSofieReader is a new class, which takes as input a model file (in ONNX, Keras, PyTorch or ROOT format) and generates and compiles the C++ code for the inference at run time using the ROOT JITing capabilities of CLING. An example of using this class is the tutorial TMVA_SOFIE_RSofieReader.C.

TMVA Pythonizations

New Pythonizations are available for TMVA allowing to replace the option string passed to several TMVA functions such as the TMVA::Factory constructor, the DataLoader::PrepareTrainingAndTestTree and Factory::BookMethod using Python function arguments. For example instead of writing an option string "NTrees=500:BoostType=AdaBoost" one can use in Python NTrees=500,BoostType='AdaBoost'. The new tmva tutorials TMVA_Higgs_Classification.py, TMVA_CNN_Classificaion.py and TMVA_RNN_Classificaton.py provide examples of using these new pythonizations.

2D Graphics Libraries

   import matplotlib.pyplot as plt
   import numpy as np
   points = np.array([3, 8, 1, 10, 5, 7])
   plt.plot(ypoints)
   plt. show()

It is now possible to do the same with the ROOT TGraph:

   double y[6] = {3, 8, 1, 10, 5, 7};
   auto g = new TGraph(6,y);
   g->Draw();

3D Graphics Libraries

REve now uses RenderCore to visualize 3D objects in JavaScript, replacing the use of Three.js. RenderCore is an open source WebGL rendering engine maintained by University of Ljubljana and tailored to the needs of the REve visualization framework. The minimized version is now included in ROOT source as built-in. REve with use of RendeCore gains:

REve screenshot 1 REve screenshot 2 REve screenshot 3

Geometry Libraries

   ROOT::Experimental::RGeomViewer viewer(geom);
   viewer.SaveImage("rootgeom.jpeg", 800, 600);

This runs normal WebGL rendering in headless web browser (Chrome or Firefox) and creates png or jpeg image out of it.

Database Libraries

Networking Libraries

THttpServer

GUI Libraries

   [localnode] rootssh user@remotenode
   [remotenode] root --web -e 'new TBrowser'

Script automatically configures ssh tunnel between local and remote nodes, one the remote node unix socket with strict 0700 mode is used. When ROOT running on remote node wants to display new web widget, script will automatically start web browser on local node with appropriate URL, accessing widget via configured ssh tunnel.

JavaScript ROOT

Tutorials

Class Reference Guide

Build, Configuration and Testing Infrastructure

PyROOT

from ROOT import TFile
with TFile("file1.root", "recreate") as outfile:
    hout = ROOT.TH1F(...)
    outfile.WriteObject(hout, "myhisto")
with TDirectory.TContext():
    # Open some file here
    file = ROOT.TFile(...)
    # Retrieve contents from the file
    histo = file.Get("myhisto")
 
# After the 'with' statement, the current directory is restored to ROOT.gROOT

Bugs and Issues fixed in this release

Release 6.28/02

Published on March 21, 2023

Bugs and Issues fixed in this release

Release 6.28/04

Published on May 7, 2023

Bugs and Issues fixed in this release

Release 6.28/06

Published on August 28, 2023

Bugs and Issues fixed in this release

Release 6.28/08

Published on October 13, 2023

Bugs and Issues fixed in this release

Release 6.28/10

Published on November 27, 2023

Bugs and Issues fixed in this release

This release also addresses a security issue. More details will follow.

Release 6.28/12

Published on January 30, 2024

Bugs and Issues fixed in this release

Besides the items below, this patch release features some improvements relative to run time performance. Firstly, the memory footprint of the plugin manager has been reduced. Moreover, the mechanism by which symbols are looked up by the interpreter was improved, avoiding to open and close a large number of libraries at startup, which improves considerably the user experience as well as cpu efficiency on batch jobs. More verbose output is now provided if wrong settings are used for the web-based widgets, the loopback device is always used and only one connection is allowed to RBrowser.

List of issues solved:

HEAD of the v6-28-00-patches branch

These changes will be part of a future 6.28/14.