ROOT::RDataFrame Class Reference

ROOT's RDataFrame offers a modern, high-level interface for analysis of data stored in TTree , CSV and other data formats, in C++ or Python.

Python interface

You can use RDataFrame in Python thanks to the dynamic Python/C++ translation of PyROOT. In general, the interface is the same as for C++, a simple example follows.

df = ROOT.RDataFrame("myTree", "myFile.root")
sum = df.Filter("x > 10").Sum("y")
print(sum.GetValue())

User code in the RDataFrame workflow

C++ code

In the simple example that was shown above, a C++ expression is passed to the Filter() operation as a string ("x > 0"), even if we call the method from Python. Indeed, under the hood, the analysis computations run in C++, while Python is just the interface language.

To perform more complex operations that don't fit into a simple expression string, you can just-in-time compile C++ functions - via the C++ interpreter cling - and use those functions in an expression. See the following snippet for an example:

# JIT a C++ function from Python
ROOT.gInterpreter.Declare("""
bool myFilter(float x) {
    return x > 10;
}
""")
 
df = ROOT.RDataFrame("myTree", "myFile.root")
# Use the function in an RDF operation
sum = df.Filter("myFilter(x)").Sum("y")
print(sum.GetValue())

To increase the performance even further, you can also pre-compile a C++ library with full code optimizations and load the function into the RDataFrame computation as follows.

ROOT.gSystem.Load("path/to/myLibrary.so") # Library with the myFilter function
ROOT.gInterpreter.Declare('#include "myLibrary.h"') # Header with the declaration of the myFilter function
df = ROOT.RDataFrame("myTree", "myFile.root")
sum = df.Filter("myFilter(x)").Sum("y")
print(sum.GetValue())

A more thorough explanation of how to use C++ code from Python can be found in the PyROOT manual.

Python code

ROOT also offers the option to compile Python functions with fundamental types and arrays thereof using Numba. Such compiled functions can then be used in a C++ expression provided to RDataFrame.

The function to be compiled should be decorated with ROOT.Numba.Declare, which allows to specify the parameter and return types. See the following snippet for a simple example or the full tutorial here.

@ROOT.Numba.Declare(["float"], "bool")
def myFilter(x):
    return x > 10
 
df = ROOT.RDataFrame("myTree", "myFile.root")
sum = df.Filter("Numba::myFilter(x)").Sum("y")
print(sum.GetValue())

It also works with collections: RVec objects of fundamental types can be transparently converted to/from numpy arrays:

@ROOT.Numba.Declare(['RVec<float>', 'int'], 'RVec<float>')
def pypowarray(numpyvec, pow):
    return numpyvec**pow
 
df.Define('array', 'ROOT::RVecF{1.,2.,3.}')\
  .Define('arraySquared', 'Numba::pypowarray(array, 2)')

Note that this functionality requires the Python packages numba and cffi to be installed.

Interoperability with NumPy

Conversion to NumPy arrays

Eventually, you probably would like to inspect the content of the RDataFrame or process the data further with Python libraries. For this purpose, we provide the AsNumpy() function, which returns the columns of your RDataFrame as a dictionary of NumPy arrays. See a few simple examples below or a full tutorial here.

Scalar columns

If your column contains scalar values of fundamental types (e.g., integers, floats), AsNumpy() produces NumPy arrays with the appropriate dtype:

rdf = ROOT.RDataFrame(10).Define("int_col", "1").Define("float_col", "2.3")
print(rdf.AsNumpy(["int_col", "float_col"]))
# Output: {'int_col': array([...], dtype=int32), 'float_col': array([...], dtype=float64)}

Columns containing non-fundamental types (e.g., objects, strings) will result in NumPy arrays with dtype=object.

Collection Columns

If your column contains collections of fundamental types (e.g., std::vector<int>), AsNumpy() produces a NumPy array with dtype=object where each element is a NumPy array representing the collection for its corresponding entry in the column.

If the collection at a certain entry contains values of fundamental types, or if it is a regularly shaped multi-dimensional array of a fundamental type, then the numpy array representing the collection for that entry will have the dtype associated with the value type of the collection, for example:

rdf = rdf.Define("v_col", "std::vector<int>{{1, 2, 3}}")
print(rdf.AsNumpy(["v_col", "int_col", "float_col"]))
# Output: {'v_col': array([array([1, 2, 3], dtype=int32), ...], dtype=object), ...}

If the collection at a certain entry contains values of a non-fundamental type, AsNumpy() will fallback on the default behavior and produce a NumPy array with dtype=object for that collection.

For more complex collection types in your entries, e.g. when every entry has a jagged array value, refer to the section on interoperability with AwkwardArray.

Processing data stored in NumPy arrays

In case you have data in NumPy arrays in Python and you want to process the data with ROOT, you can easily create an RDataFrame using ROOT.RDF.FromNumpy. The factory function accepts a dictionary where the keys are the column names and the values are NumPy arrays, and returns a new RDataFrame with the provided columns.

Only arrays of fundamental types (integers and floating point values) are supported and the arrays must have the same length. Data is read directly from the arrays: no copies are performed.

# Read data from NumPy arrays
# The column names in the RDataFrame are taken from the dictionary keys
x, y = numpy.array([1, 2, 3]), numpy.array([4, 5, 6])
df = ROOT.RDF.FromNumpy({"x": x, "y": y})
 
# Use RDataFrame as usual, e.g. write out a ROOT file
df.Define("z", "x + y").Snapshot("tree", "file.root")

Interoperability with AwkwardArray

The function for RDataFrame to Awkward conversion is ak.from_rdataframe(). The argument to this function accepts a tuple of strings that are the RDataFrame column names. By default this function returns ak.Array type.

import awkward as ak
import ROOT
 
array = ak.from_rdataframe(
    df,
    columns=(
        "x",
        "y",
        "z",
    ),
)

The function for Awkward to RDataFrame conversion is ak.to_rdataframe().

The argument to this function requires a dictionary: { <column name string> : <awkward array> }. This function always returns an RDataFrame object.

The arrays given for each column have to be equal length:

array_x = ak.Array(
    [
        {"x": [1.1, 1.2, 1.3]},
        {"x": [2.1, 2.2]},
        {"x": [3.1]},
        {"x": [4.1, 4.2, 4.3, 4.4]},
        {"x": [5.1]},
    ]
)
array_y = ak.Array([1, 2, 3, 4, 5])
array_z = ak.Array([[1.1], [2.1, 2.3, 2.4], [3.1], [4.1, 4.2, 4.3], [5.1]])
 
assert len(array_x) == len(array_y) == len(array_z)
 
df = ak.to_rdataframe({"x": array_x, "y": array_y, "z": array_z})

Construct histogram and profile models from a tuple

The Histo1D(), Histo2D(), Histo3D(), Profile1D() and Profile2D() methods return histograms and profiles, respectively, which can be constructed using a model argument.

In Python, we can specify the arguments for the constructor of such histogram or profile model with a Python tuple, as shown in the example below:

# First argument is a tuple with the arguments to construct a TH1D model
h = df.Histo1D(("histName", "histTitle", 64, 0., 128.), "myColumn")

AsRNode helper function

The ROOT::RDF::AsRNode function casts an RDataFrame node to the generic ROOT::RDF::RNode type. From Python, it can be used to pass any RDataFrame node as an argument of a C++ function, as shown below:

ROOT.gInterpreter.Declare("""
ROOT::RDF::RNode MyTransformation(ROOT::RDF::RNode df) {
    auto myFunc = [](float x){ return -x;};
    return df.Define("y", myFunc, {"x"});
}
""")
 
# Cast the RDataFrame head node
df = ROOT.RDataFrame("myTree", "myFile.root")
df_transformed = ROOT.MyTransformation(ROOT.RDF.AsRNode(df))
 
# ... or any other node
df2 = df.Filter("x > 42")
df2_transformed = ROOT.MyTransformation(ROOT.RDF.AsRNode(df2))

In addition, multi-threading and other low-level optimisations allow users to exploit all the resources available on their machines completely transparently.
Skip to the class reference or keep reading for the user guide.

In a nutshell:

ROOT::EnableImplicitMT(); // Tell ROOT you want to go parallel
ROOT::RDataFrame d("myTree", "file_*.root"); // Interface to TTree and TChain
auto myHisto = d.Histo1D("Branch_A"); // This books the (lazy) filling of a histogram
myHisto->Draw(); // Event loop is run here, upon first access to a result

Calculations are expressed in terms of a type-safe functional chain of actions and transformations, RDataFrame takes care of their execution. The implementation automatically puts in place several low level optimisations such as multi-thread parallelization and caching.

For the impatient user

You can directly see RDataFrame in action in our tutorials, in C++ or Python.

Table of Contents

• Cheat sheet
• Introduction
• Crash course
• Working with collections
• Transformations: manipulating data
• Actions: getting results
• Distributed execution in Python
• Performance tips and parallel execution
• More features
  • Systematic variations
  • RDataFrame objects as function arguments and return values
  • Storing RDataFrame objects in collections
  • Executing callbacks every N events
  • Default column lists
  • Special helper columns: rdfentry_ and rdfslot_
  • Just-in-time compilation: column type inference and explicit declaration of column types
  • User-defined custom actions
  • Dataset joins with friend trees
  • Reading data formats other than ROOT trees
  • Computation graphs (storing and reusing sets of transformations)
  • Visualizing the computation graph
  • Activating RDataFrame execution logs
  • Creating an RDataFrame from a dataset specification file
  • Adding a progress bar
  • Working with missing values in the dataset
  • Dealing with NaN or Inf values in the dataset
  • Translating TTree::Draw to RDataFrame
• Python interface
• Class reference

Cheat sheet

These are the operations which can be performed with RDataFrame.

Transformations

Transformations are a way to manipulate the data.

Transformation	Description
Alias()	Introduce an alias for a particular column name.
DefaultValueFor()	If the value of the input column is missing, provide a default value instead.
Define()	Create a new column in the dataset. Example usages include adding a column that contains the invariant mass of a particle, or a selection of elements of an array (e.g. only the pts of "good" muons).
DefinePerSample()	Define a new column that is updated when the input sample changes, e.g. when switching tree being processed in a chain.
DefineSlot()	Same as Define(), but the user-defined function must take an extra unsigned int slot as its first parameter. slot will take a different value, in the range [0, nThread-1], for each thread of execution. This is meant as a helper in writing thread-safe Define() transformations when using RDataFrame after ROOT::EnableImplicitMT(). DefineSlot() works just as well with single-thread execution: in that case slot will always be 0.
DefineSlotEntry()	Same as DefineSlot(), but the entry number is passed in addition to the slot number. This is meant as a helper in case the expression depends on the entry number. For details about entry numbers in multi-threaded runs, see here.
Filter()	Filter rows based on user-defined conditions.
FilterAvailable()	Specialized Filter. If the value of the input column is available, keep the entry, otherwise discard it.
FilterMissing()	Specialized Filter. If the value of the input column is missing, keep the entry, otherwise discard it.
Range()	Filter rows based on entry number (single-thread only).
Redefine()	Overwrite the value and/or type of an existing column. See Define() for more information.
RedefineSlot()	Overwrite the value and/or type of an existing column. See DefineSlot() for more information.
RedefineSlotEntry()	Overwrite the value and/or type of an existing column. See DefineSlotEntry() for more information.
Vary()	Register systematic variations for an existing column. Varied results are then extracted via VariationsFor().

Actions

Actions aggregate data into a result. Each one is described in more detail in the reference guide.

In the following, whenever we say an action "returns" something, we always mean it returns a smart pointer to it. Actions only act on events that pass all preceding filters.

Lazy actions only trigger the event loop when one of the results is accessed for the first time, making it easy to produce many different results in one event loop. Instant actions trigger the event loop instantly.

Lazy action	Description
Aggregate()	Execute a user-defined accumulation operation on the processed column values.
Book()	Book execution of a custom action using a user-defined helper object.
Cache()	Cache column values in memory. Custom columns can be cached as well, filtered entries are not cached. Users can specify which columns to save (default is all).
Count()	Return the number of events processed. Useful e.g. to get a quick count of the number of events passing a Filter.
Display()	Provides a printable representation of the dataset contents. The method returns a ROOT::RDF::RDisplay() instance which can print a tabular representation of the data or return it as a string.
Fill()	Fill a user-defined object with the values of the specified columns, as if by calling Obj.Fill(col1, col2, ...).
Graph()	Fills a TGraph with the two columns provided. If multi-threading is enabled, the order of the points may not be the one expected, it is therefore suggested to sort if before drawing.
GraphAsymmErrors()	Fills a TGraphAsymmErrors. Should be used for any type of graph with errors, including cases with errors on one of the axes only. If multi-threading is enabled, the order of the points may not be the one expected, it is therefore suggested to sort if before drawing.
Histo1D(), Histo2D(), Histo3D()	Fill a one-, two-, three-dimensional histogram with the processed column values.
HistoND()	Fill an N-dimensional histogram with the processed column values.
HistoNSparseD()	Fill an N-dimensional sparse histogram with the processed column values. Memory is allocated only for non-empty bins.
Max()	Return the maximum of processed column values. If the type of the column is inferred, the return type is double, the type of the column otherwise.
Mean()	Return the mean of processed column values.
Min()	Return the minimum of processed column values. If the type of the column is inferred, the return type is double, the type of the column otherwise.
Profile1D(), Profile2D()	Fill a one- or two-dimensional profile with the column values that passed all filters.
Reduce()	Reduce (e.g. sum, merge) entries using the function (lambda, functor...) passed as argument. The function must have signature T(T,T) where T is the type of the column. Return the final result of the reduction operation. An optional parameter allows initialization of the result object to non-default values.
Report()	Obtain statistics on how many entries have been accepted and rejected by the filters. See the section on named filters for a more detailed explanation. The method returns a ROOT::RDF::RCutFlowReport instance which can be queried programmatically to get information about the effects of the individual cuts.
Stats()	Return a TStatistic object filled with the input columns.
StdDev()	Return the unbiased standard deviation of the processed column values.
Sum()	Return the sum of the values in the column. If the type of the column is inferred, the return type is double, the type of the column otherwise.
Take()	Extract a column from the dataset as a collection of values, e.g. a std::vector<float> for a column of type float.

Instant action	Description
Foreach()	Execute a user-defined function on each entry. Users are responsible for the thread-safety of this callable when executing with implicit multi-threading enabled.
ForeachSlot()	Same as Foreach(), but the user-defined function must take an extra unsigned int slot as its first parameter. slot will take a different value, 0 to nThreads - 1, for each thread of execution. This is meant as a helper in writing thread-safe Foreach() actions when using RDataFrame after ROOT::EnableImplicitMT(). ForeachSlot() works just as well with single-thread execution: in that case slot will always be 0.
Snapshot()	Write the processed dataset to disk, in a new TTree or RNTuple and TFile. Custom columns can be saved as well, filtered entries are not saved. Users can specify which columns to save (default is all nominal columns). Columns resulting from Vary() can be included by setting the corresponding flag in RSnapshotOptions. Snapshot, by default, overwrites the output file if it already exists. Snapshot() can be made lazy setting the appropriate flag in the snapshot options.

Queries

These operations do not modify the dataframe or book computations but simply return information on the RDataFrame object.

Operation	Description
Describe()	Get useful information describing the dataframe, e.g. columns and their types.
GetColumnNames()	Get the names of all the available columns of the dataset.
GetColumnType()	Return the type of a given column as a string.
GetColumnTypeNamesList()	Return the list of type names of columns in the dataset.
GetDefinedColumnNames()	Get the names of all the defined columns.
GetFilterNames()	Return the names of all filters in the computation graph.
GetNRuns()	Return the number of event loops run by this RDataFrame instance so far.
GetNSlots()	Return the number of processing slots that RDataFrame will use during the event loop (i.e. the concurrency level).
ROOT::RDF::SaveGraph()	Store the computation graph of an RDataFrame in DOT format (graphviz) for easy inspection. See the relevant section for details.

Introduction

Users define their analysis as a sequence of operations to be performed on the dataframe object; the framework takes care of the management of the loop over entries as well as low-level details such as I/O and parallelization. RDataFrame provides methods to perform most common operations required by ROOT analyses; at the same time, users can just as easily specify custom code that will be executed in the event loop.

RDataFrame is built with a modular and flexible workflow in mind, summarised as follows:

1. Construct a dataframe object by specifying a dataset. RDataFrame supports TTree as well as TChain, CSV files, SQLite files, RNTuples, and it can be extended to custom data formats. From Python, NumPy arrays can be imported into RDataFrame as well.
2. Transform the dataframe by:
   • Applying filters. This selects only specific rows of the dataset.
   • Creating custom columns. Custom columns can, for example, contain the results of a computation that must be performed for every row of the dataset.
3. Produce results. Actions are used to aggregate data into results. Most actions are lazy, i.e. they are not executed on the spot, but registered with RDataFrame and executed only when a result is accessed for the first time.

Make sure to book all transformations and actions before you access the contents of any of the results. This lets RDataFrame accumulate work and then produce all results at the same time, upon first access to any of them.

The following table shows how analyses based on TTreeReader and TTree::Draw() translate to RDataFrame. Follow the crash course to discover more idiomatic and flexible ways to express analyses with RDataFrame.

TTreeReader	ROOT::RDataFrame
TTreeReader reader("myTree", file); TTreeReaderValue<A_t> a(reader, "A"); TTreeReaderValue<B_t> b(reader, "B"); TTreeReaderValue<C_t> c(reader, "C"); while(reader.Next()) { if(IsGoodEvent(*a, *b, *c)) DoStuff(*a, *b, *c); }	ROOT::RDataFrame d("myTree", file, {"A", "B", "C"}); d.Filter(IsGoodEvent).Foreach(DoStuff);
TTree::Draw	ROOT::RDataFrame
auto *tree = file->Get<TTree>("myTree"); tree->Draw("x", "y > 2");	ROOT::RDataFrame df("myTree", file); auto h = df.Filter("y > 2").Histo1D("x"); h->Draw()
tree->Draw("jet_eta", "weight*(event == 1)");	df.Filter("event == 1").Histo1D("jet_eta", "weight"); // or the fully compiled version: df.Filter([] (ULong64_t e) { return e == 1; }, {"event"}).Histo1D<RVec<float>>("jet_eta", "weight");
// object selection: for each event, fill histogram with array of selected pts tree->Draw('Muon_pt', 'Muon_pt > 100');	// with RDF, arrays are read as ROOT::VecOps::RVec objects df.Define("good_pt", "Muon_pt[Muon_pt > 100]").Histo1D("good_pt")

Crash course

All snippets of code presented in the crash course can be executed in the ROOT interpreter. Simply precede them with

using namespace ROOT; // RDataFrame's namespace

which is omitted for brevity. The terms "column" and "branch" are used interchangeably.

Creating an RDataFrame

RDataFrame's constructor is where the user specifies the dataset and, optionally, a default set of columns that operations should work with. Here are the most common methods to construct an RDataFrame object:

// single file -- all constructors are equivalent
TFile *f = TFile::Open("file.root");
TTree *t = f.Get<TTree>("treeName");
 
ROOT::RDataFrame d1("treeName", "file.root");
ROOT::RDataFrame d2("treeName", f); // same as TTreeReader
ROOT::RDataFrame d3(*t);
 
// multiple files -- all constructors are equivalent
TChain chain("myTree");
chain.Add("file1.root");
chain.Add("file2.root");
 
ROOT::RDataFrame d4("myTree", {"file1.root", "file2.root"});
std::vector<std::string> files = {"file1.root", "file2.root"};
ROOT::RDataFrame d5("myTree", files);
ROOT::RDataFrame d6("myTree", "file*.root"); // the glob is passed as-is to TChain's constructor
ROOT::RDataFrame d7(chain);

Additionally, users can construct an RDataFrame with no data source by passing an integer number. This is the number of rows that will be generated by this RDataFrame.

ROOT::RDataFrame d(10); // a RDF with 10 entries (and no columns/branches, for now)
d.Foreach([] { static int i = 0; std::cout << i++ << std::endl; }); // silly example usage: count to ten

This is useful to generate simple datasets on the fly: the contents of each event can be specified with Define() (explained below). For example, we have used this method to generate Pythia events and write them to disk in parallel (with the Snapshot action).

For data sources other than TTrees and TChains, RDataFrame objects are constructed using ad-hoc factory functions (see e.g. FromCSV(), FromSqlite(), FromArrow()):

auto df = ROOT::RDF::FromCSV("input.csv");
// use df as usual

Filling a histogram

Let's now tackle a very common task, filling a histogram:

// Fill a TH1D with the "MET" branch
ROOT::RDataFrame d("myTree", "file.root");
auto h = d.Histo1D("MET");
h->Draw();

The first line creates an RDataFrame associated to the TTree "myTree". This tree has a branch named "MET".

Histo1D() is an action; it returns a smart pointer (a ROOT::RDF::RResultPtr, to be precise) to a TH1D histogram filled with the MET of all events. If the quantity stored in the column is a collection (e.g. a vector or an array), the histogram is filled with all vector elements for each event.

You can use the objects returned by actions as if they were pointers to the desired results. There are many other possible actions, and all their results are wrapped in smart pointers; we'll see why in a minute.

Applying a filter

Let's say we want to cut over the value of branch "MET" and count how many events pass this cut. This is one way to do it:

ROOT::RDataFrame d("myTree", "file.root");
auto c = d.Filter("MET > 4.").Count(); // computations booked, not run
std::cout << *c << std::endl; // computations run here, upon first access to the result

The filter string (which must contain a valid C++ expression) is applied to the specified columns for each event; the name and types of the columns are inferred automatically. The string expression is required to return a bool which signals whether the event passes the filter (true) or not (false).

You can think of your data as "flowing" through the chain of calls, being transformed, filtered and finally used to perform actions. Multiple Filter() calls can be chained one after another.

Using string filters is nice for simple things, but they are limited to specifying the equivalent of a single return statement or the body of a lambda, so it's cumbersome to use strings with more complex filters. They also add a small runtime overhead, as ROOT needs to just-in-time compile the string into C++ code. When more freedom is required or runtime performance is very important, a C++ callable can be specified instead (a lambda in the following snippet, but it can be any kind of function or even a functor class), together with a list of column names. This snippet is analogous to the one above:

ROOT::RDataFrame d("myTree", "file.root");
auto metCut = [](double x) { return x > 4.; }; // a C++11 lambda function checking "x > 4"
auto c = d.Filter(metCut, {"MET"}).Count();
std::cout << *c << std::endl;

An example of a more complex filter expressed as a string containing C++ code is shown below

ROOT::RDataFrame d("myTree", "file.root");
auto df = d.Define("p", "std::array<double, 4> p{px, py, pz}; return p;")
           .Filter("double p2 = 0.0; for (auto&& x : p) p2 += x*x; return sqrt(p2) < 10.0;");

The code snippet above defines a column p that is a fixed-size array using the component column names and then filters on its magnitude by looping over its elements. It must be noted that the usage of strings to define columns like the one above is currently the only possibility when using PyROOT. When writing expressions as such, only constants and data coming from other columns in the dataset can be involved in the code passed as a string. Local variables and functions cannot be used, since the interpreter will not know how to find them. When capturing local state is necessary, it must first be declared to the ROOT C++ interpreter.

More information on filters and how to use them to automatically generate cutflow reports can be found below.

Defining custom columns

Let's now consider the case in which "myTree" contains two quantities "x" and "y", but our analysis relies on a derived quantity z = sqrt(x*x + y*y). Using the Define() transformation, we can create a new column in the dataset containing the variable "z":

ROOT::RDataFrame d("myTree", "file.root");
auto sqrtSum = [](double x, double y) { return sqrt(x*x + y*y); };
auto zMean = d.Define("z", sqrtSum, {"x","y"}).Mean("z");
std::cout << *zMean << std::endl;

Define() creates the variable "z" by applying sqrtSum to "x" and "y". Later in the chain of calls we refer to variables created with Define() as if they were actual tree branches/columns, but they are evaluated on demand, at most once per event. As with filters, Define() calls can be chained with other transformations to create multiple custom columns. Define() and Filter() transformations can be concatenated and intermixed at will.

As with filters, it is possible to specify new columns as string expressions. This snippet is analogous to the one above:

ROOT::RDataFrame d("myTree", "file.root");
auto zMean = d.Define("z", "sqrt(x*x + y*y)").Mean("z");
std::cout << *zMean << std::endl;

Again the names of the columns used in the expression and their types are inferred automatically. The string must be valid C++ and it is just-in-time compiled. The process has a small runtime overhead and like with filters it is currently the only possible approach when using PyROOT.

Previously, when showing the different ways an RDataFrame can be created, we showed a constructor that takes a number of entries as a parameter. In the following example we show how to combine such an "empty" RDataFrame with Define() transformations to create a dataset on the fly. We then save the generated data on disk using the Snapshot() action.

ROOT::RDataFrame d(100); // an RDF that will generate 100 entries (currently empty)
int x = -1;
auto d_with_columns = d.Define("x", []()->int { return ++x; }).Define("xx", []()->int { return x*x; });
d_with_columns.Snapshot("myNewTree", "newfile.root");

This example is slightly more advanced than what we have seen so far. First, it makes use of lambda captures (a simple way to make external variables available inside the body of C++ lambdas) to act on the same variable x from both Define() transformations. Second, we have stored the transformed dataframe in a variable. This is always possible, since at each point of the transformation chain users can store the status of the dataframe for further use (more on this below).

You can read more about defining new columns here.

A graph composed of two branches, one starting with a filter and one with a define. The end point of a branch is always an action.

Running on a range of entries

It is sometimes necessary to limit the processing of the dataset to a range of entries. For this reason, the RDataFrame offers the concept of ranges as a node of the RDataFrame chain of transformations; this means that filters, columns and actions can be concatenated to and intermixed with Range()s. If a range is specified after a filter, the range will act exclusively on the entries passing the filter – it will not even count the other entries! The same goes for a Range() hanging from another Range(). Here are some commented examples:

ROOT::RDataFrame d("myTree", "file.root");
// Here we store a dataframe that loops over only the first 30 entries in a variable
auto d30 = d.Range(30);
// This is how you pick all entries from 15 onwards
auto d15on = d.Range(15, 0);
// We can specify a stride too, in this case we pick an event every 3
auto d15each3 = d.Range(0, 15, 3);

Note that ranges are not available when multi-threading is enabled. More information on ranges is available here.

Executing multiple actions in the same event loop

As a final example let us apply two different cuts on branch "MET" and fill two different histograms with the "pt_v" of the filtered events. By now, you should be able to easily understand what is happening:

RDataFrame d("treeName", "file.root");
auto h1 = d.Filter("MET > 10").Histo1D("pt_v");
auto h2 = d.Histo1D("pt_v");
h1->Draw();       // event loop is run once here
h2->Draw("SAME"); // no need to run the event loop again

RDataFrame executes all above actions by running the event-loop only once. The trick is that actions are not executed at the moment they are called, but they are lazy, i.e. delayed until the moment one of their results is accessed through the smart pointer. At that time, the event loop is triggered and all results are produced simultaneously.

Properly exploiting RDataFrame laziness

For yet another example of the difference between the correct and incorrect running of the event-loop, see the following two code snippets. We assume our ROOT file has branches a, b and c.

The correct way - the dataset is only processed once.

df_correct = ROOT.RDataFrame(treename, filename);
 
h_a = df_correct.Histo1D("a")
h_b = df_correct.Histo1D("b")
h_c = df_correct.Histo1D("c")
 
h_a_val = h_a.GetValue()
h_b_val = h_b.GetValue()
h_c_val = h_c.GetValue()
 
print(f"How many times was the data set processed? {df_wrong.GetNRuns()} time.") # The answer will be 1 time.

An incorrect way - the dataset is processed three times.

df_incorrect = ROOT.RDataFrame(treename, filename);
 
h_a = df_incorrect.Histo1D("a")
h_a_val = h_a.GetValue()
 
h_b = df_incorrect.Histo1D("b")
h_b_val = h_b.GetValue()
 
h_c = df_incorrect.Histo1D("c")
h_c_val = h_c.GetValue()
 
print(f"How many times was the data set processed? {df_wrong.GetNRuns()} times.") # The answer will be 3 times.

It is therefore good practice to declare all your transformations and actions before accessing their results, allowing RDataFrame to run the loop once and produce all results in one go.

Going parallel

Let's say we would like to run the previous examples in parallel on several cores, dividing events fairly between cores. The only modification required to the snippets would be the addition of this line before constructing the main dataframe object:

ROOT::EnableImplicitMT();

Simple as that. More details are given below.

Working with collections and object selections

RDataFrame reads collections as the special type ROOT::RVec: for example, a column containing an array of floating point numbers can be read as a ROOT::RVecF. C-style arrays (with variable or static size), STL vectors and most other collection types can be read this way.

RVec is a container similar to std::vector (and can be used just like a std::vector) but it also offers a rich interface to operate on the array elements in a vectorised fashion, similarly to Python's NumPy arrays.

For example, to fill a histogram with the "pt" of selected particles for each event, Define() can be used to create a column that contains the desired array elements as follows:

// h is filled with all the elements of `good_pts`, for each event
auto h = df.Define("good_pts", [](const ROOT::RVecF &pt) { return pt[pt > 0]; })
           .Histo1D("good_pts");

And in Python:

h = df.Define("good_pts", "pt[pt > 0]").Histo1D("good_pts")

Learn more at ROOT::VecOps::RVec.

ROOT provides convenience utility functions to work with RVec which are available in the ROOT::VecOps namespace

Transformations: manipulating data

Filters

A filter is created through a call to Filter(f, columnList) or Filter(filterString). In the first overload, f can be a function, a lambda expression, a functor class, or any other callable object. It must return a bool signalling whether the event has passed the selection (true) or not (false). It should perform "read-only" operations on the columns, and should not have side-effects (e.g. modification of an external or static variable) to ensure correctness when implicit multi-threading is active. The second overload takes a string with a valid C++ expression in which column names are used as variable names (e.g. Filter("x[0] + x[1] > 0")). This is a convenience feature that comes with a certain runtime overhead: C++ code has to be generated on the fly from this expression before using it in the event loop. See the paragraph about "Just-in-time compilation" below for more information.

RDataFrame only evaluates filters when necessary: if multiple filters are chained one after another, they are executed in order and the first one returning false causes the event to be discarded and triggers the processing of the next entry. If multiple actions or transformations depend on the same filter, that filter is not executed multiple times for each entry: after the first access it simply serves a cached result.

Named filters and cutflow reports

An optional string parameter name can be passed to the Filter() method to create a named filter. Named filters work as usual, but also keep track of how many entries they accept and reject.

Statistics are retrieved through a call to the Report() method:

• when Report() is called on the main RDataFrame object, it returns a ROOT::RDF::RResultPtr<RCutFlowReport> relative to all named filters declared up to that point
• when called on a specific node (e.g. the result of a Define() or Filter()), it returns a ROOT::RDF::RResultPtr<RCutFlowReport> relative all named filters in the section of the chain between the main RDataFrame and that node (included).

Stats are stored in the same order as named filters have been added to the graph, and refer to the latest event-loop that has been run using the relevant RDataFrame.

Ranges

When RDataFrame is not being used in a multi-thread environment (i.e. no call to EnableImplicitMT() was made), Range() transformations are available. These act very much like filters but instead of basing their decision on a filter expression, they rely on begin,end and stride parameters.

• begin: initial entry number considered for this range.
• end: final entry number (excluded) considered for this range. 0 means that the range goes until the end of the dataset.
• stride: process one entry of the [begin, end) range every stride entries. Must be strictly greater than 0.

The actual number of entries processed downstream of a Range() node will be (end - begin)/stride (or less if less entries than that are available).

Note that ranges act "locally", not based on the global entry count: Range(10,50) means "skip the first 10 entries that reach this node*, let the next 40 entries pass, then stop processing". If a range node hangs from a filter node, and the range has a begin parameter of 10, that means the range will skip the first 10 entries that pass the preceding filter.

Ranges allow "early quitting": if all branches of execution of a functional graph reached their end value of processed entries, the event-loop is immediately interrupted. This is useful for debugging and quick data explorations.

Custom columns

Custom columns are created by invoking Define(name, f, columnList). As usual, f can be any callable object (function, lambda expression, functor class...); it takes the values of the columns listed in columnList (a list of strings) as parameters, in the same order as they are listed in columnList. f must return the value that will be assigned to the temporary column.

A new variable is created called name, accessible as if it was contained in the dataset from subsequent transformations/actions.

Use cases include:

• caching the results of complex calculations for easy and efficient multiple access
• extraction of quantities of interest from complex objects
• branch aliasing, i.e. changing the name of a branch

An exception is thrown if the name of the new column/branch is already in use for another branch in the TTree.

It is also possible to specify the quantity to be stored in the new temporary column as a C++ expression with the method Define(name, expression). For example this invocation

df.Define("pt", "sqrt(px*px + py*py)");

will create a new column called "pt" the value of which is calculated starting from the columns px and py. The system builds a just-in-time compiled function starting from the expression after having deduced the list of necessary branches from the names of the variables specified by the user.

Custom columns as function of slot and entry number

It is possible to create custom columns also as a function of the processing slot and entry numbers. The methods that can be invoked are:

• DefineSlot(name, f, columnList). In this case the callable f has this signature R(unsigned int, T1, T2, ...): the first parameter is the slot number which ranges from 0 to ROOT::GetThreadPoolSize() - 1.
• DefineSlotEntry(name, f, columnList). In this case the callable f has this signature R(unsigned int, ULong64_t, T1, T2, ...): the first parameter is the slot number while the second one the number of the entry being processed.

Actions: getting results

Instant and lazy actions

Actions can be instant or lazy. Instant actions are executed as soon as they are called, while lazy actions are executed whenever the object they return is accessed for the first time. As a rule of thumb, actions with a return value are lazy, the others are instant.

Return type of a lazy action

When a lazy action is called, it returns a ROOT::RDF::RResultPtr<T>, where T is the type of the result of the action. The final result will be stored in the RResultPtr, and can be retrieved by dereferencing it or via its GetValue method. Retrieving the result also starts the event loop if the result hasn't been produced yet.

The RResultPtr shares ownership of the result object. To directly access result, use:

ROOT::RDF::RResultPtr<TH1D> histo = rdf.Histo1D(...);
histo->Draw(); // Starts running the event loop

To return results from functions, a copy of the underlying shared_ptr can be obtained:

std::shared_ptr<TH1D> ProduceResult(const char *columnname) {
   ROOT::RDF::RResultPtr<TH1D> histo = rdf.Histo1D(*h, columname);
   return histo.GetSharedPtr(); // Runs the event loop
}

If the result had been returned by reference or bare pointer, it would have gotten destroyed when the function exits.

To share ownership but not produce the result ("keep it lazy"), copy the RResultPtr:

std::vector<RResultPtr<TH1D>> allHistograms;
ROOT::RDF::RResultPtr<TH1D> BookHistogram(const char *columnname) {
   ROOT::RDF::RResultPtr<TH1D> histo = rdf.Histo1D(*h, columname);
   allHistograms.push_back(histo); // Will not produce the result yet
   return histo;
}

Actions that return collections

If the type of the return value of an action is a collection, e.g. std::vector<int>, you can iterate its elements directly through the wrapping RResultPtr:

ROOT::RDataFrame df{5};
auto df1 = df.Define("x", []{ return 42; });
for (const auto &el: df1.Take<int>("x")){
   std::cout << "Element: " << el << "\n";
}

df = ROOT.RDataFrame(5).Define("x", "42")
for el in df.Take[int]("x"):
    print(f"Element: {el}")

Actions and readers

An action that needs values for its computations will request it from a reader, e.g. a column created via Define or available from the input dataset. The action will request values from each column of the list of input columns (either inferred or specified by the user), in order. For example:

ROOT::RDataFrame df{1};
auto df1 = df.Define("x", []{ return 11; });
auto df2 = df1.Define("y", []{ return 22; });
auto graph = df2.Graph<int, int>("x","y");

The Graph action is going to request first the value from column "x", then that of column "y". Specifically, the order of execution of the operations of nodes in this branch of the computation graph is guaranteed to be top to bottom.

Distributed execution

RDataFrame applications can be executed in parallel through distributed computing frameworks on a set of remote machines thanks to the Python package ROOT.RDF.Distributed. This Python-only package allows to scale the optimized performance RDataFrame can achieve on a single machine to multiple nodes at the same time. It is designed so that different backends can be easily plugged in, currently supporting Apache Spark and Dask. Here is a minimal example usage of distributed RDataFrame:

import ROOT
from distributed import Client
# It still accepts the same constructor arguments as traditional RDataFrame
# but needs a client object which allows connecting to one of the supported
# schedulers (read more info below)
client = Client(...)
df = ROOT.RDataFrame("mytree", "myfile.root", executor=client)
 
# Continue the application with the traditional RDataFrame API
sum = df.Filter("x > 10").Sum("y")
h = df.Histo1D(("name", "title", 10, 0, 10), "x")
 
print(sum.GetValue())
h.Draw()

The main goal of this package is to support running any RDataFrame application distributedly. Nonetheless, not all parts of the RDataFrame API currently work with this package. The subset that is currently available is:

• Alias
• AsNumpy
• Count
• DefaultValueFor
• Define
• DefinePerSample
• Filter
• FilterAvailable
• FilterMissing
• Graph
• Histo[1,2,3]D
• HistoND, HistoNSparseD
• Max
• Mean
• Min
• Profile[1,2,3]D
• Redefine
• Snapshot
• Stats
• StdDev
• Sum
• Systematic variations: Vary and VariationsFor.
• Parallel submission of distributed graphs: RunGraphs.
• Information about the dataframe: GetColumnNames.

with support for more operations coming in the future. Currently, to the supported data sources belong TTree, TChain, RNTuple and RDatasetSpec.

Connecting to a Spark cluster

In order to distribute the RDataFrame workload, you can connect to a Spark cluster you have access to through the official Spark API, then hook the connection instance to the distributed RDataFrame object like so:

import pyspark
import ROOT
 
# Create a SparkContext object with the right configuration for your Spark cluster
conf = SparkConf().setAppName(appName).setMaster(master)
sc = SparkContext(conf=conf)
 
# The Spark RDataFrame constructor accepts an optional "sparkcontext" parameter
# and it will distribute the application to the connected cluster
df = ROOT.RDataFrame("mytree", "myfile.root", executor = sc)

Note that with the usage above the case of executor = None is not supported. One can explicitly create a ROOT.RDF.Distributed.Spark.RDataFrame object in order to get a default instance of SparkContext in case it is not already provided as argument.

Connecting to a Dask cluster

Similarly, you can connect to a Dask cluster by creating your own connection object which internally operates with one of the cluster schedulers supported by Dask (more information in the Dask distributed docs):

import ROOT
from dask.distributed import Client
# In a Python script the Dask client needs to be initalized in a context
# Jupyter notebooks / Python session don't need this
if __name__ == "__main__":
    # With an already setup cluster that exposes a Dask scheduler endpoint
    client = Client("dask_scheduler.domain.com:8786")
 
    # The Dask RDataFrame constructor accepts the Dask Client object as an optional argument
    df = ROOT.RDataFrame("mytree","myfile.root", executor=client)
    # Proceed as usual
    df.Define("x","someoperation").Histo1D(("name", "title", 10, 0, 10), "x")

Note that with the usage above the case of executor = None is not supported. One can explicitly create a ROOT.RDF.Distributed.Dask.RDataFrame object in order to get a default instance of distributed.Client in case it is not already provided as argument. This will run multiple processes on the local machine using all available cores.

Choosing the number of distributed tasks

A distributed RDataFrame has internal logic to define in how many chunks the input dataset will be split before sending tasks to the distributed backend. Each task reads and processes one of said chunks. The logic is backend-dependent, but generically tries to infer how many cores are available in the cluster through the connection object. The number of tasks will be equal to the inferred number of cores. There are cases where the connection object of the chosen backend doesn't have information about the actual resources of the cluster. An example of this is when using Dask to connect to a batch system. The client object created at the beginning of the application does not automatically know how many cores will be available during distributed execution, since the jobs are submitted to the batch system after the creation of the connection. In such cases, the logic is to default to process the whole dataset in 2 tasks.

The number of tasks submitted for distributed execution can be also set programmatically, by providing the optional keyword argument npartitions when creating the RDataFrame object. This parameter is accepted irrespectively of the backend used:

import ROOT
 
if __name__ == "__main__":
    # The `npartitions` optional argument tells the RDataFrame how many tasks are desired
    df = ROOT.RDataFrame("mytree", "myfile.root", executor=SupportedExecutor(...), npartitions=NPARTITIONS)
    # Proceed as usual
    df.Define("x","someoperation").Histo1D(("name", "title", 10, 0, 10), "x")

Note that when processing a TTree or TChain dataset, the npartitions value should not exceed the number of clusters in the dataset. The number of clusters in a TTree can be retrieved by typing rootls -lt myfile.root at a command line.

Distributed FromSpec

RDataFrame can be also built from a JSON sample specification file using the FromSpec function. In distributed mode, two arguments need to be provided: the path to the specification jsonFile (same as for local RDF case) and an additional executor argument - in the same manner as for the RDataFrame constructors above - an executor can either be a spark connection or a dask client. If no second argument is given, the local version of FromSpec will be run. Here is an example of FromSpec usage in distributed RDF using either spark or dask backends. For more information on FromSpec functionality itself please refer to FromSpec documentation. Note that adding metadata and friend information is supported, but adding the global range will not be respected in the distributed execution.

Using spark:

import pyspark
import ROOT
 
conf = SparkConf().setAppName(appName).setMaster(master)
sc = SparkContext(conf=conf)
 
# The FromSpec function accepts an optional "sparkcontext" parameter
# and it will distribute the application to the connected cluster
df_fromspec = ROOT.RDF.Experimental.FromSpec("myspec.json", executor = sc)
# Proceed as usual
df_fromspec.Define("x","someoperation").Histo1D(("name", "title", 10, 0, 10), "x")

Using dask:

import ROOT
from dask.distributed import Client
 
if __name__ == "__main__":
    client = Client("dask_scheduler.domain.com:8786")
 
    # The FromSpec function accepts the Dask Client object as an optional argument
    df_fromspec = ROOT.RDF.Experimental.FromSpec("myspec.json", executor=client)
    # Proceed as usual
    df_fromspec.Define("x","someoperation").Histo1D(("name", "title", 10, 0, 10), "x")

Distributed Snapshot

The Snapshot operation behaves slightly differently when executed distributedly. First off, it requires the path supplied to the Snapshot call to be accessible from any worker of the cluster and from the client machine (in general it should be provided as an absolute path). Another important difference is that n separate files will be produced, where n is the number of dataset partitions. As with local RDataFrame, the result of a Snapshot on a distributed RDataFrame is another distributed RDataFrame on which we can define a new computation graph and run more distributed computations.

Distributed RunGraphs

Submitting multiple distributed RDataFrame executions is supported through the RunGraphs function. Similarly to its local counterpart, the function expects an iterable of objects representing an RDataFrame action. Each action will be triggered concurrently to send multiple computation graphs to a distributed cluster at the same time:

import ROOT
 
# Create 3 different dataframes and book an histogram on each one
histoproxies = [
   ROOT.RDataFrame(100, executor=SupportedExecutor(...))
         .Define("x", "rdfentry_")
         .Histo1D(("name", "title", 10, 0, 100), "x")
   for _ in range(4)
]
 
# Execute the 3 computation graphs
ROOT.RDF.RunGraphs(histoproxies)
# Retrieve all the histograms in one go
histos = [histoproxy.GetValue() for histoproxy in histoproxies]

Every distributed backend supports this feature and graphs belonging to different backends can be still triggered with a single call to RunGraphs (e.g. it is possible to send a Spark job and a Dask job at the same time).

Histogram models in distributed mode

When calling a Histo*D operation in distributed mode, remember to pass to the function the model of the histogram to be computed, e.g. the axis range and the number of bins:

import ROOT
 
if __name__ == "__main__":
    df = ROOT.RDataFrame("mytree","myfile.root",executor=SupportedExecutor(...)).Define("x","someoperation")
    # The model can be passed either as a tuple with the arguments in the correct order
    df.Histo1D(("name", "title", 10, 0, 10), "x")
    # Or by creating the specific struct
    model = ROOT.RDF.TH1DModel("name", "title", 10, 0, 10)
    df.Histo1D(model, "x")

Without this, two partial histograms resulting from two distributed tasks would have incompatible binning, thus leading to errors when merging them. Failing to pass a histogram model will raise an error on the client side, before starting the distributed execution.

Live visualization in distributed mode with dask

The live visualization feature allows real-time data representation of plots generated during the execution of a distributed RDataFrame application. It enables visualizing intermediate results as they are computed across multiple nodes of a Dask cluster by creating a canvas and continuously updating it as partial results become available.

The LiveVisualize() function can be imported from the Python package ROOT.RDF.Distributed:

import ROOT
 
LiveVisualize = ROOT.RDF.Distributed.LiveVisualize

The function takes drawable objects (e.g. histograms) and optional callback functions as argument, it accepts 4 different input formats:

• Passing a list or tuple of drawables: You can pass a list or tuple containing the plots you want to visualize. For example:

LiveVisualize([h_gaus, h_exp, h_random])

• Passing a list or tuple of drawables with a global callback function: You can also include a global callback function that will be applied to all plots. For example:

def set_fill_color(hist):
    hist.SetFillColor("kBlue")
 
LiveVisualize([h_gaus, h_exp, h_random], set_fill_color)

• Passing a Dictionary of drawables and callback functions: For more control, you can create a dictionary where keys are plots and values are corresponding (optional) callback functions. For example:

plot_callback_dict = {
    graph: set_marker,
    h_exp: fit_exp,
    tprofile_2d: None
}
 
LiveVisualize(plot_callback_dict)

• Passing a Dictionary of drawables and callback functions with a global callback function: You can also combine a dictionary of plots and callbacks with a global callback function:

LiveVisualize(plot_callback_dict, write_to_tfile)

Note

The allowed operations to pass to LiveVisualize are:

• Histo1D(), Histo2D(), Histo3D()
• Graph()
• Profile1D(), Profile2D()

Warning

The Live Visualization feature is only supported for the Dask backend.

Injecting C++ code and using external files into distributed RDF script

Distributed RDF provides an interface for the users who want to inject the C++ code (via header files, shared libraries or declare the code directly) into their distributed RDF application, or their application needs to use information from external files which should be distributed to the workers (for example, a JSON or a txt file with necessary parameters information).

The examples below show the usage of these interface functions: firstly, how this is done in a local Python RDF application and secondly, how it is done distributedly.

Include and distribute header files.

# Local RDataFrame script
ROOT.gInterpreter.AddIncludePath("myheader.hxx")
df.Define(...)
 
# Distributed RDF script
ROOT.RDF.Distributed.DistributeHeaders("myheader.hxx")
df.Define(...)

Load and distribute shared libraries

# Local RDataFrame script
ROOT.gSystem.Load("my_library.so")
df.Define(...)
 
# Distributed RDF script
ROOT.RDF.Distributed.DistributeSharedLibs("my_library.so")
df.Define(...)

Declare and distribute the cpp code

The cpp code is always available to all dataframes.

# Local RDataFrame script
ROOT.gInterpreter.Declare("my_code")
df.Define(...)
 
# Distributed RDF script
ROOT.RDF.Distributed.DistributeCppCode("my_code")
df.Define(...)

Distribute additional files (other than headers or shared libraries).

# Local RDataFrame script is not applicable here as local RDF application can simply access the external files it needs.
 
# Distributed RDF script
ROOT.RDF.Distributed.DistributeFiles("my_file")
df.Define(...)

Performance tips and parallel execution

As pointed out before in this document, RDataFrame can transparently perform multi-threaded event loops to speed up the execution of its actions. Users have to call ROOT::EnableImplicitMT() before constructing the RDataFrame object to indicate that it should take advantage of a pool of worker threads. Each worker thread processes a distinct subset of entries, and their partial results are merged before returning the final values to the user.

By default, RDataFrame will use as many threads as the hardware supports, using up all the resources on a machine. This might be undesirable on shared computing resources such as a batch cluster. Therefore, when running on shared computing resources, use

ROOT::EnableImplicitMT(numThreads)

or export an environment variable:

export ROOT_MAX_THREADS=numThreads
root.exe rdfAnalysis.cxx
# or
ROOT_MAX_THREADS=4 python rdfAnalysis.py

replacing numThreads with the number of CPUs/slots that were allocated for this job.

Warning

There are no guarantees on the order in which threads will process the batches of entries. In particular, note that this means that, for multi-thread event loops, there is no guarantee on the order in which Snapshot() will write entries: they could be scrambled with respect to the input dataset. The values of the special rdfentry_ column will also not correspond to the entry numbers in the input dataset (e.g. TChain) in multi-threaded runs. Likewise, Take(), AsNumpy(), ... do not preserve the original ordering.

Thread-safety of user-defined expressions

RDataFrame operations such as Histo1D() or Snapshot() are guaranteed to work correctly in multi-thread event loops. User-defined expressions, such as strings or lambdas passed to Filter(), Define(), Foreach(), Reduce() or Aggregate() will have to be thread-safe, i.e. it should be possible to call them concurrently from different threads.

Note that simple Filter() and Define() transformations will inherently satisfy this requirement: Filter() / Define() expressions will often be pure in the functional programming sense (no side-effects, no dependency on external state), which eliminates all risks of race conditions.

In order to facilitate writing of thread-safe operations, some RDataFrame features such as Foreach(), Define() or OnPartialResult() offer thread-aware counterparts (ForeachSlot(), DefineSlot(), OnPartialResultSlot()): their only difference is that they will pass an extra slot argument (an unsigned integer) to the user-defined expression. When calling user-defined code concurrently, RDataFrame guarantees that different threads will see different values of the slot parameter, where slot will be a number between 0 and GetNSlots() - 1. Note that not all slot numbers may be reached, or some slots may be reached more often depending on how computation tasks are scheduled. In other words, within a slot, computations run sequentially, and events are processed sequentially. Note that the same slot might be associated to different threads over the course of a single event loop, but two threads will never receive the same slot at the same time. This extra parameter might facilitate writing safe parallel code by having each thread write/modify a different processing slot, e.g. a different element of a list. See here for an example usage of ForeachSlot().

Parallel execution of multiple RDataFrame event loops

A complex analysis may require multiple separate RDataFrame computation graphs to produce all desired results. This poses the challenge that the event loops of each computation graph can be parallelized, but the different loops run sequentially, one after the other. On many-core architectures it might be desirable to run different event loops concurrently to improve resource usage. ROOT::RDF::RunGraphs() allows running multiple RDataFrame event loops concurrently:

ROOT::EnableImplicitMT();
ROOT::RDataFrame df1("tree1", "f1.root");
ROOT::RDataFrame df2("tree2", "f2.root");
auto histo1 = df1.Histo1D("x");
auto histo2 = df2.Histo1D("y");
 
// just accessing result pointers, the event loops of separate RDataFrames run one after the other
histo1->Draw(); // runs first multi-thread event loop
histo2->Draw(); // runs second multi-thread event loop
 
// alternatively, with ROOT::RDF::RunGraphs, event loops for separate computation graphs can run concurrently
ROOT::RDF::RunGraphs({histo1, histo2});
histo1->Draw(); // results can then be used as usual

Performance considerations

To obtain the maximum performance out of RDataFrame, make sure to avoid just-in-time compiled versions of transformations and actions if at all possible. For instance, Filter("x > 0") requires just-in-time compilation of the corresponding C++ logic, while the equivalent Filter([](float x) { return x > 0.; }, {"x"}) does not. Similarly, Histo1D("x") requires just-in-time compilation after the type of x is retrieved from the dataset, while Histo1D<float>("x") does not; the latter spelling should be preferred for performance-critical applications.

Python applications cannot easily specify template parameters or pass C++ callables to RDataFrame. See Python interface for possible ways to speed up hot paths in this case.

Just-in-time compilation happens once, right before starting an event loop. To reduce the runtime cost of this step, make sure to book all operations for all RDataFrame computation graphs before the first event loop is triggered: just-in-time compilation will happen once for all code required to be generated up to that point, also across different computation graphs.

Also make sure not to count the just-in-time compilation time (which happens once before the event loop and does not depend on the size of the dataset) as part of the event loop runtime (which scales with the size of the dataset). RDataFrame has an experimental logging feature that simplifies measuring the time spent in just-in-time compilation and in the event loop (as well as providing some more interesting information). See Activating RDataFrame execution logs.

Memory usage

There are two reasons why RDataFrame may consume more memory than expected.

1. Histograms in multi-threaded mode

In multithreaded runs, each worker thread will create a local copy of histograms, which e.g. in case of many (possibly multi-dimensional) histograms with fine binning can result in significant memory consumption during the event loop. The thread-local copies of the results are destroyed when the final result is produced. Reducing the number of threads or using coarser binning will reduce the memory usage. For three-dimensional histograms, the number of clones can be reduced using ROOT::RDF::Experimental::ThreadsPerTH3().

#include "ROOT/RDFHelpers.hxx"
 
// Make four threads share a TH3 instance:
ROOT::RDF::Experimental::ThreadsPerTH3(4);
ROOT::RDataFrame rdf(...);

When TH3s are shared among threads, TH3D will either be filled under lock (slowing down the execution) or using atomics if C++20 is available. The latter is significantly faster. The best value for ThreadsPerTH3 depends on the computation graph that runs. Use lower numbers such as 4 for speed and higher memory consumption, and higher numbers such as 16 for slower execution and memory savings.

2. Just-in-time compilation

Secondly, just-in-time compilation of string expressions or non-templated actions (see the previous paragraph) causes Cling, ROOT's C++ interpreter, to allocate some memory for the generated code that is only released at the end of the application. This commonly results in memory usage creep in long-running applications that create many RDataFrames one after the other. Possible mitigations include creating and running each RDataFrame event loop in a sub-process, or booking all operations for all different RDataFrame computation graphs before the first event loop is triggered, so that the interpreter is invoked only once for all computation graphs:

// assuming df1 and df2 are separate computation graphs, do:
auto h1 = df1.Histo1D("x");
auto h2 = df2.Histo1D("y");
h1->Draw(); // we just-in-time compile everything needed by df1 and df2 here
h2->Draw("SAME");
 
// do not:
auto h1 = df1.Histo1D("x");
h1->Draw(); // we just-in-time compile here
auto h2 = df2.Histo1D("y");
h2->Draw("SAME"); // we just-in-time compile again here, as the second Histo1D call is new

More features

Here is a list of the most important features that have been omitted in the "Crash course" for brevity. You don't need to read all these to start using RDataFrame, but they are useful to save typing time and runtime.

Systematic variations

Starting from ROOT v6.26, RDataFrame provides a flexible syntax to define systematic variations. This is done in two steps: a) register variations for one or more existing columns using Vary() and b) extract variations of normal RDataFrame results using VariationsFor(). In between these steps, no other change to the analysis code is required: the presence of systematic variations for certain columns is automatically propagated through filters, defines and actions, and RDataFrame will take these dependencies into account when producing varied results. VariationsFor() is included in header ROOT/RDFHelpers.hxx. The compiled C++ programs must include this header explicitly, this is not required for ROOT macros.

An example usage of Vary() and VariationsFor() in C++:

auto nominal_hx =
   df.Vary("pt", "ROOT::RVecD{pt*0.9f, pt*1.1f}", {"down", "up"})
     .Filter("pt > pt_cut")
     .Define("x", someFunc, {"pt"})
     .Histo1D<float>("x");
 
// request the generation of varied results from the nominal_hx
ROOT::RDF::Experimental::RResultMap<TH1D> hx = ROOT::RDF::Experimental::VariationsFor(nominal_hx);
 
// the event loop runs here, upon first access to any of the results or varied results:
hx["nominal"].Draw(); // same effect as nominal_hx->Draw()
hx["pt:down"].Draw("SAME");
hx["pt:up"].Draw("SAME");

A shorter expression syntax is allowed for convenience (see the docs of the Vary overloads for more details):

auto nominal_hx =
   df.Vary("pt", "{pt*0.9f, pt*1.1f}", {"down", "up"})
// The rest is the same as above

A list of variation "tags" is passed as the last argument to Vary(). The tags give names to the varied values that are returned as elements of an RVec of the appropriate C++ type. The number of variation tags must correspond to the number of elements of this RVec (2 in the example above: the first element will correspond to the tag "down", the second to the tag "up"). The full variation name will be composed of the varied column name and the variation tags (e.g. "pt:down", "pt:up" in this example). Python usage looks similar.

Note how we use the "pt" column as usual in the Filter() and Define() calls and we simply use "x" as the value to fill the resulting histogram. To produce the varied results, RDataFrame will automatically execute the Filter and Define calls for each variation and fill the histogram with values and cuts that depend on the variation.

There is no limitation to the complexity of a Vary() expression. Just like for the Define() and Filter() calls, users are not limited to string expressions but they can also pass any valid C++ callable, including lambda functions and complex functors. The callable can be applied to zero or more existing columns and it will always receive their nominal value in input.

Note

- Currently, VariationsFor() and RResultMap are in the ROOT::RDF::Experimental namespace, to indicate that these interfaces can still evolve and improve based on user feedback. Please send feedback on https://github.com/root-project/root.

- The results of Display() cannot be varied (i.e. it is not possible to call VariationsFor()).

See the Vary() method for more information and this tutorial for an example usage of Vary and VariationsFor() in the analysis.

Snapshot with Variations

To combine Vary() and Snapshot(), the fIncludeVariations flag in RSnapshotOptions has to be set, as demonstrated in the following example:

ROOT::RDF::RSnapshotOptions options;
options.fIncludeVariations = true;
 
auto s = ROOT::RDataFrame(N)
   .Define("x", [](ULong64_t e) -> float { return 10.f * e; }, {"rdfentry_"})
   .Vary(
      "x", [](float x) { return ROOT::RVecF{x - 0.5f, x + 0.5f}; }, {"x"}, 2, "xVar")
   .Define("y", [](float x) -> double { return -1. * x; }, {"x"})
   .Filter(cuts, {"x", "y"})
   .Snapshot("t", filename, {"x", "y"}, options);

This snapshot action will create an output file with a dataset that includes the columns requested in the Snapshot() call and their corresponding variations:

Row	x	y	x__xVar_0	y__xVar_0	x__xVar_1	y__xVar_1	R_rdf_mask_t_0
0	0	-0	-0.5	0.5	0.5	-0.5	111
1	10	-10	9.5	-9.5	10.5	-10.5	111
2	20	-20	19.5	-19.5	20.5	-20.5	111
3	30	-30	29.5	-29.5	30.5	-30.5	111
4	40	-40	39.5	-39.5	40.5	-40.5	111
5	0	0	49.5	-49.5	0	0	010
6	* Not written *
7	0	0	0	0	70.5	-70.5	100

"x" and "y" are the nominal values, and their variations are snapshot as <ColumnName>__<VariationName>_<VariationTag>.

When Filters are employed, some variations might not pass the selection cuts (like the rows 5 and 7 in the table above). In that case, RDataFrame will snapshot the filtered columns in a memory-efficient way by writing zero into the memory of fundamental types, or write a default-constructed object in case of classes. If none of the filters pass like in row 6, the entire event is omitted from the snapshot.

To tell apart a genuine 0 (like x in row 0) from a case where nominal or variation didn't pass a selection, RDataFrame writes a bitmask for each event, see last column of the table above. Every bit indicates whether its associated columns are valid. The bitmask is implemented as a 64-bit std::bitset in memory, written to the output dataset as a std::uin64_t. For every 64 columns, a new bitmask column is added to the output dataset.

For each column that gets varied, the nominal and all variation columns are each assigned a bit to denote whether their entries are valid. A mapping of column names to the corresponding bitmask is placed in the same file as the output dataset, with a name that follows the pattern "R_rdf_column_to_bitmask_mapping_<NAME_OF_THE_DATASET>". It is of type std::unordered_map<std::string, std::pair<std::string, unsigned int>>, and maps a column name to the name of the bitmask column and the index of the relevant bit. For example, in the same file as the dataset "Events" there would be an object named R_rdf_column_to_bitmask_mapping_Events. This object for example would describe a connection such as:

muon_pt --> (R_rdf_mask_Events_0, 42)

which means that the validity of the entries in muon_pt is established by the bit 42 in the bitmask found in the column R_rdf_mask_Events_0.

When RDataFrame opens a file, it checks for the existence of this mapping between columns and bitmasks, and loads it automatically if found. As such, RDataFrame makes the treatment of the various bitmap maskings completely transparent to the user.

In case certain values are labeled invalid by the corresponding bit, this will result in reading a missing value. The semantics of such a scenario follow the rules described in the section on dealing with missing values and can be dealt with accordingly.

Note

Snapshot with variations is currently restricted to single-threaded TTree snapshots.

Varying multiple columns in lockstep

In the following Python snippet we use the Vary() signature that allows varying multiple columns simultaneously or "in lockstep":

df.Vary(["pt", "eta"],
        "RVec<RVecF>{{pt*0.9, pt*1.1}, {eta*0.9, eta*1.1}}",
        variationTags=["down", "up"],
        variationName="ptAndEta")

The expression returns an RVec of two RVecs: each inner vector contains the varied values for one column. The inner vectors follow the same ordering as the column names that are passed as the first argument. Besides the variation tags, in this case we also have to explicitly pass the variation name (here: "ptAndEta") as the default column name does not exist.

The above call will produce variations "ptAndEta:down" and "ptAndEta:up".

Combining multiple variations

Even if a result depends on multiple variations, only one variation is applied at a time, i.e. there will be no result produced by applying multiple systematic variations at the same time. For example, in the following example snippet, the RResultMap instance all_h will contain keys "nominal", "pt:down", "pt:up", "eta:0", "eta:1", but no "pt:up&&eta:0" or similar:

auto df = _df.Vary("pt",
                   "ROOT::RVecD{pt*0.9, pt*1.1}",
                   {"down", "up"})
             .Vary("eta",
                   [](float eta) { return RVecF{eta*0.9f, eta*1.1f}; },
                   {"eta"},
                   2);
 
auto nom_h = df.Histo2D(histoModel, "pt", "eta");
auto all_hs = VariationsFor(nom_h);
all_hs.GetKeys(); // returns {"nominal", "pt:down", "pt:up", "eta:0", "eta:1"}

Note how we passed the integer 2 instead of a list of variation tags to the second Vary() invocation: this is a shorthand that automatically generates tags 0 to N-1 (in this case 0 and 1).

RDataFrame objects as function arguments and return values

RDataFrame variables/nodes are relatively cheap to copy and it's possible to both pass them to (or move them into) functions and to return them from functions. However, in general each dataframe node will have a different C++ type, which includes all available compile-time information about what that node does. One way to cope with this complication is to use template functions and/or C++14 auto return types:

template <typename RDF>
auto ApplySomeFilters(RDF df)
{
   return df.Filter("x > 0").Filter([](int y) { return y < 0; }, {"y"});
}

A possibly simpler, C++11-compatible alternative is to take advantage of the fact that any dataframe node can be converted (implicitly or via an explicit cast) to the common type ROOT::RDF::RNode:

// a function that conditionally adds a Range to an RDataFrame node.
RNode MaybeAddRange(RNode df, bool mustAddRange)
{
   return mustAddRange ? df.Range(1) : df;
}
// use as :
ROOT::RDataFrame df(10);
auto maybeRangedDF = MaybeAddRange(df, true);

The conversion to ROOT::RDF::RNode is cheap, but it will introduce an extra virtual call during the RDataFrame event loop (in most cases, the resulting performance impact should be negligible). Python users can perform the conversion with the helper function ROOT.RDF.AsRNode.

Storing RDataFrame objects in collections

ROOT::RDF::RNode also makes it simple to store RDataFrame nodes in collections, e.g. a std::vector<RNode> or a std::map<std::string, RNode>:

std::vector<ROOT::RDF::RNode> dfs;
dfs.emplace_back(ROOT::RDataFrame(10));
dfs.emplace_back(dfs[0].Define("x", "42.f"));

Executing callbacks every N events

It's possible to schedule execution of arbitrary functions (callbacks) during the event loop. Callbacks can be used e.g. to inspect partial results of the analysis while the event loop is running, drawing a partially-filled histogram every time a certain number of new entries is processed, or displaying a progress bar while the event loop runs.

For example one can draw an up-to-date version of a result histogram every 100 entries like this:

auto h = df.Histo1D("x");
TCanvas c("c","x hist");
h.OnPartialResult(100, [&c](TH1D &h_) { c.cd(); h_.Draw(); c.Update(); });
// event loop runs here, this final `Draw` is executed after the event loop is finished
h->Draw();

Callbacks are registered to a ROOT::RDF::RResultPtr and must be callables that takes a reference to the result type as argument and return nothing. RDataFrame will invoke registered callbacks passing partial action results as arguments to them (e.g. a histogram filled with a part of the selected events).

Read more on ROOT::RDF::RResultPtr::OnPartialResult() and ROOT::RDF::RResultPtr::OnPartialResultSlot().

Default column lists

When constructing an RDataFrame object, it is possible to specify a default column list for your analysis, in the usual form of a list of strings representing branch/column names. The default column list will be used as a fallback whenever a list specific to the transformation/action is not present. RDataFrame will take as many of these columns as needed, ignoring trailing extra names if present.

// use "b1" and "b2" as default columns
ROOT::RDataFrame d1("myTree", "file.root", {"b1","b2"});
auto h = d1.Filter([](int b1, int b2) { return b1 > b2; }) // will act on "b1" and "b2"
           .Histo1D(); // will act on "b1"
 
// just one default column this time
ROOT::RDataFrame d2("myTree", "file.root", {"b1"});
auto d2f = d2.Filter([](double b2) { return b2 > 0; }, {"b2"}) // we can still specify non-default column lists
auto min = d2f.Min(); // returns the minimum value of "b1" for the filtered entries
auto vals = d2f.Take<double>(); // return the values for all entries passing the selection as a vector

Special helper columns: rdfentry_ and rdfslot_

Every instance of RDataFrame is created with two special columns called rdfentry_ and rdfslot_. The rdfentry_ column is of type ULong64_t and it holds the current entry number while rdfslot_ is an unsigned int holding the index of the current data processing slot. For backwards compatibility reasons, the names tdfentry_ and tdfslot_ are also accepted. These columns are ignored by operations such as Cache or Snapshot.

Warning

Note that in multi-thread event loops the values of rdfentry_ do not correspond to what would be the entry numbers of a TChain constructed over the same set of ROOT files, as the entries are processed in an unspecified order.

Just-in-time compilation: column type inference and explicit declaration of column types

C++ is a statically typed language: all types must be known at compile-time. This includes the types of the TTree branches we want to work on. For filters, defined columns and some of the actions, column types are deduced from the signature of the relevant filter function/temporary column expression/action function:

// here b1 is deduced to be `int` and b2 to be `double`
df.Filter([](int x, double y) { return x > 0 && y < 0.; }, {"b1", "b2"});

If we specify an incorrect type for one of the columns, an exception with an informative message will be thrown at runtime, when the column value is actually read from the dataset: RDataFrame detects type mismatches. The same would happen if we swapped the order of "b1" and "b2" in the column list passed to Filter().

Certain actions, on the other hand, do not take a function as argument (e.g. Histo1D()), so we cannot deduce the type of the column at compile-time. In this case RDataFrame infers the type of the column from the TTree itself. This is why we never needed to specify the column types for all actions in the above snippets.

When the column type is not a common one such as int, double, char or float it is nonetheless good practice to specify it as a template parameter to the action itself, like this:

df.Histo1D("b1"); // OK, the type of "b1" is deduced at runtime
df.Min<MyNumber_t>("myObject"); // OK, "myObject" is deduced to be of type `MyNumber_t`

Deducing types at runtime requires the just-in-time compilation of the relevant actions, which has a small runtime overhead, so specifying the type of the columns as template parameters to the action is good practice when performance is a goal.

When strings are passed as expressions to Filter() or Define(), fundamental types are passed as constants. This avoids certaincommon mistakes such as typing x = 0 rather than x == 0:

// this throws an error (note the typo)
df.Define("x", "0").Filter("x = 0");

User-defined custom actions

RDataFrame strives to offer a comprehensive set of standard actions that can be performed on each event. At the same time, it allows users to inject their own action code to perform arbitrarily complex data reductions.

Implementing custom actions with Book()

Through the Book() method, users can implement a custom action and have access to the same features that built-in RDataFrame actions have, e.g. hooks to events related to the start, end and execution of the event loop, or the possibility to return a lazy RResultPtr to an arbitrary type of result:

#include <ROOT/RDataFrame.hxx>
#include <memory>
 
class MyCounter : public ROOT::Detail::RDF::RActionImpl<MyCounter> {
   std::shared_ptr<int> fFinalResult = std::make_shared<int>(0);
   std::vector<int> fPerThreadResults;
 
public:
   // We use a public type alias to advertise the type of the result of this action
   using Result_t = int;
 
   MyCounter(unsigned int nSlots) : fPerThreadResults(nSlots) {}
 
   // Called before the event loop to retrieve the address of the result that will be filled/generated.
   std::shared_ptr<int> GetResultPtr() const { return fFinalResult; }
 
   // Called at the beginning of the event loop.
   void Initialize() {}
 
   // Called at the beginning of each processing task.
   void InitTask(TTreeReader *, int) {}
 
   /// Called at every entry.
   void Exec(unsigned int slot)
   {
      fPerThreadResults[slot]++;
   }
 
   // Called at the end of the event loop.
   void Finalize()
   {
      *fFinalResult = std::accumulate(fPerThreadResults.begin(), fPerThreadResults.end(), 0);
   }
 
   // Called by RDataFrame to retrieve the name of this action.
   std::string GetActionName() const { return "MyCounter"; }
};
 
int main() {
   ROOT::RDataFrame df(10);
   ROOT::RDF::RResultPtr<int> resultPtr = df.Book<>(MyCounter{df.GetNSlots()}, {});
   // The GetValue call triggers the event loop
   std::cout << "Number of processed entries: " <<  resultPtr.GetValue() << std::endl;
}

See the Book() method for more information and this tutorial for a more complete example.

Injecting arbitrary code in the event loop with Foreach() and ForeachSlot()

Foreach() takes a callable (lambda expression, free function, functor...) and a list of columns and executes the callable on the values of those columns for each event that passes all upstream selections. It can be used to perform actions that are not already available in the interface. For example, the following snippet evaluates the root mean square of column "x":

// Single-thread evaluation of RMS of column "x" using Foreach
double sumSq = 0.;
unsigned int n = 0;
df.Foreach([&sumSq, &n](double x) { ++n; sumSq += x*x; }, {"x"});
std::cout << "rms of x: " << std::sqrt(sumSq / n) << std::endl;

In multi-thread runs, users are responsible for the thread-safety of the expression passed to Foreach(): thread will execute the expression concurrently. The code above would need to employ some resource protection mechanism to ensure non-concurrent writing of rms; but this is probably too much head-scratch for such a simple operation.

ForeachSlot() can help in this situation. It is an alternative version of Foreach() for which the function takes an additional "processing slot" parameter besides the columns it should be applied to. RDataFrame guarantees that ForeachSlot() will invoke the user expression with different slot parameters for different concurrent executions (see Special helper columns: rdfentry_ and rdfslot_ for more information on the slot parameter). We can take advantage of ForeachSlot() to evaluate a thread-safe root mean square of column "x":

// Thread-safe evaluation of RMS of column "x" using ForeachSlot
ROOT::EnableImplicitMT();
const unsigned int nSlots = df.GetNSlots();
std::vector<double> sumSqs(nSlots, 0.);
std::vector<unsigned int> ns(nSlots, 0);
 
df.ForeachSlot([&sumSqs, &ns](unsigned int slot, double x) { sumSqs[slot] += x*x; ns[slot] += 1; }, {"x"});
double sumSq = std::accumulate(sumSqs.begin(), sumSqs.end(), 0.); // sum all squares
unsigned int n = std::accumulate(ns.begin(), ns.end(), 0); // sum all counts
std::cout << "rms of x: " << std::sqrt(sumSq / n) << std::endl;

Notice how we created one double variable for each processing slot and later merged their results via std::accumulate.

Dataset joins with friend trees

Vertically concatenating multiple trees that have the same columns (creating a logical dataset with the same columns and more rows) is trivial in RDataFrame: just pass the tree name and a list of file names to RDataFrame's constructor, or create a TChain out of the desired trees and pass that to RDataFrame.

Horizontal concatenations of trees or chains (creating a logical dataset with the same number of rows and the union of the columns of multiple trees) leverages TTree's "friend" mechanism.

Simple joins of trees that do not have the same number of rows are also possible with indexed friend trees (see below).

To use friend trees in RDataFrame, set up trees with the appropriate relationships and then instantiate an RDataFrame with the main tree:

TTree main([...]);
TTree friend([...]);
main.AddFriend(&friend, "myFriend");
 
RDataFrame df(main);
auto df2 = df.Filter("myFriend.MyCol == 42");

The same applies for TChains. Columns coming from the friend trees can be referred to by their full name, like in the example above, or the friend tree name can be omitted in case the column name is not ambiguous (e.g. "MyCol" could be used instead of "myFriend.MyCol" in the example above if there is no column "MyCol" in the main tree).

Note

A common source of confusion is that trees that are written out from a multi-thread Snapshot() call will have their entries (block-wise) shuffled with respect to the original tree. Such trees cannot be used as friends of the original one: rows will be mismatched.

Indexed friend trees provide a way to perform simple joins of multiple trees over a common column. When a certain entry in the main tree (or chain) is loaded, the friend trees (or chains) will then load an entry where the "index" columns have a value identical to the one in the main one. For example, in Python:

main_tree = ...
aux_tree = ...
 
# If a friend tree has an index on `commonColumn`, when the main tree loads
# a given row, it also loads the row of the friend tree that has the same
# value of `commonColumn`
aux_tree.BuildIndex("commonColumn")
 
mainTree.AddFriend(aux_tree)
 
df = ROOT.RDataFrame(mainTree)

RDataFrame supports indexed friend TTrees from ROOT v6.24 in single-thread mode and from v6.28/02 in multi-thread mode.

Reading data formats other than ROOT trees

RDataFrame can be interfaced with RDataSources. The ROOT::RDF::RDataSource interface defines an API that RDataFrame can use to read arbitrary columnar data formats.

RDataFrame calls into concrete RDataSource implementations to retrieve information about the data, retrieve (thread-local) readers or "cursors" for selected columns and to advance the readers to the desired data entry. Some predefined RDataSources are natively provided by ROOT such as the ROOT::RDF::RCsvDS which allows to read comma separated files:

auto tdf = ROOT::RDF::FromCSV("MuRun2010B.csv");
auto filteredEvents =
   tdf.Filter("Q1 * Q2 == -1")
      .Define("m", "sqrt(pow(E1 + E2, 2) - (pow(px1 + px2, 2) + pow(py1 + py2, 2) + pow(pz1 + pz2, 2)))");
auto h = filteredEvents.Histo1D("m");
h->Draw();

See also FromNumpy (Python-only), FromRNTuple(), FromArrow(), FromSqlite().

Computation graphs (storing and reusing sets of transformations)

As we saw, transformed dataframes can be stored as variables and reused multiple times to create modified versions of the dataset. This implicitly defines a computation graph in which several paths of filtering/creation of columns are executed simultaneously, and finally aggregated results are produced.

RDataFrame detects when several actions use the same filter or the same defined column, and only evaluates each filter or defined column once per event, regardless of how many times that result is used down the computation graph. Objects read from each column are built once and never copied, for maximum efficiency. When "upstream" filters are not passed, subsequent filters, temporary column expressions and actions are not evaluated, so it might be advisable to put the strictest filters first in the graph.

Visualizing the computation graph

It is possible to print the computation graph from any node to obtain a DOT (graphviz) representation either on the standard output or in a file.

Invoking the function ROOT::RDF::SaveGraph() on any node that is not the head node, the computation graph of the branch the node belongs to is printed. By using the head node, the entire computation graph is printed.

Following there is an example of usage:

// First, a sample computational graph is built
ROOT::RDataFrame df("tree", "f.root");
 
auto df2 = df.Define("x", []() { return 1; })
             .Filter("col0 % 1 == col0")
             .Filter([](int b1) { return b1 <2; }, {"cut1"})
             .Define("y", []() { return 1; });
 
auto count =  df2.Count();
 
// Prints the graph to the rd1.dot file in the current directory
ROOT::RDF::SaveGraph(df, "./mydot.dot");
// Prints the graph to standard output
ROOT::RDF::SaveGraph(df);

The generated graph can be rendered using one of the graphviz filters, e.g. dot. For instance, the image below can be generated with the following command:

$ dot -Tpng computation_graph.dot -ocomputation_graph.png

Activating RDataFrame execution logs

RDataFrame has experimental support for verbose logging of the event loop runtimes and other interesting related information. It is activated as follows:

#include <ROOT/RLogger.hxx>
 
// this increases RDF's verbosity level as long as the `verbosity` variable is in scope
auto verbosity = ROOT::RLogScopedVerbosity(ROOT::Detail::RDF::RDFLogChannel(), ROOT::ELogLevel::kInfo);

or in Python:

import ROOT
 
verbosity = ROOT.RLogScopedVerbosity(ROOT.Detail.RDF.RDFLogChannel(), ROOT.ELogLevel.kInfo)

More information (e.g. start and end of each multi-thread task) is printed using ELogLevel.kDebug and even more (e.g. a full dump of the generated code that RDataFrame just-in-time-compiles) using ELogLevel.kDebug+10.

Creating an RDataFrame from a dataset specification file

RDataFrame can be created using a dataset specification JSON file:

import ROOT
 
df = ROOT.RDF.Experimental.FromSpec("spec.json")

The input dataset specification JSON file needs to be provided by the user and it describes all necessary samples and their associated metadata information. The main required key is the "samples" (at least one sample is needed) and the required sub-keys for each sample are "trees" and "files". Additionally, one can specify a metadata dictionary for each sample in the "metadata" key.

A simple example for the formatting of the specification in the JSON file is the following:

{
   "samples": {
      "sampleA": {
         "trees": ["tree1", "tree2"],
         "files": ["file1.root", "file2.root"],
         "metadata": {
            "lumi": 10000.0,
            "xsec": 1.0,
            "sample_category" = "data"
            }
      },
      "sampleB": {
         "trees": ["tree3", "tree4"],
         "files": ["file3.root", "file4.root"],
         "metadata": {
            "lumi": 0.5,
            "xsec": 1.5,
            "sample_category" = "MC_background"
            }
      }
   }
}

The metadata information from the specification file can be then accessed using the DefinePerSample function. For example, to access luminosity information (stored as a double):

df.DefinePerSample("lumi", 'rdfsampleinfo_.GetD("lumi")')

or sample_category information (stored as a string):

df.DefinePerSample("sample_category", 'rdfsampleinfo_.GetS("sample_category")')

or directly the filename:

df.DefinePerSample("name", "rdfsampleinfo_.GetSampleName()")

An example implementation of the "FromSpec" method is available in tutorial: df106_HiggstoFourLeptons.py, which also provides a corresponding exemplary JSON file for the dataset specification.

Adding a progress bar

A progress bar showing the processed event statistics can be added to any RDataFrame program. The event statistics include elapsed time, currently processed file, currently processed events, the rate of event processing and an estimated remaining time (per file being processed). It is recorded and printed in the terminal every m events and every n seconds (by default m = 1000 and n = 1). The ProgressBar can be also added when the multithread (MT) mode is enabled.

ProgressBar is added after creating the dataframe object (df):

ROOT::RDataFrame df("tree", "file.root");
ROOT::RDF::Experimental::AddProgressBar(df);

Alternatively, RDataFrame can be cast to an RNode first, giving the user more flexibility For example, it can be called at any computational node, such as Filter or Define, not only the head node, with no change to the ProgressBar function itself (please see the Python interface section for appropriate usage in Python):

ROOT::RDataFrame df("tree", "file.root");
auto df_1 = ROOT::RDF::RNode(df.Filter("x>1"));
ROOT::RDF::Experimental::AddProgressBar(df_1);

Examples of implemented progress bars can be seen by running Higgs to Four Lepton tutorial and Dimuon tutorial.

Working with missing values in the dataset

In certain situations a dataset might be missing one or more values at one or more of its entries. For example:

• If the dataset is composed of multiple files and one or more files is missing one or more columns required by the analysis.
• When joining different datasets horizontally according to some index value (e.g. the event number), if the index does not find a match in one or more other datasets for a certain entry.
• If, for a certain event, a column is invalid because it results from a Snapshot with systematic variations, and that variation didn't pass its filters. For more details, see snapshot-with-variations.

For example, suppose that column "y" does not have a value for entry 42:

+-------+---+---+
| Entry | x | y |
+-------+---+---+
| 42    | 1 |   |
+-------+---+---+

If the RDataFrame application reads that column, for example if a Take() action was requested, the default behaviour is to throw an exception indicating that that column is missing an entry.

The following paragraphs discuss the functionalities provided by RDataFrame to work with missing values in the dataset.

FilterAvailable and FilterMissing

FilterAvailable and FilterMissing are specialized RDataFrame Filter operations. They take as input argument the name of a column of the dataset to watch for missing values. Like Filter, they will either keep or discard an entire entry based on whether a condition returns true or false. Specifically:

• FilterAvailable: the condition is whether the value of the column is present. If so, the entry is kept. Otherwise if the value is missing the entry is discarded.
• FilterMissing: the condition is whether the value of the column is missing. If so, the entry is kept. Otherwise if the value is present the entry is discarded.

df = ROOT.RDataFrame(dataset)
 
# Anytime an entry from "col" is missing, the entire entry will be filtered out
df_available = df.FilterAvailable("col")
df_available = df_available.Define("twice", "col * 2")
 
# Conversely, if we want to select the entries for which the column has missing
# values, we do the following
df_missingcol = df.FilterMissing("col")
# Following operations in the same branch of the computation graph clearly
# cannot access that same column, since there would be no value to read
df_missingcol = df_missingcol.Define("observable", "othercolumn * 2")

ROOT::RDataFrame df{dataset};
 
// Anytime an entry from "col" is missing, the entire entry will be filtered out
auto df_available = df.FilterAvailable("col");
auto df_twicecol = df_available.Define("twice", "col * 2");
 
// Conversely, if we want to select the entries for which the column has missing
// values, we do the following
auto df_missingcol = df.FilterMissing("col");
// Following operations in the same branch of the computation graph clearly
// cannot access that same column, since there would be no value to read
auto df_observable = df_missingcol.Define("observable", "othercolumn * 2");

DefaultValueFor

DefaultValueFor creates a node of the computation graph which just forwards the values of the columns necessary for other downstream nodes, when they are available. In case a value of the input column passed to this function is not available, the node will provide the default value passed to this function call instead. Example:

df = ROOT.RDataFrame(dataset)
# Anytime an entry from "col" is missing, the value will be the default one
default_value = ... # Some sensible default value here
df = df.DefaultValueFor("col", default_value)
df = df.Define("twice", "col * 2")

ROOT::RDataFrame df{dataset};
// Anytime an entry from "col" is missing, the value will be the default one
constexpr auto default_value = ... // Some sensible default value here
auto df_default = df.DefaultValueFor("col", default_value);
auto df_col = df_default.Define("twice", "col * 2");

Mixing different strategies to work with missing values in the same RDataFrame

All the operations presented above only act on the particular branch of the computation graph where they are called, so that different results can be obtained by mixing and matching the filtering or providing a default value strategies:

df = ROOT.RDataFrame(dataset)
# Anytime an entry from "col" is missing, the value will be the default one
default_value = ... # Some sensible default value here
df_default = df.DefaultValueFor("col", default_value).Define("twice", "col * 2")
df_filtered = df.FilterAvailable("col").Define("twice", "col * 2")
 
# Same number of total entries as the input dataset, with defaulted values
df_default.Display(["twice"]).Print()
# Only keep the entries where "col" has values
df_filtered.Display(["twice"]).Print()

ROOT::RDataFrame df{dataset};
 
// Anytime an entry from "col" is missing, the value will be the default one
constexpr auto default_value = ... // Some sensible default value here
auto df_default = df.DefaultValueFor("col", default_value).Define("twice", "col * 2");
auto df_filtered = df.FilterAvailable("col").Define("twice", "col * 2");
 
// Same number of total entries as the input dataset, with defaulted values
df_default.Display({"twice"})->Print();
// Only keep the entries where "col" has values
df_filtered.Display({"twice"})->Print();

Further considerations

Note that working with missing values is currently supported with a TTree-based data source. Support of this functionality for other data sources may come in the future.

Dealing with NaN or Inf values in the dataset

RDataFrame does not treat NaNs or infinities beyond what the floating-point standards require, i.e. they will propagate to the final result. Non-finite numbers can be suppressed using Filter(), e.g.:

df.Filter("std::isfinite(x)").Mean("x")

Translating TTree commands to RDataFrame

TTree::Draw	ROOT::RDataFrame
// Get the tree and Draw a histogram of x for selected y values auto *tree = file->Get<TTree>("myTree"); tree->Draw("x", "y > 2");	ROOT::RDataFrame df("myTree", file); df.Filter("y > 2").Histo1D("x")->Draw();
// Draw a histogram of "jet_eta" with the desired weight tree->Draw("jet_eta", "weight*(event == 1)");	df.Filter("event == 1").Histo1D("jet_eta", "weight")->Draw();
// Draw a histogram filled with values resulting from calling a method of the class of the `event` branch in the TTree. tree->Draw("event.GetNtrack()");	df.Define("NTrack","event.GetNtrack()").Histo1D("NTrack")->Draw();
// Draw only every 10th event tree->Draw("fNtrack","fEvtHdr.fEvtNum%10 == 0");	// Use the Filter operation together with the special RDF column: `rdfentry_` df.Filter("rdfentry_ % 10 == 0").Histo1D("fNtrack")->Draw();
// object selection: for each event, fill histogram with array of selected pts tree->Draw('Muon_pt', 'Muon_pt > 100');	// with RDF, arrays are read as ROOT::VecOps::RVec objects df.Define("good_pt", "Muon_pt[Muon_pt > 100]").Histo1D("good_pt")->Draw();

// Draw the histogram and fill hnew with it
tree->Draw("sqrt(x)>>hnew","y>0");
 
// Retrieve hnew from the current directory
auto hnew = gDirectory->Get<TH1F>("hnew");

// We pass histogram constructor arguments to the Histo1D operation, to easily give the histogram a name
auto hist = df.Define("sqrt_x", "sqrt(x)").Filter("y>0").Histo1D({"hnew","hnew", 10, 0, 10}, "sqrt_x");

// Draw a 1D Profile histogram instead of TH2F
tree->Draw("y:x","","prof");
 
// Draw a 2D Profile histogram instead of TH3F
tree->Draw("z:y:x","","prof");

// Draw a 1D Profile histogram
df.Profile1D("x", "y")->Draw();
 
// Draw a 2D Profile histogram
df.Profile2D("x", "y", "z")->Draw();

// This command draws 2 entries starting with entry 5
tree->Draw("x", "","", 2, 5);

// Range function with arguments begin, end
df.Range(5,7).Histo1D("x")->Draw();

// Draw the X() component of the
// ROOT::Math::DisplacementVector3D in vec_list
tree->Draw("vec_list.X()");

df.Define("x", "ROOT::RVecD out; for(const auto &el: vec_list) out.push_back(el.X()); return out;").Histo1D("x")->Draw();

// Gather all values from a branch holding a collection per event, `pt`,
// and fill a histogram so that we can count the total number of values across all events
tree->Draw("pt>>histo");
auto histo = gDirectory->Get<TH1D>("histo");
histo->GetEntries();

df.Histo1D("pt")->GetEntries();

TTree::Scan()

ROOT::RDataFrame

// Print a table of the first 10 entries for all variables in the Tree
// if the first entry in the Muon_pt collection is > 10.
tree->Scan("*", "Muon_pt[0] > 10.", "", 10);

// Selecting columns using a regular expression
df.Filter("Muon_pt[0] > 10.").Display(".*", 10)->Print();

// For 10 events, print Muon_pt and Muon_eta, starting at entry 100
tree->Scan("Muon_pt:Muon_eta", "", "", 10, 100);

// Selecting columns using a collection of names
df.Range(100, 0).Display({"Muon_pt", "Muon_eta"}, 10)->Print();

Definition at line 50 of file RDataFrame.hxx.

Public Types
using	ColumnNames_t = ROOT::RDF::ColumnNames_t

Public Member Functions
	RDataFrame (ROOT::RDF::Experimental::RDatasetSpec spec)
	Build dataframe from an RDatasetSpec object.
	RDataFrame (std::string_view treeName, ::TDirectory *dirPtr, const ColumnNames_t &defaultColumns={})
	RDataFrame (std::string_view treename, const std::vector< std::string > &filenames, const ColumnNames_t &defaultColumns={})
	Build the dataframe.
	RDataFrame (std::string_view treename, std::initializer_list< std::string > filenames, const ColumnNames_t &defaultColumns={})
	RDataFrame (std::string_view treeName, std::string_view filenameglob, const ColumnNames_t &defaultColumns={})
	Build the dataframe.
	RDataFrame (std::unique_ptr< ROOT::RDF::RDataSource >, const ColumnNames_t &defaultColumns={})
	Build dataframe associated to data source.
	RDataFrame (TTree &tree, const ColumnNames_t &defaultColumns={})
	Build the dataframe.
	RDataFrame (ULong64_t numEntries)
	Build a dataframe that generates numEntries entries.
RDFDescription	Describe ()
	Return information about the dataframe.
ColumnNames_t	GetColumnNames ()
	Returns the names of the available columns.
std::string	GetColumnType (std::string_view column)
	Return the type of a given column as a string.
ColumnNames_t	GetDatasetTopLevelFieldNames ()
	Retrieve the names of top-level field names.
ColumnNames_t	GetDefinedColumnNames ()
	Returns the names of the defined columns.
unsigned int	GetNFiles ()
unsigned int	GetNRuns () const
	Gets the number of event loops run.
unsigned int	GetNSlots () const
	Gets the number of data processing slots.
RVariationsDescription	GetVariations () const
	Return a descriptor for the systematic variations registered in this branch of the computation graph.
bool	HasColumn (std::string_view columnName)
	Checks if a column is present in the dataset.
Transformation
	operator RNode () const
	Cast any RDataFrame node to a common type ROOT::RDF::RNode.
RInterface< RDFDetail::RFilter< F, RDFDetail::RLoopManager > >	Filter (F f, const ColumnNames_t &columns={}, std::string_view name="")
	Append a filter to the call graph.
RInterface< RDFDetail::RFilterWithMissingValues< RDFDetail::RLoopManager > >	FilterAvailable (std::string_view column)
	Discard entries with missing values.
RInterface< RDFDetail::RFilterWithMissingValues< RDFDetail::RLoopManager > >	FilterMissing (std::string_view column)
	Keep only the entries that have missing values.
RInterface< RDFDetail::RLoopManager >	Define (std::string_view name, F expression, const ColumnNames_t &columns={})
	Define a new column.
RInterface< RDFDetail::RLoopManager >	DefineSlot (std::string_view name, F expression, const ColumnNames_t &columns={})
	Define a new column with a value dependent on the processing slot.
RInterface< RDFDetail::RLoopManager >	DefineSlotEntry (std::string_view name, F expression, const ColumnNames_t &columns={})
	Define a new column with a value dependent on the processing slot and the current entry.
RInterface< RDFDetail::RLoopManager >	Redefine (std::string_view name, F expression, const ColumnNames_t &columns={})
	Overwrite the value and/or type of an existing column.
RInterface< RDFDetail::RLoopManager >	RedefineSlot (std::string_view name, F expression, const ColumnNames_t &columns={})
	Overwrite the value and/or type of an existing column.
RInterface< RDFDetail::RLoopManager >	RedefineSlotEntry (std::string_view name, F expression, const ColumnNames_t &columns={})
	Overwrite the value and/or type of an existing column.
RInterface< RDFDetail::RLoopManager >	DefaultValueFor (std::string_view column, const T &defaultValue)
	In case the value in the given column is missing, provide a default value.
RInterface< RDFDetail::RLoopManager >	DefinePerSample (std::string_view name, F expression)
	Define a new column that is updated when the input sample changes.
RInterface< RDFDetail::RLoopManager >	Vary (std::string_view colName, F &&expression, const ColumnNames_t &inputColumns, const std::vector< std::string > &variationTags, std::string_view variationName="")
	Register systematic variations for a single existing column using custom variation tags.
RInterface< RDFDetail::RLoopManager >	Alias (std::string_view alias, std::string_view columnName)
	Allow to refer to a column with a different name.
RResultPtr< RInterface< RLoopManager > >	Snapshot (std::string_view treename, std::string_view filename, const ColumnNames_t &columnList, const RSnapshotOptions &options=RSnapshotOptions())
RInterface< RLoopManager >	Cache (const ColumnNames_t &columnList)
	Save selected columns in memory.
RInterface< RDFDetail::RRange< RDFDetail::RLoopManager > >	Range (unsigned int begin, unsigned int end, unsigned int stride=1)
	Creates a node that filters entries based on range: [begin, end).
void	Foreach (F f, const ColumnNames_t &columns={})
	Execute a user-defined function on each entry (instant action).
void	ForeachSlot (F f, const ColumnNames_t &columns={})
	Execute a user-defined function requiring a processing slot index on each entry (instant action).
RResultPtr< T >	Reduce (F f, std::string_view columnName="")
	Execute a user-defined reduce operation on the values of a column.
RResultPtr< ULong64_t >	Count ()
	Return the number of entries processed (lazy action).
RResultPtr< COLL >	Take (std::string_view column="")
	Return a collection of values of a column (lazy action, returns a std::vector by default).
RResultPtr<::TH1D >	Histo1D (const TH1DModel &model={"", "", 128u, 0., 0.}, std::string_view vName="")
	Fill and return a one-dimensional histogram with the values of a column (lazy action).
RResultPtr<::TH2D >	Histo2D (const TH2DModel &model, std::string_view v1Name="", std::string_view v2Name="")
	Fill and return a two-dimensional histogram (lazy action).
RResultPtr<::TH3D >	Histo3D (const TH3DModel &model, std::string_view v1Name="", std::string_view v2Name="", std::string_view v3Name="")
	Fill and return a three-dimensional histogram (lazy action).
RResultPtr<::THnD >	HistoND (const THnDModel &model, const ColumnNames_t &columnList, std::string_view wName="")
	Fill and return an N-dimensional histogram (lazy action).
RResultPtr<::THnSparseD >	HistoNSparseD (const THnSparseDModel &model, const ColumnNames_t &columnList, std::string_view wName="")
	Fill and return a sparse N-dimensional histogram (lazy action).
RResultPtr< ROOT::Experimental::RHist< BinContentType > >	Hist (std::uint64_t nNormalBins, std::pair< double, double > interval, std::string_view vName)
	Fill and return a one-dimensional RHist (lazy action).
RResultPtr<::TGraph >	Graph (std::string_view x="", std::string_view y="")
	Fill and return a TGraph object (lazy action).
RResultPtr<::TGraphAsymmErrors >	GraphAsymmErrors (std::string_view x="", std::string_view y="", std::string_view exl="", std::string_view exh="", std::string_view eyl="", std::string_view eyh="")
	Fill and return a TGraphAsymmErrors object (lazy action).
RResultPtr<::TProfile >	Profile1D (const TProfile1DModel &model, std::string_view v1Name="", std::string_view v2Name="")
	Fill and return a one-dimensional profile (lazy action).
RResultPtr<::TProfile2D >	Profile2D (const TProfile2DModel &model, std::string_view v1Name="", std::string_view v2Name="", std::string_view v3Name="")
	Fill and return a two-dimensional profile (lazy action).
RResultPtr< std::decay_t< T > >	Fill (T &&model, const ColumnNames_t &columnList)
	Return an object of type T on which T::Fill will be called once per event (lazy action).
RResultPtr< TStatistic >	Stats (std::string_view value="")
	Return a TStatistic object, filled once per event (lazy action).
RResultPtr< RDFDetail::MinReturnType_t< T > >	Min (std::string_view columnName="")
	Return the minimum of processed column values (lazy action).
RResultPtr< RDFDetail::MaxReturnType_t< T > >	Max (std::string_view columnName="")
	Return the maximum of processed column values (lazy action).
RResultPtr< double >	Mean (std::string_view columnName="")
	Return the mean of processed column values (lazy action).
RResultPtr< double >	StdDev (std::string_view columnName="")
	Return the unbiased standard deviation of processed column values (lazy action).
RResultPtr< RDFDetail::SumReturnType_t< T > >	Sum (std::string_view columnName="", const RDFDetail::SumReturnType_t< T > &initValue=RDFDetail::SumReturnType_t< T >{})
	Return the sum of processed column values (lazy action).
RResultPtr< RCutFlowReport >	Report ()
	Gather filtering statistics.
std::vector< std::string >	GetFilterNames ()
	Returns the names of the filters created.
RResultPtr< U >	Aggregate (AccFun aggregator, MergeFun merger, std::string_view columnName, const U &aggIdentity)
	Execute a user-defined accumulation operation on the processed column values in each processing slot.
RResultPtr< typename std::decay_t< Helper >::Result_t >	Book (Helper &&helper, const ColumnNames_t &columns={})
	Book execution of a custom action using a user-defined helper object.
RResultPtr< RDisplay >	Display (const ColumnNames_t &columnList, size_t nRows=5, size_t nMaxCollectionElements=10)
	Provides a representation of the columns in the dataset.

Protected Member Functions
void	AddDefaultColumns ()
template<typename... ColumnTypes>
void	CheckAndFillDSColumns (ColumnNames_t validCols, TTraits::TypeList< ColumnTypes... > typeList)
void	CheckAndFillDSColumns (const std::vector< std::string > &colNames, const std::vector< const std::type_info * > &colTypeIDs)
void	CheckIMTDisabled (std::string_view callerName)
template<typename ActionTag, typename... ColTypes, typename ActionResultType, typename RDFNode, typename HelperArgType = ActionResultType, std::enable_if_t< RDFInternal::RNeedJitting< ColTypes... >::value, int > = 0>
RResultPtr< ActionResultType >	CreateAction (const ColumnNames_t &columns, const std::shared_ptr< ActionResultType > &r, const std::shared_ptr< HelperArgType > &helperArg, const std::shared_ptr< RDFNode > &proxiedPtr, const int nColumns=-1, const bool vector2RVec=true)
	Create RAction object, return RResultPtr for the action Overload for the case in which one or more column types were not specified (RTTI + jitting).
template<typename ActionTag, typename... ColTypes, typename ActionResultType, typename RDFNode, typename HelperArgType = ActionResultType, std::enable_if_t<!RDFInternal::RNeedJitting< ColTypes... >::value, int > = 0>
RResultPtr< ActionResultType >	CreateAction (const ColumnNames_t &columns, const std::shared_ptr< ActionResultType > &r, const std::shared_ptr< HelperArgType > &helperArg, const std::shared_ptr< RDFNode > &proxiedPtr, const int=-1)
	Create RAction object, return RResultPtr for the action Overload for the case in which all column types were specified (no jitting).
std::string	DescribeDataset () const
ColumnNames_t	GetColumnTypeNamesList (const ColumnNames_t &columnList)
RDataSource *	GetDataSource () const
RDFDetail::RLoopManager *	GetLoopManager () const
const std::shared_ptr< RDFDetail::RLoopManager > &	GetProxiedPtr () const
ColumnNames_t	GetValidatedColumnNames (const unsigned int nColumns, const ColumnNames_t &columns)
template<typename RetType>
void	SanityChecksForVary (const std::vector< std::string > &colNames, const std::vector< std::string > &variationTags, std::string_view variationName)

Protected Attributes
RDFInternal::RColumnRegister	fColRegister
	Contains the columns defined up to this node.
std::shared_ptr< ROOT::Detail::RDF::RLoopManager >	fLoopManager
	< The RLoopManager at the root of this computation graph. Never null.

Private Types
using	RFilterBase
using	RLoopManager
using	RRangeBase

Private Member Functions
RInterface< RLoopManager >	CacheImpl (const ColumnNames_t &columnList, std::index_sequence< S... >)
	Implementation of cache.
auto	CallCreateActionWithoutColsIfPossible (const std::shared_ptr< ActionResultType > &resPtr, const std::shared_ptr< Helper > &hPtr, TTraits::TypeList< RDFDetail::RInferredType >) -> decltype(hPtr->Exec(0u), RResultPtr< ActionResultType >{})
std::enable_if_t< std::is_default_constructible< RetType >::value, RInterface< RDFDetail::RLoopManager > >	DefineImpl (std::string_view name, F &&expression, const ColumnNames_t &columns, const std::string &where)
RInterface< RDFDetail::RLoopManager >	JittedVaryImpl (const std::vector< std::string > &colNames, std::string_view expression, const std::vector< std::string > &variationTags, std::string_view variationName, bool isSingleColumn)
RInterface< RDFDetail::RLoopManager >	VaryImpl (const std::vector< std::string > &colNames, F &&expression, const ColumnNames_t &inputColumns, const std::vector< std::string > &variationTags, std::string_view variationName)

Private Attributes
std::shared_ptr< RDFDetail::RLoopManager >	fProxiedPtr
	Smart pointer to the graph node encapsulated by this RInterface.

#include <ROOT/RDataFrame.hxx>

Inheritance diagram for ROOT::RDataFrame:

Member Typedef Documentation

ColumnNames_t

using ROOT::RDataFrame::ColumnNames_t = ROOT::RDF::ColumnNames_t

Definition at line 52 of file RDataFrame.hxx.

RFilterBase

using ROOT::RDF::RInterface< RDFDetail::RLoopManager >::RFilterBase

privateinherited

Definition at line 125 of file RInterface.hxx.

RLoopManager

using ROOT::RDF::RInterface< RDFDetail::RLoopManager >::RLoopManager

privateinherited

Definition at line 127 of file RInterface.hxx.

RRangeBase

using ROOT::RDF::RInterface< RDFDetail::RLoopManager >::RRangeBase

privateinherited

Definition at line 126 of file RInterface.hxx.

Constructor & Destructor Documentation

RDataFrame() [1/8]

ROOT::RDataFrame::RDataFrame	(	std::string_view	treeName,
		std::string_view	fileNameGlob,
		const ColumnNames_t &	defaultColumns = {} )

Build the dataframe.

Parameters

[in]	treeName	Name of the tree contained in the directory
[in]	fileNameGlob	TDirectory where the tree is stored, e.g. a TFile.
[in]	defaultColumns	Collection of default columns.

The filename glob supports the same type of expressions as TChain::Add(), and it is passed as-is to TChain's constructor.

The default columns are looked at in case no column is specified in the booking of actions or transformations.

Note

see ROOT::RDF::RInterface for the documentation of the methods available.

Definition at line 2191 of file RDataFrame.cxx.

RDataFrame() [2/8]

ROOT::RDataFrame::RDataFrame	(	std::string_view	datasetName,
		const std::vector< std::string > &	fileNameGlobs,
		const ColumnNames_t &	defaultColumns = {} )

Build the dataframe.

Parameters

[in]	datasetName	Name of the dataset contained in the directory
[in]	fileNameGlobs	Collection of file names of filename globs
[in]	defaultColumns	Collection of default columns.

The filename globs support the same type of expressions as TChain::Add(), and each glob is passed as-is to TChain's constructor.

The default columns are looked at in case no column is specified in the booking of actions or transformations.

Note

see ROOT::RDF::RInterface for the documentation of the methods available.

Definition at line 2207 of file RDataFrame.cxx.

RDataFrame() [3/8]

ROOT::RDataFrame::RDataFrame ( std::string_view treename, std::initializer_list< std::string > filenames, const ColumnNames_t & defaultColumns = {} )

inline

Definition at line 56 of file RDataFrame.hxx.

RDataFrame() [4/8]

ROOT::RDataFrame::RDataFrame	(	std::string_view	treeName,
		::TDirectory *	dirPtr,
		const ColumnNames_t &	defaultColumns = {} )

RDataFrame() [5/8]

ROOT::RDataFrame::RDataFrame	(	TTree &	tree,
		const ColumnNames_t &	defaultColumns = {} )

Build the dataframe.

Parameters

[in]	tree	The tree or chain to be studied.
[in]	defaultColumns	Collection of default column names to fall back to when none is specified.

The default columns are looked at in case no column is specified in the booking of actions or transformations.

Note

see ROOT::RDF::RInterface for the documentation of the methods available.

Definition at line 2221 of file RDataFrame.cxx.

RDataFrame() [6/8]

ROOT::RDataFrame::RDataFrame

(

ULong64_t

numEntries

)

Build a dataframe that generates numEntries entries.

Parameters

[in]

numEntries

The number of entries to generate.

An empty-source dataframe constructed with a number of entries will generate those entries on the fly when some action is triggered, and it will do so for all the previously-defined columns.

Note

see ROOT::RDF::RInterface for the documentation of the methods available.

Definition at line 2234 of file RDataFrame.cxx.

RDataFrame() [7/8]

ROOT::RDataFrame::RDataFrame	(	std::unique_ptr< ROOT::RDF::RDataSource >	ds,
		const ColumnNames_t &	defaultColumns = {} )

Build dataframe associated to data source.

Parameters

[in]	ds	The data source object.
[in]	defaultColumns	Collection of default column names to fall back to when none is specified.

A dataframe associated to a data source will query it to access column values.

Note

see ROOT::RDF::RInterface for the documentation of the methods available.

Definition at line 2247 of file RDataFrame.cxx.

RDataFrame() [8/8]

ROOT::RDataFrame::RDataFrame

(

ROOT::RDF::Experimental::RDatasetSpec

spec

)

Build dataframe from an RDatasetSpec object.

Parameters

[in]

spec

The dataset specification object.

A dataset specification includes trees and file names, as well as an optional friend list and/or entry range.

Example usage from Python:

spec = (
    ROOT.RDF.Experimental.RDatasetSpec()
    .AddSample(("data", "tree", "file.root"))
    .WithGlobalFriends("friendTree", "friend.root", "alias")
    .WithGlobalRange((100, 200))
)
df = ROOT.RDataFrame(spec)

See also ROOT::RDataFrame::FromSpec().

Definition at line 2271 of file RDataFrame.cxx.

Member Function Documentation

AddDefaultColumns()

void ROOT::RDF::RInterfaceBase::AddDefaultColumns ( )

protectedinherited

Definition at line 360 of file RInterfaceBase.cxx.

Aggregate()

RResultPtr< U > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Aggregate ( AccFun aggregator, MergeFun merger, std::string_view columnName, const U & aggIdentity )

inlineinherited

Execute a user-defined accumulation operation on the processed column values in each processing slot.

Template Parameters

F	The type of the aggregator callable. Automatically deduced.
U	The type of the aggregator variable. Must be default-constructible, copy-constructible and copy-assignable. Automatically deduced.
T	The type of the column to apply the reduction to. Automatically deduced.

Parameters

[in]	aggregator	A callable with signature U(U,T) or void(U&,T), where T is the type of the column, U is the type of the aggregator variable
[in]	merger	A callable with signature U(U,U) or void(std::vector<U>&) used to merge the results of the accumulations of each thread
[in]	columnName	The column to be aggregated. If omitted, the first default column is used instead.
[in]	aggIdentity	The aggregator variable of each thread is initialized to this value (or is default-constructed if the parameter is omitted)

Returns

the result of the aggregation wrapped in a RResultPtr.

An aggregator callable takes two values, an aggregator variable and a column value. The aggregator variable is initialized to aggIdentity or default-constructed if aggIdentity is omitted. This action calls the aggregator callable for each processed entry, passing in the aggregator variable and the value of the column columnName. If the signature is U(U,T) the aggregator variable is then copy-assigned the result of the execution of the callable. Otherwise the signature of aggregator must be void(U&,T).

The merger callable is used to merge the partial accumulation results of each processing thread. It is only called in multi-thread executions. If its signature is U(U,U) the aggregator variables of each thread are merged two by two. If its signature is void(std::vector<U>& a) it is assumed that it merges all aggregators in a[0].

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

auto aggregator = [](double acc, double x) { return acc * x; };
ROOT::EnableImplicitMT();
// If multithread is enabled, the aggregator function will be called by more threads
// and will produce a vector of partial accumulators.
// The merger function performs the final aggregation of these partial results.
auto merger = [](std::vector<double> &accumulators) {
   for (auto i : ROOT::TSeqU(1u, accumulators.size())) {
      accumulators[0] *= accumulators[i];
   }
};
 
// The accumulator is initialized at this value by every thread.
double initValue = 1.;
 
// Multiplies all elements of the column "x"
auto result = d.Aggregate(aggregator, merger, "x", initValue);

Definition at line 3518 of file RInterface.hxx.

Alias()

RInterface< RDFDetail::RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Alias ( std::string_view alias, std::string_view columnName )

inlineinherited

Allow to refer to a column with a different name.

Parameters

[in]	alias	name of the column alias
[in]	columnName	of the column to be aliased

Returns

the first node of the computation graph for which the alias is available.

Aliasing an alias is supported.

Example usage:

auto df_with_alias = df.Alias("simple_name", "very_long&complex_name!!!");

Definition at line 1293 of file RInterface.hxx.

Book()

RResultPtr< typename std::decay_t< Helper >::Result_t > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Book ( Helper && helper, const ColumnNames_t & columns = {} )

inlineinherited

Book execution of a custom action using a user-defined helper object.

Template Parameters

FirstColumn	The type of the first column used by this action. Inferred together with OtherColumns if not present.
OtherColumns	A list of the types of the other columns used by this action
Helper	The type of the user-defined helper. See below for the required interface it should expose.

Parameters

[in]	helper	The Action Helper to be scheduled.
[in]	columns	The names of the columns on which the helper acts.

Returns

the result of the helper wrapped in a RResultPtr.

This method books a custom action for execution. The behavior of the action is completely dependent on the Helper object provided by the caller. The required interface for the helper is described below (more methods that the ones required can be present, e.g. a constructor that takes the number of worker threads is usually useful):

Mandatory interface

• Helper must publicly inherit from ROOT::Detail::RDF::RActionImpl<Helper>
• Helper::Result_t: public alias for the type of the result of this action helper. Result_t must be default-constructible.
• Helper(Helper &&): a move-constructor is required. Copy-constructors are discouraged.
• std::shared_ptr<Result_t> GetResultPtr() const: return a shared_ptr to the result of this action (of type Result_t). The RResultPtr returned by Book will point to this object. Note that this method can be called before Initialize(), because the RResultPtr is constructed before the event loop is started.
• void Initialize(): this method is called once before starting the event-loop. Useful for setup operations. It must reset the state of the helper to the expected state at the beginning of the event loop: the same helper, or copies of it, might be used for multiple event loops (e.g. in the presence of systematic variations).
• void InitTask(TTreeReader *, unsigned int slot): each working thread shall call this method during the event loop, before processing a batch of entries. The pointer passed as argument, if not null, will point to the TTreeReader that RDataFrame has set up to read the task's batch of entries. It is passed to the helper to allow certain advanced optimizations it should not usually serve any purpose for the Helper. This method is often no-op for simple helpers.
• void Exec(unsigned int slot, ColumnTypes...columnValues): each working thread shall call this method during the event-loop, possibly concurrently. No two threads will ever call Exec with the same 'slot' value: this parameter is there to facilitate writing thread-safe helpers. The other arguments will be the values of the requested columns for the particular entry being processed.
• void Finalize(): this method is called at the end of the event loop. Commonly used to finalize the contents of the result.
• std::string GetActionName(): it returns a string identifier for this type of action that RDataFrame will use in diagnostics, SaveGraph(), etc.

Optional methods

If these methods are implemented they enable extra functionality as per the description below.

• Result_t &PartialUpdate(unsigned int slot): if present, it must return the value of the partial result of this action for the given 'slot'. Different threads might call this method concurrently, but will do so with different 'slot' numbers. RDataFrame leverages this method to implement RResultPtr::OnPartialResult().
• ROOT::RDF::SampleCallback_t GetSampleCallback(): if present, it must return a callable with the appropriate signature (see ROOT::RDF::SampleCallback_t) that will be invoked at the beginning of the processing of every sample, as in DefinePerSample().
• Helper MakeNew(void *newResult, std::string_view variation = "nominal"): if implemented, it enables varying the action's result with VariationsFor(). It takes a type-erased new result that can be safely cast to a std::shared_ptr<Result_t> * (a pointer to shared pointer) and should be used as the action's output result. The function optionally takes the name of the current variation which could be useful in customizing its behaviour.

In case Book is called without specifying column types as template arguments, corresponding typed code will be just-in-time compiled by RDataFrame. In that case the Helper class needs to be known to the ROOT interpreter.

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Examples

See this tutorial for an example implementation of an action helper.

It is also possible to inspect the code used by built-in RDataFrame actions at ActionHelpers.hxx.

Definition at line 3624 of file RInterface.hxx.

Cache()

RInterface< RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Cache ( const ColumnNames_t & columnList )

inlineinherited

Save selected columns in memory.

Template Parameters

ColumnTypes

variadic list of branch/column types.

Parameters

[in]

columnList

columns to be cached in memory.

Returns

a RDataFrame that wraps the cached dataset.

This action returns a new RDataFrame object, completely detached from the originating RDataFrame. The new dataframe only contains the cached columns and stores their content in memory for fast, zero-copy subsequent access.

Use Cache if you know you will only need a subset of the (Filtered) data that fits in memory and that will be accessed many times.

Note

Cache will refuse to process columns with names of the form #columnname. These are special columns made available by some data sources (e.g. RNTupleDS) that represent the size of column columnname, and are not meant to be written out with that name (which is not a valid C++ variable name). Instead, go through an Alias(): df.Alias("nbar", "#bar").Cache<std::size_t>(..., {"nbar"}).

Example usage:

Types and columns specified:

auto cache_some_cols_df = df.Cache<double, MyClass, int>({"col0", "col1", "col2"});

Types inferred and columns specified (this invocation relies on jitting):

auto cache_some_cols_df = df.Cache({"col0", "col1", "col2"});

Types inferred and columns selected with a regexp (this invocation relies on jitting):

auto cache_all_cols_df = df.Cache(myRegexp);

Definition at line 1637 of file RInterface.hxx.

CacheImpl()

RInterface< RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::CacheImpl ( const ColumnNames_t & columnList, std::index_sequence< S... >  )

inlineprivateinherited

Implementation of cache.

Definition at line 3803 of file RInterface.hxx.

CallCreateActionWithoutColsIfPossible()

auto ROOT::RDF::RInterface< RDFDetail::RLoopManager >::CallCreateActionWithoutColsIfPossible ( const std::shared_ptr< ActionResultType > & resPtr, const std::shared_ptr< Helper > & hPtr, TTraits::TypeList< RDFDetail::RInferredType >  )-> decltype(hPtr->Exec(0u), RResultPtr< ActionResultType >{})

inlineprivateinherited

Definition at line 3895 of file RInterface.hxx.

CheckAndFillDSColumns() [1/2]

template<typename... ColumnTypes>

void ROOT::RDF::RInterfaceBase::CheckAndFillDSColumns ( ColumnNames_t validCols, TTraits::TypeList< ColumnTypes... > typeList )

inlineprotectedinherited

Definition at line 142 of file RInterfaceBase.hxx.

CheckAndFillDSColumns() [2/2]

void ROOT::RDF::RInterfaceBase::CheckAndFillDSColumns ( const std::vector< std::string > & colNames, const std::vector< const std::type_info * > & colTypeIDs )

inlineprotectedinherited

Definition at line 148 of file RInterfaceBase.hxx.

CheckIMTDisabled()

void ROOT::RDF::RInterfaceBase::CheckIMTDisabled ( std::string_view callerName )

protectedinherited

Definition at line 351 of file RInterfaceBase.cxx.

Count()

RResultPtr< ULong64_t > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Count ( )

inlineinherited

Return the number of entries processed (lazy action).

Returns

the number of entries wrapped in a RResultPtr.

Useful e.g. for counting the number of entries passing a certain filter (see also Report). This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

auto nEntriesAfterCuts = myFilteredDf.Count();

Definition at line 1903 of file RInterface.hxx.

CreateAction() [1/2]

template<typename ActionTag, typename... ColTypes, typename ActionResultType, typename RDFNode, typename HelperArgType = ActionResultType, std::enable_if_t< RDFInternal::RNeedJitting< ColTypes... >::value, int > = 0>

RResultPtr< ActionResultType > ROOT::RDF::RInterfaceBase::CreateAction ( const ColumnNames_t & columns, const std::shared_ptr< ActionResultType > & r, const std::shared_ptr< HelperArgType > & helperArg, const std::shared_ptr< RDFNode > & proxiedPtr, const int nColumns = -1, const bool vector2RVec = true )

inlineprotectedinherited

Create RAction object, return RResultPtr for the action Overload for the case in which one or more column types were not specified (RTTI + jitting).

This overload has a nColumns optional argument. If present, the number of required columns for this action is taken equal to nColumns, otherwise it is assumed to be sizeof...(ColTypes).

Definition at line 187 of file RInterfaceBase.hxx.

CreateAction() [2/2]

template<typename ActionTag, typename... ColTypes, typename ActionResultType, typename RDFNode, typename HelperArgType = ActionResultType, std::enable_if_t<!RDFInternal::RNeedJitting< ColTypes... >::value, int > = 0>

RResultPtr< ActionResultType > ROOT::RDF::RInterfaceBase::CreateAction ( const ColumnNames_t & columns, const std::shared_ptr< ActionResultType > & r, const std::shared_ptr< HelperArgType > & helperArg, const std::shared_ptr< RDFNode > & proxiedPtr, const int = -1 )

inlineprotectedinherited

Create RAction object, return RResultPtr for the action Overload for the case in which all column types were specified (no jitting).

For most actions, r and helperArg will refer to the same object, because the only argument to forward to the action helper is the result value itself. We need the distinction for actions such as Snapshot or Cache, for which the constructor arguments of the action helper are different from the returned value.

Definition at line 163 of file RInterfaceBase.hxx.

DefaultValueFor()

RInterface< RDFDetail::RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::DefaultValueFor ( std::string_view column, const T & defaultValue )

inlineinherited

In case the value in the given column is missing, provide a default value.

Template Parameters

T	The type of the column

Parameters

[in]	column	Column name where missing values should be replaced by the given default value
[in]	defaultValue	Value to provide instead of a missing value

Returns

The node of the graph that will provide a default value

This operation is useful in case an entry of the dataset is incomplete, i.e. if one or more of the columns do not have valid values. It does not modify the values of the column, but in case any entry is missing, it will provide the default value to downstream nodes instead.

Use cases include:

• When processing multiple files, one or more of them is missing a column
• In horizontal joining with entry matching, a certain dataset has no match for the current entry.

Example usage:

// Assume a dataset with columns [idx, x] matching another dataset with
// columns [idx, y]. For idx == 42, the right-hand dataset has no match
ROOT::RDataFrame df{dataset};
auto df_default = df.DefaultValueFor("y", 33)
                    .Define("z", [](int x, int y) { return x + y; }, {"x", "y"});
auto colz = df_default.Take<int>("z");

df = ROOT.RDataFrame(dataset)
df_default = df.DefaultValueFor("y", 33).Define("z", "x + y")
colz = df_default.Take[int]("z")

Definition at line 682 of file RInterface.hxx.

Define()

RInterface< RDFDetail::RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Define ( std::string_view name, F expression, const ColumnNames_t & columns = {} )

inlineinherited

Define a new column.

Parameters

[in]	name	The name of the defined column.
[in]	expression	Function, lambda expression, functor class or any other callable object producing the defined value. Returns the value that will be assigned to the defined column. This callable must be thread safe when used with multiple threads.
[in]	columns	Names of the columns/branches in input to the producer function.

Returns

the first node of the computation graph for which the new quantity is defined.

Define a column that will be visible from all subsequent nodes of the functional chain. The expression is only evaluated for entries that pass all the preceding filters. A new variable is created called name, accessible as if it was contained in the dataset from subsequent transformations/actions.

Use cases include:

• caching the results of complex calculations for easy and efficient multiple access
• extraction of quantities of interest from complex objects

An exception is thrown if the name of the new column is already in use in this branch of the computation graph. Note that the callable must be thread safe when called from multiple threads. Use DefineSlot() if needed.

Example usage:

// assuming a function with signature:
double myComplexCalculation(const RVec<float> &muon_pts);
// we can pass it directly to Define
auto df_with_define = df.Define("newColumn", myComplexCalculation, {"muon_pts"});
// alternatively, we can pass the body of the function as a string, as in Filter:
auto df_with_define = df.Define("newColumn", "x*x + y*y");

Note

If the body of the string expression contains an explicit return statement (even if it is in a nested scope), RDataFrame will not add another one in front of the expression. So this will not work:

df.Define("x2", "Map(v, [](float e) { return e*e; })")

but instead this will:

df.Define("x2", "return Map(v, [](float e) { return e*e; })")

Definition at line 453 of file RInterface.hxx.

DefineImpl()

std::enable_if_t< std::is_default_constructible< RetType >::value, RInterface< RDFDetail::RLoopManager > > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::DefineImpl ( std::string_view name, F && expression, const ColumnNames_t & columns, const std::string & where )

inlineprivateinherited

Definition at line 3742 of file RInterface.hxx.

DefinePerSample()

RInterface< RDFDetail::RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::DefinePerSample ( std::string_view name, F expression )

inlineinherited

Define a new column that is updated when the input sample changes.

Parameters

[in]	name	The name of the defined column.
[in]	expression	A C++ callable that computes the new value of the defined column.

Returns

the first node of the computation graph for which the new quantity is defined.

The signature of the callable passed as second argument should be T(unsigned int slot, const ROOT::RDF::RSampleInfo &id) where:

• T is the type of the defined column
• slot is a number in the range [0, nThreads) that is different for each processing thread. This can simplify the definition of thread-safe callables if you are interested in using parallel capabilities of RDataFrame.
• id is an instance of a ROOT::RDF::RSampleInfo object which contains information about the sample which is being processed (see the class docs for more information).

DefinePerSample() is useful to e.g. define a quantity that depends on which TTree in which TFile is being processed or to inject a callback into the event loop that is only called when the processing of a new sample starts rather than at every entry.

The callable will be invoked once per input TTree or once per multi-thread task, whichever is more often.

Example usage:

ROOT::RDataFrame df{"mytree", {"sample1.root","sample2.root"}};
df.DefinePerSample("weightbysample",
                   [](unsigned int slot, const ROOT::RDF::RSampleInfo &id)
                   { return id.Contains("sample1") ? 1.0f : 2.0f; });

Definition at line 745 of file RInterface.hxx.

DefineSlot()

RInterface< RDFDetail::RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::DefineSlot ( std::string_view name, F expression, const ColumnNames_t & columns = {} )

inlineinherited

Define a new column with a value dependent on the processing slot.

Parameters

[in]	name	The name of the defined column.
[in]	expression	Function, lambda expression, functor class or any other callable object producing the defined value. Returns the value that will be assigned to the defined column.
[in]	columns	Names of the columns/branches in input to the producer function (excluding the slot number).

Returns

the first node of the computation graph for which the new quantity is defined.

This alternative implementation of Define is meant as a helper to evaluate new column values in a thread-safe manner. The expression must be a callable of signature R(unsigned int, T1, T2, ...) where T1, T2... are the types of the columns that the expression takes as input. The first parameter is reserved for an unsigned integer representing a "slot number". RDataFrame guarantees that different threads will invoke the expression with different slot numbers - slot numbers will range from zero to ROOT::GetThreadPoolSize()-1. Note that there is no guarantee as to how often each slot will be reached during the event loop.

The following two calls are equivalent, although DefineSlot is slightly more performant:

int function(unsigned int, double, double);
df.Define("x", function, {"rdfslot_", "column1", "column2"})
df.DefineSlot("x", function, {"column1", "column2"})

See Define() for more information.

Definition at line 483 of file RInterface.hxx.

DefineSlotEntry()

RInterface< RDFDetail::RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::DefineSlotEntry ( std::string_view name, F expression, const ColumnNames_t & columns = {} )

inlineinherited

Define a new column with a value dependent on the processing slot and the current entry.

Parameters

[in]	name	The name of the defined column.
[in]	expression	Function, lambda expression, functor class or any other callable object producing the defined value. Returns the value that will be assigned to the defined column.
[in]	columns	Names of the columns/branches in input to the producer function (excluding slot and entry).

Returns

the first node of the computation graph for which the new quantity is defined.

This alternative implementation of Define is meant as a helper in writing entry-specific, thread-safe custom columns. The expression must be a callable of signature R(unsigned int, ULong64_t, T1, T2, ...) where T1, T2... are the types of the columns that the expression takes as input. The first parameter is reserved for an unsigned integer representing a "slot number". RDataFrame guarantees that different threads will invoke the expression with different slot numbers - slot numbers will range from zero to ROOT::GetThreadPoolSize()-1. Note that there is no guarantee as to how often each slot will be reached during the event loop. The second parameter is reserved for a ULong64_t representing the current entry being processed by the current thread.

The following two Defines are equivalent, although DefineSlotEntry is slightly more performant:

int function(unsigned int, ULong64_t, double, double);
Define("x", function, {"rdfslot_", "rdfentry_", "column1", "column2"})
DefineSlotEntry("x", function, {"column1", "column2"})

See Define() for more information.

Definition at line 514 of file RInterface.hxx.

Describe()

ROOT::RDF::RDFDescription ROOT::RDF::RInterfaceBase::Describe ( )

inherited

Return information about the dataframe.

Returns

information about the dataframe as RDFDescription object

This convenience function describes the dataframe and combines the following information:

• Number of event loops run, see GetNRuns()
• Number of total and defined columns, see GetColumnNames() and GetDefinedColumnNames()
• Column names, see GetColumnNames()
• Column types, see GetColumnType()
• Number of processing slots, see GetNSlots()

This is not an action nor a transformation, just a query to the RDataFrame object. The result is dependent on the node from which this method is called, e.g. the list of defined columns returned by GetDefinedColumnNames().

Please note that this is a convenience feature and the layout of the output can be subject to change and should be parsed via RDFDescription methods.

Example usage:

RDataFrame df(10);
auto df2 = df.Define("x", "1.f").Define("s", "\"myStr\"");
// Describe the dataframe
df2.Describe().Print()
df2.Describe().Print(/*shortFormat=*‍/true)
std::cout << df2.Describe().AsString() << std::endl;
std::cout << df2.Describe().AsString(/*shortFormat=*&zwj;/true) << std::endl;

Definition at line 178 of file RInterfaceBase.cxx.

DescribeDataset()

std::string ROOT::RDF::RInterfaceBase::DescribeDataset ( ) const

protectedinherited

Definition at line 35 of file RInterfaceBase.cxx.

Display()

RResultPtr< RDisplay > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Display ( const ColumnNames_t & columnList, size_t nRows = 5, size_t nMaxCollectionElements = 10 )

inlineinherited

Provides a representation of the columns in the dataset.

Template Parameters

ColumnTypes

variadic list of branch/column types.

Parameters

[in]	columnList	Names of the columns to be displayed.
[in]	nRows	Number of events for each column to be displayed.
[in]	nMaxCollectionElements	Maximum number of collection elements to display per row.

Returns

the RDisplay instance wrapped in a RResultPtr.

This function returns a RResultPtr<RDisplay> containing all the entries to be displayed, organized in a tabular form. RDisplay will either print on the standard output a summarized version through RDisplay::Print() or will return a complete version through RDisplay::AsString().

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Preparing the RResultPtr<RDisplay> object with all columns and default number of entries
auto d1 = rdf.Display("");
// Preparing the RResultPtr<RDisplay> object with two columns and 128 entries
auto d2 = d.Display({"x", "y"}, 128);
// Printing the short representations, the event loop will run
d1->Print();
d2->Print();

Definition at line 3669 of file RInterface.hxx.

Fill()

RResultPtr< std::decay_t< T > > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Fill ( T && model, const ColumnNames_t & columnList )

inlineinherited

Return an object of type T on which T::Fill will be called once per event (lazy action).

Type T must provide at least:

• a copy-constructor
• a Fill method that accepts as many arguments and with same types as the column names passed as columnList (these types can also be passed as template parameters to this method)
• a Merge method with signature Merge(TCollection *) or Merge(const std::vector<T *>&) that merges the objects passed as argument into the object on which Merge was called (an analogous of TH1::Merge). Note that if the signature that takes a TCollection* is used, then T must inherit from TObject (to allow insertion in the TCollection*).

Template Parameters

FirstColumn	The first type of the column the values of which are used to fill the object. Inferred together with OtherColumns if not present.
OtherColumns	A list of the other types of the columns the values of which are used to fill the object.
T	The type of the object to fill. Automatically deduced.

Parameters

[in]	model	The model to be considered to build the new return value.
[in]	columnList	A list containing the names of the columns that will be passed when calling Fill

Returns

the filled object wrapped in a RResultPtr.

The user gives up ownership of the model object. The list of column names to be used for filling must always be specified. This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

MyClass obj;
// Deduce column types (this invocation needs jitting internally, and in this case
// MyClass needs to be known to the interpreter)
auto myFilledObj = myDf.Fill(obj, {"col0", "col1"});
// explicit column types
auto myFilledObj = myDf.Fill<float, float>(obj, {"col0", "col1"});

Definition at line 3173 of file RInterface.hxx.

Filter()

RInterface< RDFDetail::RFilter< F, RDFDetail::RLoopManager > > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Filter ( F f, const ColumnNames_t & columns = {}, std::string_view name = "" )

inlineinherited

Append a filter to the call graph.

Parameters

[in]	f	Function, lambda expression, functor class or any other callable object. It must return a bool signalling whether the event has passed the selection (true) or not (false).
[in]	columns	Names of the columns/branches in input to the filter function.
[in]	name	Optional name of this filter. See Report.

Returns

the filter node of the computation graph.

Append a filter node at the point of the call graph corresponding to the object this method is called on. The callable f should not have side-effects (e.g. modification of an external or static variable) to ensure correct results when implicit multi-threading is active.

RDataFrame only evaluates filters when necessary: if multiple filters are chained one after another, they are executed in order and the first one returning false causes the event to be discarded. Even if multiple actions or transformations depend on the same filter, it is executed once per entry. If its result is requested more than once, the cached result is served.

Example usage:

// C++ callable (function, functor class, lambda...) that takes two parameters of the types of "x" and "y"
auto filtered = df.Filter(myCut, {"x", "y"});
 
// String: it must contain valid C++ except that column names can be used instead of variable names
auto filtered = df.Filter("x*y > 0");

Note

If the body of the string expression contains an explicit return statement (even if it is in a nested scope), RDataFrame will not add another one in front of the expression. So this will not work:

df.Filter("Sum(Map(vec, [](float e) { return e*e > 0.5; }))")

but instead this will:

df.Filter("return Sum(Map(vec, [](float e) { return e*e > 0.5; }))")

Definition at line 235 of file RInterface.hxx.

FilterAvailable()

RInterface< RDFDetail::RFilterWithMissingValues< RDFDetail::RLoopManager > > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::FilterAvailable ( std::string_view column )

inlineinherited

Discard entries with missing values.

Parameters

[in]

column

Column name whose entries with missing values should be discarded

Returns

The filter node of the computation graph

This operation is useful in case an entry of the dataset is incomplete, i.e. if one or more of the columns do not have valid values. If the value of the input column is missing for an entry, the entire entry will be discarded from the rest of this branch of the computation graph.

Use cases include:

• When processing multiple files, one or more of them is missing a column
• In horizontal joining with entry matching, a certain dataset has no match for the current entry.

Example usage:

# Assume a dataset with columns [idx, x] matching another dataset with
# columns [idx, y]. For idx == 42, the right-hand dataset has no match
df = ROOT.RDataFrame(dataset)
df_nomissing = df.FilterAvailable("idx").Define("z", "x + y")
colz = df_nomissing.Take[int]("z")

// Assume a dataset with columns [idx, x] matching another dataset with
// columns [idx, y]. For idx == 42, the right-hand dataset has no match
ROOT::RDataFrame df{dataset};
auto df_nomissing = df.FilterAvailable("idx")
                      .Define("z", [](int x, int y) { return x + y; }, {"x", "y"});
auto colz = df_nomissing.Take<int>("z");

Note

See FilterMissing() if you want to keep only the entries with missing values instead.

Definition at line 348 of file RInterface.hxx.

FilterMissing()

RInterface< RDFDetail::RFilterWithMissingValues< RDFDetail::RLoopManager > > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::FilterMissing ( std::string_view column )

inlineinherited

Keep only the entries that have missing values.

Parameters

[in]

column

Column name whose entries with missing values should be kept

Returns

The filter node of the computation graph

This operation is useful in case an entry of the dataset is incomplete, i.e. if one or more of the columns do not have valid values. It only keeps the entries for which the value of the input column is missing.

Use cases include:

• When processing multiple files, one or more of them is missing a column
• In horizontal joining with entry matching, a certain dataset has no match for the current entry.

Example usage:

# Assume a dataset made of two files vertically chained together, one has
# column "x" and the other has column "y"
df = ROOT.RDataFrame(dataset)
df_valid_col_x = df.FilterMissing("y")
df_valid_col_y = df.FilterMissing("x")
display_x = df_valid_col_x.Display(("x",))
display_y = df_valid_col_y.Display(("y",))

// Assume a dataset made of two files vertically chained together, one has
// column "x" and the other has column "y"
ROOT.RDataFrame df{dataset};
auto df_valid_col_x = df.FilterMissing("y");
auto df_valid_col_y = df.FilterMissing("x");
auto display_x = df_valid_col_x.Display<int>({"x"});
auto display_y = df_valid_col_y.Display<int>({"y"});

Note

See FilterAvailable() if you want to discard the entries in case there is a missing value instead.

Definition at line 399 of file RInterface.hxx.

Foreach()

void ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Foreach ( F f, const ColumnNames_t & columns = {} )

inlineinherited

Execute a user-defined function on each entry (instant action).

Parameters

[in]	f	Function, lambda expression, functor class or any other callable object performing user defined calculations.
[in]	columns	Names of the columns/branches in input to the user function.

The callable f is invoked once per entry. This is an instant action: upon invocation, an event loop as well as execution of all scheduled actions is triggered. Users are responsible for the thread-safety of this callable when executing with implicit multi-threading enabled (i.e. ROOT::EnableImplicitMT).

Example usage:

myDf.Foreach([](int i){ std::cout << i << std::endl;}, {"myIntColumn"});

Definition at line 1785 of file RInterface.hxx.

ForeachSlot()

void ROOT::RDF::RInterface< RDFDetail::RLoopManager >::ForeachSlot ( F f, const ColumnNames_t & columns = {} )

inlineinherited

Execute a user-defined function requiring a processing slot index on each entry (instant action).

Parameters

[in]	f	Function, lambda expression, functor class or any other callable object performing user defined calculations.
[in]	columns	Names of the columns/branches in input to the user function.

Same as Foreach, but the user-defined function takes an extra unsigned int as its first parameter, the processing slot index. This slot index will be assigned a different value, 0 to poolSize - 1, for each thread of execution. This is meant as a helper in writing thread-safe Foreach actions when using RDataFrame after ROOT::EnableImplicitMT(). The user-defined processing callable is able to follow different streams of processing indexed by the first parameter. ForeachSlot works just as well with single-thread execution: in that case slot will always be 0.

Example usage:

myDf.ForeachSlot([](unsigned int s, int i){ std::cout << "Slot " << s << ": "<< i << std::endl;}, {"myIntColumn"});

Definition at line 1815 of file RInterface.hxx.

GetColumnNames()

ROOT::RDF::ColumnNames_t ROOT::RDF::RInterfaceBase::GetColumnNames ( )

inherited

Returns the names of the available columns.

Returns

the container of column names.

This is not an action nor a transformation, just a query to the RDataFrame object.

Example usage:

auto colNames = d.GetColumnNames();
// Print columns' names
for (auto &&colName : colNames) std::cout << colName << std::endl;

Definition at line 77 of file RInterfaceBase.cxx.

GetColumnType()

std::string ROOT::RDF::RInterfaceBase::GetColumnType ( std::string_view column )

inherited

Return the type of a given column as a string.

Returns

the type of the required column.

This is not an action nor a transformation, just a query to the RDataFrame object.

Example usage:

auto colType = d.GetColumnType("columnName");
// Print column type
std::cout << "Column " << colType << " has type " << colType << std::endl;

Definition at line 138 of file RInterfaceBase.cxx.

GetColumnTypeNamesList()

ROOT::RDF::ColumnNames_t ROOT::RDF::RInterfaceBase::GetColumnTypeNamesList ( const ColumnNames_t & columnList )

protectedinherited

Definition at line 341 of file RInterfaceBase.cxx.

GetDatasetTopLevelFieldNames()

ROOT::RDF::ColumnNames_t ROOT::RDF::RInterfaceBase::GetDatasetTopLevelFieldNames ( )

inherited

Retrieve the names of top-level field names.

For data sources that support hierarchical dataset schemas, such as TTree or RNTuple, this function will retrieve the names of top-level fields. For example, if the schema contains a user class with a data member, only the name of the top-level field containing the user class object would be reported, but not the name of the data member sub-field.

For all other data sources, returns the list of all available dataset columns.

Definition at line 113 of file RInterfaceBase.cxx.

GetDataSource()

RDataSource * ROOT::RDF::RInterfaceBase::GetDataSource ( ) const

inlineprotectedinherited

Definition at line 134 of file RInterfaceBase.hxx.

GetDefinedColumnNames()

ROOT::RDF::ColumnNames_t ROOT::RDF::RInterfaceBase::GetDefinedColumnNames ( )

inherited

Returns the names of the defined columns.

Returns

the container of the defined column names.

This is not an action nor a transformation, just a simple utility to get the columns names that have been defined up to the node. If no column has been defined, e.g. on a root node, it returns an empty collection.

Example usage:

auto defColNames = d.GetDefinedColumnNames();
// Print defined columns' names
for (auto &&defColName : defColNames) std::cout << defColName << std::endl;

Definition at line 250 of file RInterfaceBase.cxx.

GetFilterNames()

std::vector< std::string > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::GetFilterNames ( )

inlineinherited

Returns the names of the filters created.

Returns

the container of filters names.

If called on a root node, all the filters in the computation graph will be printed. For any other node, only the filters upstream of that node. Filters without a name are printed as "Unnamed Filter" This is not an action nor a transformation, just a query to the RDataFrame object.

Example usage:

auto filtNames = d.GetFilterNames();
for (auto &&filtName : filtNames) std::cout << filtName << std::endl;

Definition at line 3466 of file RInterface.hxx.

GetLoopManager()

RDFDetail::RLoopManager * ROOT::RDF::RInterfaceBase::GetLoopManager ( ) const

inlineprotectedinherited

Definition at line 133 of file RInterfaceBase.hxx.

GetNFiles()

unsigned int ROOT::RDF::RInterfaceBase::GetNFiles ( )

inherited

Definition at line 27 of file RInterfaceBase.cxx.

GetNRuns()

unsigned int ROOT::RDF::RInterfaceBase::GetNRuns ( ) const

inherited

Gets the number of event loops run.

Returns

The number of event loops run by this RDataFrame instance

This method returns the number of events loops run so far by this RDataFrame instance.

Example usage:

ROOT::RDataFrame df(1);
std::cout << df.GetNRuns() << std::endl; // prints "0"
df.Sum("rdfentry_").GetValue(); // trigger the event loop
std::cout << df.GetNRuns() << std::endl; // prints "1"
df.Sum("rdfentry_").GetValue(); // trigger another event loop
std::cout << df.GetNRuns() << std::endl; // prints "2"

Definition at line 336 of file RInterfaceBase.cxx.

GetNSlots()

unsigned int ROOT::RDF::RInterfaceBase::GetNSlots ( ) const

inherited

Gets the number of data processing slots.

Returns

The number of data processing slots used by this RDataFrame instance

This method returns the number of data processing slots used by this RDataFrame instance. This number is influenced by the global switch ROOT::EnableImplicitMT().

Example usage:

ROOT::EnableImplicitMT(6)
ROOT::RDataFrame df(1);
std::cout << df.GetNSlots() << std::endl; // prints "6"

Definition at line 317 of file RInterfaceBase.cxx.

GetProxiedPtr()

const std::shared_ptr< RDFDetail::RLoopManager > & ROOT::RDF::RInterface< RDFDetail::RLoopManager >::GetProxiedPtr ( ) const

inlineprotectedinherited

Definition at line 3922 of file RInterface.hxx.

GetValidatedColumnNames()

ColumnNames_t ROOT::RDF::RInterfaceBase::GetValidatedColumnNames ( const unsigned int nColumns, const ColumnNames_t & columns )

inlineprotectedinherited

Definition at line 136 of file RInterfaceBase.hxx.

GetVariations()

ROOT::RDF::RVariationsDescription ROOT::RDF::RInterfaceBase::GetVariations ( ) const

inherited

Return a descriptor for the systematic variations registered in this branch of the computation graph.

This is not an action nor a transformation, just a simple utility to inspect the systematic variations that have been registered with Vary() up to this node. When called on the root node, it returns an empty descriptor.

Example usage:

auto variations = d.GetVariations();
variations.Print();

Definition at line 275 of file RInterfaceBase.cxx.

Graph()

RResultPtr<::TGraph > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Graph ( std::string_view x = "", std::string_view y = "" )

inlineinherited

Fill and return a TGraph object (lazy action).

Template Parameters

X	The type of the column used to fill the x axis.
Y	The type of the column used to fill the y axis.

Parameters

[in]	x	The name of the column that will fill the x axis.
[in]	y	The name of the column that will fill the y axis.

Returns

the TGraph wrapped in a RResultPtr.

Columns can be of a container type (e.g. std::vector<double>), in which case the TGraph is filled with each one of the elements of the container. If Multithreading is enabled, the order in which points are inserted is undefined. If the Graph has to be drawn, it is suggested to the user to sort it on the x before printing. A name and a title to the TGraph is given based on the input column names.

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Deduce column types (this invocation needs jitting internally)
auto myGraph1 = myDf.Graph("xValues", "yValues");
// Explicit column types
auto myGraph2 = myDf.Graph<int, float>("xValues", "yValues");

Note

Differently from other ROOT interfaces, the returned TGraph is not associated to gDirectory and the caller is responsible for its lifetime (in particular, a typical source of confusion is that if result histograms go out of scope before the end of the program, ROOT might display a blank canvas).

Definition at line 2850 of file RInterface.hxx.

GraphAsymmErrors()

RResultPtr<::TGraphAsymmErrors > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::GraphAsymmErrors ( std::string_view x = "", std::string_view y = "", std::string_view exl = "", std::string_view exh = "", std::string_view eyl = "", std::string_view eyh = "" )

inlineinherited

Fill and return a TGraphAsymmErrors object (lazy action).

Parameters

[in]	x	The name of the column that will fill the x axis.
[in]	y	The name of the column that will fill the y axis.
[in]	exl	The name of the column of X low errors
[in]	exh	The name of the column of X high errors
[in]	eyl	The name of the column of Y low errors
[in]	eyh	The name of the column of Y high errors

Returns

the TGraphAsymmErrors wrapped in a RResultPtr.

Columns can be of a container type (e.g. std::vector<double>), in which case the graph is filled with each one of the elements of the container. If Multithreading is enabled, the order in which points are inserted is undefined.

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Deduce column types (this invocation needs jitting internally)
auto myGAE1 = myDf.GraphAsymmErrors("xValues", "yValues", "exl", "exh", "eyl", "eyh");
// Explicit column types
using f = float
auto myGAE2 = myDf.GraphAsymmErrors<f, f, f, f, f, f>("xValues", "yValues", "exl", "exh", "eyl", "eyh");

GraphAsymmErrors should also be used for the cases in which values associated only with one of the axes have associated errors. For example, only ey exist and ex are equal to zero. In such cases, user should do the following:

// Create a column of zeros in RDataFrame
auto rdf_withzeros = rdf.Define("zero", "0");
// or alternatively:
auto rdf_withzeros = rdf.Define("zero", []() -> double { return 0.;});
// Create the graph with y errors only
auto rdf_errorsOnYOnly = rdf_withzeros.GraphAsymmErrors("xValues", "yValues", "zero", "zero", "eyl", "eyh");

Note

Differently from other ROOT interfaces, the returned TGraphAsymmErrors is not associated to gDirectory and the caller is responsible for its lifetime (in particular, a typical source of confusion is that if result histograms go out of scope before the end of the program, ROOT might display a blank canvas).

Definition at line 2915 of file RInterface.hxx.

HasColumn()

bool ROOT::RDF::RInterfaceBase::HasColumn ( std::string_view columnName )

inherited

Checks if a column is present in the dataset.

Returns

true if the column is available, false otherwise

This method checks if a column is part of the input ROOT dataset, has been defined or can be provided by the data source.

Example usage:

ROOT::RDataFrame base(1);
auto rdf = base.Define("definedColumn", [](){return 0;});
rdf.HasColumn("definedColumn"); // true: we defined it
rdf.HasColumn("rdfentry_"); // true: it's always there
rdf.HasColumn("foo"); // false: it is not there

Definition at line 294 of file RInterfaceBase.cxx.

Hist()

RResultPtr< ROOT::Experimental::RHist< BinContentType > > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Hist ( std::uint64_t nNormalBins, std::pair< double, double > interval, std::string_view vName )

inlineinherited

Fill and return a one-dimensional RHist (lazy action).

Template Parameters

BinContentType

The bin content type of the returned RHist.

Parameters

[in]	nNormalBins	The returned histogram will be constructed using this number of normal bins.
[in]	interval	The axis interval of the constructed histogram (lower end inclusive, upper end exclusive).
[in]	vName	The name of the column that will fill the histogram.

Returns

the histogram wrapped in a RResultPtr.

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

auto myHist = myDf.Hist(10, {5, 15}, "col0");

Definition at line 2562 of file RInterface.hxx.

Histo1D()

RResultPtr<::TH1D > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Histo1D ( const TH1DModel & model = {"", "", 128u, 0., 0.}, std::string_view vName = "" )

inlineinherited

Fill and return a one-dimensional histogram with the values of a column (lazy action).

Template Parameters

V	The type of the column used to fill the histogram.

Parameters

[in]	model	The returned histogram will be constructed using this as a model.
[in]	vName	The name of the column that will fill the histogram.

Returns

the monodimensional histogram wrapped in a RResultPtr.

Columns can be of a container type (e.g. std::vector<double>), in which case the histogram is filled with each one of the elements of the container. In case multiple columns of container type are provided (e.g. values and weights) they must have the same length for each one of the events (but possibly different lengths between events). This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Deduce column type (this invocation needs jitting internally)
auto myHist1 = myDf.Histo1D({"histName", "histTitle", 64u, 0., 128.}, "myColumn");
// Explicit column type
auto myHist2 = myDf.Histo1D<float>({"histName", "histTitle", 64u, 0., 128.}, "myColumn");

Note

Differently from other ROOT interfaces, the returned histogram is not associated to gDirectory and the caller is responsible for its lifetime (in particular, a typical source of confusion is that if result histograms go out of scope before the end of the program, ROOT might display a blank canvas).

Definition at line 1978 of file RInterface.hxx.

Histo2D()

RResultPtr<::TH2D > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Histo2D ( const TH2DModel & model, std::string_view v1Name = "", std::string_view v2Name = "" )

inlineinherited

Fill and return a two-dimensional histogram (lazy action).

Template Parameters

V1	The type of the column used to fill the x axis of the histogram.
V2	The type of the column used to fill the y axis of the histogram.

Parameters

[in]	model	The returned histogram will be constructed using this as a model.
[in]	v1Name	The name of the column that will fill the x axis.
[in]	v2Name	The name of the column that will fill the y axis.

Returns

the bidimensional histogram wrapped in a RResultPtr.

Columns can be of a container type (e.g. std::vector<double>), in which case the histogram is filled with each one of the elements of the container. In case multiple columns of container type are provided (e.g. values and weights) they must have the same length for each one of the events (but possibly different lengths between events). This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Deduce column types (this invocation needs jitting internally)
auto myHist1 = myDf.Histo2D({"histName", "histTitle", 64u, 0., 128., 32u, -4., 4.}, "myValueX", "myValueY");
// Explicit column types
auto myHist2 = myDf.Histo2D<float, float>({"histName", "histTitle", 64u, 0., 128., 32u, -4., 4.}, "myValueX", "myValueY");

Note

Differently from other ROOT interfaces, the returned histogram is not associated to gDirectory and the caller is responsible for its lifetime (in particular, a typical source of confusion is that if result histograms go out of scope before the end of the program, ROOT might display a blank canvas).

Definition at line 2130 of file RInterface.hxx.

Histo3D()

RResultPtr<::TH3D > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Histo3D ( const TH3DModel & model, std::string_view v1Name = "", std::string_view v2Name = "", std::string_view v3Name = "" )

inlineinherited

Fill and return a three-dimensional histogram (lazy action).

Template Parameters

V1	The type of the column used to fill the x axis of the histogram. Inferred if not present.
V2	The type of the column used to fill the y axis of the histogram. Inferred if not present.
V3	The type of the column used to fill the z axis of the histogram. Inferred if not present.

Parameters

[in]	model	The returned histogram will be constructed using this as a model.
[in]	v1Name	The name of the column that will fill the x axis.
[in]	v2Name	The name of the column that will fill the y axis.
[in]	v3Name	The name of the column that will fill the z axis.

Returns

the tridimensional histogram wrapped in a RResultPtr.

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Deduce column types (this invocation needs jitting internally)
auto myHist1 = myDf.Histo3D({"name", "title", 64u, 0., 128., 32u, -4., 4., 8u, -2., 2.},
                            "myValueX", "myValueY", "myValueZ");
// Explicit column types
auto myHist2 = myDf.Histo3D<double, double, float>({"name", "title", 64u, 0., 128., 32u, -4., 4., 8u, -2., 2.},
                                                   "myValueX", "myValueY", "myValueZ");

Note

If three-dimensional histograms consume too much memory in multithreaded runs, the cloning of TH3D per thread can be reduced using ROOT::RDF::Experimental::ThreadsPerTH3(). See the section "Memory Usage" in the RDataFrame description.

Differently from other ROOT interfaces, the returned histogram is not associated to gDirectory and the caller is responsible for its lifetime (in particular, a typical source of confusion is that if result histograms go out of scope before the end of the program, ROOT might display a blank canvas).

Definition at line 2227 of file RInterface.hxx.

HistoND()

RResultPtr<::THnD > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::HistoND ( const THnDModel & model, const ColumnNames_t & columnList, std::string_view wName = "" )

inlineinherited

Fill and return an N-dimensional histogram (lazy action).

Template Parameters

FirstColumn	The first type of the column the values of which are used to fill the object. Inferred if not present.
OtherColumns	A list of the other types of the columns the values of which are used to fill the object.

Parameters

[in]	model	The returned histogram will be constructed using this as a model.
[in]	columnList	A list containing the names of the columns that will be passed when calling Fill.
[in]	wName	The name of the column that will provide the weights.

Returns

the N-dimensional histogram wrapped in a RResultPtr.

This action is lazy: upon invocation of this method the calculation is booked but not executed. See RResultPtr documentation.

Example usage:

auto myFilledObj = myDf.HistoND<float, float, float, float>({"name","title", 4,
                                               {40,40,40,40}, {20.,20.,20.,20.}, {60.,60.,60.,60.}},
                                              {"col0", "col1", "col2", "col3"});

Note

A column with event weights should not be passed as part of columnList, but instead be passed in the new argument wName: HistoND(model, cols, weightCol).

Definition at line 2326 of file RInterface.hxx.

HistoNSparseD()

RResultPtr<::THnSparseD > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::HistoNSparseD ( const THnSparseDModel & model, const ColumnNames_t & columnList, std::string_view wName = "" )

inlineinherited

Fill and return a sparse N-dimensional histogram (lazy action).

Template Parameters

FirstColumn	The first type of the column the values of which are used to fill the object. Inferred if not present.
OtherColumns	A list of the other types of the columns the values of which are used to fill the object.

Parameters

[in]	model	The returned histogram will be constructed using this as a model.
[in]	columnList	A list containing the names of the columns that will be passed when calling Fill.
[in]	wName	The name of the column that will provide the weights.

Returns

the N-dimensional histogram wrapped in a RResultPtr.

This action is lazy: upon invocation of this method the calculation is booked but not executed. See RResultPtr documentation.

Example usage:

auto myFilledObj = myDf.HistoNSparseD<float, float, float, float>({"name","title", 4,
                                               {40,40,40,40}, {20.,20.,20.,20.}, {60.,60.,60.,60.}},
                                              {"col0", "col1", "col2", "col3"});

Note

A column with event weights should not be passed as part of columnList, but instead be passed in the new argument wName: HistoND(model, cols, weightCol).

Definition at line 2447 of file RInterface.hxx.

JittedVaryImpl()

RInterface< RDFDetail::RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::JittedVaryImpl ( const std::vector< std::string > & colNames, std::string_view expression, const std::vector< std::string > & variationTags, std::string_view variationName, bool isSingleColumn )

inlineprivateinherited

Definition at line 3858 of file RInterface.hxx.

Max()

RResultPtr< RDFDetail::MaxReturnType_t< T > > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Max ( std::string_view columnName = "" )

inlineinherited

Return the maximum of processed column values (lazy action).

Template Parameters

T	The type of the branch/column.

Parameters

[in]

columnName

The name of the branch/column to be treated.

Returns

the maximum value of the selected column wrapped in a RResultPtr.

If T is not specified, RDataFrame will infer it from the data and just-in-time compile the correct template specialization of this method. If the type of the column is inferred, the return type is double, the type of the column otherwise.

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Deduce column type (this invocation needs jitting internally)
auto maxVal0 = myDf.Max("values");
// Explicit column type
auto maxVal1 = myDf.Max<double>("values");

Definition at line 3307 of file RInterface.hxx.

Mean()

RResultPtr< double > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Mean ( std::string_view columnName = "" )

inlineinherited

Return the mean of processed column values (lazy action).

Template Parameters

T	The type of the branch/column.

Parameters

[in]

columnName

The name of the branch/column to be treated.

Returns

the mean value of the selected column wrapped in a RResultPtr.

If T is not specified, RDataFrame will infer it from the data and just-in-time compile the correct template specialization of this method. Note that internally, the summations are executed with Kahan sums in double precision, irrespective of the type of column that is read.

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Deduce column type (this invocation needs jitting internally)
auto meanVal0 = myDf.Mean("values");
// Explicit column type
auto meanVal1 = myDf.Mean<double>("values");

Definition at line 3338 of file RInterface.hxx.

Min()

RResultPtr< RDFDetail::MinReturnType_t< T > > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Min ( std::string_view columnName = "" )

inlineinherited

Return the minimum of processed column values (lazy action).

Template Parameters

T	The type of the branch/column.

Parameters

[in]

columnName

The name of the branch/column to be treated.

Returns

the minimum value of the selected column wrapped in a RResultPtr.

If T is not specified, RDataFrame will infer it from the data and just-in-time compile the correct template specialization of this method. If the type of the column is inferred, the return type is double, the type of the column otherwise.

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Deduce column type (this invocation needs jitting internally)
auto minVal0 = myDf.Min("values");
// Explicit column type
auto minVal1 = myDf.Min<double>("values");

Definition at line 3277 of file RInterface.hxx.

operator RNode()

ROOT::RDF::RInterface< RDFDetail::RLoopManager >::operator RNode ( ) const

inlineinherited

Cast any RDataFrame node to a common type ROOT::RDF::RNode.

Different RDataFrame methods return different C++ types. All nodes, however, can be cast to this common type at the cost of a small performance penalty. This allows, for example, storing RDataFrame nodes in a vector, or passing them around via (non-template, C++11) helper functions. Example usage:

// a function that conditionally adds a Range to a RDataFrame node.
RNode MaybeAddRange(RNode df, bool mustAddRange)
{
   return mustAddRange ? df.Range(1) : df;
}
// use as :
ROOT::RDataFrame df(10);
auto maybeRanged = MaybeAddRange(df, true);

Note that it is not a problem to pass RNode's by value.

Definition at line 190 of file RInterface.hxx.

Profile1D()

RResultPtr<::TProfile > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Profile1D ( const TProfile1DModel & model, std::string_view v1Name = "", std::string_view v2Name = "" )

inlineinherited

Fill and return a one-dimensional profile (lazy action).

Template Parameters

V1	The type of the column the values of which are used to fill the profile. Inferred if not present.
V2	The type of the column the values of which are used to fill the profile. Inferred if not present.

Parameters

[in]	model	The model to be considered to build the new return value.
[in]	v1Name	The name of the column that will fill the x axis.
[in]	v2Name	The name of the column that will fill the y axis.

Returns

the monodimensional profile wrapped in a RResultPtr.

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Deduce column types (this invocation needs jitting internally)
auto myProf1 = myDf.Profile1D({"profName", "profTitle", 64u, -4., 4.}, "xValues", "yValues");
// Explicit column types
auto myProf2 = myDf.Graph<int, float>({"profName", "profTitle", 64u, -4., 4.}, "xValues", "yValues");

Note

Differently from other ROOT interfaces, the returned profile is not associated to gDirectory and the caller is responsible for its lifetime (in particular, a typical source of confusion is that if result histograms go out of scope before the end of the program, ROOT might display a blank canvas).

Definition at line 2962 of file RInterface.hxx.

Profile2D()

RResultPtr<::TProfile2D > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Profile2D ( const TProfile2DModel & model, std::string_view v1Name = "", std::string_view v2Name = "", std::string_view v3Name = "" )

inlineinherited

Fill and return a two-dimensional profile (lazy action).

Template Parameters

V1	The type of the column used to fill the x axis of the histogram. Inferred if not present.
V2	The type of the column used to fill the y axis of the histogram. Inferred if not present.
V3	The type of the column used to fill the z axis of the histogram. Inferred if not present.

Parameters

[in]	model	The returned profile will be constructed using this as a model.
[in]	v1Name	The name of the column that will fill the x axis.
[in]	v2Name	The name of the column that will fill the y axis.
[in]	v3Name	The name of the column that will fill the z axis.

Returns

the bidimensional profile wrapped in a RResultPtr.

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Deduce column types (this invocation needs jitting internally)
auto myProf1 = myDf.Profile2D({"profName", "profTitle", 40, -4, 4, 40, -4, 4, 0, 20},
                              "xValues", "yValues", "zValues");
// Explicit column types
auto myProf2 = myDf.Profile2D<int, float, double>({"profName", "profTitle", 40, -4, 4, 40, -4, 4, 0, 20},
                                                  "xValues", "yValues", "zValues");

Note

Differently from other ROOT interfaces, the returned profile is not associated to gDirectory and the caller is responsible for its lifetime (in particular, a typical source of confusion is that if result histograms go out of scope before the end of the program, ROOT might display a blank canvas).

Definition at line 3063 of file RInterface.hxx.

Range()

RInterface< RDFDetail::RRange< RDFDetail::RLoopManager > > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Range ( unsigned int begin, unsigned int end, unsigned int stride = 1 )

inlineinherited

Creates a node that filters entries based on range: [begin, end).

Parameters

[in]	begin	Initial entry number considered for this range.
[in]	end	Final entry number (excluded) considered for this range. 0 means that the range goes until the end of the dataset.
[in]	stride	Process one entry of the [begin, end) range every stride entries. Must be strictly greater than 0.

Returns

the first node of the computation graph for which the event loop is limited to a certain range of entries.

Note that in case of previous Ranges and Filters the selected range refers to the transformed dataset. Ranges are only available if EnableImplicitMT has not been called. Multi-thread ranges are not supported.

Example usage:

auto d_0_30 = d.Range(0, 30); // Pick the first 30 entries
auto d_15_end = d.Range(15, 0); // Pick all entries from 15 onwards
auto d_15_end_3 = d.Range(15, 0, 3); // Stride: from event 15, pick an event every 3

Definition at line 1744 of file RInterface.hxx.

Redefine()

RInterface< RDFDetail::RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Redefine ( std::string_view name, F expression, const ColumnNames_t & columns = {} )

inlineinherited

Overwrite the value and/or type of an existing column.

Parameters

[in]	name	The name of the column to redefine.
[in]	expression	Function, lambda expression, functor class or any other callable object producing the defined value. Returns the value that will be assigned to the defined column.
[in]	columns	Names of the columns/branches in input to the expression.

Returns

the first node of the computation graph for which the quantity is redefined.

The old value of the column can be used as an input for the expression.

An exception is thrown in case the column to redefine does not already exist. See Define() for more information.

Definition at line 572 of file RInterface.hxx.

RedefineSlot()

RInterface< RDFDetail::RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::RedefineSlot ( std::string_view name, F expression, const ColumnNames_t & columns = {} )

inlineinherited

Overwrite the value and/or type of an existing column.

Parameters

[in]	name	The name of the column to redefine.
[in]	expression	Function, lambda expression, functor class or any other callable object producing the defined value. Returns the value that will be assigned to the defined column.
[in]	columns	Names of the columns/branches in input to the producer function (excluding slot).

Returns

the first node of the computation graph for which the new quantity is defined.

The old value of the column can be used as an input for the expression. An exception is thrown in case the column to redefine does not already exist.

See DefineSlot() for more information.

Definition at line 591 of file RInterface.hxx.

RedefineSlotEntry()

RInterface< RDFDetail::RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::RedefineSlotEntry ( std::string_view name, F expression, const ColumnNames_t & columns = {} )

inlineinherited

Overwrite the value and/or type of an existing column.

Parameters

[in]	name	The name of the column to redefine.
[in]	expression	Function, lambda expression, functor class or any other callable object producing the defined value. Returns the value that will be assigned to the defined column.
[in]	columns	Names of the columns/branches in input to the producer function (excluding slot and entry).

Returns

the first node of the computation graph for which the new quantity is defined.

The old value of the column can be used as an input for the expression. An exception is thrown in case the column to re-define does not already exist.

See DefineSlotEntry() for more information.

Definition at line 610 of file RInterface.hxx.

Reduce()

RResultPtr< T > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Reduce ( F f, std::string_view columnName = "" )

inlineinherited

Execute a user-defined reduce operation on the values of a column.

Template Parameters

F	The type of the reduce callable. Automatically deduced.
T	The type of the column to apply the reduction to. Automatically deduced.

Parameters

[in]	f	A callable with signature T(T,T)
[in]	columnName	The column to be reduced. If omitted, the first default column is used instead.

Returns

the reduced quantity wrapped in a ROOT::RDF:RResultPtr.

A reduction takes two values of a column and merges them into one (e.g. by summing them, taking the maximum, etc). This action performs the specified reduction operation on all processed column values, returning a single value of the same type. The callable f must satisfy the general requirements of a processing function besides having signature T(T,T) where T is the type of column columnName.

The returned reduced value of each thread (e.g. the initial value of a sum) is initialized to a default-constructed T object. This is commonly expected to be the neutral/identity element for the specific reduction operation f (e.g. 0 for a sum, 1 for a product). If a default-constructed T does not satisfy this requirement, users should explicitly specify an initialization value for T by calling the appropriate Reduce overload.

Example usage:

auto sumOfIntCol = d.Reduce([](int x, int y) { return x + y; }, "intCol");

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Definition at line 1862 of file RInterface.hxx.

Report()

RResultPtr< RCutFlowReport > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Report ( )

inlineinherited

Gather filtering statistics.

Returns

the resulting RCutFlowReport instance wrapped in a RResultPtr.

Calling Report on the main RDataFrame object gathers stats for all named filters in the call graph. Calling this method on a stored chain state (i.e. a graph node different from the first) gathers the stats for all named filters in the chain section between the original RDataFrame and that node (included). Stats are gathered in the same order as the named filters have been added to the graph. A RResultPtr<RCutFlowReport> is returned to allow inspection of the effects cuts had.

This action is lazy: upon invocation of this method the calculation is booked but not executed. See RResultPtr documentation.

Example usage:

auto filtered = d.Filter(cut1, {"b1"}, "Cut1").Filter(cut2, {"b2"}, "Cut2");
auto cutReport = filtered3.Report();
cutReport->Print();

Definition at line 3431 of file RInterface.hxx.

SanityChecksForVary()

template<typename RetType>

void ROOT::RDF::RInterfaceBase::SanityChecksForVary ( const std::vector< std::string > & colNames, const std::vector< std::string > & variationTags, std::string_view variationName )

inlineprotectedinherited

Definition at line 66 of file RInterfaceBase.hxx.

Snapshot()

RResultPtr< RInterface< RLoopManager > > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Snapshot ( std::string_view treename, std::string_view filename, const ColumnNames_t & columnList, const RSnapshotOptions & options = RSnapshotOptions() )

inlineinherited

Definition at line 1320 of file RInterface.hxx.

Stats()

RResultPtr< TStatistic > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Stats ( std::string_view value = "" )

inlineinherited

Return a TStatistic object, filled once per event (lazy action).

Template Parameters

V	The type of the value column

Parameters

[in]

value

The name of the column with the values to fill the statistics with.

Returns

the filled TStatistic object wrapped in a RResultPtr.

Example usage:

// Deduce column type (this invocation needs jitting internally)
auto stats0 = myDf.Stats("values");
// Explicit column type
auto stats1 = myDf.Stats<float>("values");

Definition at line 3199 of file RInterface.hxx.

StdDev()

RResultPtr< double > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::StdDev ( std::string_view columnName = "" )

inlineinherited

Return the unbiased standard deviation of processed column values (lazy action).

Template Parameters

T	The type of the branch/column.

Parameters

[in]

columnName

The name of the branch/column to be treated.

Returns

the standard deviation value of the selected column wrapped in a RResultPtr.

If T is not specified, RDataFrame will infer it from the data and just-in-time compile the correct template specialization of this method.

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Deduce column type (this invocation needs jitting internally)
auto stdDev0 = myDf.StdDev("values");
// Explicit column type
auto stdDev1 = myDf.StdDev<double>("values");

Definition at line 3366 of file RInterface.hxx.

Sum()

RResultPtr< RDFDetail::SumReturnType_t< T > > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Sum ( std::string_view columnName = "", const RDFDetail::SumReturnType_t< T > & initValue = RDFDetail::SumReturnType_t<T>{} )

inlineinherited

Return the sum of processed column values (lazy action).

Template Parameters

T	The type of the branch/column.

Parameters

[in]	columnName	The name of the branch/column.
[in]	initValue	Optional initial value for the sum. If not present, the column values must be default-constructible.

Returns

the sum of the selected column wrapped in a RResultPtr.

If T is not specified, RDataFrame will infer it from the data and just-in-time compile the correct template specialization of this method. If the type of the column is inferred, the return type is double, the type of the column otherwise.

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Example usage:

// Deduce column type (this invocation needs jitting internally)
auto sum0 = myDf.Sum("values");
// Explicit column type
auto sum1 = myDf.Sum<double>("values");

Definition at line 3398 of file RInterface.hxx.

Take()

RResultPtr< COLL > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Take ( std::string_view column = "" )

inlineinherited

Return a collection of values of a column (lazy action, returns a std::vector by default).

Template Parameters

T	The type of the column.
COLL	The type of collection used to store the values.

Parameters

[in]

column

The name of the column to collect the values of.

Returns

the content of the selected column wrapped in a RResultPtr.

The collection type to be specified for C-style array columns is RVec<T>: in this case the returned collection is a std::vector<RVec<T>>.

Example usage:

// In this case intCol is a std::vector<int>
auto intCol = rdf.Take<int>("integerColumn");
// Same content as above but in this case taken as a RVec<int>
auto intColAsRVec = rdf.Take<int, RVec<int>>("integerColumn");
// In this case intCol is a std::vector<RVec<int>>, a collection of collections
auto cArrayIntCol = rdf.Take<RVec<int>>("cArrayInt");

This action is lazy: upon invocation of this method the calculation is booked but not executed. Also see RResultPtr.

Definition at line 1935 of file RInterface.hxx.

Vary()

RInterface< RDFDetail::RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::Vary ( std::string_view colName, F && expression, const ColumnNames_t & inputColumns, const std::vector< std::string > & variationTags, std::string_view variationName = "" )

inlineinherited

Register systematic variations for a single existing column using custom variation tags.

Parameters

[in]	colName	name of the column for which varied values are provided.
[in]	expression	a callable that evaluates the varied values for the specified columns. The callable can take any column values as input, similarly to what happens during Filter and Define calls. It must return an RVec of varied values, one for each variation tag, in the same order as the tags.
[in]	inputColumns	the names of the columns to be passed to the callable.
[in]	variationTags	names for each of the varied values, e.g. "up" and "down".
[in]	variationName	a generic name for this set of varied values, e.g. "ptvariation".

Vary provides a natural and flexible syntax to define systematic variations that automatically propagate to Filters, Defines and results. RDataFrame usage of columns with attached variations does not change, but for results that depend on any varied quantity, a map/dictionary of varied results can be produced with ROOT::RDF::Experimental::VariationsFor (see the example below).

The dictionary will contain a "nominal" value (accessed with the "nominal" key) for the unchanged result, and values for each of the systematic variations that affected the result (via upstream Filters or via direct or indirect dependencies of the column values on some registered variations). The keys will be a composition of variation names and tags, e.g. "pt:up" and "pt:down" for the example below.

In the following example we add up/down variations of pt and fill a histogram with a quantity that depends on pt. We automatically obtain three histograms in output ("nominal", "pt:up" and "pt:down"):

auto nominal_hx =
    df.Vary("pt", [] (double pt) { return RVecD{pt*0.9, pt*1.1}; }, {"down", "up"})
      .Filter("pt > k")
      .Define("x", someFunc, {"pt"})
      .Histo1D("x");
 
auto hx = ROOT::RDF::Experimental::VariationsFor(nominal_hx);
hx["nominal"].Draw();
hx["pt:down"].Draw("SAME");
hx["pt:up"].Draw("SAME");

RDataFrame computes all variations as part of a single loop over the data. In particular, this means that I/O and computation of values shared among variations only happen once for all variations. Thus, the event loop run-time typically scales much better than linearly with the number of variations.

RDataFrame lazily computes the varied values required to produce the outputs of VariationsFor(). If VariationsFor() was not called for a result, the computations are only run for the nominal case.

See other overloads for examples when variations are added for multiple existing columns, or when the tags are auto-generated instead of being directly defined.

Definition at line 870 of file RInterface.hxx.

VaryImpl()

RInterface< RDFDetail::RLoopManager > ROOT::RDF::RInterface< RDFDetail::RLoopManager >::VaryImpl ( const std::vector< std::string > & colNames, F && expression, const ColumnNames_t & inputColumns, const std::vector< std::string > & variationTags, std::string_view variationName )

inlineprivateinherited

Definition at line 3825 of file RInterface.hxx.

Member Data Documentation

fColRegister

RDFInternal::RColumnRegister ROOT::RDF::RInterfaceBase::fColRegister

protectedinherited

Contains the columns defined up to this node.

Definition at line 55 of file RInterfaceBase.hxx.

fLoopManager

std::shared_ptr<ROOT::Detail::RDF::RLoopManager> ROOT::RDF::RInterfaceBase::fLoopManager

protectedinherited

< The RLoopManager at the root of this computation graph. Never null.

Definition at line 52 of file RInterfaceBase.hxx.

fProxiedPtr

std::shared_ptr<RDFDetail::RLoopManager> ROOT::RDF::RInterface< RDFDetail::RLoopManager >::fProxiedPtr

privateinherited

Smart pointer to the graph node encapsulated by this RInterface.

Definition at line 142 of file RInterface.hxx.

---

The documentation for this class was generated from the following files:

• tree/dataframe/inc/ROOT/RDataFrame.hxx
• tree/dataframe/src/RDataFrame.cxx