_rdataframe.pyzdoc

Go to the documentation of this file.

\pythondoc ROOT::RDataFrame
 
You can use RDataFrame in Python thanks to the dynamic Python/C++ translation of [PyROOT](https://root.cern/manual/python). In general, the interface
is the same as for C++, a simple example follows.
 
~~~{.py}
df = ROOT.RDataFrame("myTree", "myFile.root")
sum = df.Filter("x > 10").Sum("y")
print(sum.GetValue())
~~~
 
### User code in the RDataFrame workflow
 
#### C++ code
 
In the simple example that was shown above, a C++ expression is passed to the Filter() operation as a string
(`"x > 0"`), even if we call the method from Python. Indeed, under the hood, the analysis computations run in
C++, while Python is just the interface language.
 
To perform more complex operations that don't fit into a simple expression string, you can just-in-time compile
C++ functions - via the C++ interpreter cling - and use those functions in an expression. See the following
snippet for an example:
 
~~~{.py}
# JIT a C++ function from Python
ROOT.gInterpreter.Declare("""
bool myFilter(float x) {
    return x > 10;
}
""")
 
df = ROOT.RDataFrame("myTree", "myFile.root")
# Use the function in an RDF operation
sum = df.Filter("myFilter(x)").Sum("y")
print(sum.GetValue())
~~~
 
To increase the performance even further, you can also pre-compile a C++ library with full code optimizations
and load the function into the RDataFrame computation as follows.
 
~~~{.py}
ROOT.gSystem.Load("path/to/myLibrary.so") # Library with the myFilter function
ROOT.gInterpreter.Declare('#include "myLibrary.h"') # Header with the declaration of the myFilter function
df = ROOT.RDataFrame("myTree", "myFile.root")
sum = df.Filter("myFilter(x)").Sum("y")
print(sum.GetValue())
~~~
 
A more thorough explanation of how to use C++ code from Python can be found in the [PyROOT manual](https://root.cern/manual/python/#loading-user-libraries-and-just-in-time-compilation-jitting).
 
#### Python code
 
ROOT also offers the option to compile Python functions with fundamental types and arrays thereof using [Numba](https://numba.pydata.org/).
Such compiled functions can then be used in a C++ expression provided to RDataFrame.
 
The function to be compiled should be decorated with `ROOT.Numba.Declare`, which allows to specify the parameter and
return types. See the following snippet for a simple example or the full tutorial [here](pyroot004__NumbaDeclare_8py.html).
 
~~~{.py}
@ROOT.Numba.Declare(["float"], "bool")
def myFilter(x):
    return x > 10
 
df = ROOT.RDataFrame("myTree", "myFile.root")
sum = df.Filter("Numba::myFilter(x)").Sum("y")
print(sum.GetValue())
~~~
 
It also works with collections: `RVec` objects of fundamental types can be transparently converted to/from numpy arrays:
 
~~~{.py}
@ROOT.Numba.Declare(['RVec<float>', 'int'], 'RVec<float>')
def pypowarray(numpyvec, pow):
    return numpyvec**pow
 
df.Define('array', 'ROOT::RVecF{1.,2.,3.}')\
  .Define('arraySquared', 'Numba::pypowarray(array, 2)')
~~~
 
Note that this functionality requires the Python packages `numba` and `cffi` to be installed.
 
### Interoperability with NumPy
 
#### Conversion to NumPy arrays
 
Eventually, you probably would like to inspect the content of the RDataFrame or process the data further
with Python libraries. For this purpose, we provide the `AsNumpy()` function, which returns the columns
of your RDataFrame as a dictionary of NumPy arrays. See a few simple examples below or a full tutorial [here](df026__AsNumpyArrays_8py.html).
 
\anchor asnumpy_scalar_columns
##### Scalar columns
If your column contains scalar values of fundamental types (e.g., integers, floats), `AsNumpy()` produces NumPy arrays with the appropriate `dtype`:
~~~{.py}
rdf = ROOT.RDataFrame(10).Define("int_col", "1").Define("float_col", "2.3")
print(rdf.AsNumpy(["int_col", "float_col"]))
# Output: {'int_col': array([...], dtype=int32), 'float_col': array([...], dtype=float64)}
~~~
 
Columns containing non-fundamental types (e.g., objects, strings) will result in NumPy arrays with `dtype=object`.
 
##### Collection Columns
If your column contains collections of fundamental types (e.g., std::vector<int>), `AsNumpy()` produces a NumPy array with `dtype=object` where each 
element is a NumPy array representing the collection for its corresponding entry in the column.
 
If the collection at a certain entry contains values of fundamental types, or if it is a regularly shaped multi-dimensional array of a fundamental type, 
then the numpy array representing the collection for that entry will have the `dtype` associated with the value type of the collection, for example:
~~~{.py}
rdf = rdf.Define("v_col", "std::vector<int>{{1, 2, 3}}")
print(rdf.AsNumpy(["v_col", "int_col", "float_col"]))
# Output: {'v_col': array([array([1, 2, 3], dtype=int32), ...], dtype=object), ...}
~~~
 
If the collection at a certain entry contains values of a non-fundamental type, `AsNumpy()` will fallback on the [default behavior](\ref asnumpy_scalar_columns) and produce a NumPy array with `dtype=object` for that collection.
 
For more complex collection types in your entries, e.g. when every entry has a jagged array value, refer to the section on [interoperability with AwkwardArray](\ref awkward_interop).
 
#### Processing data stored in NumPy arrays
 
In case you have data in NumPy arrays in Python and you want to process the data with ROOT, you can easily
create an RDataFrame using `ROOT.RDF.FromNumpy`. The factory function accepts a dictionary where
the keys are the column names and the values are NumPy arrays, and returns a new RDataFrame with the provided
columns.
 
Only arrays of fundamental types (integers and floating point values) are supported and the arrays must have the same length.
Data is read directly from the arrays: no copies are performed.
 
~~~{.py}
# Read data from NumPy arrays
# The column names in the RDataFrame are taken from the dictionary keys
x, y = numpy.array([1, 2, 3]), numpy.array([4, 5, 6])
df = ROOT.RDF.FromNumpy({"x": x, "y": y})
 
# Use RDataFrame as usual, e.g. write out a ROOT file
df.Define("z", "x + y").Snapshot("tree", "file.root")
~~~
 
 
\anchor awkward_interop
### Interoperability with [AwkwardArray](https://awkward-array.org/doc/main/user-guide/how-to-convert-rdataframe.html)
 
The function for RDataFrame to Awkward conversion is ak.from_rdataframe(). The argument to this function accepts a tuple of strings that are the RDataFrame column names. By default this function returns ak.Array type.
 
~~~{.py}
import awkward as ak
import ROOT
 
array = ak.from_rdataframe(
    df,
    columns=(
        "x",
        "y",
        "z",
    ),
)
~~~
 
The function for Awkward to RDataFrame conversion is ak.to_rdataframe().
 
The argument to this function requires a dictionary: { <column name string> : <awkward array> }. This function always returns an RDataFrame object.
 
The arrays given for each column have to be equal length:
 
~~~{.py}
array_x = ak.Array(
    [
        {"x": [1.1, 1.2, 1.3]},
        {"x": [2.1, 2.2]},
        {"x": [3.1]},
        {"x": [4.1, 4.2, 4.3, 4.4]},
        {"x": [5.1]},
    ]
)
array_y = ak.Array([1, 2, 3, 4, 5])
array_z = ak.Array([[1.1], [2.1, 2.3, 2.4], [3.1], [4.1, 4.2, 4.3], [5.1]])
 
assert len(array_x) == len(array_y) == len(array_z)
 
df = ak.to_rdataframe({"x": array_x, "y": array_y, "z": array_z})
~~~
 
### Construct histogram and profile models from a tuple
 
The Histo1D(), Histo2D(), Histo3D(), Profile1D() and Profile2D() methods return
histograms and profiles, respectively, which can be constructed using a model
argument.
 
In Python, we can specify the arguments for the constructor of such histogram or
profile model with a Python tuple, as shown in the example below:
 
~~~{.py}
# First argument is a tuple with the arguments to construct a TH1D model
h = df.Histo1D(("histName", "histTitle", 64, 0., 128.), "myColumn")
~~~
 
### AsRNode helper function
 
The ROOT::RDF::AsRNode function casts an RDataFrame node to the generic ROOT::RDF::RNode type. From Python, it can be used to pass any RDataFrame node as an argument of a C++ function, as shown below:
 
~~~{.py}
ROOT.gInterpreter.Declare("""
ROOT::RDF::RNode MyTransformation(ROOT::RDF::RNode df) {
    auto myFunc = [](float x){ return -x;};
    return df.Define("y", myFunc, {"x"});
}
""")
 
# Cast the RDataFrame head node
df = ROOT.RDataFrame("myTree", "myFile.root")
df_transformed = ROOT.MyTransformation(ROOT.RDF.AsRNode(df))
 
# ... or any other node
df2 = df.Filter("x > 42")
df2_transformed = ROOT.MyTransformation(ROOT.RDF.AsRNode(df2))
~~~
 
\endpythondoc