Logo ROOT  
Reference Guide
 
Loading...
Searching...
No Matches
ROOT::TTreeProcessorMP Class Reference

This class provides an interface to process a TTree dataset in parallel with multi-process technology.

ROOT::TTreeProcessorMP::Process

The possible usages of the Process method are the following:

  • Process(<dataset>, F func, const std::string& treeName, ULong64_t nToProcess): func is executed nToProcess times with argument a TTreeReader&, initialized for the TTree with name treeName, from the dataset <dataset>. The dataset can be expressed as:
    const std::string& fileName -> single file name
    const std::vector<std::string>& fileNames -> vector of file names
    TFileCollection& files -> collection of TFileInfo objects
    TChain& files -> TChain with the file paths
    TTree& tree -> Reference to an existing TTree object
    char name[80]
    Definition TGX11.cxx:110
    A chain is a collection of files containing TTree objects.
    Definition TChain.h:33
    Class that contains a list of TFileInfo's and accumulated meta data information about its entries.
    Class describing a generic file including meta information.
    Definition TFileInfo.h:39
    A TTree represents a columnar dataset.
    Definition TTree.h:79
    For legacy, the following signature is also supported:
  • Process(<dataset>, TSelector& selector, const std::string& treeName, ULong64_t nToProcess): where selector is a TSelector derived class describing the analysis and the other arguments have the same meaning as above.

For either set of signatures, the processing function is executed as many times as needed by a pool of fNWorkers workers; the number of workers can be passed to the constructor or set via SetNWorkers. It defaults to the number of cores.
A collection containing the result of each execution is returned.
Note: the user is responsible for the deletion of any object that might be created upon execution of func, returned objects included: ROOT::TTreeProcessorMP never deletes what it returns, it simply forgets it.
Note: that the usage of ROOT::TTreeProcessorMP::Process is indicated only when the task to be executed takes more than a few seconds, otherwise the overhead introduced by Process will outrun the benefits of parallel execution on most machines.

Parameters
funca lambda expression, an std::function, a loaded macro, a functor class or a function that takes zero arguments (for the first signature) or one (for the second signature).
argsa standard container (vector, list, deque), an initializer list or a pointer to a TCollection (TList*, TObjArray*, ...).

Note: the version of ROOT::TTreeProcessorMP::Process that takes a TFileCollection* as argument incurs in the overhead of copying data from the TCollection to an STL container. Only use it when absolutely necessary.
Note: in cases where the function to be executed takes more than zero/one argument but all are fixed except zero/one, the function can be wrapped in a lambda or via std::bind to give it the right signature.
Note: the user should take care of initializing random seeds differently in each process (e.g. using the process id in the seed). Otherwise several parallel executions might generate the same sequence of pseudo-random numbers.

Return value:

Methods taking 'F func' return the return type of F. Methods taking a TSelector return a 'TList *' with the selector output list; the output list content is owned by the caller.

Examples:

See tutorials/multicore/mp102_readNtuplesFillHistosAndFit.C and tutorials/multicore/mp103__processSelector.C .

Definition at line 38 of file TTreeProcessorMP.hxx.

Public Member Functions

 TTreeProcessorMP (const TTreeProcessorMP &)=delete
 
 TTreeProcessorMP (UInt_t nWorkers=0)
 Class constructor.
 
 ~TTreeProcessorMP ()=default
 
unsigned GetNWorkers () const
 
TTreeProcessorMPoperator= (const TTreeProcessorMP &)=delete
 
template<class F >
auto Process (const std::string &fileName, F procFunc, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
 
template<class F >
auto Process (const std::string &fileName, F procFunc, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
 
TListProcess (const std::string &fileName, TSelector &selector, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
 
TListProcess (const std::string &fileName, TSelector &selector, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
 TSelector-based tree processing: dataset as a single file.
 
template<class F >
auto Process (const std::vector< std::string > &fileNames, F procFunc, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
 Process a TTree dataset with a functor: version without entry list.
 
template<class F >
auto Process (const std::vector< std::string > &fileNames, F procFunc, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
 Process a TTree dataset with a functor.
 
TListProcess (const std::vector< std::string > &fileNames, TSelector &selector, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
 Process a TTree dataset with a selector: version without entry list.
 
TListProcess (const std::vector< std::string > &fileNames, TSelector &selector, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
 Process a TTree dataset with a selector.
 
template<class F >
auto Process (TChain &chain, F procFunc, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
 
template<class F >
auto Process (TChain &files, F procFunc, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
 
TListProcess (TChain &files, TSelector &selector, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
 
TListProcess (TChain &files, TSelector &selector, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
 TSelector-based tree processing: dataset as a TChain.
 
template<class F >
auto Process (TFileCollection &collection, F procFunc, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
 
template<class F >
auto Process (TFileCollection &files, F procFunc, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
 
TListProcess (TFileCollection &files, TSelector &selector, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
 
TListProcess (TFileCollection &files, TSelector &selector, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
 TSelector-based tree processing: dataset as a TFileCollection.
 
template<class F >
auto Process (TTree &tree, F procFunc, TEntryList &entries, ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
 
template<class F >
auto Process (TTree &tree, F procFunc, ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
 
TListProcess (TTree &tree, TSelector &selector, TEntryList &entries, ULong64_t nToProcess=0, ULong64_t jFirst=0)
 TSelector-based tree processing: memory resident tree.
 
TListProcess (TTree &tree, TSelector &selector, ULong64_t nToProcess=0, ULong64_t jFirst=0)
 
void SetNWorkers (unsigned n)
 

Private Types

enum class  ETask : unsigned char { kNoTask , kProcByRange , kProcByFile }
 A collection of the types of tasks that TTreeProcessorMP can execute. More...
 
template<typename F , typename... Args>
using InvokeResult_t = ROOT::TypeTraits::InvokeResult_t< F, Args... >
 

Private Member Functions

template<class T >
void Collect (std::vector< T > &reslist)
 Listen for messages sent by the workers and call the appropriate handler function.
 
void FixLists (std::vector< TObject * > &lists)
 Fix list of lists before merging (to avoid errors about duplicated objects)
 
template<class T >
void HandlePoolCode (MPCodeBufPair &msg, TSocket *sender, std::vector< T > &reslist)
 Handle message and reply to the worker.
 
void ReplyToIdle (TSocket *s)
 Reply to a worker who is idle.
 
void Reset ()
 Reset TTreeProcessorMP's state.
 
- Private Member Functions inherited from TMPClient
 TMPClient (const TMPClient &)=delete
 
 TMPClient (unsigned nWorkers=0)
 Class constructor.
 
 ~TMPClient ()
 Class destructor.
 
template<class T >
unsigned Broadcast (unsigned code, const std::vector< T > &objs)
 Send a message with a different object to each server.
 
template<class T >
unsigned Broadcast (unsigned code, std::initializer_list< T > &objs)
 Send a message with a different object to each server.
 
template<class T >
unsigned Broadcast (unsigned code, T obj, unsigned nMessages=0)
 Send a message containing code and obj to each worker, up to a maximum number of nMessages workers.
 
unsigned Broadcast (unsigned code, unsigned nMessages=0)
 Send a message with the specified code to at most nMessages workers.
 
void DeActivate (TSocket *s)
 DeActivate a certain socket.
 
bool Fork (TMPWorker &server)
 This method forks the ROOT session into fNWorkers children processes.
 
bool GetIsParent () const
 
TMonitorGetMonitor ()
 
unsigned GetNWorkers () const
 
void HandleMPCode (MPCodeBufPair &msg, TSocket *sender)
 Handle messages containing an EMPCode.
 
TMPClientoperator= (const TMPClient &)=delete
 
void ReapWorkers ()
 Wait on worker processes and remove their pids from fWorkerPids.
 
void Remove (TSocket *s)
 Remove a certain socket from the monitor.
 
void SetNWorkers (unsigned n)
 Set the number of workers that will be spawned by the next call to Fork()
 

Private Attributes

unsigned fNProcessed
 number of arguments already passed to the workers
 
unsigned fNToProcess
 total number of arguments to pass to the workers
 
ETask fTaskType = ETask::kNoTask
 the kind of task that is being executed, if any
 

#include <ROOT/TTreeProcessorMP.hxx>

Inheritance diagram for ROOT::TTreeProcessorMP:
[legend]

Member Typedef Documentation

◆ InvokeResult_t

template<typename F , typename... Args>
using ROOT::TTreeProcessorMP::InvokeResult_t = ROOT::TypeTraits::InvokeResult_t<F, Args...>
private

Definition at line 40 of file TTreeProcessorMP.hxx.

Member Enumeration Documentation

◆ ETask

enum class ROOT::TTreeProcessorMP::ETask : unsigned char
strongprivate

A collection of the types of tasks that TTreeProcessorMP can execute.

It is used to interpret in the right way and properly reply to the messages received (see, for example, TTreeProcessorMP::HandleInput)

Enumerator
kNoTask 

no task is being executed

kProcByRange 

a Process method is being executed and each worker will process a certain range of each file

kProcByFile 

a Process method is being executed and each worker will process a different file

Definition at line 182 of file TTreeProcessorMP.hxx.

Constructor & Destructor Documentation

◆ TTreeProcessorMP() [1/2]

ROOT::TTreeProcessorMP::TTreeProcessorMP ( UInt_t  nWorkers = 0)
explicit

Class constructor.

nWorkers is the number of times this ROOT session will be forked, i.e. the number of workers that will be spawned.

Definition at line 90 of file TTreeProcessorMP.cxx.

◆ ~TTreeProcessorMP()

ROOT::TTreeProcessorMP::~TTreeProcessorMP ( )
default

◆ TTreeProcessorMP() [2/2]

ROOT::TTreeProcessorMP::TTreeProcessorMP ( const TTreeProcessorMP )
delete

Member Function Documentation

◆ Collect()

template<class T >
void ROOT::TTreeProcessorMP::Collect ( std::vector< T > &  reslist)
private

Listen for messages sent by the workers and call the appropriate handler function.

TTreeProcessorMP::HandlePoolCode is called on messages with a code < 1000 and TMPClient::HandleMPCode is called on messages with a code >= 1000.

Definition at line 433 of file TTreeProcessorMP.hxx.

◆ FixLists()

void ROOT::TTreeProcessorMP::FixLists ( std::vector< TObject * > &  lists)
private

Fix list of lists before merging (to avoid errors about duplicated objects)

Definition at line 324 of file TTreeProcessorMP.cxx.

◆ GetNWorkers()

unsigned ROOT::TTreeProcessorMP::GetNWorkers ( ) const
inline

Definition at line 166 of file TTreeProcessorMP.hxx.

◆ HandlePoolCode()

template<class T >
void ROOT::TTreeProcessorMP::HandlePoolCode ( MPCodeBufPair msg,
TSocket sender,
std::vector< T > &  reslist 
)
private

Handle message and reply to the worker.

Definition at line 407 of file TTreeProcessorMP.hxx.

◆ operator=()

TTreeProcessorMP & ROOT::TTreeProcessorMP::operator= ( const TTreeProcessorMP )
delete

◆ Process() [1/20]

template<class F >
auto ROOT::TTreeProcessorMP::Process ( const std::string &  fileName,
F  procFunc,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
) -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 366 of file TTreeProcessorMP.hxx.

◆ Process() [2/20]

template<class F >
auto ROOT::TTreeProcessorMP::Process ( const std::string &  fileName,
F  procFunc,
TEntryList entries,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
) -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 259 of file TTreeProcessorMP.hxx.

◆ Process() [3/20]

TList * ROOT::TTreeProcessorMP::Process ( const std::string &  fileName,
TSelector selector,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
)

Definition at line 296 of file TTreeProcessorMP.cxx.

◆ Process() [4/20]

TList * ROOT::TTreeProcessorMP::Process ( const std::string &  fileName,
TSelector selector,
TEntryList entries,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
)

TSelector-based tree processing: dataset as a single file.

Definition at line 278 of file TTreeProcessorMP.cxx.

◆ Process() [5/20]

template<class F >
auto ROOT::TTreeProcessorMP::Process ( const std::vector< std::string > &  fileNames,
F  procFunc,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
) -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Process a TTree dataset with a functor: version without entry list.

No TEntryList versions of generic processor.

Template Parameters
Ffunctor returning a pointer to TObject or inheriting classes and taking a TTreeReader& (both enforced at compile-time)

Dataset definition:

Parameters
[in]fileNamesvector of strings with the paths of the files with the TTree to process
[in]fileNamestring with the path of the files with the TTree to process
[in]collectionTFileCollection with the files with the TTree to process
[in]chainTChain with the files with the TTree to process
[in]treeTTree to process
[in]treeNameName of the TTree to process
[in]nToProcessNumber of entries to process (0 means all)
[in]jFirstFirst entry to process (0 means the first of the first file)

Definition at line 356 of file TTreeProcessorMP.hxx.

◆ Process() [6/20]

template<class F >
auto ROOT::TTreeProcessorMP::Process ( const std::vector< std::string > &  fileNames,
F  procFunc,
TEntryList entries,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
) -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Process a TTree dataset with a functor.

Template Parameters
Ffunctor returning a pointer to TObject or inheriting classes and taking a TTreeReader& (both enforced at compile-time)

Dataset definition:

Parameters
[in]fileNamesvector of strings with the paths of the files with the TTree to process
[in]fileNamestring with the path of the files with the TTree to process
[in]collectionTFileCollection with the files with the TTree to process
[in]chainTChain with the files with the TTree to process
[in]treeTTree to process
[in]entriesTEntryList to filter the dataset
[in]treeNameName of the TTree to process
[in]nToProcessNumber of entries to process (0 means all)
[in]jFirstFirst entry to process (0 means the first of the first file)

Definition at line 192 of file TTreeProcessorMP.hxx.

◆ Process() [7/20]

TList * ROOT::TTreeProcessorMP::Process ( const std::vector< std::string > &  fileNames,
TSelector selector,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
)

Process a TTree dataset with a selector: version without entry list.

No TEntryList versions of selector processor.

Dataset definition:

Parameters
[in]fileNamesvector of strings with the paths of the files with the TTree to process
[in]fileNamestring with the path of the files with the TTree to process
[in]collectionTFileCollection with the files with the TTree to process
[in]chainTChain with the files with the TTree to process
[in]treeTTree to process
[in]selectorInstance of TSelector to be applied to the dataset
[in]treeNameName of the TTree to process
[in]nToProcessNumber of entries to process (0 means all)
[in]jFirstFirst entry to process (0 means the first of the first file)

Definition at line 289 of file TTreeProcessorMP.cxx.

◆ Process() [8/20]

TList * ROOT::TTreeProcessorMP::Process ( const std::vector< std::string > &  fileNames,
TSelector selector,
TEntryList entries,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
)

Process a TTree dataset with a selector.

TSelector-based tree processing: dataset as a vector of files.

Dataset definition:

Parameters
[in]fileNamesvector of strings with the paths of the files with the TTree to process
[in]fileNamestring with the path of the files with the TTree to process
[in]collectionTFileCollection with the files with the TTree to process
[in]chainTChain with the files with the TTree to process
[in]treeTTree to process
[in]selectorInstance of TSelector to be applied to the dataset
[in]entriesTEntryList to filter the dataset
[in]treeNameName of the TTree to process
[in]nToProcessNumber of entries to process (0 means all)
[in]jFirstFirst entry to process (0 means the first of the first file)

Definition at line 160 of file TTreeProcessorMP.cxx.

◆ Process() [9/20]

template<class F >
auto ROOT::TTreeProcessorMP::Process ( TChain chain,
F  procFunc,
TEntryList entries,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
) -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 283 of file TTreeProcessorMP.hxx.

◆ Process() [10/20]

template<class F >
auto ROOT::TTreeProcessorMP::Process ( TChain files,
F  procFunc,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
) -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 386 of file TTreeProcessorMP.hxx.

◆ Process() [11/20]

TList * ROOT::TTreeProcessorMP::Process ( TChain files,
TSelector selector,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
)

Definition at line 310 of file TTreeProcessorMP.cxx.

◆ Process() [12/20]

TList * ROOT::TTreeProcessorMP::Process ( TChain files,
TSelector selector,
TEntryList entries,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
)

TSelector-based tree processing: dataset as a TChain.

Definition at line 264 of file TTreeProcessorMP.cxx.

◆ Process() [13/20]

template<class F >
auto ROOT::TTreeProcessorMP::Process ( TFileCollection collection,
F  procFunc,
TEntryList entries,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
) -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 269 of file TTreeProcessorMP.hxx.

◆ Process() [14/20]

template<class F >
auto ROOT::TTreeProcessorMP::Process ( TFileCollection files,
F  procFunc,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
) -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 376 of file TTreeProcessorMP.hxx.

◆ Process() [15/20]

TList * ROOT::TTreeProcessorMP::Process ( TFileCollection files,
TSelector selector,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
)

Definition at line 303 of file TTreeProcessorMP.cxx.

◆ Process() [16/20]

TList * ROOT::TTreeProcessorMP::Process ( TFileCollection files,
TSelector selector,
TEntryList entries,
const std::string &  treeName = "",
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
)

TSelector-based tree processing: dataset as a TFileCollection.

Definition at line 250 of file TTreeProcessorMP.cxx.

◆ Process() [17/20]

template<class F >
auto ROOT::TTreeProcessorMP::Process ( TTree tree,
F  procFunc,
TEntryList entries,
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
) -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 298 of file TTreeProcessorMP.hxx.

◆ Process() [18/20]

template<class F >
auto ROOT::TTreeProcessorMP::Process ( TTree tree,
F  procFunc,
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
) -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 396 of file TTreeProcessorMP.hxx.

◆ Process() [19/20]

TList * ROOT::TTreeProcessorMP::Process ( TTree tree,
TSelector selector,
TEntryList entries,
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
)

TSelector-based tree processing: memory resident tree.

Definition at line 97 of file TTreeProcessorMP.cxx.

◆ Process() [20/20]

TList * ROOT::TTreeProcessorMP::Process ( TTree tree,
TSelector selector,
ULong64_t  nToProcess = 0,
ULong64_t  jFirst = 0 
)

Definition at line 317 of file TTreeProcessorMP.cxx.

◆ ReplyToIdle()

void ROOT::TTreeProcessorMP::ReplyToIdle ( TSocket s)
private

Reply to a worker who is idle.

If still events to process, tell the worker. Otherwise ask for a result

Definition at line 351 of file TTreeProcessorMP.cxx.

◆ Reset()

void ROOT::TTreeProcessorMP::Reset ( )
private

Reset TTreeProcessorMP's state.

Definition at line 340 of file TTreeProcessorMP.cxx.

◆ SetNWorkers()

void ROOT::TTreeProcessorMP::SetNWorkers ( unsigned  n)
inline

Definition at line 165 of file TTreeProcessorMP.hxx.

Member Data Documentation

◆ fNProcessed

unsigned ROOT::TTreeProcessorMP::fNProcessed
private

number of arguments already passed to the workers

Definition at line 176 of file TTreeProcessorMP.hxx.

◆ fNToProcess

unsigned ROOT::TTreeProcessorMP::fNToProcess
private

total number of arguments to pass to the workers

Definition at line 177 of file TTreeProcessorMP.hxx.

◆ fTaskType

ETask ROOT::TTreeProcessorMP::fTaskType = ETask::kNoTask
private

the kind of task that is being executed, if any

Definition at line 188 of file TTreeProcessorMP.hxx.

Libraries for ROOT::TTreeProcessorMP:

The documentation for this class was generated from the following files: