This class provides an interface to process a TTree dataset in parallel with multi-process technology.
The possible usages of the Process method are the following:
Process(<dataset>, F func, const std::string& treeName, ULong64_t nToProcess)
: func is executed nToProcess times with argument a TTreeReader&, initialized for the TTree with name treeName, from the dataset <dataset>
. The dataset can be expressed as: Process(<dataset>, TSelector& selector, const std::string& treeName, ULong64_t nToProcess)
: where selector is a TSelector derived class describing the analysis and the other arguments have the same meaning as above.For either set of signatures, the processing function is executed as many times as needed by a pool of fNWorkers workers; the number of workers can be passed to the constructor or set via SetNWorkers. It defaults to the number of cores.
A collection containing the result of each execution is returned.
Note: the user is responsible for the deletion of any object that might be created upon execution of func, returned objects included: ROOT::TTreeProcessorMP never deletes what it returns, it simply forgets it.
Note: that the usage of ROOT::TTreeProcessorMP::Process is indicated only when the task to be executed takes more than a few seconds, otherwise the overhead introduced by Process will outrun the benefits of parallel execution on most machines.
func | a lambda expression, an std::function, a loaded macro, a functor class or a function that takes zero arguments (for the first signature) or one (for the second signature). |
args | a standard container (vector, list, deque), an initializer list or a pointer to a TCollection (TList*, TObjArray*, ...). |
Note: the version of ROOT::TTreeProcessorMP::Process that takes a TFileCollection* as argument incurs in the overhead of copying data from the TCollection to an STL container. Only use it when absolutely necessary.
Note: in cases where the function to be executed takes more than zero/one argument but all are fixed except zero/one, the function can be wrapped in a lambda or via std::bind to give it the right signature.
Note: the user should take care of initializing random seeds differently in each process (e.g. using the process id in the seed). Otherwise several parallel executions might generate the same sequence of pseudo-random numbers.
Methods taking 'F func' return the return type of F. Methods taking a TSelector return a 'TList *' with the selector output list; the output list content is owned by the caller.
See tutorials/multicore/mp102_readNtuplesFillHistosAndFit.C and tutorials/multicore/mp103__processSelector.C .
Definition at line 38 of file TTreeProcessorMP.hxx.
Public Member Functions | |
TTreeProcessorMP (const TTreeProcessorMP &)=delete | |
TTreeProcessorMP (UInt_t nWorkers=0) | |
Class constructor. | |
~TTreeProcessorMP ()=default | |
unsigned | GetNWorkers () const |
TTreeProcessorMP & | operator= (const TTreeProcessorMP &)=delete |
template<class F > | |
auto | Process (const std::string &fileName, F procFunc, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > > |
template<class F > | |
auto | Process (const std::string &fileName, F procFunc, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > > |
TList * | Process (const std::string &fileName, TSelector &selector, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) |
TList * | Process (const std::string &fileName, TSelector &selector, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) |
TSelector-based tree processing: dataset as a single file. | |
template<class F > | |
auto | Process (const std::vector< std::string > &fileNames, F procFunc, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > > |
Process a TTree dataset with a functor: version without entry list. | |
template<class F > | |
auto | Process (const std::vector< std::string > &fileNames, F procFunc, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > > |
Process a TTree dataset with a functor. | |
TList * | Process (const std::vector< std::string > &fileNames, TSelector &selector, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) |
Process a TTree dataset with a selector: version without entry list. | |
TList * | Process (const std::vector< std::string > &fileNames, TSelector &selector, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) |
Process a TTree dataset with a selector. | |
template<class F > | |
auto | Process (TChain &chain, F procFunc, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > > |
template<class F > | |
auto | Process (TChain &files, F procFunc, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > > |
TList * | Process (TChain &files, TSelector &selector, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) |
TList * | Process (TChain &files, TSelector &selector, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) |
TSelector-based tree processing: dataset as a TChain. | |
template<class F > | |
auto | Process (TFileCollection &collection, F procFunc, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > > |
template<class F > | |
auto | Process (TFileCollection &files, F procFunc, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > > |
TList * | Process (TFileCollection &files, TSelector &selector, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) |
TList * | Process (TFileCollection &files, TSelector &selector, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) |
TSelector-based tree processing: dataset as a TFileCollection. | |
template<class F > | |
auto | Process (TTree &tree, F procFunc, TEntryList &entries, ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > > |
template<class F > | |
auto | Process (TTree &tree, F procFunc, ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > > |
TList * | Process (TTree &tree, TSelector &selector, TEntryList &entries, ULong64_t nToProcess=0, ULong64_t jFirst=0) |
TSelector-based tree processing: memory resident tree. | |
TList * | Process (TTree &tree, TSelector &selector, ULong64_t nToProcess=0, ULong64_t jFirst=0) |
void | SetNWorkers (unsigned n) |
Private Types | |
enum class | ETask : unsigned char { kNoTask , kProcByRange , kProcByFile } |
A collection of the types of tasks that TTreeProcessorMP can execute. More... | |
template<typename F , typename... Args> | |
using | InvokeResult_t = ROOT::TypeTraits::InvokeResult_t< F, Args... > |
Private Member Functions | |
template<class T > | |
void | Collect (std::vector< T > &reslist) |
Listen for messages sent by the workers and call the appropriate handler function. | |
void | FixLists (std::vector< TObject * > &lists) |
Fix list of lists before merging (to avoid errors about duplicated objects) | |
template<class T > | |
void | HandlePoolCode (MPCodeBufPair &msg, TSocket *sender, std::vector< T > &reslist) |
Handle message and reply to the worker. | |
void | ReplyToIdle (TSocket *s) |
Reply to a worker who is idle. | |
void | Reset () |
Reset TTreeProcessorMP's state. | |
Private Member Functions inherited from TMPClient | |
TMPClient (const TMPClient &)=delete | |
TMPClient (unsigned nWorkers=0) | |
Class constructor. | |
~TMPClient () | |
Class destructor. | |
template<class T > | |
unsigned | Broadcast (unsigned code, const std::vector< T > &objs) |
Send a message with a different object to each server. | |
template<class T > | |
unsigned | Broadcast (unsigned code, std::initializer_list< T > &objs) |
Send a message with a different object to each server. | |
template<class T > | |
unsigned | Broadcast (unsigned code, T obj, unsigned nMessages=0) |
Send a message containing code and obj to each worker, up to a maximum number of nMessages workers. | |
unsigned | Broadcast (unsigned code, unsigned nMessages=0) |
Send a message with the specified code to at most nMessages workers. | |
void | DeActivate (TSocket *s) |
DeActivate a certain socket. | |
bool | Fork (TMPWorker &server) |
This method forks the ROOT session into fNWorkers children processes. | |
bool | GetIsParent () const |
TMonitor & | GetMonitor () |
unsigned | GetNWorkers () const |
void | HandleMPCode (MPCodeBufPair &msg, TSocket *sender) |
Handle messages containing an EMPCode. | |
TMPClient & | operator= (const TMPClient &)=delete |
void | ReapWorkers () |
Wait on worker processes and remove their pids from fWorkerPids. | |
void | Remove (TSocket *s) |
Remove a certain socket from the monitor. | |
void | SetNWorkers (unsigned n) |
Set the number of workers that will be spawned by the next call to Fork() | |
Private Attributes | |
unsigned | fNProcessed |
number of arguments already passed to the workers | |
unsigned | fNToProcess |
total number of arguments to pass to the workers | |
ETask | fTaskType = ETask::kNoTask |
the kind of task that is being executed, if any | |
#include <ROOT/TTreeProcessorMP.hxx>
|
private |
Definition at line 40 of file TTreeProcessorMP.hxx.
|
strongprivate |
A collection of the types of tasks that TTreeProcessorMP can execute.
It is used to interpret in the right way and properly reply to the messages received (see, for example, TTreeProcessorMP::HandleInput)
Definition at line 182 of file TTreeProcessorMP.hxx.
|
explicit |
Class constructor.
nWorkers is the number of times this ROOT session will be forked, i.e. the number of workers that will be spawned.
Definition at line 90 of file TTreeProcessorMP.cxx.
|
default |
|
delete |
|
private |
Listen for messages sent by the workers and call the appropriate handler function.
TTreeProcessorMP::HandlePoolCode is called on messages with a code < 1000 and TMPClient::HandleMPCode is called on messages with a code >= 1000.
Definition at line 433 of file TTreeProcessorMP.hxx.
|
private |
Fix list of lists before merging (to avoid errors about duplicated objects)
Definition at line 324 of file TTreeProcessorMP.cxx.
|
inline |
Definition at line 166 of file TTreeProcessorMP.hxx.
|
private |
Handle message and reply to the worker.
Definition at line 407 of file TTreeProcessorMP.hxx.
|
delete |
auto ROOT::TTreeProcessorMP::Process | ( | const std::string & | fileName, |
F | procFunc, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) | -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>> |
Definition at line 366 of file TTreeProcessorMP.hxx.
auto ROOT::TTreeProcessorMP::Process | ( | const std::string & | fileName, |
F | procFunc, | ||
TEntryList & | entries, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) | -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>> |
Definition at line 259 of file TTreeProcessorMP.hxx.
TList * ROOT::TTreeProcessorMP::Process | ( | const std::string & | fileName, |
TSelector & | selector, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) |
Definition at line 296 of file TTreeProcessorMP.cxx.
TList * ROOT::TTreeProcessorMP::Process | ( | const std::string & | fileName, |
TSelector & | selector, | ||
TEntryList & | entries, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) |
TSelector-based tree processing: dataset as a single file.
Definition at line 278 of file TTreeProcessorMP.cxx.
auto ROOT::TTreeProcessorMP::Process | ( | const std::vector< std::string > & | fileNames, |
F | procFunc, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) | -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>> |
Process a TTree dataset with a functor: version without entry list.
No TEntryList versions of generic processor.
F | functor returning a pointer to TObject or inheriting classes and taking a TTreeReader& (both enforced at compile-time) |
Dataset definition:
[in] | fileNames | vector of strings with the paths of the files with the TTree to process |
[in] | fileName | string with the path of the files with the TTree to process |
[in] | collection | TFileCollection with the files with the TTree to process |
[in] | chain | TChain with the files with the TTree to process |
[in] | tree | TTree to process |
[in] | treeName | Name of the TTree to process |
[in] | nToProcess | Number of entries to process (0 means all) |
[in] | jFirst | First entry to process (0 means the first of the first file) |
Definition at line 356 of file TTreeProcessorMP.hxx.
auto ROOT::TTreeProcessorMP::Process | ( | const std::vector< std::string > & | fileNames, |
F | procFunc, | ||
TEntryList & | entries, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) | -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>> |
Process a TTree dataset with a functor.
F | functor returning a pointer to TObject or inheriting classes and taking a TTreeReader& (both enforced at compile-time) |
Dataset definition:
[in] | fileNames | vector of strings with the paths of the files with the TTree to process |
[in] | fileName | string with the path of the files with the TTree to process |
[in] | collection | TFileCollection with the files with the TTree to process |
[in] | chain | TChain with the files with the TTree to process |
[in] | tree | TTree to process |
[in] | entries | TEntryList to filter the dataset |
[in] | treeName | Name of the TTree to process |
[in] | nToProcess | Number of entries to process (0 means all) |
[in] | jFirst | First entry to process (0 means the first of the first file) |
Definition at line 192 of file TTreeProcessorMP.hxx.
TList * ROOT::TTreeProcessorMP::Process | ( | const std::vector< std::string > & | fileNames, |
TSelector & | selector, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) |
Process a TTree dataset with a selector: version without entry list.
No TEntryList versions of selector processor.
Dataset definition:
[in] | fileNames | vector of strings with the paths of the files with the TTree to process |
[in] | fileName | string with the path of the files with the TTree to process |
[in] | collection | TFileCollection with the files with the TTree to process |
[in] | chain | TChain with the files with the TTree to process |
[in] | tree | TTree to process |
[in] | selector | Instance of TSelector to be applied to the dataset |
[in] | treeName | Name of the TTree to process |
[in] | nToProcess | Number of entries to process (0 means all) |
[in] | jFirst | First entry to process (0 means the first of the first file) |
Definition at line 289 of file TTreeProcessorMP.cxx.
TList * ROOT::TTreeProcessorMP::Process | ( | const std::vector< std::string > & | fileNames, |
TSelector & | selector, | ||
TEntryList & | entries, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) |
Process a TTree dataset with a selector.
TSelector-based tree processing: dataset as a vector of files.
Dataset definition:
[in] | fileNames | vector of strings with the paths of the files with the TTree to process |
[in] | fileName | string with the path of the files with the TTree to process |
[in] | collection | TFileCollection with the files with the TTree to process |
[in] | chain | TChain with the files with the TTree to process |
[in] | tree | TTree to process |
[in] | selector | Instance of TSelector to be applied to the dataset |
[in] | entries | TEntryList to filter the dataset |
[in] | treeName | Name of the TTree to process |
[in] | nToProcess | Number of entries to process (0 means all) |
[in] | jFirst | First entry to process (0 means the first of the first file) |
Definition at line 160 of file TTreeProcessorMP.cxx.
auto ROOT::TTreeProcessorMP::Process | ( | TChain & | chain, |
F | procFunc, | ||
TEntryList & | entries, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) | -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>> |
Definition at line 283 of file TTreeProcessorMP.hxx.
auto ROOT::TTreeProcessorMP::Process | ( | TChain & | files, |
F | procFunc, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) | -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>> |
Definition at line 386 of file TTreeProcessorMP.hxx.
TList * ROOT::TTreeProcessorMP::Process | ( | TChain & | files, |
TSelector & | selector, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) |
Definition at line 310 of file TTreeProcessorMP.cxx.
TList * ROOT::TTreeProcessorMP::Process | ( | TChain & | files, |
TSelector & | selector, | ||
TEntryList & | entries, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) |
TSelector-based tree processing: dataset as a TChain.
Definition at line 264 of file TTreeProcessorMP.cxx.
auto ROOT::TTreeProcessorMP::Process | ( | TFileCollection & | collection, |
F | procFunc, | ||
TEntryList & | entries, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) | -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>> |
Definition at line 269 of file TTreeProcessorMP.hxx.
auto ROOT::TTreeProcessorMP::Process | ( | TFileCollection & | files, |
F | procFunc, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) | -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>> |
Definition at line 376 of file TTreeProcessorMP.hxx.
TList * ROOT::TTreeProcessorMP::Process | ( | TFileCollection & | files, |
TSelector & | selector, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) |
Definition at line 303 of file TTreeProcessorMP.cxx.
TList * ROOT::TTreeProcessorMP::Process | ( | TFileCollection & | files, |
TSelector & | selector, | ||
TEntryList & | entries, | ||
const std::string & | treeName = "" , |
||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) |
TSelector-based tree processing: dataset as a TFileCollection.
Definition at line 250 of file TTreeProcessorMP.cxx.
auto ROOT::TTreeProcessorMP::Process | ( | TTree & | tree, |
F | procFunc, | ||
TEntryList & | entries, | ||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) | -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>> |
Definition at line 298 of file TTreeProcessorMP.hxx.
auto ROOT::TTreeProcessorMP::Process | ( | TTree & | tree, |
F | procFunc, | ||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) | -> InvokeResult_t<F, std::reference_wrapper<TTreeReader>> |
Definition at line 396 of file TTreeProcessorMP.hxx.
TList * ROOT::TTreeProcessorMP::Process | ( | TTree & | tree, |
TSelector & | selector, | ||
TEntryList & | entries, | ||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) |
TSelector-based tree processing: memory resident tree.
Definition at line 97 of file TTreeProcessorMP.cxx.
TList * ROOT::TTreeProcessorMP::Process | ( | TTree & | tree, |
TSelector & | selector, | ||
ULong64_t | nToProcess = 0 , |
||
ULong64_t | jFirst = 0 |
||
) |
Definition at line 317 of file TTreeProcessorMP.cxx.
|
private |
Reply to a worker who is idle.
If still events to process, tell the worker. Otherwise ask for a result
Definition at line 351 of file TTreeProcessorMP.cxx.
|
private |
Reset TTreeProcessorMP's state.
Definition at line 340 of file TTreeProcessorMP.cxx.
|
inline |
Definition at line 165 of file TTreeProcessorMP.hxx.
|
private |
number of arguments already passed to the workers
Definition at line 176 of file TTreeProcessorMP.hxx.
|
private |
total number of arguments to pass to the workers
Definition at line 177 of file TTreeProcessorMP.hxx.
|
private |
the kind of task that is being executed, if any
Definition at line 188 of file TTreeProcessorMP.hxx.