ROOT::TTreeProcessorMP Class Reference

This class provides an interface to process a TTree dataset in parallel with multi-process technology.

ROOT::TTreeProcessorMP::Process

The possible usages of the Process method are the following:

• Process(<dataset>, F func, const std::string& treeName, ULong64_t nToProcess): func is executed nToProcess times with argument a TTreeReader&, initialized for the TTree with name treeName, from the dataset <dataset>. The dataset can be expressed as:

  const std::string& fileName  -> single file name
  const std::vector<std::string>& fileNames -> vector of file names
  TFileCollection& files       -> collection of TFileInfo objects
  TChain& files                -> TChain with the file paths
  TTree& tree                  -> Reference to an existing TTree object

  For legacy, the following signature is also supported:
• Process(<dataset>, TSelector& selector, const std::string& treeName, ULong64_t nToProcess): where selector is a TSelector derived class describing the analysis and the other arguments have the same meaning as above.

For either set of signatures, the processing function is executed as many times as needed by a pool of fNWorkers workers; the number of workers can be passed to the constructor or set via SetNWorkers. It defaults to the number of cores.
A collection containing the result of each execution is returned.
Note: the user is responsible for the deletion of any object that might be created upon execution of func, returned objects included: ROOT::TTreeProcessorMP never deletes what it returns, it simply forgets it.
Note: that the usage of ROOT::TTreeProcessorMP::Process is indicated only when the task to be executed takes more than a few seconds, otherwise the overhead introduced by Process will outrun the benefits of parallel execution on most machines.

Parameters

func	a lambda expression, an std::function, a loaded macro, a functor class or a function that takes zero arguments (for the first signature) or one (for the second signature).
args	a standard container (vector, list, deque), an initializer list or a pointer to a TCollection (TList*, TObjArray*, ...).

Note: the version of ROOT::TTreeProcessorMP::Process that takes a TFileCollection* as argument incurs in the overhead of copying data from the TCollection to an STL container. Only use it when absolutely necessary.
Note: in cases where the function to be executed takes more than zero/one argument but all are fixed except zero/one, the function can be wrapped in a lambda or via std::bind to give it the right signature.
Note: the user should take care of initializing random seeds differently in each process (e.g. using the process id in the seed). Otherwise several parallel executions might generate the same sequence of pseudo-random numbers.

Return value:

Methods taking 'F func' return the return type of F. Methods taking a TSelector return a 'TList *' with the selector output list; the output list content is owned by the caller.

Examples:

See tutorials/multicore/mp102_readNtuplesFillHistosAndFit.C and tutorials/multicore/mp103__processSelector.C .

Definition at line 38 of file TTreeProcessorMP.hxx.

Public Member Functions
	TTreeProcessorMP (const TTreeProcessorMP &)=delete
	TTreeProcessorMP (UInt_t nWorkers=0)
	Class constructor.
	~TTreeProcessorMP ()=default
unsigned	GetNWorkers () const
TTreeProcessorMP &	operator= (const TTreeProcessorMP &)=delete
template<class F >
auto	Process (const std::string &fileName, F procFunc, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
template<class F >
auto	Process (const std::string &fileName, F procFunc, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
TList *	Process (const std::string &fileName, TSelector &selector, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
TList *	Process (const std::string &fileName, TSelector &selector, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
	TSelector-based tree processing: dataset as a single file.
template<class F >
auto	Process (const std::vector< std::string > &fileNames, F procFunc, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
	Process a TTree dataset with a functor: version without entry list.
template<class F >
auto	Process (const std::vector< std::string > &fileNames, F procFunc, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
	Process a TTree dataset with a functor.
TList *	Process (const std::vector< std::string > &fileNames, TSelector &selector, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
	Process a TTree dataset with a selector: version without entry list.
TList *	Process (const std::vector< std::string > &fileNames, TSelector &selector, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
	Process a TTree dataset with a selector.
template<class F >
auto	Process (TChain &chain, F procFunc, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
template<class F >
auto	Process (TChain &files, F procFunc, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
TList *	Process (TChain &files, TSelector &selector, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
TList *	Process (TChain &files, TSelector &selector, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
	TSelector-based tree processing: dataset as a TChain.
template<class F >
auto	Process (TFileCollection &collection, F procFunc, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
template<class F >
auto	Process (TFileCollection &files, F procFunc, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
TList *	Process (TFileCollection &files, TSelector &selector, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
TList *	Process (TFileCollection &files, TSelector &selector, TEntryList &entries, const std::string &treeName="", ULong64_t nToProcess=0, ULong64_t jFirst=0)
	TSelector-based tree processing: dataset as a TFileCollection.
template<class F >
auto	Process (TTree &tree, F procFunc, TEntryList &entries, ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
template<class F >
auto	Process (TTree &tree, F procFunc, ULong64_t nToProcess=0, ULong64_t jFirst=0) -> InvokeResult_t< F, std::reference_wrapper< TTreeReader > >
TList *	Process (TTree &tree, TSelector &selector, TEntryList &entries, ULong64_t nToProcess=0, ULong64_t jFirst=0)
	TSelector-based tree processing: memory resident tree.
TList *	Process (TTree &tree, TSelector &selector, ULong64_t nToProcess=0, ULong64_t jFirst=0)
void	SetNWorkers (unsigned n)

Private Types
enum class	ETask : unsigned char { kNoTask , kProcByRange , kProcByFile }
	A collection of the types of tasks that TTreeProcessorMP can execute. More...
template<typename F , typename... Args>
using	InvokeResult_t = ROOT::TypeTraits::InvokeResult_t< F, Args... >

Private Member Functions
template<class T >
void	Collect (std::vector< T > &reslist)
	Listen for messages sent by the workers and call the appropriate handler function.
void	FixLists (std::vector< TObject * > &lists)
	Fix list of lists before merging (to avoid errors about duplicated objects)
template<class T >
void	HandlePoolCode (MPCodeBufPair &msg, TSocket *sender, std::vector< T > &reslist)
	Handle message and reply to the worker.
void	ReplyToIdle (TSocket *s)
	Reply to a worker who is idle.
void	Reset ()
	Reset TTreeProcessorMP's state.
Private Member Functions inherited from TMPClient
	TMPClient (const TMPClient &)=delete
	TMPClient (unsigned nWorkers=0)
	Class constructor.
	~TMPClient ()
	Class destructor.
template<class T >
unsigned	Broadcast (unsigned code, const std::vector< T > &objs)
	Send a message with a different object to each server.
template<class T >
unsigned	Broadcast (unsigned code, std::initializer_list< T > &objs)
	Send a message with a different object to each server.
template<class T >
unsigned	Broadcast (unsigned code, T obj, unsigned nMessages=0)
	Send a message containing code and obj to each worker, up to a maximum number of nMessages workers.
unsigned	Broadcast (unsigned code, unsigned nMessages=0)
	Send a message with the specified code to at most nMessages workers.
void	DeActivate (TSocket *s)
	DeActivate a certain socket.
bool	Fork (TMPWorker &server)
	This method forks the ROOT session into fNWorkers children processes.
bool	GetIsParent () const
TMonitor &	GetMonitor ()
unsigned	GetNWorkers () const
void	HandleMPCode (MPCodeBufPair &msg, TSocket *sender)
	Handle messages containing an EMPCode.
TMPClient &	operator= (const TMPClient &)=delete
void	ReapWorkers ()
	Wait on worker processes and remove their pids from fWorkerPids.
void	Remove (TSocket *s)
	Remove a certain socket from the monitor.
void	SetNWorkers (unsigned n)
	Set the number of workers that will be spawned by the next call to Fork()

Private Attributes
unsigned	fNProcessed
	number of arguments already passed to the workers
unsigned	fNToProcess
	total number of arguments to pass to the workers
ETask	fTaskType = ETask::kNoTask
	the kind of task that is being executed, if any

#include <ROOT/TTreeProcessorMP.hxx>

Inheritance diagram for ROOT::TTreeProcessorMP:

Member Typedef Documentation

InvokeResult_t

template<typename F , typename... Args>

using ROOT::TTreeProcessorMP::InvokeResult_t = ROOT::TypeTraits::InvokeResult_t<F, Args...>

private

Definition at line 40 of file TTreeProcessorMP.hxx.

Member Enumeration Documentation

ETask

enum class ROOT::TTreeProcessorMP::ETask : unsigned char

strongprivate

A collection of the types of tasks that TTreeProcessorMP can execute.

It is used to interpret in the right way and properly reply to the messages received (see, for example, TTreeProcessorMP::HandleInput)

Enumerator
kNoTask	no task is being executed
kProcByRange	a Process method is being executed and each worker will process a certain range of each file
kProcByFile	a Process method is being executed and each worker will process a different file

Definition at line 182 of file TTreeProcessorMP.hxx.

Constructor & Destructor Documentation

TTreeProcessorMP() [1/2]

ROOT::TTreeProcessorMP::TTreeProcessorMP ( UInt_t  nWorkers = 0 )

explicit

Class constructor.

nWorkers is the number of times this ROOT session will be forked, i.e. the number of workers that will be spawned.

Definition at line 90 of file TTreeProcessorMP.cxx.

~TTreeProcessorMP()

ROOT::TTreeProcessorMP::~TTreeProcessorMP ( )

default

TTreeProcessorMP() [2/2]

ROOT::TTreeProcessorMP::TTreeProcessorMP ( const TTreeProcessorMP &  )

delete

Member Function Documentation

Collect()

template<class T >

void ROOT::TTreeProcessorMP::Collect ( std::vector< T > &  reslist )

private

Listen for messages sent by the workers and call the appropriate handler function.

TTreeProcessorMP::HandlePoolCode is called on messages with a code < 1000 and TMPClient::HandleMPCode is called on messages with a code >= 1000.

Definition at line 433 of file TTreeProcessorMP.hxx.

FixLists()

void ROOT::TTreeProcessorMP::FixLists ( std::vector< TObject * > &  lists )

private

Fix list of lists before merging (to avoid errors about duplicated objects)

Definition at line 324 of file TTreeProcessorMP.cxx.

GetNWorkers()

unsigned ROOT::TTreeProcessorMP::GetNWorkers ( ) const

inline

Definition at line 166 of file TTreeProcessorMP.hxx.

HandlePoolCode()

template<class T >

void ROOT::TTreeProcessorMP::HandlePoolCode ( MPCodeBufPair &  msg, TSocket *  sender, std::vector< T > &  reslist  )

private

Handle message and reply to the worker.

Definition at line 407 of file TTreeProcessorMP.hxx.

operator=()

TTreeProcessorMP & ROOT::TTreeProcessorMP::operator= ( const TTreeProcessorMP &  )

delete

Process() [1/20]

template<class F >

auto ROOT::TTreeProcessorMP::Process	(	const std::string &	fileName,
		F	procFunc,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)		-> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 366 of file TTreeProcessorMP.hxx.

Process() [2/20]

template<class F >

auto ROOT::TTreeProcessorMP::Process	(	const std::string &	fileName,
		F	procFunc,
		TEntryList &	entries,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)		-> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 259 of file TTreeProcessorMP.hxx.

Process() [3/20]

TList * ROOT::TTreeProcessorMP::Process	(	const std::string &	fileName,
		TSelector &	selector,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)

Definition at line 296 of file TTreeProcessorMP.cxx.

Process() [4/20]

TList * ROOT::TTreeProcessorMP::Process	(	const std::string &	fileName,
		TSelector &	selector,
		TEntryList &	entries,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)

TSelector-based tree processing: dataset as a single file.

Definition at line 278 of file TTreeProcessorMP.cxx.

Process() [5/20]

template<class F >

auto ROOT::TTreeProcessorMP::Process	(	const std::vector< std::string > &	fileNames,
		F	procFunc,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)		-> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Process a TTree dataset with a functor: version without entry list.

No TEntryList versions of generic processor.

Template Parameters

F	functor returning a pointer to TObject or inheriting classes and taking a TTreeReader& (both enforced at compile-time)

Dataset definition:

Parameters

[in]	fileNames	vector of strings with the paths of the files with the TTree to process
[in]	fileName	string with the path of the files with the TTree to process
[in]	collection	TFileCollection with the files with the TTree to process
[in]	chain	TChain with the files with the TTree to process
[in]	tree	TTree to process
[in]	treeName	Name of the TTree to process
[in]	nToProcess	Number of entries to process (0 means all)
[in]	jFirst	First entry to process (0 means the first of the first file)

Definition at line 356 of file TTreeProcessorMP.hxx.

Process() [6/20]

template<class F >

auto ROOT::TTreeProcessorMP::Process	(	const std::vector< std::string > &	fileNames,
		F	procFunc,
		TEntryList &	entries,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)		-> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Process a TTree dataset with a functor.

Template Parameters

F	functor returning a pointer to TObject or inheriting classes and taking a TTreeReader& (both enforced at compile-time)

Dataset definition:

Parameters

[in]	fileNames	vector of strings with the paths of the files with the TTree to process
[in]	fileName	string with the path of the files with the TTree to process
[in]	collection	TFileCollection with the files with the TTree to process
[in]	chain	TChain with the files with the TTree to process
[in]	tree	TTree to process
[in]	entries	TEntryList to filter the dataset
[in]	treeName	Name of the TTree to process
[in]	nToProcess	Number of entries to process (0 means all)
[in]	jFirst	First entry to process (0 means the first of the first file)

Definition at line 192 of file TTreeProcessorMP.hxx.

Process() [7/20]

TList * ROOT::TTreeProcessorMP::Process	(	const std::vector< std::string > &	fileNames,
		TSelector &	selector,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)

Process a TTree dataset with a selector: version without entry list.

No TEntryList versions of selector processor.

Dataset definition:

Parameters

[in]	fileNames	vector of strings with the paths of the files with the TTree to process
[in]	fileName	string with the path of the files with the TTree to process
[in]	collection	TFileCollection with the files with the TTree to process
[in]	chain	TChain with the files with the TTree to process
[in]	tree	TTree to process
[in]	selector	Instance of TSelector to be applied to the dataset
[in]	treeName	Name of the TTree to process
[in]	nToProcess	Number of entries to process (0 means all)
[in]	jFirst	First entry to process (0 means the first of the first file)

Definition at line 289 of file TTreeProcessorMP.cxx.

Process() [8/20]

TList * ROOT::TTreeProcessorMP::Process	(	const std::vector< std::string > &	fileNames,
		TSelector &	selector,
		TEntryList &	entries,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)

Process a TTree dataset with a selector.

TSelector-based tree processing: dataset as a vector of files.

Dataset definition:

Parameters

[in]	fileNames	vector of strings with the paths of the files with the TTree to process
[in]	fileName	string with the path of the files with the TTree to process
[in]	collection	TFileCollection with the files with the TTree to process
[in]	chain	TChain with the files with the TTree to process
[in]	tree	TTree to process
[in]	selector	Instance of TSelector to be applied to the dataset
[in]	entries	TEntryList to filter the dataset
[in]	treeName	Name of the TTree to process
[in]	nToProcess	Number of entries to process (0 means all)
[in]	jFirst	First entry to process (0 means the first of the first file)

Definition at line 160 of file TTreeProcessorMP.cxx.

Process() [9/20]

template<class F >

auto ROOT::TTreeProcessorMP::Process	(	TChain &	chain,
		F	procFunc,
		TEntryList &	entries,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)		-> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 283 of file TTreeProcessorMP.hxx.

Process() [10/20]

template<class F >

auto ROOT::TTreeProcessorMP::Process	(	TChain &	files,
		F	procFunc,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)		-> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 386 of file TTreeProcessorMP.hxx.

Process() [11/20]

TList * ROOT::TTreeProcessorMP::Process	(	TChain &	files,
		TSelector &	selector,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)

Definition at line 310 of file TTreeProcessorMP.cxx.

Process() [12/20]

TList * ROOT::TTreeProcessorMP::Process	(	TChain &	files,
		TSelector &	selector,
		TEntryList &	entries,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)

TSelector-based tree processing: dataset as a TChain.

Definition at line 264 of file TTreeProcessorMP.cxx.

Process() [13/20]

template<class F >

auto ROOT::TTreeProcessorMP::Process	(	TFileCollection &	collection,
		F	procFunc,
		TEntryList &	entries,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)		-> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 269 of file TTreeProcessorMP.hxx.

Process() [14/20]

template<class F >

auto ROOT::TTreeProcessorMP::Process	(	TFileCollection &	files,
		F	procFunc,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)		-> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 376 of file TTreeProcessorMP.hxx.

Process() [15/20]

TList * ROOT::TTreeProcessorMP::Process	(	TFileCollection &	files,
		TSelector &	selector,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)

Definition at line 303 of file TTreeProcessorMP.cxx.

Process() [16/20]

TList * ROOT::TTreeProcessorMP::Process	(	TFileCollection &	files,
		TSelector &	selector,
		TEntryList &	entries,
		const std::string &	treeName = "",
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)

TSelector-based tree processing: dataset as a TFileCollection.

Definition at line 250 of file TTreeProcessorMP.cxx.

Process() [17/20]

template<class F >

auto ROOT::TTreeProcessorMP::Process	(	TTree &	tree,
		F	procFunc,
		TEntryList &	entries,
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)		-> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 298 of file TTreeProcessorMP.hxx.

Process() [18/20]

template<class F >

auto ROOT::TTreeProcessorMP::Process	(	TTree &	tree,
		F	procFunc,
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)		-> InvokeResult_t<F, std::reference_wrapper<TTreeReader>>

Definition at line 396 of file TTreeProcessorMP.hxx.

Process() [19/20]

TList * ROOT::TTreeProcessorMP::Process	(	TTree &	tree,
		TSelector &	selector,
		TEntryList &	entries,
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)

TSelector-based tree processing: memory resident tree.

Definition at line 97 of file TTreeProcessorMP.cxx.

Process() [20/20]

TList * ROOT::TTreeProcessorMP::Process	(	TTree &	tree,
		TSelector &	selector,
		ULong64_t	nToProcess = 0,
		ULong64_t	jFirst = 0
	)

Definition at line 317 of file TTreeProcessorMP.cxx.

ReplyToIdle()

void ROOT::TTreeProcessorMP::ReplyToIdle ( TSocket *  s )

private

Reply to a worker who is idle.

If still events to process, tell the worker. Otherwise ask for a result

Definition at line 351 of file TTreeProcessorMP.cxx.

Reset()

void ROOT::TTreeProcessorMP::Reset ( )

private

Reset TTreeProcessorMP's state.

Definition at line 340 of file TTreeProcessorMP.cxx.

SetNWorkers()

void ROOT::TTreeProcessorMP::SetNWorkers ( unsigned  n )

inline

Definition at line 165 of file TTreeProcessorMP.hxx.

Member Data Documentation

fNProcessed

unsigned ROOT::TTreeProcessorMP::fNProcessed

private

number of arguments already passed to the workers

Definition at line 176 of file TTreeProcessorMP.hxx.

fNToProcess

unsigned ROOT::TTreeProcessorMP::fNToProcess

private

total number of arguments to pass to the workers

Definition at line 177 of file TTreeProcessorMP.hxx.

fTaskType

ETask ROOT::TTreeProcessorMP::fTaskType = ETask::kNoTask

private

the kind of task that is being executed, if any

Definition at line 188 of file TTreeProcessorMP.hxx.

Libraries for ROOT::TTreeProcessorMP:

---

The documentation for this class was generated from the following files:

• tree/treeplayer/inc/ROOT/TTreeProcessorMP.hxx
• tree/treeplayer/src/TTreeProcessorMP.cxx