ROOT::RDF::Experimental::ProgressHelper Class Reference

RDF progress helper.

This class provides callback functions to the RDataFrame. The event statistics (including elapsed time, currently processed file, currently processed events, the rate of event processing and an estimated remaining time (per file being processed)) are recorded and printed in the terminal every m events and every n seconds. ProgressHelper::operator()(unsigned int, T&) is thread safe, and can be used as a callback in MT mode. ProgressBar should be added after creating the dataframe object (df):

ROOT::RDataFrame df("tree", "file.root");
ROOT::RDF::Experimental::AddProgressBar(df);

alternatively RDataFrame can be cast to an RNode first giving it more flexibility. For example, it can be called at any computational node, such as Filter or Define, not only the head node, with no change to the ProgressBar function itself:

ROOT::RDataFrame df("tree", "file.root");
auto df_1 = ROOT::RDF::RNode(df.Filter("x>1"));
ROOT::RDF::Experimental::AddProgressBar(df_1);

Definition at line 332 of file RDFHelpers.hxx.

Public Member Functions
	ProgressHelper (ProgressHelper &&)=delete
	ProgressHelper (ProgressHelper const &)=delete
	ProgressHelper (std::size_t increment, unsigned int totalFiles, unsigned int printInterval=0, bool useColors=true)
	Create a progress helper.
	~ProgressHelper ()=default
template<typename T >
void	operator() (unsigned int, T &)
	Thread-safe callback for RDataFrame.
ProgressHelper &	operator= (ProgressHelper &&)=delete
ProgressHelper &	operator= (ProgressHelper const &)=delete
void	PrintStatsFinal () const
void	RegisterNewSample (unsigned int, const ROOT::RDF::RSampleInfo &id)
	Register a new sample for completion statistics.

Private Member Functions
std::size_t	ComputeTotalEvents () const
	Compute total events in all open files.
double	EvtPerSec () const
	Compute a running mean of events/s.
void	PrintProgressAndStats (std::ostream &stream, std::size_t currentEventCount, std::chrono::seconds totalElapsedSeconds) const
	Print event and time statistics.
std::pair< std::size_t, std::chrono::seconds >	RecordEvtCountAndTime ()
	Record current event counts and time stamp, populate evts/s statistics array.
void	Update ()
	Record number of events processed and update progress bar.

Private Attributes
std::chrono::time_point< std::chrono::system_clock > const	fBeginTime = std::chrono::system_clock::now()
std::array< double, 10 >	fEventsPerSecondStatistics
unsigned int	fEventsPerSecondStatisticsCounter {0}
std::size_t const	fIncrement
bool const	fIsTTY
std::chrono::time_point< std::chrono::system_clock >	fLastPrintTime = fBeginTime
std::size_t	fLastProcessedEvents {0}
unsigned int const	fNColumns
std::chrono::seconds const	fPrintInterval
std::atomic< std::size_t >	fProcessedEvents {0}
std::map< std::string, ULong64_t >	fSampleNameToEventEntries
std::mutex	fSampleNameToEventEntriesMutex
unsigned int const	fTotalFiles
std::mutex	fUpdateMutex
bool const	fUseShellColours

#include <ROOT/RDFHelpers.hxx>

Constructor & Destructor Documentation

ProgressHelper() [1/3]

ROOT::RDF::Experimental::ProgressHelper::ProgressHelper	(	std::size_t	increment,
		unsigned int	totalFiles,
		unsigned int	printInterval = 0,
		bool	useColors = true )

Create a progress helper.

Parameters

increment	RDF callbacks are called every n events. Pass this n here.
totalFiles	number of files read in the RDF.
printInterval	Update stats every n seconds.
useColors	Use shell colour codes to colour the output. Automatically disabled when we are not writing to a tty.

Definition at line 178 of file RDFHelpers.cxx.

ProgressHelper() [2/3]

ROOT::RDF::Experimental::ProgressHelper::ProgressHelper ( ProgressHelper const & )

delete

ProgressHelper() [3/3]

ROOT::RDF::Experimental::ProgressHelper::ProgressHelper ( ProgressHelper && )

delete

~ProgressHelper()

ROOT::RDF::Experimental::ProgressHelper::~ProgressHelper ( )

default

Member Function Documentation

ComputeTotalEvents()

std::size_t ROOT::RDF::Experimental::ProgressHelper::ComputeTotalEvents ( ) const

private

Compute total events in all open files.

Definition at line 229 of file RDFHelpers.cxx.

EvtPerSec()

double ROOT::RDF::Experimental::ProgressHelper::EvtPerSec ( ) const

private

Compute a running mean of events/s.

Definition at line 214 of file RDFHelpers.cxx.

operator()()

template<typename T >

void ROOT::RDF::Experimental::ProgressHelper::operator() ( unsigned int , T &  )

inline

Thread-safe callback for RDataFrame.

It will record elapsed times and event statistics, and print a progress bar every n seconds (set by the fPrintInterval).

Parameters

slot	Ignored.
value	Ignored.

Definition at line 385 of file RDFHelpers.hxx.

operator=() [1/2]

ProgressHelper & ROOT::RDF::Experimental::ProgressHelper::operator= ( ProgressHelper && )

delete

operator=() [2/2]

ProgressHelper & ROOT::RDF::Experimental::ProgressHelper::operator= ( ProgressHelper const & )

delete

PrintProgressAndStats()

void ROOT::RDF::Experimental::ProgressHelper::PrintProgressAndStats ( std::ostream & stream, std::size_t currentEventCount, std::chrono::seconds totalElapsedSeconds ) const

private

Print event and time statistics.

Definition at line 261 of file RDFHelpers.cxx.

PrintStatsFinal()

void ROOT::RDF::Experimental::ProgressHelper::PrintStatsFinal

(

)

const

Definition at line 356 of file RDFHelpers.cxx.

RecordEvtCountAndTime()

std::pair< std::size_t, std::chrono::seconds > ROOT::RDF::Experimental::ProgressHelper::RecordEvtCountAndTime ( )

private

Record current event counts and time stamp, populate evts/s statistics array.

The function assumes that a lock on the update mutex is held.

Definition at line 240 of file RDFHelpers.cxx.

RegisterNewSample()

void ROOT::RDF::Experimental::ProgressHelper::RegisterNewSample	(	unsigned int	,
		const ROOT::RDF::RSampleInfo &	id )

Register a new sample for completion statistics.

See also

ROOT::RDF::RInterface::DefinePerSample(). The id.AsString() refers to the name of the currently processed file. The idea is to populate the event entries in the fSampleNameToEventEntries map by selecting the greater of the two values: id.EntryRange().second which is the upper event entry range of the processed sample and the current value of the event entries in the fSampleNameToEventEntries map. In the single threaded case, the two numbers are the same as the entry range corresponds to the number of events in an individual file (each sample is simply a single file). In the multithreaded case, the idea is to accumulate the higher event entry value until the total number of events in a given file is reached.

Definition at line 207 of file RDFHelpers.cxx.

Update()

void ROOT::RDF::Experimental::ProgressHelper::Update ( )

private

Record number of events processed and update progress bar.

This function will atomically record elapsed times and event statistics, and one thread will udpate the progress bar every n seconds (set by the fPrintInterval).

Definition at line 397 of file RDFHelpers.cxx.

Member Data Documentation

fBeginTime

std::chrono::time_point<std::chrono::system_clock> const ROOT::RDF::Experimental::ProgressHelper::fBeginTime = std::chrono::system_clock::now()

private

Definition at line 353 of file RDFHelpers.hxx.

fEventsPerSecondStatistics

std::array<double, 10> ROOT::RDF::Experimental::ProgressHelper::fEventsPerSecondStatistics

private

Definition at line 350 of file RDFHelpers.hxx.

fEventsPerSecondStatisticsCounter

unsigned int ROOT::RDF::Experimental::ProgressHelper::fEventsPerSecondStatisticsCounter {0}

private

Definition at line 351 of file RDFHelpers.hxx.

fIncrement

std::size_t const ROOT::RDF::Experimental::ProgressHelper::fIncrement

private

Definition at line 346 of file RDFHelpers.hxx.

fIsTTY

bool const ROOT::RDF::Experimental::ProgressHelper::fIsTTY

private

Definition at line 341 of file RDFHelpers.hxx.

fLastPrintTime

std::chrono::time_point<std::chrono::system_clock> ROOT::RDF::Experimental::ProgressHelper::fLastPrintTime = fBeginTime

private

Definition at line 354 of file RDFHelpers.hxx.

fLastProcessedEvents

std::size_t ROOT::RDF::Experimental::ProgressHelper::fLastProcessedEvents {0}

private

Definition at line 345 of file RDFHelpers.hxx.

fNColumns

unsigned int const ROOT::RDF::Experimental::ProgressHelper::fNColumns

private

Definition at line 347 of file RDFHelpers.hxx.

fPrintInterval

std::chrono::seconds const ROOT::RDF::Experimental::ProgressHelper::fPrintInterval

private

Definition at line 355 of file RDFHelpers.hxx.

fProcessedEvents

std::atomic<std::size_t> ROOT::RDF::Experimental::ProgressHelper::fProcessedEvents {0}

private

Definition at line 344 of file RDFHelpers.hxx.

fSampleNameToEventEntries

std::map<std::string, ULong64_t> ROOT::RDF::Experimental::ProgressHelper::fSampleNameToEventEntries

private

Definition at line 362 of file RDFHelpers.hxx.

fSampleNameToEventEntriesMutex

std::mutex ROOT::RDF::Experimental::ProgressHelper::fSampleNameToEventEntriesMutex

mutableprivate

Definition at line 361 of file RDFHelpers.hxx.

fTotalFiles

unsigned int const ROOT::RDF::Experimental::ProgressHelper::fTotalFiles

private

Definition at line 348 of file RDFHelpers.hxx.

fUpdateMutex

std::mutex ROOT::RDF::Experimental::ProgressHelper::fUpdateMutex

private

Definition at line 359 of file RDFHelpers.hxx.

fUseShellColours

bool const ROOT::RDF::Experimental::ProgressHelper::fUseShellColours

private

Definition at line 342 of file RDFHelpers.hxx.

Libraries for ROOT::RDF::Experimental::ProgressHelper:

---

The documentation for this class was generated from the following files:

• tree/dataframe/inc/ROOT/RDFHelpers.hxx
• tree/dataframe/src/RDFHelpers.cxx