Logo ROOT  
Reference Guide
 
Loading...
Searching...
No Matches
ROOT::RDF::Experimental::ProgressHelper Class Reference

RDF progress helper.

This class provides callback functions to the RDataFrame. The event statistics (including elapsed time, currently processed file, currently processed events, the rate of event processing and an estimated remaining time (per file being processed)) are recorded and printed in the terminal every m events and every n seconds. ProgressHelper::operator()(unsigned int, T&) is thread safe, and can be used as a callback in MT mode. ProgressBar should be added after creating the dataframe object (df):

ROOT::RDataFrame df("tree", "file.root");
ROOT's RDataFrame offers a modern, high-level interface for analysis of data stored in TTree ,...
void AddProgressBar(ROOT::RDF::RNode df)
Add ProgressBar to a ROOT::RDF::RNode.

alternatively RDataFrame can be cast to an RNode first giving it more flexibility. For example, it can be called at any computational node, such as Filter or Define, not only the head node, with no change to the ProgressBar function itself:

ROOT::RDataFrame df("tree", "file.root");
auto df_1 = ROOT::RDF::RNode(df.Filter("x>1"));
RInterface<::ROOT::Detail::RDF::RNodeBase, void > RNode

Definition at line 321 of file RDFHelpers.hxx.

Public Member Functions

 ProgressHelper (std::size_t increment, unsigned int totalFiles=1, unsigned int progressBarWidth=40, unsigned int printInterval=1, bool useColors=true)
 Create a progress helper.
 
 ~ProgressHelper ()=default
 
unsigned int ComputeCurrentFileIdx () const
 
std::size_t ComputeNEventsSoFar () const
 
template<typename T >
void operator() (T &)
 Thread-safe callback for RDataFrame.
 
template<typename T >
void operator() (unsigned int, T &value)
 Thread-safe callback for RDataFrame.
 
void registerNewSample (unsigned int, const ROOT::RDF::RSampleInfo &id)
 Register a new sample for completion statistics.
 

Private Member Functions

double EvtPerSec () const
 Compute a running mean of events/s.
 
void PrintProgressBar (std::ostream &stream, std::size_t currentEventCount) const
 Print a progress bar of width ProgressHelper::fBarWidth if fGetNEventsOfCurrentFile is known.
 
void PrintStats (std::ostream &stream, std::size_t currentEventCount, std::chrono::seconds totalElapsedSeconds) const
 Print event and time statistics.
 
void PrintStatsFinal (std::ostream &stream, std::chrono::seconds totalElapsedSeconds) const
 
std::pair< std::size_t, std::chrono::seconds > RecordEvtCountAndTime ()
 Record current event counts and time stamp, populate evts/s statistics array.
 

Private Attributes

unsigned int fBarWidth
 
std::chrono::time_point< std::chrono::system_clock > fBeginTime = std::chrono::system_clock::now()
 
std::array< double, 20 > fEventsPerSecondStatistics
 
std::size_t fEventsPerSecondStatisticsIndex {0}
 
std::size_t fIncrement
 
bool fIsTTY
 
std::chrono::time_point< std::chrono::system_clock > fLastPrintTime = fBeginTime
 
std::size_t fLastProcessedEvents {0}
 
std::chrono::seconds fPrintInterval {1}
 
std::mutex fPrintMutex
 
std::atomic< std::size_t > fProcessedEvents {0}
 
std::map< std::string, ULong64_tfSampleNameToEventEntries
 
std::mutex fSampleNameToEventEntriesMutex
 
unsigned int fTotalFiles
 
std::shared_ptr< TTreefTree {nullptr}
 
bool fUseShellColours
 

Friends

class ProgressBarAction
 

#include <ROOT/RDFHelpers.hxx>

Constructor & Destructor Documentation

◆ ProgressHelper()

ROOT::RDF::Experimental::ProgressHelper::ProgressHelper ( std::size_t  increment,
unsigned int  totalFiles = 1,
unsigned int  progressBarWidth = 40,
unsigned int  printInterval = 1,
bool  useColors = true 
)

Create a progress helper.

Parameters
incrementRDF callbacks are called every n events. Pass this n here.
totalFilesread total number of files in the RDF.
progressBarWidthNumber of characters the progress bar will occupy.
printIntervalUpdate every stats every n seconds.
useColorsUse shell colour codes to colour the output. Automatically disabled when we are not writing to a tty.

Definition at line 145 of file RDFHelpers.cxx.

◆ ~ProgressHelper()

ROOT::RDF::Experimental::ProgressHelper::~ProgressHelper ( )
default

Member Function Documentation

◆ ComputeCurrentFileIdx()

unsigned int ROOT::RDF::Experimental::ProgressHelper::ComputeCurrentFileIdx ( ) const
inline

Definition at line 444 of file RDFHelpers.hxx.

◆ ComputeNEventsSoFar()

std::size_t ROOT::RDF::Experimental::ProgressHelper::ComputeNEventsSoFar ( ) const
inline

Definition at line 435 of file RDFHelpers.hxx.

◆ EvtPerSec()

double ROOT::RDF::Experimental::ProgressHelper::EvtPerSec ( ) const
private

Compute a running mean of events/s.

Definition at line 162 of file RDFHelpers.cxx.

◆ operator()() [1/2]

template<typename T >
void ROOT::RDF::Experimental::ProgressHelper::operator() ( T &  )
inline

Thread-safe callback for RDataFrame.

It will record elapsed times and event statistics, and print a progress bar every n seconds (set by the fPrintInterval).

Parameters
valueIgnored.

Definition at line 399 of file RDFHelpers.hxx.

◆ operator()() [2/2]

template<typename T >
void ROOT::RDF::Experimental::ProgressHelper::operator() ( unsigned int  ,
T &  value 
)
inline

Thread-safe callback for RDataFrame.

It will record elapsed times and event statistics, and print a progress bar every n seconds (set by the fPrintInterval).

Parameters
slotIgnored.
valueIgnored.

Definition at line 389 of file RDFHelpers.hxx.

◆ PrintProgressBar()

void ROOT::RDF::Experimental::ProgressHelper::PrintProgressBar ( std::ostream &  stream,
std::size_t  currentEventCount 
) const
private

Print a progress bar of width ProgressHelper::fBarWidth if fGetNEventsOfCurrentFile is known.

Definition at line 297 of file RDFHelpers.cxx.

◆ PrintStats()

void ROOT::RDF::Experimental::ProgressHelper::PrintStats ( std::ostream &  stream,
std::size_t  currentEventCount,
std::chrono::seconds  totalElapsedSeconds 
) const
private

Print event and time statistics.

Definition at line 221 of file RDFHelpers.cxx.

◆ PrintStatsFinal()

void ROOT::RDF::Experimental::ProgressHelper::PrintStatsFinal ( std::ostream &  stream,
std::chrono::seconds  totalElapsedSeconds 
) const
private

Definition at line 268 of file RDFHelpers.cxx.

◆ RecordEvtCountAndTime()

std::pair< std::size_t, std::chrono::seconds > ROOT::RDF::Experimental::ProgressHelper::RecordEvtCountAndTime ( )
private

Record current event counts and time stamp, populate evts/s statistics array.

Definition at line 174 of file RDFHelpers.cxx.

◆ registerNewSample()

void ROOT::RDF::Experimental::ProgressHelper::registerNewSample ( unsigned int  ,
const ROOT::RDF::RSampleInfo id 
)
inline

Register a new sample for completion statistics.

See also
ROOT::RDF::RInterface::DefinePerSample(). The id.AsString() refers to the name of the currently processed file. The idea is to populate the event entries in the fSampleNameToEventEntries map by selecting the greater of the two values: id.EntryRange().second which is the upper event entry range of the processed sample and the current value of the event entries in the fSampleNameToEventEntries map. In the single threaded case, the two numbers are the same as the entry range corresponds to the number of events in an individual file (each sample is simply a single file). In the multithreaded case, the idea is to accumulate the higher event entry value until the total number of events in a given file is reached.

Definition at line 378 of file RDFHelpers.hxx.

Friends And Related Symbol Documentation

◆ ProgressBarAction

friend class ProgressBarAction
friend

Definition at line 365 of file RDFHelpers.hxx.

Member Data Documentation

◆ fBarWidth

unsigned int ROOT::RDF::Experimental::ProgressHelper::fBarWidth
private

Definition at line 343 of file RDFHelpers.hxx.

◆ fBeginTime

std::chrono::time_point<std::chrono::system_clock> ROOT::RDF::Experimental::ProgressHelper::fBeginTime = std::chrono::system_clock::now()
private

Definition at line 329 of file RDFHelpers.hxx.

◆ fEventsPerSecondStatistics

std::array<double, 20> ROOT::RDF::Experimental::ProgressHelper::fEventsPerSecondStatistics
private

Definition at line 340 of file RDFHelpers.hxx.

◆ fEventsPerSecondStatisticsIndex

std::size_t ROOT::RDF::Experimental::ProgressHelper::fEventsPerSecondStatisticsIndex {0}
private

Definition at line 341 of file RDFHelpers.hxx.

◆ fIncrement

std::size_t ROOT::RDF::Experimental::ProgressHelper::fIncrement
private

Definition at line 335 of file RDFHelpers.hxx.

◆ fIsTTY

bool ROOT::RDF::Experimental::ProgressHelper::fIsTTY
private

Definition at line 347 of file RDFHelpers.hxx.

◆ fLastPrintTime

std::chrono::time_point<std::chrono::system_clock> ROOT::RDF::Experimental::ProgressHelper::fLastPrintTime = fBeginTime
private

Definition at line 330 of file RDFHelpers.hxx.

◆ fLastProcessedEvents

std::size_t ROOT::RDF::Experimental::ProgressHelper::fLastProcessedEvents {0}
private

Definition at line 334 of file RDFHelpers.hxx.

◆ fPrintInterval

std::chrono::seconds ROOT::RDF::Experimental::ProgressHelper::fPrintInterval {1}
private

Definition at line 331 of file RDFHelpers.hxx.

◆ fPrintMutex

std::mutex ROOT::RDF::Experimental::ProgressHelper::fPrintMutex
private

Definition at line 346 of file RDFHelpers.hxx.

◆ fProcessedEvents

std::atomic<std::size_t> ROOT::RDF::Experimental::ProgressHelper::fProcessedEvents {0}
private

Definition at line 333 of file RDFHelpers.hxx.

◆ fSampleNameToEventEntries

std::map<std::string, ULong64_t> ROOT::RDF::Experimental::ProgressHelper::fSampleNameToEventEntries
private

Definition at line 338 of file RDFHelpers.hxx.

◆ fSampleNameToEventEntriesMutex

std::mutex ROOT::RDF::Experimental::ProgressHelper::fSampleNameToEventEntriesMutex
mutableprivate

Definition at line 337 of file RDFHelpers.hxx.

◆ fTotalFiles

unsigned int ROOT::RDF::Experimental::ProgressHelper::fTotalFiles
private

Definition at line 344 of file RDFHelpers.hxx.

◆ fTree

std::shared_ptr<TTree> ROOT::RDF::Experimental::ProgressHelper::fTree {nullptr}
private

Definition at line 350 of file RDFHelpers.hxx.

◆ fUseShellColours

bool ROOT::RDF::Experimental::ProgressHelper::fUseShellColours
private

Definition at line 348 of file RDFHelpers.hxx.

Libraries for ROOT::RDF::Experimental::ProgressHelper:

The documentation for this class was generated from the following files: