RDF progress helper.
This class provides callback functions to the RDataFrame. The event statistics (including elapsed time, currently processed file, currently processed events, the rate of event processing and an estimated remaining time (per file being processed)) are recorded and printed in the terminal every m events and every n seconds. ProgressHelper::operator()(unsigned int, T&) is thread safe, and can be used as a callback in MT mode. ProgressBar should be added after creating the dataframe object (df):
alternatively RDataFrame can be cast to an RNode first giving it more flexibility. For example, it can be called at any computational node, such as Filter or Define, not only the head node, with no change to the ProgressBar function itself:
Definition at line 296 of file RDFHelpers.hxx.
Public Member Functions | |
ProgressHelper (std::size_t increment, unsigned int totalFiles=1, unsigned int progressBarWidth=40, unsigned int printInterval=1, bool useColors=true) | |
Create a progress helper. | |
~ProgressHelper ()=default | |
unsigned int | ComputeCurrentFileIdx () const |
std::size_t | ComputeNEventsSoFar () const |
template<typename T > | |
void | operator() (T &) |
Thread-safe callback for RDataFrame. | |
template<typename T > | |
void | operator() (unsigned int, T &value) |
Thread-safe callback for RDataFrame. | |
void | registerNewSample (unsigned int, const ROOT::RDF::RSampleInfo &id) |
Register a new sample for completion statistics. | |
Private Member Functions | |
double | EvtPerSec () const |
Compute a running mean of events/s. | |
void | PrintProgressBar (std::ostream &stream, std::size_t currentEventCount) const |
Print a progress bar of width ProgressHelper::fBarWidth if fGetNEventsOfCurrentFile is known. | |
void | PrintStats (std::ostream &stream, std::size_t currentEventCount, std::chrono::seconds totalElapsedSeconds) const |
Print event and time statistics. | |
void | PrintStatsFinal (std::ostream &stream, std::chrono::seconds totalElapsedSeconds) const |
std::pair< std::size_t, std::chrono::seconds > | RecordEvtCountAndTime () |
Record current event counts and time stamp, populate evts/s statistics array. | |
Private Attributes | |
unsigned int | fBarWidth |
std::chrono::time_point< std::chrono::system_clock > | fBeginTime = std::chrono::system_clock::now() |
std::array< double, 20 > | fEventsPerSecondStatistics |
std::size_t | fEventsPerSecondStatisticsIndex {0} |
std::size_t | fIncrement |
bool | fIsTTY |
std::chrono::time_point< std::chrono::system_clock > | fLastPrintTime = fBeginTime |
std::size_t | fLastProcessedEvents {0} |
std::chrono::seconds | fPrintInterval {1} |
std::mutex | fPrintMutex |
std::atomic< std::size_t > | fProcessedEvents {0} |
std::map< std::string, ULong64_t > | fSampleNameToEventEntries |
std::mutex | fSampleNameToEventEntriesMutex |
unsigned int | fTotalFiles |
std::shared_ptr< TTree > | fTree {nullptr} |
bool | fUseShellColours |
Friends | |
class | ProgressBarAction |
#include <ROOT/RDFHelpers.hxx>
ROOT::RDF::Experimental::ProgressHelper::ProgressHelper | ( | std::size_t | increment, |
unsigned int | totalFiles = 1 , |
||
unsigned int | progressBarWidth = 40 , |
||
unsigned int | printInterval = 1 , |
||
bool | useColors = true |
||
) |
Create a progress helper.
increment | RDF callbacks are called every n events. Pass this n here. |
totalFiles | read total number of files in the RDF. |
progressBarWidth | Number of characters the progress bar will occupy. |
printInterval | Update every stats every n seconds. |
useColors | Use shell colour codes to colour the output. Automatically disabled when we are not writing to a tty. |
Definition at line 145 of file RDFHelpers.cxx.
|
default |
|
inline |
Definition at line 419 of file RDFHelpers.hxx.
|
inline |
Definition at line 410 of file RDFHelpers.hxx.
|
private |
Compute a running mean of events/s.
Definition at line 162 of file RDFHelpers.cxx.
|
inline |
Thread-safe callback for RDataFrame.
It will record elapsed times and event statistics, and print a progress bar every n seconds (set by the fPrintInterval).
value | Ignored. |
Definition at line 374 of file RDFHelpers.hxx.
|
inline |
Thread-safe callback for RDataFrame.
It will record elapsed times and event statistics, and print a progress bar every n seconds (set by the fPrintInterval).
slot | Ignored. |
value | Ignored. |
Definition at line 364 of file RDFHelpers.hxx.
|
private |
Print a progress bar of width ProgressHelper::fBarWidth
if fGetNEventsOfCurrentFile
is known.
Definition at line 297 of file RDFHelpers.cxx.
|
private |
Print event and time statistics.
Definition at line 221 of file RDFHelpers.cxx.
|
private |
Definition at line 268 of file RDFHelpers.cxx.
|
private |
Record current event counts and time stamp, populate evts/s statistics array.
Definition at line 174 of file RDFHelpers.cxx.
|
inline |
Register a new sample for completion statistics.
Definition at line 353 of file RDFHelpers.hxx.
|
friend |
Definition at line 340 of file RDFHelpers.hxx.
|
private |
Definition at line 318 of file RDFHelpers.hxx.
|
private |
Definition at line 304 of file RDFHelpers.hxx.
|
private |
Definition at line 315 of file RDFHelpers.hxx.
|
private |
Definition at line 316 of file RDFHelpers.hxx.
|
private |
Definition at line 310 of file RDFHelpers.hxx.
|
private |
Definition at line 322 of file RDFHelpers.hxx.
|
private |
Definition at line 305 of file RDFHelpers.hxx.
|
private |
Definition at line 309 of file RDFHelpers.hxx.
|
private |
Definition at line 306 of file RDFHelpers.hxx.
|
private |
Definition at line 321 of file RDFHelpers.hxx.
|
private |
Definition at line 308 of file RDFHelpers.hxx.
|
private |
Definition at line 313 of file RDFHelpers.hxx.
|
mutableprivate |
Definition at line 312 of file RDFHelpers.hxx.
|
private |
Definition at line 319 of file RDFHelpers.hxx.
|
private |
Definition at line 325 of file RDFHelpers.hxx.
|
private |
Definition at line 323 of file RDFHelpers.hxx.