ROOT::Experimental::RNTupleParallelWriter Class Reference

A writer to fill an RNTuple from multiple contexts.

Compared to the sequential RNTupleWriter, a parallel writer enables the creation of multiple RNTupleFillContext (see RNTupleParallelWriter::CreateFillContext). Each fill context prepares independent clusters that are appended to the common ntuple with internal synchronization. Before destruction, all fill contexts must have flushed their data and been destroyed (or data could be lost!).

For user convenience, RNTupleParallelWriter::CreateFillContext is thread-safe and may be called from multiple threads in parallel at any time, also after some data has already been written. Internally, the original model is cloned and ownership is passed to a newly created RNTupleFillContext. For that reason, it is recommended to use RNTupleModel::CreateBare when creating the model for parallel writing and avoid the allocation of a useless default REntry per context.

Note that the sequence of independently prepared clusters is indeterminate and therefore entries are only partially ordered: Entries from one context are totally ordered as they were filled. However, there is no orderering with other contexts and the entries may be appended to the ntuple either before or after other entries written in parallel into other contexts. In addition, two consecutive entries in one fill context can end up separated in the final ntuple, if they happen to fall onto a cluster boundary and other contexts append more entries before the next cluster is full.

At the moment, the parallel writer does not (yet) support incremental updates of the underlying model. Please refer to RNTupleWriter::CreateModelUpdater if required for your use case.

Definition at line 64 of file RNTupleParallelWriter.hxx.

Public Member Functions
	~RNTupleParallelWriter ()
void	CommitDataset ()
	Automatically called by the destructor.
std::shared_ptr< RNTupleFillContext >	CreateFillContext ()
	Create a new RNTupleFillContext that can be used to fill entries and prepare clusters in parallel.
void	EnableMetrics ()
const Detail::RNTupleMetrics &	GetMetrics () const

Static Public Member Functions
static std::unique_ptr< RNTupleParallelWriter >	Append (std::unique_ptr< RNTupleModel > model, std::string_view ntupleName, TDirectory &fileOrDirectory, const RNTupleWriteOptions &options=RNTupleWriteOptions())
	Append an ntuple to the existing file, which must not be accessed while data is filled into any created context.
static std::unique_ptr< RNTupleParallelWriter >	Recreate (std::unique_ptr< RNTupleModel > model, std::string_view ntupleName, std::string_view storage, const RNTupleWriteOptions &options=RNTupleWriteOptions())
	Recreate a new file and return a writer to write an ntuple.

Private Member Functions
	RNTupleParallelWriter (const RNTupleParallelWriter &)=delete
	RNTupleParallelWriter (std::unique_ptr< RNTupleModel > model, std::unique_ptr< Internal::RPageSink > sink)
RNTupleParallelWriter &	operator= (const RNTupleParallelWriter &)=delete

Private Attributes
std::vector< std::weak_ptr< RNTupleFillContext > >	fFillContexts
	List of all created helpers. They must be destroyed before this RNTupleParallelWriter is destructed.
Detail::RNTupleMetrics	fMetrics
std::unique_ptr< RNTupleModel >	fModel
	The original RNTupleModel connected to fSink; needs to be destructed before it.
std::mutex	fMutex
	A global mutex to protect the internal data structures of this object.
std::unique_ptr< Internal::RPageSink >	fSink
	The final RPageSink that represents the synchronization point.
std::mutex	fSinkMutex
	A mutex to synchronize the final page sink.

#include <ROOT/RNTupleParallelWriter.hxx>

Constructor & Destructor Documentation

RNTupleParallelWriter() [1/2]

ROOT::Experimental::RNTupleParallelWriter::RNTupleParallelWriter ( std::unique_ptr< RNTupleModel >  model, std::unique_ptr< Internal::RPageSink >  sink  )

private

Definition at line 116 of file RNTupleParallelWriter.cxx.

RNTupleParallelWriter() [2/2]

ROOT::Experimental::RNTupleParallelWriter::RNTupleParallelWriter ( const RNTupleParallelWriter &  )

privatedelete

~RNTupleParallelWriter()

ROOT::Experimental::RNTupleParallelWriter::~RNTupleParallelWriter

(

)

Definition at line 128 of file RNTupleParallelWriter.cxx.

Member Function Documentation

Append()

std::unique_ptr< ROOT::Experimental::RNTupleParallelWriter > ROOT::Experimental::RNTupleParallelWriter::Append ( std::unique_ptr< RNTupleModel >  model, std::string_view  ntupleName, TDirectory &  fileOrDirectory, const RNTupleWriteOptions &  options = RNTupleWriteOptions()  )

static

Append an ntuple to the existing file, which must not be accessed while data is filled into any created context.

Definition at line 168 of file RNTupleParallelWriter.cxx.

CommitDataset()

void ROOT::Experimental::RNTupleParallelWriter::CommitDataset

(

)

Automatically called by the destructor.

Definition at line 137 of file RNTupleParallelWriter.cxx.

CreateFillContext()

std::shared_ptr< ROOT::Experimental::RNTupleFillContext > ROOT::Experimental::RNTupleParallelWriter::CreateFillContext

(

)

Create a new RNTupleFillContext that can be used to fill entries and prepare clusters in parallel.

This method is thread-safe and may be called from multiple threads in parallel at any time, also after some data has already been written.

Note that all fill contexts must be destroyed before RNTupleParallelWriter::CommitDataset() is called.

Definition at line 180 of file RNTupleParallelWriter.cxx.

EnableMetrics()

void ROOT::Experimental::RNTupleParallelWriter::EnableMetrics ( )

inline

Definition at line 104 of file RNTupleParallelWriter.hxx.

GetMetrics()

const Detail::RNTupleMetrics & ROOT::Experimental::RNTupleParallelWriter::GetMetrics ( ) const

inline

Definition at line 105 of file RNTupleParallelWriter.hxx.

operator=()

RNTupleParallelWriter & ROOT::Experimental::RNTupleParallelWriter::operator= ( const RNTupleParallelWriter &  )

privatedelete

Recreate()

std::unique_ptr< ROOT::Experimental::RNTupleParallelWriter > ROOT::Experimental::RNTupleParallelWriter::Recreate ( std::unique_ptr< RNTupleModel >  model, std::string_view  ntupleName, std::string_view  storage, const RNTupleWriteOptions &  options = RNTupleWriteOptions()  )

static

Recreate a new file and return a writer to write an ntuple.

Definition at line 155 of file RNTupleParallelWriter.cxx.

Member Data Documentation

fFillContexts

std::vector<std::weak_ptr<RNTupleFillContext> > ROOT::Experimental::RNTupleParallelWriter::fFillContexts

private

List of all created helpers. They must be destroyed before this RNTupleParallelWriter is destructed.

Definition at line 76 of file RNTupleParallelWriter.hxx.

fMetrics

Detail::RNTupleMetrics ROOT::Experimental::RNTupleParallelWriter::fMetrics

private

Definition at line 74 of file RNTupleParallelWriter.hxx.

fModel

std::unique_ptr<RNTupleModel> ROOT::Experimental::RNTupleParallelWriter::fModel

private

The original RNTupleModel connected to fSink; needs to be destructed before it.

Definition at line 73 of file RNTupleParallelWriter.hxx.

fMutex

std::mutex ROOT::Experimental::RNTupleParallelWriter::fMutex

private

A global mutex to protect the internal data structures of this object.

Definition at line 67 of file RNTupleParallelWriter.hxx.

fSink

std::unique_ptr<Internal::RPageSink> ROOT::Experimental::RNTupleParallelWriter::fSink

private

The final RPageSink that represents the synchronization point.

Definition at line 71 of file RNTupleParallelWriter.hxx.

fSinkMutex

std::mutex ROOT::Experimental::RNTupleParallelWriter::fSinkMutex

private

A mutex to synchronize the final page sink.

Definition at line 69 of file RNTupleParallelWriter.hxx.

Libraries for ROOT::Experimental::RNTupleParallelWriter:

---

The documentation for this class was generated from the following files:

• tree/ntuple/v7/inc/ROOT/RNTupleParallelWriter.hxx
• tree/ntuple/v7/src/RNTupleParallelWriter.cxx