Wrapper sink that coalesces cluster column page writes.

Definition at line 38 of file RPageSinkBuf.hxx.

Classes
class	RColumnBuf
	A buffered column. More...

struct	RCounters
	I/O performance counters that get registered in fMetrics. More...

Public Member Functions
	RPageSinkBuf (const RPageSinkBuf &)=delete

	RPageSinkBuf (RPageSinkBuf &&)=default

	RPageSinkBuf (std::unique_ptr< RPageSink > inner)

virtual	~RPageSinkBuf ()=default

RNTupleMetrics &	GetMetrics () final
	Returns the default metrics object. Subclasses might alternatively provide their own metrics object by overriding this.

RPageSinkBuf &	operator= (const RPageSinkBuf &)=delete

RPageSinkBuf &	operator= (RPageSinkBuf &&)=default

void	ReleasePage (RPage &page) final
	Every page store needs to be able to free pages it handed out.

RPage	ReservePage (ColumnHandle_t columnHandle, std::size_t nElements) final
	Get a new, empty page for the given column that can be filled with up to nElements.

Public Member Functions inherited from ROOT::Experimental::Detail::RPageSink
	RPageSink (const RPageSink &)=delete

	RPageSink (RPageSink &&)=default

	RPageSink (std::string_view ntupleName, const RNTupleWriteOptions &options)

virtual	~RPageSink ()

ColumnHandle_t	AddColumn (DescriptorId_t fieldId, const RColumn &column) final
	Register a new column.

std::uint64_t	CommitCluster (NTupleSize_t nEntries)
	Finalize the current cluster and create a new one for the following data.

void	CommitDataset ()
	Finalize the current cluster and the entrire data set.

void	CommitPage (ColumnHandle_t columnHandle, const RPage &page)
	Write a page to the storage. The column must have been added before.

void	CommitSealedPage (DescriptorId_t columnId, const RPageStorage::RSealedPage &sealedPage)
	Write a preprocessed page to storage.

void	Create (RNTupleModel &model)
	Physically creates the storage container to hold the ntuple (e.g., a keys a TFile or an S3 bucket) To do so, Create() calls CreateImpl() after updating the descriptor.

void	DropColumn (ColumnHandle_t) final
	Unregisters a column.

EPageStorageType	GetType () final
	Whether the concrete implementation is a sink or a source.

const RNTupleWriteOptions &	GetWriteOptions () const
	Returns the sink's write options.

RPageSink &	operator= (const RPageSink &)=delete

RPageSink &	operator= (RPageSink &&)=default

Public Member Functions inherited from ROOT::Experimental::Detail::RPageStorage
	RPageStorage (const RPageStorage &other)=delete

	RPageStorage (RPageStorage &&other)=default

	RPageStorage (std::string_view name)

virtual	~RPageStorage ()

const std::string &	GetNTupleName () const
	Returns the NTuple name.

RPageStorage &	operator= (const RPageStorage &other)=delete

RPageStorage &	operator= (RPageStorage &&other)=default

void	SetTaskScheduler (RTaskScheduler *taskScheduler)

Protected Member Functions
std::uint64_t	CommitClusterImpl (NTupleSize_t nEntries) final
	Returns the number of bytes written to storage (excluding metadata)

void	CommitDatasetImpl () final

RNTupleLocator	CommitPageImpl (ColumnHandle_t columnHandle, const RPage &page) final

RNTupleLocator	CommitSealedPageImpl (DescriptorId_t columnId, const RSealedPage &sealedPage) final

void	CreateImpl (const RNTupleModel &model) final

Protected Member Functions inherited from ROOT::Experimental::Detail::RPageSink
void	EnableDefaultMetrics (const std::string &prefix)
	Enables the default set of metrics provided by RPageSink.

RSealedPage	SealPage (const RPage &page, const RColumnElementBase &element, int compressionSetting)
	Helper for streaming a page.

Private Attributes
std::vector< RColumnBuf >	fBufferedColumns
	Vector of buffered column pages. Indexed by column id.

std::unique_ptr< RCounters >	fCounters

std::unique_ptr< RNTupleModel >	fInnerModel
	The buffered page sink maintains a copy of the RNTupleModel for the inner sink.

std::unique_ptr< RPageSink >	fInnerSink
	The inner sink, responsible for actually performing I/O.

RNTupleMetrics	fMetrics

Additional Inherited Members
Public Types inherited from ROOT::Experimental::Detail::RPageStorage
using	ColumnHandle_t = RColumnHandle
	The column handle identifies a column with the current open page storage.

Static Public Member Functions inherited from ROOT::Experimental::Detail::RPageSink
static std::unique_ptr< RPageSink >	Create (std::string_view ntupleName, std::string_view location, const RNTupleWriteOptions &options=RNTupleWriteOptions())
	Guess the concrete derived page source from the file name (location)

Static Protected Member Functions inherited from ROOT::Experimental::Detail::RPageSink
static RSealedPage	SealPage (const RPage &page, const RColumnElementBase &element, int compressionSetting, void *buf)
	Seal a page using the provided buffer.

Protected Attributes inherited from ROOT::Experimental::Detail::RPageSink
std::unique_ptr< RNTupleCompressor >	fCompressor
	Helper to zip pages and header/footer; includes a 16MB (kMAXZIPBUF) zip buffer.

std::unique_ptr< RCounters >	fCounters

RNTupleDescriptorBuilder	fDescriptorBuilder

DescriptorId_t	fLastClusterId = 0

DescriptorId_t	fLastColumnId = 0

DescriptorId_t	fLastFieldId = 0
	Building the ntuple descriptor while writing is done in the same way for all the storage sink implementations.

RNTupleMetrics	fMetrics

std::vector< RClusterDescriptor::RColumnRange >	fOpenColumnRanges
	Keeps track of the number of elements in the currently open cluster. Indexed by column id.

std::vector< RClusterDescriptor::RPageRange >	fOpenPageRanges
	Keeps track of the written pages in the currently open cluster. Indexed by column id.

std::unique_ptr< RNTupleWriteOptions >	fOptions

NTupleSize_t	fPrevClusterNEntries = 0

Protected Attributes inherited from ROOT::Experimental::Detail::RPageStorage
std::string	fNTupleName

RTaskScheduler *	fTaskScheduler = nullptr

#include <ROOT/RPageSinkBuf.hxx>

Inheritance diagram for ROOT::Experimental::Detail::RPageSinkBuf:

[legend]

Constructor & Destructor Documentation

◆ RPageSinkBuf() [1/3]

ROOT::Experimental::Detail::RPageSinkBuf::RPageSinkBuf ( std::unique_ptr< RPageSink > inner )

explicit

Definition at line 22 of file RPageSinkBuf.cxx.

◆ RPageSinkBuf() [2/3]

ROOT::Experimental::Detail::RPageSinkBuf::RPageSinkBuf ( const RPageSinkBuf & )

delete

◆ RPageSinkBuf() [3/3]

ROOT::Experimental::Detail::RPageSinkBuf::RPageSinkBuf ( RPageSinkBuf && )

default

◆ ~RPageSinkBuf()

virtual ROOT::Experimental::Detail::RPageSinkBuf::~RPageSinkBuf ( )

virtualdefault

Member Function Documentation

◆ CommitClusterImpl()

std::uint64_t ROOT::Experimental::Detail::RPageSinkBuf::CommitClusterImpl ( NTupleSize_t nEntries )

finalprotectedvirtual

Returns the number of bytes written to storage (excluding metadata)

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 86 of file RPageSinkBuf.cxx.

◆ CommitDatasetImpl()

void ROOT::Experimental::Detail::RPageSinkBuf::CommitDatasetImpl ( )

finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 106 of file RPageSinkBuf.cxx.

◆ CommitPageImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Detail::RPageSinkBuf::CommitPageImpl	(	ColumnHandle_t	columnHandle,
		const RPage &	page
	)

finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 42 of file RPageSinkBuf.cxx.

◆ CommitSealedPageImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Detail::RPageSinkBuf::CommitSealedPageImpl	(	DescriptorId_t	columnId,
		const RSealedPage &	sealedPage
	)

finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 76 of file RPageSinkBuf.cxx.

◆ CreateImpl()

void ROOT::Experimental::Detail::RPageSinkBuf::CreateImpl ( const RNTupleModel & model )

finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 34 of file RPageSinkBuf.cxx.

◆ GetMetrics()

RNTupleMetrics & ROOT::Experimental::Detail::RPageSinkBuf::GetMetrics ( )

inlinefinalvirtual

Returns the default metrics object. Subclasses might alternatively provide their own metrics object by overriding this.

Reimplemented from ROOT::Experimental::Detail::RPageSink.

Definition at line 128 of file RPageSinkBuf.hxx.

◆ operator=() [1/2]

RPageSinkBuf & ROOT::Experimental::Detail::RPageSinkBuf::operator= ( const RPageSinkBuf & )

delete

◆ operator=() [2/2]

RPageSinkBuf & ROOT::Experimental::Detail::RPageSinkBuf::operator= ( RPageSinkBuf && )

default

◆ ReleasePage()

void ROOT::Experimental::Detail::RPageSinkBuf::ReleasePage ( RPage & page )

finalvirtual

Every page store needs to be able to free pages it handed out.

But Sinks and sources have different means of allocating pages.

Implements ROOT::Experimental::Detail::RPageStorage.

Definition at line 117 of file RPageSinkBuf.cxx.

◆ ReservePage()

ROOT::Experimental::Detail::RPage ROOT::Experimental::Detail::RPageSinkBuf::ReservePage	(	ColumnHandle_t	columnHandle,
		std::size_t	nElements
	)

finalvirtual

Get a new, empty page for the given column that can be filled with up to nElements.

If nElements is zero, the page sink picks an appropriate size.

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 112 of file RPageSinkBuf.cxx.

Member Data Documentation

◆ fBufferedColumns

std::vector<RColumnBuf> ROOT::Experimental::Detail::RPageSinkBuf::fBufferedColumns

private

Vector of buffered column pages. Indexed by column id.

Definition at line 108 of file RPageSinkBuf.hxx.

◆ fCounters

std::unique_ptr<RCounters> ROOT::Experimental::Detail::RPageSinkBuf::fCounters

private

Definition at line 100 of file RPageSinkBuf.hxx.

◆ fInnerModel

std::unique_ptr<RNTupleModel> ROOT::Experimental::Detail::RPageSinkBuf::fInnerModel

private

The buffered page sink maintains a copy of the RNTupleModel for the inner sink.

For the unbuffered case, the RNTupleModel is instead managed by a RNTupleWriter.

Definition at line 106 of file RPageSinkBuf.hxx.

◆ fInnerSink

std::unique_ptr<RPageSink> ROOT::Experimental::Detail::RPageSinkBuf::fInnerSink

private

The inner sink, responsible for actually performing I/O.

Definition at line 103 of file RPageSinkBuf.hxx.

◆ fMetrics

RNTupleMetrics ROOT::Experimental::Detail::RPageSinkBuf::fMetrics

private

Definition at line 101 of file RPageSinkBuf.hxx.

Libraries for ROOT::Experimental::Detail::RPageSinkBuf:

[legend]

The documentation for this class was generated from the following files:

tree/ntuple/v7/inc/ROOT/RPageSinkBuf.hxx
tree/ntuple/v7/src/RPageSinkBuf.cxx

Classes

Public Member Functions

Protected Member Functions

Private Attributes

Additional Inherited Members

Constructor & Destructor Documentation

◆ RPageSinkBuf() [1/3]

◆ RPageSinkBuf() [2/3]

◆ RPageSinkBuf() [3/3]

◆ ~RPageSinkBuf()

Member Function Documentation

◆ CommitClusterImpl()

◆ CommitDatasetImpl()

◆ CommitPageImpl()

◆ CommitSealedPageImpl()

◆ CreateImpl()

◆ GetMetrics()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ ReleasePage()

◆ ReservePage()

Member Data Documentation

◆ fBufferedColumns

◆ fCounters

◆ fInnerModel

◆ fInnerSink

◆ fMetrics