Abstract interface to write data into an ntuple.

The page sink takes the list of columns and afterwards a series of page commits and cluster commits. The user is responsible to commit clusters at a consistent point, i.e. when all pages corresponding to data up to the given entry number are committed.

An object of this class may either be a wrapper (for example a RPageSinkBuf) or a "persistent" sink, inheriting from RPagePersistentSink.

Definition at line 256 of file RPageStorage.hxx.

Classes
struct	RSealPageConfig
	Parameters for the SealPage() method. More...
class	RSinkGuard
	An RAII wrapper used to synchronize a page sink. See GetSinkGuard(). More...
struct	RStagedCluster
	Cluster that was staged, but not yet logically appended to the RNTuple. More...

Public Types
using	Callback_t = std::function<void(RPageSink &)>

Public Member Functions
	RPageSink (const RPageSink &)=delete
	RPageSink (RPageSink &&)=default
	RPageSink (std::string_view ntupleName, const ROOT::RNTupleWriteOptions &options)
	~RPageSink () override
virtual std::unique_ptr< RPageSink >	CloneAsHidden (std::string_view name, const RNTupleWriteOptions &opts) const=0
	Creates a new sink with the same underlying storage as this but writing to a different RNTuple named name.
virtual void	CommitAttributeSet (std::string_view attrSetName, const RNTupleLink &attrAnchorInfo)=0
	Adds the given anchor information (name + locator) into the main RNTuple's descriptor as an attribute set linked to it with the given name.
virtual std::uint64_t	CommitCluster (ROOT::NTupleSize_t nNewEntries)
	Finalize the current cluster and create a new one for the following data.
virtual void	CommitClusterGroup ()=0
	Write out the page locations (page list envelope) for all the committed clusters since the last call of CommitClusterGroup (or the beginning of writing).
RNTupleLink	CommitDataset ()
	Run the registered callbacks and finalize the current cluster and the entrire data set.
virtual void	CommitPage (ColumnHandle_t columnHandle, const ROOT::Internal::RPage &page)=0
	Write a page to the storage. The column must have been added before.
virtual void	CommitSealedPage (ROOT::DescriptorId_t physicalColumnId, const RPageStorage::RSealedPage &sealedPage)=0
	Write a preprocessed page to storage. The column must have been added before.
virtual void	CommitSealedPageV (std::span< RPageStorage::RSealedPageGroup > ranges)=0
	Write a vector of preprocessed pages to storage. The corresponding columns must have been added before.
virtual void	CommitStagedClusters (std::span< RStagedCluster > clusters)=0
	Commit staged clusters, logically appending them to the ntuple descriptor.
virtual void	CommitSuppressedColumn (ColumnHandle_t columnHandle)=0
	Commits a suppressed column for the current cluster.
void	DropColumn (ColumnHandle_t) final
	Unregisters a column.
virtual const ROOT::RNTupleDescriptor &	GetDescriptor () const=0
	Return the RNTupleDescriptor being constructed.
virtual ROOT::NTupleSize_t	GetNEntries () const=0
virtual RSinkGuard	GetSinkGuard ()
EPageStorageType	GetType () final
	Whether the concrete implementation is a sink or a source.
const ROOT::RNTupleWriteOptions &	GetWriteOptions () const
	Returns the sink's write options.
void	Init (RNTupleModel &model)
	Physically creates the storage container to hold the ntuple (e.g., a keys a TFile or an S3 bucket) Init() associates column handles to the columns referenced by the model.
bool	IsInitialized () const
RPageSink &	operator= (const RPageSink &)=delete
RPageSink &	operator= (RPageSink &&)=default
void	RegisterOnCommitDatasetCallback (Callback_t callback)
	The registered callback is executed at the beginning of CommitDataset();.
virtual ROOT::Internal::RPage	ReservePage (ColumnHandle_t columnHandle, std::size_t nElements)
	Get a new, empty page for the given column that can be filled with up to nElements; nElements must be larger than zero.
virtual RStagedCluster	StageCluster (ROOT::NTupleSize_t nNewEntries)=0
	Stage the current cluster and create a new one for the following data.
virtual void	UpdateExtraTypeInfo (const ROOT::RExtraTypeInfoDescriptor &extraTypeInfo)=0
	Adds an extra type information record to schema.
virtual void	UpdateSchema (const ROOT::Internal::RNTupleModelChangeset &changeset, ROOT::NTupleSize_t firstEntry)=0
	Incorporate incremental changes to the model into the ntuple descriptor.

Static Public Member Functions
static RSealedPage	SealPage (const RSealPageConfig &config)
	Seal a page using the provided info.

Protected Member Functions
virtual RNTupleLink	CommitDatasetImpl ()=0
virtual void	InitImpl (RNTupleModel &model)=0
RSealedPage	SealPage (const ROOT::Internal::RPage &page, const ROOT::Internal::RColumnElementBase &element)
	Helper for streaming a page.

Protected Attributes
bool	fIsInitialized = false
	Flag if sink was initialized.
std::unique_ptr< ROOT::RNTupleWriteOptions >	fOptions

Private Attributes
std::vector< Callback_t >	fOnDatasetCommitCallbacks
std::vector< unsigned char >	fSealPageBuffer
	Used as destination buffer in the simple SealPage overload.
RWritePageMemoryManager	fWritePageMemoryManager
	Used in ReservePage to maintain the page buffer budget.

#include <ROOT/RPageStorage.hxx>

Inheritance diagram for RPageSink:

Member Typedef Documentation

◆ Callback_t

using ROOT::Internal::RPageSink::Callback_t = std::function<void(RPageSink &)>

Definition at line 258 of file RPageStorage.hxx.

Constructor & Destructor Documentation

◆ RPageSink() [1/3]

ROOT::Internal::RPageSink::RPageSink	(	std::string_view	ntupleName,
		const ROOT::RNTupleWriteOptions &	options )

Definition at line 711 of file RPageStorage.cxx.

◆ RPageSink() [2/3]

ROOT::Internal::RPageSink::RPageSink ( const RPageSink & )

delete

◆ RPageSink() [3/3]

ROOT::Internal::RPageSink::RPageSink ( RPageSink && )

default

◆ ~RPageSink()

ROOT::Internal::RPageSink::~RPageSink ( )

override

Definition at line 717 of file RPageStorage.cxx.

Member Function Documentation

◆ CloneAsHidden()

virtual std::unique_ptr< RPageSink > ROOT::Internal::RPageSink::CloneAsHidden	(	std::string_view	name,
		const RNTupleWriteOptions &	opts ) const

pure virtual

Creates a new sink with the same underlying storage as this but writing to a different RNTuple named name.

Only one of the two sinks can safely write at the same time. The RNTuple written by this cloned sink will be stored in a hidden key (this is a convenient assumption we make now since this method is only used to create attribute RNTuples).

◆ CommitAttributeSet()

virtual void ROOT::Internal::RPageSink::CommitAttributeSet	(	std::string_view	attrSetName,
		const RNTupleLink &	attrAnchorInfo )

pure virtual

Adds the given anchor information (name + locator) into the main RNTuple's descriptor as an attribute set linked to it with the given name.

The attribute set must have already been written to storage via RNTupleAttrSetWriter::Commit(). Note that, by RNTuple specs, this is only legal to call on a non-attribute RNTuple's sink.

◆ CommitCluster()

virtual std::uint64_t ROOT::Internal::RPageSink::CommitCluster ( ROOT::NTupleSize_t nNewEntries )

inlinevirtual

Finalize the current cluster and create a new one for the following data.

Returns the number of bytes written to storage (excluding metadata).

Definition at line 379 of file RPageStorage.hxx.

◆ CommitClusterGroup()

virtual void ROOT::Internal::RPageSink::CommitClusterGroup ( )

pure virtual

Write out the page locations (page list envelope) for all the committed clusters since the last call of CommitClusterGroup (or the beginning of writing).

◆ CommitDataset()

ROOT::Internal::RNTupleLink ROOT::Internal::RPageSink::CommitDataset ( )

Run the registered callbacks and finalize the current cluster and the entrire data set.

Definition at line 774 of file RPageStorage.cxx.

◆ CommitDatasetImpl()

virtual RNTupleLink ROOT::Internal::RPageSink::CommitDatasetImpl ( )

protectedpure virtual

◆ CommitPage()

virtual void ROOT::Internal::RPageSink::CommitPage	(	ColumnHandle_t	columnHandle,
		const ROOT::Internal::RPage &	page )

pure virtual

Write a page to the storage. The column must have been added before.

◆ CommitSealedPage()

virtual void ROOT::Internal::RPageSink::CommitSealedPage	(	ROOT::DescriptorId_t	physicalColumnId,
		const RPageStorage::RSealedPage &	sealedPage )

pure virtual

Write a preprocessed page to storage. The column must have been added before.

◆ CommitSealedPageV()

virtual void ROOT::Internal::RPageSink::CommitSealedPageV ( std::span< RPageStorage::RSealedPageGroup > ranges )

pure virtual

Write a vector of preprocessed pages to storage. The corresponding columns must have been added before.

◆ CommitStagedClusters()

virtual void ROOT::Internal::RPageSink::CommitStagedClusters ( std::span< RStagedCluster > clusters )

pure virtual

Commit staged clusters, logically appending them to the ntuple descriptor.

◆ CommitSuppressedColumn()

virtual void ROOT::Internal::RPageSink::CommitSuppressedColumn ( ColumnHandle_t columnHandle )

pure virtual

Commits a suppressed column for the current cluster.

Can be called anytime before CommitCluster(). For any given column and cluster, there must be no calls to both CommitSuppressedColumn() and page commits.

◆ DropColumn()

void ROOT::Internal::RPageSink::DropColumn ( ColumnHandle_t columnHandle )

inlinefinalvirtual

Unregisters a column.

A page source decreases the reference counter for the corresponding active column. For a page sink, dropping columns is currently a no-op.

Implements ROOT::Internal::RPageStorage.

Definition at line 307 of file RPageStorage.hxx.

◆ GetDescriptor()

virtual const ROOT::RNTupleDescriptor & ROOT::Internal::RPageSink::GetDescriptor ( ) const

pure virtual

Return the RNTupleDescriptor being constructed.

◆ GetNEntries()

virtual ROOT::NTupleSize_t ROOT::Internal::RPageSink::GetNEntries ( ) const

pure virtual

◆ GetSinkGuard()

virtual RSinkGuard ROOT::Internal::RPageSink::GetSinkGuard ( )

inlinevirtual

Definition at line 433 of file RPageStorage.hxx.

◆ GetType()

EPageStorageType ROOT::Internal::RPageSink::GetType ( )

inlinefinalvirtual

Whether the concrete implementation is a sink or a source.

Implements ROOT::Internal::RPageStorage.

Definition at line 303 of file RPageStorage.hxx.

◆ GetWriteOptions()

const ROOT::RNTupleWriteOptions & ROOT::Internal::RPageSink::GetWriteOptions ( ) const

inline

Returns the sink's write options.

Definition at line 305 of file RPageStorage.hxx.

◆ Init()

void ROOT::Internal::RPageSink::Init ( RNTupleModel & model )

inline

Physically creates the storage container to hold the ntuple (e.g., a keys a TFile or an S3 bucket) Init() associates column handles to the columns referenced by the model.

Definition at line 318 of file RPageStorage.hxx.

◆ InitImpl()

virtual void ROOT::Internal::RPageSink::InitImpl ( RNTupleModel & model )

protectedpure virtual

◆ IsInitialized()

bool ROOT::Internal::RPageSink::IsInitialized ( ) const

inline

Definition at line 309 of file RPageStorage.hxx.

◆ operator=() [1/2]

RPageSink & ROOT::Internal::RPageSink::operator= ( const RPageSink & )

delete

◆ operator=() [2/2]

RPageSink & ROOT::Internal::RPageSink::operator= ( RPageSink && )

default

◆ RegisterOnCommitDatasetCallback()

void ROOT::Internal::RPageSink::RegisterOnCommitDatasetCallback ( Callback_t callback )

inline

The registered callback is executed at the beginning of CommitDataset();.

Definition at line 402 of file RPageStorage.hxx.

◆ ReservePage()

ROOT::Internal::RPage ROOT::Internal::RPageSink::ReservePage	(	ColumnHandle_t	columnHandle,
		std::size_t	nElements )

virtual

Get a new, empty page for the given column that can be filled with up to nElements; nElements must be larger than zero.

Definition at line 781 of file RPageStorage.cxx.

◆ SealPage() [1/2]

ROOT::Internal::RPageStorage::RSealedPage ROOT::Internal::RPageSink::SealPage	(	const ROOT::Internal::RPage &	page,
		const ROOT::Internal::RColumnElementBase &	element )

protected

Helper for streaming a page.

This is commonly used in derived, concrete page sinks. Note that if compressionSetting is 0 (uncompressed) and the page is mappable and not checksummed, the returned sealed page will point directly to the input page buffer. Otherwise, the sealed page references fSealPageBuffer. Thus, the buffer pointed to by the RSealedPage should never be freed.

Definition at line 757 of file RPageStorage.cxx.

◆ SealPage() [2/2]

ROOT::Internal::RPageStorage::RSealedPage ROOT::Internal::RPageSink::SealPage ( const RSealPageConfig & config )

static

Seal a page using the provided info.

Definition at line 719 of file RPageStorage.cxx.

◆ StageCluster()

virtual RStagedCluster ROOT::Internal::RPageSink::StageCluster ( ROOT::NTupleSize_t nNewEntries )

pure virtual

Stage the current cluster and create a new one for the following data.

Returns the object that must be passed to CommitStagedClusters to logically append the staged cluster to the ntuple descriptor.

◆ UpdateExtraTypeInfo()

virtual void ROOT::Internal::RPageSink::UpdateExtraTypeInfo ( const ROOT::RExtraTypeInfoDescriptor & extraTypeInfo )

pure virtual

Adds an extra type information record to schema.

The extra type information will be written to the extension header. The information in the record will be merged with the existing information, e.g. duplicate streamer info records will be removed. This method is called by the "on commit dataset" callback registered by specific fields (e.g., streamer field) and during merging.

◆ UpdateSchema()

virtual void ROOT::Internal::RPageSink::UpdateSchema	(	const ROOT::Internal::RNTupleModelChangeset &	changeset,
		ROOT::NTupleSize_t	firstEntry )

pure virtual

Incorporate incremental changes to the model into the ntuple descriptor.

This happens, e.g. if new fields were added after the initial call to RPageSink::Init(RNTupleModel &). firstEntry specifies the global index for the first stored element in the added columns.

Member Data Documentation

◆ fIsInitialized

bool ROOT::Internal::RPageSink::fIsInitialized = false

protected

Flag if sink was initialized.

Definition at line 279 of file RPageStorage.hxx.

◆ fOnDatasetCommitCallbacks

std::vector<Callback_t> ROOT::Internal::RPageSink::fOnDatasetCommitCallbacks

private

Definition at line 288 of file RPageStorage.hxx.

◆ fOptions

std::unique_ptr<ROOT::RNTupleWriteOptions> ROOT::Internal::RPageSink::fOptions

protected

Definition at line 276 of file RPageStorage.hxx.

◆ fSealPageBuffer

std::vector<unsigned char> ROOT::Internal::RPageSink::fSealPageBuffer

private

Used as destination buffer in the simple SealPage overload.

Definition at line 289 of file RPageStorage.hxx.

◆ fWritePageMemoryManager

RWritePageMemoryManager ROOT::Internal::RPageSink::fWritePageMemoryManager

private

Used in ReservePage to maintain the page buffer budget.

Definition at line 292 of file RPageStorage.hxx.

The documentation for this class was generated from the following files:

tree/ntuple/inc/ROOT/RPageStorage.hxx
tree/ntuple/src/RPageStorage.cxx

Classes

Public Types

Public Member Functions

Static Public Member Functions

Protected Member Functions

Protected Attributes

Private Attributes

Member Typedef Documentation

◆ Callback_t

Constructor & Destructor Documentation

◆ RPageSink() [1/3]

◆ RPageSink() [2/3]

◆ RPageSink() [3/3]

◆ ~RPageSink()

Member Function Documentation

◆ CloneAsHidden()

◆ CommitAttributeSet()

◆ CommitCluster()

◆ CommitClusterGroup()

◆ CommitDataset()

◆ CommitDatasetImpl()

◆ CommitPage()

◆ CommitSealedPage()

◆ CommitSealedPageV()

◆ CommitStagedClusters()

◆ CommitSuppressedColumn()

◆ DropColumn()

◆ GetDescriptor()

◆ GetNEntries()

◆ GetSinkGuard()

◆ GetType()

◆ GetWriteOptions()

◆ Init()

◆ InitImpl()

◆ IsInitialized()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ RegisterOnCommitDatasetCallback()

◆ ReservePage()

◆ SealPage() [1/2]

◆ SealPage() [2/2]

◆ StageCluster()

◆ UpdateExtraTypeInfo()

◆ UpdateSchema()

Member Data Documentation

◆ fIsInitialized

◆ fOnDatasetCommitCallbacks

◆ fOptions

◆ fSealPageBuffer

◆ fWritePageMemoryManager