Storage provider that writes ntuple pages to into a DAOS container.

Currently, an object is allocated for ntuple metadata (anchor/header/footer). Objects can correspond to pages or clusters of pages depending on the RNTuple-DAOS mapping strategy.

Definition at line 99 of file RPageStorageDaos.hxx.

Public Types
using	Callback_t = std::function<void(RPageSink &)>
using	ColumnHandle_t = RColumnHandle
	The column handle identifies a column with the current open page storage.
using	SealedPageSequence_t = std::deque<RSealedPage>

Public Member Functions
	RPageSinkDaos (std::string_view ntupleName, std::string_view uri, const ROOT::RNTupleWriteOptions &options)
	~RPageSinkDaos () override
ColumnHandle_t	AddColumn (ROOT::DescriptorId_t fieldId, ROOT::Internal::RColumn &column) final
	Register a new column.
std::unique_ptr< ROOT::Internal::RPageSink >	CloneAsHidden (std::string_view name, const ROOT::RNTupleWriteOptions &opts) const final
	Creates a new sink with the same underlying storage as this but writing to a different RNTuple named name.
void	CommitAttributeSet (std::string_view attrSetName, const RNTupleLink &attrAnchorInfo) final
	Adds the given anchor information (name + locator) into the main RNTuple's descriptor as an attribute set linked to it with the given name.
virtual std::uint64_t	CommitCluster (ROOT::NTupleSize_t nNewEntries)
	Finalize the current cluster and create a new one for the following data.
void	CommitClusterGroup () final
	Write out the page locations (page list envelope) for all the committed clusters since the last call of CommitClusterGroup (or the beginning of writing).
RNTupleLink	CommitDataset ()
	Run the registered callbacks and finalize the current cluster and the entrire data set.
void	CommitPage (ColumnHandle_t columnHandle, const ROOT::Internal::RPage &page) final
	Write a page to the storage. The column must have been added before.
void	CommitSealedPage (ROOT::DescriptorId_t physicalColumnId, const RPageStorage::RSealedPage &sealedPage) final
	Write a preprocessed page to storage. The column must have been added before.
void	CommitSealedPageV (std::span< RPageStorage::RSealedPageGroup > ranges) final
	Write a vector of preprocessed pages to storage. The corresponding columns must have been added before.
void	CommitStagedClusters (std::span< RStagedCluster > clusters) final
	Commit staged clusters, logically appending them to the ntuple descriptor.
void	CommitSuppressedColumn (ColumnHandle_t columnHandle) final
	Commits a suppressed column for the current cluster.
void	DropColumn (ColumnHandle_t) final
	Unregisters a column.
ROOT::DescriptorId_t	GetColumnId (ColumnHandle_t columnHandle) const
const ROOT::RNTupleDescriptor &	GetDescriptor () const final
	Return the RNTupleDescriptor being constructed.
virtual ROOT::Experimental::Detail::RNTupleMetrics &	GetMetrics ()
	Returns the default metrics object.
ROOT::NTupleSize_t	GetNEntries () const final
const std::string &	GetNTupleName () const
	Returns the NTuple name.
virtual RSinkGuard	GetSinkGuard ()
EPageStorageType	GetType () final
	Whether the concrete implementation is a sink or a source.
const ROOT::RNTupleWriteOptions &	GetWriteOptions () const
	Returns the sink's write options.
void	Init (RNTupleModel &model)
	Physically creates the storage container to hold the ntuple (e.g., a keys a TFile or an S3 bucket) Init() associates column handles to the columns referenced by the model.
std::unique_ptr< RNTupleModel >	InitFromDescriptor (const ROOT::RNTupleDescriptor &descriptor, bool copyClusters)
	Initialize sink based on an existing descriptor and fill into the descriptor builder, optionally copying over the descriptor's clusters to this sink's descriptor.
bool	IsInitialized () const
void	RegisterOnCommitDatasetCallback (Callback_t callback)
	The registered callback is executed at the beginning of CommitDataset();.
virtual ROOT::Internal::RPage	ReservePage (ColumnHandle_t columnHandle, std::size_t nElements)
	Get a new, empty page for the given column that can be filled with up to nElements; nElements must be larger than zero.
void	SetTaskScheduler (RTaskScheduler *taskScheduler)
RStagedCluster	StageCluster (ROOT::NTupleSize_t nNewEntries) final
	Stage the current cluster and create a new one for the following data.
void	UpdateExtraTypeInfo (const ROOT::RExtraTypeInfoDescriptor &extraTypeInfo) final
	Adds an extra type information record to schema.
void	UpdateSchema (const ROOT::Internal::RNTupleModelChangeset &changeset, ROOT::NTupleSize_t firstEntry) override
	Incorporate incremental changes to the model into the ntuple descriptor.

Static Public Member Functions
static std::unique_ptr< RPageSink >	Create (std::string_view ntupleName, std::string_view location, const ROOT::RNTupleWriteOptions &options=ROOT::RNTupleWriteOptions())
	Guess the concrete derived page source from the location.
static RSealedPage	SealPage (const RSealPageConfig &config)
	Seal a page using the provided info.

Static Public Attributes
static constexpr std::size_t	kNBytesPageChecksum = sizeof(std::uint64_t)
	The page checksum is a 64bit xxhash3.

Protected Member Functions
RNTupleLocator	CommitClusterGroupImpl (unsigned char *serializedPageList, std::uint32_t length) final
	Returns the locator of the page list envelope of the given buffer that contains the serialized page list.
RNTupleLink	CommitDatasetImpl () final
ROOT::Internal::RNTupleLink	CommitDatasetImpl (unsigned char *serializedFooter, std::uint32_t length) final
RNTupleLocator	CommitPageImpl (ColumnHandle_t columnHandle, const ROOT::Internal::RPage &page) final
RNTupleLocator	CommitSealedPageImpl (ROOT::DescriptorId_t physicalColumnId, const RPageStorage::RSealedPage &sealedPage) final
std::vector< RNTupleLocator >	CommitSealedPageVImpl (std::span< RPageStorage::RSealedPageGroup > ranges, const std::vector< bool > &mask) final
	Vector commit of preprocessed pages.
void	EnableDefaultMetrics (const std::string &prefix)
	Enables the default set of metrics provided by RPageSink.
void	InitImpl (RNTupleModel &model) final
	Updates the descriptor and calls InitImpl() that handles the backend-specific details (file, DAOS, etc.).
void	InitImpl (unsigned char *serializedHeader, std::uint32_t length) final
RSealedPage	SealPage (const ROOT::Internal::RPage &page, const ROOT::Internal::RColumnElementBase &element)
	Helper for streaming a page.
std::uint64_t	StageClusterImpl () final
	Returns the number of bytes written to storage (excluding metadata).
void	WaitForAllTasks ()
void	WriteNTupleAnchor ()
void	WriteNTupleFooter (const void *data, size_t nbytes, size_t lenFooter)
void	WriteNTupleHeader (const void *data, size_t nbytes, size_t lenHeader)

Protected Attributes
std::unique_ptr< RCounters >	fCounters
ROOT::Internal::RNTupleDescriptorBuilder	fDescriptorBuilder
RFeatures	fFeatures
bool	fIsInitialized = false
	Flag if sink was initialized.
ROOT::Experimental::Detail::RNTupleMetrics	fMetrics
std::string	fNTupleName
std::unique_ptr< ROOT::RNTupleWriteOptions >	fOptions
std::unique_ptr< ROOT::Internal::RPageAllocator >	fPageAllocator
	For the time being, we will use the heap allocator for all sources and sinks. This may change in the future.
RTaskScheduler *	fTaskScheduler = nullptr

Private Attributes
std::atomic< std::uint64_t >	fClusterGroupId {0}
	Cluster group counter for the next committed cluster pagelist; incremented in CommitClusterGroupImpl().
std::unique_ptr< RDaosContainer >	fDaosContainer
	Underlying DAOS container.
ROOT::Internal::RNTupleSerializer::StreamerInfoMap_t	fInfosOfStreamerFields
	Union of the streamer info records that are sent from streamer fields to the sink before committing the dataset.
std::uint64_t	fNBytesCurrentCluster {0}
	Tracks the number of bytes committed to the current cluster.
std::uint64_t	fNextClusterInGroup = 0
	Remembers the starting cluster id for the next cluster group.
RDaosNTupleAnchor	fNTupleAnchor
ntuple_index_t	fNTupleIndex {0}
std::vector< Callback_t >	fOnDatasetCommitCallbacks
std::vector< ROOT::RClusterDescriptor::RColumnRange >	fOpenColumnRanges
	Keeps track of the number of elements in the currently open cluster. Indexed by column id.
std::vector< ROOT::RClusterDescriptor::RPageRange >	fOpenPageRanges
	Keeps track of the written pages in the currently open cluster. Indexed by column id.
std::atomic< std::uint64_t >	fPageId {0}
	Page identifier for the next committed page; it is automatically incremented in CommitSealedPageImpl().
ROOT::NTupleSize_t	fPrevClusterNEntries = 0
	Used to calculate the number of entries in the current cluster.
std::vector< unsigned char >	fSealPageBuffer
	Used as destination buffer in the simple SealPage overload.
ROOT::Internal::RNTupleSerializer::RContext	fSerializationContext
	Used to map the IDs of the descriptor to the physical IDs issued during header/footer serialization.
std::string	fURI
	A URI to a DAOS pool of the form 'daos://pool-label/container-label'.
RWritePageMemoryManager	fWritePageMemoryManager
	Used in ReservePage to maintain the page buffer budget.

#include <ROOT/RPageStorageDaos.hxx>

Inheritance diagram for ROOT::Experimental::Internal::RPageSinkDaos:

Member Typedef Documentation

◆ Callback_t

using ROOT::Internal::RPageSink::Callback_t = std::function<void(RPageSink &)>

inherited

Definition at line 258 of file RPageStorage.hxx.

◆ ColumnHandle_t

using ROOT::Internal::RPageStorage::ColumnHandle_t = RColumnHandle

inherited

The column handle identifies a column with the current open page storage.

Definition at line 180 of file RPageStorage.hxx.

◆ SealedPageSequence_t

using ROOT::Internal::RPageStorage::SealedPageSequence_t = std::deque<RSealedPage>

inherited

Definition at line 130 of file RPageStorage.hxx.

Constructor & Destructor Documentation

◆ RPageSinkDaos()

ROOT::Experimental::Internal::RPageSinkDaos::RPageSinkDaos	(	std::string_view	ntupleName,
		std::string_view	uri,
		const ROOT::RNTupleWriteOptions &	options )

Definition at line 259 of file RPageStorageDaos.cxx.

◆ ~RPageSinkDaos()

ROOT::Experimental::Internal::RPageSinkDaos::~RPageSinkDaos ( )

overridedefault

Member Function Documentation

◆ AddColumn()

ROOT::Internal::RPageStorage::ColumnHandle_t ROOT::Internal::RPagePersistentSink::AddColumn	(	ROOT::DescriptorId_t	fieldId,
		ROOT::Internal::RColumn &	column )

finalvirtualinherited

Register a new column.

When reading, the column must exist in the ntuple on disk corresponding to the metadata. When writing, every column can only be attached once.

Implements ROOT::Internal::RPageStorage.

Definition at line 824 of file RPageStorage.cxx.

◆ CloneAsHidden()

std::unique_ptr< ROOT::Internal::RPageSink > ROOT::Experimental::Internal::RPageSinkDaos::CloneAsHidden	(	std::string_view	name,
		const ROOT::RNTupleWriteOptions &	opts ) const

finalvirtual

Creates a new sink with the same underlying storage as this but writing to a different RNTuple named name.

Only one of the two sinks can safely write at the same time. The RNTuple written by this cloned sink will be stored in a hidden key (this is a convenient assumption we make now since this method is only used to create attribute RNTuples).

Implements ROOT::Internal::RPageSink.

Definition at line 453 of file RPageStorageDaos.cxx.

◆ CommitAttributeSet()

void ROOT::Internal::RPagePersistentSink::CommitAttributeSet	(	std::string_view	attrSetName,
		const RNTupleLink &	attrAnchorInfo )

finalvirtualinherited

Adds the given anchor information (name + locator) into the main RNTuple's descriptor as an attribute set linked to it with the given name.

The attribute set must have already been written to storage via RNTupleAttrSetWriter::Commit(). Note that, by RNTuple specs, this is only legal to call on a non-attribute RNTuple's sink.

Implements ROOT::Internal::RPageSink.

Definition at line 1274 of file RPageStorage.cxx.

◆ CommitCluster()

virtual std::uint64_t ROOT::Internal::RPageSink::CommitCluster ( ROOT::NTupleSize_t nNewEntries )

inlinevirtualinherited

Finalize the current cluster and create a new one for the following data.

Returns the number of bytes written to storage (excluding metadata).

Reimplemented in ROOT::Internal::RPageSinkBuf.

Definition at line 379 of file RPageStorage.hxx.

◆ CommitClusterGroup()

void ROOT::Internal::RPagePersistentSink::CommitClusterGroup ( )

finalvirtualinherited

Write out the page locations (page list envelope) for all the committed clusters since the last call of CommitClusterGroup (or the beginning of writing).

Implements ROOT::Internal::RPageSink.

Definition at line 1232 of file RPageStorage.cxx.

◆ CommitClusterGroupImpl()

ROOT::RNTupleLocator ROOT::Experimental::Internal::RPageSinkDaos::CommitClusterGroupImpl	(	unsigned char *	serializedPageList,
		std::uint32_t	length )

finalprotectedvirtual

Returns the locator of the page list envelope of the given buffer that contains the serialized page list.

Typically, the implementation takes care of compressing and writing the provided buffer.

Implements ROOT::Internal::RPagePersistentSink.

Definition at line 391 of file RPageStorageDaos.cxx.

◆ CommitDataset()

ROOT::Internal::RNTupleLink ROOT::Internal::RPageSink::CommitDataset ( )

inherited

Run the registered callbacks and finalize the current cluster and the entrire data set.

Definition at line 774 of file RPageStorage.cxx.

◆ CommitDatasetImpl() [1/2]

ROOT::Internal::RNTupleLink ROOT::Internal::RPagePersistentSink::CommitDatasetImpl ( )

finalprotectedvirtual

Returns: The locator and length of the written anchor.

Reimplemented from ROOT::Internal::RPagePersistentSink.

Definition at line 555 of file RPageStorage.cxx.

◆ CommitDatasetImpl() [2/2]

ROOT::Internal::RNTupleLink ROOT::Experimental::Internal::RPageSinkDaos::CommitDatasetImpl	(	unsigned char *	serializedFooter,
		std::uint32_t	length )

finalprotectedvirtual

Implements ROOT::Internal::RPagePersistentSink.

Definition at line 412 of file RPageStorageDaos.cxx.

◆ CommitPage()

void ROOT::Internal::RPagePersistentSink::CommitPage	(	ColumnHandle_t	columnHandle,
		const ROOT::Internal::RPage &	page )

finalvirtualinherited

Write a page to the storage. The column must have been added before.

Implements ROOT::Internal::RPageSink.

Definition at line 1056 of file RPageStorage.cxx.

◆ CommitPageImpl()

ROOT::RNTupleLocator ROOT::Experimental::Internal::RPageSinkDaos::CommitPageImpl	(	ColumnHandle_t	columnHandle,
		const ROOT::Internal::RPage &	page )

finalprotectedvirtual

Implements ROOT::Internal::RPagePersistentSink.

Definition at line 296 of file RPageStorageDaos.cxx.

◆ CommitSealedPage()

void ROOT::Internal::RPagePersistentSink::CommitSealedPage	(	ROOT::DescriptorId_t	physicalColumnId,
		const RPageStorage::RSealedPage &	sealedPage )

finalvirtualinherited

Write a preprocessed page to storage. The column must have been added before.

Implements ROOT::Internal::RPageSink.

Definition at line 1067 of file RPageStorage.cxx.

◆ CommitSealedPageImpl()

ROOT::RNTupleLocator ROOT::Experimental::Internal::RPageSinkDaos::CommitSealedPageImpl	(	ROOT::DescriptorId_t	physicalColumnId,
		const RPageStorage::RSealedPage &	sealedPage )

finalprotectedvirtual

Implements ROOT::Internal::RPagePersistentSink.

Definition at line 311 of file RPageStorageDaos.cxx.

◆ CommitSealedPageV()

void ROOT::Internal::RPagePersistentSink::CommitSealedPageV ( std::span< RPageStorage::RSealedPageGroup > ranges )

finalvirtualinherited

Write a vector of preprocessed pages to storage. The corresponding columns must have been added before.

Implements ROOT::Internal::RPageSink.

Definition at line 1096 of file RPageStorage.cxx.

◆ CommitSealedPageVImpl()

std::vector< ROOT::RNTupleLocator > ROOT::Experimental::Internal::RPageSinkDaos::CommitSealedPageVImpl	(	std::span< RPageStorage::RSealedPageGroup >	ranges,
		const std::vector< bool > &	mask )

finalprotectedvirtual

Vector commit of preprocessed pages.

The ranges array specifies a range of sealed pages to be committed for each column. The returned vector contains, in order, the RNTupleLocator for each page on each range in ranges, i.e. the first N entries refer to the N pages in ranges[0], followed by M entries that refer to the M pages in ranges[1], etc. The mask allows to skip writing out certain pages. The vector has the size of all the pages. For every false value in the mask, the corresponding locator is skipped (missing) in the output vector. The default is to call CommitSealedPageImpl for each page; derived classes may provide an optimized implementation though.

Reimplemented from ROOT::Internal::RPagePersistentSink.

Definition at line 335 of file RPageStorageDaos.cxx.

◆ CommitStagedClusters()

void ROOT::Internal::RPagePersistentSink::CommitStagedClusters ( std::span< RStagedCluster > clusters )

finalvirtualinherited

Commit staged clusters, logically appending them to the ntuple descriptor.

Implements ROOT::Internal::RPageSink.

Definition at line 1195 of file RPageStorage.cxx.

◆ CommitSuppressedColumn()

void ROOT::Internal::RPagePersistentSink::CommitSuppressedColumn ( ColumnHandle_t columnHandle )

finalvirtualinherited

Commits a suppressed column for the current cluster.

Can be called anytime before CommitCluster(). For any given column and cluster, there must be no calls to both CommitSuppressedColumn() and page commits.

Implements ROOT::Internal::RPageSink.

Definition at line 1051 of file RPageStorage.cxx.

◆ Create()

std::unique_ptr< ROOT::Internal::RPageSink > ROOT::Internal::RPagePersistentSink::Create	(	std::string_view	ntupleName,
		std::string_view	location,
		const ROOT::RNTupleWriteOptions &	options = ROOT::RNTupleWriteOptions() )

staticinherited

Guess the concrete derived page source from the location.

Definition at line 794 of file RPageStorage.cxx.

◆ DropColumn()

void ROOT::Internal::RPageSink::DropColumn ( ColumnHandle_t columnHandle )

inlinefinalvirtualinherited

Unregisters a column.

A page source decreases the reference counter for the corresponding active column. For a page sink, dropping columns is currently a no-op.

Implements ROOT::Internal::RPageStorage.

Definition at line 307 of file RPageStorage.hxx.

◆ EnableDefaultMetrics()

void ROOT::Internal::RPagePersistentSink::EnableDefaultMetrics ( const std::string & prefix )

protectedinherited

Enables the default set of metrics provided by RPageSink.

prefix will be used as the prefix for the counters registered in the internal RNTupleMetrics object. This set of counters can be extended by a subclass by calling fMetrics.MakeCounter<...>().

A subclass using the default set of metrics is always responsible for updating the counters appropriately, e.g. fCounters->fNPageCommited.Inc()

Definition at line 1320 of file RPageStorage.cxx.

◆ GetColumnId()

ROOT::DescriptorId_t ROOT::Internal::RPageStorage::GetColumnId ( ColumnHandle_t columnHandle ) const

inlineinherited

Definition at line 188 of file RPageStorage.hxx.

◆ GetDescriptor()

const ROOT::RNTupleDescriptor & ROOT::Internal::RPagePersistentSink::GetDescriptor ( ) const

inlinefinalvirtualinherited

Return the RNTupleDescriptor being constructed.

Implements ROOT::Internal::RPageSink.

Definition at line 530 of file RPageStorage.hxx.

◆ GetMetrics()

virtual ROOT::Experimental::Detail::RNTupleMetrics & ROOT::Internal::RPageStorage::GetMetrics ( )

inlinevirtualinherited

Returns the default metrics object.

Subclasses might alternatively provide their own metrics object by overriding this.

Definition at line 192 of file RPageStorage.hxx.

◆ GetNEntries()

ROOT::NTupleSize_t ROOT::Internal::RPagePersistentSink::GetNEntries ( ) const

inlinefinalvirtualinherited

Implements ROOT::Internal::RPageSink.

Definition at line 532 of file RPageStorage.hxx.

◆ GetNTupleName()

const std::string & ROOT::Internal::RPageStorage::GetNTupleName ( ) const

inlineinherited

Returns the NTuple name.

Definition at line 195 of file RPageStorage.hxx.

◆ GetSinkGuard()

virtual RSinkGuard ROOT::Internal::RPageSink::GetSinkGuard ( )

inlinevirtualinherited

Definition at line 433 of file RPageStorage.hxx.

◆ GetType()

EPageStorageType ROOT::Internal::RPageSink::GetType ( )

inlinefinalvirtualinherited

Whether the concrete implementation is a sink or a source.

Implements ROOT::Internal::RPageStorage.

Definition at line 303 of file RPageStorage.hxx.

◆ GetWriteOptions()

const ROOT::RNTupleWriteOptions & ROOT::Internal::RPageSink::GetWriteOptions ( ) const

inlineinherited

Returns the sink's write options.

Definition at line 305 of file RPageStorage.hxx.

◆ Init()

void ROOT::Internal::RPageSink::Init ( RNTupleModel & model )

inlineinherited

Physically creates the storage container to hold the ntuple (e.g., a keys a TFile or an S3 bucket) Init() associates column handles to the columns referenced by the model.

Definition at line 318 of file RPageStorage.hxx.

◆ InitFromDescriptor()

std::unique_ptr< ROOT::RNTupleModel > ROOT::Internal::RPagePersistentSink::InitFromDescriptor	(	const ROOT::RNTupleDescriptor &	descriptor,
		bool	copyClusters )

nodiscardinherited

Initialize sink based on an existing descriptor and fill into the descriptor builder, optionally copying over the descriptor's clusters to this sink's descriptor.

Returns: The model created from the new sink's descriptor. This model should be kept alive for at least as long as the sink.

Definition at line 980 of file RPageStorage.cxx.

◆ InitImpl() [1/2]

void ROOT::Internal::RPagePersistentSink::InitImpl ( ROOT::RNTupleModel & model )

finalprotectedvirtual

Updates the descriptor and calls InitImpl() that handles the backend-specific details (file, DAOS, etc.).

Reimplemented from ROOT::Internal::RPagePersistentSink.

Definition at line 535 of file RPageStorage.cxx.

◆ InitImpl() [2/2]

void ROOT::Experimental::Internal::RPageSinkDaos::InitImpl	(	unsigned char *	serializedHeader,
		std::uint32_t	length )

finalprotectedvirtual

Implements ROOT::Internal::RPagePersistentSink.

Definition at line 273 of file RPageStorageDaos.cxx.

◆ IsInitialized()

bool ROOT::Internal::RPageSink::IsInitialized ( ) const

inlineinherited

Definition at line 309 of file RPageStorage.hxx.

◆ RegisterOnCommitDatasetCallback()

void ROOT::Internal::RPageSink::RegisterOnCommitDatasetCallback ( Callback_t callback )

inlineinherited

The registered callback is executed at the beginning of CommitDataset();.

Definition at line 402 of file RPageStorage.hxx.

◆ ReservePage()

ROOT::Internal::RPage ROOT::Internal::RPageSink::ReservePage	(	ColumnHandle_t	columnHandle,
		std::size_t	nElements )

virtualinherited

Get a new, empty page for the given column that can be filled with up to nElements; nElements must be larger than zero.

Reimplemented in ROOT::Internal::RPageSinkBuf.

Definition at line 781 of file RPageStorage.cxx.

◆ SealPage() [1/2]

ROOT::Internal::RPageStorage::RSealedPage ROOT::Internal::RPageSink::SealPage	(	const ROOT::Internal::RPage &	page,
		const ROOT::Internal::RColumnElementBase &	element )

protectedinherited

Helper for streaming a page.

This is commonly used in derived, concrete page sinks. Note that if compressionSetting is 0 (uncompressed) and the page is mappable and not checksummed, the returned sealed page will point directly to the input page buffer. Otherwise, the sealed page references fSealPageBuffer. Thus, the buffer pointed to by the RSealedPage should never be freed.

Definition at line 757 of file RPageStorage.cxx.

◆ SealPage() [2/2]

ROOT::Internal::RPageStorage::RSealedPage ROOT::Internal::RPageSink::SealPage ( const RSealPageConfig & config )

staticinherited

Seal a page using the provided info.

Definition at line 719 of file RPageStorage.cxx.

◆ SetTaskScheduler()

void ROOT::Internal::RPageStorage::SetTaskScheduler ( RTaskScheduler * taskScheduler )

inlineinherited

Definition at line 197 of file RPageStorage.hxx.

◆ StageCluster()

ROOT::Internal::RPageSink::RStagedCluster ROOT::Internal::RPagePersistentSink::StageCluster ( ROOT::NTupleSize_t nNewEntries )

finalvirtualinherited

Stage the current cluster and create a new one for the following data.

Returns the object that must be passed to CommitStagedClusters to logically append the staged cluster to the ntuple descriptor.

Implements ROOT::Internal::RPageSink.

Definition at line 1166 of file RPageStorage.cxx.

◆ StageClusterImpl()

std::uint64_t ROOT::Experimental::Internal::RPageSinkDaos::StageClusterImpl ( )

finalprotectedvirtual

Returns the number of bytes written to storage (excluding metadata).

Implements ROOT::Internal::RPagePersistentSink.

Definition at line 385 of file RPageStorageDaos.cxx.

◆ UpdateExtraTypeInfo()

void ROOT::Internal::RPagePersistentSink::UpdateExtraTypeInfo ( const ROOT::RExtraTypeInfoDescriptor & extraTypeInfo )

finalvirtualinherited

Adds an extra type information record to schema.

The extra type information will be written to the extension header. The information in the record will be merged with the existing information, e.g. duplicate streamer info records will be removed. This method is called by the "on commit dataset" callback registered by specific fields (e.g., streamer field) and during merging.

Implements ROOT::Internal::RPageSink.

Definition at line 943 of file RPageStorage.cxx.

◆ UpdateSchema()

void ROOT::Internal::RPagePersistentSink::UpdateSchema	(	const ROOT::Internal::RNTupleModelChangeset &	changeset,
		ROOT::NTupleSize_t	firstEntry )

overridevirtualinherited

Incorporate incremental changes to the model into the ntuple descriptor.

This happens, e.g. if new fields were added after the initial call to RPageSink::Init(RNTupleModel &). firstEntry specifies the global index for the first stored element in the added columns.

Implements ROOT::Internal::RPageSink.

Reimplemented in ROOT::Internal::RPageSinkFile.

Definition at line 845 of file RPageStorage.cxx.

◆ WaitForAllTasks()

void ROOT::Internal::RPageStorage::WaitForAllTasks ( )

inlineprotectedinherited

Definition at line 153 of file RPageStorage.hxx.

◆ WriteNTupleAnchor()

void ROOT::Experimental::Internal::RPageSinkDaos::WriteNTupleAnchor ( )

protected

Definition at line 442 of file RPageStorageDaos.cxx.

◆ WriteNTupleFooter()

void ROOT::Experimental::Internal::RPageSinkDaos::WriteNTupleFooter	(	const void *	data,
		size_t	nbytes,
		size_t	lenFooter )

protected

Definition at line 433 of file RPageStorageDaos.cxx.

◆ WriteNTupleHeader()

void ROOT::Experimental::Internal::RPageSinkDaos::WriteNTupleHeader	(	const void *	data,
		size_t	nbytes,
		size_t	lenHeader )

protected

Definition at line 424 of file RPageStorageDaos.cxx.

Member Data Documentation

◆ fClusterGroupId

std::atomic<std::uint64_t> ROOT::Experimental::Internal::RPageSinkDaos::fClusterGroupId {0}

private

Cluster group counter for the next committed cluster pagelist; incremented in CommitClusterGroupImpl().

Definition at line 109 of file RPageStorageDaos.hxx.

◆ fCounters

std::unique_ptr<RCounters> ROOT::Internal::RPagePersistentSink::fCounters

protectedinherited

Definition at line 483 of file RPageStorage.hxx.

◆ fDaosContainer

std::unique_ptr<RDaosContainer> ROOT::Experimental::Internal::RPageSinkDaos::fDaosContainer

private

Underlying DAOS container.

An internal std::shared_ptr keep the pool connection alive. ISO C++ ensures the correct destruction order, i.e., ~RDaosContainer is invoked first (which calls daos_cont_close(); the destructor for the std::shared_ptr<RDaosPool> is invoked after (which calls daos_pool_disconect()).

Definition at line 105 of file RPageStorageDaos.hxx.

◆ fDescriptorBuilder

ROOT::Internal::RNTupleDescriptorBuilder ROOT::Internal::RPagePersistentSink::fDescriptorBuilder

protectedinherited

Definition at line 471 of file RPageStorage.hxx.

◆ fFeatures

RFeatures ROOT::Internal::RPagePersistentSink::fFeatures

protectedinherited

Definition at line 470 of file RPageStorage.hxx.

◆ fInfosOfStreamerFields

ROOT::Internal::RNTupleSerializer::StreamerInfoMap_t ROOT::Internal::RPagePersistentSink::fInfosOfStreamerFields

privateinherited

Union of the streamer info records that are sent from streamer fields to the sink before committing the dataset.

Definition at line 462 of file RPageStorage.hxx.

◆ fIsInitialized

bool ROOT::Internal::RPageSink::fIsInitialized = false

protectedinherited

Flag if sink was initialized.

Definition at line 279 of file RPageStorage.hxx.

◆ fMetrics

ROOT::Experimental::Detail::RNTupleMetrics ROOT::Internal::RPageStorage::fMetrics

protectedinherited

Definition at line 146 of file RPageStorage.hxx.

◆ fNBytesCurrentCluster

std::uint64_t ROOT::Experimental::Internal::RPageSinkDaos::fNBytesCurrentCluster {0}

private

Tracks the number of bytes committed to the current cluster.

Definition at line 113 of file RPageStorageDaos.hxx.

◆ fNextClusterInGroup

std::uint64_t ROOT::Internal::RPagePersistentSink::fNextClusterInGroup = 0

privateinherited

Remembers the starting cluster id for the next cluster group.

Definition at line 453 of file RPageStorage.hxx.

◆ fNTupleAnchor

RDaosNTupleAnchor ROOT::Experimental::Internal::RPageSinkDaos::fNTupleAnchor

private

Definition at line 115 of file RPageStorageDaos.hxx.

◆ fNTupleIndex

ntuple_index_t ROOT::Experimental::Internal::RPageSinkDaos::fNTupleIndex {0}

private

Definition at line 116 of file RPageStorageDaos.hxx.

◆ fNTupleName

std::string ROOT::Internal::RPageStorage::fNTupleName

protectedinherited

Definition at line 151 of file RPageStorage.hxx.

◆ fOnDatasetCommitCallbacks

std::vector<Callback_t> ROOT::Internal::RPageSink::fOnDatasetCommitCallbacks

privateinherited

Definition at line 288 of file RPageStorage.hxx.

◆ fOpenColumnRanges

std::vector<ROOT::RClusterDescriptor::RColumnRange> ROOT::Internal::RPagePersistentSink::fOpenColumnRanges

privateinherited

Keeps track of the number of elements in the currently open cluster. Indexed by column id.

Definition at line 457 of file RPageStorage.hxx.

◆ fOpenPageRanges

std::vector<ROOT::RClusterDescriptor::RPageRange> ROOT::Internal::RPagePersistentSink::fOpenPageRanges

privateinherited

Keeps track of the written pages in the currently open cluster. Indexed by column id.

Definition at line 459 of file RPageStorage.hxx.

◆ fOptions

std::unique_ptr<ROOT::RNTupleWriteOptions> ROOT::Internal::RPageSink::fOptions

protectedinherited

Definition at line 276 of file RPageStorage.hxx.

◆ fPageAllocator

std::unique_ptr<ROOT::Internal::RPageAllocator> ROOT::Internal::RPageStorage::fPageAllocator

protectedinherited

For the time being, we will use the heap allocator for all sources and sinks. This may change in the future.

Definition at line 149 of file RPageStorage.hxx.

◆ fPageId

std::atomic<std::uint64_t> ROOT::Experimental::Internal::RPageSinkDaos::fPageId {0}

private

Page identifier for the next committed page; it is automatically incremented in CommitSealedPageImpl().

Definition at line 107 of file RPageStorageDaos.hxx.

◆ fPrevClusterNEntries

ROOT::NTupleSize_t ROOT::Internal::RPagePersistentSink::fPrevClusterNEntries = 0

privateinherited

Used to calculate the number of entries in the current cluster.

Definition at line 455 of file RPageStorage.hxx.

◆ fSealPageBuffer

std::vector<unsigned char> ROOT::Internal::RPageSink::fSealPageBuffer

privateinherited

Used as destination buffer in the simple SealPage overload.

Definition at line 289 of file RPageStorage.hxx.

◆ fSerializationContext

ROOT::Internal::RNTupleSerializer::RContext ROOT::Internal::RPagePersistentSink::fSerializationContext

privateinherited

Used to map the IDs of the descriptor to the physical IDs issued during header/footer serialization.

Definition at line 450 of file RPageStorage.hxx.

◆ fTaskScheduler

RTaskScheduler* ROOT::Internal::RPageStorage::fTaskScheduler = nullptr

protectedinherited

Definition at line 152 of file RPageStorage.hxx.

◆ fURI

std::string ROOT::Experimental::Internal::RPageSinkDaos::fURI

private

A URI to a DAOS pool of the form 'daos://pool-label/container-label'.

Definition at line 111 of file RPageStorageDaos.hxx.

◆ fWritePageMemoryManager

RWritePageMemoryManager ROOT::Internal::RPageSink::fWritePageMemoryManager

privateinherited

Used in ReservePage to maintain the page buffer budget.

Definition at line 292 of file RPageStorage.hxx.

◆ kNBytesPageChecksum

std::size_t ROOT::Internal::RPageStorage::kNBytesPageChecksum = sizeof(std::uint64_t)

staticconstexprinherited

The page checksum is a 64bit xxhash3.

Definition at line 73 of file RPageStorage.hxx.

The documentation for this class was generated from the following files:

tree/ntuple/inc/ROOT/RPageStorageDaos.hxx
tree/ntuple/src/RPageStorageDaos.cxx

Public Types

Public Member Functions

Static Public Member Functions

Static Public Attributes

Protected Member Functions

Protected Attributes

Private Attributes

Member Typedef Documentation

◆ Callback_t

◆ ColumnHandle_t

◆ SealedPageSequence_t

Constructor & Destructor Documentation

◆ RPageSinkDaos()

◆ ~RPageSinkDaos()

Member Function Documentation

◆ AddColumn()

◆ CloneAsHidden()

◆ CommitAttributeSet()

◆ CommitCluster()

◆ CommitClusterGroup()

◆ CommitClusterGroupImpl()

◆ CommitDataset()

◆ CommitDatasetImpl() [1/2]

◆ CommitDatasetImpl() [2/2]

◆ CommitPage()

◆ CommitPageImpl()

◆ CommitSealedPage()

◆ CommitSealedPageImpl()

◆ CommitSealedPageV()

◆ CommitSealedPageVImpl()

◆ CommitStagedClusters()

◆ CommitSuppressedColumn()

◆ Create()

◆ DropColumn()

◆ EnableDefaultMetrics()

◆ GetColumnId()

◆ GetDescriptor()

◆ GetMetrics()

◆ GetNEntries()

◆ GetNTupleName()

◆ GetSinkGuard()

◆ GetType()

◆ GetWriteOptions()

◆ Init()

◆ InitFromDescriptor()

◆ InitImpl() [1/2]

◆ InitImpl() [2/2]

◆ IsInitialized()

◆ RegisterOnCommitDatasetCallback()

◆ ReservePage()

◆ SealPage() [1/2]

◆ SealPage() [2/2]

◆ SetTaskScheduler()

◆ StageCluster()

◆ StageClusterImpl()

◆ UpdateExtraTypeInfo()

◆ UpdateSchema()

◆ WaitForAllTasks()

◆ WriteNTupleAnchor()

◆ WriteNTupleFooter()

◆ WriteNTupleHeader()

Member Data Documentation

◆ fClusterGroupId

◆ fCounters

◆ fDaosContainer

◆ fDescriptorBuilder

◆ fFeatures

◆ fInfosOfStreamerFields

◆ fIsInitialized

◆ fMetrics

◆ fNBytesCurrentCluster

◆ fNextClusterInGroup

◆ fNTupleAnchor

◆ fNTupleIndex

◆ fNTupleName

◆ fOnDatasetCommitCallbacks

◆ fOpenColumnRanges

◆ fOpenPageRanges

◆ fOptions

◆ fPageAllocator