Storage provider that writes ntuple pages to into a DAOS container.

Currently, an object is allocated for each page + 3 additional objects (anchor/header/footer).

Definition at line 92 of file RPageStorageDaos.hxx.

Public Member Functions
	RPageSinkDaos (std::string_view ntupleName, std::string_view uri, const RNTupleWriteOptions &options)

virtual	~RPageSinkDaos ()

void	ReleasePage (RPage &page) final
	Every page store needs to be able to free pages it handed out.

RPage	ReservePage (ColumnHandle_t columnHandle, std::size_t nElements) final
	Get a new, empty page for the given column that can be filled with up to nElements.

Public Member Functions inherited from ROOT::Experimental::Detail::RPageSink
	RPageSink (const RPageSink &)=delete

	RPageSink (RPageSink &&)=default

	RPageSink (std::string_view ntupleName, const RNTupleWriteOptions &options)

virtual	~RPageSink ()

ColumnHandle_t	AddColumn (DescriptorId_t fieldId, const RColumn &column) final
	Register a new column.

std::uint64_t	CommitCluster (NTupleSize_t nEntries)
	Finalize the current cluster and create a new one for the following data.

void	CommitDataset ()
	Finalize the current cluster and the entrire data set.

void	CommitPage (ColumnHandle_t columnHandle, const RPage &page)
	Write a page to the storage. The column must have been added before.

void	CommitSealedPage (DescriptorId_t columnId, const RPageStorage::RSealedPage &sealedPage)
	Write a preprocessed page to storage.

void	Create (RNTupleModel &model)
	Physically creates the storage container to hold the ntuple (e.g., a keys a TFile or an S3 bucket) To do so, Create() calls CreateImpl() after updating the descriptor.

void	DropColumn (ColumnHandle_t) final
	Unregisters a column.

virtual RNTupleMetrics &	GetMetrics () override
	Returns the default metrics object. Subclasses might alternatively provide their own metrics object by overriding this.

EPageStorageType	GetType () final
	Whether the concrete implementation is a sink or a source.

const RNTupleWriteOptions &	GetWriteOptions () const
	Returns the sink's write options.

RPageSink &	operator= (const RPageSink &)=delete

RPageSink &	operator= (RPageSink &&)=default

Public Member Functions inherited from ROOT::Experimental::Detail::RPageStorage
	RPageStorage (const RPageStorage &other)=delete

	RPageStorage (RPageStorage &&other)=default

	RPageStorage (std::string_view name)

virtual	~RPageStorage ()

const std::string &	GetNTupleName () const
	Returns the NTuple name.

RPageStorage &	operator= (const RPageStorage &other)=delete

RPageStorage &	operator= (RPageStorage &&other)=default

void	SetTaskScheduler (RTaskScheduler *taskScheduler)

Protected Member Functions
std::uint64_t	CommitClusterImpl (NTupleSize_t nEntries) final
	Returns the number of bytes written to storage (excluding metadata)

void	CommitDatasetImpl () final

RNTupleLocator	CommitPageImpl (ColumnHandle_t columnHandle, const RPage &page) final

RNTupleLocator	CommitSealedPageImpl (DescriptorId_t columnId, const RPageStorage::RSealedPage &sealedPage) final

void	CreateImpl (const RNTupleModel &model) final

void	WriteNTupleAnchor ()

void	WriteNTupleFooter (const void *data, size_t nbytes, size_t lenFooter)

void	WriteNTupleHeader (const void *data, size_t nbytes, size_t lenHeader)

Protected Member Functions inherited from ROOT::Experimental::Detail::RPageSink
void	EnableDefaultMetrics (const std::string &prefix)
	Enables the default set of metrics provided by RPageSink.

RSealedPage	SealPage (const RPage &page, const RColumnElementBase &element, int compressionSetting)
	Helper for streaming a page.

Private Attributes
std::unique_ptr< RDaosContainer >	fDaosContainer
	Underlying DAOS container.

std::uint64_t	fNBytesCurrentCluster {0}
	Tracks the number of bytes committed to the current cluster.

RDaosNTupleAnchor	fNTupleAnchor

std::atomic< std::uint64_t >	fOid {0}
	OID for the next committed page; it is automatically incremented in `CommitSealedPageImpl()`

std::unique_ptr< RPageAllocatorHeap >	fPageAllocator

Internal::RNTupleSerializer::RContext	fSerializationContext
	Used to keep the column and field IDs issued during header serialization for the footer serialization.

std::string	fURI
	A URI to a DAOS pool of the form 'daos://pool-uuid:svc_replicas/container-uuid'.

Additional Inherited Members
Public Types inherited from ROOT::Experimental::Detail::RPageStorage
using	ColumnHandle_t = RColumnHandle
	The column handle identifies a column with the current open page storage.

Static Public Member Functions inherited from ROOT::Experimental::Detail::RPageSink
static std::unique_ptr< RPageSink >	Create (std::string_view ntupleName, std::string_view location, const RNTupleWriteOptions &options=RNTupleWriteOptions())
	Guess the concrete derived page source from the file name (location)

Static Protected Member Functions inherited from ROOT::Experimental::Detail::RPageSink
static RSealedPage	SealPage (const RPage &page, const RColumnElementBase &element, int compressionSetting, void *buf)
	Seal a page using the provided buffer.

Protected Attributes inherited from ROOT::Experimental::Detail::RPageSink
std::unique_ptr< RNTupleCompressor >	fCompressor
	Helper to zip pages and header/footer; includes a 16MB (kMAXZIPBUF) zip buffer.

std::unique_ptr< RCounters >	fCounters

RNTupleDescriptorBuilder	fDescriptorBuilder

DescriptorId_t	fLastClusterId = 0

DescriptorId_t	fLastColumnId = 0

DescriptorId_t	fLastFieldId = 0
	Building the ntuple descriptor while writing is done in the same way for all the storage sink implementations.

RNTupleMetrics	fMetrics

std::vector< RClusterDescriptor::RColumnRange >	fOpenColumnRanges
	Keeps track of the number of elements in the currently open cluster. Indexed by column id.

std::vector< RClusterDescriptor::RPageRange >	fOpenPageRanges
	Keeps track of the written pages in the currently open cluster. Indexed by column id.

std::unique_ptr< RNTupleWriteOptions >	fOptions

NTupleSize_t	fPrevClusterNEntries = 0

Protected Attributes inherited from ROOT::Experimental::Detail::RPageStorage
std::string	fNTupleName

RTaskScheduler *	fTaskScheduler = nullptr

#include <ROOT/RPageStorageDaos.hxx>

Inheritance diagram for ROOT::Experimental::Detail::RPageSinkDaos:

[legend]

Constructor & Destructor Documentation

◆ RPageSinkDaos()

ROOT::Experimental::Detail::RPageSinkDaos::RPageSinkDaos	(	std::string_view	ntupleName,
		std::string_view	uri,
		const RNTupleWriteOptions &	options
	)

Definition at line 131 of file RPageStorageDaos.cxx.

◆ ~RPageSinkDaos()

ROOT::Experimental::Detail::RPageSinkDaos::~RPageSinkDaos ( )

virtualdefault

Member Function Documentation

◆ CommitClusterImpl()

std::uint64_t ROOT::Experimental::Detail::RPageSinkDaos::CommitClusterImpl ( NTupleSize_t nEntries )

finalprotectedvirtual

Returns the number of bytes written to storage (excluding metadata)

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 210 of file RPageStorageDaos.cxx.

◆ CommitDatasetImpl()

void ROOT::Experimental::Detail::RPageSinkDaos::CommitDatasetImpl ( )

finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 216 of file RPageStorageDaos.cxx.

◆ CommitPageImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Detail::RPageSinkDaos::CommitPageImpl	(	ColumnHandle_t	columnHandle,
		const RPage &	page
	)

finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 174 of file RPageStorageDaos.cxx.

◆ CommitSealedPageImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Detail::RPageSinkDaos::CommitSealedPageImpl	(	DescriptorId_t	columnId,
		const RPageStorage::RSealedPage &	sealedPage
	)

finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 189 of file RPageStorageDaos.cxx.

◆ CreateImpl()

void ROOT::Experimental::Detail::RPageSinkDaos::CreateImpl ( const RNTupleModel & model )

finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 147 of file RPageStorageDaos.cxx.

◆ ReleasePage()

void ROOT::Experimental::Detail::RPageSinkDaos::ReleasePage ( RPage & page )

finalvirtual

Every page store needs to be able to free pages it handed out.

But Sinks and sources have different means of allocating pages.

Implements ROOT::Experimental::Detail::RPageStorage.

Definition at line 290 of file RPageStorageDaos.cxx.

◆ ReservePage()

ROOT::Experimental::Detail::RPage ROOT::Experimental::Detail::RPageSinkDaos::ReservePage	(	ColumnHandle_t	columnHandle,
		std::size_t	nElements
	)

finalvirtual

Get a new, empty page for the given column that can be filled with up to nElements.

If nElements is zero, the page sink picks an appropriate size.

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 282 of file RPageStorageDaos.cxx.

◆ WriteNTupleAnchor()

void ROOT::Experimental::Detail::RPageSinkDaos::WriteNTupleAnchor ( )

protected

Definition at line 273 of file RPageStorageDaos.cxx.

◆ WriteNTupleFooter()

void ROOT::Experimental::Detail::RPageSinkDaos::WriteNTupleFooter	(	const void *	data,
		size_t	nbytes,
		size_t	lenFooter
	)

protected

Definition at line 264 of file RPageStorageDaos.cxx.

◆ WriteNTupleHeader()

void ROOT::Experimental::Detail::RPageSinkDaos::WriteNTupleHeader	(	const void *	data,
		size_t	nbytes,
		size_t	lenHeader
	)

protected

Definition at line 255 of file RPageStorageDaos.cxx.

Member Data Documentation

◆ fDaosContainer

std::unique_ptr<RDaosContainer> ROOT::Experimental::Detail::RPageSinkDaos::fDaosContainer

private

Underlying DAOS container.

An internal std::shared_ptr keep the pool connection alive. ISO C++ ensures the correct destruction order, i.e., ~RDaosContainer is invoked first (which calls daos_cont_close(); the destructor for the std::shared_ptr<RDaosPool> is invoked after (which calls daos_pool_disconect()).

Definition at line 100 of file RPageStorageDaos.hxx.

◆ fNBytesCurrentCluster

std::uint64_t ROOT::Experimental::Detail::RPageSinkDaos::fNBytesCurrentCluster {0}

private

Tracks the number of bytes committed to the current cluster.

Definition at line 106 of file RPageStorageDaos.hxx.

◆ fNTupleAnchor

RDaosNTupleAnchor ROOT::Experimental::Detail::RPageSinkDaos::fNTupleAnchor

private

Definition at line 110 of file RPageStorageDaos.hxx.

◆ fOid

std::atomic<std::uint64_t> ROOT::Experimental::Detail::RPageSinkDaos::fOid {0}

private

OID for the next committed page; it is automatically incremented in CommitSealedPageImpl()

Definition at line 102 of file RPageStorageDaos.hxx.

◆ fPageAllocator

std::unique_ptr<RPageAllocatorHeap> ROOT::Experimental::Detail::RPageSinkDaos::fPageAllocator

private

Definition at line 94 of file RPageStorageDaos.hxx.

◆ fSerializationContext

Internal::RNTupleSerializer::RContext ROOT::Experimental::Detail::RPageSinkDaos::fSerializationContext

private

Used to keep the column and field IDs issued during header serialization for the footer serialization.

Definition at line 108 of file RPageStorageDaos.hxx.

◆ fURI

std::string ROOT::Experimental::Detail::RPageSinkDaos::fURI

private

A URI to a DAOS pool of the form 'daos://pool-uuid:svc_replicas/container-uuid'.

Definition at line 104 of file RPageStorageDaos.hxx.

tree/ntuple/v7/inc/ROOT/RPageStorageDaos.hxx

tree/ntuple/v7/src/RPageStorageDaos.cxx

Public Member Functions

Protected Member Functions

Private Attributes

Additional Inherited Members

Constructor & Destructor Documentation

◆ RPageSinkDaos()

◆ ~RPageSinkDaos()

Member Function Documentation

◆ CommitClusterImpl()

◆ CommitDatasetImpl()

◆ CommitPageImpl()

◆ CommitSealedPageImpl()

◆ CreateImpl()

◆ ReleasePage()

◆ ReservePage()

◆ WriteNTupleAnchor()

◆ WriteNTupleFooter()

◆ WriteNTupleHeader()

Member Data Documentation

◆ fDaosContainer

◆ fNBytesCurrentCluster

◆ fNTupleAnchor

◆ fOid

◆ fPageAllocator

◆ fSerializationContext

◆ fURI