Storage provider that reads ntuple pages from a file.

Definition at line 100 of file RPageStorageFile.hxx.

Classes
struct	RClusterInfo
	Summarizes cluster-level information that are necessary to populate a certain page. More...

Public Member Functions
	RPageSourceFile (const RPageSourceFile &)=delete

	RPageSourceFile (RPageSourceFile &&)=delete

	RPageSourceFile (std::string_view ntupleName, std::string_view path, const RNTupleReadOptions &options)

	RPageSourceFile (std::string_view ntupleName, std::unique_ptr< ROOT::Internal::RRawFile > file, const RNTupleReadOptions &options)

	~RPageSourceFile () override

std::unique_ptr< RPageSource >	Clone () const final
	The cloned page source creates a new raw file and reader and opens its own file descriptor to the data.

std::vector< std::unique_ptr< RCluster > >	LoadClusters (std::span< RCluster::RKey > clusterKeys) final
	Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages.

void	LoadSealedPage (DescriptorId_t physicalColumnId, RClusterIndex clusterIndex, RSealedPage &sealedPage) final
	Read the packed and compressed bytes of a page into the memory buffer provided by selaedPage.

RPageSourceFile &	operator= (const RPageSourceFile &)=delete

RPageSourceFile &	operator= (RPageSourceFile &&)=delete

RPage	PopulatePage (ColumnHandle_t columnHandle, NTupleSize_t globalIndex) final
	Allocates and fills a page that contains the index-th element.

RPage	PopulatePage (ColumnHandle_t columnHandle, RClusterIndex clusterIndex) final
	Another version of PopulatePage that allows to specify cluster-relative indexes.

void	ReleasePage (RPage &page) final
	Every page store needs to be able to free pages it handed out.

Public Member Functions inherited from ROOT::Experimental::Internal::RPageSource
	RPageSource (const RPageSource &)=delete

	RPageSource (RPageSource &&)=delete

	RPageSource (std::string_view ntupleName, const RNTupleReadOptions &fOptions)

	~RPageSource () override

ColumnHandle_t	AddColumn (DescriptorId_t fieldId, const RColumn &column) override
	Register a new column.

void	Attach ()
	Open the physical storage container for the tree.

void	DropColumn (ColumnHandle_t columnHandle) override
	Unregisters a column.

ColumnId_t	GetColumnId (ColumnHandle_t columnHandle)

REntryRange	GetEntryRange () const

NTupleSize_t	GetNElements (ColumnHandle_t columnHandle)

NTupleSize_t	GetNEntries ()

const RNTupleReadOptions &	GetReadOptions () const

const RSharedDescriptorGuard	GetSharedDescriptorGuard () const
	Takes the read lock for the descriptor.

EPageStorageType	GetType () final
	Whether the concrete implementation is a sink or a source.

RPageSource &	operator= (const RPageSource &)=delete

RPageSource &	operator= (RPageSource &&)=delete

void	SetEntryRange (const REntryRange &range)
	Promise to only read from the given entry range.

RPage	UnsealPage (const RSealedPage &sealedPage, const RColumnElementBase &element, DescriptorId_t physicalColumnId)
	Helper for unstreaming a page.

void	UnzipCluster (RCluster *cluster)
	Parallel decompression and unpacking of the pages in the given cluster.

Public Member Functions inherited from ROOT::Experimental::Internal::RPageStorage
	RPageStorage (const RPageStorage &other)=delete

	RPageStorage (RPageStorage &&other)=default

	RPageStorage (std::string_view name)

virtual	~RPageStorage ()

virtual Detail::RNTupleMetrics &	GetMetrics ()
	Returns the default metrics object.

const std::string &	GetNTupleName () const
	Returns the NTuple name.

RPageStorage &	operator= (const RPageStorage &other)=delete

RPageStorage &	operator= (RPageStorage &&other)=default

void	SetTaskScheduler (RTaskScheduler *taskScheduler)

Static Public Member Functions
static std::unique_ptr< RPageSourceFile >	CreateFromAnchor (const RNTuple &anchor, const RNTupleReadOptions &options=RNTupleReadOptions())
	Used from the RNTuple class to build a datasource if the anchor is already available.

Static Public Member Functions inherited from ROOT::Experimental::Internal::RPageSource
static std::unique_ptr< RPageSource >	Create (std::string_view ntupleName, std::string_view location, const RNTupleReadOptions &options=RNTupleReadOptions())
	Guess the concrete derived page source from the file name (location)

Protected Member Functions
RNTupleDescriptor	AttachImpl () final

void	UnzipClusterImpl (RCluster *cluster) final

Protected Member Functions inherited from ROOT::Experimental::Internal::RPageSource
void	EnableDefaultMetrics (const std::string &prefix)
	Enables the default set of metrics provided by RPageSource.

RExclDescriptorGuard	GetExclDescriptorGuard ()
	Note that the underlying lock is not recursive. See GetSharedDescriptorGuard() for further information.

void	PrepareLoadCluster (const RCluster::RKey &clusterKey, ROnDiskPageMap &pageZeroMap, std::function< void(DescriptorId_t, NTupleSize_t, const RClusterDescriptor::RPageRange::RPageInfo &)> perPageFunc)
	Prepare a page range read for the column set in `clusterKey`.

Protected Member Functions inherited from ROOT::Experimental::Internal::RPageStorage
void	WaitForAllTasks ()

Private Member Functions
	RPageSourceFile (std::string_view ntupleName, const RNTupleReadOptions &options)

void	InitDescriptor (const RNTuple &anchor)
	Deserialized header and footer into a minimal descriptor held by fDescriptorBuilder.

RPage	PopulatePageFromCluster (ColumnHandle_t columnHandle, const RClusterInfo &clusterInfo, ClusterSize_t::ValueType idxInCluster)

std::unique_ptr< RCluster >	PrepareSingleCluster (const RCluster::RKey &clusterKey, std::vector< ROOT::Internal::RRawFile::RIOVec > &readRequests)
	Helper function for LoadClusters: it prepares the memory buffer (page map) and the read requests for a given cluster and columns.

Private Attributes
std::unique_ptr< RClusterPool >	fClusterPool
	The cluster pool asynchronously preloads the next few clusters.

RCluster *	fCurrentCluster = nullptr
	The last cluster from which a page got populated. Points into fClusterPool->fPool.

RNTupleDescriptorBuilder	fDescriptorBuilder
	The descriptor is created from the header and footer either in AttachImpl or in CreateFromAnchor.

std::unique_ptr< ROOT::Internal::RRawFile >	fFile
	An RRawFile is used to request the necessary byte ranges from a local or a remote file.

std::shared_ptr< RPagePool >	fPagePool
	Populated pages might be shared; the page pool might, at some point, be used by multiple page sources.

RMiniFileReader	fReader
	Takes the fFile to read ntuple blobs from it.

Friends
class	ROOT::Experimental::RNTuple

Additional Inherited Members
Public Types inherited from ROOT::Experimental::Internal::RPageStorage
using	ColumnHandle_t = RColumnHandle
	The column handle identifies a column with the current open page storage.

using	SealedPageSequence_t = std::deque< RSealedPage >

Protected Attributes inherited from ROOT::Experimental::Internal::RPageSource
RActivePhysicalColumns	fActivePhysicalColumns
	The active columns are implicitly defined by the model fields or views.

std::unique_ptr< RCounters >	fCounters

std::unique_ptr< RNTupleDecompressor >	fDecompressor
	Helper to unzip pages and header/footer; comprises a 16MB (kMAXZIPBUF) unzip buffer.

RNTupleReadOptions	fOptions

Protected Attributes inherited from ROOT::Experimental::Internal::RPageStorage
Detail::RNTupleMetrics	fMetrics

std::string	fNTupleName

RTaskScheduler *	fTaskScheduler = nullptr

#include <ROOT/RPageStorageFile.hxx>

Inheritance diagram for ROOT::Experimental::Internal::RPageSourceFile:

[legend]

Constructor & Destructor Documentation

◆ RPageSourceFile() [1/5]

ROOT::Experimental::Internal::RPageSourceFile::RPageSourceFile	(	std::string_view	ntupleName,
		const RNTupleReadOptions &	options
	)

private

Definition at line 227 of file RPageStorageFile.cxx.

◆ RPageSourceFile() [2/5]

ROOT::Experimental::Internal::RPageSourceFile::RPageSourceFile	(	std::string_view	ntupleName,
		std::string_view	path,
		const RNTupleReadOptions &	options
	)

Definition at line 247 of file RPageStorageFile.cxx.

◆ RPageSourceFile() [3/5]

ROOT::Experimental::Internal::RPageSourceFile::RPageSourceFile	(	std::string_view	ntupleName,
		std::unique_ptr< ROOT::Internal::RRawFile >	file,
		const RNTupleReadOptions &	options
	)

Definition at line 237 of file RPageStorageFile.cxx.

◆ RPageSourceFile() [4/5]

ROOT::Experimental::Internal::RPageSourceFile::RPageSourceFile ( const RPageSourceFile & )

delete

◆ RPageSourceFile() [5/5]

ROOT::Experimental::Internal::RPageSourceFile::RPageSourceFile ( RPageSourceFile && )

delete

◆ ~RPageSourceFile()

ROOT::Experimental::Internal::RPageSourceFile::~RPageSourceFile ( )

overridedefault

Member Function Documentation

◆ AttachImpl()

ROOT::Experimental::RNTupleDescriptor ROOT::Experimental::Internal::RPageSourceFile::AttachImpl ( )

finalprotectedvirtual

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 310 of file RPageStorageFile.cxx.

◆ Clone()

std::unique_ptr< ROOT::Experimental::Internal::RPageSource > ROOT::Experimental::Internal::RPageSourceFile::Clone ( ) const

finalvirtual

The cloned page source creates a new raw file and reader and opens its own file descriptor to the data.

The meta-data (header and footer) is reread and parsed by the clone.

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 479 of file RPageStorageFile.cxx.

◆ CreateFromAnchor()

std::unique_ptr< ROOT::Experimental::Internal::RPageSourceFile > ROOT::Experimental::Internal::RPageSourceFile::CreateFromAnchor	(	const RNTuple &	anchor,
		const RNTupleReadOptions &	options = `RNTupleReadOptions()`
	)

static

Used from the RNTuple class to build a datasource if the anchor is already available.

Requires the RNTuple object to be streamed from a file.

Definition at line 282 of file RPageStorageFile.cxx.

◆ InitDescriptor()

void ROOT::Experimental::Internal::RPageSourceFile::InitDescriptor ( const RNTuple & anchor )

private

Deserialized header and footer into a minimal descriptor held by fDescriptorBuilder.

Definition at line 253 of file RPageStorageFile.cxx.

◆ LoadClusters()

std::vector< std::unique_ptr< ROOT::Experimental::Internal::RCluster > > ROOT::Experimental::Internal::RPageSourceFile::LoadClusters ( std::span< RCluster::RKey > clusterKeys )

finalvirtual

Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages.

The page source may load more columns than the minimal necessary set from columns. To indicate which columns have been loaded, LoadClusters() must mark them with SetColumnAvailable(). That includes the ones from the columns that don't have pages; otherwise subsequent requests for the cluster would assume an incomplete cluster and trigger loading again. LoadClusters() is typically called from the I/O thread of a cluster pool, i.e. the method runs concurrently to other methods of the page source.

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 598 of file RPageStorageFile.cxx.

◆ LoadSealedPage()

void ROOT::Experimental::Internal::RPageSourceFile::LoadSealedPage	(	DescriptorId_t	physicalColumnId,
		RClusterIndex	clusterIndex,
		RSealedPage &	sealedPage
	)

finalvirtual

Read the packed and compressed bytes of a page into the memory buffer provided by selaedPage.

The sealed page can be used subsequently in a call to RPageSink::CommitSealedPage. The fSize and fNElements member of the sealedPage parameters are always set. If sealedPage.fBuffer is nullptr, no data will be copied but the returned size information can be used by the caller to allocate a large enough buffer and call LoadSealedPage again.

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 335 of file RPageStorageFile.cxx.

◆ operator=() [1/2]

RPageSourceFile & ROOT::Experimental::Internal::RPageSourceFile::operator= ( const RPageSourceFile & )

delete

◆ operator=() [2/2]

RPageSourceFile & ROOT::Experimental::Internal::RPageSourceFile::operator= ( RPageSourceFile && )

delete

◆ PopulatePage() [1/2]

ROOT::Experimental::Internal::RPage ROOT::Experimental::Internal::RPageSourceFile::PopulatePage	(	ColumnHandle_t	columnHandle,
		NTupleSize_t	globalIndex
	)

finalvirtual

Allocates and fills a page that contains the index-th element.

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 423 of file RPageStorageFile.cxx.

◆ PopulatePage() [2/2]

ROOT::Experimental::Internal::RPage ROOT::Experimental::Internal::RPageSourceFile::PopulatePage	(	ColumnHandle_t	columnHandle,
		RClusterIndex	clusterIndex
	)

finalvirtual

Another version of PopulatePage that allows to specify cluster-relative indexes.

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 450 of file RPageStorageFile.cxx.

◆ PopulatePageFromCluster()

ROOT::Experimental::Internal::RPage ROOT::Experimental::Internal::RPageSourceFile::PopulatePageFromCluster	(	ColumnHandle_t	columnHandle,
		const RClusterInfo &	clusterInfo,
		ClusterSize_t::ValueType	idxInCluster
	)

private

Definition at line 361 of file RPageStorageFile.cxx.

◆ PrepareSingleCluster()

std::unique_ptr< ROOT::Experimental::Internal::RCluster > ROOT::Experimental::Internal::RPageSourceFile::PrepareSingleCluster	(	const RCluster::RKey &	clusterKey,
		std::vector< ROOT::Internal::RRawFile::RIOVec > &	readRequests
	)

private

Helper function for LoadClusters: it prepares the memory buffer (page map) and the read requests for a given cluster and columns.

The reead requests are appended to the provided vector. This way, requests can be collected for multiple clusters before sending them to RRawFile::ReadV().

Definition at line 488 of file RPageStorageFile.cxx.

◆ ReleasePage()

void ROOT::Experimental::Internal::RPageSourceFile::ReleasePage ( RPage & page )

finalvirtual

Every page store needs to be able to free pages it handed out.

But Sinks and sources have different means of allocating pages.

Implements ROOT::Experimental::Internal::RPageStorage.

Definition at line 474 of file RPageStorageFile.cxx.

◆ UnzipClusterImpl()

void ROOT::Experimental::Internal::RPageSourceFile::UnzipClusterImpl ( RCluster * cluster )

finalprotectedvirtual

Reimplemented from ROOT::Experimental::Internal::RPageSource.

Definition at line 650 of file RPageStorageFile.cxx.

Friends And Related Symbol Documentation

◆ ROOT::Experimental::RNTuple

friend class ROOT::Experimental::RNTuple

friend

Definition at line 101 of file RPageStorageFile.hxx.

Member Data Documentation

◆ fClusterPool

std::unique_ptr<RClusterPool> ROOT::Experimental::Internal::RPageSourceFile::fClusterPool

private

The cluster pool asynchronously preloads the next few clusters.

Definition at line 125 of file RPageStorageFile.hxx.

◆ fCurrentCluster

RCluster* ROOT::Experimental::Internal::RPageSourceFile::fCurrentCluster = nullptr

private

The last cluster from which a page got populated. Points into fClusterPool->fPool.

Definition at line 117 of file RPageStorageFile.hxx.

◆ fDescriptorBuilder

RNTupleDescriptorBuilder ROOT::Experimental::Internal::RPageSourceFile::fDescriptorBuilder

private

The descriptor is created from the header and footer either in AttachImpl or in CreateFromAnchor.

Definition at line 123 of file RPageStorageFile.hxx.

◆ fFile

std::unique_ptr<ROOT::Internal::RRawFile> ROOT::Experimental::Internal::RPageSourceFile::fFile

private

An RRawFile is used to request the necessary byte ranges from a local or a remote file.

Definition at line 119 of file RPageStorageFile.hxx.

◆ fPagePool

std::shared_ptr<RPagePool> ROOT::Experimental::Internal::RPageSourceFile::fPagePool

private

Populated pages might be shared; the page pool might, at some point, be used by multiple page sources.

Definition at line 115 of file RPageStorageFile.hxx.

◆ fReader

RMiniFileReader ROOT::Experimental::Internal::RPageSourceFile::fReader

private

Takes the fFile to read ntuple blobs from it.

Definition at line 121 of file RPageStorageFile.hxx.

Libraries for ROOT::Experimental::Internal::RPageSourceFile:

[legend]

The documentation for this class was generated from the following files:

tree/ntuple/v7/inc/ROOT/RPageStorageFile.hxx
tree/ntuple/v7/src/RPageStorageFile.cxx

Classes

Public Member Functions

Static Public Member Functions

Protected Member Functions

Private Member Functions

Private Attributes

Friends

Additional Inherited Members

Constructor & Destructor Documentation

◆ RPageSourceFile() [1/5]

◆ RPageSourceFile() [2/5]

◆ RPageSourceFile() [3/5]

◆ RPageSourceFile() [4/5]

◆ RPageSourceFile() [5/5]

◆ ~RPageSourceFile()

Member Function Documentation

◆ AttachImpl()

◆ Clone()

◆ CreateFromAnchor()

◆ InitDescriptor()

◆ LoadClusters()

◆ LoadSealedPage()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ PopulatePage() [1/2]

◆ PopulatePage() [2/2]

◆ PopulatePageFromCluster()

◆ PrepareSingleCluster()

◆ ReleasePage()

◆ UnzipClusterImpl()

Friends And Related Symbol Documentation

◆ ROOT::Experimental::RNTuple

Member Data Documentation

◆ fClusterPool

◆ fCurrentCluster

◆ fDescriptorBuilder

◆ fFile

◆ fPagePool

◆ fReader