Virtual storage that combines several other sources horizontally.

Definition at line 39 of file RPageSourceFriends.hxx.

Classes
struct	RIdBiMap
	A bi-directional map of descriptor IDs that translates from physical to virtual column, field, and cluster IDs and vice versa. More...

struct	ROriginId

Public Member Functions
	RPageSourceFriends (std::string_view ntupleName, std::span< std::unique_ptr< RPageSource > > sources)

	~RPageSourceFriends () final

ColumnHandle_t	AddColumn (DescriptorId_t fieldId, RColumn &column) final
	Register a new column.

void	DropColumn (ColumnHandle_t columnHandle) final
	Unregisters a column.

Detail::RNTupleMetrics &	GetMetrics () final
	Returns the default metrics object.

std::vector< std::unique_ptr< RCluster > >	LoadClusters (std::span< RCluster::RKey > clusterKeys) final
	Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages.

RPageRef	LoadPage (ColumnHandle_t columnHandle, NTupleSize_t globalIndex) final
	Allocates and fills a page that contains the index-th element.

RPageRef	LoadPage (ColumnHandle_t columnHandle, RClusterIndex clusterIndex) final
	Another version of `LoadPage` that allows to specify cluster-relative indexes.

void	LoadSealedPage (DescriptorId_t physicalColumnId, RClusterIndex clusterIndex, RSealedPage &sealedPage) final
	Read the packed and compressed bytes of a page into the memory buffer provided by `sealedPage`.

Public Member Functions inherited from ROOT::Experimental::Internal::RPageSource
	RPageSource (const RPageSource &)=delete

	RPageSource (RPageSource &&)=delete

	RPageSource (std::string_view ntupleName, const RNTupleReadOptions &fOptions)

	~RPageSource () override

void	Attach ()
	Open the physical storage container and deserialize header and footer.

std::unique_ptr< RPageSource >	Clone () const
	Open the same storage multiple time, e.g.

REntryRange	GetEntryRange () const

NTupleSize_t	GetNElements (ColumnHandle_t columnHandle)

NTupleSize_t	GetNEntries ()

const RNTupleReadOptions &	GetReadOptions () const

const RSharedDescriptorGuard	GetSharedDescriptorGuard () const
	Takes the read lock for the descriptor.

EPageStorageType	GetType () final
	Whether the concrete implementation is a sink or a source.

void	LoadStructure ()
	Loads header and footer without decompressing or deserializing them.

RPageSource &	operator= (const RPageSource &)=delete

RPageSource &	operator= (RPageSource &&)=delete

void	SetEntryRange (const REntryRange &range)
	Promise to only read from the given entry range.

RResult< RPage >	UnsealPage (const RSealedPage &sealedPage, const RColumnElementBase &element, DescriptorId_t physicalColumnId)

void	UnzipCluster (RCluster *cluster)
	Parallel decompression and unpacking of the pages in the given cluster.

Public Member Functions inherited from ROOT::Experimental::Internal::RPageStorage
	RPageStorage (const RPageStorage &other)=delete

	RPageStorage (RPageStorage &&other)=default

	RPageStorage (std::string_view name)

virtual	~RPageStorage ()

ColumnId_t	GetColumnId (ColumnHandle_t columnHandle) const

const std::string &	GetNTupleName () const
	Returns the NTuple name.

RPageStorage &	operator= (const RPageStorage &other)=delete

RPageStorage &	operator= (RPageStorage &&other)=default

void	SetTaskScheduler (RTaskScheduler *taskScheduler)

Protected Member Functions
RNTupleDescriptor	AttachImpl () final
	`LoadStructureImpl()` has been called before `AttachImpl()` is called

std::unique_ptr< RPageSource >	CloneImpl () const final
	Returns a new, unattached page source for the same data set.

virtual RPageRef	LoadPageImpl (ColumnHandle_t, const RClusterInfo &, ClusterSize_t::ValueType)

void	LoadStructureImpl () final

Protected Member Functions inherited from ROOT::Experimental::Internal::RPageSource
void	EnableDefaultMetrics (const std::string &prefix)
	Enables the default set of metrics provided by RPageSource.

RExclDescriptorGuard	GetExclDescriptorGuard ()
	Note that the underlying lock is not recursive. See `GetSharedDescriptorGuard()` for further information.

void	PrepareLoadCluster (const RCluster::RKey &clusterKey, ROnDiskPageMap &pageZeroMap, std::function< void(DescriptorId_t, NTupleSize_t, const RClusterDescriptor::RPageRange::RPageInfo &)> perPageFunc)
	Prepare a page range read for the column set in `clusterKey`.

virtual void	UnzipClusterImpl (RCluster *cluster)

Protected Member Functions inherited from ROOT::Experimental::Internal::RPageStorage
void	WaitForAllTasks ()

Private Member Functions
void	AddVirtualField (const RNTupleDescriptor &originDesc, std::size_t originIdx, const RFieldDescriptor &originField, DescriptorId_t virtualParent, const std::string &virtualName)

Private Attributes
RNTupleDescriptorBuilder	fBuilder

RIdBiMap	fColumnMap
	TODO(jblomer): Not only the columns, but actually all the different objects of the descriptor would need their own maps to avoid descriptor ID clashes.

RIdBiMap	fIdBiMap

Detail::RNTupleMetrics	fMetrics

DescriptorId_t	fNextId = 1
	0 is reserved for the friend zero field

std::vector< std::unique_ptr< RPageSource > >	fSources

Additional Inherited Members
Public Types inherited from ROOT::Experimental::Internal::RPageStorage
using	ColumnHandle_t = RColumnHandle
	The column handle identifies a column with the current open page storage.

using	SealedPageSequence_t = std::deque< RSealedPage >

Static Public Member Functions inherited from ROOT::Experimental::Internal::RPageSource
static std::unique_ptr< RPageSource >	Create (std::string_view ntupleName, std::string_view location, const RNTupleReadOptions &options=RNTupleReadOptions())
	Guess the concrete derived page source from the file name (location)

static RResult< RPage >	UnsealPage (const RSealedPage &sealedPage, const RColumnElementBase &element, DescriptorId_t physicalColumnId, RPageAllocator &pageAlloc)
	Helper for unstreaming a page.

Static Public Attributes inherited from ROOT::Experimental::Internal::RPageStorage
static constexpr std::size_t	kNBytesPageChecksum = sizeof(std::uint64_t)
	The page checksum is a 64bit xxhash3.

Protected Attributes inherited from ROOT::Experimental::Internal::RPageSource
RActivePhysicalColumns	fActivePhysicalColumns
	The active columns are implicitly defined by the model fields or views.

std::unique_ptr< RCounters >	fCounters

RNTupleReadOptions	fOptions

RPagePool	fPagePool
	Pages that are unzipped with IMT are staged into the page pool.

Protected Attributes inherited from ROOT::Experimental::Internal::RPageStorage
Detail::RNTupleMetrics	fMetrics

std::string	fNTupleName

std::unique_ptr< RPageAllocator >	fPageAllocator
	For the time being, we will use the heap allocator for all sources and sinks. This may change in the future.

RTaskScheduler *	fTaskScheduler = nullptr

#include <ROOT/RPageSourceFriends.hxx>

Inheritance diagram for ROOT::Experimental::Internal::RPageSourceFriends:

[legend]

Constructor & Destructor Documentation

◆ RPageSourceFriends()

ROOT::Experimental::Internal::RPageSourceFriends::RPageSourceFriends	(	std::string_view	ntupleName,
		std::span< std::unique_ptr< RPageSource > >	sources
	)

Definition at line 24 of file RPageSourceFriends.cxx.

◆ ~RPageSourceFriends()

ROOT::Experimental::Internal::RPageSourceFriends::~RPageSourceFriends ( )

finaldefault

Member Function Documentation

◆ AddColumn()

ROOT::Experimental::Internal::RPageStorage::ColumnHandle_t ROOT::Experimental::Internal::RPageSourceFriends::AddColumn	(	DescriptorId_t	fieldId,
		RColumn &	column
	)

finalvirtual

Register a new column.

When reading, the column must exist in the ntuple on disk corresponding to the meta-data. When writing, every column can only be attached once.

Reimplemented from ROOT::Experimental::Internal::RPageSource.

Definition at line 148 of file RPageSourceFriends.cxx.

◆ AddVirtualField()

void ROOT::Experimental::Internal::RPageSourceFriends::AddVirtualField	(	const RNTupleDescriptor &	originDesc,
		std::size_t	originIdx,
		const RFieldDescriptor &	originField,
		DescriptorId_t	virtualParent,
		const std::string &	virtualName
	)

private

Definition at line 36 of file RPageSourceFriends.cxx.

◆ AttachImpl()

ROOT::Experimental::RNTupleDescriptor ROOT::Experimental::Internal::RPageSourceFriends::AttachImpl ( )

finalprotectedvirtual

LoadStructureImpl() has been called before AttachImpl() is called

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 69 of file RPageSourceFriends.cxx.

◆ CloneImpl()

std::unique_ptr< ROOT::Experimental::Internal::RPageSource > ROOT::Experimental::Internal::RPageSourceFriends::CloneImpl ( ) const

finalprotectedvirtual

Returns a new, unattached page source for the same data set.

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 135 of file RPageSourceFriends.cxx.

◆ DropColumn()

void ROOT::Experimental::Internal::RPageSourceFriends::DropColumn ( ColumnHandle_t columnHandle )

finalvirtual

Unregisters a column.

A page source decreases the reference counter for the corresponding active column. For a page sink, dropping columns is currently a no-op.

Reimplemented from ROOT::Experimental::Internal::RPageSource.

Definition at line 155 of file RPageSourceFriends.cxx.

◆ GetMetrics()

Detail::RNTupleMetrics & ROOT::Experimental::Internal::RPageSourceFriends::GetMetrics ( )

inlinefinalvirtual

Returns the default metrics object.

Subclasses might alternatively provide their own metrics object by overriding this.

Reimplemented from ROOT::Experimental::Internal::RPageStorage.

Definition at line 117 of file RPageSourceFriends.hxx.

◆ LoadClusters()

std::vector< std::unique_ptr< ROOT::Experimental::Internal::RCluster > > ROOT::Experimental::Internal::RPageSourceFriends::LoadClusters ( std::span< RCluster::RKey > clusterKeys )

finalvirtual

Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages.

The page source may load more columns than the minimal necessary set from columns. To indicate which columns have been loaded, LoadClusters()must mark them withSetColumnAvailable(). That includes the ones from thecolumnsthat don't have pages; otherwise subsequent requests for the cluster would assume an incomplete cluster and trigger loading again. LoadClusters()` is typically called from the I/O thread of a cluster pool, i.e. the method runs concurrently to other methods of the page source.

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 209 of file RPageSourceFriends.cxx.

◆ LoadPage() [1/2]

ROOT::Experimental::Internal::RPageRef ROOT::Experimental::Internal::RPageSourceFriends::LoadPage	(	ColumnHandle_t	columnHandle,
		NTupleSize_t	globalIndex
	)

finalvirtual

Allocates and fills a page that contains the index-th element.

The default implementation searches the page and calls LoadPageImpl(). Returns a default-constructed RPage for suppressed columns.

Reimplemented from ROOT::Experimental::Internal::RPageSource.

Definition at line 164 of file RPageSourceFriends.cxx.

◆ LoadPage() [2/2]

ROOT::Experimental::Internal::RPageRef ROOT::Experimental::Internal::RPageSourceFriends::LoadPage	(	ColumnHandle_t	columnHandle,
		RClusterIndex	clusterIndex
	)

finalvirtual

Another version of LoadPage that allows to specify cluster-relative indexes.

Returns a default-constructed RPage for suppressed columns.

Reimplemented from ROOT::Experimental::Internal::RPageSource.

Definition at line 182 of file RPageSourceFriends.cxx.

◆ LoadPageImpl()

virtual RPageRef ROOT::Experimental::Internal::RPageSourceFriends::LoadPageImpl	(	ColumnHandle_t	,
		const RClusterInfo &	,
		ClusterSize_t::ValueType
	)

inlineprotectedvirtual

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 97 of file RPageSourceFriends.hxx.

◆ LoadSealedPage()

void ROOT::Experimental::Internal::RPageSourceFriends::LoadSealedPage	(	DescriptorId_t	physicalColumnId,
		RClusterIndex	clusterIndex,
		RSealedPage &	sealedPage
	)

finalvirtual

Read the packed and compressed bytes of a page into the memory buffer provided by sealedPage.

The sealed page can be used subsequently in a call to RPageSink::CommitSealedPage. The fSize and fNElements member of the sealedPage parameters are always set. If sealedPage.fBuffer is nullptr, no data will be copied but the returned size information can be used by the caller to allocate a large enough buffer and call LoadSealedPage again.

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 198 of file RPageSourceFriends.cxx.

◆ LoadStructureImpl()

void ROOT::Experimental::Internal::RPageSourceFriends::LoadStructureImpl ( )

inlinefinalprotectedvirtual

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 92 of file RPageSourceFriends.hxx.

Member Data Documentation

◆ fBuilder

RNTupleDescriptorBuilder ROOT::Experimental::Internal::RPageSourceFriends::fBuilder

private

Definition at line 85 of file RPageSourceFriends.hxx.

◆ fColumnMap

RIdBiMap ROOT::Experimental::Internal::RPageSourceFriends::fColumnMap

private

TODO(jblomer): Not only the columns, but actually all the different objects of the descriptor would need their own maps to avoid descriptor ID clashes.

The need for the distinct column map was triggered by adding support for multi-column representations. A follow-up patch should either fix the friend page source properly or remove it in favor of the RNTupleProcessor.

Definition at line 83 of file RPageSourceFriends.hxx.

◆ fIdBiMap

RIdBiMap ROOT::Experimental::Internal::RPageSourceFriends::fIdBiMap

private

Definition at line 78 of file RPageSourceFriends.hxx.

◆ fMetrics

Detail::RNTupleMetrics ROOT::Experimental::Internal::RPageSourceFriends::fMetrics

private

Definition at line 76 of file RPageSourceFriends.hxx.

◆ fNextId

DescriptorId_t ROOT::Experimental::Internal::RPageSourceFriends::fNextId = 1

private

0 is reserved for the friend zero field

Definition at line 86 of file RPageSourceFriends.hxx.

◆ fSources

std::vector<std::unique_ptr<RPageSource> > ROOT::Experimental::Internal::RPageSourceFriends::fSources

private

Definition at line 77 of file RPageSourceFriends.hxx.

Libraries for ROOT::Experimental::Internal::RPageSourceFriends:

[legend]

The documentation for this class was generated from the following files:

tree/ntuple/v7/inc/ROOT/RPageSourceFriends.hxx
tree/ntuple/v7/src/RPageSourceFriends.cxx

Classes

Public Member Functions

Protected Member Functions

Private Member Functions

Private Attributes

Additional Inherited Members

Constructor & Destructor Documentation

◆ RPageSourceFriends()

◆ ~RPageSourceFriends()

Member Function Documentation

◆ AddColumn()

◆ AddVirtualField()

◆ AttachImpl()

◆ CloneImpl()

◆ DropColumn()

◆ GetMetrics()

◆ LoadClusters()

◆ LoadPage() [1/2]

◆ LoadPage() [2/2]

◆ LoadPageImpl()

◆ LoadSealedPage()

◆ LoadStructureImpl()

Member Data Documentation

◆ fBuilder

◆ fColumnMap

◆ fIdBiMap

◆ fMetrics

◆ fNextId

◆ fSources