Logo ROOT  
Reference Guide
 
Loading...
Searching...
No Matches
ROOT::Experimental::Detail::RPageSource Class Referenceabstract

Abstract interface to read data from an ntuple.

The page source is initialized with the columns of interest. Pages from those columns can then be mapped into memory. The page source also gives access to the ntuple's meta-data.

Definition at line 270 of file RPageStorage.hxx.

Classes

struct  RCounters
 Default I/O performance counters that get registered in fMetrics. More...
 

Public Member Functions

 RPageSource (const RPageSource &)=delete
 
 RPageSource (RPageSource &&)=default
 
 RPageSource (std::string_view ntupleName, const RNTupleReadOptions &fOptions)
 
virtual ~RPageSource ()
 
ColumnHandle_t AddColumn (DescriptorId_t fieldId, const RColumn &column) override
 Register a new column.
 
void Attach ()
 Open the physical storage container for the tree.
 
virtual std::unique_ptr< RPageSourceClone () const =0
 Open the same storage multiple time, e.g. for reading in multiple threads.
 
void DropColumn (ColumnHandle_t columnHandle) override
 Unregisters a column.
 
ColumnId_t GetColumnId (ColumnHandle_t columnHandle)
 
const RNTupleDescriptorGetDescriptor () const
 
virtual RNTupleMetricsGetMetrics () override
 Returns the default metrics object. Subclasses might alternatively override the method and provide their own metrics object.
 
NTupleSize_t GetNElements (ColumnHandle_t columnHandle)
 
NTupleSize_t GetNEntries ()
 
const RNTupleReadOptionsGetReadOptions () const
 
EPageStorageType GetType () final
 Whether the concrete implementation is a sink or a source.
 
virtual std::vector< std::unique_ptr< RCluster > > LoadClusters (std::span< RCluster::RKey > clusterKeys)=0
 Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages.
 
virtual void LoadSealedPage (DescriptorId_t columnId, const RClusterIndex &clusterIndex, RSealedPage &sealedPage)=0
 Read the packed and compressed bytes of a page into the memory buffer provided by selaedPage.
 
RPageSourceoperator= (const RPageSource &)=delete
 
RPageSourceoperator= (RPageSource &&)=default
 
virtual RPage PopulatePage (ColumnHandle_t columnHandle, const RClusterIndex &clusterIndex)=0
 Another version of PopulatePage that allows to specify cluster-relative indexes.
 
virtual RPage PopulatePage (ColumnHandle_t columnHandle, NTupleSize_t globalIndex)=0
 Allocates and fills a page that contains the index-th element.
 
void UnzipCluster (RCluster *cluster)
 Parallel decompression and unpacking of the pages in the given cluster.
 
- Public Member Functions inherited from ROOT::Experimental::Detail::RPageStorage
 RPageStorage (const RPageStorage &other)=delete
 
 RPageStorage (RPageStorage &&other)=default
 
 RPageStorage (std::string_view name)
 
virtual ~RPageStorage ()
 
const std::string & GetNTupleName () const
 Returns the NTuple name.
 
RPageStorageoperator= (const RPageStorage &other)=delete
 
RPageStorageoperator= (RPageStorage &&other)=default
 
virtual void ReleasePage (RPage &page)=0
 Every page store needs to be able to free pages it handed out.
 
void SetTaskScheduler (RTaskScheduler *taskScheduler)
 

Static Public Member Functions

static std::unique_ptr< RPageSourceCreate (std::string_view ntupleName, std::string_view location, const RNTupleReadOptions &options=RNTupleReadOptions())
 Guess the concrete derived page source from the file name (location)
 

Protected Member Functions

virtual RNTupleDescriptor AttachImpl ()=0
 
void EnableDefaultMetrics (const std::string &prefix)
 Enables the default set of metrics provided by RPageSource.
 
std::unique_ptr< unsigned char[]> UnsealPage (const RSealedPage &sealedPage, const RColumnElementBase &element)
 Helper for unstreaming a page.
 
virtual void UnzipClusterImpl (RCluster *)
 

Protected Attributes

RCluster::ColumnSet_t fActiveColumns
 The active columns are implicitly defined by the model fields or views.
 
std::unique_ptr< RCountersfCounters
 
std::unique_ptr< RNTupleDecompressorfDecompressor
 Helper to unzip pages and header/footer; comprises a 16MB (kMAXZIPBUF) unzip buffer.
 
RNTupleDescriptor fDescriptor
 
RNTupleMetrics fMetrics
 Wraps the I/O counters and is observed by the RNTupleReader metrics.
 
RNTupleReadOptions fOptions
 
- Protected Attributes inherited from ROOT::Experimental::Detail::RPageStorage
std::string fNTupleName
 
RTaskSchedulerfTaskScheduler = nullptr
 

Additional Inherited Members

- Public Types inherited from ROOT::Experimental::Detail::RPageStorage
using ColumnHandle_t = RColumnHandle
 The column handle identifies a column with the current open page storage.
 

#include <ROOT/RPageStorage.hxx>

Inheritance diagram for ROOT::Experimental::Detail::RPageSource:
[legend]

Constructor & Destructor Documentation

◆ RPageSource() [1/3]

ROOT::Experimental::Detail::RPageSource::RPageSource ( std::string_view  ntupleName,
const RNTupleReadOptions fOptions 
)

Definition at line 49 of file RPageStorage.cxx.

◆ RPageSource() [2/3]

ROOT::Experimental::Detail::RPageSource::RPageSource ( const RPageSource )
delete

◆ RPageSource() [3/3]

ROOT::Experimental::Detail::RPageSource::RPageSource ( RPageSource &&  )
default

◆ ~RPageSource()

ROOT::Experimental::Detail::RPageSource::~RPageSource ( )
virtual

Definition at line 54 of file RPageStorage.cxx.

Member Function Documentation

◆ AddColumn()

ROOT::Experimental::Detail::RPageStorage::ColumnHandle_t ROOT::Experimental::Detail::RPageSource::AddColumn ( DescriptorId_t  fieldId,
const RColumn column 
)
overridevirtual

Register a new column.

When reading, the column must exist in the ntuple on disk corresponding to the meta-data. When writing, every column can only be attached once.

Implements ROOT::Experimental::Detail::RPageStorage.

Reimplemented in ROOT::Experimental::Detail::RPageSourceFriends.

Definition at line 78 of file RPageStorage.cxx.

◆ Attach()

void ROOT::Experimental::Detail::RPageSource::Attach ( )
inline

Open the physical storage container for the tree.

Definition at line 345 of file RPageStorage.hxx.

◆ AttachImpl()

virtual RNTupleDescriptor ROOT::Experimental::Detail::RPageSource::AttachImpl ( )
protectedpure virtual

◆ Clone()

virtual std::unique_ptr< RPageSource > ROOT::Experimental::Detail::RPageSource::Clone ( ) const
pure virtual

Open the same storage multiple time, e.g. for reading in multiple threads.

Implemented in ROOT::Experimental::Detail::RPageSourceFriends, ROOT::Experimental::Detail::RPageSourceDaos, and ROOT::Experimental::Detail::RPageSourceFile.

◆ Create()

std::unique_ptr< ROOT::Experimental::Detail::RPageSource > ROOT::Experimental::Detail::RPageSource::Create ( std::string_view  ntupleName,
std::string_view  location,
const RNTupleReadOptions options = RNTupleReadOptions() 
)
static

Guess the concrete derived page source from the file name (location)

Definition at line 58 of file RPageStorage.cxx.

◆ DropColumn()

void ROOT::Experimental::Detail::RPageSource::DropColumn ( ColumnHandle_t  columnHandle)
overridevirtual

Unregisters a column.

A page source decreases the reference counter for the corresponding active column. For a page sink, dropping columns is currently a no-op.

Implements ROOT::Experimental::Detail::RPageStorage.

Reimplemented in ROOT::Experimental::Detail::RPageSourceFriends.

Definition at line 87 of file RPageStorage.cxx.

◆ EnableDefaultMetrics()

void ROOT::Experimental::Detail::RPageSource::EnableDefaultMetrics ( const std::string &  prefix)
protected

Enables the default set of metrics provided by RPageSource.

prefix will be used as the prefix for the counters registered in the internal RNTupleMetrics object. A subclass using the default set of metrics is responsible for updating the counters appropriately, e.g. fCounters->fNRead.Inc() Alternatively, a subclass might provide its own RNTupleMetrics object by overriding the GetMetrics() member function.

Definition at line 142 of file RPageStorage.cxx.

◆ GetColumnId()

ROOT::Experimental::ColumnId_t ROOT::Experimental::Detail::RPageSource::GetColumnId ( ColumnHandle_t  columnHandle)

Definition at line 102 of file RPageStorage.cxx.

◆ GetDescriptor()

const RNTupleDescriptor & ROOT::Experimental::Detail::RPageSource::GetDescriptor ( ) const
inline

Definition at line 339 of file RPageStorage.hxx.

◆ GetMetrics()

virtual RNTupleMetrics & ROOT::Experimental::Detail::RPageSource::GetMetrics ( )
inlineoverridevirtual

Returns the default metrics object. Subclasses might alternatively override the method and provide their own metrics object.

Implements ROOT::Experimental::Detail::RPageStorage.

Reimplemented in ROOT::Experimental::Detail::RPageSourceFriends.

Definition at line 379 of file RPageStorage.hxx.

◆ GetNElements()

ROOT::Experimental::NTupleSize_t ROOT::Experimental::Detail::RPageSource::GetNElements ( ColumnHandle_t  columnHandle)

Definition at line 97 of file RPageStorage.cxx.

◆ GetNEntries()

ROOT::Experimental::NTupleSize_t ROOT::Experimental::Detail::RPageSource::GetNEntries ( )

Definition at line 92 of file RPageStorage.cxx.

◆ GetReadOptions()

const RNTupleReadOptions & ROOT::Experimental::Detail::RPageSource::GetReadOptions ( ) const
inline

Definition at line 340 of file RPageStorage.hxx.

◆ GetType()

EPageStorageType ROOT::Experimental::Detail::RPageSource::GetType ( )
inlinefinalvirtual

Whether the concrete implementation is a sink or a source.

Implements ROOT::Experimental::Detail::RPageStorage.

Definition at line 338 of file RPageStorage.hxx.

◆ LoadClusters()

virtual std::vector< std::unique_ptr< RCluster > > ROOT::Experimental::Detail::RPageSource::LoadClusters ( std::span< RCluster::RKey clusterKeys)
pure virtual

Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages.

The page source may load more columns than the minimal necessary set from columns. To indicate which columns have been loaded, LoadClusters() must mark them with SetColumnAvailable(). That includes the ones from the columns that don't have pages; otherwise subsequent requests for the cluster would assume an incomplete cluster and trigger loading again. LoadClusters() is typically called from the I/O thread of a cluster pool, i.e. the method runs concurrently to other methods of the page source.

Implemented in ROOT::Experimental::Detail::RPageSourceFriends, ROOT::Experimental::Detail::RPageSourceDaos, and ROOT::Experimental::Detail::RPageSourceFile.

◆ LoadSealedPage()

virtual void ROOT::Experimental::Detail::RPageSource::LoadSealedPage ( DescriptorId_t  columnId,
const RClusterIndex clusterIndex,
RSealedPage sealedPage 
)
pure virtual

Read the packed and compressed bytes of a page into the memory buffer provided by selaedPage.

The sealed page can be used subsequently in a call to RPageSink::CommitSealedPage. The fSize and fNElements member of the sealedPage parameters are always set. If sealedPage.fBuffer is nullptr, no data will be copied but the returned size information can be used by the caller to allocate a large enough buffer and call LoadSealedPage again.

Implemented in ROOT::Experimental::Detail::RPageSourceFriends, ROOT::Experimental::Detail::RPageSourceDaos, and ROOT::Experimental::Detail::RPageSourceFile.

◆ operator=() [1/2]

RPageSource & ROOT::Experimental::Detail::RPageSource::operator= ( const RPageSource )
delete

◆ operator=() [2/2]

RPageSource & ROOT::Experimental::Detail::RPageSource::operator= ( RPageSource &&  )
default

◆ PopulatePage() [1/2]

virtual RPage ROOT::Experimental::Detail::RPageSource::PopulatePage ( ColumnHandle_t  columnHandle,
const RClusterIndex clusterIndex 
)
pure virtual

Another version of PopulatePage that allows to specify cluster-relative indexes.

Implemented in ROOT::Experimental::Detail::RPageSourceFriends, ROOT::Experimental::Detail::RPageSourceDaos, and ROOT::Experimental::Detail::RPageSourceFile.

◆ PopulatePage() [2/2]

virtual RPage ROOT::Experimental::Detail::RPageSource::PopulatePage ( ColumnHandle_t  columnHandle,
NTupleSize_t  globalIndex 
)
pure virtual

Allocates and fills a page that contains the index-th element.

Implemented in ROOT::Experimental::Detail::RPageSourceFriends, ROOT::Experimental::Detail::RPageSourceDaos, and ROOT::Experimental::Detail::RPageSourceFile.

◆ UnsealPage()

std::unique_ptr< unsigned char[]> ROOT::Experimental::Detail::RPageSource::UnsealPage ( const RSealedPage sealedPage,
const RColumnElementBase element 
)
protected

Helper for unstreaming a page.

This is commonly used in derived, concrete page sources. The implementation currently always makes a memory copy, even if the sealed page is uncompressed and in the final memory layout. The optimization of directly mapping pages is left to the concrete page source implementations. Usage of this method requires construction of fDecompressor.

Definition at line 115 of file RPageStorage.cxx.

◆ UnzipCluster()

void ROOT::Experimental::Detail::RPageSource::UnzipCluster ( RCluster cluster)

Parallel decompression and unpacking of the pages in the given cluster.

The unzipped pages are supposed to be preloaded in a page pool attached to the source. The method is triggered by the cluster pool's unzip thread. It is an optional optimization, the method can safely do nothing. In particular, the actual implementation will only run if a task scheduler is set. In practice, a task scheduler is set if implicit multi-threading is turned on.

Definition at line 108 of file RPageStorage.cxx.

◆ UnzipClusterImpl()

virtual void ROOT::Experimental::Detail::RPageSource::UnzipClusterImpl ( RCluster )
inlineprotectedvirtual

Member Data Documentation

◆ fActiveColumns

RCluster::ColumnSet_t ROOT::Experimental::Detail::RPageSource::fActiveColumns
protected

The active columns are implicitly defined by the model fields or views.

Definition at line 299 of file RPageStorage.hxx.

◆ fCounters

std::unique_ptr<RCounters> ROOT::Experimental::Detail::RPageSource::fCounters
protected

Definition at line 292 of file RPageStorage.hxx.

◆ fDecompressor

std::unique_ptr<RNTupleDecompressor> ROOT::Experimental::Detail::RPageSource::fDecompressor
protected

Helper to unzip pages and header/footer; comprises a 16MB (kMAXZIPBUF) unzip buffer.

Not all page sources need a decompressor (e.g. virtual ones for chains and friends don't), thus we leave it up to the derived class whether or not the decompressor gets constructed.

Definition at line 304 of file RPageStorage.hxx.

◆ fDescriptor

RNTupleDescriptor ROOT::Experimental::Detail::RPageSource::fDescriptor
protected

Definition at line 297 of file RPageStorage.hxx.

◆ fMetrics

RNTupleMetrics ROOT::Experimental::Detail::RPageSource::fMetrics
protected

Wraps the I/O counters and is observed by the RNTupleReader metrics.

Definition at line 294 of file RPageStorage.hxx.

◆ fOptions

RNTupleReadOptions ROOT::Experimental::Detail::RPageSource::fOptions
protected

Definition at line 296 of file RPageStorage.hxx.

Libraries for ROOT::Experimental::Detail::RPageSource:

The documentation for this class was generated from the following files: