Abstract interface to read data from an ntuple.
The page source is initialized with the columns of interest. Pages from those columns can then be mapped into memory. The page source also gives access to the ntuple's meta-data.
Definition at line 270 of file RPageStorage.hxx.
Classes | |
struct | RCounters |
Default I/O performance counters that get registered in fMetrics. More... | |
Public Member Functions | |
RPageSource (const RPageSource &)=delete | |
RPageSource (RPageSource &&)=default | |
RPageSource (std::string_view ntupleName, const RNTupleReadOptions &fOptions) | |
virtual | ~RPageSource () |
ColumnHandle_t | AddColumn (DescriptorId_t fieldId, const RColumn &column) override |
Register a new column. | |
void | Attach () |
Open the physical storage container for the tree. | |
virtual std::unique_ptr< RPageSource > | Clone () const =0 |
Open the same storage multiple time, e.g. for reading in multiple threads. | |
void | DropColumn (ColumnHandle_t columnHandle) override |
Unregisters a column. | |
ColumnId_t | GetColumnId (ColumnHandle_t columnHandle) |
const RNTupleDescriptor & | GetDescriptor () const |
virtual RNTupleMetrics & | GetMetrics () override |
Returns the default metrics object. Subclasses might alternatively override the method and provide their own metrics object. | |
NTupleSize_t | GetNElements (ColumnHandle_t columnHandle) |
NTupleSize_t | GetNEntries () |
const RNTupleReadOptions & | GetReadOptions () const |
EPageStorageType | GetType () final |
Whether the concrete implementation is a sink or a source. | |
virtual std::vector< std::unique_ptr< RCluster > > | LoadClusters (std::span< RCluster::RKey > clusterKeys)=0 |
Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages. | |
virtual void | LoadSealedPage (DescriptorId_t columnId, const RClusterIndex &clusterIndex, RSealedPage &sealedPage)=0 |
Read the packed and compressed bytes of a page into the memory buffer provided by selaedPage. | |
RPageSource & | operator= (const RPageSource &)=delete |
RPageSource & | operator= (RPageSource &&)=default |
virtual RPage | PopulatePage (ColumnHandle_t columnHandle, const RClusterIndex &clusterIndex)=0 |
Another version of PopulatePage that allows to specify cluster-relative indexes. | |
virtual RPage | PopulatePage (ColumnHandle_t columnHandle, NTupleSize_t globalIndex)=0 |
Allocates and fills a page that contains the index-th element. | |
void | UnzipCluster (RCluster *cluster) |
Parallel decompression and unpacking of the pages in the given cluster. | |
![]() | |
RPageStorage (const RPageStorage &other)=delete | |
RPageStorage (RPageStorage &&other)=default | |
RPageStorage (std::string_view name) | |
virtual | ~RPageStorage () |
const std::string & | GetNTupleName () const |
Returns the NTuple name. | |
RPageStorage & | operator= (const RPageStorage &other)=delete |
RPageStorage & | operator= (RPageStorage &&other)=default |
virtual void | ReleasePage (RPage &page)=0 |
Every page store needs to be able to free pages it handed out. | |
void | SetTaskScheduler (RTaskScheduler *taskScheduler) |
Static Public Member Functions | |
static std::unique_ptr< RPageSource > | Create (std::string_view ntupleName, std::string_view location, const RNTupleReadOptions &options=RNTupleReadOptions()) |
Guess the concrete derived page source from the file name (location) | |
Protected Member Functions | |
virtual RNTupleDescriptor | AttachImpl ()=0 |
void | EnableDefaultMetrics (const std::string &prefix) |
Enables the default set of metrics provided by RPageSource. | |
std::unique_ptr< unsigned char[]> | UnsealPage (const RSealedPage &sealedPage, const RColumnElementBase &element) |
Helper for unstreaming a page. | |
virtual void | UnzipClusterImpl (RCluster *) |
Protected Attributes | |
RCluster::ColumnSet_t | fActiveColumns |
The active columns are implicitly defined by the model fields or views. | |
std::unique_ptr< RCounters > | fCounters |
std::unique_ptr< RNTupleDecompressor > | fDecompressor |
Helper to unzip pages and header/footer; comprises a 16MB (kMAXZIPBUF) unzip buffer. | |
RNTupleDescriptor | fDescriptor |
RNTupleMetrics | fMetrics |
Wraps the I/O counters and is observed by the RNTupleReader metrics. | |
RNTupleReadOptions | fOptions |
![]() | |
std::string | fNTupleName |
RTaskScheduler * | fTaskScheduler = nullptr |
Additional Inherited Members | |
![]() | |
using | ColumnHandle_t = RColumnHandle |
The column handle identifies a column with the current open page storage. | |
#include <ROOT/RPageStorage.hxx>
ROOT::Experimental::Detail::RPageSource::RPageSource | ( | std::string_view | ntupleName, |
const RNTupleReadOptions & | fOptions | ||
) |
Definition at line 49 of file RPageStorage.cxx.
|
delete |
|
default |
|
virtual |
Definition at line 54 of file RPageStorage.cxx.
|
overridevirtual |
Register a new column.
When reading, the column must exist in the ntuple on disk corresponding to the meta-data. When writing, every column can only be attached once.
Implements ROOT::Experimental::Detail::RPageStorage.
Reimplemented in ROOT::Experimental::Detail::RPageSourceFriends.
Definition at line 78 of file RPageStorage.cxx.
|
inline |
Open the physical storage container for the tree.
Definition at line 345 of file RPageStorage.hxx.
|
protectedpure virtual |
|
pure virtual |
Open the same storage multiple time, e.g. for reading in multiple threads.
Implemented in ROOT::Experimental::Detail::RPageSourceFriends, ROOT::Experimental::Detail::RPageSourceDaos, and ROOT::Experimental::Detail::RPageSourceFile.
|
static |
Guess the concrete derived page source from the file name (location)
Definition at line 58 of file RPageStorage.cxx.
|
overridevirtual |
Unregisters a column.
A page source decreases the reference counter for the corresponding active column. For a page sink, dropping columns is currently a no-op.
Implements ROOT::Experimental::Detail::RPageStorage.
Reimplemented in ROOT::Experimental::Detail::RPageSourceFriends.
Definition at line 87 of file RPageStorage.cxx.
|
protected |
Enables the default set of metrics provided by RPageSource.
prefix
will be used as the prefix for the counters registered in the internal RNTupleMetrics object. A subclass using the default set of metrics is responsible for updating the counters appropriately, e.g. fCounters->fNRead.Inc()
Alternatively, a subclass might provide its own RNTupleMetrics object by overriding the GetMetrics() member function.
Definition at line 142 of file RPageStorage.cxx.
ROOT::Experimental::ColumnId_t ROOT::Experimental::Detail::RPageSource::GetColumnId | ( | ColumnHandle_t | columnHandle | ) |
Definition at line 102 of file RPageStorage.cxx.
|
inline |
Definition at line 339 of file RPageStorage.hxx.
|
inlineoverridevirtual |
Returns the default metrics object. Subclasses might alternatively override the method and provide their own metrics object.
Implements ROOT::Experimental::Detail::RPageStorage.
Reimplemented in ROOT::Experimental::Detail::RPageSourceFriends.
Definition at line 379 of file RPageStorage.hxx.
ROOT::Experimental::NTupleSize_t ROOT::Experimental::Detail::RPageSource::GetNElements | ( | ColumnHandle_t | columnHandle | ) |
Definition at line 97 of file RPageStorage.cxx.
ROOT::Experimental::NTupleSize_t ROOT::Experimental::Detail::RPageSource::GetNEntries | ( | ) |
Definition at line 92 of file RPageStorage.cxx.
|
inline |
Definition at line 340 of file RPageStorage.hxx.
|
inlinefinalvirtual |
Whether the concrete implementation is a sink or a source.
Implements ROOT::Experimental::Detail::RPageStorage.
Definition at line 338 of file RPageStorage.hxx.
|
pure virtual |
Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages.
The page source may load more columns than the minimal necessary set from columns
. To indicate which columns have been loaded, LoadClusters() must mark them with SetColumnAvailable(). That includes the ones from the columns
that don't have pages; otherwise subsequent requests for the cluster would assume an incomplete cluster and trigger loading again. LoadClusters() is typically called from the I/O thread of a cluster pool, i.e. the method runs concurrently to other methods of the page source.
Implemented in ROOT::Experimental::Detail::RPageSourceFriends, ROOT::Experimental::Detail::RPageSourceDaos, and ROOT::Experimental::Detail::RPageSourceFile.
|
pure virtual |
Read the packed and compressed bytes of a page into the memory buffer provided by selaedPage.
The sealed page can be used subsequently in a call to RPageSink::CommitSealedPage. The fSize and fNElements member of the sealedPage parameters are always set. If sealedPage.fBuffer is nullptr, no data will be copied but the returned size information can be used by the caller to allocate a large enough buffer and call LoadSealedPage again.
Implemented in ROOT::Experimental::Detail::RPageSourceFriends, ROOT::Experimental::Detail::RPageSourceDaos, and ROOT::Experimental::Detail::RPageSourceFile.
|
delete |
|
default |
|
pure virtual |
Another version of PopulatePage that allows to specify cluster-relative indexes.
Implemented in ROOT::Experimental::Detail::RPageSourceFriends, ROOT::Experimental::Detail::RPageSourceDaos, and ROOT::Experimental::Detail::RPageSourceFile.
|
pure virtual |
Allocates and fills a page that contains the index-th element.
Implemented in ROOT::Experimental::Detail::RPageSourceFriends, ROOT::Experimental::Detail::RPageSourceDaos, and ROOT::Experimental::Detail::RPageSourceFile.
|
protected |
Helper for unstreaming a page.
This is commonly used in derived, concrete page sources. The implementation currently always makes a memory copy, even if the sealed page is uncompressed and in the final memory layout. The optimization of directly mapping pages is left to the concrete page source implementations. Usage of this method requires construction of fDecompressor.
Definition at line 115 of file RPageStorage.cxx.
Parallel decompression and unpacking of the pages in the given cluster.
The unzipped pages are supposed to be preloaded in a page pool attached to the source. The method is triggered by the cluster pool's unzip thread. It is an optional optimization, the method can safely do nothing. In particular, the actual implementation will only run if a task scheduler is set. In practice, a task scheduler is set if implicit multi-threading is turned on.
Definition at line 108 of file RPageStorage.cxx.
|
inlineprotectedvirtual |
Reimplemented in ROOT::Experimental::Detail::RPageSourceDaos, and ROOT::Experimental::Detail::RPageSourceFile.
Definition at line 308 of file RPageStorage.hxx.
|
protected |
The active columns are implicitly defined by the model fields or views.
Definition at line 299 of file RPageStorage.hxx.
|
protected |
Definition at line 292 of file RPageStorage.hxx.
|
protected |
Helper to unzip pages and header/footer; comprises a 16MB (kMAXZIPBUF) unzip buffer.
Not all page sources need a decompressor (e.g. virtual ones for chains and friends don't), thus we leave it up to the derived class whether or not the decompressor gets constructed.
Definition at line 304 of file RPageStorage.hxx.
|
protected |
Definition at line 297 of file RPageStorage.hxx.
|
protected |
Wraps the I/O counters and is observed by the RNTupleReader metrics.
Definition at line 294 of file RPageStorage.hxx.
|
protected |
Definition at line 296 of file RPageStorage.hxx.