Storage provider that reads ntuple pages from a file.
Definition at line 119 of file RPageStorageFile.hxx.
Public Member Functions | |
RPageSourceFile (const RPageSourceFile &)=delete | |
RPageSourceFile (RPageSourceFile &&)=default | |
RPageSourceFile (std::string_view ntupleName, std::string_view path, const RNTupleReadOptions &options) | |
virtual | ~RPageSourceFile () |
std::unique_ptr< RPageSource > | Clone () const final |
The cloned page source creates a new raw file and reader and opens its own file descriptor to the data. | |
std::vector< std::unique_ptr< RCluster > > | LoadClusters (std::span< RCluster::RKey > clusterKeys) final |
Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages. | |
void | LoadSealedPage (DescriptorId_t columnId, const RClusterIndex &clusterIndex, RSealedPage &sealedPage) final |
Read the packed and compressed bytes of a page into the memory buffer provided by selaedPage. | |
RPageSourceFile & | operator= (const RPageSourceFile &)=delete |
RPageSourceFile & | operator= (RPageSourceFile &&)=default |
RPage | PopulatePage (ColumnHandle_t columnHandle, const RClusterIndex &clusterIndex) final |
Another version of PopulatePage that allows to specify cluster-relative indexes. | |
RPage | PopulatePage (ColumnHandle_t columnHandle, NTupleSize_t globalIndex) final |
Allocates and fills a page that contains the index-th element. | |
void | ReleasePage (RPage &page) final |
Every page store needs to be able to free pages it handed out. | |
![]() | |
RPageSource (const RPageSource &)=delete | |
RPageSource (RPageSource &&)=default | |
RPageSource (std::string_view ntupleName, const RNTupleReadOptions &fOptions) | |
virtual | ~RPageSource () |
ColumnHandle_t | AddColumn (DescriptorId_t fieldId, const RColumn &column) override |
Register a new column. | |
void | Attach () |
Open the physical storage container for the tree. | |
void | DropColumn (ColumnHandle_t columnHandle) override |
Unregisters a column. | |
ColumnId_t | GetColumnId (ColumnHandle_t columnHandle) |
const RNTupleDescriptor & | GetDescriptor () const |
virtual RNTupleMetrics & | GetMetrics () override |
Returns the default metrics object. Subclasses might alternatively override the method and provide their own metrics object. | |
NTupleSize_t | GetNElements (ColumnHandle_t columnHandle) |
NTupleSize_t | GetNEntries () |
const RNTupleReadOptions & | GetReadOptions () const |
EPageStorageType | GetType () final |
Whether the concrete implementation is a sink or a source. | |
RPageSource & | operator= (const RPageSource &)=delete |
RPageSource & | operator= (RPageSource &&)=default |
void | UnzipCluster (RCluster *cluster) |
Parallel decompression and unpacking of the pages in the given cluster. | |
![]() | |
RPageStorage (const RPageStorage &other)=delete | |
RPageStorage (RPageStorage &&other)=default | |
RPageStorage (std::string_view name) | |
virtual | ~RPageStorage () |
const std::string & | GetNTupleName () const |
Returns the NTuple name. | |
RPageStorage & | operator= (const RPageStorage &other)=delete |
RPageStorage & | operator= (RPageStorage &&other)=default |
void | SetTaskScheduler (RTaskScheduler *taskScheduler) |
Static Public Attributes | |
static constexpr std::size_t | kMaxPageSize = 1024 * 1024 |
Cannot process pages larger than 1MB. | |
Protected Member Functions | |
RNTupleDescriptor | AttachImpl () final |
void | UnzipClusterImpl (RCluster *cluster) final |
![]() | |
void | EnableDefaultMetrics (const std::string &prefix) |
Enables the default set of metrics provided by RPageSource. | |
std::unique_ptr< unsigned char[]> | UnsealPage (const RSealedPage &sealedPage, const RColumnElementBase &element) |
Helper for unstreaming a page. | |
Private Member Functions | |
RPageSourceFile (std::string_view ntupleName, const RNTupleReadOptions &options) | |
RPage | PopulatePageFromCluster (ColumnHandle_t columnHandle, const RClusterDescriptor &clusterDescriptor, ClusterSize_t::ValueType idxInCluster) |
std::unique_ptr< RCluster > | PrepareSingleCluster (const RCluster::RKey &clusterKey, std::vector< ROOT::Internal::RRawFile::RIOVec > &readRequests) |
Helper function for LoadClusters: it prepares the memory buffer (page map) and the read requests for a given cluster and columns. | |
Private Attributes | |
std::unique_ptr< RClusterPool > | fClusterPool |
The cluster pool asynchronously preloads the next few clusters. | |
RCluster * | fCurrentCluster = nullptr |
The last cluster from which a page got populated. Points into fClusterPool->fPool. | |
std::unique_ptr< ROOT::Internal::RRawFile > | fFile |
An RRawFile is used to request the necessary byte ranges from a local or a remote file. | |
std::unique_ptr< RPageAllocatorFile > | fPageAllocator |
Populated pages might be shared; there memory buffer is managed by the RPageAllocatorFile. | |
std::shared_ptr< RPagePool > | fPagePool |
The page pool might, at some point, be used by multiple page sources. | |
Internal::RMiniFileReader | fReader |
Takes the fFile to read ntuple blobs from it. | |
Additional Inherited Members | |
![]() | |
using | ColumnHandle_t = RColumnHandle |
The column handle identifies a column with the current open page storage. | |
![]() | |
static std::unique_ptr< RPageSource > | Create (std::string_view ntupleName, std::string_view location, const RNTupleReadOptions &options=RNTupleReadOptions()) |
Guess the concrete derived page source from the file name (location) | |
![]() | |
RCluster::ColumnSet_t | fActiveColumns |
The active columns are implicitly defined by the model fields or views. | |
std::unique_ptr< RCounters > | fCounters |
std::unique_ptr< RNTupleDecompressor > | fDecompressor |
Helper to unzip pages and header/footer; comprises a 16MB (kMAXZIPBUF) unzip buffer. | |
RNTupleDescriptor | fDescriptor |
RNTupleMetrics | fMetrics |
Wraps the I/O counters and is observed by the RNTupleReader metrics. | |
RNTupleReadOptions | fOptions |
![]() | |
std::string | fNTupleName |
RTaskScheduler * | fTaskScheduler = nullptr |
#include <ROOT/RPageStorageFile.hxx>
|
private |
Definition at line 237 of file RPageStorageFile.cxx.
ROOT::Experimental::Detail::RPageSourceFile::RPageSourceFile | ( | std::string_view | ntupleName, |
std::string_view | path, | ||
const RNTupleReadOptions & | options | ||
) |
Definition at line 249 of file RPageStorageFile.cxx.
|
delete |
|
default |
|
virtualdefault |
|
finalprotectedvirtual |
Implements ROOT::Experimental::Detail::RPageSource.
Definition at line 262 of file RPageStorageFile.cxx.
|
finalvirtual |
The cloned page source creates a new raw file and reader and opens its own file descriptor to the data.
The meta-data (header and footer) is reread and parsed by the clone.
Implements ROOT::Experimental::Detail::RPageSource.
Definition at line 411 of file RPageStorageFile.cxx.
|
finalvirtual |
Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages.
The page source may load more columns than the minimal necessary set from columns
. To indicate which columns have been loaded, LoadClusters() must mark them with SetColumnAvailable(). That includes the ones from the columns
that don't have pages; otherwise subsequent requests for the cluster would assume an incomplete cluster and trigger loading again. LoadClusters() is typically called from the I/O thread of a cluster pool, i.e. the method runs concurrently to other methods of the page source.
Implements ROOT::Experimental::Detail::RPageSource.
Definition at line 535 of file RPageStorageFile.cxx.
|
finalvirtual |
Read the packed and compressed bytes of a page into the memory buffer provided by selaedPage.
The sealed page can be used subsequently in a call to RPageSink::CommitSealedPage. The fSize and fNElements member of the sealedPage parameters are always set. If sealedPage.fBuffer is nullptr, no data will be copied but the returned size information can be used by the caller to allocate a large enough buffer and call LoadSealedPage again.
Implements ROOT::Experimental::Detail::RPageSource.
Definition at line 302 of file RPageStorageFile.cxx.
|
delete |
|
default |
|
finalvirtual |
Another version of PopulatePage that allows to specify cluster-relative indexes.
Implements ROOT::Experimental::Detail::RPageSource.
Definition at line 391 of file RPageStorageFile.cxx.
|
finalvirtual |
Allocates and fills a page that contains the index-th element.
Implements ROOT::Experimental::Detail::RPageSource.
Definition at line 374 of file RPageStorageFile.cxx.
|
private |
Definition at line 317 of file RPageStorageFile.cxx.
|
private |
Helper function for LoadClusters: it prepares the memory buffer (page map) and the read requests for a given cluster and columns.
The reead requests are appended to the provided vector. This way, requests can be collected for multiple clusters before sending them to RRawFile::ReadV().
Definition at line 420 of file RPageStorageFile.cxx.
Every page store needs to be able to free pages it handed out.
But Sinks and sources have different means of allocating pages.
Implements ROOT::Experimental::Detail::RPageStorage.
Definition at line 406 of file RPageStorageFile.cxx.
|
finalprotectedvirtual |
Reimplemented from ROOT::Experimental::Detail::RPageSource.
Definition at line 558 of file RPageStorageFile.cxx.
|
private |
The cluster pool asynchronously preloads the next few clusters.
Definition at line 136 of file RPageStorageFile.hxx.
|
private |
The last cluster from which a page got populated. Points into fClusterPool->fPool.
Definition at line 130 of file RPageStorageFile.hxx.
|
private |
An RRawFile is used to request the necessary byte ranges from a local or a remote file.
Definition at line 132 of file RPageStorageFile.hxx.
|
private |
Populated pages might be shared; there memory buffer is managed by the RPageAllocatorFile.
Definition at line 126 of file RPageStorageFile.hxx.
|
private |
The page pool might, at some point, be used by multiple page sources.
Definition at line 128 of file RPageStorageFile.hxx.
|
private |
Takes the fFile to read ntuple blobs from it.
Definition at line 134 of file RPageStorageFile.hxx.
|
staticconstexpr |
Cannot process pages larger than 1MB.
Definition at line 122 of file RPageStorageFile.hxx.