Storage provider that reads ntuple pages from a file.
Definition at line 117 of file RPageStorageFile.hxx.
Classes | |
struct | RStructureBuffer |
Holds the uncompressed header and footer. More... | |
Public Member Functions | |
RPageSourceFile (const RPageSourceFile &)=delete | |
RPageSourceFile (RPageSourceFile &&)=delete | |
RPageSourceFile (std::string_view ntupleName, std::string_view path, const RNTupleReadOptions &options) | |
RPageSourceFile (std::string_view ntupleName, std::unique_ptr< ROOT::Internal::RRawFile > file, const RNTupleReadOptions &options) | |
~RPageSourceFile () override | |
std::vector< std::unique_ptr< RCluster > > | LoadClusters (std::span< RCluster::RKey > clusterKeys) final |
Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages. | |
void | LoadSealedPage (DescriptorId_t physicalColumnId, RClusterIndex clusterIndex, RSealedPage &sealedPage) final |
Read the packed and compressed bytes of a page into the memory buffer provided by sealedPage . | |
RPageSourceFile & | operator= (const RPageSourceFile &)=delete |
RPageSourceFile & | operator= (RPageSourceFile &&)=delete |
Public Member Functions inherited from ROOT::Experimental::Internal::RPageSource | |
RPageSource (const RPageSource &)=delete | |
RPageSource (RPageSource &&)=delete | |
RPageSource (std::string_view ntupleName, const RNTupleReadOptions &fOptions) | |
~RPageSource () override | |
ColumnHandle_t | AddColumn (DescriptorId_t fieldId, RColumn &column) override |
Register a new column. | |
void | Attach () |
Open the physical storage container and deserialize header and footer. | |
std::unique_ptr< RPageSource > | Clone () const |
Open the same storage multiple time, e.g. | |
void | DropColumn (ColumnHandle_t columnHandle) override |
Unregisters a column. | |
REntryRange | GetEntryRange () const |
NTupleSize_t | GetNElements (ColumnHandle_t columnHandle) |
NTupleSize_t | GetNEntries () |
const RNTupleReadOptions & | GetReadOptions () const |
const RSharedDescriptorGuard | GetSharedDescriptorGuard () const |
Takes the read lock for the descriptor. | |
EPageStorageType | GetType () final |
Whether the concrete implementation is a sink or a source. | |
virtual RPageRef | LoadPage (ColumnHandle_t columnHandle, NTupleSize_t globalIndex) |
Allocates and fills a page that contains the index-th element. | |
virtual RPageRef | LoadPage (ColumnHandle_t columnHandle, RClusterIndex clusterIndex) |
Another version of LoadPage that allows to specify cluster-relative indexes. | |
void | LoadStructure () |
Loads header and footer without decompressing or deserializing them. | |
RPageSource & | operator= (const RPageSource &)=delete |
RPageSource & | operator= (RPageSource &&)=delete |
void | SetEntryRange (const REntryRange &range) |
Promise to only read from the given entry range. | |
RResult< RPage > | UnsealPage (const RSealedPage &sealedPage, const RColumnElementBase &element, DescriptorId_t physicalColumnId) |
void | UnzipCluster (RCluster *cluster) |
Parallel decompression and unpacking of the pages in the given cluster. | |
Public Member Functions inherited from ROOT::Experimental::Internal::RPageStorage | |
RPageStorage (const RPageStorage &other)=delete | |
RPageStorage (RPageStorage &&other)=default | |
RPageStorage (std::string_view name) | |
virtual | ~RPageStorage () |
ColumnId_t | GetColumnId (ColumnHandle_t columnHandle) const |
virtual Detail::RNTupleMetrics & | GetMetrics () |
Returns the default metrics object. | |
const std::string & | GetNTupleName () const |
Returns the NTuple name. | |
RPageStorage & | operator= (const RPageStorage &other)=delete |
RPageStorage & | operator= (RPageStorage &&other)=default |
void | SetTaskScheduler (RTaskScheduler *taskScheduler) |
Static Public Member Functions | |
static std::unique_ptr< RPageSourceFile > | CreateFromAnchor (const RNTuple &anchor, const RNTupleReadOptions &options=RNTupleReadOptions()) |
Used from the RNTuple class to build a datasource if the anchor is already available. | |
Static Public Member Functions inherited from ROOT::Experimental::Internal::RPageSource | |
static std::unique_ptr< RPageSource > | Create (std::string_view ntupleName, std::string_view location, const RNTupleReadOptions &options=RNTupleReadOptions()) |
Guess the concrete derived page source from the file name (location) | |
static RResult< RPage > | UnsealPage (const RSealedPage &sealedPage, const RColumnElementBase &element, DescriptorId_t physicalColumnId, RPageAllocator &pageAlloc) |
Helper for unstreaming a page. | |
Protected Member Functions | |
RNTupleDescriptor | AttachImpl () final |
LoadStructureImpl() has been called before AttachImpl() is called | |
std::unique_ptr< RPageSource > | CloneImpl () const final |
The cloned page source creates a new raw file and reader and opens its own file descriptor to the data. | |
RPageRef | LoadPageImpl (ColumnHandle_t columnHandle, const RClusterInfo &clusterInfo, ClusterSize_t::ValueType idxInCluster) final |
void | LoadStructureImpl () final |
Protected Member Functions inherited from ROOT::Experimental::Internal::RPageSource | |
void | EnableDefaultMetrics (const std::string &prefix) |
Enables the default set of metrics provided by RPageSource. | |
RExclDescriptorGuard | GetExclDescriptorGuard () |
Note that the underlying lock is not recursive. See GetSharedDescriptorGuard() for further information. | |
void | PrepareLoadCluster (const RCluster::RKey &clusterKey, ROnDiskPageMap &pageZeroMap, std::function< void(DescriptorId_t, NTupleSize_t, const RClusterDescriptor::RPageRange::RPageInfo &)> perPageFunc) |
Prepare a page range read for the column set in clusterKey . | |
virtual void | UnzipClusterImpl (RCluster *cluster) |
Protected Member Functions inherited from ROOT::Experimental::Internal::RPageStorage | |
void | WaitForAllTasks () |
Private Member Functions | |
RPageSourceFile (std::string_view ntupleName, const RNTupleReadOptions &options) | |
std::unique_ptr< RCluster > | PrepareSingleCluster (const RCluster::RKey &clusterKey, std::vector< ROOT::Internal::RRawFile::RIOVec > &readRequests) |
Helper function for LoadClusters: it prepares the memory buffer (page map) and the read requests for a given cluster and columns. | |
Private Attributes | |
std::optional< RNTuple > | fAnchor |
Either provided by CreateFromAnchor, or read from the ROOT file given the ntuple name. | |
std::unique_ptr< RClusterPool > | fClusterPool |
The cluster pool asynchronously preloads the next few clusters. | |
RCluster * | fCurrentCluster = nullptr |
The last cluster from which a page got loaded. Points into fClusterPool->fPool. | |
RNTupleDescriptorBuilder | fDescriptorBuilder |
The descriptor is created from the header and footer either in AttachImpl or in CreateFromAnchor. | |
std::unique_ptr< ROOT::Internal::RRawFile > | fFile |
An RRawFile is used to request the necessary byte ranges from a local or a remote file. | |
RMiniFileReader | fReader |
Takes the fFile to read ntuple blobs from it. | |
RStructureBuffer | fStructureBuffer |
Populated by LoadStructureImpl(), reset at the end of Attach() | |
Friends | |
class | ROOT::Experimental::RNTuple |
Additional Inherited Members | |
Public Types inherited from ROOT::Experimental::Internal::RPageStorage | |
using | ColumnHandle_t = RColumnHandle |
The column handle identifies a column with the current open page storage. | |
using | SealedPageSequence_t = std::deque< RSealedPage > |
Static Public Attributes inherited from ROOT::Experimental::Internal::RPageStorage | |
static constexpr std::size_t | kNBytesPageChecksum = sizeof(std::uint64_t) |
The page checksum is a 64bit xxhash3. | |
Protected Attributes inherited from ROOT::Experimental::Internal::RPageSource | |
RActivePhysicalColumns | fActivePhysicalColumns |
The active columns are implicitly defined by the model fields or views. | |
std::unique_ptr< RCounters > | fCounters |
RNTupleReadOptions | fOptions |
RPagePool | fPagePool |
Pages that are unzipped with IMT are staged into the page pool. | |
Protected Attributes inherited from ROOT::Experimental::Internal::RPageStorage | |
Detail::RNTupleMetrics | fMetrics |
std::string | fNTupleName |
std::unique_ptr< RPageAllocator > | fPageAllocator |
For the time being, we will use the heap allocator for all sources and sinks. This may change in the future. | |
RTaskScheduler * | fTaskScheduler = nullptr |
#include <ROOT/RPageStorageFile.hxx>
|
private |
Definition at line 258 of file RPageStorageFile.cxx.
ROOT::Experimental::Internal::RPageSourceFile::RPageSourceFile | ( | std::string_view | ntupleName, |
std::string_view | path, | ||
const RNTupleReadOptions & | options | ||
) |
Definition at line 276 of file RPageStorageFile.cxx.
ROOT::Experimental::Internal::RPageSourceFile::RPageSourceFile | ( | std::string_view | ntupleName, |
std::unique_ptr< ROOT::Internal::RRawFile > | file, | ||
const RNTupleReadOptions & | options | ||
) |
Definition at line 266 of file RPageStorageFile.cxx.
|
delete |
|
delete |
|
overridedefault |
|
finalprotectedvirtual |
LoadStructureImpl()
has been called before AttachImpl()
is called
Implements ROOT::Experimental::Internal::RPageSource.
Definition at line 361 of file RPageStorageFile.cxx.
|
finalprotectedvirtual |
The cloned page source creates a new raw file and reader and opens its own file descriptor to the data.
Implements ROOT::Experimental::Internal::RPageSource.
Definition at line 488 of file RPageStorageFile.cxx.
|
static |
Used from the RNTuple class to build a datasource if the anchor is already available.
Requires the RNTuple object to be streamed from a file.
Definition at line 283 of file RPageStorageFile.cxx.
|
finalvirtual |
Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages.
The page source may load more columns than the minimal necessary set from columns
. To indicate which columns have been loaded, LoadClusters()
must mark them with
SetColumnAvailable(). That includes the ones from the
columnsthat don't have pages; otherwise subsequent requests for the cluster would assume an incomplete cluster and trigger loading again.
LoadClusters()` is typically called from the I/O thread of a cluster pool, i.e. the method runs concurrently to other methods of the page source.
Implements ROOT::Experimental::Internal::RPageSource.
Definition at line 620 of file RPageStorageFile.cxx.
|
finalprotectedvirtual |
Implements ROOT::Experimental::Internal::RPageSource.
Definition at line 423 of file RPageStorageFile.cxx.
|
finalvirtual |
Read the packed and compressed bytes of a page into the memory buffer provided by sealedPage
.
The sealed page can be used subsequently in a call to RPageSink::CommitSealedPage
. The fSize
and fNElements
member of the sealedPage parameters are always set. If sealedPage.fBuffer
is nullptr
, no data will be copied but the returned size information can be used by the caller to allocate a large enough buffer and call LoadSealedPage
again.
Implements ROOT::Experimental::Internal::RPageSource.
Definition at line 394 of file RPageStorageFile.cxx.
|
finalprotectedvirtual |
Implements ROOT::Experimental::Internal::RPageSource.
Definition at line 312 of file RPageStorageFile.cxx.
|
delete |
|
delete |
|
private |
Helper function for LoadClusters: it prepares the memory buffer (page map) and the read requests for a given cluster and columns.
The reead requests are appended to the provided vector. This way, requests can be collected for multiple clusters before sending them to RRawFile::ReadV().
Definition at line 497 of file RPageStorageFile.cxx.
|
friend |
Definition at line 118 of file RPageStorageFile.hxx.
|
private |
Either provided by CreateFromAnchor, or read from the ROOT file given the ntuple name.
Definition at line 136 of file RPageStorageFile.hxx.
|
private |
The cluster pool asynchronously preloads the next few clusters.
Definition at line 146 of file RPageStorageFile.hxx.
|
private |
The last cluster from which a page got loaded. Points into fClusterPool->fPool.
Definition at line 138 of file RPageStorageFile.hxx.
|
private |
The descriptor is created from the header and footer either in AttachImpl or in CreateFromAnchor.
Definition at line 144 of file RPageStorageFile.hxx.
|
private |
An RRawFile is used to request the necessary byte ranges from a local or a remote file.
Definition at line 140 of file RPageStorageFile.hxx.
|
private |
Takes the fFile to read ntuple blobs from it.
Definition at line 142 of file RPageStorageFile.hxx.
|
private |
Populated by LoadStructureImpl(), reset at the end of Attach()
Definition at line 148 of file RPageStorageFile.hxx.