Logo ROOT  
Reference Guide
 
Loading...
Searching...
No Matches
ROOT::Experimental::Internal::RPageSourceFile Class Reference

Storage provider that reads ntuple pages from a file.

Definition at line 117 of file RPageStorageFile.hxx.

Classes

struct  RStructureBuffer
 Holds the uncompressed header and footer. More...
 

Public Member Functions

 RPageSourceFile (const RPageSourceFile &)=delete
 
 RPageSourceFile (RPageSourceFile &&)=delete
 
 RPageSourceFile (std::string_view ntupleName, std::string_view path, const RNTupleReadOptions &options)
 
 RPageSourceFile (std::string_view ntupleName, std::unique_ptr< ROOT::Internal::RRawFile > file, const RNTupleReadOptions &options)
 
 ~RPageSourceFile () override
 
std::vector< std::unique_ptr< RCluster > > LoadClusters (std::span< RCluster::RKey > clusterKeys) final
 Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages.
 
void LoadSealedPage (DescriptorId_t physicalColumnId, RClusterIndex clusterIndex, RSealedPage &sealedPage) final
 Read the packed and compressed bytes of a page into the memory buffer provided by sealedPage.
 
RPageSourceFileoperator= (const RPageSourceFile &)=delete
 
RPageSourceFileoperator= (RPageSourceFile &&)=delete
 
- Public Member Functions inherited from ROOT::Experimental::Internal::RPageSource
 RPageSource (const RPageSource &)=delete
 
 RPageSource (RPageSource &&)=delete
 
 RPageSource (std::string_view ntupleName, const RNTupleReadOptions &fOptions)
 
 ~RPageSource () override
 
ColumnHandle_t AddColumn (DescriptorId_t fieldId, RColumn &column) override
 Register a new column.
 
void Attach ()
 Open the physical storage container and deserialize header and footer.
 
std::unique_ptr< RPageSourceClone () const
 Open the same storage multiple time, e.g.
 
void DropColumn (ColumnHandle_t columnHandle) override
 Unregisters a column.
 
REntryRange GetEntryRange () const
 
NTupleSize_t GetNElements (ColumnHandle_t columnHandle)
 
NTupleSize_t GetNEntries ()
 
const RNTupleReadOptionsGetReadOptions () const
 
const RSharedDescriptorGuard GetSharedDescriptorGuard () const
 Takes the read lock for the descriptor.
 
EPageStorageType GetType () final
 Whether the concrete implementation is a sink or a source.
 
virtual RPageRef LoadPage (ColumnHandle_t columnHandle, NTupleSize_t globalIndex)
 Allocates and fills a page that contains the index-th element.
 
virtual RPageRef LoadPage (ColumnHandle_t columnHandle, RClusterIndex clusterIndex)
 Another version of LoadPage that allows to specify cluster-relative indexes.
 
void LoadStructure ()
 Loads header and footer without decompressing or deserializing them.
 
RPageSourceoperator= (const RPageSource &)=delete
 
RPageSourceoperator= (RPageSource &&)=delete
 
void SetEntryRange (const REntryRange &range)
 Promise to only read from the given entry range.
 
RResult< RPageUnsealPage (const RSealedPage &sealedPage, const RColumnElementBase &element, DescriptorId_t physicalColumnId)
 
void UnzipCluster (RCluster *cluster)
 Parallel decompression and unpacking of the pages in the given cluster.
 
- Public Member Functions inherited from ROOT::Experimental::Internal::RPageStorage
 RPageStorage (const RPageStorage &other)=delete
 
 RPageStorage (RPageStorage &&other)=default
 
 RPageStorage (std::string_view name)
 
virtual ~RPageStorage ()
 
ColumnId_t GetColumnId (ColumnHandle_t columnHandle) const
 
virtual Detail::RNTupleMetricsGetMetrics ()
 Returns the default metrics object.
 
const std::string & GetNTupleName () const
 Returns the NTuple name.
 
RPageStorageoperator= (const RPageStorage &other)=delete
 
RPageStorageoperator= (RPageStorage &&other)=default
 
void SetTaskScheduler (RTaskScheduler *taskScheduler)
 

Static Public Member Functions

static std::unique_ptr< RPageSourceFileCreateFromAnchor (const RNTuple &anchor, const RNTupleReadOptions &options=RNTupleReadOptions())
 Used from the RNTuple class to build a datasource if the anchor is already available.
 
- Static Public Member Functions inherited from ROOT::Experimental::Internal::RPageSource
static std::unique_ptr< RPageSourceCreate (std::string_view ntupleName, std::string_view location, const RNTupleReadOptions &options=RNTupleReadOptions())
 Guess the concrete derived page source from the file name (location)
 
static RResult< RPageUnsealPage (const RSealedPage &sealedPage, const RColumnElementBase &element, DescriptorId_t physicalColumnId, RPageAllocator &pageAlloc)
 Helper for unstreaming a page.
 

Protected Member Functions

RNTupleDescriptor AttachImpl () final
 LoadStructureImpl() has been called before AttachImpl() is called
 
std::unique_ptr< RPageSourceCloneImpl () const final
 The cloned page source creates a new raw file and reader and opens its own file descriptor to the data.
 
RPageRef LoadPageImpl (ColumnHandle_t columnHandle, const RClusterInfo &clusterInfo, ClusterSize_t::ValueType idxInCluster) final
 
void LoadStructureImpl () final
 
- Protected Member Functions inherited from ROOT::Experimental::Internal::RPageSource
void EnableDefaultMetrics (const std::string &prefix)
 Enables the default set of metrics provided by RPageSource.
 
RExclDescriptorGuard GetExclDescriptorGuard ()
 Note that the underlying lock is not recursive. See GetSharedDescriptorGuard() for further information.
 
void PrepareLoadCluster (const RCluster::RKey &clusterKey, ROnDiskPageMap &pageZeroMap, std::function< void(DescriptorId_t, NTupleSize_t, const RClusterDescriptor::RPageRange::RPageInfo &)> perPageFunc)
 Prepare a page range read for the column set in clusterKey.
 
virtual void UnzipClusterImpl (RCluster *cluster)
 
- Protected Member Functions inherited from ROOT::Experimental::Internal::RPageStorage
void WaitForAllTasks ()
 

Private Member Functions

 RPageSourceFile (std::string_view ntupleName, const RNTupleReadOptions &options)
 
std::unique_ptr< RClusterPrepareSingleCluster (const RCluster::RKey &clusterKey, std::vector< ROOT::Internal::RRawFile::RIOVec > &readRequests)
 Helper function for LoadClusters: it prepares the memory buffer (page map) and the read requests for a given cluster and columns.
 

Private Attributes

std::optional< RNTuplefAnchor
 Either provided by CreateFromAnchor, or read from the ROOT file given the ntuple name.
 
std::unique_ptr< RClusterPoolfClusterPool
 The cluster pool asynchronously preloads the next few clusters.
 
RClusterfCurrentCluster = nullptr
 The last cluster from which a page got loaded. Points into fClusterPool->fPool.
 
RNTupleDescriptorBuilder fDescriptorBuilder
 The descriptor is created from the header and footer either in AttachImpl or in CreateFromAnchor.
 
std::unique_ptr< ROOT::Internal::RRawFilefFile
 An RRawFile is used to request the necessary byte ranges from a local or a remote file.
 
RMiniFileReader fReader
 Takes the fFile to read ntuple blobs from it.
 
RStructureBuffer fStructureBuffer
 Populated by LoadStructureImpl(), reset at the end of Attach()
 

Friends

class ROOT::Experimental::RNTuple
 

Additional Inherited Members

- Public Types inherited from ROOT::Experimental::Internal::RPageStorage
using ColumnHandle_t = RColumnHandle
 The column handle identifies a column with the current open page storage.
 
using SealedPageSequence_t = std::deque< RSealedPage >
 
- Static Public Attributes inherited from ROOT::Experimental::Internal::RPageStorage
static constexpr std::size_t kNBytesPageChecksum = sizeof(std::uint64_t)
 The page checksum is a 64bit xxhash3.
 
- Protected Attributes inherited from ROOT::Experimental::Internal::RPageSource
RActivePhysicalColumns fActivePhysicalColumns
 The active columns are implicitly defined by the model fields or views.
 
std::unique_ptr< RCountersfCounters
 
RNTupleReadOptions fOptions
 
RPagePool fPagePool
 Pages that are unzipped with IMT are staged into the page pool.
 
- Protected Attributes inherited from ROOT::Experimental::Internal::RPageStorage
Detail::RNTupleMetrics fMetrics
 
std::string fNTupleName
 
std::unique_ptr< RPageAllocatorfPageAllocator
 For the time being, we will use the heap allocator for all sources and sinks. This may change in the future.
 
RTaskSchedulerfTaskScheduler = nullptr
 

#include <ROOT/RPageStorageFile.hxx>

Inheritance diagram for ROOT::Experimental::Internal::RPageSourceFile:
[legend]

Constructor & Destructor Documentation

◆ RPageSourceFile() [1/5]

ROOT::Experimental::Internal::RPageSourceFile::RPageSourceFile ( std::string_view  ntupleName,
const RNTupleReadOptions options 
)
private

Definition at line 258 of file RPageStorageFile.cxx.

◆ RPageSourceFile() [2/5]

ROOT::Experimental::Internal::RPageSourceFile::RPageSourceFile ( std::string_view  ntupleName,
std::string_view  path,
const RNTupleReadOptions options 
)

Definition at line 276 of file RPageStorageFile.cxx.

◆ RPageSourceFile() [3/5]

ROOT::Experimental::Internal::RPageSourceFile::RPageSourceFile ( std::string_view  ntupleName,
std::unique_ptr< ROOT::Internal::RRawFile file,
const RNTupleReadOptions options 
)

Definition at line 266 of file RPageStorageFile.cxx.

◆ RPageSourceFile() [4/5]

ROOT::Experimental::Internal::RPageSourceFile::RPageSourceFile ( const RPageSourceFile )
delete

◆ RPageSourceFile() [5/5]

ROOT::Experimental::Internal::RPageSourceFile::RPageSourceFile ( RPageSourceFile &&  )
delete

◆ ~RPageSourceFile()

ROOT::Experimental::Internal::RPageSourceFile::~RPageSourceFile ( )
overridedefault

Member Function Documentation

◆ AttachImpl()

ROOT::Experimental::RNTupleDescriptor ROOT::Experimental::Internal::RPageSourceFile::AttachImpl ( )
finalprotectedvirtual

LoadStructureImpl() has been called before AttachImpl() is called

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 361 of file RPageStorageFile.cxx.

◆ CloneImpl()

std::unique_ptr< ROOT::Experimental::Internal::RPageSource > ROOT::Experimental::Internal::RPageSourceFile::CloneImpl ( ) const
finalprotectedvirtual

The cloned page source creates a new raw file and reader and opens its own file descriptor to the data.

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 488 of file RPageStorageFile.cxx.

◆ CreateFromAnchor()

std::unique_ptr< ROOT::Experimental::Internal::RPageSourceFile > ROOT::Experimental::Internal::RPageSourceFile::CreateFromAnchor ( const RNTuple anchor,
const RNTupleReadOptions options = RNTupleReadOptions() 
)
static

Used from the RNTuple class to build a datasource if the anchor is already available.

Requires the RNTuple object to be streamed from a file.

Definition at line 283 of file RPageStorageFile.cxx.

◆ LoadClusters()

std::vector< std::unique_ptr< ROOT::Experimental::Internal::RCluster > > ROOT::Experimental::Internal::RPageSourceFile::LoadClusters ( std::span< RCluster::RKey clusterKeys)
finalvirtual

Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages.

The page source may load more columns than the minimal necessary set from columns. To indicate which columns have been loaded, LoadClusters()must mark them withSetColumnAvailable(). That includes the ones from thecolumnsthat don't have pages; otherwise subsequent requests for the cluster would assume an incomplete cluster and trigger loading again. LoadClusters()` is typically called from the I/O thread of a cluster pool, i.e. the method runs concurrently to other methods of the page source.

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 620 of file RPageStorageFile.cxx.

◆ LoadPageImpl()

ROOT::Experimental::Internal::RPageRef ROOT::Experimental::Internal::RPageSourceFile::LoadPageImpl ( ColumnHandle_t  columnHandle,
const RClusterInfo clusterInfo,
ClusterSize_t::ValueType  idxInCluster 
)
finalprotectedvirtual

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 423 of file RPageStorageFile.cxx.

◆ LoadSealedPage()

void ROOT::Experimental::Internal::RPageSourceFile::LoadSealedPage ( DescriptorId_t  physicalColumnId,
RClusterIndex  clusterIndex,
RSealedPage sealedPage 
)
finalvirtual

Read the packed and compressed bytes of a page into the memory buffer provided by sealedPage.

The sealed page can be used subsequently in a call to RPageSink::CommitSealedPage. The fSize and fNElements member of the sealedPage parameters are always set. If sealedPage.fBuffer is nullptr, no data will be copied but the returned size information can be used by the caller to allocate a large enough buffer and call LoadSealedPage again.

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 394 of file RPageStorageFile.cxx.

◆ LoadStructureImpl()

void ROOT::Experimental::Internal::RPageSourceFile::LoadStructureImpl ( )
finalprotectedvirtual

Implements ROOT::Experimental::Internal::RPageSource.

Definition at line 312 of file RPageStorageFile.cxx.

◆ operator=() [1/2]

RPageSourceFile & ROOT::Experimental::Internal::RPageSourceFile::operator= ( const RPageSourceFile )
delete

◆ operator=() [2/2]

RPageSourceFile & ROOT::Experimental::Internal::RPageSourceFile::operator= ( RPageSourceFile &&  )
delete

◆ PrepareSingleCluster()

std::unique_ptr< ROOT::Experimental::Internal::RCluster > ROOT::Experimental::Internal::RPageSourceFile::PrepareSingleCluster ( const RCluster::RKey clusterKey,
std::vector< ROOT::Internal::RRawFile::RIOVec > &  readRequests 
)
private

Helper function for LoadClusters: it prepares the memory buffer (page map) and the read requests for a given cluster and columns.

The reead requests are appended to the provided vector. This way, requests can be collected for multiple clusters before sending them to RRawFile::ReadV().

Definition at line 497 of file RPageStorageFile.cxx.

Friends And Related Symbol Documentation

◆ ROOT::Experimental::RNTuple

friend class ROOT::Experimental::RNTuple
friend

Definition at line 118 of file RPageStorageFile.hxx.

Member Data Documentation

◆ fAnchor

std::optional<RNTuple> ROOT::Experimental::Internal::RPageSourceFile::fAnchor
private

Either provided by CreateFromAnchor, or read from the ROOT file given the ntuple name.

Definition at line 136 of file RPageStorageFile.hxx.

◆ fClusterPool

std::unique_ptr<RClusterPool> ROOT::Experimental::Internal::RPageSourceFile::fClusterPool
private

The cluster pool asynchronously preloads the next few clusters.

Definition at line 146 of file RPageStorageFile.hxx.

◆ fCurrentCluster

RCluster* ROOT::Experimental::Internal::RPageSourceFile::fCurrentCluster = nullptr
private

The last cluster from which a page got loaded. Points into fClusterPool->fPool.

Definition at line 138 of file RPageStorageFile.hxx.

◆ fDescriptorBuilder

RNTupleDescriptorBuilder ROOT::Experimental::Internal::RPageSourceFile::fDescriptorBuilder
private

The descriptor is created from the header and footer either in AttachImpl or in CreateFromAnchor.

Definition at line 144 of file RPageStorageFile.hxx.

◆ fFile

std::unique_ptr<ROOT::Internal::RRawFile> ROOT::Experimental::Internal::RPageSourceFile::fFile
private

An RRawFile is used to request the necessary byte ranges from a local or a remote file.

Definition at line 140 of file RPageStorageFile.hxx.

◆ fReader

RMiniFileReader ROOT::Experimental::Internal::RPageSourceFile::fReader
private

Takes the fFile to read ntuple blobs from it.

Definition at line 142 of file RPageStorageFile.hxx.

◆ fStructureBuffer

RStructureBuffer ROOT::Experimental::Internal::RPageSourceFile::fStructureBuffer
private

Populated by LoadStructureImpl(), reset at the end of Attach()

Definition at line 148 of file RPageStorageFile.hxx.

Libraries for ROOT::Experimental::Internal::RPageSourceFile:

The documentation for this class was generated from the following files: