The on-storage meta-data of an ntuple.
Represents the on-disk (on storage) information about an ntuple. The meta-data consists of a header and one or several footers. The header carries the ntuple schema, i.e. the fields and the associated columns and their relationships. The footer(s) carry information about one or several clusters. For every cluster, a footer stores its location and size, and for every column the range of element indexes as well as a list of pages and page locations.
The descriptor provide machine-independent (de-)serialization of headers and footers, and it provides lookup routines for ntuple objects (pages, clusters, ...). It is supposed to be usable by all RPageStorage implementations.
The serialization does not use standard ROOT streamers in order to not let it depend on libCore. The serialization uses the concept of frames: header, footer, and substructures have a preamble with version numbers and the size of the writte struct. This allows for forward and backward compatibility when the meta-data evolves.
Definition at line 416 of file RNTupleDescriptor.hxx.
Classes | |
class | RClusterDescriptorIterable |
Used to loop over all the clusters of an ntuple (in unspecified order) More... | |
class | RClusterGroupDescriptorIterable |
Used to loop over all the cluster groups of an ntuple (in unspecified order) More... | |
class | RColumnDescriptorIterable |
Used to loop over a field's associated columns. More... | |
class | RFieldDescriptorIterable |
Used to loop over a field's child fields. More... | |
class | RHeaderExtension |
Summarizes information about fields and the corresponding columns that were added after the header has been serialized. More... | |
Public Member Functions | |
RNTupleDescriptor ()=default | |
RNTupleDescriptor (const RNTupleDescriptor &other)=delete | |
RNTupleDescriptor (RNTupleDescriptor &&other)=default | |
RResult< void > | AddClusterGroupDetails (DescriptorId_t clusterGroupId, std::vector< RClusterDescriptor > &clusterDescs) |
Methods to load and drop cluster group details (cluster IDs and page locations) | |
std::unique_ptr< RNTupleDescriptor > | Clone () const |
std::unique_ptr< RNTupleModel > | CreateModel () const |
Re-create the C++ model from the stored meta-data. | |
RResult< void > | DropClusterGroupDetails (DescriptorId_t clusterGroupId) |
DescriptorId_t | FindClusterId (DescriptorId_t physicalColumnId, NTupleSize_t index) const |
DescriptorId_t | FindFieldId (std::string_view fieldName) const |
Searches for a top-level field. | |
DescriptorId_t | FindFieldId (std::string_view fieldName, DescriptorId_t parentId) const |
DescriptorId_t | FindLogicalColumnId (DescriptorId_t fieldId, std::uint32_t columnIndex) const |
DescriptorId_t | FindNextClusterId (DescriptorId_t clusterId) const |
DescriptorId_t | FindPhysicalColumnId (DescriptorId_t fieldId, std::uint32_t columnIndex) const |
DescriptorId_t | FindPrevClusterId (DescriptorId_t clusterId) const |
const RClusterDescriptor & | GetClusterDescriptor (DescriptorId_t clusterId) const |
const RClusterGroupDescriptor & | GetClusterGroupDescriptor (DescriptorId_t clusterGroupId) const |
RClusterGroupDescriptorIterable | GetClusterGroupIterable () const |
RClusterDescriptorIterable | GetClusterIterable () const |
const RColumnDescriptor & | GetColumnDescriptor (DescriptorId_t columnId) const |
RColumnDescriptorIterable | GetColumnIterable () const |
RColumnDescriptorIterable | GetColumnIterable (const RFieldDescriptor &fieldDesc) const |
RColumnDescriptorIterable | GetColumnIterable (DescriptorId_t fieldId) const |
std::string | GetDescription () const |
std::vector< std::uint64_t > | GetFeatureFlags () const |
const RFieldDescriptor & | GetFieldDescriptor (DescriptorId_t fieldId) const |
RFieldDescriptorIterable | GetFieldIterable (const RFieldDescriptor &fieldDesc) const |
RFieldDescriptorIterable | GetFieldIterable (const RFieldDescriptor &fieldDesc, const std::function< bool(DescriptorId_t, DescriptorId_t)> &comparator) const |
RFieldDescriptorIterable | GetFieldIterable (DescriptorId_t fieldId) const |
RFieldDescriptorIterable | GetFieldIterable (DescriptorId_t fieldId, const std::function< bool(DescriptorId_t, DescriptorId_t)> &comparator) const |
const RFieldDescriptor & | GetFieldZero () const |
DescriptorId_t | GetFieldZeroId () const |
Returns the logical parent of all top-level NTuple data fields. | |
std::uint64_t | GetGeneration () const |
const RHeaderExtension * | GetHeaderExtension () const |
Return header extension information; if the descriptor does not have a header extension, return nullptr | |
std::size_t | GetNActiveClusters () const |
std::string | GetName () const |
std::size_t | GetNClusterGroups () const |
std::size_t | GetNClusters () const |
NTupleSize_t | GetNElements (DescriptorId_t physicalColumnId) const |
NTupleSize_t | GetNEntries () const |
We know the number of entries from adding the cluster summaries. | |
std::size_t | GetNFields () const |
std::size_t | GetNLogicalColumns () const |
std::size_t | GetNPhysicalColumns () const |
std::uint64_t | GetOnDiskFooterSize () const |
std::uint64_t | GetOnDiskHeaderSize () const |
std::uint64_t | GetOnDiskHeaderXxHash3 () const |
std::string | GetQualifiedFieldName (DescriptorId_t fieldId) const |
Walks up the parents of the field ID and returns a field name of the form a.b.c.d In case of invalid field ID, an empty string is returned. | |
RFieldDescriptorIterable | GetTopLevelFields () const |
RFieldDescriptorIterable | GetTopLevelFields (const std::function< bool(DescriptorId_t, DescriptorId_t)> &comparator) const |
bool | HasFeature (unsigned int flag) const |
void | IncGeneration () |
RNTupleDescriptor & | operator= (const RNTupleDescriptor &other)=delete |
RNTupleDescriptor & | operator= (RNTupleDescriptor &&other)=default |
bool | operator== (const RNTupleDescriptor &other) const |
void | PrintInfo (std::ostream &output) const |
Static Public Attributes | |
static constexpr unsigned int | kFeatureFlagTest = 137 |
Private Attributes | |
std::unordered_map< DescriptorId_t, RClusterDescriptor > | fClusterDescriptors |
May contain only a subset of all the available clusters, e.g. | |
std::unordered_map< DescriptorId_t, RClusterGroupDescriptor > | fClusterGroupDescriptors |
std::unordered_map< DescriptorId_t, RColumnDescriptor > | fColumnDescriptors |
std::string | fDescription |
Free text from the user. | |
std::set< unsigned int > | fFeatureFlags |
std::unordered_map< DescriptorId_t, RFieldDescriptor > | fFieldDescriptors |
std::uint64_t | fGeneration = 0 |
Once constructed by an RNTupleDescriptorBuilder, the descriptor is mostly immutable except for set of active the page locations. | |
std::unique_ptr< RHeaderExtension > | fHeaderExtension |
std::string | fName |
The ntuple name needs to be unique in a given storage location (file) | |
std::uint64_t | fNClusters = 0 |
Updated by the descriptor builder when the cluster groups are added. | |
std::uint64_t | fNEntries = 0 |
Updated by the descriptor builder when the cluster groups are added. | |
std::uint64_t | fNPhysicalColumns = 0 |
Updated by the descriptor builder when columns are added. | |
std::uint64_t | fOnDiskFooterSize = 0 |
Like fOnDiskHeaderSize, contains both cluster summaries and page locations. | |
std::uint64_t | fOnDiskHeaderSize = 0 |
Set by the descriptor builder when deserialized. | |
std::uint64_t | fOnDiskHeaderXxHash3 = 0 |
Set by the descriptor builder when deserialized. | |
Friends | |
class | Internal::RNTupleDescriptorBuilder |
#include <ROOT/RNTupleDescriptor.hxx>
|
default |
|
delete |
|
default |
ROOT::Experimental::RResult< void > ROOT::Experimental::RNTupleDescriptor::AddClusterGroupDetails | ( | DescriptorId_t | clusterGroupId, |
std::vector< RClusterDescriptor > & | clusterDescs | ||
) |
Methods to load and drop cluster group details (cluster IDs and page locations)
Definition at line 420 of file RNTupleDescriptor.cxx.
std::unique_ptr< ROOT::Experimental::RNTupleDescriptor > ROOT::Experimental::RNTupleDescriptor::Clone | ( | ) | const |
Definition at line 471 of file RNTupleDescriptor.cxx.
std::unique_ptr< ROOT::Experimental::RNTupleModel > ROOT::Experimental::RNTupleDescriptor::CreateModel | ( | ) | const |
Re-create the C++ model from the stored meta-data.
Definition at line 460 of file RNTupleDescriptor.cxx.
ROOT::Experimental::RResult< void > ROOT::Experimental::RNTupleDescriptor::DropClusterGroupDetails | ( | DescriptorId_t | clusterGroupId | ) |
Definition at line 446 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindClusterId | ( | DescriptorId_t | physicalColumnId, |
NTupleSize_t | index | ||
) | const |
Definition at line 309 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindFieldId | ( | std::string_view | fieldName | ) | const |
Searches for a top-level field.
Definition at line 284 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindFieldId | ( | std::string_view | fieldName, |
DescriptorId_t | parentId | ||
) | const |
Definition at line 246 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindLogicalColumnId | ( | DescriptorId_t | fieldId, |
std::uint32_t | columnIndex | ||
) | const |
Definition at line 290 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindNextClusterId | ( | DescriptorId_t | clusterId | ) | const |
Definition at line 325 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindPhysicalColumnId | ( | DescriptorId_t | fieldId, |
std::uint32_t | columnIndex | ||
) | const |
Definition at line 300 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindPrevClusterId | ( | DescriptorId_t | clusterId | ) | const |
Definition at line 340 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 715 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 711 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 757 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 759 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 708 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 744 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 748 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 752 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 765 of file RNTupleDescriptor.hxx.
std::vector< std::uint64_t > ROOT::Experimental::RNTupleDescriptor::GetFeatureFlags | ( | ) | const |
Definition at line 399 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 705 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 719 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 722 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 727 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 730 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 780 of file RNTupleDescriptor.hxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::GetFieldZeroId | ( | ) | const |
Returns the logical parent of all top-level NTuple data fields.
Definition at line 277 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 804 of file RNTupleDescriptor.hxx.
|
inline |
Return header extension information; if the descriptor does not have a header extension, return nullptr
Definition at line 798 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 772 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 764 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 770 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 771 of file RNTupleDescriptor.hxx.
ROOT::Experimental::NTupleSize_t ROOT::Experimental::RNTupleDescriptor::GetNElements | ( | DescriptorId_t | physicalColumnId | ) | const |
Definition at line 232 of file RNTupleDescriptor.cxx.
|
inline |
We know the number of entries from adding the cluster summaries.
Definition at line 775 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 767 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 768 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 769 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 703 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 702 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 701 of file RNTupleDescriptor.hxx.
std::string ROOT::Experimental::RNTupleDescriptor::GetQualifiedFieldName | ( | DescriptorId_t | fieldId | ) | const |
Walks up the parents of the field ID and returns a field name of the form a.b.c.d In case of invalid field ID, an empty string is returned.
Definition at line 263 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 735 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 738 of file RNTupleDescriptor.hxx.
Definition at line 794 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 805 of file RNTupleDescriptor.hxx.
|
delete |
|
default |
bool ROOT::Experimental::RNTupleDescriptor::operator== | ( | const RNTupleDescriptor & | other | ) | const |
Definition at line 217 of file RNTupleDescriptor.cxx.
void ROOT::Experimental::RNTupleDescriptor::PrintInfo | ( | std::ostream & | output | ) | const |
Definition at line 83 of file RNTupleDescriptorFmt.cxx.
|
friend |
Definition at line 417 of file RNTupleDescriptor.hxx.
|
private |
May contain only a subset of all the available clusters, e.g.
the clusters of the current file from a chain of files
Definition at line 451 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 448 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 447 of file RNTupleDescriptor.hxx.
|
private |
Free text from the user.
Definition at line 426 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 445 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 446 of file RNTupleDescriptor.hxx.
|
private |
Once constructed by an RNTupleDescriptorBuilder, the descriptor is mostly immutable except for set of active the page locations.
During the lifetime of the descriptor, page location information for clusters can be added or removed. When this happens, the generation should be increased, so that users of the descriptor know that the information changed. The generation is increased, e.g., by the page source's exclusive lock guard around the descriptor. It is used, e.g., by the descriptor cache in RNTupleReader.
Definition at line 443 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 452 of file RNTupleDescriptor.hxx.
|
private |
The ntuple name needs to be unique in a given storage location (file)
Definition at line 424 of file RNTupleDescriptor.hxx.
|
private |
Updated by the descriptor builder when the cluster groups are added.
Definition at line 433 of file RNTupleDescriptor.hxx.
|
private |
Updated by the descriptor builder when the cluster groups are added.
Definition at line 432 of file RNTupleDescriptor.hxx.
|
private |
Updated by the descriptor builder when columns are added.
Definition at line 434 of file RNTupleDescriptor.hxx.
|
private |
Like fOnDiskHeaderSize, contains both cluster summaries and page locations.
Definition at line 430 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder when deserialized.
Definition at line 429 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder when deserialized.
Definition at line 428 of file RNTupleDescriptor.hxx.
|
staticconstexpr |
Definition at line 455 of file RNTupleDescriptor.hxx.