The on-storage metadata of an RNTuple.
Represents the on-disk (on storage) information about an RNTuple. The metadata consists of a header, a footer, and potentially multiple page lists. The header carries the RNTuple schema, i.e. the fields and the associated columns and their relationships. The footer carries information about one or several cluster groups and links to their page lists. For every cluster group, a page list envelope stores cluster summaries and page locations. For every cluster, it stores for every column the range of element indexes as well as a list of pages and page locations.
The descriptor provides machine-independent (de-)serialization of headers and footers, and it provides lookup routines for RNTuple objects (pages, clusters, ...). It is supposed to be usable by all RPageStorage implementations.
The serialization does not use standard ROOT streamers in order to not let it depend on libCore. The serialization uses the concept of envelopes and frames: header, footer, and page list envelopes have a preamble with a type ID and length. Substructures are serialized in frames and have a size and number of items (for list frames). This allows for forward and backward compatibility when the metadata evolves.
Definition at line 699 of file RNTupleDescriptor.hxx.
Classes | |
| class | RClusterDescriptorIterable |
| Used to loop over all the clusters of an RNTuple (in unspecified order). More... | |
| class | RClusterGroupDescriptorIterable |
| Used to loop over all the cluster groups of an RNTuple (in unspecified order). More... | |
| class | RColumnDescriptorIterable |
| Used to loop over a field's associated columns. More... | |
| struct | RCreateModelOptions |
| Modifiers passed to CreateModel(). More... | |
| class | RExtraTypeInfoDescriptorIterable |
| Used to loop over all the extra type info record of an RNTuple (in unspecified order). More... | |
| class | RFieldDescriptorIterable |
| Used to loop over a field's child fields. More... | |
| class | RHeaderExtension |
| Summarizes information about fields and the corresponding columns that were added after the header has been serialized. More... | |
Public Types | |
| enum | EFeatureFlags { kFeatureFlag_COUNT , kFeatureFlag_Test = 137 } |
| All known feature flags. More... | |
Public Member Functions | |
| RNTupleDescriptor ()=default | |
| RNTupleDescriptor (const RNTupleDescriptor &other)=delete | |
| RNTupleDescriptor (RNTupleDescriptor &&other)=default | |
| RResult< void > | AddClusterGroupDetails (ROOT::DescriptorId_t clusterGroupId, std::vector< RClusterDescriptor > &clusterDescs) |
| Methods to load and drop cluster group details (cluster IDs and page locations). | |
| RNTupleDescriptor | Clone () const |
| std::unique_ptr< ROOT::RNTupleModel > | CreateModel (const RCreateModelOptions &options=RCreateModelOptions()) const |
| Re-create the C++ model from the stored metadata. | |
| RResult< void > | DropClusterGroupDetails (ROOT::DescriptorId_t clusterGroupId) |
| ROOT::DescriptorId_t | FindClusterId (ROOT::DescriptorId_t physicalColumnId, ROOT::NTupleSize_t index) const |
| ROOT::DescriptorId_t | FindFieldId (std::string_view fieldName) const |
| Searches for a top-level field. | |
| ROOT::DescriptorId_t | FindFieldId (std::string_view fieldName, ROOT::DescriptorId_t parentId) const |
| ROOT::DescriptorId_t | FindLogicalColumnId (ROOT::DescriptorId_t fieldId, std::uint32_t columnIndex, std::uint16_t representationIndex) const |
| ROOT::DescriptorId_t | FindNextClusterId (ROOT::DescriptorId_t clusterId) const |
| ROOT::DescriptorId_t | FindPhysicalColumnId (ROOT::DescriptorId_t fieldId, std::uint32_t columnIndex, std::uint16_t representationIndex) const |
| ROOT::DescriptorId_t | FindPrevClusterId (ROOT::DescriptorId_t clusterId) const |
| ROOT::Experimental::RNTupleAttrSetDescriptorIterable | GetAttrSetIterable () const |
| const RClusterDescriptor & | GetClusterDescriptor (ROOT::DescriptorId_t clusterId) const |
| const RClusterGroupDescriptor & | GetClusterGroupDescriptor (ROOT::DescriptorId_t clusterGroupId) const |
| RClusterGroupDescriptorIterable | GetClusterGroupIterable () const |
| RClusterDescriptorIterable | GetClusterIterable () const |
| const RColumnDescriptor & | GetColumnDescriptor (ROOT::DescriptorId_t columnId) const |
| RColumnDescriptorIterable | GetColumnIterable () const |
| RColumnDescriptorIterable | GetColumnIterable (const RFieldDescriptor &fieldDesc) const |
| RColumnDescriptorIterable | GetColumnIterable (ROOT::DescriptorId_t fieldId) const |
| const std::string & | GetDescription () const |
| RExtraTypeInfoDescriptorIterable | GetExtraTypeInfoIterable () const |
| std::vector< std::uint64_t > | GetFeatureFlags () const |
| const RFieldDescriptor & | GetFieldDescriptor (ROOT::DescriptorId_t fieldId) const |
| RFieldDescriptorIterable | GetFieldIterable (const RFieldDescriptor &fieldDesc) const |
| RFieldDescriptorIterable | GetFieldIterable (const RFieldDescriptor &fieldDesc, const std::function< bool(ROOT::DescriptorId_t, ROOT::DescriptorId_t)> &comparator) const |
| RFieldDescriptorIterable | GetFieldIterable (ROOT::DescriptorId_t fieldId) const |
| RFieldDescriptorIterable | GetFieldIterable (ROOT::DescriptorId_t fieldId, const std::function< bool(ROOT::DescriptorId_t, ROOT::DescriptorId_t)> &comparator) const |
| const RFieldDescriptor & | GetFieldZero () const |
| ROOT::DescriptorId_t | GetFieldZeroId () const |
| Returns the logical parent of all top-level RNTuple data fields. | |
| std::uint64_t | GetGeneration () const |
| const RHeaderExtension * | GetHeaderExtension () const |
| Return header extension information; if the descriptor does not have a header extension, return nullptr. | |
| std::size_t | GetNActiveClusters () const |
| const std::string & | GetName () const |
| std::size_t | GetNAttributeSets () const |
| std::size_t | GetNClusterGroups () const |
| std::size_t | GetNClusters () const |
| ROOT::NTupleSize_t | GetNElements (ROOT::DescriptorId_t physicalColumnId) const |
| ROOT::NTupleSize_t | GetNEntries () const |
| We know the number of entries from adding the cluster summaries. | |
| std::size_t | GetNExtraTypeInfos () const |
| std::size_t | GetNFields () const |
| std::size_t | GetNLogicalColumns () const |
| std::size_t | GetNPhysicalColumns () const |
| std::uint64_t | GetOnDiskFooterSize () const |
| std::uint64_t | GetOnDiskHeaderSize () const |
| std::uint64_t | GetOnDiskHeaderXxHash3 () const |
| std::string | GetQualifiedFieldName (ROOT::DescriptorId_t fieldId) const |
| Walks up the parents of the field ID and returns a field name of the form a.b.c.d In case of invalid field ID, an empty string is returned. | |
| RFieldDescriptorIterable | GetTopLevelFields () const |
| RFieldDescriptorIterable | GetTopLevelFields (const std::function< bool(ROOT::DescriptorId_t, ROOT::DescriptorId_t)> &comparator) const |
| std::string | GetTypeNameForComparison (const RFieldDescriptor &fieldDesc) const |
| Adjust the type name of the passed RFieldDescriptor for comparison with another renormalized type name. | |
| std::uint64_t | GetVersion () const |
| bool | HasFeature (unsigned int flag) const |
| void | IncGeneration () |
| RNTupleDescriptor & | operator= (const RNTupleDescriptor &other)=delete |
| RNTupleDescriptor & | operator= (RNTupleDescriptor &&other)=default |
| bool | operator== (const RNTupleDescriptor &other) const |
| void | PrintInfo (std::ostream &output) const |
Private Member Functions | |
| RNTupleDescriptor | CloneSchema () const |
| Creates a descriptor containing only the schema information about this RNTuple, i.e. | |
| ROOT::DescriptorId_t | FindClusterId (ROOT::NTupleSize_t entryIdx) const |
Private Attributes | |
| std::vector< Experimental::RNTupleAttrSetDescriptor > | fAttributeSets |
| List of AttributeSets linked to this RNTuple. | |
| std::unordered_map< ROOT::DescriptorId_t, RClusterDescriptor > | fClusterDescriptors |
| Potentially a subset of all the available clusters. | |
| std::unordered_map< ROOT::DescriptorId_t, RClusterGroupDescriptor > | fClusterGroupDescriptors |
| std::unordered_map< ROOT::DescriptorId_t, RColumnDescriptor > | fColumnDescriptors |
| std::string | fDescription |
| Free text from the user. | |
| std::vector< RExtraTypeInfoDescriptor > | fExtraTypeInfoDescriptors |
| std::set< unsigned int > | fFeatureFlags |
| std::unordered_map< ROOT::DescriptorId_t, RFieldDescriptor > | fFieldDescriptors |
| ROOT::DescriptorId_t | fFieldZeroId = ROOT::kInvalidDescriptorId |
| Set by the descriptor builder. | |
| std::uint64_t | fGeneration = 0 |
| The generation of the descriptor. | |
| std::unique_ptr< RHeaderExtension > | fHeaderExtension |
| std::string | fName |
| The RNTuple name needs to be unique in a given storage location (file). | |
| std::uint64_t | fNClusters = 0 |
| Updated by the descriptor builder when the cluster groups are added. | |
| std::uint64_t | fNEntries = 0 |
| Updated by the descriptor builder when the cluster groups are added. | |
| std::uint64_t | fNPhysicalColumns = 0 |
| Updated by the descriptor builder when columns are added. | |
| std::uint64_t | fOnDiskFooterSize = 0 |
| Like fOnDiskHeaderSize, contains both cluster summaries and page locations. | |
| std::uint64_t | fOnDiskHeaderSize = 0 |
| Set by the descriptor builder when deserialized. | |
| std::uint64_t | fOnDiskHeaderXxHash3 = 0 |
| Set by the descriptor builder when deserialized. | |
| std::vector< ROOT::DescriptorId_t > | fSortedClusterGroupIds |
| References cluster groups sorted by entry range and thus allows for binary search. | |
| std::uint16_t | fVersionEpoch = 0 |
| Set by the descriptor builder when deserialized. | |
| std::uint16_t | fVersionMajor = 0 |
| Set by the descriptor builder when deserialized. | |
| std::uint16_t | fVersionMinor = 0 |
| Set by the descriptor builder when deserialized. | |
| std::uint16_t | fVersionPatch = 0 |
| Set by the descriptor builder when deserialized. | |
Friends | |
| class | Experimental::RNTupleAttrSetDescriptorIterable |
| DescriptorId_t | Internal::CallFindClusterIdOn (const RNTupleDescriptor &desc, NTupleSize_t entryIdx) |
| RNTupleDescriptor | Internal::CloneDescriptorSchema (const RNTupleDescriptor &desc) |
| class | Internal::RNTupleDescriptorBuilder |
#include <ROOT/RNTupleDescriptor.hxx>
All known feature flags.
Note that the flag values represent the bit index, not the already-bitshifted integer.
| Enumerator | |
|---|---|
| kFeatureFlag_COUNT | |
| kFeatureFlag_Test | Reserved for forward-compatibility testing. |
Definition at line 769 of file RNTupleDescriptor.hxx.
|
default |
|
delete |
|
default |
| ROOT::RResult< void > ROOT::RNTupleDescriptor::AddClusterGroupDetails | ( | ROOT::DescriptorId_t | clusterGroupId, |
| std::vector< RClusterDescriptor > & | clusterDescs ) |
Methods to load and drop cluster group details (cluster IDs and page locations).
Definition at line 630 of file RNTupleDescriptor.cxx.
| ROOT::RNTupleDescriptor ROOT::RNTupleDescriptor::Clone | ( | ) | const |
Definition at line 766 of file RNTupleDescriptor.cxx.
|
private |
Creates a descriptor containing only the schema information about this RNTuple, i.e.
all the information needed to create a new RNTuple with the same schema as this one but not necessarily the same clustering. This is used when merging two RNTuples.
Definition at line 741 of file RNTupleDescriptor.cxx.
| std::unique_ptr< ROOT::RNTupleModel > ROOT::RNTupleDescriptor::CreateModel | ( | const RCreateModelOptions & | options = RCreateModelOptions() | ) | const |
Re-create the C++ model from the stored metadata.
Definition at line 672 of file RNTupleDescriptor.cxx.
| ROOT::RResult< void > ROOT::RNTupleDescriptor::DropClusterGroupDetails | ( | ROOT::DescriptorId_t | clusterGroupId | ) |
Definition at line 658 of file RNTupleDescriptor.cxx.
| ROOT::DescriptorId_t ROOT::RNTupleDescriptor::FindClusterId | ( | ROOT::DescriptorId_t | physicalColumnId, |
| ROOT::NTupleSize_t | index ) const |
Definition at line 437 of file RNTupleDescriptor.cxx.
|
private |
Definition at line 499 of file RNTupleDescriptor.cxx.
| ROOT::DescriptorId_t ROOT::RNTupleDescriptor::FindFieldId | ( | std::string_view | fieldName | ) | const |
Searches for a top-level field.
Definition at line 406 of file RNTupleDescriptor.cxx.
| ROOT::DescriptorId_t ROOT::RNTupleDescriptor::FindFieldId | ( | std::string_view | fieldName, |
| ROOT::DescriptorId_t | parentId ) const |
Definition at line 362 of file RNTupleDescriptor.cxx.
| ROOT::DescriptorId_t ROOT::RNTupleDescriptor::FindLogicalColumnId | ( | ROOT::DescriptorId_t | fieldId, |
| std::uint32_t | columnIndex, | ||
| std::uint16_t | representationIndex ) const |
Definition at line 411 of file RNTupleDescriptor.cxx.
| ROOT::DescriptorId_t ROOT::RNTupleDescriptor::FindNextClusterId | ( | ROOT::DescriptorId_t | clusterId | ) | const |
Definition at line 551 of file RNTupleDescriptor.cxx.
| ROOT::DescriptorId_t ROOT::RNTupleDescriptor::FindPhysicalColumnId | ( | ROOT::DescriptorId_t | fieldId, |
| std::uint32_t | columnIndex, | ||
| std::uint16_t | representationIndex ) const |
Definition at line 426 of file RNTupleDescriptor.cxx.
| ROOT::DescriptorId_t ROOT::RNTupleDescriptor::FindPrevClusterId | ( | ROOT::DescriptorId_t | clusterId | ) | const |
Definition at line 562 of file RNTupleDescriptor.cxx.
| ROOT::Experimental::RNTupleAttrSetDescriptorIterable ROOT::RNTupleDescriptor::GetAttrSetIterable | ( | ) | const |
Definition at line 1473 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 852 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 848 of file RNTupleDescriptor.hxx.
| ROOT::RNTupleDescriptor::RClusterGroupDescriptorIterable ROOT::RNTupleDescriptor::GetClusterGroupIterable | ( | ) | const |
Definition at line 1458 of file RNTupleDescriptor.cxx.
| ROOT::RNTupleDescriptor::RClusterDescriptorIterable ROOT::RNTupleDescriptor::GetClusterIterable | ( | ) | const |
Definition at line 1463 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 844 of file RNTupleDescriptor.hxx.
| ROOT::RNTupleDescriptor::RColumnDescriptorIterable ROOT::RNTupleDescriptor::GetColumnIterable | ( | ) | const |
Definition at line 1441 of file RNTupleDescriptor.cxx.
| ROOT::RNTupleDescriptor::RColumnDescriptorIterable ROOT::RNTupleDescriptor::GetColumnIterable | ( | const RFieldDescriptor & | fieldDesc | ) | const |
Definition at line 1447 of file RNTupleDescriptor.cxx.
| ROOT::RNTupleDescriptor::RColumnDescriptorIterable ROOT::RNTupleDescriptor::GetColumnIterable | ( | ROOT::DescriptorId_t | fieldId | ) | const |
Definition at line 1453 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 883 of file RNTupleDescriptor.hxx.
| ROOT::RNTupleDescriptor::RExtraTypeInfoDescriptorIterable ROOT::RNTupleDescriptor::GetExtraTypeInfoIterable | ( | ) | const |
Definition at line 1468 of file RNTupleDescriptor.cxx.
| std::vector< std::uint64_t > ROOT::RNTupleDescriptor::GetFeatureFlags | ( | ) | const |
Definition at line 610 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 840 of file RNTupleDescriptor.hxx.
| ROOT::RNTupleDescriptor::RFieldDescriptorIterable ROOT::RNTupleDescriptor::GetFieldIterable | ( | const RFieldDescriptor & | fieldDesc | ) | const |
Definition at line 1405 of file RNTupleDescriptor.cxx.
| ROOT::RNTupleDescriptor::RFieldDescriptorIterable ROOT::RNTupleDescriptor::GetFieldIterable | ( | const RFieldDescriptor & | fieldDesc, |
| const std::function< bool(ROOT::DescriptorId_t, ROOT::DescriptorId_t)> & | comparator ) const |
Definition at line 1410 of file RNTupleDescriptor.cxx.
| ROOT::RNTupleDescriptor::RFieldDescriptorIterable ROOT::RNTupleDescriptor::GetFieldIterable | ( | ROOT::DescriptorId_t | fieldId | ) | const |
Definition at line 1418 of file RNTupleDescriptor.cxx.
| ROOT::RNTupleDescriptor::RFieldDescriptorIterable ROOT::RNTupleDescriptor::GetFieldIterable | ( | ROOT::DescriptorId_t | fieldId, |
| const std::function< bool(ROOT::DescriptorId_t, ROOT::DescriptorId_t)> & | comparator ) const |
Definition at line 1423 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 900 of file RNTupleDescriptor.hxx.
|
inline |
Returns the logical parent of all top-level RNTuple data fields.
Definition at line 899 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 930 of file RNTupleDescriptor.hxx.
|
inline |
Return header extension information; if the descriptor does not have a header extension, return nullptr.
Definition at line 923 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 890 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 882 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 892 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 888 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 889 of file RNTupleDescriptor.hxx.
| ROOT::NTupleSize_t ROOT::RNTupleDescriptor::GetNElements | ( | ROOT::DescriptorId_t | physicalColumnId | ) | const |
Definition at line 331 of file RNTupleDescriptor.cxx.
|
inline |
We know the number of entries from adding the cluster summaries.
Definition at line 895 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 891 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 885 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 886 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 887 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 832 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 831 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 830 of file RNTupleDescriptor.hxx.
| std::string ROOT::RNTupleDescriptor::GetQualifiedFieldName | ( | ROOT::DescriptorId_t | fieldId | ) | const |
Walks up the parents of the field ID and returns a field name of the form a.b.c.d In case of invalid field ID, an empty string is returned.
Definition at line 381 of file RNTupleDescriptor.cxx.
| ROOT::RNTupleDescriptor::RFieldDescriptorIterable ROOT::RNTupleDescriptor::GetTopLevelFields | ( | ) | const |
Definition at line 1430 of file RNTupleDescriptor.cxx.
| ROOT::RNTupleDescriptor::RFieldDescriptorIterable ROOT::RNTupleDescriptor::GetTopLevelFields | ( | const std::function< bool(ROOT::DescriptorId_t, ROOT::DescriptorId_t)> & | comparator | ) | const |
Definition at line 1435 of file RNTupleDescriptor.cxx.
| std::string ROOT::RNTupleDescriptor::GetTypeNameForComparison | ( | const RFieldDescriptor & | fieldDesc | ) | const |
Adjust the type name of the passed RFieldDescriptor for comparison with another renormalized type name.
Definition at line 393 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 834 of file RNTupleDescriptor.hxx.
Definition at line 919 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 931 of file RNTupleDescriptor.hxx.
|
delete |
|
default |
| bool ROOT::RNTupleDescriptor::operator== | ( | const RNTupleDescriptor & | other | ) | const |
Definition at line 316 of file RNTupleDescriptor.cxx.
| void ROOT::RNTupleDescriptor::PrintInfo | ( | std::ostream & | output | ) | const |
Definition at line 78 of file RNTupleDescriptorFmt.cxx.
|
friend |
Definition at line 785 of file RNTupleDescriptor.hxx.
|
friend |
|
friend |
|
friend |
Definition at line 700 of file RNTupleDescriptor.hxx.
|
private |
List of AttributeSets linked to this RNTuple.
Definition at line 756 of file RNTupleDescriptor.hxx.
|
private |
Potentially a subset of all the available clusters.
Definition at line 754 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 748 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 719 of file RNTupleDescriptor.hxx.
|
private |
Free text from the user.
Definition at line 711 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 721 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 717 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 718 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder.
Definition at line 713 of file RNTupleDescriptor.hxx.
|
private |
The generation of the descriptor.
Once constructed by an RNTupleDescriptorBuilder, the descriptor is mostly immutable except for the set of active page locations. During the lifetime of the descriptor, page location information for clusters can be added or removed. When this happens, the generation should be increased, so that users of the descriptor know that the information changed. The generation is increased, e.g., by the page source's exclusive lock guard around the descriptor. It is used, e.g., by the descriptor cache in RNTupleReader.
Definition at line 746 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 722 of file RNTupleDescriptor.hxx.
|
private |
The RNTuple name needs to be unique in a given storage location (file).
Definition at line 709 of file RNTupleDescriptor.hxx.
|
private |
Updated by the descriptor builder when the cluster groups are added.
Definition at line 737 of file RNTupleDescriptor.hxx.
|
private |
Updated by the descriptor builder when the cluster groups are added.
Definition at line 736 of file RNTupleDescriptor.hxx.
|
private |
Updated by the descriptor builder when columns are added.
Definition at line 715 of file RNTupleDescriptor.hxx.
|
private |
Like fOnDiskHeaderSize, contains both cluster summaries and page locations.
Definition at line 734 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder when deserialized.
Definition at line 732 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder when deserialized.
Definition at line 733 of file RNTupleDescriptor.hxx.
|
private |
References cluster groups sorted by entry range and thus allows for binary search.
Note that this list is empty during the descriptor building process and will only be created when the final descriptor is extracted from the builder.
Definition at line 752 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder when deserialized.
Definition at line 727 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder when deserialized.
Definition at line 728 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder when deserialized.
Definition at line 729 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder when deserialized.
Definition at line 730 of file RNTupleDescriptor.hxx.