The on-storage meta-data of an ntuple.
Represents the on-disk (on storage) information about an ntuple. The meta-data consists of a header and one or several footers. The header carries the ntuple schema, i.e. the fields and the associated columns and their relationships. The footer(s) carry information about one or several clusters. For every cluster, a footer stores its location and size, and for every column the range of element indexes as well as a list of pages and page locations.
The descriptor provide machine-independent (de-)serialization of headers and footers, and it provides lookup routines for ntuple objects (pages, clusters, ...). It is supposed to be usable by all RPageStorage implementations.
The serialization does not use standard ROOT streamers in order to not let it depend on libCore. The serialization uses the concept of frames: header, footer, and substructures have a preamble with version numbers and the size of the writte struct. This allows for forward and backward compatibility when the meta-data evolves.
Definition at line 539 of file RNTupleDescriptor.hxx.
Classes | |
class | RClusterDescriptorIterable |
Used to loop over all the clusters of an ntuple (in unspecified order) More... | |
class | RClusterGroupDescriptorIterable |
Used to loop over all the cluster groups of an ntuple (in unspecified order) More... | |
class | RColumnDescriptorIterable |
Used to loop over a field's associated columns. More... | |
struct | RCreateModelOptions |
Modifiers passed to CreateModel More... | |
class | RExtraTypeInfoDescriptorIterable |
Used to loop over all the extra type info record of an ntuple (in unspecified order) More... | |
class | RFieldDescriptorIterable |
Used to loop over a field's child fields. More... | |
class | RHeaderExtension |
Summarizes information about fields and the corresponding columns that were added after the header has been serialized. More... | |
Static Public Attributes | |
static constexpr unsigned int | kFeatureFlagTest = 137 |
Private Member Functions | |
DescriptorId_t | FindClusterId (NTupleSize_t entryIdx) const |
Private Attributes | |
std::unordered_map< DescriptorId_t, RClusterDescriptor > | fClusterDescriptors |
May contain only a subset of all the available clusters, e.g. | |
std::unordered_map< DescriptorId_t, RClusterGroupDescriptor > | fClusterGroupDescriptors |
std::unordered_map< DescriptorId_t, RColumnDescriptor > | fColumnDescriptors |
std::string | fDescription |
Free text from the user. | |
std::vector< RExtraTypeInfoDescriptor > | fExtraTypeInfoDescriptors |
std::set< unsigned int > | fFeatureFlags |
std::unordered_map< DescriptorId_t, RFieldDescriptor > | fFieldDescriptors |
DescriptorId_t | fFieldZeroId = kInvalidDescriptorId |
Set by the descriptor builder. | |
std::uint64_t | fGeneration = 0 |
Once constructed by an RNTupleDescriptorBuilder, the descriptor is mostly immutable except for set of active the page locations. | |
std::unique_ptr< RHeaderExtension > | fHeaderExtension |
std::string | fName |
The ntuple name needs to be unique in a given storage location (file) | |
std::uint64_t | fNClusters = 0 |
Updated by the descriptor builder when the cluster groups are added. | |
std::uint64_t | fNEntries = 0 |
Updated by the descriptor builder when the cluster groups are added. | |
std::uint64_t | fNPhysicalColumns = 0 |
Updated by the descriptor builder when columns are added. | |
std::uint64_t | fOnDiskFooterSize = 0 |
Like fOnDiskHeaderSize, contains both cluster summaries and page locations. | |
std::uint64_t | fOnDiskHeaderSize = 0 |
Set by the descriptor builder when deserialized. | |
std::uint64_t | fOnDiskHeaderXxHash3 = 0 |
Set by the descriptor builder when deserialized. | |
std::vector< DescriptorId_t > | fSortedClusterGroupIds |
References cluster groups sorted by entry range and thus allows for binary search. | |
Friends | |
class | Internal::RNTupleDescriptorBuilder |
#include <ROOT/RNTupleDescriptor.hxx>
|
default |
|
delete |
|
default |
ROOT::RResult< void > ROOT::Experimental::RNTupleDescriptor::AddClusterGroupDetails | ( | DescriptorId_t | clusterGroupId, |
std::vector< RClusterDescriptor > & | clusterDescs ) |
Methods to load and drop cluster group details (cluster IDs and page locations)
Definition at line 597 of file RNTupleDescriptor.cxx.
ROOT::Experimental::RNTupleDescriptor ROOT::Experimental::RNTupleDescriptor::Clone | ( | ) | const |
Definition at line 666 of file RNTupleDescriptor.cxx.
std::unique_ptr< ROOT::Experimental::RNTupleModel > ROOT::Experimental::RNTupleDescriptor::CreateModel | ( | const RCreateModelOptions & | options = RCreateModelOptions() | ) | const |
Re-create the C++ model from the stored meta-data.
Definition at line 640 of file RNTupleDescriptor.cxx.
ROOT::RResult< void > ROOT::Experimental::RNTupleDescriptor::DropClusterGroupDetails | ( | DescriptorId_t | clusterGroupId | ) |
Definition at line 625 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindClusterId | ( | DescriptorId_t | physicalColumnId, |
NTupleSize_t | index ) const |
Definition at line 402 of file RNTupleDescriptor.cxx.
|
private |
Definition at line 460 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindFieldId | ( | std::string_view | fieldName | ) | const |
Searches for a top-level field.
Definition at line 371 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindFieldId | ( | std::string_view | fieldName, |
DescriptorId_t | parentId ) const |
Definition at line 340 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindLogicalColumnId | ( | DescriptorId_t | fieldId, |
std::uint32_t | columnIndex, | ||
std::uint16_t | representationIndex ) const |
Definition at line 377 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindNextClusterId | ( | DescriptorId_t | clusterId | ) | const |
Definition at line 513 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindPhysicalColumnId | ( | DescriptorId_t | fieldId, |
std::uint32_t | columnIndex, | ||
std::uint16_t | representationIndex ) const |
Definition at line 392 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindPrevClusterId | ( | DescriptorId_t | clusterId | ) | const |
Definition at line 525 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 636 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 632 of file RNTupleDescriptor.hxx.
ROOT::Experimental::RNTupleDescriptor::RClusterGroupDescriptorIterable ROOT::Experimental::RNTupleDescriptor::GetClusterGroupIterable | ( | ) | const |
Definition at line 1357 of file RNTupleDescriptor.cxx.
ROOT::Experimental::RNTupleDescriptor::RClusterDescriptorIterable ROOT::Experimental::RNTupleDescriptor::GetClusterIterable | ( | ) | const |
Definition at line 1363 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 628 of file RNTupleDescriptor.hxx.
ROOT::Experimental::RNTupleDescriptor::RColumnDescriptorIterable ROOT::Experimental::RNTupleDescriptor::GetColumnIterable | ( | ) | const |
Definition at line 1339 of file RNTupleDescriptor.cxx.
ROOT::Experimental::RNTupleDescriptor::RColumnDescriptorIterable ROOT::Experimental::RNTupleDescriptor::GetColumnIterable | ( | const RFieldDescriptor & | fieldDesc | ) | const |
Definition at line 1345 of file RNTupleDescriptor.cxx.
ROOT::Experimental::RNTupleDescriptor::RColumnDescriptorIterable ROOT::Experimental::RNTupleDescriptor::GetColumnIterable | ( | DescriptorId_t | fieldId | ) | const |
Definition at line 1351 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 665 of file RNTupleDescriptor.hxx.
ROOT::Experimental::RNTupleDescriptor::RExtraTypeInfoDescriptorIterable ROOT::Experimental::RNTupleDescriptor::GetExtraTypeInfoIterable | ( | ) | const |
Definition at line 1369 of file RNTupleDescriptor.cxx.
std::vector< std::uint64_t > ROOT::Experimental::RNTupleDescriptor::GetFeatureFlags | ( | ) | const |
Definition at line 576 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 627 of file RNTupleDescriptor.hxx.
ROOT::Experimental::RNTupleDescriptor::RFieldDescriptorIterable ROOT::Experimental::RNTupleDescriptor::GetFieldIterable | ( | const RFieldDescriptor & | fieldDesc | ) | const |
Definition at line 1302 of file RNTupleDescriptor.cxx.
ROOT::Experimental::RNTupleDescriptor::RFieldDescriptorIterable ROOT::Experimental::RNTupleDescriptor::GetFieldIterable | ( | const RFieldDescriptor & | fieldDesc, |
const std::function< bool(DescriptorId_t, DescriptorId_t)> & | comparator ) const |
Definition at line 1307 of file RNTupleDescriptor.cxx.
ROOT::Experimental::RNTupleDescriptor::RFieldDescriptorIterable ROOT::Experimental::RNTupleDescriptor::GetFieldIterable | ( | DescriptorId_t | fieldId | ) | const |
Definition at line 1314 of file RNTupleDescriptor.cxx.
ROOT::Experimental::RNTupleDescriptor::RFieldDescriptorIterable ROOT::Experimental::RNTupleDescriptor::GetFieldIterable | ( | DescriptorId_t | fieldId, |
const std::function< bool(DescriptorId_t, DescriptorId_t)> & | comparator ) const |
Definition at line 1319 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 681 of file RNTupleDescriptor.hxx.
|
inline |
Returns the logical parent of all top-level NTuple data fields.
Definition at line 680 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 707 of file RNTupleDescriptor.hxx.
|
inline |
Return header extension information; if the descriptor does not have a header extension, return nullptr
Definition at line 701 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 672 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 664 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 670 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 671 of file RNTupleDescriptor.hxx.
ROOT::Experimental::NTupleSize_t ROOT::Experimental::RNTupleDescriptor::GetNElements | ( | DescriptorId_t | physicalColumnId | ) | const |
Definition at line 327 of file RNTupleDescriptor.cxx.
|
inline |
We know the number of entries from adding the cluster summaries.
Definition at line 676 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 673 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 667 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 668 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 669 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 625 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 624 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 623 of file RNTupleDescriptor.hxx.
std::string ROOT::Experimental::RNTupleDescriptor::GetQualifiedFieldName | ( | DescriptorId_t | fieldId | ) | const |
Walks up the parents of the field ID and returns a field name of the form a.b.c.d In case of invalid field ID, an empty string is returned.
Definition at line 359 of file RNTupleDescriptor.cxx.
ROOT::Experimental::RNTupleDescriptor::RFieldDescriptorIterable ROOT::Experimental::RNTupleDescriptor::GetTopLevelFields | ( | ) | const |
Definition at line 1326 of file RNTupleDescriptor.cxx.
ROOT::Experimental::RNTupleDescriptor::RFieldDescriptorIterable ROOT::Experimental::RNTupleDescriptor::GetTopLevelFields | ( | const std::function< bool(DescriptorId_t, DescriptorId_t)> & | comparator | ) | const |
Definition at line 1332 of file RNTupleDescriptor.cxx.
Definition at line 697 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 708 of file RNTupleDescriptor.hxx.
|
delete |
|
default |
bool ROOT::Experimental::RNTupleDescriptor::operator== | ( | const RNTupleDescriptor & | other | ) | const |
Definition at line 311 of file RNTupleDescriptor.cxx.
void ROOT::Experimental::RNTupleDescriptor::PrintInfo | ( | std::ostream & | output | ) | const |
Definition at line 83 of file RNTupleDescriptorFmt.cxx.
|
friend |
Definition at line 540 of file RNTupleDescriptor.hxx.
|
private |
May contain only a subset of all the available clusters, e.g.
the clusters of the current file from a chain of files
Definition at line 580 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 573 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 572 of file RNTupleDescriptor.hxx.
|
private |
Free text from the user.
Definition at line 549 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 581 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 570 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 571 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder.
Definition at line 559 of file RNTupleDescriptor.hxx.
|
private |
Once constructed by an RNTupleDescriptorBuilder, the descriptor is mostly immutable except for set of active the page locations.
During the lifetime of the descriptor, page location information for clusters can be added or removed. When this happens, the generation should be increased, so that users of the descriptor know that the information changed. The generation is increased, e.g., by the page source's exclusive lock guard around the descriptor. It is used, e.g., by the descriptor cache in RNTupleReader.
Definition at line 568 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 582 of file RNTupleDescriptor.hxx.
|
private |
The ntuple name needs to be unique in a given storage location (file)
Definition at line 547 of file RNTupleDescriptor.hxx.
|
private |
Updated by the descriptor builder when the cluster groups are added.
Definition at line 556 of file RNTupleDescriptor.hxx.
|
private |
Updated by the descriptor builder when the cluster groups are added.
Definition at line 555 of file RNTupleDescriptor.hxx.
|
private |
Updated by the descriptor builder when columns are added.
Definition at line 557 of file RNTupleDescriptor.hxx.
|
private |
Like fOnDiskHeaderSize, contains both cluster summaries and page locations.
Definition at line 553 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder when deserialized.
Definition at line 552 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder when deserialized.
Definition at line 551 of file RNTupleDescriptor.hxx.
|
private |
References cluster groups sorted by entry range and thus allows for binary search.
Note that this list is empty during the descriptor building process and will only be created when the final descriptor is extracted from the builder.
Definition at line 577 of file RNTupleDescriptor.hxx.
|
staticconstexpr |
Definition at line 588 of file RNTupleDescriptor.hxx.