The on-storage meta-data of an ntuple.
Represents the on-disk (on storage) information about an ntuple. The meta-data consists of a header and one or several footers. The header carries the ntuple schema, i.e. the fields and the associated columns and their relationships. The footer(s) carry information about one or several clusters. For every cluster, a footer stores its location and size, and for every column the range of element indexes as well as a list of pages and page locations.
The descriptor provide machine-independent (de-)serialization of headers and footers, and it provides lookup routines for ntuple objects (pages, clusters, ...). It is supposed to be usable by all RPageStorage implementations.
The serialization does not use standard ROOT streamers in order to not let it depend on libCore. The serialization uses the concept of frames: header, footer, and substructures have a preamble with version numbers and the size of the writte struct. This allows for forward and backward compatibility when the meta-data evolves.
Definition at line 304 of file RNTupleDescriptor.hxx.
Classes | |
class | RClusterDescriptorIterable |
Used to loop over all the clusters of an ntuple (in unspecified order) More... | |
class | RColumnDescriptorIterable |
Used to loop over a field's associated columns. More... | |
class | RFieldDescriptorIterable |
Used to loop over a field's child fields. More... | |
Public Member Functions | |
RNTupleDescriptor ()=default | |
RNTupleDescriptor (const RNTupleDescriptor &other)=delete | |
RNTupleDescriptor (RNTupleDescriptor &&other)=default | |
DescriptorId_t | FindClusterId (DescriptorId_t columnId, NTupleSize_t index) const |
DescriptorId_t | FindColumnId (DescriptorId_t fieldId, std::uint32_t columnIndex) const |
DescriptorId_t | FindFieldId (std::string_view fieldName) const |
Searches for a top-level field. | |
DescriptorId_t | FindFieldId (std::string_view fieldName, DescriptorId_t parentId) const |
DescriptorId_t | FindNextClusterId (DescriptorId_t clusterId) const |
DescriptorId_t | FindPrevClusterId (DescriptorId_t clusterId) const |
std::unique_ptr< RNTupleModel > | GenerateModel () const |
Re-create the C++ model from the stored meta-data. | |
std::string | GetAuthor () const |
const RClusterDescriptor & | GetClusterDescriptor (DescriptorId_t clusterId) const |
RClusterDescriptorIterable | GetClusterIterable () const |
const RColumnDescriptor & | GetColumnDescriptor (DescriptorId_t columnId) const |
RColumnDescriptorIterable | GetColumnIterable (const RFieldDescriptor &fieldDesc) const |
RColumnDescriptorIterable | GetColumnIterable (DescriptorId_t fieldId) const |
std::string | GetCustodian () const |
std::string | GetDescription () const |
const RFieldDescriptor & | GetFieldDescriptor (DescriptorId_t fieldId) const |
RFieldDescriptorIterable | GetFieldIterable (const RFieldDescriptor &fieldDesc) const |
RFieldDescriptorIterable | GetFieldIterable (const RFieldDescriptor &fieldDesc, const std::function< bool(DescriptorId_t, DescriptorId_t)> &comparator) const |
RFieldDescriptorIterable | GetFieldIterable (DescriptorId_t fieldId) const |
RFieldDescriptorIterable | GetFieldIterable (DescriptorId_t fieldId, const std::function< bool(DescriptorId_t, DescriptorId_t)> &comparator) const |
const RFieldDescriptor & | GetFieldZero () const |
DescriptorId_t | GetFieldZeroId () const |
Returns the logical parent of all top-level NTuple data fields. | |
std::uint32_t | GetFooterSize () const |
RNTupleUuid | GetGroupUuid () const |
std::uint32_t | GetHeaderSize () const |
std::string | GetName () const |
std::size_t | GetNClusters () const |
std::size_t | GetNColumns () const |
NTupleSize_t | GetNElements (DescriptorId_t columnId) const |
NTupleSize_t | GetNEntries () const |
std::size_t | GetNFields () const |
std::uint64_t | GetOnDiskFooterSize () const |
std::uint64_t | GetOnDiskHeaderSize () const |
RNTupleUuid | GetOwnUuid () const |
std::string | GetQualifiedFieldName (DescriptorId_t fieldId) const |
Walks up the parents of the field ID and returns a field name of the form a.b.c.d In case of invalid field ID, an empty string is returned. | |
std::chrono::system_clock::time_point | GetTimeStampData () const |
std::chrono::system_clock::time_point | GetTimeStampWritten () const |
RFieldDescriptorIterable | GetTopLevelFields () const |
RFieldDescriptorIterable | GetTopLevelFields (const std::function< bool(DescriptorId_t, DescriptorId_t)> &comparator) const |
RNTupleVersion | GetVersion () const |
RNTupleDescriptor & | operator= (const RNTupleDescriptor &other)=delete |
RNTupleDescriptor & | operator= (RNTupleDescriptor &&other)=default |
bool | operator== (const RNTupleDescriptor &other) const |
void | PrintInfo (std::ostream &output) const |
std::uint32_t | SerializeFooter (void *buffer) const |
Serializes cluster meta data. Returns the number of bytes and fills buffer if it is not nullptr. | |
std::uint32_t | SerializeHeader (void *buffer) const |
We deliberately do not use ROOT's built-in serialization in order to allow for use of RNTuple's without libCore Serializes the global ntuple information as well as the column and field schemata Returns the number of bytes and fills buffer if it is not nullptr. | |
Static Public Member Functions | |
static void | LocateMetadata (const void *postscript, std::uint32_t &szHeader, std::uint32_t &szFooter) |
Given kNBytesPostscript bytes, extract the header and footer lengths in bytes. | |
Static Public Attributes | |
static constexpr std::uint16_t | kFrameVersionCurrent = 0 |
In order to handle changes to the serialization routine in future ntuple versions. | |
static constexpr std::uint16_t | kFrameVersionMin = 0 |
static constexpr unsigned int | kNBytesPostscript = 16 |
The last few bytes after the footer store the length of footer and header. | |
static constexpr unsigned int | kNBytesPreamble = 8 |
The preamble is sufficient to get the length of the header. | |
Private Attributes | |
std::string | fAuthor |
The origin of the data. | |
std::unordered_map< DescriptorId_t, RClusterDescriptor > | fClusterDescriptors |
May contain only a subset of all the available clusters, e.g. | |
std::unordered_map< DescriptorId_t, RColumnDescriptor > | fColumnDescriptors |
std::string | fCustodian |
The current responsible for storing the data. | |
std::string | fDescription |
Free text from the user. | |
std::unordered_map< DescriptorId_t, RFieldDescriptor > | fFieldDescriptors |
RNTupleUuid | fGroupUuid |
Column sets that are created as derived sets from existing NTuples share the same group id. | |
std::string | fName |
The ntuple name needs to be unique in a given storage location (file) | |
std::uint64_t | fOnDiskFooterSize = 0 |
Like fOnDiskHeaderSize, contains both cluster summaries and page locations. | |
std::uint64_t | fOnDiskHeaderSize = 0 |
Set by the descriptor builder when deserialized. | |
RNTupleUuid | fOwnUuid |
Every NTuple gets a unique identifier. | |
std::chrono::system_clock::time_point | fTimeStampData |
The time stamp of the ntuple data (immutable) | |
std::chrono::system_clock::time_point | fTimeStampWritten |
The time stamp of writing the data to storage, which gets updated when re-written. | |
RNTupleVersion | fVersion |
The version evolves with the ntuple summary meta-data. | |
Friends | |
class | RNTupleDescriptorBuilder |
#include <ROOT/RNTupleDescriptor.hxx>
|
default |
|
delete |
|
default |
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindClusterId | ( | DescriptorId_t | columnId, |
NTupleSize_t | index | ||
) | const |
Definition at line 705 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindColumnId | ( | DescriptorId_t | fieldId, |
std::uint32_t | columnIndex | ||
) | const |
Definition at line 694 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindFieldId | ( | std::string_view | fieldName | ) | const |
Searches for a top-level field.
Definition at line 687 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindFieldId | ( | std::string_view | fieldName, |
DescriptorId_t | parentId | ||
) | const |
Definition at line 649 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindNextClusterId | ( | DescriptorId_t | clusterId | ) | const |
Definition at line 721 of file RNTupleDescriptor.cxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::FindPrevClusterId | ( | DescriptorId_t | clusterId | ) | const |
Definition at line 736 of file RNTupleDescriptor.cxx.
std::unique_ptr< ROOT::Experimental::RNTupleModel > ROOT::Experimental::RNTupleDescriptor::GenerateModel | ( | ) | const |
Re-create the C++ model from the stored meta-data.
Definition at line 748 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 580 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 535 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 573 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 532 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 564 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 568 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 581 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 579 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 529 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 539 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 542 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 547 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 550 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 598 of file RNTupleDescriptor.hxx.
ROOT::Experimental::DescriptorId_t ROOT::Experimental::RNTupleDescriptor::GetFieldZeroId | ( | ) | const |
Returns the logical parent of all top-level NTuple data fields.
Definition at line 680 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 522 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 586 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 519 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 578 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 590 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 589 of file RNTupleDescriptor.hxx.
ROOT::Experimental::NTupleSize_t ROOT::Experimental::RNTupleDescriptor::GetNElements | ( | DescriptorId_t | columnId | ) | const |
Definition at line 635 of file RNTupleDescriptor.cxx.
ROOT::Experimental::NTupleSize_t ROOT::Experimental::RNTupleDescriptor::GetNEntries | ( | ) | const |
Definition at line 626 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 588 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 527 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 526 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 585 of file RNTupleDescriptor.hxx.
std::string ROOT::Experimental::RNTupleDescriptor::GetQualifiedFieldName | ( | DescriptorId_t | fieldId | ) | const |
Walks up the parents of the field ID and returns a field name of the form a.b.c.d In case of invalid field ID, an empty string is returned.
Definition at line 666 of file RNTupleDescriptor.cxx.
|
inline |
Definition at line 582 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 583 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 555 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 558 of file RNTupleDescriptor.hxx.
|
inline |
Definition at line 584 of file RNTupleDescriptor.hxx.
|
static |
Given kNBytesPostscript bytes, extract the header and footer lengths in bytes.
Definition at line 614 of file RNTupleDescriptor.cxx.
|
delete |
|
default |
bool ROOT::Experimental::RNTupleDescriptor::operator== | ( | const RNTupleDescriptor & | other | ) | const |
Definition at line 513 of file RNTupleDescriptor.cxx.
void ROOT::Experimental::RNTupleDescriptor::PrintInfo | ( | std::ostream & | output | ) | const |
Definition at line 82 of file RNTupleDescriptorFmt.cxx.
std::uint32_t ROOT::Experimental::RNTupleDescriptor::SerializeFooter | ( | void * | buffer | ) | const |
Serializes cluster meta data. Returns the number of bytes and fills buffer if it is not nullptr.
Definition at line 566 of file RNTupleDescriptor.cxx.
std::uint32_t ROOT::Experimental::RNTupleDescriptor::SerializeHeader | ( | void * | buffer | ) | const |
We deliberately do not use ROOT's built-in serialization in order to allow for use of RNTuple's without libCore Serializes the global ntuple information as well as the column and field schemata Returns the number of bytes and fills buffer if it is not nullptr.
TODO(jblomer): instead of runtime testing for nullptr, there should be a template for the case where only the size of the buffer is required.
Definition at line 530 of file RNTupleDescriptor.cxx.
|
friend |
Definition at line 305 of file RNTupleDescriptor.hxx.
|
private |
The origin of the data.
Definition at line 313 of file RNTupleDescriptor.hxx.
|
private |
May contain only a subset of all the available clusters, e.g.
the clusters of the current file from a chain of files
Definition at line 335 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 332 of file RNTupleDescriptor.hxx.
|
private |
The current responsible for storing the data.
Definition at line 315 of file RNTupleDescriptor.hxx.
|
private |
Free text from the user.
Definition at line 311 of file RNTupleDescriptor.hxx.
|
private |
Definition at line 331 of file RNTupleDescriptor.hxx.
|
private |
Column sets that are created as derived sets from existing NTuples share the same group id.
NTuples in the same group have the same number of entries and are supposed to contain associated data.
Definition at line 326 of file RNTupleDescriptor.hxx.
|
private |
The ntuple name needs to be unique in a given storage location (file)
Definition at line 309 of file RNTupleDescriptor.hxx.
|
private |
Like fOnDiskHeaderSize, contains both cluster summaries and page locations.
Definition at line 329 of file RNTupleDescriptor.hxx.
|
private |
Set by the descriptor builder when deserialized.
Definition at line 328 of file RNTupleDescriptor.hxx.
|
private |
Every NTuple gets a unique identifier.
Definition at line 323 of file RNTupleDescriptor.hxx.
|
private |
The time stamp of the ntuple data (immutable)
Definition at line 317 of file RNTupleDescriptor.hxx.
|
private |
The time stamp of writing the data to storage, which gets updated when re-written.
Definition at line 319 of file RNTupleDescriptor.hxx.
|
private |
The version evolves with the ntuple summary meta-data.
Definition at line 321 of file RNTupleDescriptor.hxx.
|
staticconstexpr |
In order to handle changes to the serialization routine in future ntuple versions.
Definition at line 493 of file RNTupleDescriptor.hxx.
|
staticconstexpr |
Definition at line 494 of file RNTupleDescriptor.hxx.
|
staticconstexpr |
The last few bytes after the footer store the length of footer and header.
Definition at line 498 of file RNTupleDescriptor.hxx.
|
staticconstexpr |
The preamble is sufficient to get the length of the header.
Definition at line 496 of file RNTupleDescriptor.hxx.