Classes designed for columnar data storage in modern ROOT.
RNTuple (for n-tuple and nested tuple) is the evolution of TTree columnar data storage. RNTuple introduces new interfaces that aim to be more robust. In particular, the new interfaces are type-safe through the use of templates, and the ownership is well-defined through the use of smart pointers. For instance
tree->Branch("px", &Category, "px/F");
becomes
auto px = model->MakeField<float>("px");
// px is std::shared_ptr<float>
The physical layout changes slightly from big endian to little endian so that it matches the in-memory layout on most modern architectures. Combined with a clear separation of offset/index data and payload data for collections, uncompressed RNTuple data can be directly mapped to memory without further copies.
RNTuple shall investigate improvements of the TTree I/O in the following ways
At the logical layer, the user defines a data model using the RNTupleModel class. The data model is a collection of serializable C++ types with associated names, similar to branches in a TTree. The data model can contain (nested) collections, e.g., a type can be std::vector<std::vector<float>>.
Each serializable type is represented by a field, concretely by a templated version of RField, e.g. RField<double>. A field can generate or adopt an associated value, which represents a memory location storing a value of the given C++ type. These distinguished memory locations are the destinations and sources for the deserialization and serialization.
The (de-)serialization is a mapping from the C++ type to the more simple column type system. A column contains an arbitrary number of fixed-sized elements of a well-defined set of types: integers and floats of different bit sizes. A C++ type may be mapped to multiple columns. For instance, an std::vector<float> maps to two columns, an offset column indicating the size of the vector per entry, and a payload column with the float data.
Columns are partitioned into pages (roughly: TTree baskets) of a few kB – a MB each. The physical layer (only) needs to provide the means to store and retrieve pages. The physical layer is decoupled from the high-level C++ logic. The physical layer implements an abstract page storage interface, so that dedicated implementations for key-value stores and other storage systems are conceivable. At this point, the only provided backend stores the pages in ROOT files.
RNTuples are further grouped into clusters, which are, like TTree clusters, self-contained blocks of consecutive entries. Clusters provide a unit of writing and provide the means for parallel writing of RNTuple data.
Classes | |
| class | ENTupleColumnType |
| The available trivial, native content types of a column. More... | |
| class | RArrayAsRVecField |
| Additional classes related to sequence containers. More... | |
| class | ROOT::RFieldBase::RBulk |
| Points to an array of objects with RNTuple I/O support, used for bulk reading. More... | |
| class | ROOT::Experimental::Internal::RCluster |
| An in-memory subset of the packed and compressed pages of a cluster. More... | |
| class | ROOT::RClusterDescriptor |
| Metadata for RNTuple clusters. More... | |
| class | ROOT::Internal::RClusterDescriptorBuilder |
| A helper class for piece-wise construction of an RClusterDescriptor. More... | |
| class | ROOT::RNTupleDescriptor::RClusterDescriptorIterable |
| Used to loop over all the clusters of an RNTuple (in unspecified order) More... | |
| class | ROOT::RClusterGroupDescriptor |
| Clusters are bundled in cluster groups. More... | |
| class | ROOT::Internal::RClusterGroupDescriptorBuilder |
| A helper class for piece-wise construction of an RClusterGroupDescriptor. More... | |
| class | ROOT::RNTupleDescriptor::RClusterGroupDescriptorIterable |
| Used to loop over all the cluster groups of an RNTuple (in unspecified order) More... | |
| class | ROOT::Experimental::Internal::RClusterPool |
| Managed a set of clusters containing compressed and packed pages. More... | |
| class | ROOT::Internal::RColumn |
| A column is a storage-backed array of a simple, fixed-size type, from which pages can be mapped into memory. More... | |
| class | ROOT::RColumnDescriptor |
| Metadata stored for every column of an RNTuple. More... | |
| class | ROOT::Internal::RColumnDescriptorBuilder |
| A helper class for piece-wise construction of an RColumnDescriptor. More... | |
| class | ROOT::RNTupleDescriptor::RColumnDescriptorIterable |
| Used to loop over a field's associated columns. More... | |
| class | ROOT::Internal::RColumnElementBase |
| A column element encapsulates the translation between basic C++ types and their column representation. More... | |
| class | RColumnRange |
| The window of element indexes of a particular column in a particular cluster. More... | |
| class | ROOT::RFieldBase::RColumnRepresentations |
| The list of column representations a field can have. More... | |
| class | ROOT::Experimental::Internal::RDaosNTupleAnchor |
| Entry point for an RNTuple in a DAOS container. More... | |
| class | ROOT::REntry |
| The REntry is a collection of values in an ntuple corresponding to a complete row in the data set. More... | |
| class | ROOT::Experimental::Internal::RNTupleJoinTable::REntryMapping |
| Provides a mapping from one or several join field values to an entry index. More... | |
| class | ROOT::RExtraTypeInfoDescriptor |
| Field specific extra type information from the header / extenstion header. More... | |
| class | ROOT::Internal::RExtraTypeInfoDescriptorBuilder |
| A helper class for piece-wise construction of an RExtraTypeInfoDescriptor. More... | |
| class | ROOT::RNTupleDescriptor::RExtraTypeInfoDescriptorIterable |
| Used to loop over all the extra type info record of an RNTuple (in unspecified order) More... | |
| class | ROOT::RFieldBase |
| A field translates read and write calls from/to underlying columns to/from tree values. More... | |
| class | ROOT::Experimental::RNTupleProcessor::RFieldContext |
| Manager for a field as part of the RNTupleProcessor. More... | |
| class | ROOT::RFieldDescriptor |
| Metadata stored for every field of an RNTuple. More... | |
| class | ROOT::Internal::RFieldDescriptorBuilder |
| A helper class for piece-wise construction of an RFieldDescriptor. More... | |
| class | ROOT::RNTupleDescriptor::RFieldDescriptorIterable |
| Used to loop over a field's child fields. More... | |
| class | ROOT::RFieldToken |
| A field token identifies a (sub)field in an entry. More... | |
| class | ROOT::Detail::RFieldVisitor |
| Abstract base class for classes implementing the visitor design pattern. More... | |
| class | ROOT::RNTupleDescriptor::RHeaderExtension |
| Summarizes information about fields and the corresponding columns that were added after the header has been serialized. More... | |
| class | ROOT::Experimental::RNTupleProcessor::RIterator |
| Iterator over the entries of an RNTuple, or vertical concatenation thereof. More... | |
| class | ROOT::Internal::RMiniFileReader |
| Read RNTuple data blocks from a TFile container, provided by a RRawFile. More... | |
| class | ROOT::RNTuple |
| Representation of an RNTuple data set in a ROOT file. More... | |
| class | ROOT::Experimental::Detail::RNTupleAtomicCounter |
| A thread-safe integral performance counter. More... | |
| class | ROOT::Experimental::Detail::RNTupleCalcPerf |
| A metric element that computes its floating point value from other counters. More... | |
| class | ROOT::Experimental::RNTupleChainProcessor |
| Processor specialization for vertically combined (chained) RNTupleProcessors. More... | |
| class | ROOT::RNTupleCollectionView |
| A view for a collection, that can itself generate new ntuple views for its nested fields. More... | |
| class | ROOT::Internal::RNTupleCompressor |
| Helper class to compress data blocks in the ROOT compression frame format. More... | |
| class | ROOT::Internal::RNTupleDecompressor |
| Helper class to uncompress data blocks in the ROOT compression frame format. More... | |
| class | ROOT::RNTupleDescriptor |
| The on-storage metadata of an RNTuple. More... | |
| class | ROOT::Internal::RNTupleDescriptorBuilder |
| A helper class for piece-wise construction of an RNTupleDescriptor. More... | |
| class | ROOT::RNTupleDirectAccessView< T > |
| A view variant that provides direct access to the I/O buffers. More... | |
| class | ROOT::Internal::RNTupleFileWriter |
| Write RNTuple data blocks in a TFile or a bare file container. More... | |
| class | ROOT::Experimental::RNTupleFillContext |
| A context for filling entries (data) into clusters of an RNTuple. More... | |
| class | ROOT::RNTupleFillStatus |
| A status object after filling an entry. More... | |
| class | ROOT::Internal::RNTupleFormatter |
| Contains helper functions for RNTupleReader::PrintInfo() and RPrintSchemaVisitor::VisitField() More... | |
| class | ROOT::RNTupleGlobalRange |
| Used to loop over indexes (entries or collections) between start and end. More... | |
| class | ROOT::Experimental::RNTupleImporter |
| Converts a TTree into an RNTuple. More... | |
| class | ROOT::Experimental::RNTupleInspector |
| Inspect on-disk and storage-related information of an RNTuple. More... | |
| class | ROOT::Experimental::RNTupleJoinProcessor |
| Processor specialization for horizontally combined (joined) RNTuples. More... | |
| class | ROOT::Experimental::Internal::RNTupleJoinTable |
| Builds a join table on one or several fields of an RNTuple so it can be joined onto other RNTuples. More... | |
| class | ROOT::RNTupleLocalRange |
| Used to loop over entries of collections in a single cluster. More... | |
| class | ROOT::Experimental::Internal::RNTupleMerger |
| Given a set of RPageSources merge them into an RPagePersistentSink, optionally changing their compression. More... | |
| class | ROOT::Experimental::Detail::RNTupleMetrics |
| A collection of Counter objects with a name, a unit, and a description. More... | |
| class | ROOT::RNTupleModel |
| The RNTupleModel encapulates the schema of an RNTuple. More... | |
| class | ROOT::Internal::RNTupleModelChangeset |
The incremental changes to a RNTupleModel More... | |
| class | ROOT::Experimental::RNTupleOpenSpec |
| Specification of the name and location of an RNTuple, used for creating a new RNTupleProcessor. More... | |
| class | ROOT::Experimental::RNTupleParallelWriter |
| A writer to fill an RNTuple from multiple contexts. More... | |
| class | ROOT::Experimental::Detail::RNTuplePerfCounter |
| A performance counter with a name and a unit, which can be activated on demand. More... | |
| class | ROOT::Experimental::Detail::RNTuplePlainCounter |
| A non thread-safe integral performance counter. More... | |
| class | ROOT::Experimental::RNTupleProcessor |
| Interface for iterating over entries of RNTuples and vertically concatenated RNTuples (chains). More... | |
| class | ROOT::RNTupleReader |
| Reads RNTuple data from storage. More... | |
| class | ROOT::RNTupleReadOptions |
| Common user-tunable settings for reading RNTuples. More... | |
| class | ROOT::Internal::RNTupleSerializer |
| A helper class for serializing and deserialization of the RNTuple binary format. More... | |
| class | ROOT::Experimental::RNTupleSingleProcessor |
| Processor specialization for processing a single RNTuple. More... | |
| class | ROOT::Experimental::Detail::RNTupleTickCounter< BaseCounterT > |
| A counter for CPU ticks. More... | |
| class | ROOT::Experimental::Detail::RNTupleTimer< WallTimeT, CpuTimeT > |
| Record wall time and CPU time between construction and destruction. More... | |
| class | ROOT::RNTupleView< T > |
| An RNTupleView for a known type. More... | |
| class | ROOT::RNTupleViewBase< T > |
| An RNTupleView provides read-only access to a single field of an RNTuple. More... | |
| class | ROOT::RNTupleWriteOptions |
| Common user-tunable settings for storing RNTuples. More... | |
| class | ROOT::Experimental::RNTupleWriteOptionsDaos |
| DAOS-specific user-tunable settings for storing ntuples. More... | |
| class | ROOT::RNTupleWriter |
| An RNTuple that gets filled with entries (data) and writes them to storage. More... | |
| class | ROOT::Experimental::Internal::ROnDiskPage |
| A page as being stored on disk, that is packed and compressed. More... | |
| class | ROOT::Experimental::Internal::ROnDiskPageMap |
| A memory region that contains packed and compressed pages. More... | |
| class | ROOT::Experimental::Internal::ROnDiskPageMapHeap |
| An ROnDiskPageMap that is used for an fMemory allocated as an array of unsigned char. More... | |
| class | ROOT::Internal::RPage |
| A page is a slice of a column that is mapped into memory. More... | |
| class | ROOT::Internal::RPageAllocator |
| Abstract interface to allocate and release pages. More... | |
| class | ROOT::Internal::RPageAllocatorHeap |
| Uses standard C++ memory allocation for the column data pages. More... | |
| class | ROOT::RClusterDescriptor::RPageInfo |
| Information about a single page in the context of a cluster's page range. More... | |
| class | ROOT::RClusterDescriptor::RPageInfoExtended |
| Additional information about a page in an in-memory RPageRange. More... | |
| class | ROOT::Experimental::Internal::RPageNullSink |
| Dummy sink that discards all pages. More... | |
| class | ROOT::Internal::RPagePersistentSink |
| Base class for a sink with a physical storage backend. More... | |
| class | ROOT::Internal::RPagePool |
| A thread-safe cache of pages loaded from the page source. More... | |
| class | ROOT::RClusterDescriptor::RPageRange |
| Records the partition of data into pages for a particular column in a particular cluster. More... | |
| class | ROOT::Internal::RPageRef |
| Reference to a page stored in the page pool. More... | |
| class | ROOT::Internal::RPageSink |
| Abstract interface to write data into an ntuple. More... | |
| class | ROOT::Internal::RPageSinkBuf |
| Wrapper sink that coalesces cluster column page writes. More... | |
| class | ROOT::Experimental::Internal::RPageSinkDaos |
| Storage provider that writes ntuple pages to into a DAOS container. More... | |
| class | ROOT::Internal::RPageSinkFile |
| Storage provider that write ntuple pages into a file. More... | |
| class | ROOT::Internal::RPageSource |
| Abstract interface to read data from an ntuple. More... | |
| class | ROOT::Experimental::Internal::RPageSourceDaos |
| Storage provider that reads ntuple pages from a DAOS container. More... | |
| class | ROOT::Internal::RPageSourceFile |
| Storage provider that reads ntuple pages from a file. More... | |
| class | ROOT::Internal::RPageStorage |
| Common functionality of an ntuple storage for both reading and writing. More... | |
| class | ROOT::Internal::RPrepareVisitor |
| Visitor used for a pre-processing run to collect information needed by another visitor class. More... | |
| class | ROOT::Internal::RPrintSchemaVisitor |
| Contains settings for printing and prints a summary of an RField instance. More... | |
| class | ROOT::Internal::RPrintValueVisitor |
| Renders a JSON value corresponding to the field. More... | |
| class | ROOT::Internal::RProjectedFields |
| Container for the projected fields of an RNTupleModel. More... | |
| class | ROOT::Experimental::Detail::RRawPtrWriteEntry |
| A container of const raw pointers, corresponding to a row in the data set. More... | |
| class | ROOT::Internal::RWritePageMemoryManager |
| Helper to maintain a memory budget for the write pages of a set of columns. More... | |
