Logo ROOT  
v6.34v6.32v6.30v6.28v6.26v6.24v6.22v6.20v6.18v6.16v6.14v6.12v6.10v6.08v6.06

Reference Guide
 
•All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Modules Pages
Loading...
Searching...
No Matches
List of all members | Public Types | Public Member Functions | Static Public Attributes | Protected Attributes | List of all members
ROOT::RNTupleWriteOptions Class Reference
NTuple-related classes

Common user-tunable settings for storing RNTuples.

All page sink classes need to support the common options.

Option name Type Default

Description

Compression std::uint32_t RCompressionSetting::EDefaults::kUseGeneralPurpose

The compression settings for this RNTuple

ApproxZippedClusterSize std::size_t 128 MiB

Approximation of the target compressed cluster size

MaxUnzippedClusterSize std::size_t 1280 MiB

Memory limit for committing a cluster: with very high compression ratio, we need a limit on how large the I/O buffer can grow during writing.

InitialUnzippedPageSize std::size_t 256

Initially, columns start with a page of this size. The default value is chosen to accomodate at least 32 elements of 64 bits, or 64 elements of 32 bits. If more elements are needed, pages are increased up until the byte limit given by the option MaxUnzippedPageSize or until the total page buffer limit is reached (as a sum of all page buffers). The total write buffer limit needs to be large enough to hold the initial pages of all columns.

MaxUnzippedPageSize std::size_t 1 MiB

Pages can grow only to the given limit in bytes.

PageBufferBudget std::size_t 0 / auto

The maximum size that the sum of all page buffers used for writing into a persistent sink are allowed to use. If set to zero, RNTuple will auto-adjust the budget based on the value of ApproxZippedClusterSize. If set manually, the size needs to be large enough to hold all initial page buffers. The total amount of memory for writing is larger, e.g. for the additional compressed buffers etc. Use RNTupleModel::EstimateWriteMemoryUsage() for the total estimated memory use for writing. The default values are tuned for a total write memory of around 400 MiB per fill context.

UseBufferedWrite bool true

Whether to use buffered writing (with RPageSinkBuf). This buffers compressed pages in memory, reorders them to keep pages of the same column adjacent, and coalesces the writes when committing a cluster.

UseDirectIO bool false

Whether to use Direct I/O for writing. Note that this introduces alignment requirements that may very between filesystems and platforms.

WriteBufferSize std::size_t 4 MiB

Buffer size to use for writing to files, must be a multiple of 4096 bytes. Testing suggests that 4MiB gives best performance (with Direct I/O) at a reasonable memory consumption.

UseImplicitMT EImplicitMT EImplicitMT::kDefault

Whether to use implicit multi-threading to compress pages. Only has an effect if buffered writing is turned on.

EnablePageChecksums bool true

If set, checksums will be calculated and written for every page. If turned off, will also turn off EnableSamePageMerging.

EnableSamePageMerging bool true

If set, identical pages are deduplicated and aliased on disk. Requires EnablePageChecksums and will throw if previously disabled.

Definition at line 179 of file RNTupleWriteOptions.hxx.

Public Types

enum class  EImplicitMT { kOff , kDefault }
 

Public Member Functions

virtual ~RNTupleWriteOptions ()=default
 
virtual std::unique_ptr< RNTupleWriteOptions > Clone () const
 
std::size_t GetApproxZippedClusterSize () const
 
std::uint32_t GetCompression () const
 
bool GetEnablePageChecksums () const
 
bool GetEnableSamePageMerging () const
 
std::size_t GetInitialUnzippedPageSize () const
 
std::uint64_t GetMaxKeySize () const
 
std::size_t GetMaxUnzippedClusterSize () const
 
std::size_t GetMaxUnzippedPageSize () const
 
std::size_t GetPageBufferBudget () const
 
bool GetUseBufferedWrite () const
 
bool GetUseDirectIO () const
 
EImplicitMT GetUseImplicitMT () const
 
std::size_t GetWriteBufferSize () const
 
void SetApproxZippedClusterSize (std::size_t val)
 
void SetCompression (RCompressionSetting::EAlgorithm::EValues algorithm, int compressionLevel)
 
void SetCompression (std::uint32_t val)
 
void SetEnablePageChecksums (bool val)
 Note that turning off page checksums will also turn off the same page merging optimization (see tuning.md)
 
void SetEnableSamePageMerging (bool val)
 
void SetInitialUnzippedPageSize (std::size_t val)
 
void SetMaxUnzippedClusterSize (std::size_t val)
 
void SetMaxUnzippedPageSize (std::size_t val)
 
void SetPageBufferBudget (std::size_t val)
 
void SetUseBufferedWrite (bool val)
 
void SetUseDirectIO (bool val)
 
void SetUseImplicitMT (EImplicitMT val)
 
void SetWriteBufferSize (std::size_t val)
 

Static Public Attributes

static constexpr std::uint64_t kDefaultMaxKeySize = 0x4000'0000
 

Protected Attributes

std::size_t fApproxZippedClusterSize = 128 * 1024 * 1024
 
std::uint32_t fCompression {RCompressionSetting::EDefaults::kUseGeneralPurpose}
 
bool fEnablePageChecksums = true
 
bool fEnableSamePageMerging = true
 
std::size_t fInitialUnzippedPageSize = 256
 
std::uint64_t fMaxKeySize = kDefaultMaxKeySize
 Specifies the max size of a payload storeable into a single TKey.
 
std::size_t fMaxUnzippedClusterSize = 10 * fApproxZippedClusterSize
 
std::size_t fMaxUnzippedPageSize = 1024 * 1024
 
std::size_t fPageBufferBudget = 0
 
bool fUseBufferedWrite = true
 
bool fUseDirectIO = false
 
EImplicitMT fUseImplicitMT = EImplicitMT::kDefault
 
std::size_t fWriteBufferSize = 4 * 1024 * 1024
 

#include <ROOT/RNTupleWriteOptions.hxx>

Inheritance diagram for ROOT::RNTupleWriteOptions:
[legend]

Member Enumeration Documentation

◆ EImplicitMT

enum class ROOT::RNTupleWriteOptions::EImplicitMT
strong
Enumerator
kOff 
kDefault 

Definition at line 181 of file RNTupleWriteOptions.hxx.

Constructor & Destructor Documentation

◆ ~RNTupleWriteOptions()

virtual ROOT::RNTupleWriteOptions::~RNTupleWriteOptions ( )
virtualdefault

Member Function Documentation

◆ Clone()

std::unique_ptr< ROOT::RNTupleWriteOptions > ROOT::RNTupleWriteOptions::Clone ( ) const
virtual

Reimplemented in ROOT::Experimental::RNTupleWriteOptionsDaos.

Definition at line 50 of file RNTupleWriteOptions.cxx.

◆ GetApproxZippedClusterSize()

std::size_t ROOT::RNTupleWriteOptions::GetApproxZippedClusterSize ( ) const
inline

Definition at line 221 of file RNTupleWriteOptions.hxx.

◆ GetCompression()

std::uint32_t ROOT::RNTupleWriteOptions::GetCompression ( ) const
inline

Definition at line 214 of file RNTupleWriteOptions.hxx.

◆ GetEnablePageChecksums()

bool ROOT::RNTupleWriteOptions::GetEnablePageChecksums ( ) const
inline

Definition at line 248 of file RNTupleWriteOptions.hxx.

◆ GetEnableSamePageMerging()

bool ROOT::RNTupleWriteOptions::GetEnableSamePageMerging ( ) const
inline

Definition at line 258 of file RNTupleWriteOptions.hxx.

◆ GetInitialUnzippedPageSize()

std::size_t ROOT::RNTupleWriteOptions::GetInitialUnzippedPageSize ( ) const
inline

Definition at line 227 of file RNTupleWriteOptions.hxx.

◆ GetMaxKeySize()

std::uint64_t ROOT::RNTupleWriteOptions::GetMaxKeySize ( ) const
inline

Definition at line 261 of file RNTupleWriteOptions.hxx.

◆ GetMaxUnzippedClusterSize()

std::size_t ROOT::RNTupleWriteOptions::GetMaxUnzippedClusterSize ( ) const
inline

Definition at line 224 of file RNTupleWriteOptions.hxx.

◆ GetMaxUnzippedPageSize()

std::size_t ROOT::RNTupleWriteOptions::GetMaxUnzippedPageSize ( ) const
inline

Definition at line 230 of file RNTupleWriteOptions.hxx.

◆ GetPageBufferBudget()

std::size_t ROOT::RNTupleWriteOptions::GetPageBufferBudget ( ) const

Definition at line 87 of file RNTupleWriteOptions.cxx.

◆ GetUseBufferedWrite()

bool ROOT::RNTupleWriteOptions::GetUseBufferedWrite ( ) const
inline

Definition at line 236 of file RNTupleWriteOptions.hxx.

◆ GetUseDirectIO()

bool ROOT::RNTupleWriteOptions::GetUseDirectIO ( ) const
inline

Definition at line 239 of file RNTupleWriteOptions.hxx.

◆ GetUseImplicitMT()

EImplicitMT ROOT::RNTupleWriteOptions::GetUseImplicitMT ( ) const
inline

Definition at line 245 of file RNTupleWriteOptions.hxx.

◆ GetWriteBufferSize()

std::size_t ROOT::RNTupleWriteOptions::GetWriteBufferSize ( ) const
inline

Definition at line 242 of file RNTupleWriteOptions.hxx.

◆ SetApproxZippedClusterSize()

void ROOT::RNTupleWriteOptions::SetApproxZippedClusterSize ( std::size_t val)

Definition at line 55 of file RNTupleWriteOptions.cxx.

◆ SetCompression() [1/2]

void ROOT::RNTupleWriteOptions::SetCompression ( RCompressionSetting::EAlgorithm::EValues algorithm,
int compressionLevel )
inline

Definition at line 216 of file RNTupleWriteOptions.hxx.

◆ SetCompression() [2/2]

void ROOT::RNTupleWriteOptions::SetCompression ( std::uint32_t val)
inline

Definition at line 215 of file RNTupleWriteOptions.hxx.

◆ SetEnablePageChecksums()

void ROOT::RNTupleWriteOptions::SetEnablePageChecksums ( bool val)
inline

Note that turning off page checksums will also turn off the same page merging optimization (see tuning.md)

Definition at line 250 of file RNTupleWriteOptions.hxx.

◆ SetEnableSamePageMerging()

void ROOT::RNTupleWriteOptions::SetEnableSamePageMerging ( bool val)

Definition at line 79 of file RNTupleWriteOptions.cxx.

◆ SetInitialUnzippedPageSize()

void ROOT::RNTupleWriteOptions::SetInitialUnzippedPageSize ( std::size_t val)

Definition at line 67 of file RNTupleWriteOptions.cxx.

◆ SetMaxUnzippedClusterSize()

void ROOT::RNTupleWriteOptions::SetMaxUnzippedClusterSize ( std::size_t val)

Definition at line 61 of file RNTupleWriteOptions.cxx.

◆ SetMaxUnzippedPageSize()

void ROOT::RNTupleWriteOptions::SetMaxUnzippedPageSize ( std::size_t val)

Definition at line 73 of file RNTupleWriteOptions.cxx.

◆ SetPageBufferBudget()

void ROOT::RNTupleWriteOptions::SetPageBufferBudget ( std::size_t val)
inline

Definition at line 234 of file RNTupleWriteOptions.hxx.

◆ SetUseBufferedWrite()

void ROOT::RNTupleWriteOptions::SetUseBufferedWrite ( bool val)
inline

Definition at line 237 of file RNTupleWriteOptions.hxx.

◆ SetUseDirectIO()

void ROOT::RNTupleWriteOptions::SetUseDirectIO ( bool val)
inline

Definition at line 240 of file RNTupleWriteOptions.hxx.

◆ SetUseImplicitMT()

void ROOT::RNTupleWriteOptions::SetUseImplicitMT ( EImplicitMT val)
inline

Definition at line 246 of file RNTupleWriteOptions.hxx.

◆ SetWriteBufferSize()

void ROOT::RNTupleWriteOptions::SetWriteBufferSize ( std::size_t val)
inline

Definition at line 243 of file RNTupleWriteOptions.hxx.

Member Data Documentation

◆ fApproxZippedClusterSize

std::size_t ROOT::RNTupleWriteOptions::fApproxZippedClusterSize = 128 * 1024 * 1024
protected

Definition at line 194 of file RNTupleWriteOptions.hxx.

◆ fCompression

std::uint32_t ROOT::RNTupleWriteOptions::fCompression {RCompressionSetting::EDefaults::kUseGeneralPurpose}
protected

Definition at line 193 of file RNTupleWriteOptions.hxx.

◆ fEnablePageChecksums

bool ROOT::RNTupleWriteOptions::fEnablePageChecksums = true
protected

Definition at line 203 of file RNTupleWriteOptions.hxx.

◆ fEnableSamePageMerging

bool ROOT::RNTupleWriteOptions::fEnableSamePageMerging = true
protected

Definition at line 204 of file RNTupleWriteOptions.hxx.

◆ fInitialUnzippedPageSize

std::size_t ROOT::RNTupleWriteOptions::fInitialUnzippedPageSize = 256
protected

Definition at line 196 of file RNTupleWriteOptions.hxx.

◆ fMaxKeySize

std::uint64_t ROOT::RNTupleWriteOptions::fMaxKeySize = kDefaultMaxKeySize
protected

Specifies the max size of a payload storeable into a single TKey.

When writing an RNTuple to a ROOT file, any payload whose size exceeds this will be split into multiple keys.

Definition at line 207 of file RNTupleWriteOptions.hxx.

◆ fMaxUnzippedClusterSize

std::size_t ROOT::RNTupleWriteOptions::fMaxUnzippedClusterSize = 10 * fApproxZippedClusterSize
protected

Definition at line 195 of file RNTupleWriteOptions.hxx.

◆ fMaxUnzippedPageSize

std::size_t ROOT::RNTupleWriteOptions::fMaxUnzippedPageSize = 1024 * 1024
protected

Definition at line 197 of file RNTupleWriteOptions.hxx.

◆ fPageBufferBudget

std::size_t ROOT::RNTupleWriteOptions::fPageBufferBudget = 0
protected

Definition at line 198 of file RNTupleWriteOptions.hxx.

◆ fUseBufferedWrite

bool ROOT::RNTupleWriteOptions::fUseBufferedWrite = true
protected

Definition at line 199 of file RNTupleWriteOptions.hxx.

◆ fUseDirectIO

bool ROOT::RNTupleWriteOptions::fUseDirectIO = false
protected

Definition at line 200 of file RNTupleWriteOptions.hxx.

◆ fUseImplicitMT

EImplicitMT ROOT::RNTupleWriteOptions::fUseImplicitMT = EImplicitMT::kDefault
protected

Definition at line 202 of file RNTupleWriteOptions.hxx.

◆ fWriteBufferSize

std::size_t ROOT::RNTupleWriteOptions::fWriteBufferSize = 4 * 1024 * 1024
protected

Definition at line 201 of file RNTupleWriteOptions.hxx.

◆ kDefaultMaxKeySize

constexpr std::uint64_t ROOT::RNTupleWriteOptions::kDefaultMaxKeySize = 0x4000'0000
staticconstexpr

Definition at line 187 of file RNTupleWriteOptions.hxx.

Libraries for ROOT::RNTupleWriteOptions:
[legend]
The documentation for this class was generated from the following files: