Common user-tunable settings for storing RNTuples.

All page sink classes need to support the common options.

Option name	Type	Default	Description
Compression	std::uint32_t	RCompressionSetting::EDefaults::kUseGeneralPurpose	The compression settings for this RNTuple
ApproxZippedClusterSize	std::size_t	128 MiB	Approximation of the target compressed cluster size
MaxUnzippedClusterSize	std::size_t	1280 MiB	Memory limit for committing a cluster: with very high compression ratio, we need a limit on how large the I/O buffer can grow during writing.
InitialUnzippedPageSize	std::size_t	256	Initially, columns start with a page of this size. The default value is chosen to accommodate at least 32 elements of 64 bits, or 64 elements of 32 bits. If more elements are needed, pages are increased up until the byte limit given by the option MaxUnzippedPageSize or until the total page buffer limit is reached (as a sum of all page buffers). The total write buffer limit needs to be large enough to hold the initial pages of all columns.
MaxUnzippedPageSize	std::size_t	1 MiB	Pages can grow only to the given limit in bytes.
PageBufferBudget	std::size_t	0 / auto	The maximum size that the sum of all page buffers used for writing into a persistent sink are allowed to use. If set to zero, RNTuple will auto-adjust the budget based on the value of ApproxZippedClusterSize. If set manually, the size needs to be large enough to hold all initial page buffers. The total amount of memory for writing is larger, e.g. for the additional compressed buffers etc. Use RNTupleModel::EstimateWriteMemoryUsage() for the total estimated memory use for writing. The default values are tuned for a total write memory of around 400 MiB per fill context.
UseBufferedWrite	bool	true	Whether to use buffered writing (with RPageSinkBuf). This buffers compressed pages in memory, reorders them to keep pages of the same column adjacent, and coalesces the writes when committing a cluster.
UseDirectIO	bool	false	Whether to use Direct I/O for writing. Note that this introduces alignment requirements that may very between filesystems and platforms.
WriteBufferSize	std::size_t	4 MiB	Buffer size to use for writing to files, must be a multiple of 4096 bytes. Testing suggests that 4MiB gives best performance (with Direct I/O) at a reasonable memory consumption.
UseImplicitMT	EImplicitMT	EImplicitMT::kDefault	Whether to use implicit multi-threading to compress pages. Only has an effect if buffered writing is turned on. The meaning of EImplicitMT::kDefault depends on the used writer: For the (sequential) RNTupleWriter, it translates to EImplicitMT::kOn and the user has to manually disable the use of implicit multi-threading if it is not wanted. For the RNTupleParalellWriter, the implementation defaults to EImplicitMT::kOff in order to avoid interference with explicit parallelism that might create one RNTupleFillContext per thread. If implicit multi-threading is wanted on top of this, the user has to explicitly request EImplicitMT::kOn.
EnablePageChecksums	bool	true	If set, checksums will be calculated and written for every page. If turned off, will also turn off EnableSamePageMerging.
EnableSamePageMerging	bool	true	If set, identical pages are deduplicated and aliased on disk. Requires EnablePageChecksums and will throw if previously disabled.

Definition at line 181 of file RNTupleWriteOptions.hxx.

Public Types
enum class	EImplicitMT { kOff , kOn , kDefault }

Public Member Functions
virtual	~RNTupleWriteOptions ()=default
virtual std::unique_ptr< RNTupleWriteOptions >	Clone () const
std::size_t	GetApproxZippedClusterSize () const
std::uint32_t	GetCompression () const
bool	GetEnablePageChecksums () const
bool	GetEnableSamePageMerging () const
std::size_t	GetInitialUnzippedPageSize () const
std::uint64_t	GetMaxKeySize () const
std::size_t	GetMaxUnzippedClusterSize () const
std::size_t	GetMaxUnzippedPageSize () const
std::size_t	GetPageBufferBudget () const
bool	GetUseBufferedWrite () const
bool	GetUseDirectIO () const
EImplicitMT	GetUseImplicitMT () const
std::size_t	GetWriteBufferSize () const
void	SetApproxZippedClusterSize (std::size_t val)
void	SetCompression (RCompressionSetting::EAlgorithm::EValues algorithm, int compressionLevel)
void	SetCompression (std::uint32_t val)
void	SetEnablePageChecksums (bool val)
	Note that turning off page checksums will also turn off the same page merging optimization (see tuning.md).
void	SetEnableSamePageMerging (bool val)
void	SetInitialUnzippedPageSize (std::size_t val)
void	SetMaxUnzippedClusterSize (std::size_t val)
void	SetMaxUnzippedPageSize (std::size_t val)
void	SetPageBufferBudget (std::size_t val)
void	SetUseBufferedWrite (bool val)
void	SetUseDirectIO (bool val)
void	SetUseImplicitMT (EImplicitMT val)
void	SetWriteBufferSize (std::size_t val)

Static Public Attributes
static constexpr std::uint64_t	kDefaultMaxKeySize = 0x4000'0000

Protected Attributes
std::size_t	fApproxZippedClusterSize = 128 * 1024 * 1024
std::uint32_t	fCompression {RCompressionSetting::EDefaults::kUseGeneralPurpose}
bool	fEnablePageChecksums = true
bool	fEnableSamePageMerging = true
std::size_t	fInitialUnzippedPageSize = 256
std::uint64_t	fMaxKeySize = kDefaultMaxKeySize
	Specifies the max size of a payload storeable into a single TKey.
std::size_t	fMaxUnzippedClusterSize = 10 * fApproxZippedClusterSize
std::size_t	fMaxUnzippedPageSize = 1024 * 1024
std::size_t	fPageBufferBudget = 0
bool	fUseBufferedWrite = true
bool	fUseDirectIO = false
EImplicitMT	fUseImplicitMT = EImplicitMT::kDefault
std::size_t	fWriteBufferSize = 4 * 1024 * 1024

Friends
bool	operator!= (const RNTupleWriteOptions &lhs, const RNTupleWriteOptions &rhs)
bool	operator== (const RNTupleWriteOptions &lhs, const RNTupleWriteOptions &rhs)

#include <ROOT/RNTupleWriteOptions.hxx>

Inheritance diagram for ROOT::RNTupleWriteOptions:

Member Enumeration Documentation

◆ EImplicitMT

enum class ROOT::RNTupleWriteOptions::EImplicitMT

strong

Enumerator
kOff
kOn
kDefault

Definition at line 183 of file RNTupleWriteOptions.hxx.

Constructor & Destructor Documentation

◆ ~RNTupleWriteOptions()

virtual ROOT::RNTupleWriteOptions::~RNTupleWriteOptions ( )

virtualdefault

Member Function Documentation

◆ Clone()

std::unique_ptr< ROOT::RNTupleWriteOptions > ROOT::RNTupleWriteOptions::Clone ( ) const

virtual

Reimplemented in ROOT::Experimental::RNTupleWriteOptionsDaos.

Definition at line 47 of file RNTupleWriteOptions.cxx.

◆ GetApproxZippedClusterSize()

std::size_t ROOT::RNTupleWriteOptions::GetApproxZippedClusterSize ( ) const

inline

Definition at line 224 of file RNTupleWriteOptions.hxx.

◆ GetCompression()

std::uint32_t ROOT::RNTupleWriteOptions::GetCompression ( ) const

inline

Definition at line 217 of file RNTupleWriteOptions.hxx.

◆ GetEnablePageChecksums()

bool ROOT::RNTupleWriteOptions::GetEnablePageChecksums ( ) const

inline

Definition at line 251 of file RNTupleWriteOptions.hxx.

◆ GetEnableSamePageMerging()

bool ROOT::RNTupleWriteOptions::GetEnableSamePageMerging ( ) const

inline

Definition at line 261 of file RNTupleWriteOptions.hxx.

◆ GetInitialUnzippedPageSize()

std::size_t ROOT::RNTupleWriteOptions::GetInitialUnzippedPageSize ( ) const

inline

Definition at line 230 of file RNTupleWriteOptions.hxx.

◆ GetMaxKeySize()

std::uint64_t ROOT::RNTupleWriteOptions::GetMaxKeySize ( ) const

inline

Definition at line 264 of file RNTupleWriteOptions.hxx.

◆ GetMaxUnzippedClusterSize()

std::size_t ROOT::RNTupleWriteOptions::GetMaxUnzippedClusterSize ( ) const

inline

Definition at line 227 of file RNTupleWriteOptions.hxx.

◆ GetMaxUnzippedPageSize()

std::size_t ROOT::RNTupleWriteOptions::GetMaxUnzippedPageSize ( ) const

inline

Definition at line 233 of file RNTupleWriteOptions.hxx.

◆ GetPageBufferBudget()

std::size_t ROOT::RNTupleWriteOptions::GetPageBufferBudget ( ) const

Definition at line 84 of file RNTupleWriteOptions.cxx.

◆ GetUseBufferedWrite()

bool ROOT::RNTupleWriteOptions::GetUseBufferedWrite ( ) const

inline

Definition at line 239 of file RNTupleWriteOptions.hxx.

◆ GetUseDirectIO()

bool ROOT::RNTupleWriteOptions::GetUseDirectIO ( ) const

inline

Definition at line 242 of file RNTupleWriteOptions.hxx.

◆ GetUseImplicitMT()

EImplicitMT ROOT::RNTupleWriteOptions::GetUseImplicitMT ( ) const

inline

Definition at line 248 of file RNTupleWriteOptions.hxx.

◆ GetWriteBufferSize()

std::size_t ROOT::RNTupleWriteOptions::GetWriteBufferSize ( ) const

inline

Definition at line 245 of file RNTupleWriteOptions.hxx.

◆ SetApproxZippedClusterSize()

void ROOT::RNTupleWriteOptions::SetApproxZippedClusterSize ( std::size_t val )

Definition at line 52 of file RNTupleWriteOptions.cxx.

◆ SetCompression() [1/2]

void ROOT::RNTupleWriteOptions::SetCompression	(	RCompressionSetting::EAlgorithm::EValues	algorithm,
		int	compressionLevel )

inline

Definition at line 219 of file RNTupleWriteOptions.hxx.

◆ SetCompression() [2/2]

void ROOT::RNTupleWriteOptions::SetCompression ( std::uint32_t val )

inline

Definition at line 218 of file RNTupleWriteOptions.hxx.

◆ SetEnablePageChecksums()

void ROOT::RNTupleWriteOptions::SetEnablePageChecksums ( bool val )

inline

Note that turning off page checksums will also turn off the same page merging optimization (see tuning.md).

Definition at line 253 of file RNTupleWriteOptions.hxx.

◆ SetEnableSamePageMerging()

void ROOT::RNTupleWriteOptions::SetEnableSamePageMerging ( bool val )

Definition at line 76 of file RNTupleWriteOptions.cxx.

◆ SetInitialUnzippedPageSize()

void ROOT::RNTupleWriteOptions::SetInitialUnzippedPageSize ( std::size_t val )

Definition at line 64 of file RNTupleWriteOptions.cxx.

◆ SetMaxUnzippedClusterSize()

void ROOT::RNTupleWriteOptions::SetMaxUnzippedClusterSize ( std::size_t val )

Definition at line 58 of file RNTupleWriteOptions.cxx.

◆ SetMaxUnzippedPageSize()

void ROOT::RNTupleWriteOptions::SetMaxUnzippedPageSize ( std::size_t val )

Definition at line 70 of file RNTupleWriteOptions.cxx.

◆ SetPageBufferBudget()

void ROOT::RNTupleWriteOptions::SetPageBufferBudget ( std::size_t val )

inline

Definition at line 237 of file RNTupleWriteOptions.hxx.

◆ SetUseBufferedWrite()

void ROOT::RNTupleWriteOptions::SetUseBufferedWrite ( bool val )

inline

Definition at line 240 of file RNTupleWriteOptions.hxx.

◆ SetUseDirectIO()

void ROOT::RNTupleWriteOptions::SetUseDirectIO ( bool val )

inline

Definition at line 243 of file RNTupleWriteOptions.hxx.

◆ SetUseImplicitMT()

void ROOT::RNTupleWriteOptions::SetUseImplicitMT ( EImplicitMT val )

inline

Definition at line 249 of file RNTupleWriteOptions.hxx.

◆ SetWriteBufferSize()

void ROOT::RNTupleWriteOptions::SetWriteBufferSize ( std::size_t val )

inline

Definition at line 246 of file RNTupleWriteOptions.hxx.

◆ operator!=

bool operator!=	(	const RNTupleWriteOptions &	lhs,
		const RNTupleWriteOptions &	rhs )

friend

Definition at line 278 of file RNTupleWriteOptions.hxx.

◆ operator==

bool operator==	(	const RNTupleWriteOptions &	lhs,
		const RNTupleWriteOptions &	rhs )

friend

Definition at line 266 of file RNTupleWriteOptions.hxx.

Member Data Documentation

◆ fApproxZippedClusterSize

std::size_t ROOT::RNTupleWriteOptions::fApproxZippedClusterSize = 128 * 1024 * 1024

protected

Definition at line 197 of file RNTupleWriteOptions.hxx.

◆ fCompression

std::uint32_t ROOT::RNTupleWriteOptions::fCompression {RCompressionSetting::EDefaults::kUseGeneralPurpose}

protected

Definition at line 196 of file RNTupleWriteOptions.hxx.

◆ fEnablePageChecksums

bool ROOT::RNTupleWriteOptions::fEnablePageChecksums = true

protected

Definition at line 206 of file RNTupleWriteOptions.hxx.

◆ fEnableSamePageMerging

bool ROOT::RNTupleWriteOptions::fEnableSamePageMerging = true

protected

Definition at line 207 of file RNTupleWriteOptions.hxx.

◆ fInitialUnzippedPageSize

std::size_t ROOT::RNTupleWriteOptions::fInitialUnzippedPageSize = 256

protected

Definition at line 199 of file RNTupleWriteOptions.hxx.

◆ fMaxKeySize

std::uint64_t ROOT::RNTupleWriteOptions::fMaxKeySize = kDefaultMaxKeySize

protected

Specifies the max size of a payload storeable into a single TKey.

When writing an RNTuple to a ROOT file, any payload whose size exceeds this will be split into multiple keys.

Definition at line 210 of file RNTupleWriteOptions.hxx.

◆ fMaxUnzippedClusterSize

std::size_t ROOT::RNTupleWriteOptions::fMaxUnzippedClusterSize = 10 * fApproxZippedClusterSize

protected

Definition at line 198 of file RNTupleWriteOptions.hxx.

◆ fMaxUnzippedPageSize

std::size_t ROOT::RNTupleWriteOptions::fMaxUnzippedPageSize = 1024 * 1024

protected

Definition at line 200 of file RNTupleWriteOptions.hxx.

◆ fPageBufferBudget

std::size_t ROOT::RNTupleWriteOptions::fPageBufferBudget = 0

protected

Definition at line 201 of file RNTupleWriteOptions.hxx.

◆ fUseBufferedWrite

bool ROOT::RNTupleWriteOptions::fUseBufferedWrite = true

protected

Definition at line 202 of file RNTupleWriteOptions.hxx.

◆ fUseDirectIO

bool ROOT::RNTupleWriteOptions::fUseDirectIO = false

protected

Definition at line 203 of file RNTupleWriteOptions.hxx.

◆ fUseImplicitMT

EImplicitMT ROOT::RNTupleWriteOptions::fUseImplicitMT = EImplicitMT::kDefault

protected

Definition at line 205 of file RNTupleWriteOptions.hxx.

◆ fWriteBufferSize

std::size_t ROOT::RNTupleWriteOptions::fWriteBufferSize = 4 * 1024 * 1024

protected

Definition at line 204 of file RNTupleWriteOptions.hxx.

◆ kDefaultMaxKeySize

std::uint64_t ROOT::RNTupleWriteOptions::kDefaultMaxKeySize = 0x4000'0000

staticconstexpr

Definition at line 190 of file RNTupleWriteOptions.hxx.

The documentation for this class was generated from the following files:

tree/ntuple/inc/ROOT/RNTupleWriteOptions.hxx
tree/ntuple/src/RNTupleWriteOptions.cxx

Public Types

Public Member Functions

Static Public Attributes

Protected Attributes

Friends

Member Enumeration Documentation

◆ EImplicitMT

Constructor & Destructor Documentation

◆ ~RNTupleWriteOptions()

Member Function Documentation

◆ Clone()

◆ GetApproxZippedClusterSize()

◆ GetCompression()

◆ GetEnablePageChecksums()

◆ GetEnableSamePageMerging()

◆ GetInitialUnzippedPageSize()

◆ GetMaxKeySize()

◆ GetMaxUnzippedClusterSize()

◆ GetMaxUnzippedPageSize()

◆ GetPageBufferBudget()

◆ GetUseBufferedWrite()

◆ GetUseDirectIO()

◆ GetUseImplicitMT()

◆ GetWriteBufferSize()

◆ SetApproxZippedClusterSize()

◆ SetCompression() [1/2]

◆ SetCompression() [2/2]

◆ SetEnablePageChecksums()

◆ SetEnableSamePageMerging()

◆ SetInitialUnzippedPageSize()

◆ SetMaxUnzippedClusterSize()

◆ SetMaxUnzippedPageSize()

◆ SetPageBufferBudget()

◆ SetUseBufferedWrite()

◆ SetUseDirectIO()

◆ SetUseImplicitMT()

◆ SetWriteBufferSize()

Friends And Related Symbol Documentation

◆ operator!=

◆ operator==

Member Data Documentation

◆ fApproxZippedClusterSize

◆ fCompression

◆ fEnablePageChecksums

◆ fEnableSamePageMerging

◆ fInitialUnzippedPageSize

◆ fMaxKeySize

◆ fMaxUnzippedClusterSize

◆ fMaxUnzippedPageSize

◆ fPageBufferBudget

◆ fUseBufferedWrite

◆ fUseDirectIO

◆ fUseImplicitMT

◆ fWriteBufferSize

◆ kDefaultMaxKeySize