Logo ROOT  

Reference Guide
 
Loading...
Searching...
No Matches
List of all members | Public Member Functions | Public Attributes | List of all members
ROOT::Experimental::Internal::ML::RChunkConstructor Class Reference

The logic for constructing chunks from a dataset.

This struct handles the logic for splitting a dataset into smaller subsets known as chunks, which are constructed from blocks.

A chunk is the largest portion of the dataset loaded into memory at once, and each chunk is further divided into batches for machine learning training.

The dataset is split into disjoint chunks based on a user-defined chunk size. There are two types of chunks:

  • Full chunks: contain exactly the number of entries specified by the chunk size.
  • Leftover chunk: contains any remaining entries that don't make up a full chunk.

Each chunk is constructed from blocks based on a user-defined block size. There are two types of blocks:

  • Full blocks: contain exactly the number of entries specified by the block size.
  • Leftover block: contains any remaining entries that don't make up a full block.

The blocks are defined by their start and end entries, which correspond to positions within the dataset’s total number of entries.

Definition at line 48 of file RChunkConstructor.hxx.

Public Member Functions

 RChunkConstructor (const std::size_t numEntries, const std::size_t chunkSize, const std::size_t blockSize)
 
void CreateChunksIntervals ()
 Creates chunks from the dataset consisting of blocks with the begin and end entry.
 
void DistributeBlockIntervals ()
 Group the blocks based on the block type (full or leftover) based on the size of the block.
 
void SizeOfChunks ()
 Fills a vector with the size of every chunk from the dataset.
 

Public Attributes

std::vector< std::pair< Long_t, Long_t > > BlockIntervals
 
std::size_t BlockPerFullChunk
 
std::size_t BlockPerLeftoverChunk
 
std::size_t Chunks
 
std::vector< std::vector< std::pair< Long_t, Long_t > > > ChunksIntervals
 
std::vector< std::size_t > ChunksSizes
 
std::size_t fBlockSize {}
 
std::size_t fChunkSize {}
 
std::size_t fNumEntries {}
 
std::vector< std::pair< Long_t, Long_t > > FullBlockIntervalsInFullChunks
 
std::vector< std::pair< Long_t, Long_t > > FullBlockIntervalsInLeftoverChunks
 
std::size_t FullBlocksInFullChunks
 
std::size_t FullBlocksInLeftoverChunks
 
std::size_t FullBlocksPerFullChunk
 
std::size_t FullBlocksPerLeftoverChunk
 
std::size_t FullChunks
 
std::vector< std::pair< Long_t, Long_t > > LeftoverBlockIntervalsInFullChunks
 
std::vector< std::pair< Long_t, Long_t > > LeftoverBlockIntervalsInLeftoverChunks
 
std::size_t LeftoverBlocksInFullChunks
 
std::size_t LeftoverBlocksInLeftoverChunks
 
std::size_t LeftoverBlocksPerFullChunk
 
std::size_t LeftoverBlocksPerLeftoverChunk
 
std::size_t LeftoverChunks
 
std::size_t NumberOfBlocks
 
std::vector< std::size_t > NumberOfDifferentBlocks
 
std::vector< std::size_t > SizeOfBlocks
 
std::size_t SizeOfFullBlockInFullChunk
 
std::size_t SizeOfFullBlockInLeftoverChunk
 
std::size_t SizeOfFullChunk
 
std::size_t SizeOfLeftoverBlockInFullChunk
 
std::size_t SizeOfLeftoverBlockInLeftoverChunk
 
std::size_t SizeOfLeftoverChunk
 

#include <ROOT/ML/RChunkConstructor.hxx>

Constructor & Destructor Documentation

◆ RChunkConstructor()

ROOT::Experimental::Internal::ML::RChunkConstructor::RChunkConstructor ( const std::size_t numEntries,
const std::size_t chunkSize,
const std::size_t blockSize )
inline

Definition at line 108 of file RChunkConstructor.hxx.

Member Function Documentation

◆ CreateChunksIntervals()

void ROOT::Experimental::Internal::ML::RChunkConstructor::CreateChunksIntervals ( )
inline

Creates chunks from the dataset consisting of blocks with the begin and end entry.

Definition at line 180 of file RChunkConstructor.hxx.

◆ DistributeBlockIntervals()

void ROOT::Experimental::Internal::ML::RChunkConstructor::DistributeBlockIntervals ( )
inline

Group the blocks based on the block type (full or leftover) based on the size of the block.

Definition at line 158 of file RChunkConstructor.hxx.

◆ SizeOfChunks()

void ROOT::Experimental::Internal::ML::RChunkConstructor::SizeOfChunks ( )
inline

Fills a vector with the size of every chunk from the dataset.

Definition at line 219 of file RChunkConstructor.hxx.

Member Data Documentation

◆ BlockIntervals

std::vector<std::pair<Long_t, Long_t> > ROOT::Experimental::Internal::ML::RChunkConstructor::BlockIntervals

Definition at line 96 of file RChunkConstructor.hxx.

◆ BlockPerFullChunk

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::BlockPerFullChunk

Definition at line 71 of file RChunkConstructor.hxx.

◆ BlockPerLeftoverChunk

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::BlockPerLeftoverChunk

Definition at line 76 of file RChunkConstructor.hxx.

◆ Chunks

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::Chunks

Definition at line 66 of file RChunkConstructor.hxx.

◆ ChunksIntervals

std::vector<std::vector<std::pair<Long_t, Long_t> > > ROOT::Experimental::Internal::ML::RChunkConstructor::ChunksIntervals

Definition at line 104 of file RChunkConstructor.hxx.

◆ ChunksSizes

std::vector<std::size_t> ROOT::Experimental::Internal::ML::RChunkConstructor::ChunksSizes

Definition at line 106 of file RChunkConstructor.hxx.

◆ fBlockSize

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::fBlockSize {}

Definition at line 51 of file RChunkConstructor.hxx.

◆ fChunkSize

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::fChunkSize {}

Definition at line 50 of file RChunkConstructor.hxx.

◆ fNumEntries

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::fNumEntries {}

Definition at line 49 of file RChunkConstructor.hxx.

◆ FullBlockIntervalsInFullChunks

std::vector<std::pair<Long_t, Long_t> > ROOT::Experimental::Internal::ML::RChunkConstructor::FullBlockIntervalsInFullChunks

Definition at line 98 of file RChunkConstructor.hxx.

◆ FullBlockIntervalsInLeftoverChunks

std::vector<std::pair<Long_t, Long_t> > ROOT::Experimental::Internal::ML::RChunkConstructor::FullBlockIntervalsInLeftoverChunks

Definition at line 101 of file RChunkConstructor.hxx.

◆ FullBlocksInFullChunks

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::FullBlocksInFullChunks

Definition at line 79 of file RChunkConstructor.hxx.

◆ FullBlocksInLeftoverChunks

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::FullBlocksInLeftoverChunks

Definition at line 83 of file RChunkConstructor.hxx.

◆ FullBlocksPerFullChunk

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::FullBlocksPerFullChunk

Definition at line 69 of file RChunkConstructor.hxx.

◆ FullBlocksPerLeftoverChunk

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::FullBlocksPerLeftoverChunk

Definition at line 74 of file RChunkConstructor.hxx.

◆ FullChunks

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::FullChunks

Definition at line 64 of file RChunkConstructor.hxx.

◆ LeftoverBlockIntervalsInFullChunks

std::vector<std::pair<Long_t, Long_t> > ROOT::Experimental::Internal::ML::RChunkConstructor::LeftoverBlockIntervalsInFullChunks

Definition at line 99 of file RChunkConstructor.hxx.

◆ LeftoverBlockIntervalsInLeftoverChunks

std::vector<std::pair<Long_t, Long_t> > ROOT::Experimental::Internal::ML::RChunkConstructor::LeftoverBlockIntervalsInLeftoverChunks

Definition at line 102 of file RChunkConstructor.hxx.

◆ LeftoverBlocksInFullChunks

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::LeftoverBlocksInFullChunks

Definition at line 80 of file RChunkConstructor.hxx.

◆ LeftoverBlocksInLeftoverChunks

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::LeftoverBlocksInLeftoverChunks

Definition at line 84 of file RChunkConstructor.hxx.

◆ LeftoverBlocksPerFullChunk

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::LeftoverBlocksPerFullChunk

Definition at line 70 of file RChunkConstructor.hxx.

◆ LeftoverBlocksPerLeftoverChunk

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::LeftoverBlocksPerLeftoverChunk

Definition at line 75 of file RChunkConstructor.hxx.

◆ LeftoverChunks

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::LeftoverChunks

Definition at line 65 of file RChunkConstructor.hxx.

◆ NumberOfBlocks

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::NumberOfBlocks

Definition at line 93 of file RChunkConstructor.hxx.

◆ NumberOfDifferentBlocks

std::vector<std::size_t> ROOT::Experimental::Internal::ML::RChunkConstructor::NumberOfDifferentBlocks

Definition at line 90 of file RChunkConstructor.hxx.

◆ SizeOfBlocks

std::vector<std::size_t> ROOT::Experimental::Internal::ML::RChunkConstructor::SizeOfBlocks

Definition at line 87 of file RChunkConstructor.hxx.

◆ SizeOfFullBlockInFullChunk

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::SizeOfFullBlockInFullChunk

Definition at line 58 of file RChunkConstructor.hxx.

◆ SizeOfFullBlockInLeftoverChunk

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::SizeOfFullBlockInLeftoverChunk

Definition at line 60 of file RChunkConstructor.hxx.

◆ SizeOfFullChunk

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::SizeOfFullChunk

Definition at line 54 of file RChunkConstructor.hxx.

◆ SizeOfLeftoverBlockInFullChunk

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::SizeOfLeftoverBlockInFullChunk

Definition at line 59 of file RChunkConstructor.hxx.

◆ SizeOfLeftoverBlockInLeftoverChunk

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::SizeOfLeftoverBlockInLeftoverChunk

Definition at line 61 of file RChunkConstructor.hxx.

◆ SizeOfLeftoverChunk

std::size_t ROOT::Experimental::Internal::ML::RChunkConstructor::SizeOfLeftoverChunk

Definition at line 55 of file RChunkConstructor.hxx.

Collaboration diagram for ROOT::Experimental::Internal::ML::RChunkConstructor:
[legend]
The documentation for this class was generated from the following file: