ROOT   Reference Guide
ROOT::Experimental::Detail::RClusterPool Class Reference

Managed a set of clusters containing compressed and packed pages.

The unzipping step of the pipeline therefore behaves differently depending on whether or not implicit multi-threadin is turned on. If it is turned off, i.e. in a single-threaded environment, the cluster pool will only read the compressed pages and the page source has to uncompresses pages at a later point when data from the page is requested.

Definition at line 55 of file RClusterPool.hxx.

## Classes

struct  RInFlightCluster
Clusters that are currently being processed by the pipeline. More...

Request to load a subset of the columns of a particular cluster. More...

struct  RUnzipItem
Request to decompress and if necessary unpack compressed pages. More...

## Public Member Functions

RClusterPool (const RClusterPool &other)=delete

RClusterPool (RPageSource &pageSource)

RClusterPool (RPageSource &pageSource, unsigned int clusterBunchSize)

~RClusterPool ()

RClusterGetCluster (DescriptorId_t clusterId, const RCluster::ColumnSet_t &columns)
Returns the requested cluster either from the pool or, in case of a cache miss, lets the I/O thread load the cluster in the pool, blocks until done, and then returns it. More...

RClusterPooloperator= (const RClusterPool &other)=delete

void WaitForInFlightClusters ()
Used by the unit tests to drain the queue of clusters to be preloaded. More...

## Static Public Attributes

static constexpr unsigned int kDefaultClusterBunchSize = 1

## Private Member Functions

The I/O thread routine, there is exactly one I/O thread in-flight for every cluster pool. More...

void ExecUnzipClusters ()
The unzip thread routine which takes a loaded cluster and passes it to fPageSource.UnzipCluster (which might be a no-op if IMT is off). More...

size_t FindFreeSlot () const
Returns an index of an unused element in fPool; callers of this function (GetCluster() and WaitFor()) make sure that a free slot actually exists. More...

RClusterFindInPool (DescriptorId_t clusterId) const
Every cluster id has at most one corresponding RCluster pointer in the pool. More...

RClusterWaitFor (DescriptorId_t clusterId, const RCluster::ColumnSet_t &columns)
Returns the given cluster from the pool, which needs to contain at least the columns columns. More...

## Private Attributes

std::int64_t fBunchId = 0
Used as an ever-growing counter in GetCluster() to separate bunches of clusters from each other. More...

unsigned int fClusterBunchSize
The number of clusters that are being read in a single vector read. More...

Signals a non-empty I/O work queue. More...

std::condition_variable fCvHasUnzipWork
Signals non-empty unzip work queue. More...

std::vector< RInFlightClusterfInFlightClusters
The clusters that were handed off to the I/O thread. More...

std::mutex fLockUnzipQueue
The lock associated with the fCvHasUnzipWork conditional variable. More...

std::mutex fLockWorkQueue
Protects the shared state between the main thread and the pipeline threads, namely the read and unzip work queues and the in-flight clusters vector. More...

RPageSourcefPageSource
Every cluster pool is responsible for exactly one page source that triggers loading of the clusters (GetCluster()) and is used for implementing the I/O and cluster memory allocation (PageSource::LoadClusters()). More...

std::vector< std::unique_ptr< RCluster > > fPool
The cache of clusters around the currently active cluster. More...

The communication channel to the I/O thread. More...

The unzip thread takes a loaded cluster and passes it to fPageSource->UnzipCluster() on it. More...

std::queue< RUnzipItemfUnzipQueue
The communication channel between the I/O thread and the unzip thread. More...

unsigned int fWindowPre = 0
The number of clusters before the currently active cluster that should stay in the pool if present Reserved for later use. More...

#include <ROOT/RClusterPool.hxx>

## ◆ RClusterPool() [1/3]

 ROOT::Experimental::Detail::RClusterPool::RClusterPool ( RPageSource & pageSource, unsigned int clusterBunchSize )

Definition at line 52 of file RClusterPool.cxx.

## ◆ RClusterPool() [2/3]

 ROOT::Experimental::Detail::RClusterPool::RClusterPool ( RPageSource & pageSource )
inlineexplicit

Definition at line 146 of file RClusterPool.hxx.

## ◆ RClusterPool() [3/3]

 ROOT::Experimental::Detail::RClusterPool::RClusterPool ( const RClusterPool & other )
delete

## ◆ ~RClusterPool()

 ROOT::Experimental::Detail::RClusterPool::~RClusterPool ( )

Definition at line 62 of file RClusterPool.cxx.

## Member Function Documentation

private

The I/O thread routine, there is exactly one I/O thread in-flight for every cluster pool.

Definition at line 106 of file RClusterPool.cxx.

## ◆ ExecUnzipClusters()

 void ROOT::Experimental::Detail::RClusterPool::ExecUnzipClusters ( )
private

The unzip thread routine which takes a loaded cluster and passes it to fPageSource.UnzipCluster (which might be a no-op if IMT is off).

Marks the cluster as ready to be picked up by the main thread.

Definition at line 81 of file RClusterPool.cxx.

## ◆ FindFreeSlot()

 size_t ROOT::Experimental::Detail::RClusterPool::FindFreeSlot ( ) const
private

Returns an index of an unused element in fPool; callers of this function (GetCluster() and WaitFor()) make sure that a free slot actually exists.

Definition at line 168 of file RClusterPool.cxx.

## ◆ FindInPool()

 ROOT::Experimental::Detail::RCluster * ROOT::Experimental::Detail::RClusterPool::FindInPool ( DescriptorId_t clusterId ) const
private

Every cluster id has at most one corresponding RCluster pointer in the pool.

Definition at line 159 of file RClusterPool.cxx.

## ◆ GetCluster()

 ROOT::Experimental::Detail::RCluster * ROOT::Experimental::Detail::RClusterPool::GetCluster ( DescriptorId_t clusterId, const RCluster::ColumnSet_t & columns )

Returns the requested cluster either from the pool or, in case of a cache miss, lets the I/O thread load the cluster in the pool, blocks until done, and then returns it.

Triggers along the way the background loading of the following fWindowPost number of clusters. The returned cluster has at least all the pages of columns and possibly pages of other columns, too. If implicit multi-threading is turned on, the uncompressed pages of the returned cluster are already pushed into the page pool associated with the page source upon return. The cluster remains valid until the next call to GetCluster().

Definition at line 235 of file RClusterPool.cxx.

## ◆ operator=()

 RClusterPool & ROOT::Experimental::Detail::RClusterPool::operator= ( const RClusterPool & other )
delete

## ◆ WaitFor()

 ROOT::Experimental::Detail::RCluster * ROOT::Experimental::Detail::RClusterPool::WaitFor ( DescriptorId_t clusterId, const RCluster::ColumnSet_t & columns )
private

Returns the given cluster from the pool, which needs to contain at least the columns columns.

Executed at the end of GetCluster when all missing data pieces have been sent to the load queue. Ideally, the function returns without blocking if the cluster is already in the pool.

Definition at line 375 of file RClusterPool.cxx.

## ◆ WaitForInFlightClusters()

 void ROOT::Experimental::Detail::RClusterPool::WaitForInFlightClusters ( )

Used by the unit tests to drain the queue of clusters to be preloaded.

Definition at line 424 of file RClusterPool.cxx.

## ◆ fBunchId

 std::int64_t ROOT::Experimental::Detail::RClusterPool::fBunchId = 0
private

Used as an ever-growing counter in GetCluster() to separate bunches of clusters from each other.

Definition at line 99 of file RClusterPool.hxx.

## ◆ fClusterBunchSize

 unsigned int ROOT::Experimental::Detail::RClusterPool::fClusterBunchSize
private

The number of clusters that are being read in a single vector read.

Definition at line 97 of file RClusterPool.hxx.

private

Signals a non-empty I/O work queue.

Definition at line 109 of file RClusterPool.hxx.

## ◆ fCvHasUnzipWork

 std::condition_variable ROOT::Experimental::Detail::RClusterPool::fCvHasUnzipWork
private

Signals non-empty unzip work queue.

Definition at line 115 of file RClusterPool.hxx.

## ◆ fInFlightClusters

 std::vector ROOT::Experimental::Detail::RClusterPool::fInFlightClusters
private

The clusters that were handed off to the I/O thread.

Definition at line 107 of file RClusterPool.hxx.

## ◆ fLockUnzipQueue

 std::mutex ROOT::Experimental::Detail::RClusterPool::fLockUnzipQueue
private

The lock associated with the fCvHasUnzipWork conditional variable.

Definition at line 113 of file RClusterPool.hxx.

## ◆ fLockWorkQueue

 std::mutex ROOT::Experimental::Detail::RClusterPool::fLockWorkQueue
private

Protects the shared state between the main thread and the pipeline threads, namely the read and unzip work queues and the in-flight clusters vector.

Definition at line 105 of file RClusterPool.hxx.

## ◆ fPageSource

 RPageSource& ROOT::Experimental::Detail::RClusterPool::fPageSource
private

Every cluster pool is responsible for exactly one page source that triggers loading of the clusters (GetCluster()) and is used for implementing the I/O and cluster memory allocation (PageSource::LoadClusters()).

Definition at line 92 of file RClusterPool.hxx.

## ◆ fPool

 std::vector > ROOT::Experimental::Detail::RClusterPool::fPool
private

The cache of clusters around the currently active cluster.

Definition at line 101 of file RClusterPool.hxx.

private

The communication channel to the I/O thread.

Definition at line 111 of file RClusterPool.hxx.

private

The thread is mostly waiting for the data to arrive (blocked by the kernel) and therefore can safely run in addition to the application main threads.

Definition at line 122 of file RClusterPool.hxx.

private

The unzip thread takes a loaded cluster and passes it to fPageSource->UnzipCluster() on it.

If implicit multi-threading is turned off, the UnzipCluster() call is a no-op. Otherwise, the UnzipCluster() call schedules the unzipping of pages using the application's task scheduler.

Definition at line 126 of file RClusterPool.hxx.

## ◆ fUnzipQueue

 std::queue ROOT::Experimental::Detail::RClusterPool::fUnzipQueue
private

Definition at line 117 of file RClusterPool.hxx.

## ◆ fWindowPre

 unsigned int ROOT::Experimental::Detail::RClusterPool::fWindowPre = 0
private

The number of clusters before the currently active cluster that should stay in the pool if present Reserved for later use.

Definition at line 95 of file RClusterPool.hxx.

## ◆ kDefaultClusterBunchSize

 constexpr unsigned int ROOT::Experimental::Detail::RClusterPool::kDefaultClusterBunchSize = 1
staticconstexpr

Definition at line 144 of file RClusterPool.hxx.

Libraries for ROOT::Experimental::Detail::RClusterPool:
[legend]

The documentation for this class was generated from the following files: