A cache to speed-up the reading of ROOT datasets

Motivation: why having a cache is needed?

When writing a TTree, the branch buffers are kept in memory. A typical branch buffersize (before compression) is typically 32 KBytes. After compression, the zipped buffer may be just a few Kbytes. The branch buffers cannot be much larger in case of TTrees with several hundred or thousand branches.

When writing, this does not generate a performance problem because branch buffers are always written sequentially and, thanks to OS optimisations, content is flushed to the output file when a few MBytes of data are available. On the other hand, when reading, one may hit performance problems because of latencies e.g imposed by network. For example in a WAN with 10ms latency, reading 1000 buffers of 10 KBytes each with no cache will imply 10s penalty where a local read of the 10 MBytes would take about 1 second.

The TreeCache tries to prefetch all the buffers for the selected branches in order to transfer a few multi-Megabytes large buffers instead of many multi-kilobytes small buffers. In addition, TTreeCache can sort the blocks to be read in increasing order such that the file is read sequentially.

Systems like xrootd, dCache or httpd take advantage of the TTreeCache in reading ahead as much data as they can and return to the application the maximum data specified in the cache and have the next chunk of data ready when the next request comes.

Are there cases for which the usage of TTreeCache is detrimental for performance?

Yes, some corner cases. For example, when reading only a small fraction of all entries such that not all branch buffers are read.

General Description

This class acts as a file cache, registering automatically the baskets from the branches being processed via direct manipulation of TTrees or with tools such as TTree::Draw, TTree::Process, TSelector, TTreeReader and RDataFrame when in the learning phase. The learning phase is by default 100 entries. It can be changed via TTreeCache::SetLearnEntries.

The usage of a TTreeCache can considerably improve the runtime performance at the price of a modest investment in memory, in particular when the TTree is accessed remotely, e.g. via a high latency network.

For each TTree being processed a TTreeCache object is created. This object is automatically deleted when the Tree is deleted or when the file is deleted. The user can change the size of the cache with the TTree::SetCacheSize method (by default the size is 30 Megabytes). This feature can be controlled with the environment variable ROOT_TTREECACHE_SIZE or the TTreeCache.Size option. The entry range for which the cache is active can also be set with the SetEntryRange method.

Changes of behavior when using TChain and TEventList

The usage of TChain or TEventList have influence on the behaviour of the cache:

Special case of a TChain Once the training is done on the first Tree, the list of branches in the cache is kept for the following files.
Special case of a TEventlist if the Tree or TChain has a TEventlist, only the buffers referenced by the list are put in the cache.

The learning phase is started or restarted when:

TTree automatically creates a cache.
TTree::SetCacheSize is called with a non-zero size and a cache did not previously exist
TTreeCache::StartLearningPhase is called.
TTreeCache::SetEntryRange is called and the learning is not yet finished and has not been set to manual and the new minimun entry is different.

The learning period is stopped (and prefetching is started) when:

TTreeCache::StopLearningPhase is called.
An entry outside the 'learning' range is requested The 'learning range is from fEntryMin (default to 0) to fEntryMin + fgLearnEntries.
A 'cached' TChain switches over to a new file.

Self-optimization in presence of cache misses

The TTreeCache can optimize its behavior on a cache miss. When miss optimization is enabled (see the SetOptimizeMisses method), it tracks all branches utilized after the learning phase which caused a cache miss. When one cache miss occurs, all the utilized branches are be prefetched for that event. This optimization utilizes the observation that infrequently accessed branches are often accessed together. An example scenario where such behavior is desirable, is an analysis where a set of collections are read only for a few events in which a certain condition is respected, e.g. a trigger fired.

Additional memory and CPU usage when optimizing for cache misses

When this mode is enabled, the memory dedicated to the cache can increase by at most a factor two in the case of cache miss. Additionally, on the first miss of an event, we must iterate through all the "active branches" for the miss cache and find the correct basket. This can be potentially a CPU-expensive operation compared to, e.g., the latency of a SSD. This is why the miss cache is currently disabled by default.

Example usages of TTreeCache

A few use cases are discussed below. A cache may be created with automatic sizing when a TTree is used:

In some applications, e.g. central processing workflows of experiments, the list of branches to read is known a priori. For these cases, the TTreeCache can be instructed about the branches which will be read via explicit calls to the TTree or TTreeCache interfaces. In less streamlined applications such as analysis, predicting the branches which will be read can be difficult. In such cases, ROOT I/O flags used branches automatically when a branch buffer is read during the learning phase.

In the examples below, portions of analysis code are shown. The few statements involving the TreeCache are marked with //<<<

ROOT::RDataFrame and TTreeReader Examples

If you use RDataFrame or TTreeReader, the system will automatically cache the best set of branches: no action is required by the user.

TTree::Draw Example

The TreeCache is automatically used by TTree::Draw. The method knows which branches are used in the query and it puts automatically these branches in the cache. The entry range is also inferred automatically.

TTree::Process and TSelectors Examples

The user must enable the cache and tell the system which branches to cache and also specify the entry range. It is important to specify the entry range in case only a subset of the events is processed to avoid wasteful caching.

Reading all branches

TTree *T;
f->GetObject(T, "mytree");
auto nentries = T->GetEntries();
auto cachesize = 10000000U; // 10 MBytes
T->SetCacheSize(cachesize); //<<<
T->AddBranchToCache("*", true);    //<<< add all branches to the cache
T->Process("myselector.C+");
// In the TSelector::Process function we read all branches
T->GetEntry(i);
// ... Here the entry is processed

Reading a subset of all branches

In the Process function we read a subset of the branches. Only the branches used in the first entry will be put in the cache

TTree *T;
f->GetObject(T, "mytree");
// We want to process only the 200 first entries
auto nentries=200UL;
auto efirst = 0;
auto elast = efirst+nentries;
auto cachesize = 10000000U; // 10 MBytes
TTreeCache::SetLearnEntries(1);  //<<< we can take the decision after 1 entry
T->SetCacheSize(cachesize);      //<<<
T->SetCacheEntryRange(efirst,elast); //<<<
T->Process("myselector.C+","",nentries,efirst);
// In the TSelector::Process we read only 2 branches
auto b1 = T->GetBranch("branch1");
b1->GetEntry(i);
if (somecondition) return;
auto b2 = T->GetBranch("branch2");
b2->GetEntry(i);
... Here the entry is processed

Custom event loop

Always using the same two branches

In this example, exactly two branches are always used: those need to be prefetched.

TTree *T;
f->GetObject(T, "mytree");
auto b1 = T->GetBranch("branch1");
auto b2 = T->GetBranch("branch2");
auto nentries = T->GetEntries();
auto cachesize = 10000000U; //10 MBytes
T->SetCacheSize(cachesize);     //<<<
T->AddBranchToCache(b1, true);  //<<< add branch1 and branch2 to the cache
T->AddBranchToCache(b2, true);  //<<<
T->StopCacheLearningPhase();    //<<< we do not need the system to guess anything
for (auto i : TSeqL(nentries)) {
   T->LoadTree(i); //<<< important call when calling TBranch::GetEntry after
   b1->GetEntry(i);
   if (some condition not met) continue;
   b2->GetEntry(i);
   if (some condition not met) continue;
   // Here we read the full event only in some rare cases.
   // There is no point in caching the other branches as it might be
   // more economical to read only the branch buffers really used.
   T->GetEntry(i);
   ... Here the entry is processed
}

Always using at least the same two branches

In this example, two branches are always used: in addition, some analysis functions are invoked and those may trigger the reading of other branches which are a priori not known. There is no point in prefetching branches that will be used very rarely: we can rely on the system to cache the right branches.

TTree *T;
f->GetObject(T, "mytree");
auto nentries = T->GetEntries();
auto cachesize = 10000000;   //10 MBytes
T->SetCacheSize(cachesize);   //<<<
T->SetCacheLearnEntries(5);   //<<< we can take the decision after 5 entries
auto b1 = T->GetBranch("branch1");
auto b2 = T->GetBranch("branch2");
for (auto i : TSeqL(nentries)) {
   T->LoadTree(i);
   b1->GetEntry(i);
   if (some condition not met) continue;
   b2->GetEntry(i);
   // At this point we may call a user function where a few more branches
   // will be read conditionally. These branches will be put in the cache
   // if they have been used in the first 10 entries
   if (some condition not met) continue;
   // Here we read the full event only in some rare cases.
   // There is no point in caching the other branches as it might be
   // more economical to read only the branch buffers really used.
   T->GetEntry(i);
   .. process the rare but interesting cases.
   ... Here the entry is processed
}

How can the usage and performance of TTreeCache be verified?

Once the event loop terminated, the number of effective system reads for a given file can be checked with a code like the following:

printf("Reading %lld bytes in %d transactions\n",myTFilePtr->GetBytesRead(), f->GetReadCalls());

Another handy command is:

myTreeOrChain.GetTree()->PrintCacheStats();

Definition at line 35 of file TTreeCache.h.

Classes
struct	IOPos

struct	MissCache

Public Types
enum	EPrefillType { kNoPrefill , kAllBranches }

Public Types inherited from TObject
enum	{ kIsOnHeap = 0x01000000 , kNotDeleted = 0x02000000 , kZombie = 0x04000000 , kInconsistent = 0x08000000 , kBitMask = 0x00ffffff }

enum	{ kSingleKey = BIT(0) , kOverwrite = BIT(1) , kWriteDelete = BIT(2) }

enum	EDeprecatedStatusBits { kObjInCanvas = BIT(3) }

enum	EStatusBits { kCanDelete = BIT(0) , kMustCleanup = BIT(3) , kIsReferenced = BIT(4) , kHasUUID = BIT(5) , kCannotPick = BIT(6) , kNoContextMenu = BIT(8) , kInvalidObject = BIT(13) }

Public Member Functions
	TTreeCache ()
	Default Constructor. More...

	TTreeCache (TTree *tree, Int_t buffersize=0)
	Constructor. More...

virtual	~TTreeCache ()
	Destructor. (in general called by the TFile destructor) More...

virtual Int_t	AddBranch (const char *branch, Bool_t subbranches=kFALSE)
	Add a branch to the list of branches to be stored in the cache this is to be used by user (thats why we pass the name of the branch). More...

virtual Int_t	AddBranch (TBranch *b, Bool_t subgbranches=kFALSE)
	Add a branch to the list of branches to be stored in the cache this function is called by the user via TTree::AddBranchToCache. More...

virtual void	Disable ()

virtual Int_t	DropBranch (const char *branch, Bool_t subbranches=kFALSE)
	Remove a branch to the list of branches to be stored in the cache this is to be used by user (thats why we pass the name of the branch). More...

virtual Int_t	DropBranch (TBranch *b, Bool_t subbranches=kFALSE)
	Remove a branch to the list of branches to be stored in the cache this function is called by TBranch::GetBasket. More...

virtual void	Enable ()

virtual Bool_t	FillBuffer ()
	Fill the cache buffer with the branches in the cache. More...

const TObjArray *	GetCachedBranches () const

EPrefillType	GetConfiguredPrefillType () const
	Return the desired prefill type from the environment or resource variable. More...

Double_t	GetEfficiency () const
	Give the total efficiency of the primary cache... defined as the ratio of blocks found in the cache vs. More...

Double_t	GetEfficiencyRel () const
	This will indicate a sort of relative efficiency... a ratio of the reads found in the cache to the number of reads so far. More...

virtual Int_t	GetEntryMax () const

virtual Int_t	GetEntryMin () const

virtual EPrefillType	GetLearnPrefill () const

Double_t	GetMissEfficiency () const
	The total efficiency of the 'miss cache' - defined as the ratio of blocks found in the cache versus the number of blocks prefetched. More...

Double_t	GetMissEfficiencyRel () const
	Relative efficiency of the 'miss cache' - ratio of the reads found in cache to the number of reads so far. More...

Bool_t	GetOptimizeMisses () const

TTree *	GetTree () const

Bool_t	IsAutoCreated () const

virtual Bool_t	IsEnabled () const

virtual Bool_t	IsLearning () const

virtual Int_t	LearnBranch (TBranch *b, Bool_t subgbranches=kFALSE)
	Add a branch discovered by actual usage to the list of branches to be stored in the cache this function is called by TBranch::GetBasket If we are not longer in the training phase this is an error. More...

virtual void	LearnPrefill ()
	Perform an initial prefetch, attempting to read as much of the learning phase baskets for all branches at once. More...

virtual void	Print (Option_t *option="") const
	Print cache statistics. More...

virtual Int_t	ReadBuffer (char *buf, Long64_t pos, Int_t len)
	Read buffer at position pos if the request is in the list of prefetched blocks read from fBuffer. More...

virtual Int_t	ReadBufferNormal (char *buf, Long64_t pos, Int_t len)
	Old method ReadBuffer before the addition of the prefetch mechanism. More...

virtual Int_t	ReadBufferPrefetch (char *buf, Long64_t pos, Int_t len)
	Used to read a chunk from a block previously fetched. More...

virtual void	ResetCache ()
	This will simply clear the cache. More...

void	ResetMissCache ()
	Reset all the miss cache training. More...

void	SetAutoCreated (Bool_t val)

virtual Int_t	SetBufferSize (Int_t buffersize)
	Change the underlying buffer size of the cache. More...

virtual void	SetEntryRange (Long64_t emin, Long64_t emax)
	Set the minimum and maximum entry number to be processed this information helps to optimize the number of baskets to read when prefetching the branch buffers. More...

virtual void	SetFile (TFile *file, TFile::ECacheAction action=TFile::kDisconnect)
	Overload to make sure that the object specific. More...

virtual void	SetLearnPrefill (EPrefillType type=kNoPrefill)
	Set whether the learning period is started with a prefilling of the cache and which type of prefilling is used. More...

void	SetOptimizeMisses (Bool_t opt)
	Start of methods for the miss cache. More...

void	StartLearningPhase ()
	The name should be enough to explain the method. More...

virtual void	StopLearningPhase ()
	This is the counterpart of StartLearningPhase() and can be used to stop the learning phase. More...

virtual void	UpdateBranches (TTree *tree)
	Update pointer to current Tree and recompute pointers to the branches in the cache. More...

Public Member Functions inherited from TFileCacheRead
	TFileCacheRead ()
	Default Constructor. More...

	TFileCacheRead (TFile file, Int_t buffersize, TObject tree=0)
	Creates a TFileCacheRead data structure. More...

virtual	~TFileCacheRead ()
	Destructor. More...

virtual Int_t	AddBranch (const char *, Bool_t=kFALSE)

virtual Int_t	AddBranch (TBranch *, Bool_t=kFALSE)

virtual void	AddNoCacheBytesRead (Long64_t len)

virtual void	AddNoCacheReadCalls (Int_t reads)

virtual void	Close (Option_t *option="")
	Close out any threads or asynchronous fetches used by the underlying implementation. More...

virtual Int_t	GetBufferSize () const

virtual Long64_t	GetBytesRead () const

virtual Long64_t	GetBytesReadExtra () const

TFile *	GetFile () const

virtual Long64_t	GetNoCacheBytesRead () const

virtual Int_t	GetNoCacheReadCalls () const

Int_t	GetNseek () const

Int_t	GetNtot () const

Long64_t	GetPrefetchedBlocks () const

virtual TFilePrefetch *	GetPrefetchObj ()

virtual Int_t	GetReadCalls () const

virtual Int_t	GetUnzipBuffer (char *, Long64_t, Int_t, Bool_t )

virtual Bool_t	IsAsyncReading () const

virtual Bool_t	IsEnablePrefetching () const

virtual Bool_t	IsLearning () const

virtual Int_t	LearnBranch (TBranch *, Bool_t=kFALSE)

virtual void	Prefetch (Long64_t pos, Int_t len)
	Add block of length len at position pos in the list of blocks to be prefetched. More...

virtual void	Print (Option_t *option="") const
	Print cache statistics. More...

virtual Int_t	ReadBuffer (char *buf, Long64_t pos, Int_t len)
	Read buffer at position pos. More...

virtual Int_t	ReadBufferExt (char *buf, Long64_t pos, Int_t len, Int_t &loc)

virtual Int_t	ReadBufferExtNormal (char *buf, Long64_t pos, Int_t len, Int_t &loc)
	Base function for ReadBuffer. More...

virtual Int_t	ReadBufferExtPrefetch (char *buf, Long64_t pos, Int_t len, Int_t &loc)
	prefetch the first block More...

virtual void	SecondPrefetch (Long64_t, Int_t)

virtual void	SecondSort ()
	Sort buffers to be prefetched in increasing order of positions. More...

virtual Int_t	SetBufferSize (Int_t buffersize)
	Sets the buffer size. More...

virtual void	SetEnablePrefetching (Bool_t setPrefetching=kFALSE)
	Set the prefetching mode of this file. More...

virtual void	SetFile (TFile *file, TFile::ECacheAction action=TFile::kDisconnect)
	Set the file using this cache and reset the current blocks (if any). More...

virtual void	SetSkipZip (Bool_t=kTRUE)

virtual void	Sort ()
	Sort buffers to be prefetched in increasing order of positions. More...

virtual void	WaitFinishPrefetch ()

Public Member Functions inherited from TObject
	TObject ()
	TObject constructor. More...

	TObject (const TObject &object)
	TObject copy ctor. More...

virtual	~TObject ()
	TObject destructor. More...

void	AbstractMethod (const char *method) const
	Use this method to implement an "abstract" method that you don't want to leave purely abstract. More...

virtual void	AppendPad (Option_t *option="")
	Append graphics object to current pad. More...

virtual void	Browse (TBrowser *b)
	Browse object. May be overridden for another default action. More...

ULong_t	CheckedHash ()
	Check and record whether this class has a consistent Hash/RecursiveRemove setup (*) and then return the regular Hash value for this object. More...

virtual const char *	ClassName () const
	Returns name of class to which the object belongs. More...

virtual void	Clear (Option_t *="")

virtual TObject *	Clone (const char *newname="") const
	Make a clone of an object using the Streamer facility. More...

virtual Int_t	Compare (const TObject *obj) const
	Compare abstract method. More...

virtual void	Copy (TObject &object) const
	Copy this to obj. More...

virtual void	Delete (Option_t *option="")
	Delete this object. More...

virtual Int_t	DistancetoPrimitive (Int_t px, Int_t py)
	Computes distance from point (px,py) to the object. More...

virtual void	Draw (Option_t *option="")
	Default Draw method for all objects. More...

virtual void	DrawClass () const
	Draw class inheritance tree of the class to which this object belongs. More...

virtual TObject *	DrawClone (Option_t *option="") const
	Draw a clone of this object in the current selected pad for instance with: `gROOT->SetSelectedPad(gPad)`. More...

virtual void	Dump () const
	Dump contents of object on stdout. More...

virtual void	Error (const char method, const char msgfmt,...) const
	Issue error message. More...

virtual void	Execute (const char method, const char params, Int_t *error=0)
	Execute method on this object with the given parameter string, e.g. More...

virtual void	Execute (TMethod method, TObjArray params, Int_t *error=0)
	Execute method on this object with parameters stored in the TObjArray. More...

virtual void	ExecuteEvent (Int_t event, Int_t px, Int_t py)
	Execute action corresponding to an event at (px,py). More...

virtual void	Fatal (const char method, const char msgfmt,...) const
	Issue fatal error message. More...

virtual TObject *	FindObject (const char *name) const
	Must be redefined in derived classes. More...

virtual TObject *	FindObject (const TObject *obj) const
	Must be redefined in derived classes. More...

virtual Option_t *	GetDrawOption () const
	Get option used by the graphics system to draw this object. More...

virtual const char *	GetIconName () const
	Returns mime type name of object. More...

virtual const char *	GetName () const
	Returns name of object. More...

virtual char *	GetObjectInfo (Int_t px, Int_t py) const
	Returns string containing info about the object at position (px,py). More...

virtual Option_t *	GetOption () const

virtual const char *	GetTitle () const
	Returns title of object. More...

virtual UInt_t	GetUniqueID () const
	Return the unique object id. More...

virtual Bool_t	HandleTimer (TTimer *timer)
	Execute action in response of a timer timing out. More...

virtual ULong_t	Hash () const
	Return hash value for this object. More...

Bool_t	HasInconsistentHash () const
	Return true is the type of this object is known to have an inconsistent setup for Hash and RecursiveRemove (i.e. More...

virtual void	Info (const char method, const char msgfmt,...) const
	Issue info message. More...

virtual Bool_t	InheritsFrom (const char *classname) const
	Returns kTRUE if object inherits from class "classname". More...

virtual Bool_t	InheritsFrom (const TClass *cl) const
	Returns kTRUE if object inherits from TClass cl. More...

virtual void	Inspect () const
	Dump contents of this object in a graphics canvas. More...

void	InvertBit (UInt_t f)

virtual Bool_t	IsEqual (const TObject *obj) const
	Default equal comparison (objects are equal if they have the same address in memory). More...

virtual Bool_t	IsFolder () const
	Returns kTRUE in case object contains browsable objects (like containers or lists of other objects). More...

R__ALWAYS_INLINE Bool_t	IsOnHeap () const

virtual Bool_t	IsSortable () const

R__ALWAYS_INLINE Bool_t	IsZombie () const

virtual void	ls (Option_t *option="") const
	The ls function lists the contents of a class on stdout. More...

void	MayNotUse (const char *method) const
	Use this method to signal that a method (defined in a base class) may not be called in a derived class (in principle against good design since a child class should not provide less functionality than its parent, however, sometimes it is necessary). More...

virtual Bool_t	Notify ()
	This method must be overridden to handle object notification. More...

void	Obsolete (const char method, const char asOfVers, const char *removedFromVers) const
	Use this method to declare a method obsolete. More...

void	operator delete (void *ptr)
	Operator delete. More...

void	operator delete[] (void *ptr)
	Operator delete []. More...

void *	operator new (size_t sz)

void *	operator new (size_t sz, void *vp)

void *	operator new[] (size_t sz)

void *	operator new[] (size_t sz, void *vp)

TObject &	operator= (const TObject &rhs)
	TObject assignment operator. More...

virtual void	Paint (Option_t *option="")
	This method must be overridden if a class wants to paint itself. More...

virtual void	Pop ()
	Pop on object drawn in a pad to the top of the display list. More...

virtual void	Print (Option_t *option="") const
	This method must be overridden when a class wants to print itself. More...

virtual Int_t	Read (const char *name)
	Read contents of object with specified name from the current directory. More...

virtual void	RecursiveRemove (TObject *obj)
	Recursively remove this object from a list. More...

void	ResetBit (UInt_t f)

virtual void	SaveAs (const char filename="", Option_t option="") const
	Save this object in the file specified by filename. More...

virtual void	SavePrimitive (std::ostream &out, Option_t *option="")
	Save a primitive as a C++ statement(s) on output stream "out". More...

void	SetBit (UInt_t f)

void	SetBit (UInt_t f, Bool_t set)
	Set or unset the user status bits as specified in f. More...

virtual void	SetDrawOption (Option_t *option="")
	Set drawing option for object. More...

virtual void	SetUniqueID (UInt_t uid)
	Set the unique object id. More...

virtual void	SysError (const char method, const char msgfmt,...) const
	Issue system error message. More...

R__ALWAYS_INLINE Bool_t	TestBit (UInt_t f) const

Int_t	TestBits (UInt_t f) const

virtual void	UseCurrentStyle ()
	Set current style settings in this object This function is called when either TCanvas::UseCurrentStyle or TROOT::ForceStyle have been invoked. More...

virtual void	Warning (const char method, const char msgfmt,...) const
	Issue warning message. More...

virtual Int_t	Write (const char *name=0, Int_t option=0, Int_t bufsize=0)
	Write this object to the current directory. More...

virtual Int_t	Write (const char *name=0, Int_t option=0, Int_t bufsize=0) const
	Write this object to the current directory. More...

Static Public Member Functions
static Int_t	GetLearnEntries ()
	Static function returning the number of entries used to train the cache see SetLearnEntries. More...

static void	SetLearnEntries (Int_t n=10)
	Static function to set the number of entries to be used in learning mode The default value for n is 10. More...

Static Public Member Functions inherited from TObject
static Long_t	GetDtorOnly ()
	Return destructor only flag. More...

static Bool_t	GetObjectStat ()
	Get status of object stat flag. More...

static void	SetDtorOnly (void *obj)
	Set destructor only flag. More...

static void	SetObjectStat (Bool_t stat)
	Turn on/off tracking of objects in the TObjectTable. More...

Protected Attributes
Bool_t	fAutoCreated {kFALSE}
	! true if cache was automatically created More...

TObjArray *	fBranches {nullptr}
	! List of branches to be stored in the cache More...

TList *	fBrNames {nullptr}
	! list of branch names in the cache More...

Long64_t	fCurrentClusterStart {-1}
	! Start of the cluster(s) where the current content was picked out More...

Bool_t	fEnabled {kTRUE}
	! cache enabled for cached reading More...

Long64_t	fEntryCurrent {-1}
	! current lowest entry number in the cache More...

Long64_t	fEntryMax {1}
	! last entry in the cache More...

Long64_t	fEntryMin {0}
	! first entry in the cache More...

Long64_t	fEntryNext {-1}
	! next entry number where cache must be filled More...

Int_t	fFillTimes {0}
	! how many times we can fill the current buffer More...

Bool_t	fFirstBuffer {kTRUE}
	! true if first buffer is used for prefetching More...

Long64_t	fFirstEntry {-1}
	! save the value of the first entry More...

Long64_t	fFirstMiss {-1}
	! set to the event # of the first miss. More...

Bool_t	fFirstTime {kTRUE}
	! save the fact that we processes the first entry More...

Bool_t	fIsLearning {kTRUE}
	! true if cache is in learning mode More...

Bool_t	fIsManual {kFALSE}
	! true if cache is StopLearningPhase was used More...

Long64_t	fLastMiss {-1}
	! set to the event # of the last miss. More...

Bool_t	fLearnPrefilling {kFALSE}
	! true if we are in the process of executing LearnPrefill More...

std::unique_ptr< MissCache >	fMissCache
	! Cache contents for misses More...

Int_t	fNbranches {0}
	! Number of branches in the cache More...

Long64_t	fNextClusterStart {-1}
	! End+1 of the cluster(s) where the current content was picked out More...

Int_t	fNMissReadMiss {0}
	Number of blocks read and not found in either cache. More...

Int_t	fNMissReadOk {0}
	Number of blocks read, not found in the primary cache, and found in the secondary cache. More...

Int_t	fNMissReadPref {0}
	Number of blocks read into the secondary ("miss") cache. More...

Int_t	fNReadMiss {0}
	Number of blocks read and not found in the cache. More...

Int_t	fNReadOk {0}
	Number of blocks read and found in the cache. More...

Int_t	fNReadPref {0}
	Number of blocks that were prefetched. More...

Bool_t	fOneTime {kFALSE}
	! used in the learning phase More...

Bool_t	fOptimizeMisses {kFALSE}
	! true if we should optimize cache misses. More...

EPrefillType	fPrefillType
	Whether a pre-filling is enabled (and if applicable which type) More...

Bool_t	fReadDirectionSet {kFALSE}
	! read direction established More...

Bool_t	fReverseRead {kFALSE}
	! reading in reverse mode More...

TTree *	fTree {nullptr}
	! pointer to the current Tree More...

Protected Attributes inherited from TFileCacheRead
Bool_t	fAsyncReading

Bool_t	fBIsSorted

Bool_t	fBIsTransferred

Int_t *	fBLen
	[fBNb] More...

Int_t	fBNb

Int_t	fBNseek

Int_t	fBNtot

Long64_t *	fBPos
	[fBNb] More...

Long64_t *	fBSeek
	[fBNseek] More...

Int_t *	fBSeekIndex
	[fBNseek] More...

Int_t *	fBSeekLen
	[fBNseek] More...

Int_t *	fBSeekPos
	[fBNseek] More...

Int_t	fBSeekSize

Long64_t *	fBSeekSort
	[fBNseek] More...

Int_t *	fBSeekSortLen
	[fBNseek] More...

char *	fBuffer
	[fBufferSize] buffer of contiguous prefetched blocks More...

Int_t	fBufferLen
	Current buffer length (<= fBufferSize) More...

Int_t	fBufferSize
	Allocated size of fBuffer (at a given time) More...

Int_t	fBufferSizeMin
	Original size of fBuffer. More...

Long64_t	fBytesRead
	Number of bytes read for this cache. More...

Long64_t	fBytesReadExtra
	Number of extra bytes (overhead) read by the readahead buffer. More...

Bool_t	fEnablePrefetching
	reading by prefetching asynchronously More...

TFile *	fFile
	Pointer to file. More...

Bool_t	fIsSorted
	True if fSeek array is sorted. More...

Bool_t	fIsTransferred
	True when fBuffer contains something valid. More...

Int_t *	fLen
	[fNb] Length of long buffers More...

Int_t	fNb
	Number of long buffers. More...

Long64_t	fNoCacheBytesRead
	Number of bytes read by basket to fill cached tree. More...

Int_t	fNoCacheReadCalls
	Number of read calls by basket to fill cached tree. More...

Int_t	fNseek
	Number of blocks to be prefetched. More...

Int_t	fNtot
	Total size of prefetched blocks. More...

Long64_t *	fPos
	[fNb] start of long buffers More...

TFilePrefetch *	fPrefetch
	! Object that does the asynchronous reading in another thread More...

Long64_t	fPrefetchedBlocks
	Number of blocks prefetched. More...

Int_t	fReadCalls
	Number of read calls for this cache. More...

Long64_t *	fSeek
	[fNseek] Position on file of buffers to be prefetched More...

Int_t *	fSeekIndex
	[fNseek] sorted index table of fSeek More...

Int_t *	fSeekLen
	[fNseek] Length of buffers to be prefetched More...

Int_t *	fSeekPos
	[fNseek] Position of sorted blocks in fBuffer More...

Int_t	fSeekSize
	Allocated size of fSeek. More...

Long64_t *	fSeekSort
	[fNseek] Position on file of buffers to be prefetched (sorted) More...

Int_t *	fSeekSortLen
	[fNseek] Length of buffers to be prefetched (sorted) More...

Static Protected Attributes
static Int_t	fgLearnEntries = 100
	number of entries used for learning mode More...

Private Member Functions
	TTreeCache (const TTreeCache &)=delete
	this class cannot be copied More...

Bool_t	CalculateMissCache ()
	Calculate the appropriate miss cache to fetch; helper function for FillMissCache. More...

TBranch *	CalculateMissEntries (Long64_t, int, bool)
	Given an file read, try to determine the corresponding branch. More...

Bool_t	CheckMissCache (char *buf, Long64_t pos, int len)
	Check the miss cache for a particular buffer, fetching if deemed necessary. More...

Bool_t	FillMissCache ()
	Fill the miss cache from the current set of active branches. More...

IOPos	FindBranchBasketPos (TBranch &, Long64_t entry)
	Given a branch and an entry, determine the file location (offset / size) of the corresponding basket. More...

TTreeCache &	operator= (const TTreeCache &)=delete

Bool_t	ProcessMiss (Long64_t pos, int len)
	! Given a file read not in the miss cache, handle (possibly) loading the data. More...

Additional Inherited Members
Protected Member Functions inherited from TFileCacheRead
void	SetEnablePrefetchingImpl (Bool_t setPrefetching=kFALSE)
	TFileCacheRead implementation of SetEnablePrefetching. More...

Protected Member Functions inherited from TObject
virtual void	DoError (int level, const char location, const char fmt, va_list va) const
	Interface to ErrorHandler (protected). More...

void	MakeZombie ()

#include <TTreeCache.h>

Inheritance diagram for TTreeCache:

[legend]

Member Enumeration Documentation

◆ EPrefillType

enum TTreeCache::EPrefillType

Enumerator
kNoPrefill
kAllBranches

Definition at line 38 of file TTreeCache.h.

Constructor & Destructor Documentation

◆ TTreeCache() [1/3]

TTreeCache::TTreeCache ( const TTreeCache & )

privatedelete

this class cannot be copied

◆ TTreeCache() [2/3]

TTreeCache::TTreeCache ( )

Default Constructor.

Definition at line 304 of file TTreeCache.cxx.

◆ TTreeCache() [3/3]

TTreeCache::TTreeCache	(	TTree *	tree,
		Int_t	buffersize = `0`
	)

Constructor.

Definition at line 311 of file TTreeCache.cxx.

◆ ~TTreeCache()

TTreeCache::~TTreeCache ( )

virtual

Destructor. (in general called by the TFile destructor)

Definition at line 323 of file TTreeCache.cxx.

Member Function Documentation

◆ AddBranch() [1/2]

Int_t TTreeCache::AddBranch	(	const char *	bname,
		Bool_t	subbranches = `kFALSE`
	)

virtual

Add a branch to the list of branches to be stored in the cache this is to be used by user (thats why we pass the name of the branch).

It works in exactly the same way as TTree::SetBranchStatus so you probably want to look over there for details about the use of bname with regular expressions. The branches are taken with respect to the Owner of this TTreeCache (i.e. the original Tree) NB: if bname="*" all branches are put in the cache and the learning phase stopped Returns:

0 branch added or already included
-1 on error

Reimplemented from TFileCacheRead.

Reimplemented in TTreeCacheUnzip.

Definition at line 437 of file TTreeCache.cxx.

◆ AddBranch() [2/2]

Int_t TTreeCache::AddBranch	(	TBranch *	b,
		Bool_t	subbranches = `kFALSE`
	)

virtual

Add a branch to the list of branches to be stored in the cache this function is called by the user via TTree::AddBranchToCache.

The branch is added even if we are outside of the training phase. Returns:

0 branch added or already included
-1 on error

Reimplemented from TFileCacheRead.

Reimplemented in TTreeCacheUnzip.

Definition at line 367 of file TTreeCache.cxx.

◆ CalculateMissCache()

Bool_t TTreeCache::CalculateMissCache ( )

private

Calculate the appropriate miss cache to fetch; helper function for FillMissCache.

◆ CalculateMissEntries()

TBranch * TTreeCache::CalculateMissEntries	(	Long64_t	pos,
		int	len,
		bool	all
	)

private

Given an file read, try to determine the corresponding branch.

Given a particular IO description (offset / length) representing a 'miss' of the TTreeCache's primary cache, calculate all the corresponding IO that should be performed.

all indicates that this function should search the set of all branches in this TTree. When set to false, we only search through branches that have previously incurred a miss.

Returns:

TBranch pointer corresponding to the basket that will be retrieved by this IO operation.
If no corresponding branch could be found (or an error occurs), this returns nullptr.

Definition at line 781 of file TTreeCache.cxx.

◆ CheckMissCache()

Bool_t TTreeCache::CheckMissCache	(	char *	buf,
		Long64_t	pos,
		int	len
	)

private

Check the miss cache for a particular buffer, fetching if deemed necessary.

Given an IO operation (pos, len) that was a cache miss in the primary TTC, try the operation again with the miss cache.

Returns true if the IO operation was successful and the contents of buf were populated with the requested data.

Definition at line 912 of file TTreeCache.cxx.

◆ Disable()

virtual void TTreeCache::Disable ( )

inlinevirtual

Definition at line 139 of file TTreeCache.h.

◆ DropBranch() [1/2]

Int_t TTreeCache::DropBranch	(	const char *	bname,
		Bool_t	subbranches = `kFALSE`
	)

virtual

Remove a branch to the list of branches to be stored in the cache this is to be used by user (thats why we pass the name of the branch).

It works in exactly the same way as TTree::SetBranchStatus so you probably want to look over there for details about the use of bname with regular expressions. The branches are taken with respect to the Owner of this TTreeCache (i.e. the original Tree) NB: if bname="*" all branches are put in the cache and the learning phase stopped Returns:

0 branch dropped or not in cache
-1 on error

Definition at line 577 of file TTreeCache.cxx.

◆ DropBranch() [2/2]

Int_t TTreeCache::DropBranch	(	TBranch *	b,
		Bool_t	subbranches = `kFALSE`
	)

virtual

Remove a branch to the list of branches to be stored in the cache this function is called by TBranch::GetBasket.

Returns:

0 branch dropped or not in cache
-1 on error

Definition at line 532 of file TTreeCache.cxx.

◆ Enable()

virtual void TTreeCache::Enable ( )

inlinevirtual

Definition at line 140 of file TTreeCache.h.

◆ FillBuffer()

Bool_t TTreeCache::FillBuffer ( )

virtual

Fill the cache buffer with the branches in the cache.

Reimplemented in TTreeCacheUnzip.

Definition at line 1105 of file TTreeCache.cxx.

◆ FillMissCache()

Bool_t TTreeCache::FillMissCache ( )

private

Fill the miss cache from the current set of active branches.

◆ FindBranchBasketPos()

TTreeCache::IOPos TTreeCache::FindBranchBasketPos	(	TBranch &	b,
		Long64_t	entry
	)

private

Given a branch and an entry, determine the file location (offset / size) of the corresponding basket.

For the event currently being fetched into the miss cache, find the IO (offset / length tuple) to pull in the current basket for a given branch.

Returns:

IOPos describing the IO operation necessary for the basket on this branch
On failure, IOPos.length will be set to 0.

Definition at line 707 of file TTreeCache.cxx.

◆ GetCachedBranches()

const TObjArray * TTreeCache::GetCachedBranches ( ) const

inline

Definition at line 142 of file TTreeCache.h.

◆ GetConfiguredPrefillType()

TTreeCache::EPrefillType TTreeCache::GetConfiguredPrefillType ( ) const

Return the desired prefill type from the environment or resource variable.

0 - No prefill
1 - All branches

Definition at line 1780 of file TTreeCache.cxx.

◆ GetEfficiency()

Double_t TTreeCache::GetEfficiency ( ) const

Give the total efficiency of the primary cache... defined as the ratio of blocks found in the cache vs.

the number of blocks prefetched ( it could be more than 1 if we read the same block from the cache more than once )

Note: This should eb used at the end of the processing or we will get incomplete stats

Definition at line 1803 of file TTreeCache.cxx.

◆ GetEfficiencyRel()

Double_t TTreeCache::GetEfficiencyRel ( ) const

This will indicate a sort of relative efficiency... a ratio of the reads found in the cache to the number of reads so far.

Definition at line 1827 of file TTreeCache.cxx.

◆ GetEntryMax()

virtual Int_t TTreeCache::GetEntryMax ( ) const

inlinevirtual

Definition at line 147 of file TTreeCache.h.

◆ GetEntryMin()

virtual Int_t TTreeCache::GetEntryMin ( ) const

inlinevirtual

Definition at line 146 of file TTreeCache.h.

◆ GetLearnEntries()

Int_t TTreeCache::GetLearnEntries ( )

static

Static function returning the number of entries used to train the cache see SetLearnEntries.

Definition at line 1852 of file TTreeCache.cxx.

◆ GetLearnPrefill()

virtual EPrefillType TTreeCache::GetLearnPrefill ( ) const

inlinevirtual

Definition at line 149 of file TTreeCache.h.

◆ GetMissEfficiency()

Double_t TTreeCache::GetMissEfficiency ( ) const

The total efficiency of the 'miss cache' - defined as the ratio of blocks found in the cache versus the number of blocks prefetched.

Definition at line 1815 of file TTreeCache.cxx.

◆ GetMissEfficiencyRel()

Double_t TTreeCache::GetMissEfficiencyRel ( ) const

Relative efficiency of the 'miss cache' - ratio of the reads found in cache to the number of reads so far.

Definition at line 1839 of file TTreeCache.cxx.

◆ GetOptimizeMisses()

Bool_t TTreeCache::GetOptimizeMisses ( ) const

inline

Definition at line 141 of file TTreeCache.h.

◆ GetTree()

TTree * TTreeCache::GetTree ( ) const

inline

Definition at line 152 of file TTreeCache.h.

◆ IsAutoCreated()

Bool_t TTreeCache::IsAutoCreated ( ) const

inline

Definition at line 153 of file TTreeCache.h.

◆ IsEnabled()

virtual Bool_t TTreeCache::IsEnabled ( ) const

inlinevirtual

Definition at line 154 of file TTreeCache.h.

◆ IsLearning()

virtual Bool_t TTreeCache::IsLearning ( ) const

inlinevirtual

Reimplemented from TFileCacheRead.

Definition at line 155 of file TTreeCache.h.

◆ LearnBranch()

Int_t TTreeCache::LearnBranch	(	TBranch *	b,
		Bool_t	subbranches = `kFALSE`
	)

virtual

Add a branch discovered by actual usage to the list of branches to be stored in the cache this function is called by TBranch::GetBasket If we are not longer in the training phase this is an error.

Returns:

0 branch added or already included
-1 on error

Reimplemented from TFileCacheRead.

Definition at line 341 of file TTreeCache.cxx.

◆ LearnPrefill()

void TTreeCache::LearnPrefill ( )

virtual

Perform an initial prefetch, attempting to read as much of the learning phase baskets for all branches at once.

Definition at line 2215 of file TTreeCache.cxx.

◆ operator=()

TTreeCache & TTreeCache::operator= ( const TTreeCache & )

privatedelete

◆ Print()

void TTreeCache::Print ( Option_t * option = "" ) const

virtual

Print cache statistics.

Like:

******TreeCache statistics for file: cms2.root ******
Number of branches in the cache ...: 1093
Cache Efficiency ..................: 0.997372
Cache Efficiency Rel...............: 1.000000
Learn entries......................: 100
Reading............................: 72761843 bytes in 7 transactions
Readahead..........................: 256000 bytes with overhead = 0 bytes
Average transaction................: 10394.549000 Kbytes
Number of blocks in current cache..: 210, total size: 6280352

if option = "a" the list of blocks in the cache is printed see also class TTreePerfStats.
if option contains 'cachedbranches', the list of branches being cached is printed.

Reimplemented from TFileCacheRead.

Reimplemented in TTreeCacheUnzip.

Definition at line 1877 of file TTreeCache.cxx.

◆ ProcessMiss()

Bool_t TTreeCache::ProcessMiss	(	Long64_t	pos,
		int	len
	)

private

! Given a file read not in the miss cache, handle (possibly) loading the data.

Process a cache miss; (pos, len) isn't in the buffer.

The first time we have a miss, we buffer as many baskets we can (up to the maximum size of the TTreeCache) in memory from all branches that are not in the prefetch list.

Subsequent times, we fetch all the buffers corresponding to branches that had previously seen misses. If it turns out the (pos, len) isn't in the list of branches, we treat this as if it was the first miss.

Returns true if we were able to pull the data into the miss cache.

Definition at line 854 of file TTreeCache.cxx.

◆ ReadBuffer()

Int_t TTreeCache::ReadBuffer	(	char *	buf,
		Long64_t	pos,
		Int_t	len
	)

virtual

Read buffer at position pos if the request is in the list of prefetched blocks read from fBuffer.

Otherwise try to fill the cache from the list of selected branches, and recheck if pos is now in the list. Returns:

-1 in case of read failure,
0 in case not in cache,
1 in case read from cache. This function overloads TFileCacheRead::ReadBuffer.

Reimplemented from TFileCacheRead.

Definition at line 2005 of file TTreeCache.cxx.

◆ ReadBufferNormal()

Int_t TTreeCache::ReadBufferNormal	(	char *	buf,
		Long64_t	pos,
		Int_t	len
	)

virtual

Old method ReadBuffer before the addition of the prefetch mechanism.

Definition at line 1905 of file TTreeCache.cxx.

◆ ReadBufferPrefetch()

Int_t TTreeCache::ReadBufferPrefetch	(	char *	buf,
		Long64_t	pos,
		Int_t	len
	)

virtual

Used to read a chunk from a block previously fetched.

It will call FillBuffer even if the cache lookup succeeds, because it will try to prefetch the next block as soon as we start reading from the current block.

Definition at line 1964 of file TTreeCache.cxx.

◆ ResetCache()

void TTreeCache::ResetCache ( )

virtual

This will simply clear the cache.

Reimplemented in TTreeCacheUnzip.

Definition at line 2018 of file TTreeCache.cxx.

◆ ResetMissCache()

void TTreeCache::ResetMissCache ( )

Reset all the miss cache training.

The contents of the miss cache will be emptied as well as the list of branches used.

Definition at line 688 of file TTreeCache.cxx.

◆ SetAutoCreated()

void TTreeCache::SetAutoCreated ( Bool_t val )

inline

Definition at line 167 of file TTreeCache.h.

◆ SetBufferSize()

Int_t TTreeCache::SetBufferSize ( Int_t buffersize )

virtual

Change the underlying buffer size of the cache.

If the change of size means some cache content is lost, or if the buffer is now larger, setup for a cache refill the next time there is a read Returns:

0 if the buffer content is still available
1 if some or all of the buffer content has been made unavailable
-1 on error

Reimplemented from TFileCacheRead.

Reimplemented in TTreeCacheUnzip.

Definition at line 2037 of file TTreeCache.cxx.

◆ SetEntryRange()

void TTreeCache::SetEntryRange	(	Long64_t	emin,
		Long64_t	emax
	)

virtual

Set the minimum and maximum entry number to be processed this information helps to optimize the number of baskets to read when prefetching the branch buffers.

Reimplemented in TTreeCacheUnzip.

Definition at line 2070 of file TTreeCache.cxx.

◆ SetFile()

void TTreeCache::SetFile	(	TFile *	file,
		TFile::ECacheAction	action = `TFile::kDisconnect`
	)

virtual

Overload to make sure that the object specific.

Reimplemented from TFileCacheRead.

Definition at line 2092 of file TTreeCache.cxx.

◆ SetLearnEntries()

void TTreeCache::SetLearnEntries ( Int_t n = 10 )

static

Static function to set the number of entries to be used in learning mode The default value for n is 10.

n must be >= 1

Definition at line 2109 of file TTreeCache.cxx.

◆ SetLearnPrefill()

void TTreeCache::SetLearnPrefill ( TTreeCache::EPrefillType type = kNoPrefill )

virtual

Set whether the learning period is started with a prefilling of the cache and which type of prefilling is used.

The two value currently supported are:

TTreeCache::kNoPrefill disable the prefilling
TTreeCache::kAllBranches fill the cache with baskets from all branches. The default prefilling behavior can be controlled by setting TTreeCache.Prefill or the environment variable ROOT_TTREECACHE_PREFILL.

Definition at line 2124 of file TTreeCache.cxx.

◆ SetOptimizeMisses()

void TTreeCache::SetOptimizeMisses ( Bool_t opt )

Start of methods for the miss cache.

Enable / disable the miss cache.

The first time this is called on a TTreeCache object, the corresponding data structures will be allocated. Subsequent enable / disables will simply turn the functionality on/off.

Definition at line 674 of file TTreeCache.cxx.

◆ StartLearningPhase()

void TTreeCache::StartLearningPhase ( )

The name should be enough to explain the method.

The only additional comments is that the cache is cleaned before the new learning phase.

Definition at line 2134 of file TTreeCache.cxx.

◆ StopLearningPhase()

void TTreeCache::StopLearningPhase ( )

virtual

This is the counterpart of StartLearningPhase() and can be used to stop the learning phase.

It's useful when the user knows exactly what branches they are going to use. For the moment it's just a call to FillBuffer() since that method will create the buffer lists from the specified branches.

Reimplemented in TTreeCacheUnzip.

Definition at line 2151 of file TTreeCache.cxx.

◆ UpdateBranches()

void TTreeCache::UpdateBranches ( TTree * tree )

virtual

Update pointer to current Tree and recompute pointers to the branches in the cache.

Reimplemented in TTreeCacheUnzip.

Definition at line 2175 of file TTreeCache.cxx.

Member Data Documentation

◆ fAutoCreated

Bool_t TTreeCache::fAutoCreated {kFALSE}

protected

! true if cache was automatically created

Definition at line 69 of file TTreeCache.h.

◆ fBranches

TObjArray* TTreeCache::fBranches {nullptr}

protected

! List of branches to be stored in the cache

Definition at line 54 of file TTreeCache.h.

◆ fBrNames

TList* TTreeCache::fBrNames {nullptr}

protected

! list of branch names in the cache

Definition at line 55 of file TTreeCache.h.

◆ fCurrentClusterStart

Long64_t TTreeCache::fCurrentClusterStart {-1}

protected

! Start of the cluster(s) where the current content was picked out

Definition at line 45 of file TTreeCache.h.

◆ fEnabled

Bool_t TTreeCache::fEnabled {kTRUE}

protected

! cache enabled for cached reading

Definition at line 66 of file TTreeCache.h.

◆ fEntryCurrent

Long64_t TTreeCache::fEntryCurrent {-1}

protected

! current lowest entry number in the cache

Definition at line 43 of file TTreeCache.h.

◆ fEntryMax

Long64_t TTreeCache::fEntryMax {1}

protected

! last entry in the cache

Definition at line 42 of file TTreeCache.h.

◆ fEntryMin

Long64_t TTreeCache::fEntryMin {0}

protected

! first entry in the cache

Definition at line 41 of file TTreeCache.h.

◆ fEntryNext

Long64_t TTreeCache::fEntryNext {-1}

protected

! next entry number where cache must be filled

Definition at line 44 of file TTreeCache.h.

◆ fFillTimes

Int_t TTreeCache::fFillTimes {0}

protected

! how many times we can fill the current buffer

Definition at line 62 of file TTreeCache.h.

◆ fFirstBuffer

Bool_t TTreeCache::fFirstBuffer {kTRUE}

protected

! true if first buffer is used for prefetching

Definition at line 59 of file TTreeCache.h.

◆ fFirstEntry

Long64_t TTreeCache::fFirstEntry {-1}

protected

! save the value of the first entry

Definition at line 64 of file TTreeCache.h.

◆ fFirstMiss

Long64_t TTreeCache::fFirstMiss {-1}

protected

! set to the event # of the first miss.

Definition at line 76 of file TTreeCache.h.

◆ fFirstTime

Bool_t TTreeCache::fFirstTime {kTRUE}

protected

! save the fact that we processes the first entry

Definition at line 63 of file TTreeCache.h.

◆ fgLearnEntries

Int_t TTreeCache::fgLearnEntries = 100

staticprotected

number of entries used for learning mode

Definition at line 68 of file TTreeCache.h.

◆ fIsLearning

Bool_t TTreeCache::fIsLearning {kTRUE}

protected

! true if cache is in learning mode

Definition at line 57 of file TTreeCache.h.

◆ fIsManual

Bool_t TTreeCache::fIsManual {kFALSE}

protected

! true if cache is StopLearningPhase was used

Definition at line 58 of file TTreeCache.h.

◆ fLastMiss

Long64_t TTreeCache::fLastMiss {-1}

protected

! set to the event # of the last miss.

Definition at line 77 of file TTreeCache.h.

◆ fLearnPrefilling

Bool_t TTreeCache::fLearnPrefilling {kFALSE}

protected

! true if we are in the process of executing LearnPrefill

Definition at line 71 of file TTreeCache.h.

◆ fMissCache

std::unique_ptr<MissCache> TTreeCache::fMissCache

protected

! Cache contents for misses

Definition at line 108 of file TTreeCache.h.

◆ fNbranches

Int_t TTreeCache::fNbranches {0}

protected

! Number of branches in the cache

Definition at line 47 of file TTreeCache.h.

◆ fNextClusterStart

Long64_t TTreeCache::fNextClusterStart {-1}

protected

! End+1 of the cluster(s) where the current content was picked out

Definition at line 46 of file TTreeCache.h.

◆ fNMissReadMiss

Int_t TTreeCache::fNMissReadMiss {0}

protected

Number of blocks read and not found in either cache.

Definition at line 51 of file TTreeCache.h.

◆ fNMissReadOk

Int_t TTreeCache::fNMissReadOk {0}

protected

Number of blocks read, not found in the primary cache, and found in the secondary cache.

Definition at line 49 of file TTreeCache.h.

◆ fNMissReadPref

Int_t TTreeCache::fNMissReadPref {0}

protected

Number of blocks read into the secondary ("miss") cache.

Definition at line 53 of file TTreeCache.h.

◆ fNReadMiss

Int_t TTreeCache::fNReadMiss {0}

protected

Number of blocks read and not found in the cache.

Definition at line 50 of file TTreeCache.h.

◆ fNReadOk

Int_t TTreeCache::fNReadOk {0}

protected

Number of blocks read and found in the cache.

Definition at line 48 of file TTreeCache.h.

◆ fNReadPref

Int_t TTreeCache::fNReadPref {0}

protected

Number of blocks that were prefetched.

Definition at line 52 of file TTreeCache.h.

◆ fOneTime

Bool_t TTreeCache::fOneTime {kFALSE}

protected

! used in the learning phase

Definition at line 60 of file TTreeCache.h.

◆ fOptimizeMisses

Bool_t TTreeCache::fOptimizeMisses {kFALSE}

protected

! true if we should optimize cache misses.

Definition at line 75 of file TTreeCache.h.

◆ fPrefillType

EPrefillType TTreeCache::fPrefillType

protected

Whether a pre-filling is enabled (and if applicable which type)

Definition at line 67 of file TTreeCache.h.

◆ fReadDirectionSet

Bool_t TTreeCache::fReadDirectionSet {kFALSE}

protected

! read direction established

Definition at line 65 of file TTreeCache.h.

◆ fReverseRead

Bool_t TTreeCache::fReverseRead {kFALSE}

protected

! reading in reverse mode

Definition at line 61 of file TTreeCache.h.

◆ fTree

TTree* TTreeCache::fTree {nullptr}

protected

! pointer to the current Tree

Definition at line 56 of file TTreeCache.h.

Libraries for TTreeCache:

[legend]

A cache to speed-up the reading of ROOT datasets

Table of Contents

Motivation: why having a cache is needed?

Are there cases for which the usage of TTreeCache is detrimental for performance?

General Description

Changes of behavior when using TChain and TEventList

Self-optimization in presence of cache misses

Additional memory and CPU usage when optimizing for cache misses

Example usages of TTreeCache

ROOT::RDataFrame and TTreeReader Examples

TTree::Draw Example

TTree::Process and TSelectors Examples

Reading all branches

Reading a subset of all branches

Custom event loop

Always using the same two branches

Always using at least the same two branches

How can the usage and performance of TTreeCache be verified?

Classes

Public Types

Public Member Functions

Static Public Member Functions

Protected Attributes

Static Protected Attributes

Private Member Functions

Additional Inherited Members

Member Enumeration Documentation

◆ EPrefillType

Constructor & Destructor Documentation

◆ TTreeCache() [1/3]

◆ TTreeCache() [2/3]

◆ TTreeCache() [3/3]

◆ ~TTreeCache()

Member Function Documentation

◆ AddBranch() [1/2]

◆ AddBranch() [2/2]

◆ CalculateMissCache()

◆ CalculateMissEntries()

◆ CheckMissCache()

◆ Disable()

◆ DropBranch() [1/2]

◆ DropBranch() [2/2]

◆ Enable()

◆ FillBuffer()

◆ FillMissCache()

◆ FindBranchBasketPos()

◆ GetCachedBranches()

◆ GetConfiguredPrefillType()

◆ GetEfficiency()

◆ GetEfficiencyRel()

◆ GetEntryMax()

◆ GetEntryMin()

◆ GetLearnEntries()

◆ GetLearnPrefill()

◆ GetMissEfficiency()

◆ GetMissEfficiencyRel()

◆ GetOptimizeMisses()

◆ GetTree()

◆ IsAutoCreated()

◆ IsEnabled()

◆ IsLearning()

◆ LearnBranch()

◆ LearnPrefill()

◆ operator=()

◆ Print()

◆ ProcessMiss()

◆ ReadBuffer()

◆ ReadBufferNormal()

◆ ReadBufferPrefetch()

◆ ResetCache()

◆ ResetMissCache()

◆ SetAutoCreated()

◆ SetBufferSize()

◆ SetEntryRange()

◆ SetFile()

◆ SetLearnEntries()

◆ SetLearnPrefill()

◆ SetOptimizeMisses()

◆ StartLearningPhase()

◆ StopLearningPhase()