Logo ROOT  
Reference Guide
TTreeCacheUnzip Class Reference

A TTreeCache which exploits parallelized decompression of its own content.

Definition at line 35 of file TTreeCacheUnzip.h.

Classes

struct  UnzipState
 

Public Types

enum  EParUnzipMode { kEnable , kDisable , kForce }
 
enum  EUnzipState { kUntouched , kProgress , kFinished }
 
- Public Types inherited from TTreeCache
enum  EPrefillType { kNoPrefill , kAllBranches }
 
- Public Types inherited from TObject
enum  {
  kIsOnHeap = 0x01000000 , kNotDeleted = 0x02000000 , kZombie = 0x04000000 , kInconsistent = 0x08000000 ,
  kBitMask = 0x00ffffff
}
 
enum  { kSingleKey = BIT(0) , kOverwrite = BIT(1) , kWriteDelete = BIT(2) }
 
enum  EDeprecatedStatusBits { kObjInCanvas = BIT(3) }
 
enum  EStatusBits {
  kCanDelete = BIT(0) , kMustCleanup = BIT(3) , kIsReferenced = BIT(4) , kHasUUID = BIT(5) ,
  kCannotPick = BIT(6) , kNoContextMenu = BIT(8) , kInvalidObject = BIT(13)
}
 

Public Member Functions

 TTreeCacheUnzip ()
 
 TTreeCacheUnzip (TTree *tree, Int_t buffersize=0)
 Constructor. More...
 
virtual ~TTreeCacheUnzip ()
 Destructor. (in general called by the TFile destructor) More...
 
virtual Int_t AddBranch (const char *branch, Bool_t subbranches=kFALSE)
 Add a branch to the list of branches to be stored in the cache this function is called by TBranch::GetBasket Returns: More...
 
virtual Int_t AddBranch (TBranch *b, Bool_t subbranches=kFALSE)
 Add a branch to the list of branches to be stored in the cache this function is called by TBranch::GetBasket Returns: More...
 
Int_t CreateTasks ()
 We create a TTaskGroup and asynchronously maps each group of baskets(> 100 kB in total) to a task. More...
 
Bool_t FillBuffer ()
 Fill the cache buffer with the branches in the cache. More...
 
Int_t GetNFound ()
 
Int_t GetNMissed ()
 
Int_t GetNUnzip ()
 
Int_t GetRecordHeader (char *buf, Int_t maxbytes, Int_t &nbytes, Int_t &objlen, Int_t &keylen)
 Read the logical record header from the buffer buf. More...
 
virtual Int_t GetUnzipBuffer (char **buf, Long64_t pos, Int_t len, Bool_t *free)
 We try to read a buffer that has already been unzipped Returns -1 in case of read failure, 0 in case it's not in the cache and n>0 in case read from cache (number of bytes copied). More...
 
Int_t GetUnzipGroupSize ()
 
void Print (Option_t *option="") const
 Print cache statistics. More...
 
virtual Int_t ReadBufferExt (char *buf, Long64_t pos, Int_t len, Int_t &loc)
 
virtual void ResetCache ()
 This will delete the list of buffers that are in the unzipping cache and will reset certain values in the cache. More...
 
virtual Int_t SetBufferSize (Int_t buffersize)
 Change the underlying buffer size of the cache. More...
 
void SetEntryRange (Long64_t emin, Long64_t emax)
 Set the minimum and maximum entry number to be processed this information helps to optimize the number of baskets to read when prefetching the branch buffers. More...
 
void SetUnzipBufferSize (Long64_t bufferSize)
 Sets the size for the unzipping cache... by default it should be two times the size of the prefetching cache. More...
 
void SetUnzipGroupSize (Int_t groupSize)
 
virtual void StopLearningPhase ()
 It's the same as TTreeCache::StopLearningPhase but we guarantee that we start the unzipping just after getting the buffers. More...
 
Int_t UnzipBuffer (char **dest, char *src)
 Unzips a ROOT specific buffer... by reading the header at the beginning. More...
 
Int_t UnzipCache (Int_t index)
 This inflates a basket in the cache. More...
 
void UpdateBranches (TTree *tree)
 update pointer to current Tree and recompute pointers to the branches in the cache More...
 
- Public Member Functions inherited from TTreeCache
 TTreeCache ()
 Default Constructor. More...
 
 TTreeCache (TTree *tree, Int_t buffersize=0)
 Constructor. More...
 
virtual ~TTreeCache ()
 Destructor. (in general called by the TFile destructor) More...
 
virtual Int_t AddBranch (const char *branch, Bool_t subbranches=kFALSE)
 Add a branch to the list of branches to be stored in the cache this is to be used by user (thats why we pass the name of the branch). More...
 
virtual Int_t AddBranch (TBranch *b, Bool_t subgbranches=kFALSE)
 Add a branch to the list of branches to be stored in the cache this function is called by the user via TTree::AddBranchToCache. More...
 
virtual void Disable ()
 
virtual Int_t DropBranch (const char *branch, Bool_t subbranches=kFALSE)
 Remove a branch to the list of branches to be stored in the cache this is to be used by user (thats why we pass the name of the branch). More...
 
virtual Int_t DropBranch (TBranch *b, Bool_t subbranches=kFALSE)
 Remove a branch to the list of branches to be stored in the cache this function is called by TBranch::GetBasket. More...
 
virtual void Enable ()
 
virtual Bool_t FillBuffer ()
 Fill the cache buffer with the branches in the cache. More...
 
const TObjArrayGetCachedBranches () const
 
EPrefillType GetConfiguredPrefillType () const
 Return the desired prefill type from the environment or resource variable. More...
 
Double_t GetEfficiency () const
 Give the total efficiency of the primary cache... defined as the ratio of blocks found in the cache vs. More...
 
Double_t GetEfficiencyRel () const
 This will indicate a sort of relative efficiency... a ratio of the reads found in the cache to the number of reads so far. More...
 
virtual Int_t GetEntryMax () const
 
virtual Int_t GetEntryMin () const
 
virtual EPrefillType GetLearnPrefill () const
 
Double_t GetMissEfficiency () const
 The total efficiency of the 'miss cache' - defined as the ratio of blocks found in the cache versus the number of blocks prefetched. More...
 
Double_t GetMissEfficiencyRel () const
 Relative efficiency of the 'miss cache' - ratio of the reads found in cache to the number of reads so far. More...
 
Bool_t GetOptimizeMisses () const
 
TTreeGetTree () const
 
Bool_t IsAutoCreated () const
 
virtual Bool_t IsEnabled () const
 
virtual Bool_t IsLearning () const
 
virtual Int_t LearnBranch (TBranch *b, Bool_t subgbranches=kFALSE)
 Add a branch discovered by actual usage to the list of branches to be stored in the cache this function is called by TBranch::GetBasket If we are not longer in the training phase this is an error. More...
 
virtual void LearnPrefill ()
 Perform an initial prefetch, attempting to read as much of the learning phase baskets for all branches at once. More...
 
virtual void Print (Option_t *option="") const
 Print cache statistics. More...
 
virtual Int_t ReadBuffer (char *buf, Long64_t pos, Int_t len)
 Read buffer at position pos if the request is in the list of prefetched blocks read from fBuffer. More...
 
virtual Int_t ReadBufferNormal (char *buf, Long64_t pos, Int_t len)
 Old method ReadBuffer before the addition of the prefetch mechanism. More...
 
virtual Int_t ReadBufferPrefetch (char *buf, Long64_t pos, Int_t len)
 Used to read a chunk from a block previously fetched. More...
 
virtual void ResetCache ()
 This will simply clear the cache. More...
 
void ResetMissCache ()
 Reset all the miss cache training. More...
 
void SetAutoCreated (Bool_t val)
 
virtual Int_t SetBufferSize (Int_t buffersize)
 Change the underlying buffer size of the cache. More...
 
virtual void SetEntryRange (Long64_t emin, Long64_t emax)
 Set the minimum and maximum entry number to be processed this information helps to optimize the number of baskets to read when prefetching the branch buffers. More...
 
virtual void SetFile (TFile *file, TFile::ECacheAction action=TFile::kDisconnect)
 Overload to make sure that the object specific. More...
 
virtual void SetLearnPrefill (EPrefillType type=kNoPrefill)
 Set whether the learning period is started with a prefilling of the cache and which type of prefilling is used. More...
 
void SetOptimizeMisses (Bool_t opt)
 Start of methods for the miss cache. More...
 
void StartLearningPhase ()
 The name should be enough to explain the method. More...
 
virtual void StopLearningPhase ()
 This is the counterpart of StartLearningPhase() and can be used to stop the learning phase. More...
 
virtual void UpdateBranches (TTree *tree)
 Update pointer to current Tree and recompute pointers to the branches in the cache. More...
 
- Public Member Functions inherited from TFileCacheRead
 TFileCacheRead ()
 Default Constructor. More...
 
 TFileCacheRead (TFile *file, Int_t buffersize, TObject *tree=nullptr)
 Creates a TFileCacheRead data structure. More...
 
virtual ~TFileCacheRead ()
 Destructor. More...
 
virtual Int_t AddBranch (const char *, Bool_t=kFALSE)
 
virtual Int_t AddBranch (TBranch *, Bool_t=kFALSE)
 
virtual void AddNoCacheBytesRead (Long64_t len)
 
virtual void AddNoCacheReadCalls (Int_t reads)
 
virtual void Close (Option_t *option="")
 Close out any threads or asynchronous fetches used by the underlying implementation. More...
 
virtual Int_t GetBufferSize () const
 
virtual Long64_t GetBytesRead () const
 
virtual Long64_t GetBytesReadExtra () const
 
TFileGetFile () const
 
virtual Long64_t GetNoCacheBytesRead () const
 
virtual Int_t GetNoCacheReadCalls () const
 
Int_t GetNseek () const
 
Int_t GetNtot () const
 
Long64_t GetPrefetchedBlocks () const
 
virtual TFilePrefetchGetPrefetchObj ()
 
virtual Int_t GetReadCalls () const
 
virtual Int_t GetUnzipBuffer (char **, Long64_t, Int_t, Bool_t *)
 
virtual Bool_t IsAsyncReading () const
 
virtual Bool_t IsEnablePrefetching () const
 
virtual Bool_t IsLearning () const
 
virtual Int_t LearnBranch (TBranch *, Bool_t=kFALSE)
 
virtual void Prefetch (Long64_t pos, Int_t len)
 Add block of length len at position pos in the list of blocks to be prefetched. More...
 
virtual void Print (Option_t *option="") const
 Print cache statistics. More...
 
virtual Int_t ReadBuffer (char *buf, Long64_t pos, Int_t len)
 Read buffer at position pos. More...
 
virtual Int_t ReadBufferExt (char *buf, Long64_t pos, Int_t len, Int_t &loc)
 
virtual Int_t ReadBufferExtNormal (char *buf, Long64_t pos, Int_t len, Int_t &loc)
 Base function for ReadBuffer. More...
 
virtual Int_t ReadBufferExtPrefetch (char *buf, Long64_t pos, Int_t len, Int_t &loc)
 prefetch the first block More...
 
virtual void SecondPrefetch (Long64_t, Int_t)
 
virtual void SecondSort ()
 Sort buffers to be prefetched in increasing order of positions. More...
 
virtual Int_t SetBufferSize (Int_t buffersize)
 Sets the buffer size. More...
 
virtual void SetEnablePrefetching (Bool_t setPrefetching=kFALSE)
 Set the prefetching mode of this file. More...
 
virtual void SetFile (TFile *file, TFile::ECacheAction action=TFile::kDisconnect)
 Set the file using this cache and reset the current blocks (if any). More...
 
virtual void SetSkipZip (Bool_t=kTRUE)
 
virtual void Sort ()
 Sort buffers to be prefetched in increasing order of positions. More...
 
virtual void WaitFinishPrefetch ()
 
- Public Member Functions inherited from TObject
 TObject ()
 TObject constructor. More...
 
 TObject (const TObject &object)
 TObject copy ctor. More...
 
virtual ~TObject ()
 TObject destructor. More...
 
void AbstractMethod (const char *method) const
 Use this method to implement an "abstract" method that you don't want to leave purely abstract. More...
 
virtual void AppendPad (Option_t *option="")
 Append graphics object to current pad. More...
 
virtual void Browse (TBrowser *b)
 Browse object. May be overridden for another default action. More...
 
ULong_t CheckedHash ()
 Check and record whether this class has a consistent Hash/RecursiveRemove setup (*) and then return the regular Hash value for this object. More...
 
virtual const char * ClassName () const
 Returns name of class to which the object belongs. More...
 
virtual void Clear (Option_t *="")
 
virtual TObjectClone (const char *newname="") const
 Make a clone of an object using the Streamer facility. More...
 
virtual Int_t Compare (const TObject *obj) const
 Compare abstract method. More...
 
virtual void Copy (TObject &object) const
 Copy this to obj. More...
 
virtual void Delete (Option_t *option="")
 Delete this object. More...
 
virtual Int_t DistancetoPrimitive (Int_t px, Int_t py)
 Computes distance from point (px,py) to the object. More...
 
virtual void Draw (Option_t *option="")
 Default Draw method for all objects. More...
 
virtual void DrawClass () const
 Draw class inheritance tree of the class to which this object belongs. More...
 
virtual TObjectDrawClone (Option_t *option="") const
 Draw a clone of this object in the current selected pad for instance with: gROOT->SetSelectedPad(gPad). More...
 
virtual void Dump () const
 Dump contents of object on stdout. More...
 
virtual void Error (const char *method, const char *msgfmt,...) const
 Issue error message. More...
 
virtual void Execute (const char *method, const char *params, Int_t *error=0)
 Execute method on this object with the given parameter string, e.g. More...
 
virtual void Execute (TMethod *method, TObjArray *params, Int_t *error=0)
 Execute method on this object with parameters stored in the TObjArray. More...
 
virtual void ExecuteEvent (Int_t event, Int_t px, Int_t py)
 Execute action corresponding to an event at (px,py). More...
 
virtual void Fatal (const char *method, const char *msgfmt,...) const
 Issue fatal error message. More...
 
virtual TObjectFindObject (const char *name) const
 Must be redefined in derived classes. More...
 
virtual TObjectFindObject (const TObject *obj) const
 Must be redefined in derived classes. More...
 
virtual Option_tGetDrawOption () const
 Get option used by the graphics system to draw this object. More...
 
virtual const char * GetIconName () const
 Returns mime type name of object. More...
 
virtual const char * GetName () const
 Returns name of object. More...
 
virtual char * GetObjectInfo (Int_t px, Int_t py) const
 Returns string containing info about the object at position (px,py). More...
 
virtual Option_tGetOption () const
 
virtual const char * GetTitle () const
 Returns title of object. More...
 
virtual UInt_t GetUniqueID () const
 Return the unique object id. More...
 
virtual Bool_t HandleTimer (TTimer *timer)
 Execute action in response of a timer timing out. More...
 
virtual ULong_t Hash () const
 Return hash value for this object. More...
 
Bool_t HasInconsistentHash () const
 Return true is the type of this object is known to have an inconsistent setup for Hash and RecursiveRemove (i.e. More...
 
virtual void Info (const char *method, const char *msgfmt,...) const
 Issue info message. More...
 
virtual Bool_t InheritsFrom (const char *classname) const
 Returns kTRUE if object inherits from class "classname". More...
 
virtual Bool_t InheritsFrom (const TClass *cl) const
 Returns kTRUE if object inherits from TClass cl. More...
 
virtual void Inspect () const
 Dump contents of this object in a graphics canvas. More...
 
void InvertBit (UInt_t f)
 
virtual Bool_t IsEqual (const TObject *obj) const
 Default equal comparison (objects are equal if they have the same address in memory). More...
 
virtual Bool_t IsFolder () const
 Returns kTRUE in case object contains browsable objects (like containers or lists of other objects). More...
 
R__ALWAYS_INLINE Bool_t IsOnHeap () const
 
virtual Bool_t IsSortable () const
 
R__ALWAYS_INLINE Bool_t IsZombie () const
 
virtual void ls (Option_t *option="") const
 The ls function lists the contents of a class on stdout. More...
 
void MayNotUse (const char *method) const
 Use this method to signal that a method (defined in a base class) may not be called in a derived class (in principle against good design since a child class should not provide less functionality than its parent, however, sometimes it is necessary). More...
 
virtual Bool_t Notify ()
 This method must be overridden to handle object notification. More...
 
void Obsolete (const char *method, const char *asOfVers, const char *removedFromVers) const
 Use this method to declare a method obsolete. More...
 
void operator delete (void *ptr)
 Operator delete. More...
 
void operator delete[] (void *ptr)
 Operator delete []. More...
 
voidoperator new (size_t sz)
 
voidoperator new (size_t sz, void *vp)
 
voidoperator new[] (size_t sz)
 
voidoperator new[] (size_t sz, void *vp)
 
TObjectoperator= (const TObject &rhs)
 TObject assignment operator. More...
 
virtual void Paint (Option_t *option="")
 This method must be overridden if a class wants to paint itself. More...
 
virtual void Pop ()
 Pop on object drawn in a pad to the top of the display list. More...
 
virtual void Print (Option_t *option="") const
 This method must be overridden when a class wants to print itself. More...
 
virtual Int_t Read (const char *name)
 Read contents of object with specified name from the current directory. More...
 
virtual void RecursiveRemove (TObject *obj)
 Recursively remove this object from a list. More...
 
void ResetBit (UInt_t f)
 
virtual void SaveAs (const char *filename="", Option_t *option="") const
 Save this object in the file specified by filename. More...
 
virtual void SavePrimitive (std::ostream &out, Option_t *option="")
 Save a primitive as a C++ statement(s) on output stream "out". More...
 
void SetBit (UInt_t f)
 
void SetBit (UInt_t f, Bool_t set)
 Set or unset the user status bits as specified in f. More...
 
virtual void SetDrawOption (Option_t *option="")
 Set drawing option for object. More...
 
virtual void SetUniqueID (UInt_t uid)
 Set the unique object id. More...
 
virtual void SysError (const char *method, const char *msgfmt,...) const
 Issue system error message. More...
 
R__ALWAYS_INLINE Bool_t TestBit (UInt_t f) const
 
Int_t TestBits (UInt_t f) const
 
virtual void UseCurrentStyle ()
 Set current style settings in this object This function is called when either TCanvas::UseCurrentStyle or TROOT::ForceStyle have been invoked. More...
 
virtual void Warning (const char *method, const char *msgfmt,...) const
 Issue warning message. More...
 
virtual Int_t Write (const char *name=0, Int_t option=0, Int_t bufsize=0)
 Write this object to the current directory. More...
 
virtual Int_t Write (const char *name=0, Int_t option=0, Int_t bufsize=0) const
 Write this object to the current directory. More...
 

Static Public Member Functions

static EParUnzipMode GetParallelUnzip ()
 Static function that returns the parallel option (to indicate an additional thread) More...
 
static Bool_t IsParallelUnzip ()
 Static function that tells wether the multithreading unzipping is activated. More...
 
static Int_t SetParallelUnzip (TTreeCacheUnzip::EParUnzipMode option=TTreeCacheUnzip::kEnable)
 Static function that (de)activates multithreading unzipping. More...
 
static void SetUnzipRelBufferSize (Float_t relbufferSize)
 static function: Sets the unzip relatibe buffer size More...
 
- Static Public Member Functions inherited from TTreeCache
static Int_t GetLearnEntries ()
 Static function returning the number of entries used to train the cache see SetLearnEntries. More...
 
static void SetLearnEntries (Int_t n=10)
 Static function to set the number of entries to be used in learning mode The default value for n is 10. More...
 
- Static Public Member Functions inherited from TObject
static Long_t GetDtorOnly ()
 Return destructor only flag. More...
 
static Bool_t GetObjectStat ()
 Get status of object stat flag. More...
 
static void SetDtorOnly (void *obj)
 Set destructor only flag. More...
 
static void SetObjectStat (Bool_t stat)
 Turn on/off tracking of objects in the TObjectTable. More...
 

Protected Types

typedef struct UnzipState UnzipState_t
 

Protected Attributes

Bool_t fAsyncReading
 
Int_t fCycle
 
Bool_t fEmpty
 
std::unique_ptr< TMutexfIOMutex
 
Int_t fNFound
 ! number of blocks that were found in the cache More...
 
Int_t fNMissed
 ! number of blocks that were not found in the cache and were unzipped More...
 
Int_t fNseekMax
 ! fNseek can change so we need to know its max size More...
 
Int_t fNStalls
 ! number of hits which caused a stall More...
 
Int_t fNUnzip
 ! number of blocks that were unzipped More...
 
Bool_t fParallel
 Indicate if we want to activate the parallelism (for this instance) More...
 
Long64_t fUnzipBufferSize
 ! Max Size for the ready unzipped blocks (default is 2*fBufferSize) More...
 
Int_t fUnzipGroupSize
 ! Min accumulated size of a group of baskets ready to be unzipped by a IMT task More...
 
UnzipState_t fUnzipState
 
std::unique_ptr< ROOT::Experimental::TTaskGroupfUnzipTaskGroup
 
- Protected Attributes inherited from TTreeCache
Bool_t fAutoCreated {kFALSE}
 ! true if cache was automatically created More...
 
TObjArrayfBranches {nullptr}
 ! List of branches to be stored in the cache More...
 
TListfBrNames {nullptr}
 ! list of branch names in the cache More...
 
Long64_t fCurrentClusterStart {-1}
 ! Start of the cluster(s) where the current content was picked out More...
 
Bool_t fEnabled {kTRUE}
 ! cache enabled for cached reading More...
 
Long64_t fEntryCurrent {-1}
 ! current lowest entry number in the cache More...
 
Long64_t fEntryMax {1}
 ! last entry in the cache More...
 
Long64_t fEntryMin {0}
 ! first entry in the cache More...
 
Long64_t fEntryNext {-1}
 ! next entry number where cache must be filled More...
 
Int_t fFillTimes {0}
 ! how many times we can fill the current buffer More...
 
Bool_t fFirstBuffer {kTRUE}
 ! true if first buffer is used for prefetching More...
 
Long64_t fFirstEntry {-1}
 ! save the value of the first entry More...
 
Long64_t fFirstMiss {-1}
 ! set to the event # of the first miss. More...
 
Bool_t fFirstTime {kTRUE}
 ! save the fact that we processes the first entry More...
 
Bool_t fIsLearning {kTRUE}
 ! true if cache is in learning mode More...
 
Bool_t fIsManual {kFALSE}
 ! true if cache is StopLearningPhase was used More...
 
Long64_t fLastMiss {-1}
 ! set to the event # of the last miss. More...
 
Bool_t fLearnPrefilling {kFALSE}
 ! true if we are in the process of executing LearnPrefill More...
 
std::unique_ptr< MissCachefMissCache
 ! Cache contents for misses More...
 
Int_t fNbranches {0}
 ! Number of branches in the cache More...
 
Long64_t fNextClusterStart {-1}
 ! End+1 of the cluster(s) where the current content was picked out More...
 
Int_t fNMissReadMiss {0}
 Number of blocks read and not found in either cache. More...
 
Int_t fNMissReadOk {0}
 Number of blocks read, not found in the primary cache, and found in the secondary cache. More...
 
Int_t fNMissReadPref {0}
 Number of blocks read into the secondary ("miss") cache. More...
 
Int_t fNReadMiss {0}
 Number of blocks read and not found in the cache. More...
 
Int_t fNReadOk {0}
 Number of blocks read and found in the cache. More...
 
Int_t fNReadPref {0}
 Number of blocks that were prefetched. More...
 
Bool_t fOneTime {kFALSE}
 ! used in the learning phase More...
 
Bool_t fOptimizeMisses {kFALSE}
 ! true if we should optimize cache misses. More...
 
EPrefillType fPrefillType
 Whether a pre-filling is enabled (and if applicable which type) More...
 
Bool_t fReadDirectionSet {kFALSE}
 ! read direction established More...
 
Bool_t fReverseRead {kFALSE}
 ! reading in reverse mode More...
 
TTreefTree {nullptr}
 ! pointer to the current Tree More...
 
- Protected Attributes inherited from TFileCacheRead
Bool_t fAsyncReading
 
Bool_t fBIsSorted
 
Bool_t fBIsTransferred
 
Int_tfBLen
 [fBNb] More...
 
Int_t fBNb
 
Int_t fBNseek
 
Int_t fBNtot
 
Long64_tfBPos
 [fBNb] More...
 
Long64_tfBSeek
 [fBNseek] More...
 
Int_tfBSeekIndex
 [fBNseek] More...
 
Int_tfBSeekLen
 [fBNseek] More...
 
Int_tfBSeekPos
 [fBNseek] More...
 
Int_t fBSeekSize
 
Long64_tfBSeekSort
 [fBNseek] More...
 
Int_tfBSeekSortLen
 [fBNseek] More...
 
char * fBuffer
 [fBufferSize] buffer of contiguous prefetched blocks More...
 
Int_t fBufferLen
 Current buffer length (<= fBufferSize) More...
 
Int_t fBufferSize
 Allocated size of fBuffer (at a given time) More...
 
Int_t fBufferSizeMin
 Original size of fBuffer. More...
 
Long64_t fBytesRead
 Number of bytes read for this cache. More...
 
Long64_t fBytesReadExtra
 Number of extra bytes (overhead) read by the readahead buffer. More...
 
Bool_t fEnablePrefetching
 reading by prefetching asynchronously More...
 
TFilefFile
 Pointer to file. More...
 
Bool_t fIsSorted
 True if fSeek array is sorted. More...
 
Bool_t fIsTransferred
 True when fBuffer contains something valid. More...
 
Int_tfLen
 [fNb] Length of long buffers More...
 
Int_t fNb
 Number of long buffers. More...
 
Long64_t fNoCacheBytesRead
 Number of bytes read by basket to fill cached tree. More...
 
Int_t fNoCacheReadCalls
 Number of read calls by basket to fill cached tree. More...
 
Int_t fNseek
 Number of blocks to be prefetched. More...
 
Int_t fNtot
 Total size of prefetched blocks. More...
 
Long64_tfPos
 [fNb] start of long buffers More...
 
TFilePrefetchfPrefetch
 ! Object that does the asynchronous reading in another thread More...
 
Long64_t fPrefetchedBlocks
 Number of blocks prefetched. More...
 
Int_t fReadCalls
 Number of read calls for this cache. More...
 
Long64_tfSeek
 [fNseek] Position on file of buffers to be prefetched More...
 
Int_tfSeekIndex
 [fNseek] sorted index table of fSeek More...
 
Int_tfSeekLen
 [fNseek] Length of buffers to be prefetched More...
 
Int_tfSeekPos
 [fNseek] Position of sorted blocks in fBuffer More...
 
Int_t fSeekSize
 Allocated size of fSeek. More...
 
Long64_tfSeekSort
 [fNseek] Position on file of buffers to be prefetched (sorted) More...
 
Int_tfSeekSortLen
 [fNseek] Length of buffers to be prefetched (sorted) More...
 

Static Protected Attributes

static TTreeCacheUnzip::EParUnzipMode fgParallel = TTreeCacheUnzip::kDisable
 Indicate if we want to activate the parallelism. More...
 
static Double_t fgRelBuffSize = .5
 This is the percentage of the TTreeCacheUnzip that will be used. More...
 
- Static Protected Attributes inherited from TTreeCache
static Int_t fgLearnEntries = 100
 number of entries used for learning mode More...
 

Private Member Functions

 TTreeCacheUnzip (const TTreeCacheUnzip &)
 
void Init ()
 Initialization procedure common to all the constructors. More...
 
TTreeCacheUnzipoperator= (const TTreeCacheUnzip &)
 

Private Attributes

char * fCompBuffer
 
Int_t fCompBufferSize
 

Additional Inherited Members

- Protected Member Functions inherited from TFileCacheRead
void SetEnablePrefetchingImpl (Bool_t setPrefetching=kFALSE)
 TFileCacheRead implementation of SetEnablePrefetching. More...
 
- Protected Member Functions inherited from TObject
virtual void DoError (int level, const char *location, const char *fmt, va_list va) const
 Interface to ErrorHandler (protected). More...
 
void MakeZombie ()
 

#include <TTreeCacheUnzip.h>

Inheritance diagram for TTreeCacheUnzip:
[legend]

Member Typedef Documentation

◆ UnzipState_t

typedef struct UnzipState TTreeCacheUnzip::UnzipState_t
protected

Definition at line 77 of file TTreeCacheUnzip.h.

Member Enumeration Documentation

◆ EParUnzipMode

Enumerator
kEnable 
kDisable 
kForce 

Definition at line 40 of file TTreeCacheUnzip.h.

◆ EUnzipState

Enumerator
kUntouched 
kProgress 
kFinished 

Definition at line 43 of file TTreeCacheUnzip.h.

Constructor & Destructor Documentation

◆ TTreeCacheUnzip() [1/3]

TTreeCacheUnzip::TTreeCacheUnzip ( const TTreeCacheUnzip )
private

◆ TTreeCacheUnzip() [2/3]

TTreeCacheUnzip::TTreeCacheUnzip ( )

Definition at line 157 of file TTreeCacheUnzip.cxx.

◆ TTreeCacheUnzip() [3/3]

TTreeCacheUnzip::TTreeCacheUnzip ( TTree tree,
Int_t  buffersize = 0 
)

Constructor.

Definition at line 176 of file TTreeCacheUnzip.cxx.

◆ ~TTreeCacheUnzip()

TTreeCacheUnzip::~TTreeCacheUnzip ( )
virtual

Destructor. (in general called by the TFile destructor)

Definition at line 233 of file TTreeCacheUnzip.cxx.

Member Function Documentation

◆ AddBranch() [1/2]

Int_t TTreeCacheUnzip::AddBranch ( const char *  branch,
Bool_t  subbranches = kFALSE 
)
virtual

Add a branch to the list of branches to be stored in the cache this function is called by TBranch::GetBasket Returns:

  • 0 branch added or already included
  • -1 on error

Reimplemented from TTreeCache.

Definition at line 258 of file TTreeCacheUnzip.cxx.

◆ AddBranch() [2/2]

Int_t TTreeCacheUnzip::AddBranch ( TBranch b,
Bool_t  subbranches = kFALSE 
)
virtual

Add a branch to the list of branches to be stored in the cache this function is called by TBranch::GetBasket Returns:

  • 0 branch added or already included
  • -1 on error

Reimplemented from TTreeCache.

Definition at line 246 of file TTreeCacheUnzip.cxx.

◆ CreateTasks()

Int_t TTreeCacheUnzip::CreateTasks ( )

We create a TTaskGroup and asynchronously maps each group of baskets(> 100 kB in total) to a task.

In TTaskGroup, we use TThreadExecutor to do the actually work of unzipping a group of basket. The purpose of creating TTaskGroup is to avoid competing with main thread.

Definition at line 609 of file TTreeCacheUnzip.cxx.

◆ FillBuffer()

Bool_t TTreeCacheUnzip::FillBuffer ( )
virtual

Fill the cache buffer with the branches in the cache.

Reimplemented from TTreeCache.

Definition at line 265 of file TTreeCacheUnzip.cxx.

◆ GetNFound()

Int_t TTreeCacheUnzip::GetNFound ( )
inline

Definition at line 154 of file TTreeCacheUnzip.h.

◆ GetNMissed()

Int_t TTreeCacheUnzip::GetNMissed ( )
inline

Definition at line 153 of file TTreeCacheUnzip.h.

◆ GetNUnzip()

Int_t TTreeCacheUnzip::GetNUnzip ( )
inline

Definition at line 152 of file TTreeCacheUnzip.h.

◆ GetParallelUnzip()

TTreeCacheUnzip::EParUnzipMode TTreeCacheUnzip::GetParallelUnzip ( )
static

Static function that returns the parallel option (to indicate an additional thread)

Definition at line 405 of file TTreeCacheUnzip.cxx.

◆ GetRecordHeader()

Int_t TTreeCacheUnzip::GetRecordHeader ( char *  buf,
Int_t  maxbytes,
Int_t nbytes,
Int_t objlen,
Int_t keylen 
)

Read the logical record header from the buffer buf.

That must be the pointer tho the header part not the object by itself and must contain data of at least maxbytes Returns nread;

In output arguments:

  • nbytes : number of bytes in record if negative, this is a deleted record if 0, cannot read record, wrong value of argument first
  • objlen : uncompressed object size
  • keylen : length of logical record header

Note that the arguments objlen and keylen are returned only if maxbytes >=16 Note: This was adapted from TFile... so some things dont apply

Definition at line 467 of file TTreeCacheUnzip.cxx.

◆ GetUnzipBuffer()

Int_t TTreeCacheUnzip::GetUnzipBuffer ( char **  buf,
Long64_t  pos,
Int_t  len,
Bool_t free 
)
virtual

We try to read a buffer that has already been unzipped Returns -1 in case of read failure, 0 in case it's not in the cache and n>0 in case read from cache (number of bytes copied).

pos and len are the original values as were passed to ReadBuffer but instead we will return the inflated buffer. Note!! : If *buf == 0 we will allocate the buffer and it will be the responsability of the caller to free it... it is useful for example to pass it to the creator of TBuffer

Reimplemented from TFileCacheRead.

Definition at line 664 of file TTreeCacheUnzip.cxx.

◆ GetUnzipGroupSize()

Int_t TTreeCacheUnzip::GetUnzipGroupSize ( )
inline

Definition at line 142 of file TTreeCacheUnzip.h.

◆ Init()

void TTreeCacheUnzip::Init ( void  )
private

Initialization procedure common to all the constructors.

Definition at line 194 of file TTreeCacheUnzip.cxx.

◆ IsParallelUnzip()

Bool_t TTreeCacheUnzip::IsParallelUnzip ( )
static

Static function that tells wether the multithreading unzipping is activated.

Definition at line 413 of file TTreeCacheUnzip.cxx.

◆ operator=()

TTreeCacheUnzip & TTreeCacheUnzip::operator= ( const TTreeCacheUnzip )
private

◆ Print()

void TTreeCacheUnzip::Print ( Option_t option = "") const
virtual

Print cache statistics.

Like:

******TreeCache statistics for file: cms2.root ******
Number of branches in the cache ...: 1093
Cache Efficiency ..................: 0.997372
Cache Efficiency Rel...............: 1.000000
Learn entries......................: 100
Reading............................: 72761843 bytes in 7 transactions
Readahead..........................: 256000 bytes with overhead = 0 bytes
Average transaction................: 10394.549000 Kbytes
Number of blocks in current cache..: 210, total size: 6280352
static unsigned int total
RooCmdArg Efficiency(const RooCategory &cat)
Definition: file.py:1
void statistics(struct mesh *m, struct behavior *b)
Definition: triangle.c:15532
  • if option = "a" the list of blocks in the cache is printed see also class TTreePerfStats.
  • if option contains 'cachedbranches', the list of branches being cached is printed.

Reimplemented from TTreeCache.

Definition at line 943 of file TTreeCacheUnzip.cxx.

◆ ReadBufferExt()

Int_t TTreeCacheUnzip::ReadBufferExt ( char *  buf,
Long64_t  pos,
Int_t  len,
Int_t loc 
)
virtual

Reimplemented from TFileCacheRead.

Definition at line 957 of file TTreeCacheUnzip.cxx.

◆ ResetCache()

void TTreeCacheUnzip::ResetCache ( )
virtual

This will delete the list of buffers that are in the unzipping cache and will reset certain values in the cache.

This name is ambiguos because the method doesn't reset the whole cache, only the part related to the unzipping Note: This method is completely different from TTreeCache::ResetCache(), in that method we were cleaning the prefetching buffer while here we delete the information about the unzipped buffers

Reimplemented from TTreeCache.

Definition at line 499 of file TTreeCacheUnzip.cxx.

◆ SetBufferSize()

Int_t TTreeCacheUnzip::SetBufferSize ( Int_t  buffersize)
virtual

Change the underlying buffer size of the cache.

Returns:

  • 0 if the buffer content is still available
  • 1 if some or all of the buffer content has been made unavailable
  • -1 on error

Reimplemented from TTreeCache.

Definition at line 357 of file TTreeCacheUnzip.cxx.

◆ SetEntryRange()

void TTreeCacheUnzip::SetEntryRange ( Long64_t  emin,
Long64_t  emax 
)
virtual

Set the minimum and maximum entry number to be processed this information helps to optimize the number of baskets to read when prefetching the branch buffers.

Reimplemented from TTreeCache.

Definition at line 373 of file TTreeCacheUnzip.cxx.

◆ SetParallelUnzip()

Int_t TTreeCacheUnzip::SetParallelUnzip ( TTreeCacheUnzip::EParUnzipMode  option = TTreeCacheUnzip::kEnable)
static

Static function that (de)activates multithreading unzipping.

The possible options are:

  • kEnable Enable it, which causes an automatic detection and launches the additional thread if the number of cores in the machine is greater than one
  • kDisable Disable will not activate the additional thread.
  • kForce Force will start the additional thread even if there is only one core. the default will be taken as kEnable.

Returns 0 if there was an error, 1 otherwise.

Definition at line 434 of file TTreeCacheUnzip.cxx.

◆ SetUnzipBufferSize()

void TTreeCacheUnzip::SetUnzipBufferSize ( Long64_t  bufferSize)

Sets the size for the unzipping cache... by default it should be two times the size of the prefetching cache.

Definition at line 835 of file TTreeCacheUnzip.cxx.

◆ SetUnzipGroupSize()

void TTreeCacheUnzip::SetUnzipGroupSize ( Int_t  groupSize)
inline

Definition at line 146 of file TTreeCacheUnzip.h.

◆ SetUnzipRelBufferSize()

void TTreeCacheUnzip::SetUnzipRelBufferSize ( Float_t  relbufferSize)
static

static function: Sets the unzip relatibe buffer size

Definition at line 826 of file TTreeCacheUnzip.cxx.

◆ StopLearningPhase()

void TTreeCacheUnzip::StopLearningPhase ( )
virtual

It's the same as TTreeCache::StopLearningPhase but we guarantee that we start the unzipping just after getting the buffers.

Reimplemented from TTreeCache.

Definition at line 382 of file TTreeCacheUnzip.cxx.

◆ UnzipBuffer()

Int_t TTreeCacheUnzip::UnzipBuffer ( char **  dest,
char *  src 
)

Unzips a ROOT specific buffer... by reading the header at the beginning.

returns the size of the inflated buffer or -1 if error Note!! : If *dest == 0 we will allocate the buffer and it will be the responsability of the caller to free it... it is useful for example to pass it to the creator of TBuffer src is the original buffer with the record (header+compressed data) *dest is the inflated buffer (including the header)

Definition at line 849 of file TTreeCacheUnzip.cxx.

◆ UnzipCache()

Int_t TTreeCacheUnzip::UnzipCache ( Int_t  index)

This inflates a basket in the cache.

. passing the data to a new buffer that will only wait there to be read... This function is responsible to update corresponding elements in fUnzipStatus, fUnzipChunks and fUnzipLen. Since we use atomic variables in fUnzipStatus to exclusively unzip the basket, we must update fUnzipStatus after fUnzipChunks and fUnzipLen and make sure fUnzipChunks and fUnzipLen are ready before main thread fetch the data.

Definition at line 524 of file TTreeCacheUnzip.cxx.

◆ UpdateBranches()

void TTreeCacheUnzip::UpdateBranches ( TTree tree)
virtual

update pointer to current Tree and recompute pointers to the branches in the cache

Reimplemented from TTreeCache.

Definition at line 390 of file TTreeCacheUnzip.cxx.

Member Data Documentation

◆ fAsyncReading

Bool_t TTreeCacheUnzip::fAsyncReading
protected

Definition at line 81 of file TTreeCacheUnzip.h.

◆ fCompBuffer

char* TTreeCacheUnzip::fCompBuffer
private

Definition at line 112 of file TTreeCacheUnzip.h.

◆ fCompBufferSize

Int_t TTreeCacheUnzip::fCompBufferSize
private

Definition at line 113 of file TTreeCacheUnzip.h.

◆ fCycle

Int_t TTreeCacheUnzip::fCycle
protected

Definition at line 83 of file TTreeCacheUnzip.h.

◆ fEmpty

Bool_t TTreeCacheUnzip::fEmpty
protected

Definition at line 82 of file TTreeCacheUnzip.h.

◆ fgParallel

TTreeCacheUnzip::EParUnzipMode TTreeCacheUnzip::fgParallel = TTreeCacheUnzip::kDisable
staticprotected

Indicate if we want to activate the parallelism.

Definition at line 88 of file TTreeCacheUnzip.h.

◆ fgRelBuffSize

Double_t TTreeCacheUnzip::fgRelBuffSize = .5
staticprotected

This is the percentage of the TTreeCacheUnzip that will be used.

Definition at line 100 of file TTreeCacheUnzip.h.

◆ fIOMutex

std::unique_ptr<TMutex> TTreeCacheUnzip::fIOMutex
protected

Definition at line 86 of file TTreeCacheUnzip.h.

◆ fNFound

Int_t TTreeCacheUnzip::fNFound
protected

! number of blocks that were found in the cache

Definition at line 103 of file TTreeCacheUnzip.h.

◆ fNMissed

Int_t TTreeCacheUnzip::fNMissed
protected

! number of blocks that were not found in the cache and were unzipped

Definition at line 104 of file TTreeCacheUnzip.h.

◆ fNseekMax

Int_t TTreeCacheUnzip::fNseekMax
protected

! fNseek can change so we need to know its max size

Definition at line 96 of file TTreeCacheUnzip.h.

◆ fNStalls

Int_t TTreeCacheUnzip::fNStalls
protected

! number of hits which caused a stall

Definition at line 105 of file TTreeCacheUnzip.h.

◆ fNUnzip

Int_t TTreeCacheUnzip::fNUnzip
protected

! number of blocks that were unzipped

Definition at line 106 of file TTreeCacheUnzip.h.

◆ fParallel

Bool_t TTreeCacheUnzip::fParallel
protected

Indicate if we want to activate the parallelism (for this instance)

Definition at line 84 of file TTreeCacheUnzip.h.

◆ fUnzipBufferSize

Long64_t TTreeCacheUnzip::fUnzipBufferSize
protected

! Max Size for the ready unzipped blocks (default is 2*fBufferSize)

Definition at line 98 of file TTreeCacheUnzip.h.

◆ fUnzipGroupSize

Int_t TTreeCacheUnzip::fUnzipGroupSize
protected

! Min accumulated size of a group of baskets ready to be unzipped by a IMT task

Definition at line 97 of file TTreeCacheUnzip.h.

◆ fUnzipState

UnzipState_t TTreeCacheUnzip::fUnzipState
protected

Definition at line 78 of file TTreeCacheUnzip.h.

◆ fUnzipTaskGroup

std::unique_ptr<ROOT::Experimental::TTaskGroup> TTreeCacheUnzip::fUnzipTaskGroup
protected

Definition at line 92 of file TTreeCacheUnzip.h.

Libraries for TTreeCacheUnzip:
[legend]

The documentation for this class was generated from the following files: