I/O
Asynchronous Prefetching
- The prefetching mechanism uses two new classes (TFilePrefetch
and TFPBlock) to prefetch in advance a block of tree entries. There
is a thread which takes care of actually transferring the blocks and
making them available to the main requesting thread. Therefore, the time
spent by the main thread waiting for the data before processing considerably
decreases. Besides the prefetching mechanisms there is also a local
caching option which can be enabled by the user. Both capabilities are
disabled by default and must be explicitly enabled by the user.
- In order to enable the prefetching the user must set the rootrc environment
variable TFile.AsyncPrefetching as follows:
gEnv->SetValue("TFile.AsyncPrefetching", 1). Only when the
prefetching is enabled can the user set the local cache directory in
which the file transferred will be saved. For subsequent reads of the
same file the system will use the local copy of the file from cache.
To set up a local cache directory, the client can use the following commands:
TString cachedir="file:/tmp/xcache/";
// or using xrootd on port 2000
// TString cachedir="root://localhost:2000//tmp/xrdcache1/";
gEnv->SetValue("Cache.Directory", cachedir.Data());
The TFilePrefetch class is responsible for actually reading
and storing the requests received from the main thread. It also creates
the working thread which will transfer all the information. Apart from
managing the block requests, it also deals with caching the blocks on
the local machine and retrieving them when necessary.
The TFPBlock class represents the encapsulation of a block
request. It contains the chunks to be prefetched and also serves as a
container for the information read.
In order to accommodate the new prefetching mechanism the following
files have been modified: TFileCacheRead.cxx/.h,
TTreeCache.cxx/.h and to a less extent TXNetFile.cxx,
TFile.h. Basically in TFileCacheRead we've added the
logic for dealing with the second buffer that is prefetched. In
TTreeCache during prefetching the method FillBuffer is called
after each read so that once the main thread starts reading from the
beginning of the last available buffer, the second thread starts
prefetching the next block.
The TZIPFile compressed archive reader now supports the Zip64 format for
archives and file members greater than 2 and 4 GB, respectively.