TDataSource defines an API that TDataFrame can use to read arbitrary data formats.

A concrete TDataSource implementation (i.e. a class that inherits from TDataSource and implements all of its pure methods) provides an adaptor that TDataFrame can leverage to read any kind of tabular data formats. TDataFrame calls into TDataSource to retrieve information about the data, retrieve (thread-local) readers or "cursors" for selected columns and to advance the readers to the desired data entry.

The sequence of calls that TDataFrame (or any other client of a TDataSource) performs is the following:

1) SetNSlots: inform TDataSource of the desired level of parallelism 2) GetColumnReaders: retrieve from TDataSource per-thread readers for the desired columns 3) Initialise: inform TDataSource that an event-loop is about to start 4) GetEntryRanges: retrieve from TDataSource a set of ranges of entries that can be processed concurrently 5) InitSlot: inform TDataSource that a certain thread is about to start working on a certain range of entries 6) SetEntry: inform TDataSource that a certain thread is about to start working on a certain entry 7) FinaliseSlot: inform TDataSource that a certain thread finished working on a certain range of entries 8) Finalise: inform TDataSource that an event-loop finished

TDataSource implementations must support running multiple event-loops consecutively (although sequentially) on the same dataset. Method 1 is called once per TDataSource object, typically when it is associated to a TDataFrame. Method 2 can be called several times, potentially with the same arguments, also in-between event-loops, but not during an event-loop. Methods 3,8 are called once per event-loop, right before starting and right after finishing. Methods 5,6,7 can be called concurrently from multiple threads, multiple times per event-loop.

Definition at line 51 of file TDataSource.hxx.

Public Member Functions
virtual	~TDataSource ()=default

virtual void	Finalise ()
	Convenience method called after concluding an event-loop. More...

virtual void	FinaliseSlot (unsigned int)
	Convenience method called at the end of the data processing associated to a slot. More...

virtual const std::vector< std::string > &	GetColumnNames () const =0
	Returns a reference to the collection of the dataset's column names. More...

template<typename T >
std::vector< T ** >	GetColumnReaders (std::string_view columnName)
	Called at most once per column by TDF. More...

virtual std::vector< std::pair< ULong64_t, ULong64_t > >	GetEntryRanges ()=0
	Return ranges of entries to distribute to tasks. More...

virtual std::string	GetTypeName (std::string_view) const =0
	Type of a column as a string, e.g. More...

virtual bool	HasColumn (std::string_view) const =0
	Checks if the dataset has a certain column. More...

virtual void	Initialise ()
	Convenience method called before starting an event-loop. More...

virtual void	InitSlot (unsigned int, ULong64_t)
	Convenience method called at the start of the data processing associated to a slot. More...

virtual void	SetEntry (unsigned int slot, ULong64_t entry)=0
	Advance the "cursors" returned by GetColumnReaders to the selected entry for a particular slot. More...

virtual void	SetNSlots (unsigned int nSlots)=0
	Inform TDataSource of the number of processing slots (i.e. More...

Protected Member Functions
virtual std::vector< void * >	GetColumnReadersImpl (std::string_view name, const std::type_info &)=0
	type-erased vector of pointers to pointers to column values - one per slot More...

#include <ROOT/TDataSource.hxx>

Inheritance diagram for ROOT::Experimental::TDF::TDataSource:

[legend]

Constructor & Destructor Documentation

◆ ~TDataSource()

virtual ROOT::Experimental::TDF::TDataSource::~TDataSource ( )

virtualdefault

Member Function Documentation

◆ Finalise()

virtual void ROOT::Experimental::TDF::TDataSource::Finalise ( )

inlinevirtual

Convenience method called after concluding an event-loop.

See Initialise for more details.

Definition at line 119 of file TDataSource.hxx.

◆ FinaliseSlot()

virtual void ROOT::Experimental::TDF::TDataSource::FinaliseSlot ( unsigned int )

inlinevirtual

Convenience method called at the end of the data processing associated to a slot.

Parameters

[in] slot The data processing slot wihch needs to be finalised This method might be called multiple times per thread per event-loop.

Definition at line 115 of file TDataSource.hxx.

◆ GetColumnNames()

virtual const std::vector<std::string>& ROOT::Experimental::TDF::TDataSource::GetColumnNames ( ) const

pure virtual

Returns a reference to the collection of the dataset's column names.

Implemented in ROOT::Experimental::TDF::TCsvDS, ROOT::Experimental::TDF::TRootDS, and ROOT::Experimental::TDF::TTrivialDS.

◆ GetColumnReaders()

template<typename T >

std::vector<T **> ROOT::Experimental::TDF::TDataSource::GetColumnReaders ( std::string_view columnName )

inline

Called at most once per column by TDF.

Return vector of pointers to pointers to column values - one per slot.

Template Parameters

T	The type of the data stored in the column

Parameters

[in] columnName The name of the column

These pointers are veritable cursors: it's a responsibility of the TDataSource implementation that they point to the "right" memory region.

Definition at line 78 of file TDataSource.hxx.

◆ GetColumnReadersImpl()

virtual std::vector<void *> ROOT::Experimental::TDF::TDataSource::GetColumnReadersImpl	(	std::string_view	name,
		const std::type_info &
	)

protectedpure virtual

type-erased vector of pointers to pointers to column values - one per slot

Implemented in ROOT::Experimental::TDF::TCsvDS, ROOT::Experimental::TDF::TRootDS, and ROOT::Experimental::TDF::TTrivialDS.

◆ GetEntryRanges()

virtual std::vector<std::pair<ULong64_t, ULong64_t> > ROOT::Experimental::TDF::TDataSource::GetEntryRanges ( )

pure virtual

Return ranges of entries to distribute to tasks.

They are required to be contiguous intervals with no entries skipped. Supposing a dataset with nEntries, the intervals must start at 0 and end at nEntries, e.g. [0-5],[5-10] for 10 entries.

Implemented in ROOT::Experimental::TDF::TCsvDS, ROOT::Experimental::TDF::TRootDS, and ROOT::Experimental::TDF::TTrivialDS.

◆ GetTypeName()

virtual std::string ROOT::Experimental::TDF::TDataSource::GetTypeName ( std::string_view ) const

pure virtual

Type of a column as a string, e.g.

GetTypeName("x") == "double". Required for jitting e.g. df.Filter("x>0").

Parameters

[in] columnName The name of the column

Implemented in ROOT::Experimental::TDF::TCsvDS, ROOT::Experimental::TDF::TRootDS, and ROOT::Experimental::TDF::TTrivialDS.

◆ HasColumn()

virtual bool ROOT::Experimental::TDF::TDataSource::HasColumn ( std::string_view ) const

pure virtual

Checks if the dataset has a certain column.

Parameters

[in] columnName The name of the column

Implemented in ROOT::Experimental::TDF::TCsvDS, ROOT::Experimental::TDF::TRootDS, and ROOT::Experimental::TDF::TTrivialDS.

◆ Initialise()

virtual void ROOT::Experimental::TDF::TDataSource::Initialise ( )

inlinevirtual

Convenience method called before starting an event-loop.

This method might be called multiple times over the lifetime of a TDataSource, since users can run multiple event-loops with the same TDataFrame. Ideally, Initialise should set the state of the TDataSource so that multiple identical event-loops will produce identical results.

Reimplemented in ROOT::Experimental::TDF::TCsvDS, ROOT::Experimental::TDF::TRootDS, and ROOT::Experimental::TDF::TTrivialDS.

Definition at line 104 of file TDataSource.hxx.

◆ InitSlot()

virtual void ROOT::Experimental::TDF::TDataSource::InitSlot	(	unsigned int	,
		ULong64_t
	)

inlinevirtual

Convenience method called at the start of the data processing associated to a slot.

Parameters

[in]	slot	The data processing slot wihch needs to be initialised
[in]	firstEntry	The first entry of the range that the task will process. This method might be called multiple times per thread per event-loop.

Reimplemented in ROOT::Experimental::TDF::TRootDS.

Definition at line 110 of file TDataSource.hxx.

◆ SetEntry()

virtual void ROOT::Experimental::TDF::TDataSource::SetEntry	(	unsigned int	slot,
		ULong64_t	entry
	)

pure virtual

Advance the "cursors" returned by GetColumnReaders to the selected entry for a particular slot.

Parameters

[in]	slot	The data processing slot that needs to be considered
[in]	entry	The entry which needs to be pointed to by the reader pointers Slots are adopted to accommodate parallel data processing. Different workers will loop over different ranges and will be labelled by different "slot" values.

Implemented in ROOT::Experimental::TDF::TCsvDS, ROOT::Experimental::TDF::TRootDS, and ROOT::Experimental::TDF::TTrivialDS.

◆ SetNSlots()

virtual void ROOT::Experimental::TDF::TDataSource::SetNSlots ( unsigned int nSlots )

pure virtual

Inform TDataSource of the number of processing slots (i.e.

worker threads) used by the associated TDataFrame. Slots numbers are used to simplify parallel execution: TDataFrame guarantees that different threads will always pass different slot values when calling methods concurrently.

Implemented in ROOT::Experimental::TDF::TCsvDS, ROOT::Experimental::TDF::TRootDS, and ROOT::Experimental::TDF::TTrivialDS.

The documentation for this class was generated from the following file:

tree/treeplayer/inc/ROOT/TDataSource.hxx

Public Member Functions

Protected Member Functions

Constructor & Destructor Documentation

◆ ~TDataSource()

Member Function Documentation

◆ Finalise()

◆ FinaliseSlot()

◆ GetColumnNames()

◆ GetColumnReaders()

◆ GetColumnReadersImpl()

◆ GetEntryRanges()

◆ GetTypeName()

◆ HasColumn()

◆ Initialise()

◆ InitSlot()

◆ SetEntry()

◆ SetNSlots()