Logo ROOT  
Reference Guide
 
Loading...
Searching...
No Matches
ROOT::RDF::RSqliteDS Class Referencefinal

RSqliteDS is an RDF data source implementation for SQL result sets from sqlite3 files.

The RSqliteDS is able to feed an RDataFrame with data from a SQlite SELECT query. One can use it like

auto rdf = ROOT::RDF::FromSqlite("/path/to/file.sqlite", "select name from table");
auto h = rdf.Define("lName", "name.length()").Histo1D("lName");

The data source has to provide column types for all the columns. Determining column types in SQlite is tricky as it is dynamically typed and in principle each row can have different column types. The following heuristics is used:

  • If a table column is queried as is ("SELECT colname FROM table"), the default/declared column type is taken.
  • For expressions ("SELECT 1+1 FROM table"), the type of the first row of the result set determines the column type. That can result in a column to be of thought of type NULL where subsequent rows actually have meaningful values. The provided SELECT query can be used to avoid such ambiguities.

Definition at line 51 of file RSqliteDS.hxx.

Classes

struct  Value_t
 Used to hold a single "cell" of the SELECT query's result table. Can be changed to std::variant once available. More...
 

Public Member Functions

 RSqliteDS (const std::string &fileName, const std::string &query)
 Build the dataframe.
 
 ~RSqliteDS ()
 Frees the sqlite resources and closes the file.
 
const std::vector< std::string > & GetColumnNames () const final
 Returns the SELECT queries names.
 
std::vector< std::pair< ULong64_t, ULong64_t > > GetEntryRanges () final
 Returns a range of size 1 as long as more rows are available in the SQL result set.
 
std::string GetLabel () final
 Return a string representation of the datasource type.
 
std::string GetTypeName (std::string_view colName) const final
 Returns the C++ type for a given column name, implemented as a linear search through all the columns.
 
bool HasColumn (std::string_view colName) const final
 A linear search through the columns for the given name.
 
void Initialize () final
 Resets the SQlite query engine at the beginning of the event loop.
 
bool SetEntry (unsigned int slot, ULong64_t entry) final
 Stores the result of the current active sqlite query row as a C++ value.
 
void SetNSlots (unsigned int nSlots) final
 Almost a no-op, many slots can in fact reduce the performance due to thread synchronization.
 
- Public Member Functions inherited from ROOT::RDF::RDataSource
virtual ~RDataSource ()=default
 
virtual void Finalize ()
 Convenience method called after concluding an event-loop.
 
virtual void FinalizeSlot (unsigned int)
 Convenience method called at the end of the data processing associated to a slot.
 
template<typename T >
std::vector< T ** > GetColumnReaders (std::string_view columnName)
 Called at most once per column by RDF.
 
virtual std::unique_ptr< ROOT::Detail::RDF::RColumnReaderBaseGetColumnReaders (unsigned int, std::string_view, const std::type_info &)
 If the other GetColumnReaders overload returns an empty vector, this overload will be called instead.
 
virtual std::size_t GetNFiles () const
 Returns the number of files from which the dataset is constructed.
 
virtual void InitSlot (unsigned int, ULong64_t)
 Convenience method called at the start of the data processing associated to a slot.
 

Protected Member Functions

Record_t GetColumnReadersImpl (std::string_view name, const std::type_info &) final
 Activates the given column's result value.
 
- Protected Member Functions inherited from ROOT::RDF::RDataSource
virtual std::string AsString ()
 

Private Types

enum class  ETypes {
  kInteger , kReal , kText , kBlob ,
  kNull
}
 All the types known to SQlite. Changes require changing fgTypeNames, too. More...
 

Private Member Functions

void SqliteError (int errcode)
 Helper function to throw an exception if there is a fatal sqlite error, e.g. an I/O error.
 

Private Attributes

std::vector< std::string > fColumnNames
 
std::vector< ETypesfColumnTypes
 
std::unique_ptr< Internal::RSqliteDSDataSetfDataSet
 
ULong64_t fNRow
 
unsigned int fNSlots
 
std::vector< Value_tfValues
 The data source is inherently single-threaded and returns only one row at a time. This vector holds the results.
 

Static Private Attributes

static constexpr char const * fgTypeNames []
 Corresponds to the types defined in ETypes.
 

Additional Inherited Members

- Protected Types inherited from ROOT::RDF::RDataSource
using Record_t = std::vector< void * >
 

#include <ROOT/RSqliteDS.hxx>

Inheritance diagram for ROOT::RDF::RSqliteDS:
[legend]

Member Enumeration Documentation

◆ ETypes

enum class ROOT::RDF::RSqliteDS::ETypes
strongprivate

All the types known to SQlite. Changes require changing fgTypeNames, too.

Enumerator
kInteger 
kReal 
kText 
kBlob 
kNull 

Definition at line 55 of file RSqliteDS.hxx.

Constructor & Destructor Documentation

◆ RSqliteDS()

ROOT::RDF::RSqliteDS::RSqliteDS ( const std::string &  fileName,
const std::string &  query 
)

Build the dataframe.

Parameters
[in]fileNameThe path to an sqlite3 file, will be opened read-only
[in]queryA valid sqlite3 SELECT query

The constructor opens the sqlite file, prepares the query engine and determines the column names and types.

Definition at line 352 of file RSqliteDS.cxx.

◆ ~RSqliteDS()

ROOT::RDF::RSqliteDS::~RSqliteDS ( )

Frees the sqlite resources and closes the file.

Definition at line 438 of file RSqliteDS.cxx.

Member Function Documentation

◆ GetColumnNames()

const std::vector< std::string > & ROOT::RDF::RSqliteDS::GetColumnNames ( ) const
finalvirtual

Returns the SELECT queries names.

The column names have been cached in the constructor. For expressions, the column name is the string of the expression unless the query defines a column name with as like in "SELECT 1 + 1 as mycolumn FROM table"

Implements ROOT::RDF::RDataSource.

Definition at line 451 of file RSqliteDS.cxx.

◆ GetColumnReadersImpl()

RDataSource::Record_t ROOT::RDF::RSqliteDS::GetColumnReadersImpl ( std::string_view  name,
const std::type_info &  ti 
)
finalprotectedvirtual

Activates the given column's result value.

Implements ROOT::RDF::RDataSource.

Definition at line 458 of file RSqliteDS.cxx.

◆ GetEntryRanges()

std::vector< std::pair< ULong64_t, ULong64_t > > ROOT::RDF::RSqliteDS::GetEntryRanges ( )
finalvirtual

Returns a range of size 1 as long as more rows are available in the SQL result set.

This inherently serialized the RDF independent of the number of slots.

Implements ROOT::RDF::RDataSource.

Definition at line 481 of file RSqliteDS.cxx.

◆ GetLabel()

std::string ROOT::RDF::RSqliteDS::GetLabel ( )
finalvirtual

Return a string representation of the datasource type.

The returned string will be used by ROOT::RDF::SaveGraph() to represent the datasource in the visualization of the computation graph. Concrete datasources can override the default implementation.

Reimplemented from ROOT::RDF::RDataSource.

Definition at line 529 of file RSqliteDS.cxx.

◆ GetTypeName()

std::string ROOT::RDF::RSqliteDS::GetTypeName ( std::string_view  colName) const
finalvirtual

Returns the C++ type for a given column name, implemented as a linear search through all the columns.

Implements ROOT::RDF::RDataSource.

Definition at line 500 of file RSqliteDS.cxx.

◆ HasColumn()

bool ROOT::RDF::RSqliteDS::HasColumn ( std::string_view  colName) const
finalvirtual

A linear search through the columns for the given name.

Implements ROOT::RDF::RDataSource.

Definition at line 514 of file RSqliteDS.cxx.

◆ Initialize()

void ROOT::RDF::RSqliteDS::Initialize ( )
finalvirtual

Resets the SQlite query engine at the beginning of the event loop.

Reimplemented from ROOT::RDF::RDataSource.

Definition at line 521 of file RSqliteDS.cxx.

◆ SetEntry()

bool ROOT::RDF::RSqliteDS::SetEntry ( unsigned int  slot,
ULong64_t  entry 
)
finalvirtual

Stores the result of the current active sqlite query row as a C++ value.

Implements ROOT::RDF::RDataSource.

Definition at line 546 of file RSqliteDS.cxx.

◆ SetNSlots()

void ROOT::RDF::RSqliteDS::SetNSlots ( unsigned int  nSlots)
finalvirtual

Almost a no-op, many slots can in fact reduce the performance due to thread synchronization.

Implements ROOT::RDF::RDataSource.

Definition at line 583 of file RSqliteDS.cxx.

◆ SqliteError()

void ROOT::RDF::RSqliteDS::SqliteError ( int  errcode)
private

Helper function to throw an exception if there is a fatal sqlite error, e.g. an I/O error.

Definition at line 594 of file RSqliteDS.cxx.

Member Data Documentation

◆ fColumnNames

std::vector<std::string> ROOT::RDF::RSqliteDS::fColumnNames
private

Definition at line 83 of file RSqliteDS.hxx.

◆ fColumnTypes

std::vector<ETypes> ROOT::RDF::RSqliteDS::fColumnTypes
private

Definition at line 84 of file RSqliteDS.hxx.

◆ fDataSet

std::unique_ptr<Internal::RSqliteDSDataSet> ROOT::RDF::RSqliteDS::fDataSet
private

Definition at line 80 of file RSqliteDS.hxx.

◆ fgTypeNames

constexpr char const * ROOT::RDF::RSqliteDS::fgTypeNames
staticconstexprprivate
Initial value:
= {
"Long64_t",
"double",
"std::string",
"std::vector<unsigned char>",
"void *"
}

Corresponds to the types defined in ETypes.

Definition at line 90 of file RSqliteDS.hxx.

◆ fNRow

ULong64_t ROOT::RDF::RSqliteDS::fNRow
private

Definition at line 82 of file RSqliteDS.hxx.

◆ fNSlots

unsigned int ROOT::RDF::RSqliteDS::fNSlots
private

Definition at line 81 of file RSqliteDS.hxx.

◆ fValues

std::vector<Value_t> ROOT::RDF::RSqliteDS::fValues
private

The data source is inherently single-threaded and returns only one row at a time. This vector holds the results.

Definition at line 86 of file RSqliteDS.hxx.

Libraries for ROOT::RDF::RSqliteDS:

The documentation for this class was generated from the following files: