Logo ROOT  
Reference Guide
 
Loading...
Searching...
No Matches
ROOT._pythonization._ml_dataloader._RDataLoader Class Reference

Definition at line 85 of file _ml_dataloader.py.

Public Member Functions

None __init__ (self, ROOT.RDF.RNode|Spec|Iterable[ROOT.RDF.RNode|Spec] inputs, *int batch_size=0, int batches_in_memory=1, list[str]|None columns=None, dict[str, int]|None max_vec_sizes=None, int vec_padding=0, str|list[str]|None target=None, str weights="", float test_size=0.0, bool shuffle=True, bool drop_remainder=True, int set_seed=0, bool load_eager=False, str sampling_type="", float sampling_ratio=1.0, bool replacement=False)
 Wrapper around the C++ DataLoader.
 
 Activate (self)
 Initialize the generator to be used for a loop, this spawns the loading thread.
 
 ActivateTrainingEpoch (self)
 Activate the training epoch of the generator.
 
 ActivateValidationEpoch (self)
 Activate the validation epoch of the generator.
 
Any ConvertBatchToJAX (self, Any batch, device=None)
 Convert the batch into a JAX array.
 
np.ndarray ConvertBatchToNumpy (self, batch)
 Convert the batch into a NumPy array.
 
torch.Tensor ConvertBatchToPyTorch (self, Any batch, device=None)
 Convert the batch into a PyTorch tensor.
 
Any ConvertBatchToTF (self, Any batch)
 Convert the batch into a TensorFlow tensor.
 
 CreateTrainBatches (self)
 Create the first training batches from the first cluster.
 
 CreateValidationBatches (self)
 Create the first validation batches from the first cluster.
 
 DeActivate (self)
 Deactivate the generator.
 
 DeActivateTrainingEpoch (self)
 Deactivate the training epoch of the generator.
 
 DeActivateValidationEpoch (self)
 Deactivate the validation epoch of the generator.
 
 GetSample (self)
 Return a sample of data that has the same size and types as the actual result.
 
Any GetTrainBatch (self)
 Return the next training batch of data from the given RDataFrame.
 
Any GetValidationBatch (self)
 Return the next training batch of data from the given RDataFrame.
 
 isActive (self)
 
 isTrainingActive (self)
 
 isValidationActive (self)
 
int num_training_batches (self)
 
int num_validation_batches (self)
 
None Save (self, str dataset_name, str filename, bool is_training, str output_format)
 
int train_remainder_rows (self)
 
int val_remainder_rows (self)
 

Public Attributes

 all_columns
 
 batch_size
 
 DeActivate
 
 engine
 
 feature_columns
 
 given_columns
 
 noded_rdfs
 
 num_columns
 
 target_columns
 
 target_given
 
 target_slice
 
 train_slice
 
 weights_column
 
 weights_given
 
 weights_index
 

Protected Member Functions

np.ndarray _get_raw_array (self, batch)
 
list[int] _resolve_columns (self, ROOT.RDF.RNode x_rdf, list[str]|None columns, list[str] target_columns, str weights_column, dict[str, int]|None max_vec_sizes=None)
 Resolve the columns to load based on the given RDataFrame and columns.
 
np.ndarray|Tuple[np.ndarray, np.ndarray]|Tuple[np.ndarray, np.ndarray, np.ndarray] _split_target_and_weights (self, np.ndarray data)
 

Constructor & Destructor Documentation

◆ __init__()

None ROOT._pythonization._ml_dataloader._RDataLoader.__init__ ( self,
ROOT.RDF.RNode | Spec | Iterable[ROOT.RDF.RNode | Spec] inputs,
*int batch_size = 0,
int batches_in_memory = 1,
list[str] | None columns = None,
dict[str, int] | None max_vec_sizes = None,
int vec_padding = 0,
str | list[str] | None target = None,
str weights = "",
float test_size = 0.0,
bool shuffle = True,
bool drop_remainder = True,
int set_seed = 0,
bool load_eager = False,
str sampling_type = "",
float sampling_ratio = 1.0,
bool replacement = False )

Wrapper around the C++ DataLoader.

Args: inputs (ROOT.RDF.RNode | Spec | Iterable[ROOT.RDF.RNode | Spec]): RDataFrame or (dataset_name, files) tuple, or a list of them, to load from. A tuple is used to construct a single RDataFrame to read dataset_name from one or more files. batch_size (int): Number of entries per batch returned by the generator. batches_in_memory (int): Approximate number of batches that should be kept in memory at the same time. Higher value results in faster loading, but also higher memory usage. Defaults to 1. columns (list[str] | None): Names of feature columns to load. If not given, all columns are used. max_vec_sizes (dict[str, int] | None): Mapping from vector column name to the maximum size of the vector. Required when using vector based columns. vec_padding (int): Value used to pad vectors with if the vector is smaller than the given max vector length. Defaults to 0. target (str | list[str] | None): Name or list of names of target column(s). weights (str): Column used to weight events. Can only be used when a target is given. test_size (float): The ratio of batches being kept for validation. Value has to be between 0 and 1. Defaults to 0.0. shuffle (bool): Batches consist of random events and are shuffled every epoch. Defaults to True. drop_remainder (bool): Drop the remainder of data that is too small to compose full batch. Defaults to True. set_seed (int): For reproducibility: Set the seed for the random number generator used to split the dataset into training and validation as well as shuffing of the entries. Defaults to 0 which means that the seed is set to the random device. load_eager (bool): If True, load the full dataset(s) into memory. If False, load data lazily in clusters. Defaults to False. sampling_type (str): Describes the mode of sampling from the minority and majority dataframes. Supported values are "undersampling" and "oversampling". Requires load_eager=True. Defaults to "". For 'undersampling' and 'oversampling' it requires a list of exactly two dataframes as input, where the dataframe with the most entries is the majority dataframe and the dataframe with the fewest entries is the minority dataframe. sampling_ratio (float): Ratio of minority and majority entries in the resampled dataset. Requires load_eager=True and sampling_type="undersampling" or "oversampling". Defaults to 1.0. replacement (bool): Whether the sampling is with (True) or without (False) replacement. Requires load_eager=True and sampling_type="undersampling". Defaults to False.

Definition at line 164 of file _ml_dataloader.py.

Member Function Documentation

◆ _get_raw_array()

np.ndarray ROOT._pythonization._ml_dataloader._RDataLoader._get_raw_array ( self,
batch )
protected

Definition at line 395 of file _ml_dataloader.py.

◆ _resolve_columns()

list[int] ROOT._pythonization._ml_dataloader._RDataLoader._resolve_columns ( self,
ROOT.RDF.RNode x_rdf,
list[str] | None columns,
list[str] target_columns,
str weights_column,
dict[str, int] | None max_vec_sizes = None )
protected

Resolve the columns to load based on the given RDataFrame and columns.

Args: x_rdf (RNode): RDataFrame or RNode object. columns (list[str], optional): Feature columns to load. Defaults to all columns in the RDataFrame, except for target_columns and weights_column. target_columns (list[str]): Name(s) of target column(s). weights_column (str): Name of the weight column, or "" for none. max_vec_sizes (dict[str, int], optional): Mapping from vector column name to the maximum size of the vector. Required when using vector based columns.

Returns: list[int]: List of max vector sizes

Definition at line 86 of file _ml_dataloader.py.

◆ _split_target_and_weights()

np.ndarray | Tuple[np.ndarray, np.ndarray] | Tuple[np.ndarray, np.ndarray, np.ndarray] ROOT._pythonization._ml_dataloader._RDataLoader._split_target_and_weights ( self,
np.ndarray data )
protected

Definition at line 405 of file _ml_dataloader.py.

◆ Activate()

ROOT._pythonization._ml_dataloader._RDataLoader.Activate ( self)

Initialize the generator to be used for a loop, this spawns the loading thread.

Definition at line 305 of file _ml_dataloader.py.

◆ ActivateTrainingEpoch()

ROOT._pythonization._ml_dataloader._RDataLoader.ActivateTrainingEpoch ( self)

Activate the training epoch of the generator.

Definition at line 313 of file _ml_dataloader.py.

◆ ActivateValidationEpoch()

ROOT._pythonization._ml_dataloader._RDataLoader.ActivateValidationEpoch ( self)

Activate the validation epoch of the generator.

Definition at line 318 of file _ml_dataloader.py.

◆ ConvertBatchToJAX()

Any ROOT._pythonization._ml_dataloader._RDataLoader.ConvertBatchToJAX ( self,
Any batch,
device = None )

Convert the batch into a JAX array.

Args: batch: Batch returned from the DataLoader

Returns: jax.Array: converted batch

Definition at line 492 of file _ml_dataloader.py.

◆ ConvertBatchToNumpy()

np.ndarray ROOT._pythonization._ml_dataloader._RDataLoader.ConvertBatchToNumpy ( self,
batch )

Convert the batch into a NumPy array.

Args: batch: Batch returned from the DataLoader

Returns: np.ndarray: converted batch

Definition at line 429 of file _ml_dataloader.py.

◆ ConvertBatchToPyTorch()

torch.Tensor ROOT._pythonization._ml_dataloader._RDataLoader.ConvertBatchToPyTorch ( self,
Any batch,
device = None )

Convert the batch into a PyTorch tensor.

Args: batch: Batch returned from the DataLoader

Returns: torch.Tensor: converted batch

Definition at line 440 of file _ml_dataloader.py.

◆ ConvertBatchToTF()

Any ROOT._pythonization._ml_dataloader._RDataLoader.ConvertBatchToTF ( self,
Any batch )

Convert the batch into a TensorFlow tensor.

Args: batch: Batch returned from the DataLoader

Returns: tensorflow.Tensor: converted batch

Definition at line 458 of file _ml_dataloader.py.

◆ CreateTrainBatches()

ROOT._pythonization._ml_dataloader._RDataLoader.CreateTrainBatches ( self)

Create the first training batches from the first cluster.

Definition at line 331 of file _ml_dataloader.py.

◆ CreateValidationBatches()

ROOT._pythonization._ml_dataloader._RDataLoader.CreateValidationBatches ( self)

Create the first validation batches from the first cluster.

Definition at line 335 of file _ml_dataloader.py.

◆ DeActivate()

ROOT._pythonization._ml_dataloader._RDataLoader.DeActivate ( self)

Deactivate the generator.

Definition at line 309 of file _ml_dataloader.py.

◆ DeActivateTrainingEpoch()

ROOT._pythonization._ml_dataloader._RDataLoader.DeActivateTrainingEpoch ( self)

Deactivate the training epoch of the generator.

Definition at line 323 of file _ml_dataloader.py.

◆ DeActivateValidationEpoch()

ROOT._pythonization._ml_dataloader._RDataLoader.DeActivateValidationEpoch ( self)

Deactivate the validation epoch of the generator.

Definition at line 327 of file _ml_dataloader.py.

◆ GetSample()

ROOT._pythonization._ml_dataloader._RDataLoader.GetSample ( self)

Return a sample of data that has the same size and types as the actual result.

This sample can be used to define the shape and size of the output

Returns: np.ndarray: data sample

Definition at line 355 of file _ml_dataloader.py.

◆ GetTrainBatch()

Any ROOT._pythonization._ml_dataloader._RDataLoader.GetTrainBatch ( self)

Return the next training batch of data from the given RDataFrame.

Returns: (np.ndarray): Batch of data of size.

Definition at line 517 of file _ml_dataloader.py.

◆ GetValidationBatch()

Any ROOT._pythonization._ml_dataloader._RDataLoader.GetValidationBatch ( self)

Return the next training batch of data from the given RDataFrame.

Returns: (np.ndarray): Batch of data of size.

Definition at line 527 of file _ml_dataloader.py.

◆ isActive()

ROOT._pythonization._ml_dataloader._RDataLoader.isActive ( self)

Definition at line 296 of file _ml_dataloader.py.

◆ isTrainingActive()

ROOT._pythonization._ml_dataloader._RDataLoader.isTrainingActive ( self)

Definition at line 299 of file _ml_dataloader.py.

◆ isValidationActive()

ROOT._pythonization._ml_dataloader._RDataLoader.isValidationActive ( self)

Definition at line 302 of file _ml_dataloader.py.

◆ num_training_batches()

int ROOT._pythonization._ml_dataloader._RDataLoader.num_training_batches ( self)

Definition at line 340 of file _ml_dataloader.py.

◆ num_validation_batches()

int ROOT._pythonization._ml_dataloader._RDataLoader.num_validation_batches ( self)

Definition at line 344 of file _ml_dataloader.py.

◆ Save()

None ROOT._pythonization._ml_dataloader._RDataLoader.Save ( self,
str dataset_name,
str filename,
bool is_training,
str output_format )

Definition at line 537 of file _ml_dataloader.py.

◆ train_remainder_rows()

int ROOT._pythonization._ml_dataloader._RDataLoader.train_remainder_rows ( self)

Definition at line 348 of file _ml_dataloader.py.

◆ val_remainder_rows()

int ROOT._pythonization._ml_dataloader._RDataLoader.val_remainder_rows ( self)

Definition at line 352 of file _ml_dataloader.py.

Member Data Documentation

◆ all_columns

ROOT._pythonization._ml_dataloader._RDataLoader.all_columns

Definition at line 124 of file _ml_dataloader.py.

◆ batch_size

ROOT._pythonization._ml_dataloader._RDataLoader.batch_size

Definition at line 262 of file _ml_dataloader.py.

◆ DeActivate

ROOT._pythonization._ml_dataloader._RDataLoader.DeActivate

Definition at line 293 of file _ml_dataloader.py.

◆ engine

ROOT._pythonization._ml_dataloader._RDataLoader.engine

Definition at line 276 of file _ml_dataloader.py.

◆ feature_columns

ROOT._pythonization._ml_dataloader._RDataLoader.feature_columns

Definition at line 160 of file _ml_dataloader.py.

◆ given_columns

ROOT._pythonization._ml_dataloader._RDataLoader.given_columns

Definition at line 123 of file _ml_dataloader.py.

◆ noded_rdfs

ROOT._pythonization._ml_dataloader._RDataLoader.noded_rdfs

Definition at line 246 of file _ml_dataloader.py.

◆ num_columns

ROOT._pythonization._ml_dataloader._RDataLoader.num_columns

Definition at line 261 of file _ml_dataloader.py.

◆ target_columns

ROOT._pythonization._ml_dataloader._RDataLoader.target_columns

Definition at line 249 of file _ml_dataloader.py.

◆ target_given

ROOT._pythonization._ml_dataloader._RDataLoader.target_given

Definition at line 251 of file _ml_dataloader.py.

◆ target_slice

ROOT._pythonization._ml_dataloader._RDataLoader.target_slice

Definition at line 159 of file _ml_dataloader.py.

◆ train_slice

ROOT._pythonization._ml_dataloader._RDataLoader.train_slice

Definition at line 158 of file _ml_dataloader.py.

◆ weights_column

ROOT._pythonization._ml_dataloader._RDataLoader.weights_column

Definition at line 250 of file _ml_dataloader.py.

◆ weights_given

ROOT._pythonization._ml_dataloader._RDataLoader.weights_given

Definition at line 252 of file _ml_dataloader.py.

◆ weights_index

ROOT._pythonization._ml_dataloader._RDataLoader.weights_index

Definition at line 266 of file _ml_dataloader.py.


The documentation for this class was generated from the following file: