Logo ROOT  
Reference Guide
No Matches
RooFit::MultiProcess::Queue Class Reference

Keeps a queue of tasks for workers and manages the queue process through its event loop.

The Queue maintains a set of tasks on the queue process by receiving them from the master process. Worker processes can request to pop them off the queue. The communication between these processes is handled inside 'Queue::loop()', the queue process's event loop that polls the Messenger's sockets for incoming messages and handles them when they come.

The reason for this class is to get automatic load balancing between workers. By allowing workers to request tasks whenever they are ready to do work, we don't need to manually distribute work over workers and they will always have something to do until all tasks have been completed. The alternative simple strategy of just distributing all tasks evenly over workers will be suboptimal when tasks have different or even varying runtimes (this simple strategy could be implemented with a PUSH-PULL ZeroMQ socket from master to workers, which would distribute tasks in a round-robin fashion, which, indeed, does not do load balancing).

Definition at line 24 of file Queue.h.

Public Member Functions

void add (JobTask job_task)
 Enqueue a task.
void loop ()
 The queue process's event loop.
bool pop (JobTask &job_task)
 Have a worker ask for a task-message from the queue.
void process_master_message (M2Q message)
 Helper function for 'Queue::loop()'.
void process_worker_message (std::size_t this_worker_id, W2Q message)
 Helper function for 'Queue::loop()'.

Private Attributes

std::size_t N_tasks_ = 0
std::size_t N_tasks_at_workers_ = 0
std::queue< JobTaskqueue_

#include </home/sftnight/build/workspace/root-makedoc-v626/rootspi/rdoc/src/v6-26-00-patches/roofit/multiprocess/res/RooFit/MultiProcess/Queue.h>

Member Function Documentation

◆ add()

void RooFit::MultiProcess::Queue::add ( JobTask  job_task)

Enqueue a task.

[in]job_taskJobTask object that contains the Job ID and the task index.

Definition at line 61 of file Queue.cxx.

◆ loop()

void RooFit::MultiProcess::Queue::loop ( )

The queue process's event loop.

Polls for incoming messages from other processes and handles them.

Definition at line 114 of file Queue.cxx.

◆ pop()

bool RooFit::MultiProcess::Queue::pop ( JobTask job_task)

Have a worker ask for a task-message from the queue.

[out]job_taskJobTask reference to put the Job ID and the task index into.
true if a task was popped from the queue successfully, false if the queue was empty.

Definition at line 47 of file Queue.cxx.

◆ process_master_message()

void RooFit::MultiProcess::Queue::process_master_message ( M2Q  message)

Helper function for 'Queue::loop()'.

Definition at line 74 of file Queue.cxx.

◆ process_worker_message()

void RooFit::MultiProcess::Queue::process_worker_message ( std::size_t  this_worker_id,
W2Q  message 

Helper function for 'Queue::loop()'.

Definition at line 91 of file Queue.cxx.

Member Data Documentation

◆ N_tasks_

std::size_t RooFit::MultiProcess::Queue::N_tasks_ = 0

Definition at line 36 of file Queue.h.

◆ N_tasks_at_workers_

std::size_t RooFit::MultiProcess::Queue::N_tasks_at_workers_ = 0

Definition at line 37 of file Queue.h.

◆ queue_

std::queue<JobTask> RooFit::MultiProcess::Queue::queue_

Definition at line 35 of file Queue.h.

  • roofit/multiprocess/res/RooFit/MultiProcess/Queue.h
  • roofit/multiprocess/src/Queue.cxx