Logo ROOT  
Reference Guide
 
Loading...
Searching...
No Matches
RNTupleParallelWriter.hxx
Go to the documentation of this file.
1/// \file ROOT/RNTupleParallelWriter.hxx
2/// \ingroup NTuple ROOT7
3/// \author Jonas Hahnfeld <jonas.hahnfeld@cern.ch>
4/// \date 2024-02-01
5/// \warning This is part of the ROOT 7 prototype! It will change without notice. It might trigger earthquakes. Feedback
6/// is welcome!
7
8/*************************************************************************
9 * Copyright (C) 1995-2024, Rene Brun and Fons Rademakers. *
10 * All rights reserved. *
11 * *
12 * For the licensing terms see $ROOTSYS/LICENSE. *
13 * For the list of contributors see $ROOTSYS/README/CREDITS. *
14 *************************************************************************/
15
16#ifndef ROOT7_RNTupleParallelWriter
17#define ROOT7_RNTupleParallelWriter
18
21
22#include <memory>
23#include <mutex>
24#include <string_view>
25#include <vector>
26
27class TFile;
28
29namespace ROOT {
30namespace Experimental {
31
32namespace Internal {
33class RPageSink;
34} // namespace Internal
35
36class RNTupleFillContext;
37class RNTupleModel;
38
39/**
40\class ROOT::Experimental::RNTupleParallelWriter
41\ingroup NTuple
42\brief A writer to fill an RNTuple from multiple contexts
43
44Compared to the sequential RNTupleWriter, a parallel writer enables the creation of multiple RNTupleFillContext (see
45RNTupleParallelWriter::CreateFillContext). Each fill context prepares independent clusters that are appended to the
46common ntuple with internal synchronization. Before destruction, all fill contexts must have flushed their data and
47been destroyed (or data could be lost!).
48
49For user convenience, RNTupleParallelWriter::CreateFillContext is thread-safe and may be called from multiple threads
50in parallel at any time, also after some data has already been written. Internally, the original model is cloned and
51ownership is passed to a newly created RNTupleFillContext. For that reason, it is recommended to use
52RNTupleModel::CreateBare when creating the model for parallel writing and avoid the allocation of a useless default
53REntry per context.
54
55Note that the sequence of independently prepared clusters is indeterminate and therefore entries are only partially
56ordered: Entries from one context are totally ordered as they were filled. However, there is no orderering with other
57contexts and the entries may be appended to the ntuple either before or after other entries written in parallel into
58other contexts. In addition, two consecutive entries in one fill context can end up separated in the final ntuple, if
59they happen to fall onto a cluster boundary and other contexts append more entries before the next cluster is full.
60
61At the moment, the parallel writer does not (yet) support incremental updates of the underlying model. Please refer to
62RNTupleWriter::CreateModelUpdater if required for your use case.
63*/
65private:
66 /// A global mutex to protect the internal data structures of this object.
67 std::mutex fMutex;
68 /// A mutex to synchronize the final page sink.
69 std::mutex fSinkMutex;
70 /// The final RPageSink that represents the synchronization point.
71 std::unique_ptr<Internal::RPageSink> fSink;
72 /// The original RNTupleModel connected to fSink; needs to be destructed before it.
73 std::unique_ptr<RNTupleModel> fModel;
75 /// List of all created helpers. They must be destroyed before this RNTupleParallelWriter is destructed.
76 std::vector<std::weak_ptr<RNTupleFillContext>> fFillContexts;
77
78 RNTupleParallelWriter(std::unique_ptr<RNTupleModel> model, std::unique_ptr<Internal::RPageSink> sink);
81
82public:
83 /// Recreate a new file and return a writer to write an ntuple.
84 static std::unique_ptr<RNTupleParallelWriter> Recreate(std::unique_ptr<RNTupleModel> model,
85 std::string_view ntupleName, std::string_view storage,
86 const RNTupleWriteOptions &options = RNTupleWriteOptions());
87 /// Append an ntuple to the existing file, which must not be accessed while data is filled into any created context.
88 static std::unique_ptr<RNTupleParallelWriter> Append(std::unique_ptr<RNTupleModel> model,
89 std::string_view ntupleName, TFile &file,
90 const RNTupleWriteOptions &options = RNTupleWriteOptions());
91
93
94 /// Create a new RNTupleFillContext that can be used to fill entries and prepare clusters in parallel. This method is
95 /// thread-safe and may be called from multiple threads in parallel at any time, also after some data has already
96 /// been written.
97 ///
98 /// Note that all fill contexts must be destroyed before the RNTupleParallelWriter is destructed.
99 std::shared_ptr<RNTupleFillContext> CreateFillContext();
100
102 const Detail::RNTupleMetrics &GetMetrics() const { return fMetrics; }
103};
104
105} // namespace Experimental
106} // namespace ROOT
107
108#endif
A collection of Counter objects with a name, a unit, and a description.
A writer to fill an RNTuple from multiple contexts.
std::vector< std::weak_ptr< RNTupleFillContext > > fFillContexts
List of all created helpers. They must be destroyed before this RNTupleParallelWriter is destructed.
RNTupleParallelWriter(const RNTupleParallelWriter &)=delete
std::unique_ptr< RNTupleModel > fModel
The original RNTupleModel connected to fSink; needs to be destructed before it.
std::unique_ptr< Internal::RPageSink > fSink
The final RPageSink that represents the synchronization point.
static std::unique_ptr< RNTupleParallelWriter > Recreate(std::unique_ptr< RNTupleModel > model, std::string_view ntupleName, std::string_view storage, const RNTupleWriteOptions &options=RNTupleWriteOptions())
Recreate a new file and return a writer to write an ntuple.
std::mutex fSinkMutex
A mutex to synchronize the final page sink.
std::shared_ptr< RNTupleFillContext > CreateFillContext()
Create a new RNTupleFillContext that can be used to fill entries and prepare clusters in parallel.
RNTupleParallelWriter & operator=(const RNTupleParallelWriter &)=delete
std::mutex fMutex
A global mutex to protect the internal data structures of this object.
static std::unique_ptr< RNTupleParallelWriter > Append(std::unique_ptr< RNTupleModel > model, std::string_view ntupleName, TFile &file, const RNTupleWriteOptions &options=RNTupleWriteOptions())
Append an ntuple to the existing file, which must not be accessed while data is filled into any creat...
const Detail::RNTupleMetrics & GetMetrics() const
Common user-tunable settings for storing ntuples.
A ROOT file is an on-disk file, usually with extension .root, that stores objects in a file-system-li...
Definition TFile.h:53
tbb::task_arena is an alias of tbb::interface7::task_arena, which doesn't allow to forward declare tb...