Logo ROOT   6.14/05
Reference Guide
runProof.C
Go to the documentation of this file.
1 /// \file
2 /// \ingroup proof
3 ///
4 /// Macro to run examples of analysis on PROOF, corresponding to the TSelector
5 /// implementations found under <ROOTSYS>/tutorials/proof .
6 /// This macro uses an existing PROOF session or starts one at the indicated URL.
7 /// In the case non existing PROOF session is found and no URL is given, the
8 /// macro tries to start a local PROOF session.
9 ///
10 /// To run the macro:
11 ///
12 /// root[] .L proof/runProof.C+
13 /// root[] runProof("<analysis>")
14 ///
15 /// Currently available analysis are (to see how all this really works check
16 /// the scope for the specified option inside the macro):
17 ///
18 /// 1. "simple"
19 ///
20 /// Selector: ProofSimple.h.C
21 ///
22 /// root[] runProof("simple")
23 ///
24 /// This will create a local PROOF session and run an analysis filling
25 /// 100 histos with 100000 gaussian random numbers, and displaying them
26 /// in a canvas with 100 pads (10x10).
27 /// The number of histograms can be passed as argument 'nhist' to 'simple',
28 /// e.g. to fill 16 histos with 1000000 entries use
29 ///
30 /// root[] runProof("simple(nevt=1000000,nhist=16)")
31 ///
32 /// The argument nhist3 controls the creation of 3d histos to simulate
33 /// merging load. By default, no 3D hitogram is created.
34 ///
35 /// 2. "h1"
36 ///
37 /// Selector: tutorials/tree/h1analysis.h.C
38 ///
39 /// root[] runProof("h1")
40 ///
41 /// This runs the 'famous' H1 analysis from $ROOTSYS/tree/h1analysis.C.h.
42 /// By default the data are read from the HTTP server at root.cern.ch,
43 /// the data source can be changed via the argument 'h1src', e.g.
44 ///
45 /// root[] runProof("h1,h1src=/data/h1")
46 ///
47 /// (the directory specified must contain the 4 H1 files).
48 ///
49 /// The 'h1' example is also used to show how to use entry-lists in PROOF.
50 /// To fill the list for the events used for the final plots add the option
51 /// 'fillList':
52 ///
53 /// root[] runProof("h1,fillList")
54 ///
55 /// To use the list previously created for the events used for the
56 /// final plots add the option 'useList':
57 ///
58 /// root[] runProof("h1,useList")
59 ///
60 /// 3. "event"
61 ///
62 /// Selector: ProofEvent.h,.C
63 ///
64 /// This is an example of using PROOF par files.
65 /// It runs event generation and simple analysis based on the 'Event'
66 /// class found under test.
67 ///
68 /// root[] runProof("event")
69 ///
70 /// 4. "eventproc"
71 ///
72 /// Selector: ProofEventProc.h.C
73 ///
74 /// This is an example of using PROOF par files and process 'event'
75 /// data from the ROOT HTTP server. It runs the ProofEventProc selector
76 /// which is derived from the EventTree_Proc one found under
77 /// test/ProofBench. The following specific arguments are available:
78 /// - 'readall' to read the whole event, by default only the branches
79 /// needed by the analysis are read (read 25% more bytes)
80 /// - 'datasrc=<dir-with-files>' to read the files from another server,
81 /// the files must be named 'event_<num>.root' where <num>=1,2,...
82 /// or
83 /// - 'datasrc=<file-with-files>' to take the file content from a text file,
84 /// specified one file per line; usefull when testing differences
85 /// between several sources and distributions
86 /// - 'files=N' to change the number of files to be analysed (default
87 /// is 10, max is 50 for the HTTP server).
88 /// - 'uneven' to process uneven entries from files following the scheme
89 /// {50000,5000,5000,5000,5000} and so on
90 ///
91 /// root[] runProof("eventproc")
92 ///
93 /// 5. "pythia8"
94 ///
95 /// Selector: ProofPythia.h.C
96 ///
97 /// This runs Pythia8 generation based on main03.cc example in Pythia 8.1
98 ///
99 /// To run this analysis ROOT must be configured with pythia8.
100 ///
101 /// Note that before executing this analysis, the env variable PYTHIA8
102 /// must point to the pythia8100 (or newer) directory, in particular,
103 /// $PYTHIA8/xmldoc must contain the file Index.xml. The tutorial assumes
104 /// that the Pythia8 directory is the same on all machines, i.e. local
105 /// and worker ones.
106 ///
107 /// root[] runProof("pythia8")
108 ///
109 /// 6. "ntuple"
110 ///
111 /// Selector: ProofNtuple.h.C
112 ///
113 /// This is an example of final merging via files created on the workers,
114 /// using TProofOutputFile. The final file is called ProofNtuple.root
115 /// and it is created in the directory where the tutorial is run. If
116 /// the PROOF cluster is remote, the file is received by a local xrootd
117 /// daemon started for the purpose. Because of this, this example can be
118 /// run only on unix clients.
119 ///
120 /// root[] runProof("ntuple")
121 ///
122 /// By default the random numbers are generate anew. There is the
123 /// possibility use a file of random numbers (to have reproducible results)
124 /// by specify the option 'inputrndm', e.g.
125 ///
126 /// root[] runProof("ntuple(inputrndm)")
127 ///
128 /// By default the output will be saved in the local file SimpleNtuple.root;
129 /// location and name of the file can be changed via the argument 'outfile',
130 /// e.g.
131 ///
132 /// root[] runProof("simplefile(outfile=/data0/testntuple.root)")
133 /// root[] runProof("simplefile(outfile=root://aserver//data/testntuple.root)")
134 ///
135 /// 7. "dataset"
136 ///
137 /// Selector: ProofNtuple.h.C
138 ///
139 /// This is an example of automatic creation of a dataset from files
140 /// created on the workers, using TProofOutputFile. The dataset is
141 /// called testNtuple and it is automatically registered and verified.
142 /// The files contain the same ntuple as in previous example/tutorial 6
143 /// (the same selector ProofNTuple is used with a slightly different
144 /// configuration). The dataset is then used to produce the same plot
145 /// as in 5 but using the DrawSelect methods of PROOF, which also show
146 /// how to set style, color and other drawing attributes in PROOF.
147 /// Depending on the relative worker perforance, some of the produced
148 /// files may result in having no entries. If this happens, the file
149 /// will be added to the missing (skipped) file list. Increasing the
150 /// number of events (via nevt=...) typically solves this issue.
151 ///
152 /// root[] runProof("dataset")
153 ///
154 /// 8. "friends"
155 ///
156 /// Selectors: ProofFriends.h(.C), ProofAux.h(.C)
157 ///
158 /// This is an example of TTree friend processing in PROOF. It also shows
159 /// how to use the TPacketizerFile to steer creation of files.
160 ///
161 /// root[] runProof("friends")
162 ///
163 /// The trees are by default created in separate files; to create
164 /// them in the same file use option 'samefile', e.g.
165 ///
166 /// root[] runProof("friends(samefile)")
167 ///
168 /// 9. "simplefile"
169 ///
170 /// Selector: ProofSimpleFile.h.C
171 ///
172 /// root[] runProof("simplefile")
173 ///
174 /// This will create a local PROOF session and run an analysis filling
175 /// 16+16 histos with 100000 gaussian random numbers. The merging of
176 /// these histos goes via file; 16 histos are saved in the top directory,
177 /// the other 16 into a subdirectory called 'blue'. The final display
178 /// is done in two canvanses, one for each set of histograms and with
179 /// 16 pads each (4x4).
180 /// The number of histograms in each set can be passed as argument
181 /// 'nhist' to 'simplefile', e.g. to fill 25 histos with 1000000 entries use
182 ///
183 /// root[] runProof("simplefile(nevt=1000000,nhist=25)")
184 ///
185 /// By default the output will be saved in the local file SimpleFile.root;
186 /// location and name of the file can be changed via the argument 'outfile',
187 /// e.g.
188 ///
189 /// root[] runProof("simplefile(outfile=/data0/testsimple.root)")
190 /// root[] runProof("simplefile(outfile=root://aserver//data/testsimple.root)")
191 ///
192 /// 10. "stdvec"
193 ///
194 /// Selector: ProofStdVect.h.C
195 ///
196 /// This is an example of using standard vectors (vector<vector<bool> > and
197 /// vector<vector<float> >) in a TSelector. The same selector is run twice:
198 /// in 'create' mode it creates a dataset with the tree 'stdvec' containing
199 /// 3 branches, a vector<vector<bool> > and two vector<vector<float> >. The
200 /// tree is saved into a file on each worker and a dataset is created with
201 /// these files (the dataset is called 'TestStdVect'); in 'read' mode the
202 /// dataset is read and a couple fo histograms filled and displayed.
203 ///
204 /// root[] runProof("stdvec")
205 ///
206 /// General arguments
207 /// -----------------
208 ///
209 /// The following arguments are valid for all examples (the ones specific
210 /// to each tutorial have been explained above)
211 ///
212 /// 1. ACLiC mode
213 ///
214 /// By default all processing is done with ACLiC mode '+', i.e. compile
215 /// if changed. However, this may lead to problems if the available
216 /// selector libs were compiled in previous sessions with a different
217 /// set of loaded libraries (this is a general problem in ROOT). When
218 /// this happens the best solution is to force recompilation (ACLiC
219 /// mode '++'). To do this just add one or more '+' to the name of the
220 /// tutorial, e.g. runProof("simple++")
221 ///
222 /// 2. debug=[what:]level
223 ///
224 /// Controls verbosity; 'level' is an integer number and the optional string
225 /// 'what' one or more of the enum names in TProofDebug.h .
226 /// e.g. runProof("eventproc(debug=kPacketizer|kCollect:2)") runs 'eventproc' enabling
227 /// all printouts matching TProofDebug::kPacketizer and having level
228 /// equal or larger than 2 .
229 ///
230 /// 3. nevt=N and/or first=F
231 ///
232 /// Set the number of entries to N, eventually (when it makes sense, i.e. when
233 /// processing existing files) starting from F
234 /// e.g. runProof("simple(nevt=1000000000)") runs simple with 1000000000
235 /// runProof("eventproc(first=65000)") runs eventproc processing
236 /// starting with event 65000
237 /// runProof("eventproc(nevt=100000,first=65000)") runs eventproc processing
238 /// 100000 events starting with event 65000
239 ///
240 /// 4. asyn
241 ///
242 /// Run in non blocking mode
243 /// e.g. root[] runProof("h1(asyn)")
244 ///
245 /// 5. nwrk=N
246 ///
247 /// Set the number of active workers to N, usefull to test performance
248 /// on a remote cluster where control about the number of workers is
249 /// not possible, e.g. runProof("event(nwrk=2)") runs 'event' with
250 /// 2 workers.
251 ///
252 /// 6. punzip
253 ///
254 /// Use parallel unzipping in reading files where relevant
255 /// e.g. root[] runProof("eventproc(punzip)")
256 ///
257 /// 7. cache=<bytes> (or <kbytes>K or <mbytes>M)
258 ///
259 /// Change the size of the tree cache; 0 or <0 disables the cache,
260 /// value cane be in bytes (no suffix), kilobytes (suffix 'K') or
261 /// megabytes (suffix 'M'), e.g. root[] runProof("eventproc(cache=0)")
262 ///
263 /// 8. submergers[=S]
264 ///
265 /// Enabling merging via S submergers or the optimal number if S is
266 /// not specified, e.g. root[] runProof("simple(hist=1000,submergers)")
267 ///
268 /// 9. rateest=average
269 ///
270 /// Enable processed entries estimation for constant progress reporting based on
271 /// the measured average. This may screw up the progress bar in some cases, which
272 /// is the reason why it is not on by default .
273 /// e.g. root[] runProof("eventproc(rateest=average)")
274 ///
275 /// 10. perftree=perftreefile.root
276 ///
277 /// Generate the perfomance tree and save it to file 'perftreefile.root',
278 /// e.g. root[] runProof("eventproc(perftree=perftreefile.root)")
279 ///
280 /// 11. feedback=name1[,name2,name3,...]|off
281 ///
282 /// Enable feedback for the specified names or switch it off; by default it is
283 /// enabled for the 'stats' histograms (events,packest, packets-being processed).
284 ///
285 /// In all cases, to run on a remote PROOF cluster, the master URL must
286 /// be passed as second argument; e.g.
287 ///
288 /// root[] runProof("simple","master.do.main")
289 ///
290 /// A rough parsing of the URL is done to determine the locality of the cluster.
291 /// If using a tunnel the URL can start by localhost even for external clusters:
292 /// in such cases the default locality determination will be wrong, so one has
293 /// to tell explicity that the cluster is external via the option field, e.g.
294 ///
295 /// root[] runProof("simple","localhost:33002/?external")
296 ///
297 /// In the case of local running it is possible to specify the number of
298 /// workers to start as third argument (the default is the number of cores
299 /// of the machine), e.g.
300 ///
301 /// root[] runProof("simple",0,4)
302 ///
303 /// will start 4 workers. Note that the real number of workers is changed
304 /// only the first time you call runProof into a ROOT session. Following
305 /// calls can reduce the number of active workers, but not increase it.
306 /// For example, in the same session of the call above starting 4 workers,
307 /// this
308 ///
309 /// root[] runProof("simple",0,8)
310 ///
311 /// will still use 4 workers, while this
312 ///
313 /// root[] runProof("simple",0,2)
314 ///
315 /// will disable 2 workers and use the other 2.
316 ///
317 /// Finally, it is possible to pass as 4th argument a list of objects to be added
318 /// to the input list to further control the PROOF behaviour:
319 ///
320 /// root [] TList *ins = new TList
321 /// root [] ins->Add(new TParameter<Int_t>("MyParm", 3))
322 /// root [] runProof("simple",0,4,ins)
323 ///
324 /// the content of 'ins' will then be copied to the input list before processing.
325 ///
326 ///
327 /// \macro_code
328 ///
329 /// \author Gerardo Ganis
330 
331 
332 #include "TCanvas.h"
333 #include "TChain.h"
334 #include "TDSet.h"
335 #include "TEnv.h"
336 #include "TEntryList.h"
337 #include "TFile.h"
338 #include "TFileCollection.h"
339 #include "TFrame.h"
340 #include "THashList.h"
341 #include "TList.h"
342 #include "TPad.h"
343 #include "TPaveText.h"
344 #include "TProof.h"
345 #include "TProofDebug.h"
346 #include "TString.h"
347 
348 #include "getProof.C"
349 void plotNtuple(TProof *p, const char *ds, const char *ntptitle);
350 void SavePerfTree(TProof *proof, const char *fn);
351 
352 // Variable used to locate the Pythia8 directory for the Pythia8 example
353 const char *pythia8dir = 0;
354 const char *pythia8data = 0;
355 
356 void runProof(const char *what = "simple",
357  const char *masterurl = "proof://localhost:40000",
358  Int_t nwrks = -1, TList *ins = 0)
359 {
360 
361  gEnv->SetValue("Proof.StatsHist",1);
362 
363  TString u(masterurl);
364  // Determine locality of this session
365  Bool_t isProofLocal = kFALSE;
366  if (!u.IsNull() && u != "lite://") {
367  TUrl uu(masterurl);
368  TString uopts(uu.GetOptions());
369  if ((!strcmp(uu.GetHost(), "localhost") && !uopts.Contains("external")) ||
370  !strcmp(uu.GetHostFQDN(), TUrl(gSystem->HostName()).GetHostFQDN())) {
371  isProofLocal = kTRUE;
372  }
373  // Adjust URL
374  if (!u.BeginsWith(uu.GetProtocol())) uu.SetProtocol("proof");
375  uopts.ReplaceAll("external", "");
376  uu.SetOptions(uopts.Data());
377  u = uu.GetUrl();
378  }
379  const char *url = u.Data();
380 
381  // Temp dir for PROOF tutorials
382  // Force "/tmp/<user>" whenever possible to avoid length problems on MacOsX
383  TString tmpdir("/tmp");
384  if (gSystem->AccessPathName(tmpdir, kWritePermission)) tmpdir = gSystem->TempDirectory();
385  TString us;
387  if (!ug) {
388  Printf("runProof: could not get user info");
389  return;
390  }
391  us.Form("/%s", ug->fUser.Data());
392  if (!tmpdir.EndsWith(us.Data())) tmpdir += us;
393  gSystem->mkdir(tmpdir.Data(), kTRUE);
394  if (gSystem->AccessPathName(tmpdir, kWritePermission)) {
395  Printf("runProof: unable to get a writable tutorial directory (tried: %s)"
396  " - cannot continue", tmpdir.Data());
397  return;
398  }
399  TString tutdir = Form("%s/.proof-tutorial", tmpdir.Data());
400  if (gSystem->AccessPathName(tutdir)) {
401  Printf("runProof: creating the temporary directory"
402  " for the tutorial (%s) ... ", tutdir.Data());
403  if (gSystem->mkdir(tutdir, kTRUE) != 0) {
404  Printf("runProof: could not assert / create the temporary directory"
405  " for the tutorial (%s)", tutdir.Data());
406  return;
407  }
408  }
409 
410  // For the Pythia8 example we need to set some environment variable;
411  // This must be done BEFORE starting the PROOF session
412  if (what && !strncmp(what, "pythia8", 7)) {
413  // We assume that the remote location of Pythia8 is the same as the local one
414  pythia8dir = gSystem->Getenv("PYTHIA8");
415  if (!pythia8dir || strlen(pythia8dir) <= 0) {
416  Printf("runProof: pythia8: environment variable PYTHIA8 undefined:"
417  " it must contain the path to pythia81xx root directory (local and remote) !");
418  return;
419  }
420  pythia8data = gSystem->Getenv("PYTHIA8DATA");
421  if (!pythia8data || strlen(pythia8data) <= 0) {
422  gSystem->Setenv("PYTHIA8DATA", Form("%s/xmldoc", pythia8dir));
423  pythia8data = gSystem->Getenv("PYTHIA8DATA");
424  if (!pythia8data || strlen(pythia8data) <= 0) {
425  Printf("runProof: pythia8: environment variable PYTHIA8DATA undefined:"
426  " it one must contain the path to pythia81xx/xmldoc"
427  " subdirectory (local and remote) !");
428  return;
429  }
430  }
431  TString env = Form("echo export PYTHIA8=%s; export PYTHIA8DATA=%s",
432  pythia8dir, pythia8data);
433  TProof::AddEnvVar("PROOF_INITCMD", env.Data());
434  }
435 
436  Printf("tutorial dir:\t%s", tutdir.Data());
437 
438  // Get the PROOF Session
439  TProof *proof = getProof(url, nwrks, tutdir.Data(), "ask");
440  if (!proof) {
441  Printf("runProof: could not start/attach a PROOF session");
442  return;
443  }
444 
445  // Refine locality (PROOF-Lite always local)
446  if (proof->IsLite()) isProofLocal = kTRUE;
447 
448 #ifdef WIN32
449  if (isProofLocal && what && !strcmp(what, "ntuple", 6)) {
450  // Not support on windows
451  Printf("runProof: the 'ntuple' example needs to run xrootd to receive the output file, \n"
452  " but xrootd is not supported on Windows - cannot continue");
453  return;
454  }
455 #endif
456 
457  TString proofsessions(Form("%s/sessions",tutdir.Data()));
458  // Save tag of the used session
459  FILE *fs = fopen(proofsessions.Data(), "a");
460  if (!fs) {
461  Printf("runProof: could not create files for sessions tags");
462  } else {
463  fprintf(fs,"session-%s\n", proof->GetSessionTag());
464  fclose(fs);
465  }
466  if (!proof) {
467  Printf("runProof: could not start/attach a PROOF session");
468  return;
469  }
470 
471  // Set the number of workers (may only reduce the number of active workers
472  // in the session)
473  if (nwrks > 0)
474  proof->SetParallel(nwrks);
475 
476  // Where is the code to run
477  char *rootbin = gSystem->Which(gSystem->Getenv("PATH"), "root.exe", kExecutePermission);
478  if (!rootbin) {
479  Printf("runProof: root.exe not found: please check the environment!");
480  return;
481  }
482  TString rootsys(gSystem->DirName(rootbin));
483  rootsys = gSystem->DirName(rootsys);
484  TString tutorials(Form("%s/tutorials", rootsys.Data()));
485  delete[] rootbin;
486 
487  // Parse 'what'; it is in the form 'analysis(arg1,arg2,...)'
488  TString args(what);
489  args.ReplaceAll("("," ");
490  args.ReplaceAll(")"," ");
491  args.ReplaceAll(","," ");
492  Ssiz_t from = 0;
493  TString act, tok;
494  if (!args.Tokenize(act, from, " ")) {
495  // Cannot continue
496  Printf("runProof: action not found: check your arguments (%s)", what);
497  return;
498  }
499  // Extract ACLiC mode
500  TString aMode = "+";
501  if (act.EndsWith("+")) {
502  aMode += "+";
503  while (act.EndsWith("+")) { act.Remove(TString::kTrailing,'+'); }
504  }
505  Printf("runProof: %s: ACLiC mode: '%s'", act.Data(), aMode.Data());
506 
507  // Parse out number of events and 'asyn' option, used almost by every test
508  TString aNevt, aFirst, aNwrk, opt, sel, punzip("off"), aCache, aOutFile,
509  aH1Src("http://root.cern.ch/files/h1"),
510  aDebug, aDebugEnum, aRateEst, aPerfTree("perftree.root"),
511  aFeedback("fb=stats");
512  Long64_t suf = 1;
513  Int_t aSubMg = -1;
514  Bool_t useList = kFALSE, makePerfTree = kFALSE;
515  while (args.Tokenize(tok, from, " ")) {
516  // Debug controllers
517  if (tok.BeginsWith("debug=")) {
518  aDebug = tok;
519  aDebug.ReplaceAll("debug=","");
520  Int_t icol = kNPOS;
521  if ((icol = aDebug.Index(":")) != kNPOS) {
522  aDebugEnum = aDebug(0, icol);
523  aDebug.Remove(0, icol+1);
524  }
525  if (!aDebug.IsDigit()) {
526  Printf("runProof: %s: error parsing the 'debug=' option (%s) - ignoring", act.Data(), tok.Data());
527  aDebug = "";
528  aDebugEnum = "";
529  }
530  }
531  // Number of events
532  if (tok.BeginsWith("nevt=")) {
533  aNevt = tok;
534  aNevt.ReplaceAll("nevt=","");
535  if (!aNevt.IsDigit()) {
536  Printf("runProof: %s: error parsing the 'nevt=' option (%s) - ignoring", act.Data(), tok.Data());
537  aNevt = "";
538  }
539  }
540  // First event
541  if (tok.BeginsWith("first=")) {
542  aFirst = tok;
543  aFirst.ReplaceAll("first=","");
544  if (!aFirst.IsDigit()) {
545  Printf("runProof: %s: error parsing the 'first=' option (%s) - ignoring", act.Data(), tok.Data());
546  aFirst = "";
547  }
548  }
549  // Sync or async ?
550  if (tok.BeginsWith("asyn"))
551  opt = "ASYN";
552  // Number of workers
553  if (tok.BeginsWith("nwrk=")) {
554  aNwrk = tok;
555  aNwrk.ReplaceAll("nwrk=","");
556  if (!aNwrk.IsDigit()) {
557  Printf("runProof: %s: error parsing the 'nwrk=' option (%s) - ignoring", act.Data(), tok.Data());
558  aNwrk = "";
559  }
560  }
561  // Parallel unzipping ?
562  if (tok.BeginsWith("punzip"))
563  punzip = "on";
564  // Number of workers
565  if (tok.BeginsWith("cache=")) {
566  aCache = tok;
567  aCache.ReplaceAll("cache=","");
568  if (aCache.EndsWith("k")) { aCache.Remove(TString::kTrailing, 'k'); suf = 1024; }
569  if (aCache.EndsWith("K")) { aCache.Remove(TString::kTrailing, 'K'); suf = 1024; }
570  if (aCache.EndsWith("M")) { aCache.Remove(TString::kTrailing, 'M'); suf = 1024*1024; }
571  if (!aCache.IsDigit()) {
572  Printf("runProof: %s: error parsing the 'cache=' option (%s) - ignoring", act.Data(), tok.Data());
573  aCache = "";
574  }
575  }
576  // Use submergers?
577  if (tok.BeginsWith("submergers")) {
578  tok.ReplaceAll("submergers","");
579  aSubMg = 0;
580  if (tok.BeginsWith("=")) {
581  tok.ReplaceAll("=","");
582  if (tok.IsDigit()) aSubMg = tok.Atoi();
583  }
584  }
585  // H1: use entry-lists ?
586  if (tok.BeginsWith("useList")) {
587  useList = kTRUE;
588  }
589  if (tok.BeginsWith("fillList")) {
590  opt += "fillList";
591  }
592  // H1: change location of files?
593  if (tok.BeginsWith("h1src=")) {
594  tok.ReplaceAll("h1src=","");
595  if (!(tok.IsNull())) aH1Src = tok;
596  Printf("runProof: %s: reading data files from '%s'", act.Data(), aH1Src.Data());
597  }
598  // Rate estimation technique
599  if (tok.BeginsWith("rateest=")) {
600  tok.ReplaceAll("rateest=","");
601  if (!(tok.IsNull())) aRateEst = tok;
602  Printf("runProof: %s: progress-bar rate estimation option: '%s'", act.Data(), aRateEst.Data());
603  }
604  // Create and save the preformance tree?
605  if (tok.BeginsWith("perftree")) {
606  makePerfTree = kTRUE;
607  if (tok.BeginsWith("perftree=")) {
608  tok.ReplaceAll("perftree=","");
609  if (!(tok.IsNull())) aPerfTree = tok;
610  }
611  Printf("runProof: %s: saving performance tree to '%s'", act.Data(), aPerfTree.Data());
612  }
613  // Location of the output file, if any
614  if (tok.BeginsWith("outfile")) {
615  if (tok.BeginsWith("outfile=")) {
616  tok.ReplaceAll("outfile=","");
617  if (!(tok.IsNull())) aOutFile = tok;
618  }
619  Printf("runProof: %s: output file: '%s'", act.Data(), aOutFile.Data());
620  }
621  // Feedback
622  if (tok.BeginsWith("feedback=")) {
623  tok.ReplaceAll("feedback=","");
624  if (tok == "off" || tok == "OFF" || tok == "0") {
625  aFeedback = "";
626  } else if (!(tok.IsNull())) {
627  if (tok.BeginsWith("+")) {
628  tok[0] = ',';
629  aFeedback += tok;
630  } else {
631  aFeedback.Form("fb=%s", tok.Data());
632  }
633  }
634  Printf("runProof: %s: feedback: '%s'", act.Data(), aFeedback.Data());
635  }
636  }
637  Long64_t nevt = (aNevt.IsNull()) ? -1 : aNevt.Atoi();
638  Long64_t first = (aFirst.IsNull()) ? 0 : aFirst.Atoi();
639  Long64_t nwrk = (aNwrk.IsNull()) ? -1 : aNwrk.Atoi();
640  from = 0;
641 
642  // Set number workers
643  if (nwrk > 0) {
644  if (proof->GetParallel() < nwrk) {
645  Printf("runProof: %s: request for a number of workers larger then available - ignored", act.Data());
646  } else {
647  proof->SetParallel(nwrk);
648  }
649  }
650 
651  // Debug controllers
652  if (!aDebug.IsNull()) {
653  Int_t dbg = aDebug.Atoi();
654  Int_t scope = TProofDebug::kAll;
655  if (!aDebugEnum.IsNull()) scope = getDebugEnum(aDebugEnum.Data());
656  proof->SetLogLevel(dbg, scope);
657  Printf("runProof: %s: verbose mode for '%s'; level: %d", act.Data(), aDebugEnum.Data(), dbg);
658  }
659 
660  // Have constant progress reporting based on estimated info
661  // (NB: may screw up the progress bar in some cases)
662  if (aRateEst == "average")
663  proof->SetParameter("PROOF_RateEstimation", aRateEst);
664 
665  // Parallel unzip
666  if (punzip == "on") {
667  proof->SetParameter("PROOF_UseParallelUnzip", (Int_t)1);
668  Printf("runProof: %s: parallel unzip enabled", act.Data());
669  } else {
670  proof->SetParameter("PROOF_UseParallelUnzip", (Int_t)0);
671  }
672 
673  // Tree cache
674  if (!aCache.IsNull()) {
675  Long64_t cachesz = aCache.Atoi() * suf;
676  if (cachesz <= 0) {
677  proof->SetParameter("PROOF_UseTreeCache", (Int_t)0);
678  Printf("runProof: %s: disabling tree cache", act.Data());
679  } else {
680  proof->SetParameter("PROOF_UseTreeCache", (Int_t)1);
681  proof->SetParameter("PROOF_CacheSize", cachesz);
682  Printf("runProof: %s: setting cache size to %lld", act.Data(), cachesz);
683  }
684  } else {
685  // Use defaults
686  proof->DeleteParameters("PROOF_UseTreeCache");
687  proof->DeleteParameters("PROOF_CacheSize");
688  }
689 
690  // Enable submergers, if required
691  if (aSubMg >= 0) {
692  proof->SetParameter("PROOF_UseMergers", aSubMg);
693  if (aSubMg > 0) {
694  Printf("runProof: %s: enabling merging via %d sub-mergers", act.Data(), aSubMg);
695  } else {
696  Printf("runProof: %s: enabling merging via sub-mergers (optimal number)", act.Data());
697  }
698  } else {
699  proof->DeleteParameters("PROOF_UseMergers");
700  }
701 
702  // The performance tree
703  if (makePerfTree) {
704  proof->SetParameter("PROOF_StatsHist", "");
705  proof->SetParameter("PROOF_StatsTrace", "");
706  proof->SetParameter("PROOF_SlaveStatsTrace", "");
707  }
708 
709  // Additional inputs from the argument 'ins'
710  if (ins && ins->GetSize() > 0) {
711  TObject *oin = 0;
712  TIter nxo(ins);
713  while ((oin = nxo())) { proof->AddInput(oin); }
714  }
715 
716  // Full lits of inputs so far
717  proof->GetInputList()->Print();
718 
719  // Action
720  if (act == "simple") {
721  // ProofSimple is an example of non-data driven analysis; it
722  // creates and fills with random numbers a given number of histos
723 
724  if (first > 0)
725  // Meaningless for this tutorial
726  Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
727  " - ignored", act.Data());
728 
729  // Default 10000 events
730  nevt = (nevt < 0) ? 100000 : nevt;
731  // Find out the number of histograms
732  TString aNhist, aNhist3;
733  while (args.Tokenize(tok, from, " ")) {
734  // Number of histos
735  if (tok.BeginsWith("nhist=")) {
736  aNhist = tok;
737  aNhist.ReplaceAll("nhist=","");
738  if (!aNhist.IsDigit()) {
739  Printf("runProof: error parsing the 'nhist=' option (%s) - ignoring", tok.Data());
740  aNhist = "";
741  }
742  } else if (tok.BeginsWith("nhist3=")) {
743  aNhist3 = tok;
744  aNhist3.ReplaceAll("nhist3=","");
745  if (!aNhist3.IsDigit()) {
746  Printf("runProof: error parsing the 'nhist3=' option (%s) - ignoring", tok.Data());
747  aNhist3 = "";
748  }
749  }
750  }
751  Int_t nhist = (aNhist.IsNull()) ? 100 : aNhist.Atoi();
752  Int_t nhist3 = (aNhist3.IsNull()) ? -1 : aNhist3.Atoi();
753  Printf("\nrunProof: running \"simple\" with nhist= %d, nhist3=%d and nevt= %lld\n", nhist, nhist3, nevt);
754 
755  // The number of histograms is added as parameter in the input list
756  proof->SetParameter("ProofSimple_NHist", (Long_t)nhist);
757  // The number of histograms is added as parameter in the input list
758  if (nhist3 > 0) proof->SetParameter("ProofSimple_NHist3", (Long_t)nhist3);
759  // The selector string
760  sel.Form("%s/proof/ProofSimple.C%s", tutorials.Data(), aMode.Data());
761  //
762  // Run it for nevt times
763  TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
764  proof->Process(sel.Data(), nevt, xopt);
765 
766  } else if (act == "h1") {
767  // This is the famous 'h1' example analysis run on Proof reading the
768  // data from the ROOT http server.
769 
770  // Create the chain
771  TChain *chain = new TChain("h42");
772  chain->Add(TString::Format("%s/dstarmb.root", aH1Src.Data()));
773  chain->Add(TString::Format("%s/dstarp1a.root", aH1Src.Data()));
774  chain->Add(TString::Format("%s/dstarp1b.root", aH1Src.Data()));
775  chain->Add(TString::Format("%s/dstarp2.root", aH1Src.Data()));
776  chain->ls();
777  // We run on Proof
778  chain->SetProof();
779  // Set entrylist, if required
780  if (useList) {
781  TString eln("elist"), elfn("elist.root");
782  if (gSystem->AccessPathName(elfn)) {
783  Printf("\nrunProof: asked to use an entry list but '%s' not found or not readable", elfn.Data());
784  Printf("\nrunProof: did you forget to run with 'fillList=%s'?\n", elfn.Data());
785  } else {
786  TFile f(elfn);
787  if (!(f.IsZombie())) {
788  TEntryList *elist = (TEntryList *)f.Get(eln);
789  if (elist) {
790  elist->SetDirectory(0); //otherwise the file destructor will delete elist
791  chain->SetEntryList(elist);
792  } else {
793  Printf("\nrunProof: could not find entry-list '%s' in file '%s': ignoring",
794  eln.Data(), elfn.Data());
795  }
796  } else {
797  Printf("\nrunProof: requested entry-list file '%s' not existing (or not readable):"
798  " ignoring", elfn.Data());
799  }
800  }
801  }
802  // The selector
803  sel.Form("%s/tree/h1analysis.C%s", tutorials.Data(), aMode.Data());
804  // Run it
805  Printf("\nrunProof: running \"h1\"\n");
806  TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
807  chain->Process(sel.Data(),xopt,nevt,first);
808  // Cleanup the input list
809  gProof->ClearInputData("elist");
810  gProof->ClearInputData("elist.root");
811  TIter nxi(gProof->GetInputList());
812  TObject *o = 0;
813  while ((o = nxi())) {
814  if (!strncmp(o->GetName(), "elist", 5)) {
815  gProof->GetInputList()->Remove(o);
816  delete o;
817  }
818  }
819 
820  } else if (act == "pythia8") {
821 
822  if (first > 0)
823  Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
824  " - ignored", act.Data());
825 
826  TString path(Form("%s/Index.xml", pythia8data));
827  gSystem->ExpandPathName(path);
828  if (gSystem->AccessPathName(path)) {
829  Printf("runProof: pythia8: PYTHIA8DATA directory (%s) must"
830  " contain the Index.xml file !", pythia8data);
831  return;
832  }
833  TString pythia8par = TString::Format("%s/proof/pythia8.par", tutorials.Data());
834  if (gSystem->AccessPathName(pythia8par.Data())) {
835  Printf("runProof: pythia8: par file not found (tried %s)", pythia8par.Data());
836  return;
837  }
838  proof->UploadPackage(pythia8par);
839  proof->EnablePackage("pythia8");
840  // Show enabled packages
841  proof->ShowEnabledPackages();
842  Printf("runProof: pythia8: check settings:");
843  proof->Exec(".!echo hostname = `hostname`; echo \"ls pythia8:\"; ls pythia8");
844  // Loading libraries needed
845  if (gSystem->Load("libEG.so") < 0) {
846  Printf("runProof: pythia8: libEG not found \n");
847  return;
848  }
849  if (gSystem->Load("libEGPythia8.so") < 0) {
850  Printf("runProof: pythia8: libEGPythia8 not found \n");
851  return;
852  }
853  // Setting the default number of events, if needed
854  nevt = (nevt < 0) ? 100 : nevt;
855  Printf("\nrunProof: running \"Pythia01\" nevt= %lld\n", nevt);
856  // The selector string
857  sel.Form("%s/proof/ProofPythia.C%s", tutorials.Data(), aMode.Data());
858  // Run it for nevt times
859  TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
860  proof->Process(sel.Data(), nevt, xopt);
861 
862  } else if (act == "event") {
863 
864  if (first > 0)
865  // Meaningless for this tutorial
866  Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
867  " - ignored", act.Data());
868 
869  TString eventpar = TString::Format("%s/proof/event.par", tutorials.Data());
870  if (gSystem->AccessPathName(eventpar.Data())) {
871  Printf("runProof: event: par file not found (tried %s)", eventpar.Data());
872  return;
873  }
874 
875  proof->UploadPackage(eventpar);
876  proof->EnablePackage("event");
877  Printf("Enabled packages...\n");
878  proof->ShowEnabledPackages();
879 
880  // Setting the default number of events, if needed
881  nevt = (nevt < 0) ? 100 : nevt;
882  Printf("\nrunProof: running \"event\" nevt= %lld\n", nevt);
883  // The selector string
884  sel.Form("%s/proof/ProofEvent.C%s", tutorials.Data(), aMode.Data());
885  // Run it for nevt times
886  TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
887  proof->Process(sel.Data(), nevt, xopt);
888 
889  } else if (act == "eventproc") {
890 
891  TString eventpar = TString::Format("%s/proof/event.par", tutorials.Data());
892  gSystem->ExpandPathName(eventpar);
893  if (gSystem->AccessPathName(eventpar.Data())) {
894  Printf("runProof: eventproc: par file not found (tried %s)", eventpar.Data());
895  return;
896  }
897 
898  proof->UploadPackage(eventpar);
899  proof->EnablePackage("event");
900  Printf("Enabled packages...\n");
901  proof->ShowEnabledPackages();
902 
903  // Load ProcFileElements (to check processed ranges)
904  TString pfelem = TString::Format("%s/proof/ProcFileElements.C", tutorials.Data());
905  gSystem->ExpandPathName(pfelem);
906  if (gSystem->AccessPathName(pfelem.Data())) {
907  Printf("runProof: eventproc: ProcFileElements.C not found (tried %s)", pfelem.Data());
908  return;
909  }
910  pfelem += aMode;
911  // Add include to test trasmission
912  pfelem += TString::Format(",%s/proof/EmptyInclude.h", tutorials.Data());
913  proof->Load(pfelem);
914 
915  // Extract the number of files to process, data source and
916  // other parameters controlling the run ...
917  Bool_t uneven = kFALSE;
918  TString aFiles, aDataSrc("http://root.cern.ch/files/data"), aPartitions;
919  proof->SetParameter("ProofEventProc_Read", "optimized");
920  while (args.Tokenize(tok, from, " ")) {
921  // Number of events
922  if (tok.BeginsWith("files=")) {
923  aFiles = tok;
924  aFiles.ReplaceAll("files=","");
925  if (!aFiles.IsDigit()) {
926  Printf("runProof: error parsing the 'files=' option (%s) - ignoring", tok.Data());
927  aFiles = "";
928  }
929  } else if (tok.BeginsWith("datasrc=")) {
930  tok.ReplaceAll("datasrc=","");
931  if (tok.IsDigit()) {
932  Printf("runProof: error parsing the 'datasrc=' option (%s) - ignoring", tok.Data());
933  } else {
934  aDataSrc = tok;
935  Printf("runProof: reading files from: %s", aDataSrc.Data());
936  }
937  } else if (tok == "readall") {
938  proof->SetParameter("ProofEventProc_Read", "readall");
939  Printf("runProof: eventproc: reading the full event");
940  } else if (tok == "uneven") {
941  uneven = kTRUE;
942  } else if (tok.BeginsWith("partitions=")) {
943  tok.ReplaceAll("partitions=","");
944  if (tok.IsDigit()) {
945  Printf("runProof: error parsing the 'partitions=' option (%s) - ignoring", tok.Data());
946  } else {
947  aPartitions = tok;
948  Printf("runProof: partitions: %s included in packetizer operations", aPartitions.Data());
949  }
950  }
951  }
952  Int_t nFiles = (aFiles.IsNull()) ? 10 : aFiles.Atoi();
953  Printf("runProof: found aFiles: '%s', nFiles: %d", aFiles.Data(), nFiles);
954  if (nFiles > 50) {
955  Printf("runProof: max number of files is 50 - resizing request");
956  nFiles = 50;
957  }
958 
959  // We create the chain now
960  TChain *c = new TChain("EventTree");
961 
962  FileStat_t fst;
963  if (gSystem->GetPathInfo(aDataSrc, fst) == 0 && R_ISREG(fst.fMode) &&
964  !gSystem->AccessPathName(aDataSrc, kReadPermission)) {
965  // It is a local file, we get the TFileCollection and we inject it into the chain
966  TFileCollection *fc = new TFileCollection("", "", aDataSrc, nFiles);
967  c->AddFileInfoList(fc->GetList());
968  delete fc;
969 
970  } else {
971 
972  // Tokenize the source: if more than 1 we rotate the assignment. More sources can be specified
973  // separating them by a '|'
974  TObjArray *dsrcs = aDataSrc.Tokenize("|");
975  Int_t nds = dsrcs->GetEntries();
976 
977  // Fill the chain
978  Int_t i = 1, k = 0;
979  TString fn;
980  for (i = 1; i <= nFiles; i++) {
981  k = (i - 1) % nds;
982  TObjString *os = (TObjString *) (*dsrcs)[k];
983  if (os) {
984  fn.Form("%s/event_%d.root", os->GetName(), i);
985  if (uneven) {
986  if ((i - 1)%5 == 0)
987  c->AddFile(fn.Data(), 50000);
988  else
989  c->AddFile(fn.Data(), 5000);
990  } else {
991  c->AddFile(fn.Data());
992  }
993  }
994  }
995  dsrcs->SetOwner();
996  delete dsrcs;
997  }
998  // Show the chain
999  c->ls();
1000  c->SetProof();
1001 
1002  // Only validate the files really needed for the analysis
1003  proof->SetParameter("PROOF_ValidateByFile", 1);
1004 
1005  // Send over the partition information, if any
1006  if (!aPartitions.IsNull()) {
1007  aPartitions.ReplaceAll("|", ",");
1008  proof->SetParameter("PROOF_PacketizerPartitions", aPartitions);
1009  }
1010 
1011  // The selector
1012  sel.Form("%s/proof/ProofEventProc.C%s", tutorials.Data(), aMode.Data());
1013  // Run it
1014  Printf("\nrunProof: running \"eventproc\"\n");
1015  TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
1016  c->Process(sel.Data(), xopt, nevt, first);
1017 
1018  } else if (act == "ntuple") {
1019 
1020  // ProofNtuple is an example of non-data driven analysis; it
1021  // creates and fills a disk resident ntuple with automatic file merging
1022 
1023  if (first > 0)
1024  // Meaningless for this tutorial
1025  Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
1026  " - ignored", act.Data());
1027 
1028  // Set the default number of events, if needed
1029  nevt = (nevt < 0) ? 1000 : nevt;
1030  Printf("\nrunProof: running \"ntuple\" with nevt= %lld\n", nevt);
1031 
1032  // Which randoms to use
1033  Bool_t usentprndm = kFALSE;
1034  while (args.Tokenize(tok, from, " ")) {
1035  if (tok == "inputrndm") {
1036  usentprndm = kTRUE;
1037  break;
1038  }
1039  }
1040  if (usentprndm) Printf("runProof: taking randoms from input ntuple\n");
1041 
1042  // Output file
1043  TString fout(aOutFile);
1044  if (fout.IsNull()) {
1045  fout.Form("%s/ProofNtuple.root", gSystem->WorkingDirectory());
1046  // Cleanup any existing instance of the output file
1047  gSystem->Unlink(fout);
1048 
1049  if (!isProofLocal) {
1050  // Setup a local basic xrootd to receive the file
1051  Bool_t xrdok = kFALSE;
1052  Int_t port = 9000;
1053  while (port < 9010) {
1054  if (checkXrootdAt(port) != 1) {
1055  if (startXrootdAt(port, gSystem->WorkingDirectory(), kTRUE) == 0) {
1056  xrdok = kTRUE;
1057  break;
1058  }
1059  }
1060  port++;
1061  }
1062  if (!xrdok) {
1063  Printf("runProof: could not start basic xrootd on ports 9000-9009 - cannot continue");
1064  return;
1065  }
1066  fout.Insert(0, TString::Format("root://%s:%d/", TUrl(gSystem->HostName()).GetHostFQDN(), port));
1067  // Make a copy of the files on the master before merging
1068  proof->AddInput(new TNamed("PROOF_OUTPUTFILE_LOCATION", "LOCAL"));
1069  }
1070  }
1071  proof->AddInput(new TNamed("PROOF_OUTPUTFILE", fout.Data()));
1072 
1073  // If using the 'NtpRndm' for a fixed values of randoms, send over the file
1074  if (usentprndm) {
1075  // The file with 'NtpRndm'
1076  TString fnr = TString::Format("%s/proof/ntprndm.root", tutorials.Data());
1077  // Set as input data
1078  proof->SetInputDataFile(fnr);
1079  // Set the related parameter
1080  proof->SetParameter("PROOF_USE_NTP_RNDM","yes");
1081  // Notify
1082  Printf("runProof: taking randoms from '%s'", fnr.Data());
1083  }
1084 
1085  // The selector string
1086  sel.Form("%s/proof/ProofNtuple.C%s", tutorials.Data(), aMode.Data());
1087 
1088  // Run it for nevt times
1089  TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
1090  Printf("runProof: selector file '%s', options: '%s'", sel.Data(), xopt.Data());
1091  proof->Process(sel.Data(), nevt, xopt);
1092 
1093  // Reset input variables
1094  if (usentprndm) {
1095  proof->DeleteParameters("PROOF_USE_NTP_RNDM");
1096  proof->SetInputDataFile(0);
1097  }
1098 
1099  } else if (act == "dataset") {
1100 
1101  // This is an example of analysis creating data files on each node which are
1102  // automatically registered as dataset; the newly created dataset is used to create
1103  // the final plots. The data are of the same type as for the 'ntuple' example.
1104  // Selector used: ProofNtuple
1105 
1106  if (first > 0)
1107  // Meaningless for this tutorial
1108  Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
1109  " - ignored", act.Data());
1110 
1111  // Set the default number of events, if needed
1112  nevt = (nevt < 0) ? 1000000 : nevt;
1113  Printf("\nrunProof: running \"dataset\" with nevt= %lld\n", nevt);
1114 
1115  // Ask for registration of the dataset (the default is the the TFileCollection is return
1116  // without registration; the name of the TFileCollection is the name of the dataset
1117  proof->SetParameter("SimpleNtuple.root","testNtuple");
1118 
1119  // Do not plot the ntuple at this level
1120  proof->SetParameter("PROOF_NTUPLE_DONT_PLOT", "");
1121 
1122  // The selector string
1123  sel.Form("%s/proof/ProofNtuple.C%s", tutorials.Data(), aMode.Data());
1124  //
1125  // Run it for nevt times
1126  TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
1127  proof->Process(sel.Data(), nevt, xopt);
1128 
1129  // The TFileCollection must be in the output
1130  if (proof->GetOutputList()->FindObject("testNtuple")) {
1131 
1132  // Plot the ntuple via PROOF (example of drawing PROOF actions)
1133  plotNtuple(proof, "testNtuple", "proof ntuple from dataset");
1134 
1135  } else {
1136  Printf("runProof: dataset 'testNtuple' not found in the output list");
1137  }
1138  // Do not plot the ntuple at this level
1139  proof->DeleteParameters("PROOF_NTUPLE_DONT_PLOT");
1140  proof->DeleteParameters("SimpleNtuple.root");
1141 
1142  } else if (act == "friends") {
1143 
1144  // This is an example of analysis creating two data files on each node (the main tree
1145  // and its friend) which are then processed as 'friends' to create the final plots.
1146  // Selector used: ProofFriends, ProofAux
1147 
1148  if (first > 0)
1149  // Meaningless for this tutorial
1150  Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
1151  " - ignored", act.Data());
1152 
1153  // Find out whether to use the same file or separate files
1154  Bool_t sameFile = kFALSE;
1155  while (args.Tokenize(tok, from, " ")) {
1156  // Number of histos
1157  if (tok == "samefile") {
1158  sameFile = kTRUE;
1159  break;
1160  }
1161  }
1162 
1163  // File generation: we use TPacketizerFile in here to create two files per node
1164  TList *wrks = proof->GetListOfSlaveInfos();
1165  if (!wrks) {
1166  Printf("runProof: could not get the list of information about the workers");
1167  return;
1168  }
1169  // Create the map
1170  TString fntree;
1171  TMap *files = new TMap;
1172  files->SetName("PROOF_FilesToProcess");
1173  TIter nxwi(wrks);
1174  TSlaveInfo *wi = 0;
1175  while ((wi = (TSlaveInfo *) nxwi())) {
1176  fntree.Form("tree_%s.root", wi->GetOrdinal());
1177  THashList *wrklist = (THashList *) files->GetValue(wi->GetName());
1178  if (!wrklist) {
1179  wrklist = new THashList;
1180  wrklist->SetName(wi->GetName());
1181  files->Add(new TObjString(wi->GetName()), wrklist);
1182  }
1183  wrklist->Add(new TObjString(fntree));
1184  }
1185 
1186  // Generate the files
1187  proof->AddInput(files);
1188  if (sameFile) {
1189  Printf("runProof: friend tree stored in the same file as the main tree");
1190  proof->SetParameter("ProofAux_Action", "GenerateTreesSameFile");
1191  } else {
1192  proof->SetParameter("ProofAux_Action", "GenerateTrees");
1193  }
1194  // Default 1000 events
1195  nevt = (nevt < 0) ? 10000 : nevt;
1196  proof->SetParameter("ProofAux_NEvents", (Long64_t)nevt);
1197  // Special Packetizer
1198  proof->SetParameter("PROOF_Packetizer", "TPacketizerFile");
1199  // Now process
1200  sel.Form("%s/proof/ProofAux.C%s", tutorials.Data(), aMode.Data());
1201  proof->Process(sel.Data(), 1);
1202  // Remove the packetizer specifications
1203  proof->DeleteParameters("PROOF_Packetizer");
1204 
1205  // Print the lists and create the TDSet objects
1206  TDSet *dset = new TDSet("Tmain", "Tmain");
1207  TDSet *dsetf = new TDSet("Tfrnd", "Tfrnd");
1208  if (proof->GetOutputList()) {
1209  TIter nxo(proof->GetOutputList());
1210  TObject *o = 0;
1211  TObjString *os = 0;
1212  while ((o = nxo())) {
1213  TList *l = dynamic_cast<TList *> (o);
1214  if (l && !strncmp(l->GetName(), "MainList-", 9)) {
1215  TIter nxf(l);
1216  while ((os = (TObjString *) nxf()))
1217  dset->Add(os->GetName());
1218  }
1219  }
1220  nxo.Reset();
1221  while ((o = nxo())) {
1222  TList *l = dynamic_cast<TList *> (o);
1223  if (l && !strncmp(l->GetName(), "FriendList-", 11)) {
1224  TIter nxf(l);
1225  while ((os = (TObjString *) nxf()))
1226  dsetf->Add(os->GetName());
1227  }
1228  }
1229  }
1230  // Process with friends
1231  dset->AddFriend(dsetf, "friend");
1232  sel.Form("%s/proof/ProofFriends.C%s", tutorials.Data(), aMode.Data());
1233  TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
1234  dset->Process(sel, xopt);
1235  // Clear the files created by this run
1236  proof->ClearData(TProof::kUnregistered | TProof::kForceClear);
1237 
1238  } else if (act == "simplefile") {
1239 
1240  // ProofSimpleFile is an example of non-data driven analysis with merging
1241  // via file and objcets saved in different directories; it creates and
1242  // fills with random numbers two sets of a given number of histos
1243 
1244  if (first > 0)
1245  // Meaningless for this tutorial
1246  Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
1247  " - ignored", act.Data());
1248 
1249  // Default 100000 events
1250  nevt = (nevt < 0) ? 1000000 : nevt;
1251  // Find out the number of histograms
1252  TString aNhist;
1253  while (args.Tokenize(tok, from, " ")) {
1254  // Number of histos
1255  if (tok.BeginsWith("nhist=")) {
1256  aNhist = tok;
1257  aNhist.ReplaceAll("nhist=","");
1258  if (!aNhist.IsDigit()) {
1259  Printf("runProof: error parsing the 'nhist=' option (%s) - ignoring", tok.Data());
1260  aNhist = "";
1261  }
1262  }
1263  }
1264  Int_t nhist = (aNhist.IsNull()) ? 16 : aNhist.Atoi();
1265  Printf("\nrunProof: running \"simplefile\" with nhist= %d and nevt= %lld\n", nhist, nevt);
1266 
1267  // The number of histograms is added as parameter in the input list
1268  proof->SetParameter("ProofSimple_NHist", (Long_t)nhist);
1269 
1270  // Output file
1271  TString fout(aOutFile);
1272  if (fout.IsNull()) {
1273  fout.Form("%s/SimpleFile.root", gSystem->WorkingDirectory());
1274  // Cleanup any existing instance of the output file
1275  gSystem->Unlink(fout);
1276 
1277  if (!isProofLocal) {
1278  // Setup a local basic xrootd to receive the file
1279  Bool_t xrdok = kFALSE;
1280  Int_t port = 9000;
1281  while (port < 9010) {
1282  if (checkXrootdAt(port) != 1) {
1283  if (startXrootdAt(port, gSystem->WorkingDirectory(), kTRUE) == 0) {
1284  xrdok = kTRUE;
1285  break;
1286  }
1287  }
1288  port++;
1289  }
1290  if (!xrdok) {
1291  Printf("runProof: could not start basic xrootd on ports 9000-9009 - cannot continue");
1292  return;
1293  }
1294  fout.Insert(0, TString::Format("root://%s:%d/", TUrl(gSystem->HostName()).GetHostFQDN(), port));
1295  // Make a copy of the files on the master before merging
1296  proof->AddInput(new TNamed("PROOF_OUTPUTFILE_LOCATION", "LOCAL"));
1297  }
1298  }
1299  proof->AddInput(new TNamed("PROOF_OUTPUTFILE", fout.Data()));
1300 
1301  // The selector string
1302  sel.Form("%s/proof/ProofSimpleFile.C%s", tutorials.Data(), aMode.Data());
1303  //
1304  // Run it for nevt times
1305  TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
1306  proof->Process(sel.Data(), nevt, xopt);
1307 
1308  } else if (act == "stdvec") {
1309 
1310  // This is an example of runnign a TSelector using standard vectors
1311  // Selector used: ProofStdVect
1312 
1313  if (first > 0)
1314  // Meaningless for this tutorial
1315  Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
1316  " - ignored", act.Data());
1317 
1318  // Set the default number of events, if needed
1319  nevt = (nevt < 0) ? 50000 * proof->GetParallel() : nevt;
1320  Printf("\nrunProof: running \"stdvec\" with nevt= %lld\n", nevt);
1321 
1322  // The selector string
1323  sel.Form("%s/proof/ProofStdVect.C%s", tutorials.Data(), aMode.Data());
1324 
1325  TString xopt;
1326  // Create the dataset 'TestStdVect' with 'nevt' events
1327  xopt.Form("%s %s create", aFeedback.Data(), opt.Data());
1328  proof->Process(sel.Data(), nevt, xopt);
1329 
1330  // The dataset must have been registered
1331  if (proof->ExistsDataSet("TestStdVect")) {
1332 
1333  // Use dataset 'TestStdVect'
1334  xopt.Form("%s %s", aFeedback.Data(), opt.Data());
1335  proof->Process("TestStdVect", sel.Data(), xopt);
1336 
1337  } else {
1338  Printf("runProof: dataset 'TestStdVect' not available!");
1339  }
1340 
1341  } else {
1342  // Do not know what to run
1343  Printf("runProof: unknown tutorial: %s", what);
1344  }
1345 
1346  // Save the performance tree
1347  if (makePerfTree) {
1348  SavePerfTree(proof, aPerfTree.Data());
1349  // Cleanup parameters
1350  gProof->DeleteParameters("PROOF_StatsHist");
1351  gProof->DeleteParameters("PROOF_StatsTrace");
1352  gProof->DeleteParameters("PROOF_SlaveStatsTrace");
1353  }
1354 }
1355 
1356 //_______________________________________________________________________________________
1357 void plotNtuple(TProof *p, const char *ds, const char *ntptitle)
1358 {
1359  // Make some plots from the ntuple 'ntp' via PROOF
1360 
1361  //
1362  // Create a canvas, with 2 pads
1363  //
1364  TCanvas *c1 = new TCanvas(Form("cv-%s", ds), ntptitle,800,10,700,780);
1365  c1->Divide(1,2);
1366  TPad *pad1 = (TPad *) c1->GetPad(1);
1367  TPad *pad2 = (TPad *) c1->GetPad(2);
1368  //
1369  // Display a function of one ntuple column imposing a condition
1370  // on another column.
1371  pad1->cd();
1372  pad1->SetGrid();
1373  pad1->SetLogy();
1374  pad1->GetFrame()->SetFillColor(15);
1375 
1376  p->SetParameter("PROOF_LineColor", (Int_t)1);
1377  p->SetParameter("PROOF_FillStyle", (Int_t)1001);
1378  p->SetParameter("PROOF_FillColor", (Int_t)45);
1379  p->DrawSelect(ds, "3*px+2","px**2+py**2>1");
1380  p->SetParameter("PROOF_FillColor", (Int_t)38);
1381  p->DrawSelect(ds, "2*px+2","pz>2","same");
1382  p->SetParameter("PROOF_FillColor", (Int_t)5);
1383  p->DrawSelect(ds, "1.3*px+2","(px^2+py^2>4) && py>0","same");
1384  pad1->RedrawAxis();
1385 
1386  //
1387  // Display a 3-D scatter plot of 3 columns. Superimpose a different selection.
1388  pad2->cd();
1389  p->DrawSelect(ds, "pz:py:px","(pz<10 && pz>6)+(pz<4 && pz>3)");
1390  p->SetParameter("PROOF_MarkerColor", (Int_t)4);
1391  p->DrawSelect(ds, "pz:py:px","pz<6 && pz>4","same");
1392  p->SetParameter("PROOF_MarkerColor", (Int_t)5);
1393  p->DrawSelect(ds, "pz:py:px","pz<4 && pz>3","same");
1394  TPaveText *l2 = new TPaveText(0.,0.6,0.9,0.95);
1395  l2->SetFillColor(42);
1396  l2->SetTextAlign(12);
1397  l2->AddText("You can interactively rotate this view in 2 ways:");
1398  l2->AddText(" - With the RotateCube in clicking in this pad");
1399  l2->AddText(" - Selecting View with x3d in the View menu");
1400  l2->Draw();
1401 
1402  // Final update
1403  c1->cd();
1404  c1->Update();
1405 
1406  // Clear parameters used for the plots
1407  p->DeleteParameters("PROOF_*Color");
1408  p->DeleteParameters("PROOF_*Style");
1409 }
1410 
1411 //______________________________________________________________________________
1412 void SavePerfTree(TProof *proof, const char *fn)
1413 {
1414  // Save PROOF timing information from TPerfStats to file 'fn'
1415 
1416  if (!proof) {
1417  Printf("PROOF must be run to save output performance information");;
1418  return;
1419  }
1420  if (!proof->GetOutputList() || proof->GetOutputList()->GetSize() <= 0) {
1421  Printf("PROOF outputlist undefined or empty");;
1422  return;
1423  }
1424 
1425  TFile f(fn, "RECREATE");
1426  if (f.IsZombie()) {
1427  Printf("ERROR: could not open file '%s' for writing", fn);;
1428  } else {
1429  f.cd();
1430  TIter nxo(proof->GetOutputList());
1431  TObject* obj = 0;
1432  while ((obj = nxo())) {
1433  TString objname(obj->GetName());
1434  if (objname.BeginsWith("PROOF_")) {
1435  // Must list the objects since other PROOF_ objects exist
1436  // besides timing objects
1437  if (objname == "PROOF_PerfStats" ||
1438  objname == "PROOF_PacketsHist" ||
1439  objname == "PROOF_EventsHist" ||
1440  objname == "PROOF_NodeHist" ||
1441  objname == "PROOF_LatencyHist" ||
1442  objname == "PROOF_ProcTimeHist" ||
1443  objname == "PROOF_CpuTimeHist")
1444  obj->Write();
1445  }
1446  }
1447  f.Close();
1448  }
1449 
1450 }
virtual Bool_t AccessPathName(const char *path, EAccessMode mode=kFileExists)
Returns FALSE if one can access a file using the specified access mode.
Definition: TSystem.cxx:1276
An array of TObjects.
Definition: TObjArray.h:37
const char * GetName() const
Returns name of object.
Definition: TProof.h:231
virtual void Draw(Option_t *option="")
Draw this pavetext with its current attributes.
Definition: TPaveText.cxx:233
long long Long64_t
Definition: RtypesCore.h:69
virtual void ls(Option_t *option="") const
List the chain.
Definition: TChain.cxx:1805
virtual TVirtualPad * GetPad(Int_t subpadnumber) const
Get a pointer to subpadnumber of this pad.
Definition: TPad.cxx:2864
virtual const char * WorkingDirectory()
Return working directory.
Definition: TSystem.cxx:869
virtual Long64_t DrawSelect(TDSet *dset, const char *varexp, const char *selection="", Option_t *option="", Long64_t nentries=-1, Long64_t firstentry=0)
Execute the specified drawing action on a data set (TDSet).
Definition: TProof.cxx:6118
Attaches to a PROOF session, possibly at the indicated URL.
Collectable string class.
Definition: TObjString.h:28
return c1
Definition: legend1.C:41
static constexpr double us
virtual Long64_t Process(TSelector *selector, Option_t *option="", Long64_t nentries=-1, Long64_t firstentry=0, TObject *enl=0)
Process TDSet on currently active PROOF session.
Definition: TDSet.cxx:919
const Ssiz_t kNPOS
Definition: RtypesCore.h:111
This class represents a WWW compatible URL.
Definition: TUrl.h:35
TString & ReplaceAll(const TString &s1, const TString &s2)
Definition: TString.h:687
int GetPathInfo(const char *path, Long_t *id, Long_t *size, Long_t *flags, Long_t *modtime)
Get info about a file: id, size, flags, modification time.
Definition: TSystem.cxx:1374
This class implements a data set to be used for PROOF processing.
Definition: TDSet.h:153
virtual void SetOwner(Bool_t enable=kTRUE)
Set whether this collection is the owner (enable==true) of its content.
void SetParameter(const char *par, const char *value)
Set input list parameter.
Definition: TProof.cxx:9794
A ROOT file is a suite of consecutive data records (TKey instances) with a well defined format...
Definition: TFile.h:47
TVirtualPad * cd(Int_t subpadnumber=0)
Set current canvas & pad.
Definition: TCanvas.cxx:688
TList * GetOutputList()
Get list with all object created during processing (see Process()).
Definition: TProof.cxx:9780
virtual TText * AddText(Double_t x1, Double_t y1, const char *label)
Add a new Text line to this pavetext at given coordinates.
Definition: TPaveText.cxx:182
void Add(TObject *obj)
This function may not be used (but we need to provide it since it is a pure virtual in TCollection)...
Definition: TMap.cxx:53
const char * GetOrdinal() const
Definition: TProof.h:232
virtual int Load(const char *module, const char *entry="", Bool_t system=kFALSE)
Load a shared library.
Definition: TSystem.cxx:1829
Basic string class.
Definition: TString.h:131
#define f(i)
Definition: RSha256.hxx:104
int Int_t
Definition: RtypesCore.h:41
virtual const char * DirName(const char *pathname)
Return the directory name in pathname.
Definition: TSystem.cxx:1004
bool Bool_t
Definition: RtypesCore.h:59
virtual char * Which(const char *search, const char *file, EAccessMode mode=kFileExists)
Find location of file in a search path.
Definition: TSystem.cxx:1522
Bool_t R_ISREG(Int_t mode)
Definition: TSystem.h:119
virtual int mkdir(const char *name, Bool_t recursive=kFALSE)
Make a file system directory.
Definition: TSystem.cxx:904
virtual void SetValue(const char *name, const char *value, EEnvLevel level=kEnvChange, const char *type=0)
Set the value of a resource or create a new resource.
Definition: TEnv.cxx:736
Int_t fMode
Definition: TSystem.h:128
virtual void SetDirectory(TDirectory *dir)
Add reference to directory dir. dir can be 0.
virtual Int_t AddFileInfoList(TCollection *list, Long64_t nfiles=TTree::kMaxEntries)
Add all files referenced in the list to the chain.
Definition: TChain.cxx:554
static struct mg_connection * fc(struct mg_context *ctx)
Definition: civetweb.c:3352
virtual int Unlink(const char *name)
Unlink, i.e. remove, a file.
Definition: TSystem.cxx:1357
TVirtualPad * cd(Int_t subpadnumber=0)
Set Current pad.
Definition: TPad.cxx:594
static TString Format(const char *fmt,...)
Static method which formats a string using a printf style format descriptor and return a TString...
Definition: TString.cxx:2286
THashList implements a hybrid collection class consisting of a hash table and a list to store TObject...
Definition: THashList.h:34
TFrame * GetFrame()
Get frame.
Definition: TPad.cxx:2823
The TNamed class is the base class for all named ROOT classes.
Definition: TNamed.h:29
virtual void SetGrid(Int_t valuex=1, Int_t valuey=1)
Definition: TPad.h:327
virtual void SetEntryList(TEntryList *elist, Option_t *opt="")
Set the input entry list (processing the entries of the chain will then be limited to the entries in ...
Definition: TChain.cxx:2549
virtual const char * Getenv(const char *env)
Get environment variable.
Definition: TSystem.cxx:1638
Bool_t EndsWith(const char *pat, ECaseCompare cmp=kExact) const
Return true if string ends with the specified string.
Definition: TString.cxx:2152
virtual UserGroup_t * GetUserInfo(Int_t uid)
Returns all user info in the UserGroup_t structure.
Definition: TSystem.cxx:1574
virtual const char * TempDirectory() const
Return a user configured or systemwide directory to create temporary files in.
Definition: TSystem.cxx:1458
virtual void RedrawAxis(Option_t *option="")
Redraw the frame axis Redrawing axis may be necessary in case of superimposed histograms when one or ...
Definition: TPad.cxx:5203
Int_t getProof(const char *where, Int_t verbose=1)
Open a PROOF session at gUrl.
virtual void SetTextAlign(Short_t align=11)
Set the text alignment.
Definition: TAttText.h:41
A doubly linked list.
Definition: TList.h:44
const char * GetName() const
Returns name of object.
Definition: TObjString.h:39
TString fUser
Definition: TSystem.h:142
virtual void Setenv(const char *name, const char *value)
Set environment variable.
Definition: TSystem.cxx:1622
R__EXTERN TSystem * gSystem
Definition: TSystem.h:540
virtual void SetFillColor(Color_t fcolor)
Set the fill area color.
Definition: TAttFill.h:37
THashList * GetList()
virtual TObject * Remove(TObject *obj)
Remove object from the list.
Definition: TList.cxx:818
Bool_t BeginsWith(const char *s, ECaseCompare cmp=kExact) const
Definition: TString.h:610
void Form(const char *fmt,...)
Formats a string using a printf style format descriptor.
Definition: TString.cxx:2264
virtual void SetProof(Bool_t on=kTRUE, Bool_t refresh=kFALSE, Bool_t gettreeheader=kFALSE)
Enable/Disable PROOF processing on the current default Proof (gProof).
Definition: TChain.cxx:2809
The most important graphics class in the ROOT system.
Definition: TPad.h:29
char * Form(const char *fmt,...)
static void AddEnvVar(const char *name, const char *value)
Add an variable to the list of environment variables passed to proofserv on the master and slaves...
Definition: TProof.cxx:11732
void SetName(const char *name)
Definition: TCollection.h:204
#define Printf
Definition: TGeoToOCC.h:18
virtual Int_t AddFile(const char *name, Long64_t nentries=TTree::kMaxEntries, const char *tname="")
Add a new file to this chain.
Definition: TChain.cxx:457
const Bool_t kFALSE
Definition: RtypesCore.h:88
void ClearInputData(TObject *obj=0)
Remove obj form the input data list; if obj is null (default), clear the input data info...
Definition: TProof.cxx:9510
TString & Remove(Ssiz_t pos)
Definition: TString.h:668
long Long_t
Definition: RtypesCore.h:50
int Ssiz_t
Definition: RtypesCore.h:63
The Canvas class.
Definition: TCanvas.h:31
TList * GetInputList()
Get input list.
Definition: TProof.cxx:9725
R__EXTERN TProof * gProof
Definition: TProof.h:1077
A Pave (see TPave) with text, lines or/and boxes inside.
Definition: TPaveText.h:21
void DeleteParameters(const char *wildcard)
Delete the input list parameters specified by a wildcard (e.g.
Definition: TProof.cxx:9905
virtual const char * HostName()
Return the system&#39;s host name.
Definition: TSystem.cxx:311
TMap implements an associative array of (key,value) pairs using a THashTable for efficient retrieval ...
Definition: TMap.h:40
R__EXTERN TEnv * gEnv
Definition: TEnv.h:171
This class controls a Parallel ROOT Facility, PROOF, cluster.
Definition: TProof.h:316
virtual Int_t GetUid(const char *user=0)
Returns the user&#39;s id. If user = 0, returns current user&#39;s id.
Definition: TSystem.cxx:1535
Bool_t IsNull() const
Definition: TString.h:402
Mother of all ROOT objects.
Definition: TObject.h:37
virtual Bool_t Add(const char *file, const char *objname=0, const char *dir=0, Long64_t first=0, Long64_t num=-1, const char *msd=0)
Add file to list of files to be analyzed.
Definition: TDSet.cxx:1052
virtual Long64_t Process(const char *filename, Option_t *option="", Long64_t nentries=kMaxEntries, Long64_t firstentry=0)
Process all entries in this chain, calling functions in filename.
Definition: TChain.cxx:2190
virtual void Divide(Int_t nx=1, Int_t ny=1, Float_t xmargin=0.01, Float_t ymargin=0.01, Int_t color=0)
Automatic pad generation by division.
Definition: TPad.cxx:1162
virtual void Add(TObject *obj)
Definition: TList.h:87
auto * l
Definition: textangle.C:4
Class that contains a list of TFileInfo&#39;s and accumulated meta data information about its entries...
A chain is a collection of files containing TTree objects.
Definition: TChain.h:33
Int_t GetEntries() const
Return the number of objects in array (i.e.
Definition: TObjArray.cxx:522
#define c(i)
Definition: RSha256.hxx:101
TObject * GetValue(const char *keyname) const
Returns a pointer to the value associated with keyname as name of the key.
Definition: TMap.cxx:235
Int_t Atoi() const
Return integer value of string.
Definition: TString.cxx:1896
Bool_t IsDigit() const
Returns true if all characters in string are digits (0-9) or white spaces, i.e.
Definition: TString.cxx:1738
virtual Bool_t ExpandPathName(TString &path)
Expand a pathname getting rid of special shell characters like ~.
Definition: TSystem.cxx:1254
Definition: first.py:1
virtual void Update()
Update canvas pad buffers.
Definition: TCanvas.cxx:2248
virtual Int_t GetSize() const
Return the capacity of the collection, i.e.
Definition: TCollection.h:182
virtual void AddFriend(TDSet *friendset, const char *alias)
Add friend dataset to this set.
Definition: TDSet.cxx:1335
A List of entry numbers in a TTree or TChain.
Definition: TEntryList.h:25
const Bool_t kTRUE
Definition: RtypesCore.h:87
virtual const char * GetName() const
Return name of this collection.
virtual Int_t Add(TChain *chain)
Add all files referenced by the passed chain to this chain.
Definition: TChain.cxx:222
const char * Data() const
Definition: TString.h:364
virtual void SetLogy(Int_t value=1)
Set Lin/Log scale for Y.
Definition: TPad.cxx:5816