Logo ROOT  
Reference Guide
 
Loading...
Searching...
No Matches
runProof.C
Go to the documentation of this file.
1/// \file
2/// \ingroup proof
3///
4/// Macro to run examples of analysis on PROOF, corresponding to the TSelector
5/// implementations found under <ROOTSYS>/tutorials/proof .
6/// This macro uses an existing PROOF session or starts one at the indicated URL.
7/// In the case non existing PROOF session is found and no URL is given, the
8/// macro tries to start a local PROOF session.
9///
10/// To run the macro:
11///
12/// root[] .L proof/runProof.C+
13/// root[] runProof("<analysis>")
14///
15/// Currently available analysis are (to see how all this really works check
16/// the scope for the specified option inside the macro):
17///
18/// 1. "simple"
19///
20/// Selector: ProofSimple.h.C
21///
22/// root[] runProof("simple")
23///
24/// This will create a local PROOF session and run an analysis filling
25/// 100 histos with 100000 gaussian random numbers, and displaying them
26/// in a canvas with 100 pads (10x10).
27/// The number of histograms can be passed as argument 'nhist' to 'simple',
28/// e.g. to fill 16 histos with 1000000 entries use
29///
30/// root[] runProof("simple(nevt=1000000,nhist=16)")
31///
32/// The argument nhist3 controls the creation of 3d histos to simulate
33/// merging load. By default, no 3D hitogram is created.
34///
35/// 2. "h1"
36///
37/// Selector: tutorials/tree/h1analysis.h.C
38///
39/// root[] runProof("h1")
40///
41/// This runs the 'famous' H1 analysis from $ROOTSYS/tree/h1analysis.C.h.
42/// By default the data are read from the HTTP server at root.cern.ch,
43/// the data source can be changed via the argument 'h1src', e.g.
44///
45/// root[] runProof("h1,h1src=/data/h1")
46///
47/// (the directory specified must contain the 4 H1 files).
48///
49/// The 'h1' example is also used to show how to use entry-lists in PROOF.
50/// To fill the list for the events used for the final plots add the option
51/// 'fillList':
52///
53/// root[] runProof("h1,fillList")
54///
55/// To use the list previously created for the events used for the
56/// final plots add the option 'useList':
57///
58/// root[] runProof("h1,useList")
59///
60/// 3. "event"
61///
62/// Selector: ProofEvent.h,.C
63///
64/// This is an example of using PROOF par files.
65/// It runs event generation and simple analysis based on the 'Event'
66/// class found under test.
67///
68/// root[] runProof("event")
69///
70/// 4. "eventproc"
71///
72/// Selector: ProofEventProc.h.C
73///
74/// This is an example of using PROOF par files and process 'event'
75/// data from the ROOT HTTP server. It runs the ProofEventProc selector
76/// which is derived from the EventTree_Proc one found under
77/// test/ProofBench. The following specific arguments are available:
78/// - 'readall' to read the whole event, by default only the branches
79/// needed by the analysis are read (read 25% more bytes)
80/// - 'datasrc=<dir-with-files>' to read the files from another server,
81/// the files must be named 'event_<num>.root' where <num>=1,2,...
82/// or
83/// - 'datasrc=<file-with-files>' to take the file content from a text file,
84/// specified one file per line; usefull when testing differences
85/// between several sources and distributions
86/// - 'files=N' to change the number of files to be analysed (default
87/// is 10, max is 50 for the HTTP server).
88/// - 'uneven' to process uneven entries from files following the scheme
89/// {50000,5000,5000,5000,5000} and so on
90///
91/// root[] runProof("eventproc")
92///
93/// 5. "pythia8"
94///
95/// Selector: ProofPythia.h.C
96///
97/// This runs Pythia8 generation based on main03.cc example in Pythia 8.1
98///
99/// To run this analysis ROOT must be configured with pythia8.
100///
101/// Note that before executing this analysis, the env variable PYTHIA8
102/// must point to the pythia8100 (or newer) directory, in particular,
103/// $PYTHIA8/xmldoc must contain the file Index.xml. The tutorial assumes
104/// that the Pythia8 directory is the same on all machines, i.e. local
105/// and worker ones.
106///
107/// root[] runProof("pythia8")
108///
109/// 6. "ntuple"
110///
111/// Selector: ProofNtuple.h.C
112///
113/// This is an example of final merging via files created on the workers,
114/// using TProofOutputFile. The final file is called ProofNtuple.root
115/// and it is created in the directory where the tutorial is run. If
116/// the PROOF cluster is remote, the file is received by a local xrootd
117/// daemon started for the purpose. Because of this, this example can be
118/// run only on unix clients.
119///
120/// root[] runProof("ntuple")
121///
122/// By default the random numbers are generate anew. There is the
123/// possibility use a file of random numbers (to have reproducible results)
124/// by specify the option 'inputrndm', e.g.
125///
126/// root[] runProof("ntuple(inputrndm)")
127///
128/// By default the output will be saved in the local file SimpleNtuple.root;
129/// location and name of the file can be changed via the argument 'outfile',
130/// e.g.
131///
132/// root[] runProof("simplefile(outfile=/data0/testntuple.root)")
133/// root[] runProof("simplefile(outfile=root://aserver//data/testntuple.root)")
134///
135/// 7. "dataset"
136///
137/// Selector: ProofNtuple.h.C
138///
139/// This is an example of automatic creation of a dataset from files
140/// created on the workers, using TProofOutputFile. The dataset is
141/// called testNtuple and it is automatically registered and verified.
142/// The files contain the same ntuple as in previous example/tutorial 6
143/// (the same selector ProofNTuple is used with a slightly different
144/// configuration). The dataset is then used to produce the same plot
145/// as in 5 but using the DrawSelect methods of PROOF, which also show
146/// how to set style, color and other drawing attributes in PROOF.
147/// Depending on the relative worker perforance, some of the produced
148/// files may result in having no entries. If this happens, the file
149/// will be added to the missing (skipped) file list. Increasing the
150/// number of events (via nevt=...) typically solves this issue.
151///
152/// root[] runProof("dataset")
153///
154/// 8. "friends"
155///
156/// Selectors: ProofFriends.h(.C), ProofAux.h(.C)
157///
158/// This is an example of TTree friend processing in PROOF. It also shows
159/// how to use the TPacketizerFile to steer creation of files.
160///
161/// root[] runProof("friends")
162///
163/// The trees are by default created in separate files; to create
164/// them in the same file use option 'samefile', e.g.
165///
166/// root[] runProof("friends(samefile)")
167///
168/// 9. "simplefile"
169///
170/// Selector: ProofSimpleFile.h.C
171///
172/// root[] runProof("simplefile")
173///
174/// This will create a local PROOF session and run an analysis filling
175/// 16+16 histos with 100000 gaussian random numbers. The merging of
176/// these histos goes via file; 16 histos are saved in the top directory,
177/// the other 16 into a subdirectory called 'blue'. The final display
178/// is done in two canvanses, one for each set of histograms and with
179/// 16 pads each (4x4).
180/// The number of histograms in each set can be passed as argument
181/// 'nhist' to 'simplefile', e.g. to fill 25 histos with 1000000 entries use
182///
183/// root[] runProof("simplefile(nevt=1000000,nhist=25)")
184///
185/// By default the output will be saved in the local file SimpleFile.root;
186/// location and name of the file can be changed via the argument 'outfile',
187/// e.g.
188///
189/// root[] runProof("simplefile(outfile=/data0/testsimple.root)")
190/// root[] runProof("simplefile(outfile=root://aserver//data/testsimple.root)")
191///
192/// 10. "stdvec"
193///
194/// Selector: ProofStdVect.h.C
195///
196/// This is an example of using standard vectors (vector<vector<bool> > and
197/// vector<vector<float> >) in a TSelector. The same selector is run twice:
198/// in 'create' mode it creates a dataset with the tree 'stdvec' containing
199/// 3 branches, a vector<vector<bool> > and two vector<vector<float> >. The
200/// tree is saved into a file on each worker and a dataset is created with
201/// these files (the dataset is called 'TestStdVect'); in 'read' mode the
202/// dataset is read and a couple fo histograms filled and displayed.
203///
204/// root[] runProof("stdvec")
205///
206/// General arguments
207/// -----------------
208///
209/// The following arguments are valid for all examples (the ones specific
210/// to each tutorial have been explained above)
211///
212/// 1. ACLiC mode
213///
214/// By default all processing is done with ACLiC mode '+', i.e. compile
215/// if changed. However, this may lead to problems if the available
216/// selector libs were compiled in previous sessions with a different
217/// set of loaded libraries (this is a general problem in ROOT). When
218/// this happens the best solution is to force recompilation (ACLiC
219/// mode '++'). To do this just add one or more '+' to the name of the
220/// tutorial, e.g. runProof("simple++")
221///
222/// 2. debug=[what:]level
223///
224/// Controls verbosity; 'level' is an integer number and the optional string
225/// 'what' one or more of the enum names in TProofDebug.h .
226/// e.g. runProof("eventproc(debug=kPacketizer|kCollect:2)") runs 'eventproc' enabling
227/// all printouts matching TProofDebug::kPacketizer and having level
228/// equal or larger than 2 .
229///
230/// 3. nevt=N and/or first=F
231///
232/// Set the number of entries to N, eventually (when it makes sense, i.e. when
233/// processing existing files) starting from F
234/// e.g. runProof("simple(nevt=1000000000)") runs simple with 1000000000
235/// runProof("eventproc(first=65000)") runs eventproc processing
236/// starting with event 65000
237/// runProof("eventproc(nevt=100000,first=65000)") runs eventproc processing
238/// 100000 events starting with event 65000
239///
240/// 4. asyn
241///
242/// Run in non blocking mode
243/// e.g. root[] runProof("h1(asyn)")
244///
245/// 5. nwrk=N
246///
247/// Set the number of active workers to N, usefull to test performance
248/// on a remote cluster where control about the number of workers is
249/// not possible, e.g. runProof("event(nwrk=2)") runs 'event' with
250/// 2 workers.
251///
252/// 6. punzip
253///
254/// Use parallel unzipping in reading files where relevant
255/// e.g. root[] runProof("eventproc(punzip)")
256///
257/// 7. cache=<bytes> (or <kbytes>K or <mbytes>M)
258///
259/// Change the size of the tree cache; 0 or <0 disables the cache,
260/// value cane be in bytes (no suffix), kilobytes (suffix 'K') or
261/// megabytes (suffix 'M'), e.g. root[] runProof("eventproc(cache=0)")
262///
263/// 8. submergers[=S]
264///
265/// Enabling merging via S submergers or the optimal number if S is
266/// not specified, e.g. root[] runProof("simple(hist=1000,submergers)")
267///
268/// 9. rateest=average
269///
270/// Enable processed entries estimation for constant progress reporting based on
271/// the measured average. This may screw up the progress bar in some cases, which
272/// is the reason why it is not on by default .
273/// e.g. root[] runProof("eventproc(rateest=average)")
274///
275/// 10. perftree=perftreefile.root
276///
277/// Generate the perfomance tree and save it to file 'perftreefile.root',
278/// e.g. root[] runProof("eventproc(perftree=perftreefile.root)")
279///
280/// 11. feedback=name1[,name2,name3,...]|off
281///
282/// Enable feedback for the specified names or switch it off; by default it is
283/// enabled for the 'stats' histograms (events,packest, packets-being processed).
284///
285/// In all cases, to run on a remote PROOF cluster, the master URL must
286/// be passed as second argument; e.g.
287///
288/// root[] runProof("simple","master.do.main")
289///
290/// A rough parsing of the URL is done to determine the locality of the cluster.
291/// If using a tunnel the URL can start by localhost even for external clusters:
292/// in such cases the default locality determination will be wrong, so one has
293/// to tell explicity that the cluster is external via the option field, e.g.
294///
295/// root[] runProof("simple","localhost:33002/?external")
296///
297/// In the case of local running it is possible to specify the number of
298/// workers to start as third argument (the default is the number of cores
299/// of the machine), e.g.
300///
301/// root[] runProof("simple",0,4)
302///
303/// will start 4 workers. Note that the real number of workers is changed
304/// only the first time you call runProof into a ROOT session. Following
305/// calls can reduce the number of active workers, but not increase it.
306/// For example, in the same session of the call above starting 4 workers,
307/// this
308///
309/// root[] runProof("simple",0,8)
310///
311/// will still use 4 workers, while this
312///
313/// root[] runProof("simple",0,2)
314///
315/// will disable 2 workers and use the other 2.
316///
317/// Finally, it is possible to pass as 4th argument a list of objects to be added
318/// to the input list to further control the PROOF behaviour:
319///
320/// root [] TList *ins = new TList
321/// root [] ins->Add(new TParameter<Int_t>("MyParm", 3))
322/// root [] runProof("simple",0,4,ins)
323///
324/// the content of 'ins' will then be copied to the input list before processing.
325///
326///
327/// \macro_code
328///
329/// \author Gerardo Ganis
330
331
332#include "TCanvas.h"
333#include "TChain.h"
334#include "TDSet.h"
335#include "TEnv.h"
336#include "TEntryList.h"
337#include "TFile.h"
338#include "TFileCollection.h"
339#include "TFrame.h"
340#include "THashList.h"
341#include "TList.h"
342#include "TPad.h"
343#include "TPaveText.h"
344#include "TProof.h"
345#include "TProofDebug.h"
346#include "TString.h"
347
348#include "getProof.C"
349void plotNtuple(TProof *p, const char *ds, const char *ntptitle);
350void SavePerfTree(TProof *proof, const char *fn);
351
352// Variable used to locate the Pythia8 directory for the Pythia8 example
353const char *pythia8dir = 0;
354const char *pythia8data = 0;
355
356void runProof(const char *what = "simple",
357 const char *masterurl = "proof://localhost:40000",
358 Int_t nwrks = -1, TList *ins = 0)
359{
360
361 gEnv->SetValue("Proof.StatsHist",1);
362
363 TString u(masterurl);
364 // Determine locality of this session
365 Bool_t isProofLocal = kFALSE;
366 if (!u.IsNull() && u != "lite://") {
367 TUrl uu(masterurl);
368 TString uopts(uu.GetOptions());
369 if ((!strcmp(uu.GetHost(), "localhost") && !uopts.Contains("external")) ||
370 !strcmp(uu.GetHostFQDN(), TUrl(gSystem->HostName()).GetHostFQDN())) {
371 isProofLocal = kTRUE;
372 }
373 // Adjust URL
374 if (!u.BeginsWith(uu.GetProtocol())) uu.SetProtocol("proof");
375 uopts.ReplaceAll("external", "");
376 uu.SetOptions(uopts.Data());
377 u = uu.GetUrl();
378 }
379 const char *url = u.Data();
380
381 // Temp dir for PROOF tutorials
382 // Force "/tmp/<user>" whenever possible to avoid length problems on MacOsX
383 TString tmpdir("/tmp");
385 TString us;
387 if (!ug) {
388 Printf("runProof: could not get user info");
389 return;
390 }
391 us.Form("/%s", ug->fUser.Data());
392 if (!tmpdir.EndsWith(us.Data())) tmpdir += us;
393 gSystem->mkdir(tmpdir.Data(), kTRUE);
395 Printf("runProof: unable to get a writable tutorial directory (tried: %s)"
396 " - cannot continue", tmpdir.Data());
397 return;
398 }
399 TString tutdir = Form("%s/.proof-tutorial", tmpdir.Data());
400 if (gSystem->AccessPathName(tutdir)) {
401 Printf("runProof: creating the temporary directory"
402 " for the tutorial (%s) ... ", tutdir.Data());
403 if (gSystem->mkdir(tutdir, kTRUE) != 0) {
404 Printf("runProof: could not assert / create the temporary directory"
405 " for the tutorial (%s)", tutdir.Data());
406 return;
407 }
408 }
409
410 // For the Pythia8 example we need to set some environment variable;
411 // This must be done BEFORE starting the PROOF session
412 if (what && !strncmp(what, "pythia8", 7)) {
413 // We assume that the remote location of Pythia8 is the same as the local one
414 pythia8dir = gSystem->Getenv("PYTHIA8");
415 if (!pythia8dir || strlen(pythia8dir) <= 0) {
416 Printf("runProof: pythia8: environment variable PYTHIA8 undefined:"
417 " it must contain the path to pythia81xx root directory (local and remote) !");
418 return;
419 }
420 pythia8data = gSystem->Getenv("PYTHIA8DATA");
421 if (!pythia8data || strlen(pythia8data) <= 0) {
422 gSystem->Setenv("PYTHIA8DATA", Form("%s/xmldoc", pythia8dir));
423 pythia8data = gSystem->Getenv("PYTHIA8DATA");
424 if (!pythia8data || strlen(pythia8data) <= 0) {
425 Printf("runProof: pythia8: environment variable PYTHIA8DATA undefined:"
426 " it one must contain the path to pythia81xx/xmldoc"
427 " subdirectory (local and remote) !");
428 return;
429 }
430 }
431 TString env = Form("echo export PYTHIA8=%s; export PYTHIA8DATA=%s",
432 pythia8dir, pythia8data);
433 TProof::AddEnvVar("PROOF_INITCMD", env.Data());
434 }
435
436 Printf("tutorial dir:\t%s", tutdir.Data());
437
438 // Get the PROOF Session
439 TProof *proof = getProof(url, nwrks, tutdir.Data(), "ask");
440 if (!proof) {
441 Printf("runProof: could not start/attach a PROOF session");
442 return;
443 }
444
445 // Refine locality (PROOF-Lite always local)
446 if (proof->IsLite()) isProofLocal = kTRUE;
447
448#ifdef WIN32
449 if (isProofLocal && what && !strcmp(what, "ntuple", 6)) {
450 // Not support on windows
451 Printf("runProof: the 'ntuple' example needs to run xrootd to receive the output file, \n"
452 " but xrootd is not supported on Windows - cannot continue");
453 return;
454 }
455#endif
456
457 TString proofsessions(Form("%s/sessions",tutdir.Data()));
458 // Save tag of the used session
459 FILE *fs = fopen(proofsessions.Data(), "a");
460 if (!fs) {
461 Printf("runProof: could not create files for sessions tags");
462 } else {
463 fprintf(fs,"session-%s\n", proof->GetSessionTag());
464 fclose(fs);
465 }
466 if (!proof) {
467 Printf("runProof: could not start/attach a PROOF session");
468 return;
469 }
470
471 // Set the number of workers (may only reduce the number of active workers
472 // in the session)
473 if (nwrks > 0)
474 proof->SetParallel(nwrks);
475
476 // Where is the code to run
477 char *rootbin = gSystem->Which(gSystem->Getenv("PATH"), "root.exe", kExecutePermission);
478 if (!rootbin) {
479 Printf("runProof: root.exe not found: please check the environment!");
480 return;
481 }
482 TString rootsys = gSystem->GetDirName(rootbin);
483 rootsys = gSystem->GetDirName(rootsys);
484 TString tutorials(Form("%s/tutorials", rootsys.Data()));
485 delete[] rootbin;
486
487 // Parse 'what'; it is in the form 'analysis(arg1,arg2,...)'
488 TString args(what);
489 args.ReplaceAll("("," ");
490 args.ReplaceAll(")"," ");
491 args.ReplaceAll(","," ");
492 Ssiz_t from = 0;
493 TString act, tok;
494 if (!args.Tokenize(act, from, " ")) {
495 // Cannot continue
496 Printf("runProof: action not found: check your arguments (%s)", what);
497 return;
498 }
499 // Extract ACLiC mode
500 TString aMode = "+";
501 if (act.EndsWith("+")) {
502 aMode += "+";
503 while (act.EndsWith("+")) { act.Remove(TString::kTrailing,'+'); }
504 }
505 Printf("runProof: %s: ACLiC mode: '%s'", act.Data(), aMode.Data());
506
507 // Parse out number of events and 'asyn' option, used almost by every test
508 TString aNevt, aFirst, aNwrk, opt, sel, punzip("off"), aCache, aOutFile,
509 aH1Src("http://root.cern.ch/files/h1"),
510 aDebug, aDebugEnum, aRateEst, aPerfTree("perftree.root"),
511 aFeedback("fb=stats");
512 Long64_t suf = 1;
513 Int_t aSubMg = -1;
514 Bool_t useList = kFALSE, makePerfTree = kFALSE;
515 while (args.Tokenize(tok, from, " ")) {
516 // Debug controllers
517 if (tok.BeginsWith("debug=")) {
518 aDebug = tok;
519 aDebug.ReplaceAll("debug=","");
520 Int_t icol = kNPOS;
521 if ((icol = aDebug.Index(":")) != kNPOS) {
522 aDebugEnum = aDebug(0, icol);
523 aDebug.Remove(0, icol+1);
524 }
525 if (!aDebug.IsDigit()) {
526 Printf("runProof: %s: error parsing the 'debug=' option (%s) - ignoring", act.Data(), tok.Data());
527 aDebug = "";
528 aDebugEnum = "";
529 }
530 }
531 // Number of events
532 if (tok.BeginsWith("nevt=")) {
533 aNevt = tok;
534 aNevt.ReplaceAll("nevt=","");
535 if (!aNevt.IsDigit()) {
536 Printf("runProof: %s: error parsing the 'nevt=' option (%s) - ignoring", act.Data(), tok.Data());
537 aNevt = "";
538 }
539 }
540 // First event
541 if (tok.BeginsWith("first=")) {
542 aFirst = tok;
543 aFirst.ReplaceAll("first=","");
544 if (!aFirst.IsDigit()) {
545 Printf("runProof: %s: error parsing the 'first=' option (%s) - ignoring", act.Data(), tok.Data());
546 aFirst = "";
547 }
548 }
549 // Sync or async ?
550 if (tok.BeginsWith("asyn"))
551 opt = "ASYN";
552 // Number of workers
553 if (tok.BeginsWith("nwrk=")) {
554 aNwrk = tok;
555 aNwrk.ReplaceAll("nwrk=","");
556 if (!aNwrk.IsDigit()) {
557 Printf("runProof: %s: error parsing the 'nwrk=' option (%s) - ignoring", act.Data(), tok.Data());
558 aNwrk = "";
559 }
560 }
561 // Parallel unzipping ?
562 if (tok.BeginsWith("punzip"))
563 punzip = "on";
564 // Number of workers
565 if (tok.BeginsWith("cache=")) {
566 aCache = tok;
567 aCache.ReplaceAll("cache=","");
568 if (aCache.EndsWith("k")) { aCache.Remove(TString::kTrailing, 'k'); suf = 1024; }
569 if (aCache.EndsWith("K")) { aCache.Remove(TString::kTrailing, 'K'); suf = 1024; }
570 if (aCache.EndsWith("M")) { aCache.Remove(TString::kTrailing, 'M'); suf = 1024*1024; }
571 if (!aCache.IsDigit()) {
572 Printf("runProof: %s: error parsing the 'cache=' option (%s) - ignoring", act.Data(), tok.Data());
573 aCache = "";
574 }
575 }
576 // Use submergers?
577 if (tok.BeginsWith("submergers")) {
578 tok.ReplaceAll("submergers","");
579 aSubMg = 0;
580 if (tok.BeginsWith("=")) {
581 tok.ReplaceAll("=","");
582 if (tok.IsDigit()) aSubMg = tok.Atoi();
583 }
584 }
585 // H1: use entry-lists ?
586 if (tok.BeginsWith("useList")) {
587 useList = kTRUE;
588 }
589 if (tok.BeginsWith("fillList")) {
590 opt += "fillList";
591 }
592 // H1: change location of files?
593 if (tok.BeginsWith("h1src=")) {
594 tok.ReplaceAll("h1src=","");
595 if (!(tok.IsNull())) aH1Src = tok;
596 Printf("runProof: %s: reading data files from '%s'", act.Data(), aH1Src.Data());
597 }
598 // Rate estimation technique
599 if (tok.BeginsWith("rateest=")) {
600 tok.ReplaceAll("rateest=","");
601 if (!(tok.IsNull())) aRateEst = tok;
602 Printf("runProof: %s: progress-bar rate estimation option: '%s'", act.Data(), aRateEst.Data());
603 }
604 // Create and save the preformance tree?
605 if (tok.BeginsWith("perftree")) {
606 makePerfTree = kTRUE;
607 if (tok.BeginsWith("perftree=")) {
608 tok.ReplaceAll("perftree=","");
609 if (!(tok.IsNull())) aPerfTree = tok;
610 }
611 Printf("runProof: %s: saving performance tree to '%s'", act.Data(), aPerfTree.Data());
612 }
613 // Location of the output file, if any
614 if (tok.BeginsWith("outfile")) {
615 if (tok.BeginsWith("outfile=")) {
616 tok.ReplaceAll("outfile=","");
617 if (!(tok.IsNull())) aOutFile = tok;
618 }
619 Printf("runProof: %s: output file: '%s'", act.Data(), aOutFile.Data());
620 }
621 // Feedback
622 if (tok.BeginsWith("feedback=")) {
623 tok.ReplaceAll("feedback=","");
624 if (tok == "off" || tok == "OFF" || tok == "0") {
625 aFeedback = "";
626 } else if (!(tok.IsNull())) {
627 if (tok.BeginsWith("+")) {
628 tok[0] = ',';
629 aFeedback += tok;
630 } else {
631 aFeedback.Form("fb=%s", tok.Data());
632 }
633 }
634 Printf("runProof: %s: feedback: '%s'", act.Data(), aFeedback.Data());
635 }
636 }
637 Long64_t nevt = (aNevt.IsNull()) ? -1 : aNevt.Atoi();
638 Long64_t first = (aFirst.IsNull()) ? 0 : aFirst.Atoi();
639 Long64_t nwrk = (aNwrk.IsNull()) ? -1 : aNwrk.Atoi();
640 from = 0;
641
642 // Set number workers
643 if (nwrk > 0) {
644 if (proof->GetParallel() < nwrk) {
645 Printf("runProof: %s: request for a number of workers larger then available - ignored", act.Data());
646 } else {
647 proof->SetParallel(nwrk);
648 }
649 }
650
651 // Debug controllers
652 if (!aDebug.IsNull()) {
653 Int_t dbg = aDebug.Atoi();
654 Int_t scope = TProofDebug::kAll;
655 if (!aDebugEnum.IsNull()) scope = getDebugEnum(aDebugEnum.Data());
656 proof->SetLogLevel(dbg, scope);
657 Printf("runProof: %s: verbose mode for '%s'; level: %d", act.Data(), aDebugEnum.Data(), dbg);
658 }
659
660 // Have constant progress reporting based on estimated info
661 // (NB: may screw up the progress bar in some cases)
662 if (aRateEst == "average")
663 proof->SetParameter("PROOF_RateEstimation", aRateEst);
664
665 // Parallel unzip
666 if (punzip == "on") {
667 proof->SetParameter("PROOF_UseParallelUnzip", (Int_t)1);
668 Printf("runProof: %s: parallel unzip enabled", act.Data());
669 } else {
670 proof->SetParameter("PROOF_UseParallelUnzip", (Int_t)0);
671 }
672
673 // Tree cache
674 if (!aCache.IsNull()) {
675 Long64_t cachesz = aCache.Atoi() * suf;
676 if (cachesz <= 0) {
677 proof->SetParameter("PROOF_UseTreeCache", (Int_t)0);
678 Printf("runProof: %s: disabling tree cache", act.Data());
679 } else {
680 proof->SetParameter("PROOF_UseTreeCache", (Int_t)1);
681 proof->SetParameter("PROOF_CacheSize", cachesz);
682 Printf("runProof: %s: setting cache size to %lld", act.Data(), cachesz);
683 }
684 } else {
685 // Use defaults
686 proof->DeleteParameters("PROOF_UseTreeCache");
687 proof->DeleteParameters("PROOF_CacheSize");
688 }
689
690 // Enable submergers, if required
691 if (aSubMg >= 0) {
692 proof->SetParameter("PROOF_UseMergers", aSubMg);
693 if (aSubMg > 0) {
694 Printf("runProof: %s: enabling merging via %d sub-mergers", act.Data(), aSubMg);
695 } else {
696 Printf("runProof: %s: enabling merging via sub-mergers (optimal number)", act.Data());
697 }
698 } else {
699 proof->DeleteParameters("PROOF_UseMergers");
700 }
701
702 // The performance tree
703 if (makePerfTree) {
704 proof->SetParameter("PROOF_StatsHist", "");
705 proof->SetParameter("PROOF_StatsTrace", "");
706 proof->SetParameter("PROOF_SlaveStatsTrace", "");
707 }
708
709 // Additional inputs from the argument 'ins'
710 if (ins && ins->GetSize() > 0) {
711 TObject *oin = 0;
712 TIter nxo(ins);
713 while ((oin = nxo())) { proof->AddInput(oin); }
714 }
715
716 // Full lits of inputs so far
717 proof->GetInputList()->Print();
718
719 // Action
720 if (act == "simple") {
721 // ProofSimple is an example of non-data driven analysis; it
722 // creates and fills with random numbers a given number of histos
723
724 if (first > 0)
725 // Meaningless for this tutorial
726 Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
727 " - ignored", act.Data());
728
729 // Default 10000 events
730 nevt = (nevt < 0) ? 100000 : nevt;
731 // Find out the number of histograms
732 TString aNhist, aNhist3;
733 while (args.Tokenize(tok, from, " ")) {
734 // Number of histos
735 if (tok.BeginsWith("nhist=")) {
736 aNhist = tok;
737 aNhist.ReplaceAll("nhist=","");
738 if (!aNhist.IsDigit()) {
739 Printf("runProof: error parsing the 'nhist=' option (%s) - ignoring", tok.Data());
740 aNhist = "";
741 }
742 } else if (tok.BeginsWith("nhist3=")) {
743 aNhist3 = tok;
744 aNhist3.ReplaceAll("nhist3=","");
745 if (!aNhist3.IsDigit()) {
746 Printf("runProof: error parsing the 'nhist3=' option (%s) - ignoring", tok.Data());
747 aNhist3 = "";
748 }
749 }
750 }
751 Int_t nhist = (aNhist.IsNull()) ? 100 : aNhist.Atoi();
752 Int_t nhist3 = (aNhist3.IsNull()) ? -1 : aNhist3.Atoi();
753 Printf("\nrunProof: running \"simple\" with nhist= %d, nhist3=%d and nevt= %lld\n", nhist, nhist3, nevt);
754
755 // The number of histograms is added as parameter in the input list
756 proof->SetParameter("ProofSimple_NHist", (Long_t)nhist);
757 // The number of histograms is added as parameter in the input list
758 if (nhist3 > 0) proof->SetParameter("ProofSimple_NHist3", (Long_t)nhist3);
759 // The selector string
760 sel.Form("%s/proof/ProofSimple.C%s", tutorials.Data(), aMode.Data());
761 //
762 // Run it for nevt times
763 TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
764 proof->Process(sel.Data(), nevt, xopt);
765
766 } else if (act == "h1") {
767 // This is the famous 'h1' example analysis run on Proof reading the
768 // data from the ROOT http server.
769
770 // Create the chain
771 TChain *chain = new TChain("h42");
772 chain->Add(TString::Format("%s/dstarmb.root", aH1Src.Data()));
773 chain->Add(TString::Format("%s/dstarp1a.root", aH1Src.Data()));
774 chain->Add(TString::Format("%s/dstarp1b.root", aH1Src.Data()));
775 chain->Add(TString::Format("%s/dstarp2.root", aH1Src.Data()));
776 chain->ls();
777 // We run on Proof
778 chain->SetProof();
779 // Set entrylist, if required
780 if (useList) {
781 TString eln("elist"), elfn("elist.root");
782 if (gSystem->AccessPathName(elfn)) {
783 Printf("\nrunProof: asked to use an entry list but '%s' not found or not readable", elfn.Data());
784 Printf("\nrunProof: did you forget to run with 'fillList=%s'?\n", elfn.Data());
785 } else {
786 TFile f(elfn);
787 if (!(f.IsZombie())) {
788 TEntryList *elist = (TEntryList *)f.Get(eln);
789 if (elist) {
790 elist->SetDirectory(0); //otherwise the file destructor will delete elist
791 chain->SetEntryList(elist);
792 } else {
793 Printf("\nrunProof: could not find entry-list '%s' in file '%s': ignoring",
794 eln.Data(), elfn.Data());
795 }
796 } else {
797 Printf("\nrunProof: requested entry-list file '%s' not existing (or not readable):"
798 " ignoring", elfn.Data());
799 }
800 }
801 }
802 // The selector
803 sel.Form("%s/tree/h1analysis.C%s", tutorials.Data(), aMode.Data());
804 // Run it
805 Printf("\nrunProof: running \"h1\"\n");
806 TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
807 chain->Process(sel.Data(),xopt,nevt,first);
808 // Cleanup the input list
809 gProof->ClearInputData("elist");
810 gProof->ClearInputData("elist.root");
811 TIter nxi(gProof->GetInputList());
812 TObject *o = 0;
813 while ((o = nxi())) {
814 if (!strncmp(o->GetName(), "elist", 5)) {
816 delete o;
817 }
818 }
819
820 } else if (act == "pythia8") {
821
822 if (first > 0)
823 Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
824 " - ignored", act.Data());
825
826 TString path(Form("%s/Index.xml", pythia8data));
827 gSystem->ExpandPathName(path);
828 if (gSystem->AccessPathName(path)) {
829 Printf("runProof: pythia8: PYTHIA8DATA directory (%s) must"
830 " contain the Index.xml file !", pythia8data);
831 return;
832 }
833 TString pythia8par = TString::Format("%s/proof/pythia8.par", tutorials.Data());
834 if (gSystem->AccessPathName(pythia8par.Data())) {
835 Printf("runProof: pythia8: par file not found (tried %s)", pythia8par.Data());
836 return;
837 }
838 proof->UploadPackage(pythia8par);
839 proof->EnablePackage("pythia8");
840 // Show enabled packages
841 proof->ShowEnabledPackages();
842 Printf("runProof: pythia8: check settings:");
843 proof->Exec(".!echo hostname = `hostname`; echo \"ls pythia8:\"; ls pythia8");
844 // Loading libraries needed
845 if (gSystem->Load("libEG.so") < 0) {
846 Printf("runProof: pythia8: libEG not found \n");
847 return;
848 }
849 if (gSystem->Load("libEGPythia8.so") < 0) {
850 Printf("runProof: pythia8: libEGPythia8 not found \n");
851 return;
852 }
853 // Setting the default number of events, if needed
854 nevt = (nevt < 0) ? 100 : nevt;
855 Printf("\nrunProof: running \"Pythia01\" nevt= %lld\n", nevt);
856 // The selector string
857 sel.Form("%s/proof/ProofPythia.C%s", tutorials.Data(), aMode.Data());
858 // Run it for nevt times
859 TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
860 proof->Process(sel.Data(), nevt, xopt);
861
862 } else if (act == "event") {
863
864 if (first > 0)
865 // Meaningless for this tutorial
866 Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
867 " - ignored", act.Data());
868
869 TString eventpar = TString::Format("%s/proof/event.par", tutorials.Data());
870 if (gSystem->AccessPathName(eventpar.Data())) {
871 Printf("runProof: event: par file not found (tried %s)", eventpar.Data());
872 return;
873 }
874
875 proof->UploadPackage(eventpar);
876 proof->EnablePackage("event");
877 Printf("Enabled packages...\n");
878 proof->ShowEnabledPackages();
879
880 // Setting the default number of events, if needed
881 nevt = (nevt < 0) ? 100 : nevt;
882 Printf("\nrunProof: running \"event\" nevt= %lld\n", nevt);
883 // The selector string
884 sel.Form("%s/proof/ProofEvent.C%s", tutorials.Data(), aMode.Data());
885 // Run it for nevt times
886 TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
887 proof->Process(sel.Data(), nevt, xopt);
888
889 } else if (act == "eventproc") {
890
891 TString eventpar = TString::Format("%s/proof/event.par", tutorials.Data());
892 gSystem->ExpandPathName(eventpar);
893 if (gSystem->AccessPathName(eventpar.Data())) {
894 Printf("runProof: eventproc: par file not found (tried %s)", eventpar.Data());
895 return;
896 }
897
898 proof->UploadPackage(eventpar);
899 proof->EnablePackage("event");
900 Printf("Enabled packages...\n");
901 proof->ShowEnabledPackages();
902
903 // Load ProcFileElements (to check processed ranges)
904 TString pfelem = TString::Format("%s/proof/ProcFileElements.C", tutorials.Data());
905 gSystem->ExpandPathName(pfelem);
906 if (gSystem->AccessPathName(pfelem.Data())) {
907 Printf("runProof: eventproc: ProcFileElements.C not found (tried %s)", pfelem.Data());
908 return;
909 }
910 pfelem += aMode;
911 // Add include to test trasmission
912 pfelem += TString::Format(",%s/proof/EmptyInclude.h", tutorials.Data());
913 proof->Load(pfelem);
914
915 // Extract the number of files to process, data source and
916 // other parameters controlling the run ...
917 Bool_t uneven = kFALSE;
918 TString aFiles, aDataSrc("http://root.cern.ch/files/data"), aPartitions;
919 proof->SetParameter("ProofEventProc_Read", "optimized");
920 while (args.Tokenize(tok, from, " ")) {
921 // Number of events
922 if (tok.BeginsWith("files=")) {
923 aFiles = tok;
924 aFiles.ReplaceAll("files=","");
925 if (!aFiles.IsDigit()) {
926 Printf("runProof: error parsing the 'files=' option (%s) - ignoring", tok.Data());
927 aFiles = "";
928 }
929 } else if (tok.BeginsWith("datasrc=")) {
930 tok.ReplaceAll("datasrc=","");
931 if (tok.IsDigit()) {
932 Printf("runProof: error parsing the 'datasrc=' option (%s) - ignoring", tok.Data());
933 } else {
934 aDataSrc = tok;
935 Printf("runProof: reading files from: %s", aDataSrc.Data());
936 }
937 } else if (tok == "readall") {
938 proof->SetParameter("ProofEventProc_Read", "readall");
939 Printf("runProof: eventproc: reading the full event");
940 } else if (tok == "uneven") {
941 uneven = kTRUE;
942 } else if (tok.BeginsWith("partitions=")) {
943 tok.ReplaceAll("partitions=","");
944 if (tok.IsDigit()) {
945 Printf("runProof: error parsing the 'partitions=' option (%s) - ignoring", tok.Data());
946 } else {
947 aPartitions = tok;
948 Printf("runProof: partitions: %s included in packetizer operations", aPartitions.Data());
949 }
950 }
951 }
952 Int_t nFiles = (aFiles.IsNull()) ? 10 : aFiles.Atoi();
953 Printf("runProof: found aFiles: '%s', nFiles: %d", aFiles.Data(), nFiles);
954 if (nFiles > 50) {
955 Printf("runProof: max number of files is 50 - resizing request");
956 nFiles = 50;
957 }
958
959 // We create the chain now
960 TChain *c = new TChain("EventTree");
961
962 FileStat_t fst;
963 if (gSystem->GetPathInfo(aDataSrc, fst) == 0 && R_ISREG(fst.fMode) &&
965 // It is a local file, we get the TFileCollection and we inject it into the chain
966 TFileCollection *fc = new TFileCollection("", "", aDataSrc, nFiles);
967 c->AddFileInfoList(fc->GetList());
968 delete fc;
969
970 } else {
971
972 // Tokenize the source: if more than 1 we rotate the assignment. More sources can be specified
973 // separating them by a '|'
974 TObjArray *dsrcs = aDataSrc.Tokenize("|");
975 Int_t nds = dsrcs->GetEntries();
976
977 // Fill the chain
978 Int_t i = 1, k = 0;
979 TString fn;
980 for (i = 1; i <= nFiles; i++) {
981 k = (i - 1) % nds;
982 TObjString *os = (TObjString *) (*dsrcs)[k];
983 if (os) {
984 fn.Form("%s/event_%d.root", os->GetName(), i);
985 if (uneven) {
986 if ((i - 1)%5 == 0)
987 c->AddFile(fn.Data(), 50000);
988 else
989 c->AddFile(fn.Data(), 5000);
990 } else {
991 c->AddFile(fn.Data());
992 }
993 }
994 }
995 dsrcs->SetOwner();
996 delete dsrcs;
997 }
998 // Show the chain
999 c->ls();
1000 c->SetProof();
1001
1002 // Only validate the files really needed for the analysis
1003 proof->SetParameter("PROOF_ValidateByFile", 1);
1004
1005 // Send over the partition information, if any
1006 if (!aPartitions.IsNull()) {
1007 aPartitions.ReplaceAll("|", ",");
1008 proof->SetParameter("PROOF_PacketizerPartitions", aPartitions);
1009 }
1010
1011 // The selector
1012 sel.Form("%s/proof/ProofEventProc.C%s", tutorials.Data(), aMode.Data());
1013 // Run it
1014 Printf("\nrunProof: running \"eventproc\"\n");
1015 TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
1016 c->Process(sel.Data(), xopt, nevt, first);
1017
1018 } else if (act == "ntuple") {
1019
1020 // ProofNtuple is an example of non-data driven analysis; it
1021 // creates and fills a disk resident ntuple with automatic file merging
1022
1023 if (first > 0)
1024 // Meaningless for this tutorial
1025 Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
1026 " - ignored", act.Data());
1027
1028 // Set the default number of events, if needed
1029 nevt = (nevt < 0) ? 1000 : nevt;
1030 Printf("\nrunProof: running \"ntuple\" with nevt= %lld\n", nevt);
1031
1032 // Which randoms to use
1033 Bool_t usentprndm = kFALSE;
1034 while (args.Tokenize(tok, from, " ")) {
1035 if (tok == "inputrndm") {
1036 usentprndm = kTRUE;
1037 break;
1038 }
1039 }
1040 if (usentprndm) Printf("runProof: taking randoms from input ntuple\n");
1041
1042 // Output file
1043 TString fout(aOutFile);
1044 if (fout.IsNull()) {
1045 fout.Form("%s/ProofNtuple.root", gSystem->WorkingDirectory());
1046 // Cleanup any existing instance of the output file
1047 gSystem->Unlink(fout);
1048
1049 if (!isProofLocal) {
1050 // Setup a local basic xrootd to receive the file
1051 Bool_t xrdok = kFALSE;
1052 Int_t port = 9000;
1053 while (port < 9010) {
1054 if (checkXrootdAt(port) != 1) {
1055 if (startXrootdAt(port, gSystem->WorkingDirectory(), kTRUE) == 0) {
1056 xrdok = kTRUE;
1057 break;
1058 }
1059 }
1060 port++;
1061 }
1062 if (!xrdok) {
1063 Printf("runProof: could not start basic xrootd on ports 9000-9009 - cannot continue");
1064 return;
1065 }
1066 fout.Insert(0, TString::Format("root://%s:%d/", TUrl(gSystem->HostName()).GetHostFQDN(), port));
1067 // Make a copy of the files on the master before merging
1068 proof->AddInput(new TNamed("PROOF_OUTPUTFILE_LOCATION", "LOCAL"));
1069 }
1070 }
1071 proof->AddInput(new TNamed("PROOF_OUTPUTFILE", fout.Data()));
1072
1073 // If using the 'NtpRndm' for a fixed values of randoms, send over the file
1074 if (usentprndm) {
1075 // The file with 'NtpRndm'
1076 TString fnr = TString::Format("%s/proof/ntprndm.root", tutorials.Data());
1077 // Set as input data
1078 proof->SetInputDataFile(fnr);
1079 // Set the related parameter
1080 proof->SetParameter("PROOF_USE_NTP_RNDM","yes");
1081 // Notify
1082 Printf("runProof: taking randoms from '%s'", fnr.Data());
1083 }
1084
1085 // The selector string
1086 sel.Form("%s/proof/ProofNtuple.C%s", tutorials.Data(), aMode.Data());
1087
1088 // Run it for nevt times
1089 TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
1090 Printf("runProof: selector file '%s', options: '%s'", sel.Data(), xopt.Data());
1091 proof->Process(sel.Data(), nevt, xopt);
1092
1093 // Reset input variables
1094 if (usentprndm) {
1095 proof->DeleteParameters("PROOF_USE_NTP_RNDM");
1096 proof->SetInputDataFile(0);
1097 }
1098
1099 } else if (act == "dataset") {
1100
1101 // This is an example of analysis creating data files on each node which are
1102 // automatically registered as dataset; the newly created dataset is used to create
1103 // the final plots. The data are of the same type as for the 'ntuple' example.
1104 // Selector used: ProofNtuple
1105
1106 if (first > 0)
1107 // Meaningless for this tutorial
1108 Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
1109 " - ignored", act.Data());
1110
1111 // Set the default number of events, if needed
1112 nevt = (nevt < 0) ? 1000000 : nevt;
1113 Printf("\nrunProof: running \"dataset\" with nevt= %lld\n", nevt);
1114
1115 // Ask for registration of the dataset (the default is the the TFileCollection is return
1116 // without registration; the name of the TFileCollection is the name of the dataset
1117 proof->SetParameter("SimpleNtuple.root","testNtuple");
1118
1119 // Do not plot the ntuple at this level
1120 proof->SetParameter("PROOF_NTUPLE_DONT_PLOT", "");
1121
1122 // The selector string
1123 sel.Form("%s/proof/ProofNtuple.C%s", tutorials.Data(), aMode.Data());
1124 //
1125 // Run it for nevt times
1126 TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
1127 proof->Process(sel.Data(), nevt, xopt);
1128
1129 // The TFileCollection must be in the output
1130 if (proof->GetOutputList()->FindObject("testNtuple")) {
1131
1132 // Plot the ntuple via PROOF (example of drawing PROOF actions)
1133 plotNtuple(proof, "testNtuple", "proof ntuple from dataset");
1134
1135 } else {
1136 Printf("runProof: dataset 'testNtuple' not found in the output list");
1137 }
1138 // Do not plot the ntuple at this level
1139 proof->DeleteParameters("PROOF_NTUPLE_DONT_PLOT");
1140 proof->DeleteParameters("SimpleNtuple.root");
1141
1142 } else if (act == "friends") {
1143
1144 // This is an example of analysis creating two data files on each node (the main tree
1145 // and its friend) which are then processed as 'friends' to create the final plots.
1146 // Selector used: ProofFriends, ProofAux
1147
1148 if (first > 0)
1149 // Meaningless for this tutorial
1150 Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
1151 " - ignored", act.Data());
1152
1153 // Find out whether to use the same file or separate files
1154 Bool_t sameFile = kFALSE;
1155 while (args.Tokenize(tok, from, " ")) {
1156 // Number of histos
1157 if (tok == "samefile") {
1158 sameFile = kTRUE;
1159 break;
1160 }
1161 }
1162
1163 // File generation: we use TPacketizerFile in here to create two files per node
1164 TList *wrks = proof->GetListOfSlaveInfos();
1165 if (!wrks) {
1166 Printf("runProof: could not get the list of information about the workers");
1167 return;
1168 }
1169 // Create the map
1170 TString fntree;
1171 TMap *files = new TMap;
1172 files->SetName("PROOF_FilesToProcess");
1173 TIter nxwi(wrks);
1174 TSlaveInfo *wi = 0;
1175 while ((wi = (TSlaveInfo *) nxwi())) {
1176 fntree.Form("tree_%s.root", wi->GetOrdinal());
1177 THashList *wrklist = (THashList *) files->GetValue(wi->GetName());
1178 if (!wrklist) {
1179 wrklist = new THashList;
1180 wrklist->SetName(wi->GetName());
1181 files->Add(new TObjString(wi->GetName()), wrklist);
1182 }
1183 wrklist->Add(new TObjString(fntree));
1184 }
1185
1186 // Generate the files
1187 proof->AddInput(files);
1188 if (sameFile) {
1189 Printf("runProof: friend tree stored in the same file as the main tree");
1190 proof->SetParameter("ProofAux_Action", "GenerateTreesSameFile");
1191 } else {
1192 proof->SetParameter("ProofAux_Action", "GenerateTrees");
1193 }
1194 // Default 1000 events
1195 nevt = (nevt < 0) ? 10000 : nevt;
1196 proof->SetParameter("ProofAux_NEvents", (Long64_t)nevt);
1197 // Special Packetizer
1198 proof->SetParameter("PROOF_Packetizer", "TPacketizerFile");
1199 // Now process
1200 sel.Form("%s/proof/ProofAux.C%s", tutorials.Data(), aMode.Data());
1201 proof->Process(sel.Data(), 1);
1202 // Remove the packetizer specifications
1203 proof->DeleteParameters("PROOF_Packetizer");
1204
1205 // Print the lists and create the TDSet objects
1206 TDSet *dset = new TDSet("Tmain", "Tmain");
1207 TDSet *dsetf = new TDSet("Tfrnd", "Tfrnd");
1208 if (proof->GetOutputList()) {
1209 TIter nxo(proof->GetOutputList());
1210 TObject *o = 0;
1211 TObjString *os = 0;
1212 while ((o = nxo())) {
1213 TList *l = dynamic_cast<TList *> (o);
1214 if (l && !strncmp(l->GetName(), "MainList-", 9)) {
1215 TIter nxf(l);
1216 while ((os = (TObjString *) nxf()))
1217 dset->Add(os->GetName());
1218 }
1219 }
1220 nxo.Reset();
1221 while ((o = nxo())) {
1222 TList *l = dynamic_cast<TList *> (o);
1223 if (l && !strncmp(l->GetName(), "FriendList-", 11)) {
1224 TIter nxf(l);
1225 while ((os = (TObjString *) nxf()))
1226 dsetf->Add(os->GetName());
1227 }
1228 }
1229 }
1230 // Process with friends
1231 dset->AddFriend(dsetf, "friend");
1232 sel.Form("%s/proof/ProofFriends.C%s", tutorials.Data(), aMode.Data());
1233 TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
1234 dset->Process(sel, xopt);
1235 // Clear the files created by this run
1237
1238 } else if (act == "simplefile") {
1239
1240 // ProofSimpleFile is an example of non-data driven analysis with merging
1241 // via file and objcets saved in different directories; it creates and
1242 // fills with random numbers two sets of a given number of histos
1243
1244 if (first > 0)
1245 // Meaningless for this tutorial
1246 Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
1247 " - ignored", act.Data());
1248
1249 // Default 100000 events
1250 nevt = (nevt < 0) ? 1000000 : nevt;
1251 // Find out the number of histograms
1252 TString aNhist;
1253 while (args.Tokenize(tok, from, " ")) {
1254 // Number of histos
1255 if (tok.BeginsWith("nhist=")) {
1256 aNhist = tok;
1257 aNhist.ReplaceAll("nhist=","");
1258 if (!aNhist.IsDigit()) {
1259 Printf("runProof: error parsing the 'nhist=' option (%s) - ignoring", tok.Data());
1260 aNhist = "";
1261 }
1262 }
1263 }
1264 Int_t nhist = (aNhist.IsNull()) ? 16 : aNhist.Atoi();
1265 Printf("\nrunProof: running \"simplefile\" with nhist= %d and nevt= %lld\n", nhist, nevt);
1266
1267 // The number of histograms is added as parameter in the input list
1268 proof->SetParameter("ProofSimple_NHist", (Long_t)nhist);
1269
1270 // Output file
1271 TString fout(aOutFile);
1272 if (fout.IsNull()) {
1273 fout.Form("%s/SimpleFile.root", gSystem->WorkingDirectory());
1274 // Cleanup any existing instance of the output file
1275 gSystem->Unlink(fout);
1276
1277 if (!isProofLocal) {
1278 // Setup a local basic xrootd to receive the file
1279 Bool_t xrdok = kFALSE;
1280 Int_t port = 9000;
1281 while (port < 9010) {
1282 if (checkXrootdAt(port) != 1) {
1283 if (startXrootdAt(port, gSystem->WorkingDirectory(), kTRUE) == 0) {
1284 xrdok = kTRUE;
1285 break;
1286 }
1287 }
1288 port++;
1289 }
1290 if (!xrdok) {
1291 Printf("runProof: could not start basic xrootd on ports 9000-9009 - cannot continue");
1292 return;
1293 }
1294 fout.Insert(0, TString::Format("root://%s:%d/", TUrl(gSystem->HostName()).GetHostFQDN(), port));
1295 // Make a copy of the files on the master before merging
1296 proof->AddInput(new TNamed("PROOF_OUTPUTFILE_LOCATION", "LOCAL"));
1297 }
1298 }
1299 proof->AddInput(new TNamed("PROOF_OUTPUTFILE", fout.Data()));
1300
1301 // The selector string
1302 sel.Form("%s/proof/ProofSimpleFile.C%s", tutorials.Data(), aMode.Data());
1303 //
1304 // Run it for nevt times
1305 TString xopt = aFeedback; if (!opt.IsNull()) xopt += TString::Format(" %s", opt.Data());
1306 proof->Process(sel.Data(), nevt, xopt);
1307
1308 } else if (act == "stdvec") {
1309
1310 // This is an example of runnign a TSelector using standard vectors
1311 // Selector used: ProofStdVect
1312
1313 if (first > 0)
1314 // Meaningless for this tutorial
1315 Printf("runProof: %s: warning concept of 'first' meaningless for this tutorial"
1316 " - ignored", act.Data());
1317
1318 // Set the default number of events, if needed
1319 nevt = (nevt < 0) ? 50000 * proof->GetParallel() : nevt;
1320 Printf("\nrunProof: running \"stdvec\" with nevt= %lld\n", nevt);
1321
1322 // The selector string
1323 sel.Form("%s/proof/ProofStdVect.C%s", tutorials.Data(), aMode.Data());
1324
1325 TString xopt;
1326 // Create the dataset 'TestStdVect' with 'nevt' events
1327 xopt.Form("%s %s create", aFeedback.Data(), opt.Data());
1328 proof->Process(sel.Data(), nevt, xopt);
1329
1330 // The dataset must have been registered
1331 if (proof->ExistsDataSet("TestStdVect")) {
1332
1333 // Use dataset 'TestStdVect'
1334 xopt.Form("%s %s", aFeedback.Data(), opt.Data());
1335 proof->Process("TestStdVect", sel.Data(), xopt);
1336
1337 } else {
1338 Printf("runProof: dataset 'TestStdVect' not available!");
1339 }
1340
1341 } else {
1342 // Do not know what to run
1343 Printf("runProof: unknown tutorial: %s", what);
1344 }
1345
1346 // Save the performance tree
1347 if (makePerfTree) {
1348 SavePerfTree(proof, aPerfTree.Data());
1349 // Cleanup parameters
1350 gProof->DeleteParameters("PROOF_StatsHist");
1351 gProof->DeleteParameters("PROOF_StatsTrace");
1352 gProof->DeleteParameters("PROOF_SlaveStatsTrace");
1353 }
1354}
1355
1356//_______________________________________________________________________________________
1357void plotNtuple(TProof *p, const char *ds, const char *ntptitle)
1358{
1359 // Make some plots from the ntuple 'ntp' via PROOF
1360
1361 //
1362 // Create a canvas, with 2 pads
1363 //
1364 TCanvas *c1 = new TCanvas(Form("cv-%s", ds), ntptitle,800,10,700,780);
1365 c1->Divide(1,2);
1366 TPad *pad1 = (TPad *) c1->GetPad(1);
1367 TPad *pad2 = (TPad *) c1->GetPad(2);
1368 //
1369 // Display a function of one ntuple column imposing a condition
1370 // on another column.
1371 pad1->cd();
1372 pad1->SetGrid();
1373 pad1->SetLogy();
1374 pad1->GetFrame()->SetFillColor(15);
1375
1376 p->SetParameter("PROOF_LineColor", (Int_t)1);
1377 p->SetParameter("PROOF_FillStyle", (Int_t)1001);
1378 p->SetParameter("PROOF_FillColor", (Int_t)45);
1379 p->DrawSelect(ds, "3*px+2","px**2+py**2>1");
1380 p->SetParameter("PROOF_FillColor", (Int_t)38);
1381 p->DrawSelect(ds, "2*px+2","pz>2","same");
1382 p->SetParameter("PROOF_FillColor", (Int_t)5);
1383 p->DrawSelect(ds, "1.3*px+2","(px^2+py^2>4) && py>0","same");
1384 pad1->RedrawAxis();
1385
1386 //
1387 // Display a 3-D scatter plot of 3 columns. Superimpose a different selection.
1388 pad2->cd();
1389 p->DrawSelect(ds, "pz:py:px","(pz<10 && pz>6)+(pz<4 && pz>3)");
1390 p->SetParameter("PROOF_MarkerColor", (Int_t)4);
1391 p->DrawSelect(ds, "pz:py:px","pz<6 && pz>4","same");
1392 p->SetParameter("PROOF_MarkerColor", (Int_t)5);
1393 p->DrawSelect(ds, "pz:py:px","pz<4 && pz>3","same");
1394 TPaveText *l2 = new TPaveText(0.,0.6,0.9,0.95);
1395 l2->SetFillColor(42);
1396 l2->SetTextAlign(12);
1397 l2->AddText("You can interactively rotate this view in 2 ways:");
1398 l2->AddText(" - With the RotateCube in clicking in this pad");
1399 l2->AddText(" - Selecting View with x3d in the View menu");
1400 l2->Draw();
1401
1402 // Final update
1403 c1->cd();
1404 c1->Update();
1405
1406 // Clear parameters used for the plots
1407 p->DeleteParameters("PROOF_*Color");
1408 p->DeleteParameters("PROOF_*Style");
1409}
1410
1411//______________________________________________________________________________
1412void SavePerfTree(TProof *proof, const char *fn)
1413{
1414 // Save PROOF timing information from TPerfStats to file 'fn'
1415
1416 if (!proof) {
1417 Printf("PROOF must be run to save output performance information");;
1418 return;
1419 }
1420 if (!proof->GetOutputList() || proof->GetOutputList()->GetSize() <= 0) {
1421 Printf("PROOF outputlist undefined or empty");;
1422 return;
1423 }
1424
1425 TFile f(fn, "RECREATE");
1426 if (f.IsZombie()) {
1427 Printf("ERROR: could not open file '%s' for writing", fn);;
1428 } else {
1429 f.cd();
1430 TIter nxo(proof->GetOutputList());
1431 TObject* obj = 0;
1432 while ((obj = nxo())) {
1433 TString objname(obj->GetName());
1434 if (objname.BeginsWith("PROOF_")) {
1435 // Must list the objects since other PROOF_ objects exist
1436 // besides timing objects
1437 if (objname == "PROOF_PerfStats" ||
1438 objname == "PROOF_PacketsHist" ||
1439 objname == "PROOF_EventsHist" ||
1440 objname == "PROOF_NodeHist" ||
1441 objname == "PROOF_LatencyHist" ||
1442 objname == "PROOF_ProcTimeHist" ||
1443 objname == "PROOF_CpuTimeHist")
1444 obj->Write();
1445 }
1446 }
1447 f.Close();
1448 }
1449
1450}
#define f(i)
Definition RSha256.hxx:104
#define c(i)
Definition RSha256.hxx:101
const Ssiz_t kNPOS
Definition RtypesCore.h:124
int Int_t
Definition RtypesCore.h:45
int Ssiz_t
Definition RtypesCore.h:67
const Bool_t kFALSE
Definition RtypesCore.h:101
long Long_t
Definition RtypesCore.h:54
bool Bool_t
Definition RtypesCore.h:63
long long Long64_t
Definition RtypesCore.h:80
const Bool_t kTRUE
Definition RtypesCore.h:100
R__EXTERN TEnv * gEnv
Definition TEnv.h:170
R__EXTERN TProof * gProof
Definition TProof.h:1077
char * Form(const char *fmt,...)
void Printf(const char *fmt,...)
@ kExecutePermission
Definition TSystem.h:45
@ kReadPermission
Definition TSystem.h:47
@ kWritePermission
Definition TSystem.h:46
Bool_t R_ISREG(Int_t mode)
Definition TSystem.h:118
R__EXTERN TSystem * gSystem
Definition TSystem.h:559
static struct mg_connection * fc(struct mg_context *ctx)
Definition civetweb.c:3728
virtual void SetFillColor(Color_t fcolor)
Set the fill area color.
Definition TAttFill.h:37
virtual void SetTextAlign(Short_t align=11)
Set the text alignment.
Definition TAttText.h:42
The Canvas class.
Definition TCanvas.h:23
A chain is a collection of files containing TTree objects.
Definition TChain.h:33
virtual Long64_t Process(const char *filename, Option_t *option="", Long64_t nentries=kMaxEntries, Long64_t firstentry=0)
Process all entries in this chain, calling functions in filename.
Definition TChain.cxx:2233
virtual void SetEntryList(TEntryList *elist, Option_t *opt="")
Set the input entry list (processing the entries of the chain will then be limited to the entries in ...
Definition TChain.cxx:2683
virtual Int_t Add(TChain *chain)
Add all files referenced by the passed chain to this chain.
Definition TChain.cxx:218
virtual void SetProof(Bool_t on=kTRUE, Bool_t refresh=kFALSE, Bool_t gettreeheader=kFALSE)
Enable/Disable PROOF processing on the current default Proof (gProof).
Definition TChain.cxx:2992
virtual void ls(Option_t *option="") const
List the chain.
Definition TChain.cxx:1846
virtual void ls(Option_t *option="") const
List (ls) all objects in this collection.
virtual void Print(Option_t *option="") const
Default print for collections, calls Print(option, 1).
void SetName(const char *name)
virtual void SetOwner(Bool_t enable=kTRUE)
Set whether this collection is the owner (enable==true) of its content.
virtual Int_t GetSize() const
Return the capacity of the collection, i.e.
This class implements a data set to be used for PROOF processing.
Definition TDSet.h:153
virtual Bool_t Add(const char *file, const char *objname=0, const char *dir=0, Long64_t first=0, Long64_t num=-1, const char *msd=0)
Add file to list of files to be analyzed.
Definition TDSet.cxx:1052
virtual void AddFriend(TDSet *friendset, const char *alias)
Add friend dataset to this set.
Definition TDSet.cxx:1335
virtual Long64_t Process(TSelector *selector, Option_t *option="", Long64_t nentries=-1, Long64_t firstentry=0, TObject *enl=0)
Process TDSet on currently active PROOF session.
Definition TDSet.cxx:919
A List of entry numbers in a TTree or TChain.
Definition TEntryList.h:26
virtual void SetDirectory(TDirectory *dir)
Add reference to directory dir. dir can be 0.
virtual void SetValue(const char *name, const char *value, EEnvLevel level=kEnvChange, const char *type=nullptr)
Set the value of a resource or create a new resource.
Definition TEnv.cxx:736
Class that contains a list of TFileInfo's and accumulated meta data information about its entries.
A ROOT file is a suite of consecutive data records (TKey instances) with a well defined format.
Definition TFile.h:54
THashList implements a hybrid collection class consisting of a hash table and a list to store TObject...
Definition THashList.h:34
A doubly linked list.
Definition TList.h:38
virtual void Add(TObject *obj)
Definition TList.h:81
virtual TObject * Remove(TObject *obj)
Remove object from the list.
Definition TList.cxx:822
virtual TObject * FindObject(const char *name) const
Find an object in this list using its name.
Definition TList.cxx:578
TMap implements an associative array of (key,value) pairs using a THashTable for efficient retrieval ...
Definition TMap.h:40
void Add(TObject *obj)
This function may not be used (but we need to provide it since it is a pure virtual in TCollection).
Definition TMap.cxx:54
TObject * GetValue(const char *keyname) const
Returns a pointer to the value associated with keyname as name of the key.
Definition TMap.cxx:236
The TNamed class is the base class for all named ROOT classes.
Definition TNamed.h:29
An array of TObjects.
Definition TObjArray.h:31
Int_t GetEntries() const
Return the number of objects in array (i.e.
Collectable string class.
Definition TObjString.h:28
const char * GetName() const
Returns name of object.
Definition TObjString.h:38
Mother of all ROOT objects.
Definition TObject.h:41
virtual Int_t Write(const char *name=0, Int_t option=0, Int_t bufsize=0)
Write this object to the current directory.
Definition TObject.cxx:868
virtual const char * GetName() const
Returns name of object.
Definition TObject.cxx:429
The most important graphics class in the ROOT system.
Definition TPad.h:26
void SetGrid(Int_t valuex=1, Int_t valuey=1) override
Definition TPad.h:326
void SetLogy(Int_t value=1) override
Set Lin/Log scale for Y.
Definition TPad.cxx:5967
void RedrawAxis(Option_t *option="") override
Redraw the frame axis.
Definition TPad.cxx:5313
TVirtualPad * cd(Int_t subpadnumber=0) override
Set Current pad.
Definition TPad.cxx:604
TFrame * GetFrame() override
Get frame.
Definition TPad.cxx:2866
TVirtualPad * GetPad(Int_t subpadnumber) const override
Get a pointer to subpadnumber of this pad.
Definition TPad.cxx:2911
A Pave (see TPave) with text, lines or/and boxes inside.
Definition TPaveText.h:21
virtual TText * AddText(Double_t x1, Double_t y1, const char *label)
Add a new Text line to this pavetext at given coordinates.
virtual void Draw(Option_t *option="")
Draw this pavetext with its current attributes.
This class controls a Parallel ROOT Facility, PROOF, cluster.
Definition TProof.h:316
const char * GetSessionTag() const
Definition TProof.h:909
Int_t Exec(const char *cmd, ESlaves list, Bool_t plusMaster)
Send command to be executed on the PROOF master and/or slaves.
Definition TProof.cxx:6533
Int_t UploadPackage(const char *par, EUploadPackageOpt opt=kUntar, TList *workers=0)
Upload a PROOF archive (PAR file).
Definition TProof.cxx:8441
void ClearData(UInt_t what=kUnregistered, const char *dsname=0)
Remove files for the data directory.
Definition TProof.cxx:7412
Int_t SetParallel(Int_t nodes=-1, Bool_t random=kFALSE)
Tell PROOF how many slaves to use in parallel.
Definition TProof.cxx:7140
static void AddEnvVar(const char *name, const char *value)
Add an variable to the list of environment variables passed to proofserv on the master and slaves.
Definition TProof.cxx:11760
@ kUnregistered
Definition TProof.h:387
@ kForceClear
Definition TProof.h:389
void DeleteParameters(const char *wildcard)
Delete the input list parameters specified by a wildcard (e.g.
Definition TProof.cxx:9933
void SetInputDataFile(const char *datafile)
Set the file to be used to optimally distribute the input data objects.
Definition TProof.cxx:9585
void SetParameter(const char *par, const char *value)
Set input list parameter.
Definition TProof.cxx:9822
Int_t GetParallel() const
Returns number of slaves active in parallel mode.
Definition TProof.cxx:2304
virtual Long64_t DrawSelect(TDSet *dset, const char *varexp, const char *selection="", Option_t *option="", Long64_t nentries=-1, Long64_t firstentry=0)
Execute the specified drawing action on a data set (TDSet).
Definition TProof.cxx:6146
virtual Int_t Load(const char *macro, Bool_t notOnClient=kFALSE, Bool_t uniqueOnly=kTRUE, TList *wrks=0)
Load the specified macro on master, workers and, if notOnClient is kFALSE, on the client.
Definition TProof.cxx:8628
Bool_t IsLite() const
Definition TProof.h:933
TList * GetOutputList()
Get list with all object created during processing (see Process()).
Definition TProof.cxx:9808
void ClearInputData(TObject *obj=0)
Remove obj form the input data list; if obj is null (default), clear the input data info.
Definition TProof.cxx:9538
TList * GetListOfSlaveInfos()
Returns list of TSlaveInfo's. In case of error return 0.
Definition TProof.cxx:2321
void SetLogLevel(Int_t level, UInt_t mask=TProofDebug::kAll)
Set server logging level.
Definition TProof.cxx:7079
virtual Long64_t Process(TDSet *dset, const char *selector, Option_t *option="", Long64_t nentries=-1, Long64_t firstentry=0)
Process a data set (TDSet) using the specified selector (.C) file or Tselector object Entry- or event...
Definition TProof.cxx:5303
virtual Bool_t ExistsDataSet(const char *dataset)
Returns kTRUE if 'dataset' exists, kFALSE otherwise.
Definition TProof.cxx:10868
void AddInput(TObject *obj)
Add objects that might be needed during the processing of the selector (see Process()).
Definition TProof.cxx:9734
TList * GetInputList()
Get input list.
Definition TProof.cxx:9753
void ShowEnabledPackages(Bool_t all=kFALSE)
List which packages are enabled.
Definition TProof.cxx:7818
Int_t EnablePackage(const char *package, Bool_t notOnClient=kFALSE, TList *workers=0)
Enable specified package.
Definition TProof.cxx:8175
const char * GetOrdinal() const
Definition TProof.h:232
const char * GetName() const
Returns name of object.
Definition TProof.h:231
Basic string class.
Definition TString.h:136
Int_t Atoi() const
Return integer value of string.
Definition TString.cxx:1946
Bool_t EndsWith(const char *pat, ECaseCompare cmp=kExact) const
Return true if string ends with the specified string.
Definition TString.cxx:2202
const char * Data() const
Definition TString.h:369
Bool_t IsDigit() const
Returns true if all characters in string are digits (0-9) or white spaces, i.e.
Definition TString.cxx:1788
TString & ReplaceAll(const TString &s1, const TString &s2)
Definition TString.h:692
@ kTrailing
Definition TString.h:267
Bool_t BeginsWith(const char *s, ECaseCompare cmp=kExact) const
Definition TString.h:615
Bool_t IsNull() const
Definition TString.h:407
TString & Remove(Ssiz_t pos)
Definition TString.h:673
static TString Format(const char *fmt,...)
Static method which formats a string using a printf style format descriptor and return a TString.
Definition TString.cxx:2336
void Form(const char *fmt,...)
Formats a string using a printf style format descriptor.
Definition TString.cxx:2314
Ssiz_t Index(const char *pat, Ssiz_t i=0, ECaseCompare cmp=kExact) const
Definition TString.h:639
virtual Bool_t ExpandPathName(TString &path)
Expand a pathname getting rid of special shell characters like ~.
Definition TSystem.cxx:1274
virtual const char * Getenv(const char *env)
Get environment variable.
Definition TSystem.cxx:1663
virtual int mkdir(const char *name, Bool_t recursive=kFALSE)
Make a file system directory.
Definition TSystem.cxx:907
virtual int Load(const char *module, const char *entry="", Bool_t system=kFALSE)
Load a shared library.
Definition TSystem.cxx:1855
int GetPathInfo(const char *path, Long_t *id, Long_t *size, Long_t *flags, Long_t *modtime)
Get info about a file: id, size, flags, modification time.
Definition TSystem.cxx:1398
virtual Bool_t AccessPathName(const char *path, EAccessMode mode=kFileExists)
Returns FALSE if one can access a file using the specified access mode.
Definition TSystem.cxx:1296
virtual const char * HostName()
Return the system's host name.
Definition TSystem.cxx:306
virtual Int_t GetUid(const char *user=nullptr)
Returns the user's id. If user = 0, returns current user's id.
Definition TSystem.cxx:1560
virtual const char * WorkingDirectory()
Return working directory.
Definition TSystem.cxx:872
virtual char * Which(const char *search, const char *file, EAccessMode mode=kFileExists)
Find location of file in a search path.
Definition TSystem.cxx:1546
virtual void Setenv(const char *name, const char *value)
Set environment variable.
Definition TSystem.cxx:1647
virtual TString GetDirName(const char *pathname)
Return the directory name in pathname.
Definition TSystem.cxx:1032
virtual int Unlink(const char *name)
Unlink, i.e.
Definition TSystem.cxx:1381
virtual UserGroup_t * GetUserInfo(Int_t uid)
Returns all user info in the UserGroup_t structure.
Definition TSystem.cxx:1599
virtual const char * TempDirectory() const
Return a user configured or systemwide directory to create temporary files in.
Definition TSystem.cxx:1482
This class represents a WWW compatible URL.
Definition TUrl.h:33
const char * GetHostFQDN() const
Return fully qualified domain name of url host.
Definition TUrl.cxx:471
Attaches to a PROOF session, possibly at the indicated URL.
return c1
Definition legend1.C:41
Definition first.py:1
static const char * what
Definition stlLoader.cc:6
Int_t fMode
Definition TSystem.h:127
TString fUser
Definition TSystem.h:141
auto * l
Definition textangle.C:4