TProof is the interface class to the PROOF session. Everything done in the PROOF session is done using the methods of the TProof class.
TProof has many methods which we will encountered while trying to use PROOF. We start with two of them which allow to examine the session:
This method allows to print a summary status of the session. By default it shows information about the client and the master:
root [7] proof->Print()
Connected to: alice-caf.cern.ch (valid)
Port number: 1093
User: ganis
ROOT version|rev: 5.33/03|r43741
Architecture-Compiler: macosx64-gcc421
Proofd protocol version: 34
Client protocol version: 35
Remote protocol version: 34
Log level: 0
Session unique tag: lxbsq1409-1334421396-5704
os: 'root://alice-caf.cern.ch///pool/data/01/xrdnamespace'
Default data pool: root://alice-caf.cern.ch///pool/data/01/xrdnamespace
*** Master server 0 (parallel mode, 10 workers):
Master host name: lxbsq1409.cern.ch
Port number: 1093
User/Group: ganis/default
ROOT version|rev|tag: 5.33/03|current
Architecture-Compiler: linuxx8664gcc-gcc412
Protocol version: 34
Image name: lxbsq1409.cern.ch:/pool/PROOF-AAF/proof//proofbox/ganis
Working directory: /pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334421396-5704/master-0-lxbsq1409-1334421396-5704
Config directory:
Config file:
Log level: 0
Number of workers: 10
Number of active workers: 10
Number of unique workers: 9
Number of inactive workers: 0
Number of bad workers: 0
Total MBs processed: 0.00
Total real time used (s): 0.006
Total CPU time used (s): 0.000
root [8]
In the case of PROOF-Lite, the master and client collapsed in one entity:
root [9] plite->Print()
*** PROOF-Lite cluster (parallel mode, 2 workers):
Host name: macphsft12.local
User: ganis
ROOT version|rev|tag: 5.33/03|r43741
Architecture-Compiler: macosx64-gcc421
Protocol version: 35
Working directory: /Users/ganis/local/root/opt/root
Communication path: /var/folders/uC/uC0RGjQUFlmzR689bg+JJU++0gQ/-Tmp-/plite-93716
Log level: 0
Number of workers: 2
Number of active workers: 2
Number of unique workers: 1
Number of inactive workers: 0
Number of bad workers: 0
Total MBs processed: 0.00
Total real time used (s): 0.000
Total CPU time used (s): 0.000
Adding the argument "a" (for 'all') information about the workers is also displayed:
root [8] proof->Print("a")
Connected to: alice-caf.cern.ch (valid)
Port number: 1093
User: ganis
ROOT version|rev: 5.33/03|r43741
Architecture-Compiler: macosx64-gcc421
Proofd protocol version: 34
Client protocol version: 35
Remote protocol version: 34
Log level: 0
Session unique tag: lxbsq1409-1334421396-5704
Default data pool: root://alice-caf.cern.ch///pool/data/01/xrdnamespace
*** Master server 0 (parallel mode, 10 workers):
Master host name: lxbsq1409.cern.ch
Port number: 1093
User/Group: ganis/default
ROOT version|rev|tag: 5.33/03|current
Architecture-Compiler: linuxx8664gcc-gcc412
Protocol version: 34
Image name: lxbsq1409.cern.ch:/pool/PROOF-AAF/proof//proofbox/ganis
Working directory: /pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334421396-5704/master-0-lxbsq1409-1334421396-5704
Config directory:
Config file:
Log level: 0
Number of workers: 10
Number of active workers: 10
Number of unique workers: 9
Number of inactive workers: 0
Number of bad workers: 0
Total MBs processed: 0.00
Total real time used (s): 0.010
Total CPU time used (s): 0.000
List of workers:
*** Worker 0.13 (valid)
Host name: lxbsq1240.cern.ch
Port number: 1093
Worker session tag:
ROOT version|rev|tag: 5.33/03|r43580|current
Architecture-Compiler: linuxx8664gcc-gcc412
User/Group: ganis/default
Proofd protocol version: 34
Image name: lxbsq1240.cern.ch:/pool/PROOF-AAF/proof//proofbox/ganis
Working directory: /pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334421396-5704/worker-0.13-lxbsq1240-1334421397-10317
Performance index: 100
MBs processed: 0.00
MBs sent: 0.00
MBs received: 0.00
Real time used (s): 0.001
CPU time used (s): 0.000
*** Worker 0.23 (valid)
Host name: lxbsq1419.cern.ch
Port number: 1093
Worker session tag:
ROOT version|rev|tag: 5.33/03|r43580|current
Architecture-Compiler: linuxx8664gcc-gcc412
User/Group: ganis/default
Proofd protocol version: 34
Image name: lxbsq1419.cern.ch:/pool/PROOF-AAF/proof//proofbox/ganis
Working directory: /pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334421396-5704/worker-0.23-lxbsq1419-1334421397-13511
Performance index: 100
MBs processed: 0.00
MBs sent: 0.00
MBs received: 0.00
Real time used (s): 0.001
CPU time used (s): 0.000
*** Worker 0.33 (valid)
Host name: lxfssi3307.cern.ch
Port number: 1093
Worker session tag:
ROOT version|rev|tag: 5.33/03|r43580|current
Architecture-Compiler: linuxx8664gcc-gcc412
User/Group: ganis/default
Proofd protocol version: 34
Image name: lxfssi3307.cern.ch:/pool/PROOF-AAF/proof//proofbox/ganis
Working directory: /pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334421396-5704/worker-0.33-lxfssi3307-1334421397-17702
Performance index: 100
MBs processed: 0.00
MBs sent: 0.00
MBs received: 0.00
Real time used (s): 0.001
CPU time used (s): 0.000
*** Worker 0.43 (valid)
Host name: lxfssl3310.cern.ch
Port number: 1093
Worker session tag:
ROOT version|rev|tag: 5.33/03|r43580|current
Architecture-Compiler: linuxx8664gcc-gcc412
User/Group: ganis/default
Proofd protocol version: 34
Image name: lxfssl3310.cern.ch:/pool/PROOF-AAF/proof//proofbox/ganis
Working directory: /pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334421396-5704/worker-0.43-lxfssl3310-1334421397-21869
Performance index: 100
MBs processed: 0.00
MBs sent: 0.00
MBs received: 0.00
Real time used (s): 0.001
CPU time used (s): 0.000
*** Worker 0.53 (valid)
Host name: lxfssl3402.cern.ch
Port number: 1093
Worker session tag:
ROOT version|rev|tag: 5.33/03|r43580|current
Architecture-Compiler: linuxx8664gcc-gcc412
User/Group: ganis/default
Proofd protocol version: 34
Image name: lxfssl3402.cern.ch:/pool/PROOF-AAF/proof//proofbox/ganis
Working directory: /pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334421396-5704/worker-0.53-lxfssl3402-1334421397-16133
Performance index: 100
MBs processed: 0.00
MBs sent: 0.00
MBs received: 0.00
Real time used (s): 0.001
CPU time used (s): 0.000
*** Worker 0.63 (valid)
Host name: lxbsq1226.cern.ch
Port number: 1093
Worker session tag:
ROOT version|rev|tag: 5.33/03|r43580|current
Architecture-Compiler: linuxx8664gcc-gcc412
User/Group: ganis/default
Proofd protocol version: 34
Image name: lxbsq1226.cern.ch:/pool/PROOF-AAF/proof//proofbox/ganis
Working directory: /pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334421396-5704/worker-0.63-lxbsq1226-1334421397-13757
Performance index: 100
MBs processed: 0.00
MBs sent: 0.00
MBs received: 0.00
Real time used (s): 0.001
CPU time used (s): 0.000
*** Worker 0.73 (valid)
Host name: lxbsq1412.cern.ch
Port number: 1093
Worker session tag:
ROOT version|rev|tag: 5.33/03|r43580|current
Architecture-Compiler: linuxx8664gcc-gcc412
User/Group: ganis/default
Proofd protocol version: 34
Image name: lxbsq1412.cern.ch:/pool/PROOF-AAF/proof//proofbox/ganis
Working directory: /pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334421396-5704/worker-0.73-lxbsq1412-1334421397-18986
Performance index: 100
MBs processed: 0.00
MBs sent: 0.00
MBs received: 0.00
Real time used (s): 0.001
CPU time used (s): 0.000
*** Worker 0.100 (valid)
Host name: lxfssl3310.cern.ch
Port number: 1093
Worker session tag:
ROOT version|rev|tag: 5.33/03|r43580|current
Architecture-Compiler: linuxx8664gcc-gcc412
User/Group: ganis/default
Proofd protocol version: 34
Image name: lxfssl3310.cern.ch:/pool/PROOF-AAF/proof//proofbox/ganis
Working directory: /pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334421396-5704/worker-0.100-lxfssl3310-1334421396-21858
Performance index: 100
MBs processed: 0.00
MBs sent: 0.00
MBs received: 0.00
Real time used (s): 0.001
CPU time used (s): 0.000
*** Worker 0.109 (valid)
Host name: lxfssl3401.cern.ch
Port number: 1093
Worker session tag:
ROOT version|rev|tag: 5.33/03|r43580|current
Architecture-Compiler: linuxx8664gcc-gcc412
User/Group: ganis/default
Proofd protocol version: 34
Image name: lxfssl3401.cern.ch:/pool/PROOF-AAF/proof//proofbox/ganis
Working directory: /pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334421396-5704/worker-0.109-lxfssl3401-1334421396-21599
Performance index: 100
MBs processed: 0.00
MBs sent: 0.00
MBs received: 0.00
Real time used (s): 0.001
CPU time used (s): 0.000
*** Worker 0.112 (valid)
Host name: lxfssl3404.cern.ch
Port number: 1093
Worker session tag:
ROOT version|rev|tag: 5.33/03|r43580|current
Architecture-Compiler: linuxx8664gcc-gcc412
User/Group: ganis/default
Proofd protocol version: 34
Image name: lxfssl3404.cern.ch:/pool/PROOF-AAF/proof//proofbox/ganis
Working directory: /pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334421396-5704/worker-0.112-lxfssl3404-1334421396-24623
Performance index: 100
MBs processed: 0.00
MBs sent: 0.00
MBs received: 0.00
Real time used (s): 0.001
CPU time used (s): 0.000
root [9]
You may notice that each worker gets assigned a unique ordinal number in the form '0.n' .
Print() allows to find information about the ROOT versions, platforms, location of sandboxes, global statistics about CPU time and I/O used by the session actors, etc. etc. The group shown is the PROOF gorup, which defaults to 'default' if not explicitly set by the cluster administrator.
This allows to execute ROOT commands on workers or on the master. A ROOT command is anything that can be executed on the ROOT prompt, for example 'gSystem->Getenv("ROOTSYS")' or '.x mymacro.C'. Two questions may arise right away by looking at these two examples: 'how do we handle the character " delimiting strings?' and 'the mymacro.C is local, i.e. on the client machine: how does it get on the worker nodes?'. The answer to the first question is that the string delimiter " must be escaped using '\', so we will type something like this:
root [1] proof->Exec("gSystem->Getenv(\"ROOTSYS\")")
(const char* 0x14300a38)"/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a"
(const char* 0xeb61a38)"/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a"
(const char* 0x1c036a38)"/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a"
(const char* 0xc41aa38)"/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a"
(const char* 0x298ca38)"/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a"
(const char* 0xfc11a38)"/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a"
(const char* 0xfb37a38)"/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a"
(const char* 0x6e44a38)"/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a"
(const char* 0xda15a38)"/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a"
(const char* 0x76ba38)"/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a"
(Int_t)0
In the second case, PROOF detects CINT commands requiring a file (actually only '.L', '.x' and '.X') and make sure that an updated version of the file exists on the nodes (this means that PROOF checks if the file exists already on the nodes and compare the md5 checksums, so the file is uploaded only if needed). For example, with the following unnamed macro (which we call getROOTSYS.C):
{
Printf(" ROOTSYS: %s", gSystem->Getenv("ROOTSYS"));
}
we get a result similar to the above:
root [2] proof->Exec(".x getROOTSYS.C")
ROOTSYS: /pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a
ROOTSYS: /pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a
ROOTSYS: /pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a
ROOTSYS: /pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a
ROOTSYS: /pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a
ROOTSYS: /pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a
ROOTSYS: /pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a
ROOTSYS: /pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a
ROOTSYS: /pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a
ROOTSYS: /pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a
(Int_t)0
The first time you run this you will notice some latency due to the distribution of the macro to the workers. If you do not change the macro, the second time it goes much faster.
By default the command is executed only on the workers, i.e. not on the master. To execute it on the master one can do the following trick:
root [3] proof->SetParallel(0)
PROOF set to sequential mode
(Int_t)0
root [4] proof->Exec("gSystem->Getenv(\"ROOTSYS\")")
(const char* 0x706fa38)"/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a"
(Int_t)0
root [5] proof->Exec(".x getROOTSYS.C")
ROOTSYS: /pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a
(Int_t)0
root [6] proof->SetParallel(99999)
PROOF set to parallel mode (10 workers)
(Int_t)10
The call SetParallel(0) make PROF think that it has no workers, i.e. that is in sequential mode; the command is therefore executed on the unique node available, the master. Do not forget to set it in parallel mode after this by calling SetParallel again with a very large number, e.g. SetParallel(99999).
We know that ROOT allows to escape to the underlying shell (which on PROOF master and workers is always Unix) with the sequence '.!'. This make Exec() a powerful way to get information about the machines on which the workers are started. For example, this is an alternative way to access ROOTSYS on the master:
root [7] proof->SetParallel(0)
PROOF set to sequential mode
(Int_t)0
root [8] proof->Exec(".!echo $ROOTSYS")
/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a
(Int_t)0
root [9] proof->SetParallel(99999)
PROOF set to parallel mode (10 workers)
(Int_t)10
The output of Exec is displayed on the screen. This may be OK in some cases. Sometimes, however, we may need to get the output in a more usable form, i.e. a string or a number. There is no generic way to do so, but with the help of output redirection we can get something locally from which we can extract the required information. The following example shows how to get the value of any environment variable as a TString on the client session, with the help of two macros, a steering macro getEnv.C:
#include "TMacro.h"
#include "TObjString.h"
#include "TProof.h"
#include "TString.h"
#include "TSystem.h"
TString getEnv(const char *env, const char *where= "0")
{
// Return TString with the value of the environment variable 'env' on node with ordinal number 'where'.
// The ordinal numbers are '0' for the master, '0.n' for the workers.
// If the specified ordinal numebr is not found the full output is displayed so that it can be rerun with
// the correct value.
// This will be our output string
TString sout;
// We need a valid PROOF session up and running
if (!gProof || (gProof && !gProof->IsValid())) {
Printf("A valid PROOF session must be up and running!");
return sout;
}
// Master only or workers?
Bool_t onmaster = (strcmp(where, "0") == 0) ? kTRUE : kFALSE;
// If on master only, set PROOF on parallel mode
if (onmaster) gProof->SetParallel(0);
// Temporary file for the output, making sure it does not exist already
TString ftmp = TString::Format("%s/.getEnv", gSystem->TempDirectory());
gSystem->Unlink(ftmp);
// Redirect the output
gSystem->RedirectOutput(ftmp, "w");
// Run the macro to extract the information
TString cmd = TString::Format(".x getEnvOrd.C(\"%s\")", env);
gProof->Exec(cmd);
// Restore the output
gSystem->RedirectOutput(0);
// If on master only, restore in parallel mode
if (onmaster) gProof->SetParallel(99999);
// Parse the output now using TMacro
TMacro mm;
mm.ReadFile(ftmp);
TString tag = TString::Format("o:%s ", where);
TObjString *os = mm.GetLineWith(tag);
if (os) {
TString info;
Ssiz_t from = 0;
os->GetString().Tokenize(info, from, " ");
os->GetString().Tokenize(info, from, " ");
os->GetString().Tokenize(sout, from, " ");
} else {
// Print the file
Printf("Ordinal number %s not found! This is what we have:", where);
mm.Print();
}
// Done
return sout;
}
and a macro to fill the output with the wanted information:
#include "TProofServ.h"
#include "TSystem.h"
void getEnvOrd(const char *env)
{
// Macro to be used with getEnv.C to format the output
if (gSystem && gProofServ) {
Printf("o:%s h:%s %s", gProofServ->GetOrdinal(), gSystem->HostName(), gSystem->Getenv(env));
} else if (gSystem) {
Printf("o: h:%s %s", gSystem->HostName(), gSystem->Getenv(env));
} else {
Printf("o: h:");
}
}
Note the use of TSystem::RedirectOutput and of TMacro in the steering macro. Note also that a valid PROOF session must be available before running the steering macro. The result would be something like this:
root [10] .L getEnv.C+
Info in <:aclic>: creating shared library /Users/ganis/local/root/opt/root/./getEnv_C.so
root [11] TString rs = getEnv("ROOTSYS", "0.13")
root [12] Printf("%s", rs.Data());
/pool/PROOF-AAF/alien_packages/VO_ALICE/ROOT/v5-33-02a/v5-33-02a