You are here

Home » PROOF » PROOF Hands-on tutorial

The PROOF sandbox

Each user gets a working area on each node of the cluster. This area is called sandbox. The default location on a local machine is $HOME/.proof (on PROOF-Lite $HOME/.proof/path-from-where-we-started). However, on eal clusters the administrator decides where the user sandboxes will be created. You can find out the location of the sandbox with Print().

  1. The sandbox
  2. The working directory

1. The sandbox

The sandbox has several sub-directories:

  • cache
    Area where tarballs, code and binaries are cached for optimized uploads and operation 
  • packages
    Area where packages are actually build and installed
  • session-sessionUniqueID
    Working area for session sessionUniqueID
  • data
    Area to store files created by users during processing. By default is created under the sandbox, but can be redirected by the administrator to some other device.
  • queries (master only)
    Area where the processing results are stored 
  • datasets (master only)
    Area where meta-information about the datasets is stored. By default is created under the sandbox, but can be redirected by the administrator to some other device.

2. The working directory

Each PROOF session has, under the sandbox, a unique working directory identified by the unique ID. The session unique ID is in the form

          master_hostname-master_creation_time-master_process_ID

where

  • master_hostname
    The host name of the master node, e.g. lxbsql1409
  • master_creation_time
    The creation time of the master in seconds since The Epoch, i.e. 1/1/1970
  • master_process_ID
    Unix process ID of the master process 

The sessionUniqueID is created on the master and communicated to the workers where is used to create the working area. Since more workers can be started on the same worker node, each worker gets another ID identifying it in unique way. This ID has a similar form as the session ID:

          worker-ordinal-worker_hostname-workers_creation_time-worker_process_ID

Here ordinal is the ordinal number of the worker, the remaining components have the same meaning as above.

We can have a look with Exec() at a real sandbox. First on the master: 

root [1] proof->SetParallel(0)
PROOF set to sequential mode
(Int_t)0
root [2] proof->Exec(".!pwd")
/pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334473238-22773/master-0-lxbsq1409-1334473238-22773
(Int_t)0
In this case the administrator has chosen to set the user sandboxes under '/pool/PROOF-AAF/proof/proofbox'. The user sandbox is obtained by the sanbox path by adding the user name: '/pool/PROOF-AAF/proof/proofbox/ganis'. The content of the user sandbox is: 
root [3] proof->Exec(".!ls -lt /pool/PROOF-AAF/proof/proofbox/ganis")
total 160
drwxr-xr-x 3 ganis z2 4096 Apr 15 11:51 session-lxbsq1409-1334483468-8890
lrwxrwxrwx 1 ganis z2   71 Apr 15 11:51 last-master-session -> /pool/PROOF-AAF/proof//proofbox/ganis/session-lxbsq1409-1334483468-8890
drwxr-xr-x 3 ganis z2 4096 Apr 15 11:51 queries
drwxr-xr-x 2 ganis z2 4096 Apr 15 11:50 session-lxbsq1409-1334473238-22773
drwxr-xr-x 3 ganis z2 4096 Apr 15 09:01 session-lxbsq1409-1334469543-9357
drwxr-xr-x 2 ganis z2 4096 Apr 15 08:56 cache
drwxr-xr-x 2 ganis z2 4096 Apr 15 07:59 session-lxbsq1409-1334421396-5704
drwxr-xr-x 2 ganis z2 4096 Apr 14 18:37 session-lxbsq1409-1334421276-4923
drwxr-xr-x 2 ganis z2 4096 Apr 13 14:05 packages
drwxr-xr-x 2 ganis z2 4096 Apr  9 16:13 session-lxbsq1409-1333980696-18601
We see that there is a symlink 'last-master-session' pointing to the latest created session directory. We also notice that the 'data' and 'datasets' directories are created elsewhere. We can examine the content of the cache: 
root [4] proof->Exec(".!ls -lt /pool/PROOF-AAF/proof/proofbox/ganis/cache")
total 16
-rw-r--r-- 1 ganis z2 378 Apr 15 09:05 getEnvOrd.C
-rw-r--r-- 1 ganis z2  59 Apr 15 08:05 getROOTSYS.C
(Int_t)0
which contains the recently run macros, and of the packages directory: 
root [5] proof->Exec(".!ls -lt /pool/PROOF-AAF/proof/proofbox/ganis/packages")
total 0
(Int_t)0
which is currently empty. Now we look at what is inside the session directory: 
root [6] proof->Exec(".!ls -lt /pool/PROOF-AAF/proof/proofbox/ganis/last-master-session/")
total 40
-rw-r--r-- 1 ganis z2 3085 Apr 15 11:56 master-0-lxbsq1409-1334483468-8890.log
drwxr-xr-x 2 ganis z2 4096 Apr 15 11:51 master-0-lxbsq1409-1334483468-8890
-rw-r--r-- 1 ganis z2 1145 Apr 15 11:51 master-0-lxbsq1409-1334483468-8890.env
-rw-r--r-- 1 ganis z2 1932 Apr 15 11:51 master-0-lxbsq1409-1334483468-8890.rootrc
lrwxrwxrwx 1 ganis z2  113 Apr 15 11:51 session.rootrc -> /pool/PROOF-AAF/proof//proofbox/ganis/session-lxbsq1409-1334483468-8890/master-0-lxbsq1409-1334483468-8890.rootrc
(Int_t)0
there are three files and one directory, all named in the form 'master-0-sessionUniqueID' and the relevant extension (except for the directory). The directory is the real working place of the session. The extensions explain the meaning of the files: the session log file (.log), the session specific environment settings (.env) and the session specific ROOT-rc directives (.rootrc) (there is a symlink to the ROOT-rc file for internal convenience.

We now look at a worker sandbox. We activate only one worker for browsing convenience: 

root [7] proof->SetParallel(1)
PROOF set to parallel mode (1 worker)
(Int_t)1
root [8] proof->Exec(".!pwd")
/pool/PROOF-AAF/proof/proofbox/ganis/session-lxbsq1409-1334483468-8890/worker-0.13-lxbsq1240-1334483470-340
(Int_t)0
We notice that the sandbox path on the worker machine is the same (typically clusters have the same configuration patterns) and we see the session directory is named after the sessionUniqueID that we found already on the master. If we look at the content of the sandbox: 
root [12] proof->Exec(".!ls -lt /pool/PROOF-AAF/proof/proofbox/ganis")
total 152
drwxr-xr-x 3 ganis z2 4096 Apr 15 11:51 session-lxbsq1409-1334483468-8890
lrwxrwxrwx 1 ganis z2   71 Apr 15 11:51 last-worker-session -> /pool/PROOF-AAF/proof//proofbox/ganis/session-lxbsq1409-1334483468-8890
drwxr-xr-x 2 ganis z2 4096 Apr 15 11:50 session-lxbsq1409-1334473238-22773
drwxr-xr-x 2 ganis z2 4096 Apr 15 08:59 cache
drwxr-xr-x 3 ganis z2 4096 Apr 15 07:59 session-lxbsq1409-1334469543-9357
drwxr-xr-x 2 ganis z2 4096 Apr 15 07:58 session-lxbsq1409-1334421396-5704
drwxr-xr-x 2 ganis z2 4096 Apr 14 18:36 session-lxbsq1409-1334421276-4923
drwxr-xr-x 2 ganis z2 4096 Apr 13 22:35 packages
drwxr-xr-x 2 ganis z2 4096 Apr  9 16:11 session-lxbsq1409-1333980696-18601
we notice that the symlink is called now 'last-worker-session' and that there is no 'queries' directory in the sandbox. The session directory 
root [13] proof->Exec(".!ls -lt /pool/PROOF-AAF/proof/proofbox/ganis/last-worker-session/")
total 40
-rw-r--r-- 1 ganis z2 1953 Apr 15 12:09 worker-0.13-lxbsq1240-1334483470-340.log
drwxr-xr-x 2 ganis z2 4096 Apr 15 11:51 worker-0.13-lxbsq1240-1334483470-340
lrwxrwxrwx 1 ganis z2  115 Apr 15 11:51 session.rootrc -> /pool/PROOF-AAF/proof//proofbox/ganis/session-lxbsq1409-1334483468-8890/worker-0.13-lxbsq1240-1334483470-340.rootrc
-rw-r--r-- 1 ganis z2 1149 Apr 15 11:51 worker-0.13-lxbsq1240-1334483470-340.env
-rw-r--r-- 1 ganis z2 1771 Apr 15 11:51 worker-0.13-lxbsq1240-1334483470-340.rootrc
(Int_t)0
also in this case three files and one directory, which are named using the worker unique ID, as mentioned above.

Examining the sandbox has also allowed to illustrate additional usage of Exec.