Loading [MathJax]/extensions/tex2jax.js
ROOT  6.06/09
Reference Guide
All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Modules Pages
Setup a static PROOF cluster with PROOF on Demand

Introduction

Using PROOF on Demand is our current recommended way of running a PROOF cluster. The usage of PoD is in particular helpful for the following reasons:

This guide particularly refers to the setup of a static PROOF cluster running on physical hosts: the recommended setup is in practice the same as the ready-to-go Virtual Analysis Facility. If you want to use PROOF on the clouds there is no configuration to go through.

Setup a resource management system

Although PROOF on Demand can run on a cluster of nodes without using a resource management system (using pod-ssh), it is recommended to setup a dedicated one to benefit from the scheduling in a multiuser environment, or a dedicated queue on an existing one.

As there's a variety of resource management systems, this guide does not cover their setup. The RMS preconfigured for the Virtual Analysis Facility is HTCondor, which we recommend primarily because it has dynamic addition of workers built in.

Configuration steps for all nodes

Setup CernVM-FS

CernVM-FS should be installed on all machines as the preferred method for software distribution.

Configuration instructions for the latest CernVM-FS can be found here.

A brief step-by-step procedure to install CernVM-FS is hereby described.

You might need special configurations for some custom software repositories! Special cases are not covered in this guide.

Firewall configuration

PROOF on Demand is very flexible in handling various cases of network topologies. The best solution would be to allow all TCP communications between the cluster machines.

No other incoming communication is required from the outside.

Configuration steps for the head node only

Setup HTTPS+SSH (sshcertauth) authentication

Latest recommended sshcertauth version is 0.8.5.

Download and read the instructions.

If you want your users to connect to the PROOF cluster using their Grid user certificate and private key you might be interested in installing sshcertauth. Please refer to the installation guide for further information.

PROOF on Demand

Latest recommended PROOF on Demand version is 3.12.

On CernVM-FS: /cvmfs/sft.cern.ch/lcg/external/PoD/3.12

Source code: PoD download page and Installation instructions

PROOF on Demand is required on the head node and on the user's client.

In case your experiment provides a version of PoD on CernVM-FS you can use that one. Experiment-independent versions are available from the PH-SFT cvmfs repository.

Only if you have specific reasons while you want to use a customly built PoD version, download the source code and compile it using the installation instructions.

Please note that CMake and Boost are required to build PoD.

There is no need to "configure" PoD for your specific cluster: it is just enough to install it on your head node.

PoD does not have any system-wide persistent daemon running or any system-wide configuration to be performed. Also, no part of PoD will be ever run as root.

Do not worry about environment or software configuration at this time: there is no system configuration for that. All the environment for your software dependencies will be set via proper scripts from the PoD client.

PoD client configuration and running is properly covered in the appropriate manual page.

Firewall configuration

The head node only requires TCP ports 22 (SSH) and 443 (HTTPS) to accept connections from the outside. Users will get an authentication "token" from port 443 and all PROOF traffic will be automatically tunneled in a SSH connection on port 22 by PoD.

In case you are not using the HTTPS+SSH token+authentication method, access to the sole port 22 is all you need.