Resources > CGT Visualization Cluster
General
CGT's Vizualization Cluster consists of 9 nodes. Jobs are managed by OpenPBS
Torque and should be submitted through gatekeeper running on viz-login.isi.edu.
The nodes have the following characteristics:
- Debian Sarge
- 2.6 kernel
- Dual Intel Xeon 2.4GHz CPUs, HyperThreading enabled
- 2.0 GB RAM
- Gigabit network connection
- 350 GBs of /scratch space
Monitoring
Ganglia metrics (such as CPU, memory and network utilization) are available
at http://viz-login.isi.edu/ganglia/
Accessing the Cluster
The visualization cluster does not share the general ISI user account space,
but only has local accounts. Email cgt-support@isi.edu to get an account set
up.
The head node, viz-login.isi.edu, is accessible through ssh, but only with
ssh public key authentication.
File Systems
Your home directory is shared over the whole cluster, which you can use for
your jobs. Each node also has a /scratch and a /scratch-longer-term directory.
The difference is that the former gets cleaned when we reinstall the nodes,
while the latter is preserved if possible. They are roughly 175 GB each.
Note that we do not make backups of any of the filesystems, not even your home directory!
Running Jobs
Use viz-login.isi.edu/jobmanager-pbs as a resource string to submit
jobs to the cluster.
Condor-G example job:
# always use the globus universe with condor-g
universe = globus
# use the full path to the executable
executable = /nfs/home/rynge/jobs/myjob.sh
# do not transfer the executable - if this is true
# the executable will be transfered from the current
# directory
transfer_executable = false
# this specifies where the job should be submittied
# and what jobmanager to use
globusscheduler = viz-login.isi.edu/jobmanager-pbs
# additional globus rsl
globusrsl = (jobtype=multiple)(count=4)(queue=normal)
# specify the output files
output = job.$(cluster).out
error = job.$(cluster).err
log = job.$(cluster).log
# now queue it
queue
Using the X11 displays
You can start X11 programs directly from your jobs by setting the DISPLAY
environment variable to localhost:0.0 or localhost:0.1. This can be done either
with a wrapper script around the X11 program or in the RSL.
Example: RSL for showing xeyes in
the conference room:
'&(executable=/usr/bin/X11/xeyes)(jobtype=multiple)(count=3)
(nodes=3:crwall)(environment=(DISPLAY localhost:0.0)(maxWallTime=3)'
Displays available by setting DISPLAY:
| Node |
localhost:0.0 |
localhost:0.1 |
| viz-1 |
IBM T221 Pane 1 |
Left Conference Room Projector |
| viz-2 |
IBM T221 Pane 2 |
Center Conference Room Projector |
| viz-3 |
IBM T221 Pane 3 |
Right Conference Room Projector |
| viz-4 |
IBM T221 Pane 4 |
|
The X11 displays can be restarted by running /nfs/software/helper/bin/restart-all-displays
Example PBS Job
The following is an example PBS job which shows 'xeyes' on each sceeen
in the conference room. There are two files, job and
start-my-x. Run the job by running qsub job
#!/bin/bash
#
#PBS -l nodes=3:crwall
#
#PBS -q normal
#
#PBS -r n
#
#PBS -l walltime=0:05:00
#
#PBS -o stdout.txt
#
#PBS -e stderr.ext
/nfs/software/torque/default/bin/run-on-assigned-nodes \
/nfs/home/rynge/demo/cr-3-screen-test/start-my-x
#!/bin/sh
export PATH=/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games
export DISPLAY=localhost:0.1
xeyes &
sleep 30
killall xeyes
killall -9 xeyes
exit 0
Available software
Software common to the whole cluster is installed in /nfs/software .
By sourcing the setup.{sh|csh} files, you will get the right environment
for that piece of software. There are also 'default' symlinks to the most current
version.
For example, to use Globus with a bourne shell, run:
. /nfs/software/globus/default/setup.sh
or, for csh:
source /nfs/software/globus/default/setup.csh
|
