SLab:Using Sun Grid Engine
Contents
Resources
- linne cluster
- persephone cluster
- both clusters
Submitting a Job
Creating a Job Script
A job script is a standard shell script that contains the commands needed to run your job on the cluster. Sun Grid Engine uses the two character combination #$ to specify arguments to qsub. Check the qsub documentation for a complete list of command line arguments.
#!/usr/bin/env bash # # Submit this job to the all.q queue #$ -q all.q # # Send email when job begins,ends,is aborted or reschuled,or is suspended #$ -m beas # # Email address to send email to #$ -M rico@bx.psu.edu # # Shell to use to run job #$ -S /bin/bash # # Send job standart output to ~/sge-out #$ -o ~/sge-out # # Send job standard error to ~/sge-out #$ -e ~/sge-out SEQ_DIR=/afs/bx.psu.edu/depot/data/schuster_lab/sequencing RUN=100201_HWUSI-EAS610_0006 RUN_DIR=$SEQ_DIR/archive/illumina/by_date/2010/2010_02_01/$RUN cd $RUN_DIR/Data/Intensities/BaseCalls/GERALD_03-02-2010_rico wc -l s_6_sequence.txt
Adding write permissions (only for persephone cluster)
When using the persephone cluster, you need to modify the permissions of any AFS directory that is receiving output.
Create the ~/sge-out direcory.
% mkdir ~/sge-out
Allow svc/sge/persephone to access your home directory.
% fs setacl -dir ~ -acl svc/sge/persephone l
Allow svc/sge/persephone to read and write to the ~/sge-out directory.
% fs setacl -dir ~/sge-out -acl svc/sge/persephone rliw
Submitting the Job Script to SGE
You can submit jobs to Sun Grid Engine using the qsub command.
% qsub job.sh
Your job 2103 ("job.sh") has been submitted.
Checking Job Status
You can check the status of jobs using the qstat command.
The following command shows the current status of all queues and jobs.
% qstat -f -u '*'
The following command and output demonstrates checking the all.q queue on the persephone cluster.
% qstat -f -q all.q -u '*' queuename qtype resv/used/tot. load_avg arch states --------------------------------------------------------------------------------- all.q@c1.persephone.bx.psu.edu BIP 0/0/8 0.00 lx24-amd64 --------------------------------------------------------------------------------- all.q@c2.persephone.bx.psu.edu BIP 0/0/8 0.00 lx24-amd64 --------------------------------------------------------------------------------- all.q@c3.persephone.bx.psu.edu BIP 0/1/8 7.77 lx24-amd64 3646 0.55500 rico_metho suw17 r 03/23/2010 13:59:04 1 --------------------------------------------------------------------------------- all.q@c4.persephone.bx.psu.edu BIP 0/4/8 4.01 lx24-amd64 3773 0.55500 baseCallin rico r 03/23/2010 15:06:19 1 3774 0.55500 baseCallin rico r 03/23/2010 15:06:19 1 3779 0.55500 baseCallin rico r 03/23/2010 15:06:19 1 3780 0.55500 baseCallin rico r 03/23/2010 15:06:19 1 --------------------------------------------------------------------------------- all.q@c5.persephone.bx.psu.edu BIP 0/0/8 0.00 lx24-amd64 --------------------------------------------------------------------------------- all.q@c6.persephone.bx.psu.edu BIP 0/0/8 0.00 lx24-amd64 --------------------------------------------------------------------------------- all.q@c7.persephone.bx.psu.edu BIP 0/0/8 0.00 lx24-amd64 s ############################################################################ - PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING JOBS ############################################################################ 3781 0.00000 baseCallin rico hqw 03/23/2010 14:49:47 1
Deleting a Job
Jobs can be deleted using the qdel command.
The following command and output shows deleting the job with job_id 3781.
% qdel 3781 rico has deleted job 3781
The following command and output demonstrates deleting all jobs submitted by user rico.
% qdel -u rico rico has deleted job 3773 rico has deleted job 3774 rico has deleted job 3779 rico has deleted job 3780
