Difference between revisions of "SLab:File System Layout"
Line 6: | Line 6: | ||
The volume mounted at this location is RO-replicated. The main purpose of this root data.schuster_lab volume is to contain mounpoints to other volumes, as well as the symlink farm under ''sequencing/'' | The volume mounted at this location is RO-replicated. The main purpose of this root data.schuster_lab volume is to contain mounpoints to other volumes, as well as the symlink farm under ''sequencing/'' | ||
− | = | + | == scratch/ == |
+ | Individual global scratch volumes. On request only (admin-at-bx.psu.edu). | ||
+ | |||
+ | == sequencing/ == | ||
* /afs/bx.psu.edu/depot/data/schuster_lab/sequencing | * /afs/bx.psu.edu/depot/data/schuster_lab/sequencing | ||
Line 15: | Line 18: | ||
SSDFS is not currently capable of delegating commands to normal users, and requires some finesse when executing certain commands due to NFS permissions. Email admin-at-bx.psu.edu if something under sequencing/ doesn't look right. | SSDFS is not currently capable of delegating commands to normal users, and requires some finesse when executing certain commands due to NFS permissions. Email admin-at-bx.psu.edu if something under sequencing/ doesn't look right. | ||
+ | |||
+ | In short, datasets are organized into "volumes" distributed between 2 servers and multiple NFS filesystems. 454 datasets are organized by one volume per month, and Illumina datasets are one volume per dataset. SSDFS fully abstracts the actual location of the volume (server/filesystem). If you need direct access to a volume, it will be under ''/afs/bx.psu.edu/depot/data/schuster_lab/.ssdfs/vol/by-name/<name>'' | ||
Layout: | Layout: | ||
* /afs/bx.psu.edu/depot/data/schuster_lab/sequencing/ | * /afs/bx.psu.edu/depot/data/schuster_lab/sequencing/ | ||
− | ** 454/ | + | ** 454/ (all 454 runs) |
*** $YEAR/ | *** $YEAR/ | ||
**** $YEAR/$YEAR-$MONTH/ | **** $YEAR/$YEAR-$MONTH/ | ||
*** incoming/ | *** incoming/ | ||
− | ** illumina/ | + | ** illumina/ (all illumina runs) |
*** $YEAR/ | *** $YEAR/ | ||
**** $YEAR/$RUN_NAME/ | **** $YEAR/$RUN_NAME/ | ||
Line 31: | Line 36: | ||
**** illumina-5 -> /nfs/s2.persephone.bx.psu.edu/md1k-5-data/illumina/ | **** illumina-5 -> /nfs/s2.persephone.bx.psu.edu/md1k-5-data/illumina/ | ||
**** illumina-6 -> /nfs/s2.persephone.bx.psu.edu/md1k-6-data/illumina/ | **** illumina-6 -> /nfs/s2.persephone.bx.psu.edu/md1k-6-data/illumina/ | ||
+ | ** scripts/ (Various shell scripts, qsub scripts, etc) | ||
+ | *** 454/ | ||
+ | ** support/ | ||
+ | *** 454/ | ||
+ | *** illumina/ | ||
− | == support == | + | == support/ == |
* /afs/bx.psu.edu/depot/data/schuster_lab/sequencing/support/ | * /afs/bx.psu.edu/depot/data/schuster_lab/sequencing/support/ | ||
− | |||
− | |||
The sequencing support directory contains manuals, software, and reference genomes used during sequencing and sequence processing. | The sequencing support directory contains manuals, software, and reference genomes used during sequencing and sequence processing. | ||
− | = | + | = projects/ = |
* /afs/bx.psu.edu/depot/data/schuster_lab/projects | * /afs/bx.psu.edu/depot/data/schuster_lab/projects | ||
The projects directory contains directories for project specific data including the Tasmanian Devil, KB1, and Woolly Mammoth projects. | The projects directory contains directories for project specific data including the Tasmanian Devil, KB1, and Woolly Mammoth projects. | ||
− | = | + | = users/ = |
* /afs/bx.psu.edu/depot/data/schuster_lab/users | * /afs/bx.psu.edu/depot/data/schuster_lab/users | ||
The users directory contains directories for people within the Schuster Lab. | The users directory contains directories for people within the Schuster Lab. |
Latest revision as of 16:47, 4 October 2010
Contents
Schuster Lab Root Directory
- /afs/bx.psu.edu/depot/data/schuster_lab
The Schuster Lab's data can be found in the /afs/bx.psu.edu/depot/data/schuster_lab directory.
The volume mounted at this location is RO-replicated. The main purpose of this root data.schuster_lab volume is to contain mounpoints to other volumes, as well as the symlink farm under sequencing/
scratch/
Individual global scratch volumes. On request only (admin-at-bx.psu.edu).
sequencing/
- /afs/bx.psu.edu/depot/data/schuster_lab/sequencing
Access to all of the Schuster Lab's sequencing data is through the sequencing directory. This directory contains archived run directories as well as directories for runs that are currently in progress.
Most of the top-level run directories under sequencing/ are symlinks managed by the new symlink management system: http://github.com/phalenor/ssdfs
SSDFS is not currently capable of delegating commands to normal users, and requires some finesse when executing certain commands due to NFS permissions. Email admin-at-bx.psu.edu if something under sequencing/ doesn't look right.
In short, datasets are organized into "volumes" distributed between 2 servers and multiple NFS filesystems. 454 datasets are organized by one volume per month, and Illumina datasets are one volume per dataset. SSDFS fully abstracts the actual location of the volume (server/filesystem). If you need direct access to a volume, it will be under /afs/bx.psu.edu/depot/data/schuster_lab/.ssdfs/vol/by-name/<name>
Layout:
- /afs/bx.psu.edu/depot/data/schuster_lab/sequencing/
- 454/ (all 454 runs)
- $YEAR/
- $YEAR/$YEAR-$MONTH/
- incoming/
- $YEAR/
- illumina/ (all illumina runs)
- $YEAR/
- $YEAR/$RUN_NAME/
- Instruments/
- HWUSI-EAS610/
- incoming/
- illumina-4 -> /nfs/s2.persephone.bx.psu.edu/md1k-4-data/illumina/
- illumina-5 -> /nfs/s2.persephone.bx.psu.edu/md1k-5-data/illumina/
- illumina-6 -> /nfs/s2.persephone.bx.psu.edu/md1k-6-data/illumina/
- $YEAR/
- scripts/ (Various shell scripts, qsub scripts, etc)
- 454/
- support/
- 454/
- illumina/
- 454/ (all 454 runs)
support/
- /afs/bx.psu.edu/depot/data/schuster_lab/sequencing/support/
The sequencing support directory contains manuals, software, and reference genomes used during sequencing and sequence processing.
projects/
- /afs/bx.psu.edu/depot/data/schuster_lab/projects
The projects directory contains directories for project specific data including the Tasmanian Devil, KB1, and Woolly Mammoth projects.
users/
- /afs/bx.psu.edu/depot/data/schuster_lab/users
The users directory contains directories for people within the Schuster Lab.