Skip to end of metadata
Go to start of metadata

There are a number of storage options available for research data. If you have any questions about any of them, please contact us.


Overview


TypeAvailabilitySize LimitReplication1RetentionSnapshot2PerformanceCost
/n/scratch2Everyone10TBNo29 daysNoFastFree
/homeEveryone100GBYesIndefiniteYesModerateFree
Group Directories3GroupsVariesYesIndefiniteYesModerateFree5
Collaborations4On-Quad6VariesYesIndefiniteYesModerateFree
/tmpEveryoneSmall hard drive sizeNoNoneNoFastFree


  • 1Replication means that data are copied nightly to a separate location
  • 2Snapshot means that each directory contains an invisible, read-only .snapshot directory that has 60 days of weekly old versions of files, 14 days daily
  • 3Group directories are generally in /n/groups/n/data1, or /n/data2
  • 4Collaborations are generally accessible from desktops, and as /n/files from a limited set of cluster compute nodes.
  • 5There are no plans for chargebacks for group storage at this time.
  • 6Only on-Quad researchers can create collaborations, but they can invite off-Quad researchers to join collaborations (with read-only or read-write access)

A brief description of each filesystem is below. For more information, please refer to the following pages:

Filesystems

Filesystem Quotas

Details

/n/scratch2

This is a filesystem designed to handle large volumes of temporary files. If you are running large pipelines, it is recommended that you write intermediary files here. There are no backups made here, and files are automatically purged after 30 days of no access.

  • To access /n/scratch2 when running O2 jobs, please see our documentation about Filesystem Resources

/home

This is where your home directory is located. Every user on the Orchestra or O2 research cluster gets a /home directory like /home/ab987 that is limited to 100GB. This amount cannot be expanded. Your home directory is backed up inside .snapshot, an invisible directory located inside every subdirectory of your home directory, which has 14 days worth of daily snapshots, and 60 days of weekly snapshots. Every night, your /home directory is copied to an off-site location (but that copy is overwritten 24 hours later).

Group directories

Your lab might have space for you and your colleagues to share. Labs may install their own group-specific software here, as well as sharing data. Lab/group directories are generally in /n/groups/n/data1, or /n/data2. These are backed up and snapshotted in the same way as /home .

research.files

Collaborations created by HMS IT are on the research.files filesystem. This filesystem is mostly used for sharing data between labs, or for departmental shared space, and it can be mounted as a shared drive on Windows and Mac desktops. (Contact itservicedesk@hms.harvard.edu or an IT client service representative with questions about research.files.) When needed, this filesystem can be accessed as /n/files on a few "transfer compute nodes". (See File Transfer for details.)

/tmp

This is temporary local hard drive space on a single compute node. If you require the fastest I/O for your job, you can have a program write temporary intermediate files to the /tmp directory. There is not a lot of space, and you will be sharing it with anyone else that requires the use of /tmp on that node. Each node has a different /tmp, and if you need to fetch files from there, you will need to ssh directly to that compute node. Also, these files may be deleted any time after your job finishes.

Special Cases

We are evaluating other storage options that might be suited for your needs. Please contact us if you have questions.


Other Resources

For more information, please refer to the following pages:

Filesystems

Filesystem Quotas


  • No labels