Storage Questions
Storage Questions
1 Who can store data?
Only RC staff and RC users can store data on RC systems, whether those systems are meant for computation, or are dedicated exclusively to data storage or archiving.
2 Who owns the data?
The data stored under a user’s account is owned by the user’s Principal Investigator. The user’s PI may in fact legitimately request RC to suspend an account at any time. The PI is also authorized to make decisions about the disposal of the data in accounts under his/her charge.
3 What type of data may be stored?
- Data that serves as input to executables that are run on RC systems
- Data that is output from executables that are run on RC systems
- Executable files that are run on RC systems
- Other directly relevant files: source code, object files, user libraries, include files, make files, etc
4 What type of files should be regularly deleted?
Users should regularly delete non-essential files in their directories. Examples include simulation log files that would be irrelevant to research needs once job results have been correctly generated, or core files produced by application crashes.
5 How much data may be stored?
For the Lustre file system, default per-user disk quotas are defined in the user guides, but these quotas apply to all usage under /ddn, and not just to /ddn/home (in other words, usage under /ddn/scratch will count towards a user’s quota). If the need for a larger user limit can be demonstrated, RC will consider extensions to the default quotas on a case by case basis.
RC staff may employ alternative or complementary methods (to the traditional lustre quotas) to detect, warn about, and ultimately cap disk usage to enforce theses limits.
6 How long will data be retained on the system?
All disk-resident user data on all compute servers will be deleted six months after account de-activation unless a user has made prior arrangements with RC staff. An account is de-activated permanently if it is not renewed annually within 6 months of expiry. Within this 6 month period, if an account is not renewed, the account-holder’s PI can still direct RC to:
- delete the archived data
- copy the data to an external, non-RC storage device
- move the data to another user’s HPC account (without violating disk quota limits)
- extend the tenure of the data, subject to storage availability
- any combination of the above
7 How is /ddn/scratch to be used?
The /ddn/scratch directory is made available to meet the current needs of active jobs. This space is not meant in any way to provide long-term storage. Users are expected to continually delete or move elsewhere files that are not being used by current jobs. RC staff reserves the right to delete user files from /ddn/scratch at any time after the corresponding jobs have completed. Jobs intending to employ this scratch space should follow RC guidelines to name the temporary directories created therein. These guidelines are available in the user guides.