Have a Question?
ARC Storage Services
ARC Storage Services
1. Lustre – Performance and Scalability
Lustre is a high-performance, distributed file system designed for large-scale storage and data-intensive computing environments. It is particularly well-suited for computations that require rapid access to immense volumes of data. At its core, Lustre provides a parallel file system architecture, which means it can distribute data across multiple storage servers and disks, allowing for concurrent read and write operations. This parallelism ensures that Lustre can handle the massive I/O (input/output) demands of modern high-performance computing (HPC) clusters.
On ARC systems there are two Lustre storage mounts available:
- Scratch – Free storage mount for active data transfer
- Project – Paid storage mount for active data transfer with data backup
How to access Lustre
Enter the following command in your terminal after logging into the arcc2 cluster. Note that both scratch and project mounts are under the lustre directory.
$ cd /N/lustre/scratch/username
$ cd /N/lustre/project/proj-xxx
2. Bulk Storage:
Characterized by high capacity and cost-effectiveness, bulk storage serves as a repository for massive volumes of data that may not require rapid access but still demand secure and reliable preservation. This storage solution is tailored for storing large, less frequently accessed datasets and files, such as archives, backups, scientific simulations, and research data. Bulk storage ensures seamless data management, scalability, and data integrity, supporting the overall performance and productivity of the computational cluster.
3. Archival Storage:
This technology involves storing vast amounts of data on tape cartridges, which are designed for durability and data integrity over extended periods. Archival tape storage is used in HPC environments where data volumes are immense and need to be kept for years, even decades. Unlike other storage solutions, tape storage offers exceptional data longevity and energy efficiency, making it a cost-effective choice for storing data that is accessed infrequently but must be retained for compliance, historical analysis, or reference purposes.
Note: Scratch storage is limited to 20TB and is purged on a regular basis.