Have a Question?
Choosing a GPU
ARCC GPU partition guide
Which work sounds most like yours?
Choose the closest description. The guide covers general scientific computing, numerical precision, memory, multi-GPU scaling, legacy compatibility, and AI/LLM work. Every result is a starting recommendation—not a substitute for testing your own workload.
What stage best describes your AI or LLM work?
Not sure?
Begin with A100
It is the general-purpose starting point for most ARCC GPU work. Select a description to see when another partition is a better fit.
One more choice
Tell us where you are in the model lifecycle
Development, training, and inference stress GPUs differently. Choose the closest AI or LLM stage to continue.
Recommended starting point
Start with the A100 partition
A100 is the practical default for most GPU-enabled research on ARCC2.
- 10 nodes provide 20 GPUs, giving routine jobs more places to run.
- It is a strong general-purpose choice for scientific applications, data analysis, and established AI workflows.
- Starting here makes it easier to tell whether a newer or larger-memory GPU actually improves your workload.
Reconsider this choice when: Consider H100 when FP64 benchmarks, newer tensor hardware, or a tightly coupled multi-GPU job show a meaningful benefit. Consider RTX PRO 6000 when one GPU needs more memory than H100 provides or a validated research application specifically benefits from RTX visualization or rendering.
Slurm starting point
#SBATCH --partition=gpu-a100
#SBATCH --gres=gpu:1
Recommended starting point
Try the H100 partition
H100 is the strongest starting point for demanding FP64 science, modern compute-intensive AI, and jobs that scale across several GPUs on one node.
- One node provides 8 80 GB SXM GPUs for single-node multi-GPU work.
- FP64-intensive scientific computing and modern tensor-heavy training or inference are the most likely to benefit.
- The concentrated GPU node avoids spreading a tightly coupled job across multiple hosts.
Reconsider this choice when: Use A100 for routine jobs that do not benefit from H100-specific performance. Use RTX PRO 6000 when one GPU needs more memory than H100 provides or a validated research application specifically benefits from RTX visualization or rendering capabilities.
Slurm starting point
#SBATCH --partition=gpu-h100
#SBATCH --gres=gpu:1
Recommended starting point
Use the V100 partition for continuity
V100 is best treated as a compatibility and reproducibility resource rather than the default for new work.
- It provides continuity for older software stacks and established benchmark results.
- It can reduce avoidable migration work when a project was specifically validated on V100.
- New projects will usually have a better starting point on A100.
Reconsider this choice when: Choose A100 for new general-purpose GPU work unless software compatibility or reproducibility requirements specifically favor V100.
Slurm starting point
#SBATCH --partition=gpu-v100
#SBATCH --gres=gpu:1
Recommended starting point
Try the RTX PRO 6000 partition
RTX PRO 6000 is the best starting point when a workload requires its 96 GB of memory on one GPU or has a validated RTX-specific visualization or rendering requirement.
- Each GPU provides 96 GB, the largest per-GPU memory capacity in the current ARCC fleet.
- One node provides 4 GPUs for workloads that fit that single-node shape.
- Use it for research visualization or rendering only when the application has been validated to benefit from RTX-specific capabilities.
Reconsider this choice when: Use H100 for FP64-intensive science, modern tensor-heavy training, or jobs that need up to 8 tightly coupled GPUs on one node. Use A100 for routine scientific GPU work that fits its per-GPU memory.
Slurm starting point
#SBATCH --partition=gpu-rtx6000
#SBATCH --gres=gpu:1
Recommended comparison
Benchmark H100 and RTX PRO 6000
Inference is sensitive to model size, quantization, batch size, latency target, and framework support. These two partitions optimize different constraints.
NVIDIA H100
A strong candidate for high-throughput or low-latency inference, especially with optimized FP8-capable software. Benchmark against RTX PRO 6000 when 96 GB or FP4 support matters.
NVIDIA RTX PRO 6000 Blackwell Server Edition
A strong candidate for memory-heavy or quantized inference, particularly when 96 GB, FP8, or FP4 lets the model fit or serve more efficiently. Validate framework and kernel support.
Also establish a baseline: A100 remains a useful compatibility and queue-availability baseline. Compare tokens per second, time to first token, peak GPU memory, and total GPU-hours before standardizing.
Two Slurm tests
# H100 test
#SBATCH --partition=gpu-h100
#SBATCH --gres=gpu:1
# RTX PRO 6000 test
#SBATCH --partition=gpu-rtx6000
#SBATCH --gres=gpu:1
Why the recommendations differ
The table keeps the technical details available after the workload-first choice.
| Partition | GPU | Nodes | GPUs / node | Total GPUs | Memory / GPU | Type | Architecture | Numerical formats | FP64 role | Best starting point for |
|---|---|---|---|---|---|---|---|---|---|---|
gpu-a100 |
NVIDIA A100 | 10 | 2 | 20 | 40 GB | PCIe | Ampere | FP64, FP64 Tensor Core, FP32, TF32, BF16, FP16, INT8 | Strong: designed for FP64 scientific and engineering workloads | Most GPU-enabled scientific, data-analysis, and AI workloads |
gpu-h100 |
NVIDIA H100 | 1 | 8 | 8 | 80 GB | SXM | Hopper | FP64, FP64 Tensor Core, FP32, TF32, BF16, FP16, FP8, INT8 | Excellent: preferred for demanding modern FP64 workloads | Demanding FP64 science, large modern AI, and tightly coupled multi-GPU jobs |
gpu-v100 |
NVIDIA V100 | 2 | 2 | 4 | 32 GB | PCIe | Volta | FP64 (CUDA cores), FP32, FP16 Tensor Core | Capable but older: use primarily for validated legacy workflows | Reproducing or extending workflows already validated on V100 |
gpu-rtx6000 |
NVIDIA RTX PRO 6000 Blackwell Server Edition | 1 | 4 | 4 | 96 GB | PCIe | Blackwell | FP64 performance not published; FP32, TF32, BF16, FP16, FP8, FP4 | Not recommended: NVIDIA does not publish an FP64 performance specification for this GPU | Workloads requiring 96 GB on one GPU or validated RTX-specific research visualization |
How each GPU fits common research workflows
Open a GPU to compare scientific computing, numerical precision, and AI or language-model roles.
NVIDIA A100 Ampere · 40 GB
- Scientific and data-intensive computing
- The general starting point for GPU-accelerated simulation, molecular modeling, image reconstruction, data analysis, and established scientific applications that fit within 40 GB per GPU.
- Numerical precision
- A strong HPC option when FP64 matters, while TF32, BF16, FP16, and INT8 support common AI workflows.
- AI/LLM development and prototyping
- The best general starting point for notebooks, code development, debugging, and models that fit within 40 GB.
- AI/LLM training and fine-tuning
- A solid choice for training and fine-tuning that fits within 40 GB per GPU and scales within two GPUs per node. Move to H100 for larger or more tightly coupled runs.
- AI/LLM inference
- A broad-compatibility baseline for routine inference. Benchmark H100 or RTX PRO 6000 when throughput, latency, quantization, or model capacity drives the decision.
NVIDIA H100 Hopper · 80 GB
- Scientific and data-intensive computing
- A strong candidate for FP64-intensive simulation, solvers, and scientific codes that benchmark substantially faster than A100, particularly when the workload can use several tightly coupled GPUs on one node.
- Numerical precision
- Combines strong FP64 scientific computing with TF32, BF16, FP16, FP8, and INT8 acceleration for modern AI.
- AI/LLM development and prototyping
- Use when development itself needs 80 GB or Hopper-specific features. For routine coding and debugging, A100 preserves the scarcer H100 node for workloads that benefit from it.
- AI/LLM training and fine-tuning
- The preferred starting point for large modern training and fine-tuning, particularly when a job benefits from FP8 or several tightly coupled GPUs on one node.
- AI/LLM inference
- A strong candidate for high-throughput or low-latency inference, especially with optimized FP8-capable software. Benchmark against RTX PRO 6000 when 96 GB or FP4 support matters.
NVIDIA V100 Volta · 32 GB
- Scientific and data-intensive computing
- Use for scientific applications, numerical results, or software environments already validated on V100. Begin new general-purpose projects on A100 unless reproducibility requires Volta behavior.
- Numerical precision
- Provides FP64 and FP32 CUDA performance plus first-generation FP16 Tensor Cores, but lacks the newer TF32, BF16, FP8, and FP4 paths.
- AI/LLM development and prototyping
- Use only when reproducing or extending an environment already validated on V100. New LLM development should normally begin on A100.
- AI/LLM training and fine-tuning
- Appropriate for legacy training workflows whose software or results depend on V100 behavior; not the preferred platform for new large-model training.
- AI/LLM inference
- Useful for reproducing established inference results, but newer partitions provide more memory and newer low-precision capabilities.
NVIDIA RTX PRO 6000 Blackwell Server Edition Blackwell · 96 GB
- Scientific and data-intensive computing
- Use when a scientific dataset, reconstruction, mesh, volume, or other in-memory representation requires more than 80 GB on one GPU. It is not the default for ordinary GPU computing or FP64-heavy simulation.
- Numerical precision
- Optimized for FP32 and low-precision AI and visual computing. Use A100 or H100 when sustained FP64 performance is central to the workload.
- AI/LLM development and prototyping
- A strong choice when a development model or dataset needs the 96 GB available on one GPU, or when testing Blackwell-specific low-precision paths.
- AI/LLM training and fine-tuning
- Useful for memory-heavy single-GPU or four-GPU fine-tuning when the software supports Blackwell well. Prefer H100 for tightly coupled training that benefits from its SXM topology.
- AI/LLM inference
- A strong candidate for memory-heavy or quantized inference, particularly when 96 GB, FP8, or FP4 lets the model fit or serve more efficiently. Validate framework and kernel support.
Understanding numerical precision
These labels describe numeric formats—not whether a GPU is a “32-bit” or “64-bit” computer.
- FP64
- Double precision for numerically sensitive simulation, solvers, computational science, and engineering. It is usually unnecessary for LLMs.
- FP32
- Single precision used as a compatibility baseline, for debugging, and by scientific applications that do not require FP64.
- TF32
- A tensor-oriented format that accelerates many FP32 deep-learning operations while retaining FP32 numeric range.
- BF16/FP16
- Common mixed-precision formats for model development, training, and fine-tuning because they reduce memory use and accelerate tensor operations.
- FP8
- A newer low-precision format for optimized training and inference. Benefits depend on framework, kernels, and model validation.
- FP4/INT4
- Very low precision used primarily for quantized inference. It can improve fit and throughput, but quality and software support must be tested.