# A dedicated GPU for image and video generation

> Choosing a GPU for image and video generation: approximate VRAM for Stable Diffusion, Flux and video models, on dedicated GPUs. Compare the cards.

Source: https://overgpu.com/solutions/generative-media  
Updated: 2026-10-23  
Publisher: OverGPU (https://overgpu.com/)

## Key points

- **No queue behind other tenants:** The GPU is assigned to you only, all month, so the only queue is your own.
- **VRAM for video:** 96 GB of GDDR7 on one RTX PRO 6000 for video models, big batches and LoRA training.
- **Your own stack:** Root access to install ComfyUI, custom nodes, checkpoints and LoRAs, with Docker ready.
- **Room for outputs:** 20 TB of outbound traffic per month on single-GPU plans, with no egress fees.
- **Eight GPUs in one server:** The 8× RTX PRO 6000 server runs one pipeline per GPU for batch production.

## Recommended configurations

- [NVIDIA RTX PRO 6000 Blackwell](https://overgpu.com/gpus/nvidia-rtx-pro-6000): $1,390 per month — 96 GB: video models and big batches without offloading
- [NVIDIA GeForce RTX 5090](https://overgpu.com/gpus/nvidia-rtx-5090): $449 per month — 32 GB of fast GDDR7 for SDXL and FP8 Flux pipelines
- [NVIDIA L40S](https://overgpu.com/gpus/nvidia-l40s): $890 per month — 48 GB: Flux in BF16 with room for LoRAs and ControlNets

## Overview

The right GPU for image and video generation is the one that keeps your whole pipeline in VRAM: the diffusion model, its text encoders, the VAE, your LoRAs and ControlNets, plus the activations for your resolution and batch size. When everything stays resident, nothing is swapped out between steps. We rent whole NVIDIA GPUs by the month, from the 32 GB RTX 5090 to the 96 GB RTX PRO 6000, so your jobs never wait behind another tenant's.

## VRAM for Stable Diffusion, Flux and video models

The figures below are approximate and move with resolution, batch size, precision and the tool you use. Treat them as a starting point, not a promise.

| Pipeline | Approximate VRAM | Comfortable on |
| --- | --- | --- |
| SDXL at 1024 × 1024 | about 8–16 GB | RTX 5090, L40S, RTX PRO 6000 |
| Flux.1 [dev] with FP8 weights | about 20 GB | RTX 5090 and up |
| Flux.1 [dev] in BF16 | about 35–40 GB | L40S, RTX PRO 6000 |
| 13–14B video model in BF16 | about 28 GB of weights, plus activations that grow with resolution and frame count | RTX PRO 6000; L40S with FP8 weights or offloading |

The math behind them: SDXL has about 3.5 billion parameters across its UNet and text encoders, roughly 7 GB in FP16. Flux.1 [dev] is a 12-billion-parameter transformer, about 24 GB in BF16 on its own, and its T5-XXL text encoder adds roughly 9–10 GB more. Video models multiply activation memory by the number of frames, which is why 720p clips from 13–14B models often need more than 48 GB unless you quantize or offload. Training LoRAs adds optimizer state and activations on top of these inference figures.

## Choosing a GPU for image and video generation

- **RTX 5090, 32 GB of GDDR7.** A fast Blackwell card at $449 a month for SDXL, Flux in FP8, upscaling and smaller video models. It gets tight when several large models must stay loaded at once.
- **L40S, 48 GB of GDDR6.** Enough to keep Flux in BF16 resident with LoRAs and ControlNets, at $890 a month. It supports FP8 but not FP4.
- **RTX PRO 6000 Blackwell, 96 GB of GDDR7 with ECC.** The card for video models, large batches, LoRA training and workflows that chain several models, at $1,390 a month, with 1.6 TB/s of memory bandwidth per NVIDIA's specifications. More on the [RTX PRO 6000 page](https://overgpu.com/gpus/nvidia-rtx-pro-6000).
- **8× RTX PRO 6000 server.** 768 GB of total VRAM across eight GPUs for batch production, at $10,490 a month, with one pipeline per GPU.

System RAM matters too when you offload or cache models: 64 GB on the RTX 5090 plan, 128 GB on the L40S and 188 GB on the RTX PRO 6000. Deciding between the two Blackwell cards? Our [RTX 5090 vs RTX PRO 6000 comparison](https://overgpu.com/compare/rtx-5090-vs-rtx-pro-6000) puts them side by side. If you train diffusion or video models from scratch rather than run them, look at the H100, H200 and B200 instead.

## Throughput, batching and queues

On a dedicated GPU, your throughput depends on your workflow, not on other tenants' load. Three habits make the most of it:

- **Keep models resident.** ComfyUI keeps loaded models in memory between runs when VRAM allows; a workflow that swaps large models on every job spends its time moving weights. With 96 GB, an image model, a video model and an upscaler can stay loaded together.
- **Find your batch size.** Larger batches raise images per minute until the GPU saturates, then only add latency. Measure where that point is for your resolution and sampler.
- **Run one process per GPU.** ComfyUI drives one GPU per instance, so on a multi-GPU server, start one instance per GPU and spread jobs across them.

```
python main.py --cuda-device 0 --port 8188
python main.py --cuda-device 1 --port 8189
```

FP8 checkpoints roughly halve weight memory, usually with small quality differences, and the Blackwell cards add FP4. Compare a few seeds side by side before you switch production work.

## Training LoRAs on your own images

Training a LoRA adds adapter gradients, optimizer state and training activations to the inference footprint. As a rough guide, SDXL LoRA training fits in about 16–24 GB with gradient checkpointing. Flux LoRA training with a quantized base model fits in about 24 GB, while training against the full BF16 base is far more comfortable on 48 GB or more. Trainers such as kohya_ss sd-scripts, SimpleTuner and ai-toolkit run on Linux with the same CUDA stack as ComfyUI, so the same server can train a style or product LoRA and then serve it, without moving files between machines. On a dedicated GPU, a long training run never competes with another tenant's jobs, and your dataset, captions and checkpoints stay on local NVMe between runs: 1 TB on the RTX 5090 and L40S plans, 1.92 TB on the RTX PRO 6000.

## Setup, security and outputs

Our Ubuntu images include NVIDIA drivers, CUDA, Docker and the NVIDIA Container Toolkit, so you can run ComfyUI or diffusers in a container, following our [Docker guide](https://overgpu.com/docs/docker-containers), or in a Python environment. Do not expose the ComfyUI port to the internet; reach it through an SSH tunnel instead, as in our [SSH access guide](https://overgpu.com/docs/ssh-access):

```
ssh -L 8188:localhost:8188 -l your-user your-server-ip
```

Outputs count toward outbound traffic: the 20 TB a month included on single-GPU plans is about ten million 2 MB images or a million 20 MB clips, with no egress fees. Check each model's license before commercial use, since some open-weight image and video models restrict it, and keep your use within our [Acceptable Use Policy](https://overgpu.com/legal/acceptable-use).

A monthly GPU fits daily production, client work and generation backends that run every day. For a one-off weekend experiment, hourly rental elsewhere will cost less. To start, [deploy a server](https://overgpu.com/deploy): choose a configuration, create a free account or log in, and pay the first month from your prepaid balance, topped up in crypto. The server is delivered in under 10 minutes after payment, and you can describe your models, resolution and volume in the notes for our engineers.

## Frequently asked questions

### How much VRAM do I need for Flux?

Roughly 35–40 GB to keep Flux.1 [dev] and its text encoders resident in BF16, or about 20 GB with FP8 weights. A 48 GB L40S or a 96 GB RTX PRO 6000 runs BF16 comfortably, and a 32 GB RTX 5090 suits FP8.

### Is an RTX 5090 enough for Stable Diffusion?

Yes for SDXL and most image pipelines, which need roughly 8–16 GB at 1024 × 1024. Its 32 GB gets tight with large video models or with many models loaded at once.

### Can I run ComfyUI on your servers?

Yes. You get root access over SSH, install ComfyUI, custom nodes and models yourself, in Docker or a Python environment, and reach the interface through an SSH tunnel.

### Will my generations wait in a queue?

No. Each GPU is assigned to one customer only, with no slicing and no time-sharing, so the only queue is the one you run.

### Can I use the server for commercial client work?

Yes, within our Acceptable Use Policy. You are responsible for the licenses of the models and tools you run, and some open-weight models restrict commercial use.

## Related pages

- [Rent RTX PRO 6000 Blackwell servers from $1,390 per month](https://overgpu.com/gpus/nvidia-rtx-pro-6000): One 96 GB Blackwell GPU for $1,390/month (about $1.90 per GPU-hour at 24/7) or an 8-GPU server for $10,490. Dedicated, paid in crypto.
- [Rent a dedicated RTX 5090 cloud server for $449 per month](https://overgpu.com/gpus/nvidia-rtx-5090): Rent a whole RTX 5090 (32 GB GDDR7) in a dedicated VM for $449/month, about $0.62 per GPU-hour at 24/7. For image generation, small LLMs and dev work.
- [RTX 5090 vs RTX PRO 6000: when is 32 GB enough?](https://overgpu.com/compare/rtx-5090-vs-rtx-pro-6000): A 32 GB card at $449 a month or a 96 GB pro card with ECC and MIG at $1,390. See exactly when 32 GB is enough for your work.
- [RTX PRO 6000 vs L40S: is 96 GB worth the step up?](https://overgpu.com/compare/rtx-pro-6000-vs-l40s): 96 GB vs 48 GB, FP4 vs FP8, $1,390 vs $890 a month. See which card your work needs.
- [Docker GPU containers with the NVIDIA Container Toolkit](https://overgpu.com/docs/docker-containers): Use --gpus, select devices, reserve GPUs in Compose, fix PyTorch shared memory and run ROCm containers.
- [LLM VRAM calculator: will your model fit on a GPU?](https://overgpu.com/tools/vram-calculator): Pick Llama, Qwen, Mistral or gpt-oss, set the precision and context length, and see weights, KV cache and which GPUs fit.