Sizing GPU Servers for Video Analytics: Streams, FPS and Models

How is GPU server video analytics capacity calculated? Decode load, analysis FPS, model chains, a worked example, pilot measurement with nvidia-smi and N+1 redundancy.
Technical summary
Technology
NVIDIA GPU, NVDEC, nvidia-smi, FFmpeg, DeepStream, RTSP, H.264/H.265
Topic
GPU capacity calculation and redundancy in server-based video analytics
Level
Advanced
Prerequisites
  • Analytics software and model selection
  • Knowledge of camera stream profiles and codecs
  • At least one GPU server for the pilot
In this article8 sections

The most expensive mistake in server-based video analytics is buying GPU capacity according to the “N cameras per server” figure in the brochure. Real capacity is determined by the number of streams, resolution, codec, analysis frame rate, model complexity and the other services running on the same server, together. The brochure knows none of these for your site. Below we give, in order, the variables we look at in GPU server video analytics sizing, a worked example, the pilot measurement method and redundant design.

The edge-or-server decision is out of scope here; that comparison is in the edge analytics or server-based analytics article. Everything below assumes a server-based architecture has been chosen.

The variables that determine capacity

On an analytics server, each camera stream goes through this chain: RTSP ingest and parsing, video decoding, scaling and color conversion, model inference, tracking, rule evaluation, and writing metadata and events. The bottleneck can form at any of these links. The weakest link determines capacity.

Varies Resource affected Note
Number of streams All The assumption of linear growth must be verified in the pilot
Resolution and codec Hardware decoder, GPU memory H.265 support depends on the GPU generation
Camera frame rate Hardware decoder Decoded independently of the analysis rate
Analysis frame rate GPU compute Determined by rule type
Model and input size GPU compute, memory Multiple model chains multiply the load
Tracking and rules CPU Increases in busy scenes
Writing metadata and events Disk, database Significant if attribute search is enabled

Video decoding load

Most sizing work focuses on model inference and skips video decoding. Yet in large installations, the first resource to fill up is often the hardware video decoder. On NVIDIA GPUs, this work is done by NVDEC units separate from the compute cores; the number of NVDECs and supported codecs vary by GPU model. For current information, check NVIDIA’s video encode/decode GPU support matrix.

Decode load is roughly proportional to the number of pixels decoded per second. Worked example (64 cameras):

  • Main stream 1920 × 1080, 25 frames/s: 64 × 1920 × 1080 × 25 ≈ 3.3 billion pixels/s.
  • Substream 1280 × 720, 10 frames/s: 64 × 1280 × 720 × 10 ≈ 0.59 billion pixels/s.

The difference is more than fivefold. If the model input is small anyway (for example 640 pixels wide), decoding the main stream and scaling it down wastes capacity. Defining a separate stream profile for analytics is the most effective optimization in most projects; we explained profile setup in the camera multi-streaming and stream profiles article. There’s a limit: the substream resolution mustn’t leave the pixel size of distant targets too small for analytics.

Another detail often skipped on site: in H.264 and H.265, intermediate frames reference previous frames. Even if analysis uses 5 frames per second, the decoder usually has to decode every frame in the stream. In the decode calculation, use the frame rate the camera sends, not the analysis frame rate. The effect of codec differences on processing load is in the H.265 vs H.264 recording performance article.

Analysis FPS and model complexity

Inference load is the total number of inferences per second multiplied by the cost per model. 64 cameras at 5 analysis frames per camera makes 320 inferences per second. Three factors increase this value:

  • Model chain: In PPE analytics that first does person detection and then hard hat/vest classification for each person, the second model’s load grows with the number of people in the scene. Size a system that runs light in an empty scene and heavy at shift change for the busiest moment, not for the average.
  • Input size: Enlarging the input size or splitting the image into tiles for distant targets increases inference cost roughly in proportion to the number of pixels.
  • Analysis rate: Slow events such as a virtual fence work with a low analysis rate; fast vehicles and proximity warnings need a higher rate. Instead of giving every camera the same analysis rate, define profiles by rule type.

GPU memory is a separate limit: decoder buffers, model weights and batch buffers for each stream take up memory. If memory fills while compute capacity is sufficient, no new streams can be added. The usage graph looks green, and the stream still won’t open.

Worked sizing example

The values below are constructed to illustrate the method; real values must be measured in a pilot with the GPU, model and software used. Target: 64 cameras, substream 1280 × 720 / 10 frames/s, analysis rate 5 frames/s, person and vehicle detection.

  1. Pilot measurement: 16 streams from real cameras are run on a single-GPU server. Values measured at the busy hour: average GPU compute usage 45%, NVDEC usage 30%, GPU memory 9 GB of 24 GB.
  2. Target usage: At most 70% usage is targeted on each resource, for sudden loads and growth.
  3. Estimated stream limit per resource (linear assumption): compute 16 × 70 / 45 ≈ 24; decoder 16 × 70 / 30 ≈ 37; memory 16 × (0.7 × 24) / 9 ≈ 29. The bottleneck is compute, at about 24 streams per GPU.
  4. Number of GPUs: 64 / 24 ≈ 2.7, so at least 3 GPUs.
  5. Verification: The linear assumption doesn’t always hold because of batch processing and memory behavior. A second measurement is done with 24 streams and the estimate verified.

Account for the network side too: if, for example, 1.5 Mbit/s is assumed per substream, 64 streams mean about 96 Mbit/s of continuous ingest. If the analytics server is separate from the recording server, each camera sends streams to two clients at once. Some cameras limit the number of simultaneous streams; the second connection is refused, or the existing stream’s settings change. Check this camera-side limit and the network load before installation.

Measuring with a pilot

Run the pilot with cameras chosen from the real site that represent the variety of scenes, using the software version and rule set that will be used in production. Measurement should last at least a full day and include busy hours. On NVIDIA GPUs, the basic measurements are taken with nvidia-smi:

# GPU compute (sm), memory, encoder and decoder usage; 5 s interval
nvidia-smi dmon -s u -d 5

# Memory and temperature; can be logged as CSV
nvidia-smi --query-gpu=timestamp,index,utilization.gpu,memory.used,memory.total,temperature.gpu \
  --format=csv -l 5 >> gpu_pilot.csv

You can quickly test with FFmpeg whether a single stream can be decoded in hardware, and its real frame rate (FFmpeg must be built with CUDA support; the RTSP path varies by manufacturer):

ffmpeg -hide_banner -hwaccel cuda -rtsp_transport tcp \
  -i "rtsp://user:[email protected]:554/<substream-path>" \
  -t 60 -f null -

Don’t look only at the GPU during measurement. Also record: CPU usage, the processed frame rate and dropped frames reported by the analytics software, event latency (the time between the moment of the event and the alarm reaching the VMS), and the disk write rate of the metadata database. If frames are dropped while GPU usage is low, the bottleneck is in the CPU, the network or the software’s queue settings. The first thing to look at in the console is the dropped frame counter.

If the video analytics workload is being moved to a central server, measure simultaneous streams and storage needs under server hardware and capacity consulting.

Scaling and redundancy

In enterprise video analytics systems, when the analytics server fails, recording continues and alarms stop. Nobody may notice. In scenarios such as perimeter security or occupational safety warnings, this is unacceptable. Make the redundancy decision with one question: when a server is lost, is the remaining capacity enough for all cameras? If 24 streams per GPU is accepted in the example above:

Design Total capacity On losing one server For 64 cameras
2 servers × 2 GPUs 96 streams 48 streams Insufficient
2 servers × 3 GPUs 144 streams 72 streams Sufficient
3 servers × 2 GPUs 144 streams 96 streams Sufficient, with room for growth

Which server streams move to at the moment of failure depends on the analytics software’s capabilities: some platforms redistribute automatically, while others need manual assignment to the standby server. Regularly verify that rule and zone configurations are also up to date on the standby server; a standby that comes up with an old rule set produces false alarms. The failover setup on the VMS side is in the VMS high availability and N+1 design article; include analytics servers in that setup. If the NVIDIA ecosystem is used, the DeepStream developer guide is the primary source for batch processing and multi-stream settings.

Common mistakes

  • Analyzing on the main stream: When the model input is small, decoding a 4K or 1080p main stream needlessly consumes decode capacity.
  • Sizing on average load: A calculation based on the value measured in an empty scene at night leads to dropped frames at shift change.
  • Forgetting memory: GPU memory fills while compute capacity looks sufficient, and no new streams can be added.
  • Treating consumer GPUs like data center products: Check the manufacturer’s terms on continuous operation, cooling, driver license conditions and virtualization support.
  • Thermal limits: Busy GPUs generate heat in the cabinet; if clock speed drops because of temperature, capacity falls below the pilot value. A pilot run in winter misleads in summer.
  • Leaving no room for growth: When a new rule, extra model or cameras are added, capacity falls short; a target usage of around 70% leaves this margin.

Server sizing, analytics software selection and rule setup can be planned under AI video analytics deployment, and integrating analytics with the VMS and the server architecture together with our enterprise VMS platforms work.

Frequently asked questions

Why doesn’t the number of streams in a manufacturer’s benchmark hold on site?

A benchmark is usually measured with a specific model, input size and batch setting, and often with video read from a local file. On site, RTSP ingest, different codecs and resolutions, the tracker and rules engine, metadata writing and other services share the same resources. Use the benchmark as a rough upper limit; size with pilot measurements.

Does dropping analysis to 5 frames per second also reduce decode load?

Usually not directly. In H.264 and H.265 streams, intermediate frames depend on previous frames; even if analysis uses only some frames, the decoder in most cases has to decode all of them. The way to lower decode load is to take a lower-resolution, lower-frame-rate substream from the camera.

Can video analytics be done with a CPU instead of a GPU?

It’s possible with a small number of cameras and light models. When both video decoding and inference are done on the CPU, the cost per stream rises quickly. If you’re considering a CPU-based design, apply the same pilot measurement method in terms of CPU usage and latency.

Can the GPU server be the same machine as the VMS recording server?

In small installations, it can. However, separating resources is preferred so that sudden spikes in analytics load don’t affect recording continuity. If recording and analytics are on the same server, give the recording service priority and also monitor disk write performance in the load test.

Support

Contact us for support or a discovery session on this topic.

Contact form
[email protected]

Eretek Video Surveillance

IP cameras, recording and central management infrastructure. We plan blind spots, retention periods and the camera network around the risks of each site.

Knowledge Center

Other articles