The operator workstation stutters on a 16-tile grid, the mobile app gets no video, and the analytics server’s GPU is maxed out. These three complaints often share the same root: the camera’s streams haven’t been separated by task. An IP camera can produce several streams at once: a full-resolution main stream for recording, a low-resolution substream for the live grid and, on most models, a third stream for mobile access or analytics. Below we explain stream types, starting settings per stream, how the VMS chooses streams per client and the limits of the camera’s processing capacity. Bitrate and GOP settings within a single stream are the subject of the setting bitrate with CBR, VBR and smart codecs article.
Why multiple streams are needed
In a single-stream system, everything runs off the main stream. On a 16-camera grid, the client has to decode sixteen 4 MP streams, the image stutters on mobile connections, and the analytics server processes oversized frames. Worked example: decoding a 4 MP, 20 fps stream means processing 16 times more pixels per frame than a 640×360 stream (2560×1440 ÷ 640×360). If each tile on the grid already takes up around 640×360 on screen, this extra processing adds nothing to the image. It just burns the processor.
With multiple streams, each consumer gets as much data as it needs:
- Recording: The highest resolution and frame rate needed for evidential value.
- Live grid: A small stream matching the tile size, so the workstation can decode many cameras at once.
- Single camera full screen: The main stream or a stream close to it.
- Mobile and remote connections: A low-bitrate stream resilient to limited bandwidth.
- Analytics: A stream chosen according to the model’s input size and the target’s pixel density requirement.
Stream types and ONVIF profiles
In the camera interface, streams are usually called “main stream,” “substream” and “third stream.” On the ONVIF side, each stream corresponds to a media profile, and each profile has a token. When a VMS connects to a camera via ONVIF, it reads the profiles and their linked video encoder configurations with the GetProfiles call, then gets the RTSP address of the chosen profile with GetStreamUri. Profile S cameras use the Media service, and Profile T cameras the Media2 service; Media2 also defines H.265 support. Differences between profiles are explained in the ONVIF Profile T, G and M differences article and on the ONVIF profiles page.
A common problem is a mapping error. Profile tokens are named differently by each manufacturer, and on some cameras the ONVIF profiles don’t map one-to-one to the streams in the web interface. The result: the VMS has taken the substream for recording and nobody noticed, until footage of an incident is requested. After adding the camera to the VMS, check in the VMS interface which profile is assigned to which task. Don’t trust automatic mapping.
Settings table per stream
The values below are a starting point for general-purpose enterprise cameras. The exact values are determined by the scene, the target pixel density and the VMS’s stream selection behavior.
| Stream | Resolution | Frame rate | Use |
|---|---|---|---|
| Main | The camera’s full resolution | 12–25 fps (depending on the scene) | Recording, full-screen viewing |
| Sub | 640×360 (16:9) or 640×480 (4:3) | 10–15 fps | Grid viewing, long-retention secondary recording |
| Third | 1280×720 | 10–15 fps | Mobile, remote locations, medium-size tiles |
| Analytics | According to the model’s needs | As much as the analytics requires | Server-based analytics |
Three rules simplify the design:
- The same aspect ratio on all streams: If the main stream is 16:9, the substream should be 16:9 too (such as 640×360). The 704×576 (PAL D1) value common on substreams is defined for a 4:3 image; used with a 16:9 main stream, motion zones, privacy masks and analytics boxes shift. A substream left at factory settings is the most common source of this error.
- Codec consistency: If hardware-accelerated decoding on the client supports H.265, all streams can be H.265; if not, using H.264 on the substream preserves grid performance. The compatibility side of codec choice is in the H.265 vs H.264 comparison article.
- GOP length: A very long GOP on the substream delays the first image when switching to a camera in live view; the client waits for the first I-frame. The operator reports this as “the camera opens slowly.”
Stream selection and adaptive viewing in the VMS
Most enterprise VMSs choose a stream according to the pixel size of the tile on the client with an “adaptive streaming” feature. The substream is shown while the operator is on a 4×4 grid, and the main stream when they double-click a camera to go full screen. For this to work efficiently:
- At least two streams with different resolutions must be defined on the camera; two streams at the same resolution defeat the selection logic.
- In the VMS, whether each stream is used for recording, live viewing or both must be assigned explicitly.
- When playing back recorded footage, only the recorded stream can be used. If the substream isn’t recorded, adaptive selection isn’t possible during playback and the client decodes the main stream; that’s why a 16-camera playback grid runs much heavier than a live grid. Some VMSs therefore support recording both streams together or keeping the substream longer.
- For mobile clients, some VMSs transcode on the server side. This creates a heavy processor load on the mobile server; using the camera’s third stream directly is often more efficient.
The number of simultaneous sessions per camera also enters the design. The number of simultaneous RTSP connections cameras support is limited. In a setup where every client connects directly to the camera, beyond a certain point the camera rejects new connections or drops frames on all streams. The right setup is for the VMS to receive the stream once and distribute it to clients, or to use multicast. Multicast design is explained in the multicast and IGMP snooping on camera networks article.
A separate stream for analytics
Server-based video analytics takes a separate stream from the camera. Choosing this stream balances two needs: the target appearing with enough pixel density, and the analytics server’s decoding and inference load.
- Resolution: Object detection models usually scale input down to a fixed size. Decoding a 4 MP stream and then scaling it down consumes more GPU/CPU than taking a 720p stream. License plate recognition or small distant targets, however, need high resolution; then the main stream or a cropped region stream is used.
- Frame rate: Fast-moving targets (vehicles) need a high frame rate; in scenarios such as perimeter intrusion, a low frame rate may be enough. Use the frame rate recommended by the analytics manufacturer.
- Bitrate: Aggressive compression and smart codecs degrade object edges, and detection accuracy drops. Consider turning off the smart codec on the analytics stream; a setting that helps recording can harm analytics.
Whether analytics runs on the camera or the server changes stream design from the outset; the comparison is in the edge analytics vs. server-based analytics article.
Limits of camera processing capacity
A camera’s encoder is limited by the total number of pixels it can process per second. That’s why streams aren’t independent of each other. When the highest resolution and frame rate are selected on the main stream, the third stream may be disabled, the substream’s resolution options may narrow or on-camera analytics may have to be turned off. On some models, corridor mode, image rotation or high WDR modes also change stream options. There’s one budget.
Stream combinations on the manufacturer’s datasheet are best-case values. After configuring, measure that each stream really arrives with the requested values. With ffprobe from the FFmpeg package, codec, resolution and frame rate can be checked quickly (the RTSP address is the one the VMS obtained via ONVIF or the one shown in the camera interface):
ffprobe -v error -rtsp_transport tcp -select_streams v:0 \
-show_entries stream=codec_name,width,height,avg_frame_rate \
-of compact "rtsp://user:[email protected]:554/<stream-path>"
For the actual frame rate and bitrate, the VMS’s camera statistics screen is more reliable; look at an average over a few minutes. If you see dropped frames on the main stream, turn off the third stream or on-camera analytics and measure the difference. The order matters: measure first, then turn off, then measure again.
Configuration checklist
- Stream tasks (recording, live, mobile, analytics) have been defined in writing for each camera group.
- Main stream resolution and frame rate were chosen according to the target pixel density and scene motion.
- The aspect ratio is the same on all streams.
- Substream resolution matches the client tile size; clients’ decoding capacity has been tested.
- Stream–task mapping and adaptive streaming thresholds have been checked in the VMS.
- If a low-resolution stream is also needed for playback, substream recording and its retention period have been set.
- A decision has been made on a third stream or server-side encoding for mobile access.
- The analytics stream’s resolution, frame rate and smart codec settings were set according to the analytics manufacturer’s recommendation.
- The number of simultaneous sessions per camera and the need for multicast have been assessed.
- The actual resolution, frame rate and bitrate of each stream have been measured and recorded.
Stream design is an input to camera network traffic and recording storage; in the bandwidth calculation, each stream is added separately. For ONVIF profile mapping to run properly on cameras from different manufacturers, you can see the scope on the ONVIF and RTSP camera integration service page.
Frequently asked questions
Can the substream be used for recording?
It can, but the substream’s resolution is usually not enough for identification and recognition. Substream recording makes sense for general area cameras needing long retention, or as a lower-quality second recording kept longer alongside the main stream. Recordings that will carry evidential value are taken from the main stream.
Does enabling the camera’s third stream lower main stream quality?
Yes, if the camera’s encoding capacity is limited. On some models, enabling a third stream narrows the main stream’s frame rate or resolution options. Check the manufacturer’s stream combination table and measure the main stream’s actual frame rate after the change.
How does the VMS decide which stream to show in live view?
VMSs with adaptive streaming compare the pixel size of the tile on the client with the resolutions of the camera streams and choose the smallest stream that fills the tile. When the operator switches to full screen, the main stream is used. Selection thresholds are set in the VMS configuration.
What happens if the substream and main stream have different aspect ratios?
Motion detection zones, privacy masks and analytics boxes shift between streams; switching from live view to recording, the image looks cropped or shifted. Choosing the same aspect ratio on all streams eliminates this problem.