Question

What are the supported video dimensions, aspect ratios, and rendering on mobile SDKs for the Vonage Video API?

Applies To

• Mobile SDKs
• Video Dimensions
• Aspect Ratios
• Rendering
• Video API

Answer

Many of the default implementations and samples floating around the Vonage Video API might falsely suggest limitations about the layouts of video that can be sent and received through the service. Since the JS SDK is commonly using horizontal formats attached to a webcam, we will ignore endpoints in this category for this document.

Known Issues

In iOS 2.5.0, there is an issue with resolutions greater than VGA (640x480), where the scaling behavior is not correct. This will be addressed in the next release.

Update 2.7.0: Both mobile SDKs still struggle with properly encoding resolutions greater than 1920 pixels (either width or height). It is recommended that you keep captured video frames to lower than this limit, especially for screen sharing applications where lost pixel data can be particularly disruptive to the user experience.

Supported Aspect Ratios

Although we generally stick to 4:3 dimensions sets in sample code, aspect ratio is largely ignored throughout the platform. All streams with video have width and height dimensions, passed through out-of-band signaling to all clients connected to a session. Additionally, the video render driver (OTVideoRender on iOS, BaseVideoRenderer on Android) is passed raw dimensions of each decoded video frame. This means that, as long as you are producing frames of compatible size, you can send video at any aspect ratio and consume it in your endpoints.

Rotation is applied prior to encoding and transmitting published video. Images captured by the camera are always transmitted full pixel. Switching orientation values in the video capture driver will simply trigger a dimensions update, all captured pixels are encoded regardless of the orientation value.

The inversion of aspect ratio width-height values resulting from changing device orientation from landscape to portrait should be handled as with any other change in video dimensions. Subscriber renderers are typically instructed to either 'fill' or 'fit' their existing frames, or a more sophisticated layout manager might handle changes in dimensions by triggering different layout arrangements. This always just depends on the needs of your app.

Supported Video Dimensions

Like aspect ratio, video dimensions are largely ignored throughout the platform. The only restriction here is that video sources should stick to the Mod16 rule. Ensure that both height and width of a frame are divisible by 16 pixels. Beyond that, the only limitation is in what each device can handle for data throughput.

Video Rendering Modes

The default renderers will fill their view containers with content, regardless of the underlying dimensions of sent (and received) frames. This means that if there is a mismatch between the video frame and the renderer's containing frame, the aspect ratio will be preserved, therefore pixel data will be cropped when the renderer scales to fill the view container.

Android's BaseVideoRenderer has a setStyle method to force fit all of the received pixels into its frame, while preserving the original aspect ratio. There is no corresponding setter for the iOS SDK, however the sample renderer can be customized to provide similar functionality.

Some developers choose to set new frame dimensions to match received video dimensions to preserve full pixel rendering. Mobile apps should hook into the video render driver. JS apps should listen to streamPropertyChanged events to detect changes to video size.