Objective quality assessment on multi resolution adaptive video

The challenge with evaluating quality on multi resolution video is that when comparing encodes to the source the resolution varies widely and as such a programmatic comparison (PSNR, SSIM) cannot be easily done. Therefore the options available are either to scale the source for comparison or scale the output. As the output is often a low bitrate and it is unlikely to scale well the proposed approach is to scale the input for comparison.

Steps:

  1. Choose an appropriate source of approx 300 frames in a suitable broadcast contribution format for feeding to a linear encoder e.g. PAL 576i 16:9
  2. Set the encoding profiles to include all the low bitrate profiles as well as a reference profile with the output in the same format as the encode at a suitable broadcast bitrate e.g. SD 2-3Mbps
  3. Feed the source to the encoder and capture the output e.g. capture multiple TS files for a H264 encode
  4. Also capture the source as full frame lossless format e.g. raw video, mjpeg or use the original source if available in full frame visually lossless file format
  5. Scale the full frame source into the resolutions that match the target encodes, these are the new objective measurement sources
  6. Process the source for SSIM/DMOS, JND, PSNR

Process the above sources going from very low bitrates and resolutions e.g. 100k and QCIF up to a point where the encode is 80% broadcast quality and look for step jumps.

In addition for visual comparison the encodes should be dumped to bitmaps or visually lossless JPEG images for a golden eyes offline comparison.

Note: Another issue that still needs to be resolved on this is what to do in the situation where the source video is also zoomed or cropped which is often the case with low bitrate video.