Human Perceptual Evaluations for Image Compression
arXiv:1908.04187
The paper conducts human user studies to show that deep‑learning image compression methods optimized for higher MS‑SSIM scores can actually look worse to people than traditional codecs with lower MS‑SSIM, questioning the reliability of such perceptual metrics.
Abstract
Recently, there has been much interest in deep learning techniques to do image compression and there have been claims that several of these produce better results than engineered compression schemes (such as JPEG, JPEG2000 or BPG). A standard way of comparing image compression schemes today is to use perceptual similarity metrics such as PSNR or MS-SSIM (multi-scale structural similarity). This has led to some deep learning techniques which directly optimize for MS-SSIM by choosing it as a loss function. While this leads to a higher MS-SSIM for such techniques, we demonstrate using user studies that the resulting improvement may be misleading. Deep learning techniques for image compression with a higher MS-SSIM may actually be perceptually worse than engineered compression schemes with a lower MS-SSIM.
arXiv admin note: text overlap with arXiv:1907.08310