Appendix A Investigation of Image Quality of Dirac, H.264 and H.265 This appendix is based on the project by Biju Shrestha [E44]. For more details, please see Projects (Spring 2012) on UTA’s EE5359 course website: http://www-ee.uta.edu/Dip/Courses/EE5359/index.html. A.1 Introduction Shrestha [E44] has implemented H.265 for QCIF and CIF sequences and compared with Dirac and H.264. There exist several standards for video compression with additional improvements in performance and qualities in comparison to their older versions [H46]. The image quality of Dirac, H.264 and H.265 can be investigated using metrics like PSNR, CSNR, MSE, SSIM, MS SSIM,andFSIM[Q13,Q27,Q28]usingvarioustestsequences.Theconventional metrics like PSNR and MSE are a measure of intensity and cannot measure the subjectivefidelity[Q16].ThemetricslikeSSIMandFSIMtakeintoaccountofthe human visual system. A.2 H.265 H.265 is also known as HEVC [E5] and it can deliver significantly improved compression performance relative to that of the AVC (ITU-T H.264 | ISO/IEC 14496-10) [E5]. Alshina et al. [E2] investigated the coding efficiency with high resolution, HD 1080p, and concluded that it can be increased by average 37 and 36 %bitsavingsforhierarchicalBstructureandIPPPstructurewhencomparedto MPEG-4 AVC [E2]. The typical block-based video codec is composed of many processes including intra prediction and interprediction, transforms, quantization, entropy coding, and filtering [E10] as shown in Figs. A.1 and A.2. Over the decade, video coding techniques have gone through intensive research to achieve higher coding efficiencies. K.R.Raoetal.,VideoCodingStandards, 271 SignalsandCommunicationTechnology,DOI:10.1007/978-94-007-6742-3, (cid:2)SpringerScience+BusinessMediaDordrecht2014 272 AppendixA:InvestigationofImageQualityofDirac,H.264andH.265 Fig.A.1 EncoderblockdiagramofH.265.Greyboxesareproposedtoolsandwhiteboxesare H.264/AVCtools[E10](cid:2)2011ETRI Fig.A.2 DecoderblockdiagramofH.265.Greyboxesareproposedtoolsandwhiteboxesare H.264/AVCtools[E10](cid:2)2011ETRI A.3 Image Quality Assessment Using SSIM and FSIM Digital images and videos are prone to different kinds of distortions during different phases like acquisition, processing, compression, storage, transmission, and reproduction [Q2]. This degradation results in poor visual quality. There are AppendixA:InvestigationofImageQualityofDirac,H.264andH.265 273 several metrics which are widely used to quantify the image quality like FSIM, SSIM, bitrates, PSNR and MSE [Q28, Q2, Q13, Q17]. The conventional metrics likePSNRandMSEaredirectlydependentontheintensityofanimageanddonot correlatewiththesubjectivefidelityratings[Q16]. MSEcannotmodelthehuman visual system very accurately [Q19]. The measured parameters like PSNR, MSE, andSSIMofDirac,H.264,andH.265willbecomparedtostudytheircomparative characteristics and make conclusions. SSIM is the quality assessment of an image based on the degradation of structural information [Q13]. The SSIM takes an approach that the human visual system is adapted to extract structural information from images [Q17]. Thus, it is importanttoretainthestructuralsignalforimagefidelitymeasurement.Figure A.3 shows the difference between nonstructural and structural distortions. The nonstructural distortions are changes in parameters like luminance, contrast, gamma distortion, and spatial shift and are usually caused by environmental and instrumental conditions occurred during image acquisition and display [Q17]. On the other hand, structural distortion embraces additive noise, blur, and lossy compression [Q17]. The structural distortions change the structure of an image [Q17]. Figure A.4 explains the measurement system used in the calculation of SSIM. For given vectors, x¼fxji¼1;2;...;Ng and y¼fyji¼1;2;...;Ng. SSIM i i is evaluated on three different metrics like luminance, contrast, and structure which are described mathematically by Eqs. (A.1), (A.2), and (A.3) respectively [Q17]. Fig.A.3 Differencebetweennonstructuralandstructuraldistortions[Q22](cid:2)2009IEEE 274 AppendixA:InvestigationofImageQualityofDirac,H.264andH.265 Fig.A.4 BlockdiagramofSSIMmeasurementsystem[Q22](cid:2)2009IEEE (cid:2) (cid:3) 2lxlyþC1 l x;y ¼ ðA:1Þ l2þl2þC x y 1 (cid:2) (cid:3) 2rxryþC2 c x;y ¼ ðA:2Þ r2þr2þC x y 2 (cid:2) (cid:3) rxyþC3 s x;y ¼ ðA:3Þ r r þC x y 3 Here, l and l = local sample means of x and y respectively x y r and r = local sample standard deviations of x and y respectively x y r = local sample correlation coefficient between x and y xy C , C , and C = constants that stabilize the computations when denominators 1 2 3 become small General form of SSIM index can be obtained by combining equations (A.1), (A.2) and (A.3) [Q27]. (cid:2) (cid:3) h(cid:2) (cid:3)iah (cid:2) (cid:3)ibh (cid:2) (cid:3)ic SSIM x;y ¼ l x;y c x;y s x;y ðA:4Þ Here a, b, and c are parameters that mediate the relative importance of those three components. Using a¼b¼c¼1. We get [Q27], (cid:2) (cid:3)(cid:2) (cid:3) (cid:2) (cid:3) 2lxly + C1 2rxyþC2 SSIM x;y ¼(cid:2) (cid:3)(cid:2) (cid:3) ðA:5Þ l2þl2þC r2r2þC x y 1 x y 1 AppendixA:InvestigationofImageQualityofDirac,H.264andH.265 275 Fig.A.5 MSEandSSIMmeasurementsofimagesunderdifferentdistortions.aOriginalimage, (MSE=0,SSIM=1)bmeanluminanceshift,(MSE=144,SSIM=0.988)ccontraststretch, (MSE = 144, SSIM = 0.913) d impulse noise contamination, (MSE = 144, SSIM = 0.840) e blurring, (MSE = 144, SSIM = 0.694) and f JPEG [J18] compression (MSE = 142, SSIM = 0.662)[Q13] Figure A.5 shows the different distorted images which are quantified using MSE and SSIM. It is clearly visible that the different images are of different quality based on human visual system (HVS). However, all the distorted images have approximately same MSE, whereas SSIM is less for poor quality image giving much better image quality indication than that of MSE. Afeaturesimilarity(FSIM)indexisbasedonthefactthatHVSunderstandsan image mainly according to its low-level features [Q28]. The phase congruency (PC)isadimensionlessmeasureofthesignificanceofalocalstructure[Q28].PC and image gradient magnitude (GM) measurements are used as primary and secondary feature respectively in FSIM [Q28]. FSIM score is calculated by applying PC as a weighting function on the image local quality characterized by PC and GM [Q28]. FSIM is designed for gray-scale images [Q28] and FSIMc incorporates the chrominance information. FSIM can be mathematically modeled as follows [Q28]. P (cid:2) (cid:3) S ðxÞPC ðxÞ FSIM x;y ¼ x2X L m ðA:6Þ P PC ðxÞ x2X m Here, S ðxÞ = overall similarity between reference image and distorted image. L FSIMccanbemathematicallymodeledasshownin(A.7)andthecomputation process is illustrated in Fig. A.6 [Q28]. 276 AppendixA:InvestigationofImageQualityofDirac,H.264andH.265 Fig.A.6 IllustrationforFSIM/FSIMcindexcomputation.f isthereferenceimage,andf isa 1 2 distortedversionoff [Q28].(cid:2)2011IEEE 1 P S ðxÞ½S ðxÞ(cid:2)kPC ðxÞ FSIM ¼ x2X L C m ðA:7Þ C P PC ðxÞ x2X m Here, k[0isthe parameterused toadjusttheimportance ofthe chrominance components. P S ðxÞS ðxÞ½SðxÞS ðxÞ(cid:2)kPC ðxÞ FSIM ¼ x2X PC G I Q m C P PC ðxÞ x2X m All the metrics use different approaches to compare the images quantitatively. This different approach makes one method different from another. Table A.1 shows the ranking of image quality assessment metric performance on six databases.ItcanbeseenfromTable A.1thatFSIMisbetterthanSSIMandSSIM is better than PSNR when implementing an image quality assessment. TableA.1 Rankingofimagequalityassessmentmetricsperformance(FSIM,SSIMandPSNR) onsixdatabases[Q28] TID2008 CSIQ LIVE IVC MICT A57 FSIM 1 1 1 1 1 1 SSIM 2 2 2 2 2 2 PSNR 3 3 3 3 3 3 AppendixA:InvestigationofImageQualityofDirac,H.264andH.265 277 Fig.A.7 OriginalForeman QCIFsequence[V3].Video Information.QCIFsequence: foreman_qcif.yuv.Frame height:176.Framewidth: 144.Framerate:30frame/s. Total#offramesusedfor encoding:30frames A.4 Results A.4.1 Results Using Foreman QCIF Sequence Figures A.9, A.10, A.11 and Table A.2. A.4.2 Results Using Foreman CIF Sequence Figures A.14, A.15, A.16 and Table A.3. A.4.3 Results Using Container QCIF Sequence Figures A.19, A.20, A.21 and Table A.4. A.4.4 Results Using Container CIF Sequence Figures A.24, A.25, A.26, and Table A.5. A.5 Conclusions Theappendixisaimed instudyingthequalitativeperformancesofdifferent video codecs with a primary focus on Dirac, H.264 and H.265 [D24, H36, E50]. Different parameters like PSNR, MSE, and SSIM at various bitrates were measuredforallthreevideocodecstomakeacomparativestudy.Basedonvarious test sequences of different spatial/temporal resolutions, MATLAB, Microsoft VisualStudio,andMSUvideoqualitymeasurementtools(Fig. A.27)[Q30]were 278 AppendixA:InvestigationofImageQualityofDirac,H.264andH.265 Dirac at 87.32 kbps H.264 at 87.6 kbps H.265 at 76.80 kbps (baseline profile) Dirac at 152.85 kbps H.264 at 142.82 kbps H.265 at 162.46 kbps (baseline profile) Dirac at 397.60 kbps H.264 at 323.76 kbps H.265 at 398.21 kbps (baseline profile) Dirac at 4266.92 kbps H.264 at 3667.01 kbps H.265 at 2301.14 kbps (baseline profile) Fig.A.8 ForemanQCIFsequenceresultsusingdifferentcodecs[E44] AppendixA:InvestigationofImageQualityofDirac,H.264andH.265 279 Fig.A.9 PSNRachievedat PSNR vs bitrate variousbitratesforForeman 60 QCIFsequence[E44] 55 50 B d 45 n R i 40 Dirac N 35 S H.264 P 30 H.265 25 20 0 200 400 600 800 1000 Bitrate (kbps) Fig.A.10 MSEachievedat MSE vs bitrate variousbitratesforForeman 50 QCIFsequence[E44] 45 40 35 30 E S M 25 Dirac 20 H.264 15 H.265 10 5 0 0 200 400 600 800 1000 Bitrate (kbps) Fig.A.11 SSIMachievedat SSIM vs bitrate variousbitratesforForeman 1 QCIFsequence[E44] 0.98 0.96 0.94 ex 0.92 d n 0.9 M I 0.88 Dirac SI H.264 S 0.86 H.265 0.84 0.82 0.8 0 200 400 600 800 1000 Bitrate (kbps) 280 AppendixA:InvestigationofImageQualityofDirac,H.264andH.265 TableA.2 TabularresultsforY-componentusingForemanQCIFsequence[E44] Dirac Bitrate(kbps) PSNR(dB) MSE SSIM 87.31543 24.2585 243.9108 0.70942 104.1455 27.10295 126.7024 0.80875 152.8516 30.81619 53.88419 0.89656 224.4629 35.58351 17.9775 0.95434 397.6055 39.69524 6.97519 0.97947 805.6953 43.87075 2.66689 0.99047 1615.6 47.5301 1.14834 0.99547 2918.021 50.96503 0.52069 0.99801 4266.925 55.04958 0.20329 0.99934 H.264 Bitrate(kbps) PSNR(dB) MSE SSIM 3667.01 69.941 0.00723 1 3280.58 63.357 0.0309 0.9999 2903.03 58.825 0.08601 0.9996 2352.62 55.135 0.20001 0.9991 1952.06 53.245 0.30894 0.9987 1586.51 51.563 0.45541 0.9981 1205.7 49.597 0.71567 0.9971 954.55 47.961 1.04312 0.9959 736.1 46.25 1.54575 0.9942 540.58 44.37 2.38023 0.9917 418.68 42.799 3.41684 0.9885 323.76 41.293 4.83252 0.9845 240.47 39.632 7.08134 0.9784 182.57 38.186 9.88001 0.972 142.82 36.904 13.27354 0.9652 111.9 35.448 18.55763 0.9544 87.6 34.056 25.56888 0.9426 H.265 Bitrate(kbps) PSNR(dB) MSE SSIM 2301.136 61.0449 0.09232 0.99946 1810.152 56.4716 0.2055 0.99881 1407.104 53.6141 0.34358 0.99812 1080.7 51.5846 0.51131 0.99738 827.016 49.9778 0.71189 0.99655 639.616 48.5619 0.96235 0.99556 502.432 47.2557 1.27745 0.99439 398.208 45.9397 1.70959 0.99277 319.312 44.7036 2.25273 0.99081 255.792 43.3888 3.03526 0.98797 205.376 42.0663 4.09816 0.98408 162.456 40.7085 5.59949 0.97936 (continued)