Compressed Video Communications Compressed Video Communications Abdul H. Sadka UniversityofSurrey,Guildford,UK Copyright©2002byJohnWiley&Sons,Ltd BaffinsLane,Chichester, WestSussexPO191UD,England National 01243779777 International((cid:1)44)1243779777 e-mail(forordersandcustomerserviceenquiries):[email protected] VisitourHomePageon:http://www.wiley.co.uk or http://www.wiley.com AllRightsReserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,or transmitted,inanyformorbyanymeans,electronic,mechanical,photocopying,recording,scanningor otherwise,exceptunderthetermsoftheCopyright,DesignsandPatentsAct1988orunderthetermsofa licenceissuedbytheCopyrightLicensingAgency,90TottenhamCourtRoad,LondonW1P9HE,UK, withoutthepermissioninwritingofthePublisher,withtheexceptionofanymaterialsuppliedspecifically forthepurposeofbeingenteredandexecutedonacomputersystem,forexclusiveusebythepurchaserof thepublication. NeithertheauthorsnorJohnWiley&SonsLtdacceptanyresponsibilityorliabilityforlossordamage occasionedtoanypersonorpropertythroughusingthematerial,instructions,methodsorideascontained herein,oractingorrefrainingfromactingasaresultofsuchuse.TheauthorsandPublisherexpressly disclaimallimpliedwarranties,includingmerchantabilityoffitnessforanyparticularpurpose.Therewill benodutyontheauthorsorPublishertocorrectanyerrorsordefectsinthesoftware. Designationsusedbycompaniestodistinguishtheirproductsareoftenclaimedastrademarks.Inall instanceswhereJohnWiley&Sonsisawareofaclaim,theproductnamesappearininitialcapitalor capitalletters.Readers,however,shouldcontacttheappropriatecompaniesformorecomplete informationregardingtrademarksandregistration. OtherWileyEditorialOffices JohnWiley&Sons,Inc.,605ThirdAvenue, NewYork,NY10158-0012,USA Wiley-VCHVerlagGmbH,Pappelallee3, D-69469Weinheim,Germany JacarandaWileyLtd,33ParkRoad,Milton, Queensland4064,Australia JohnWiley&Sons(Canada)Ltd,22WorcesterRoad, Rexdale,OntarioM9W1L1,Canada JohnWiley&Sons(Asia)PteLtd,ClementiLoop(cid:1)02-01, JinXingDistripark,Singapore129809 LibraryofCongressCataloguinginPublicationData Sadka,AbdulH. Compressedvideocommunications/AbdulH.Sadka. p. cm. Includesbilbiographicalreferencesandindex. ISBN0-470-84312-8 1.Digitalvideo. 2.Videocompression. I.Title. TK6680.5.S24 2002 621.388—dc21 2001058144 BritishLibraryCataloguinginPublicationData AcataloguerecordforthisbookisavailablefromtheBritishLibrary ISBN0470843128 Typesetin10.5/13ptTimesbyVisionTypesetting,Manchester PrintedandboundinGreatBritainbyAntonyRoweLtd,Chippenham,Wilts Thisbookisprintedonacid-freepaperresponsiblymanufacturedfromsustainableforestry, inwhichatleasttwotreesareplantedforeachoneusedforpaperproduction. To both my parents, my lovely wife and our little daughter. Contents Preface xi Acknowledgements xiii AbouttheAuthor xv 1 Introduction 1 1.1 Background 1 1.2 SourceMaterial 3 1.3 VideoQualityAssessmentandPerformanceEvaluation 3 1.4 OutlineoftheBook 6 1.5 References 9 2 Overview of Digital Video Compression Algorithms 11 2.1 Introduction 11 2.2 WhyVideoCompression? 12 2.3 UserRequirementsfromVideo 12 2.3.1 Videoqualityandbandwidth 12 2.3.2 Complexity 13 2.3.3 Synchronisation 14 2.3.4 Delay 14 2.4 ContemporaryVideoCodingSchemes 14 2.4.1 Segmentation-basedcoding 17 2.4.2 Model-basedcoding 19 2.4.3 Sub-bandcoding 21 2.4.4 Codebookvector-basedcoding 23 2.4.5 Block-basedDCTtransformvideocoding 24 2.4.6 NoveltiesofITU-TH.263videocodingstandard 40 2.4.7 PerformanceevaluationofITU-TH.263videocodingstandard 43 2.4.8 PerformancecomparisonbetweenITU-TH.261andH.263 47 2.5 Object-basedVideoCoding 54 2.5.1 VOPencoder 55 2.5.2 Shapecoding 55 2.5.3 Motionestimationandcompensation 60 2.5.4 Paddingtechnique 60 2.5.5 Basicmotiontechniques 61 2.5.6 Texturecoding 66 2.5.7 MPEG-4VOPdecoder 66 viii CONTENTS 2.5.8 Performanceevaluation 67 2.5.9 Layeredvideocoding 68 2.6 Conclusions 71 2.7 References 72 3 Flow Control in Compressed VideoCommunications 75 3.1 Introduction 75 3.2 BitRateVariabilityofVideoCoders 76 3.3 FixedRateCoding 79 3.4 AdjustingEncodingParametersforRateControl 80 3.5 VariableQuantisationStepSizeRateControl 83 3.5.1 Buffer-basedratecontrol 84 3.5.2 Feed-forwardratecontrol 87 3.6 ImprovedQualityRateControlUsingROICoding 90 3.7 RateControlUsingPrioritisedInformationDrop 94 3.8 RateControlUsingtheInternalFeedbackLoop 100 3.9 ReducedResolutionRateControl 102 3.9.1 Reducedresolutionschemewithadaptiveframeratecontrol 107 3.10 RateControlUsingMulti-layerCoding 109 3.11 FineGranularScaleability 112 3.12 Conclusions 114 3.13 References 118 4 Error Resilience in Compressed Video Communications 121 4.1 Introduction 121 4.2 EffectsofBitErrorsonPerceptualVideoQuality 122 4.3 ErrorConcealmentTechniques(Zero-redundancy) 125 4.3.1 RecoveryoflostMVsandMBcodingmodes 127 4.3.2 Recoveryoflostcoefficients 129 4.4 DataPartitioning 130 4.4.1 Unequalerrorprotection(UEP) 132 4.5 ForwardErrorCorrection(FEC)inVideoCommunications 134 4.5.1 Rate-compatiblepuncturedcodes(RCPC) 135 4.5.2 Cyclicredundancycheck(CRC) 137 4.6 DuplicateMVInformation 138 4.7 INTRARefresh 141 4.7.1 AdaptiveINTRArefresh(AIR) 143 4.8 RobustI-frame 146 4.9 ModifiedH.263forMobileApplications(H.263/M) 148 4.9.1 Fixedlengthcoding(FLC) 149 4.9.2 Changedorderoftransmission 150 4.9.3 COD-mapcoding 150 4.9.4 AvoidingfalsesynchwordsinMVstream 151 4.9.5 Insertionofsynchwordsatfixedintervals 151 4.9.6 GOBindicatorcoding 151 4.9.7 Frametypeindicatorcoding 153 4.10 Two-wayDecodingandReversibleVLC 153 4.11 Error-resilientEntropyCoding(EREC) 161 4.12 CombinedErrorResilienceSchemes 164 4.13 ErrorResilienceBasedonReferencePictureSelection 168 CONTENTS ix 4.14 Conclusions 172 4.15 References 174 5 Video CommunicationsOver Mobile IP Networks 177 5.1 Introduction 177 5.2 Evolutionof3GMobileNetworks 177 5.3 VideoCommunicationsfromaNetworkPerspective 179 5.3.1 Whypacketvideo? 180 5.4 DescriptionofFutureMobileNetworks 182 5.5 QoSIssuesforPacketVideooverMobileNetworks 184 5.5.1 Packetisationschemes 185 5.5.2 Throughputandchannelcodingschemes 188 5.6 Real-timeVideoTransmissionsoverMobileIPNetworks 190 5.6.1 PacketisationofdatapartitionedMPEG-4videousingRTP/UDP/IP 191 5.7 QualityOptimisationforVideoTransmissionsoverMobileNetworks 196 5.7.1 Enhancedvideoqualityusingadvancederrorprotection 197 5.7.2 Content-basedadaptivequalitycontrolformobilevideotransmissions 198 5.8 PrioritisedTransportforRobustVideoTransmissionsoverMobileNetworks 200 5.9 VideoTransmissionsoverGPRS/UMTSNetworks 204 5.10 Conclusions 212 5.11 References 213 6 Video Transcoding for Inter-networkCommunications 215 6.1 Introduction 215 6.2 WhatisTranscoding? 215 6.3 HomogeneousVideoTranscoding 217 6.4 BitRateReduction 219 6.5 CascadedFullyDecoding/Re-encodingScheme 220 6.6 TranscodingwithRe-quantisationScheme 220 6.6.1 Picturedrifteffect 222 6.6.2 Drift-freetranscoder 224 6.7 TranscodingwithMotionDataRe-useScheme 226 6.8 TranscodingwithMotionDataRe-estimationScheme 228 6.9 TranscodingwithMotionRefinementScheme 228 6.9.1 MVrefinementalgorithm 229 6.9.2 Effectsofrefinementwindowsizeontranscodingquality 232 6.10 PerformanceEvaluationofRateReductionTranscodingAlgorithms 234 6.11 FrameRateReduction 234 6.12 ResolutionReduction 238 6.13 HeterogeneousVideoTranscoding 240 6.14 VideoTranscodingforError-resliencePurposes 243 6.15 VideoTranscodingforMultimediaTrafficPlanning 248 6.16 Conclusions 252 6.17 References 252 AppendixA LayeringsyntaxofITU-TH.263videocodingstandard 257 AppendixB DescriptionofthevideoclipsonthesupplementaryCD 265 GlossaryofTerms 270 Index 274 Preface This book examines the technologies underlying the compression and trans- mission of digital video sequences over networking platforms. The incorporated studycoversalargespectrumoftopicsrelatedtocompressedvideocommunica- tions.Itpresentstoreadersacomprehensiveandstructuredanalysisoftheissues encountered in the transmission of compressed video streams over networking environments. This analysis identifies the problems impeding the progress of compressedvideocommunicationtechnologiesandimpairingthequalityofnet- workedvideoservices;italsopresentsawiderangeofsolutionsthatwouldhelpin the quality optimisationof video communicationservices. The book is a unique reference in which the author has combined the discussions of several topics related to digital video technology such as compression, error resilience, rate control,video transmissionover mobilenetworks and transcoding. All the tech- niques, algorithms, tools and mechanisms connected to the provision of video communication services are explained and analysed from the application layer downtothetransportandnetworklayers.Thedescriptionofthesetechnologiesis accompaniedbyalargenumberofvideosubjectiveandobjectiveresultsinorder to help graduate students, engineers and digital video researchersmake the best understanding of the presented material and build a comprehensive view on the progress of this rather fertile and exciting field of technology. A number of videoclipshavealsobeenpreparedandputatthedisposalofreadersonthesup- plementaryCDinordertobackuptheresultsdepictedandconclusionsreached withinthebookchapters. AbdulH.Sadka