<%BANNER%>

Video Distortion Analysis and System Design for Wireless Video Communication

Permanent Link: http://ufdc.ufl.edu/UFE0042464/00001

Material Information

Title: Video Distortion Analysis and System Design for Wireless Video Communication
Physical Description: 1 online resource (170 p.)
Language: english
Creator: Chen, Zhifeng
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: clipping, clrc, distortion, ermpc, errdo, fading, jscc, rmpc, video, wireless
Electrical and Computer Engineering -- Dissertations, Academic -- UF
Genre: Electrical and Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: In this dissertation, we address the problem of minimizing the end-to-end distortion in wireless video communication. We first analytically derive transmission distortion as a function of video statistics, channel conditions and system parameters for wireless video communication systems. Then we design practical algorithms to estimate the system parameters and video statistics. Given the channel condition, we may accurately predict the instantaneous transmission distortion by our formulae and estimation algorithms. We also prove a new theorem to extend our algorithms to support rate-distortion optimized mode decision in practical video codecs. Finally, we derive a more accurate source bit rate model and quantization distortion model than existing parametric models. Our models help us to design a rate-distortion optimized cross-layer rate control algorithm for minimizing the end-to-end distortion under resource constraints in wireless video communication systems. Our results achieve remarkable performance gains over existing solutions.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Zhifeng Chen.
Thesis: Thesis (Ph.D.)--University of Florida, 2010.
Local: Adviser: Wu, Dapeng.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2011-06-30

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0042464:00001

Permanent Link: http://ufdc.ufl.edu/UFE0042464/00001

Material Information

Title: Video Distortion Analysis and System Design for Wireless Video Communication
Physical Description: 1 online resource (170 p.)
Language: english
Creator: Chen, Zhifeng
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: clipping, clrc, distortion, ermpc, errdo, fading, jscc, rmpc, video, wireless
Electrical and Computer Engineering -- Dissertations, Academic -- UF
Genre: Electrical and Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: In this dissertation, we address the problem of minimizing the end-to-end distortion in wireless video communication. We first analytically derive transmission distortion as a function of video statistics, channel conditions and system parameters for wireless video communication systems. Then we design practical algorithms to estimate the system parameters and video statistics. Given the channel condition, we may accurately predict the instantaneous transmission distortion by our formulae and estimation algorithms. We also prove a new theorem to extend our algorithms to support rate-distortion optimized mode decision in practical video codecs. Finally, we derive a more accurate source bit rate model and quantization distortion model than existing parametric models. Our models help us to design a rate-distortion optimized cross-layer rate control algorithm for minimizing the end-to-end distortion under resource constraints in wireless video communication systems. Our results achieve remarkable performance gains over existing solutions.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Zhifeng Chen.
Thesis: Thesis (Ph.D.)--University of Florida, 2010.
Local: Adviser: Wu, Dapeng.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2011-06-30

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0042464:00001


This item has the following downloads:


Full Text

PAGE 1

VIDEODISTORTIONANALYSISANDSYSTEMDESIGNFORWIRELESSVIDEOCOMMUNICATIONByZHIFENGCHENADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2010

PAGE 2

c2010ZhifengChen 2

PAGE 3

Idedicatethisdissertationtomyfather. 3

PAGE 4

ACKNOWLEDGMENTS Firstandtheforemost,IwouldliketoexpressmydeepestgratitudetomyadvisorProf.DapengWuforhisguidanceandhelpinthedevelopmentofmyresearch.Thisworkwouldnothavebeenpossiblewithouthisenlighteninginstruction,constructiveadvice,andwillingnesstoprovidefunding.Hisextensiveknowledge,stronganalyticalskills,andcommitmenttotheexcellenceofresearcharetrulytreasurestohisstudents.IwouldalsoliketothankProf.JohnHarris,Prof.TaoLi,andProf.ShigangChenforservingonmydissertationcommitteeandprovidingvaluablesuggestionsonthisdissertation.IhavebeenfortunatetobeastudentofProf.JohnM.Shea,whoisoneofthebestteachersthatIhavehadinmylife.Hisdeepknowledge,responsibleattitudeandimpressivekindnesshavehelpedmetodevelopthefundamentalandessentialacademiccompetence.IamindebtedtoTaoranLuforherexplanationofmyquestionswhenIrstencounteredchallengesinstudyingsignalprocessing.IgratefullyacknowledgethehelpofXiaochenLiformyunderstandingincommunicationtheory.IespeciallythankJunXuforhisvaluablediscussionswhenIbeganmyresearchonvideocoding.MyworkalsoowesmuchtoQianChenforherhelpintheuseofcorrectgrammar,whichimprovesthepresentationofthisdissertation.IwouldliketotakethisopportunitytothankXihuaDong,QinChen,LeiYang,BingHan,WenxingYe,ZongruiDing,YakunHu,andJiangpingWangformanyfruitfuldiscussionsrelatedtothiswork.IwishtoexpressmyspecialappreciationtoPeshalaPahalawattaandAlexisMichaelTourapisfortheirhelpinsolvingmyquestionsabouttheH.264/AVCJMreferencesoftwareandassistingmewithmorerigorousexpressionofmanyideasinthiswork.Lastbutnotleast,Ineedtoexpressmywarmestthankstomyparentsandmywifefortheircontinuedencouragementandsupport. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 10 LISTOFFIGURES ..................................... 11 ABSTRACT ......................................... 13 CHAPTER 1INTRODUCTION ................................... 14 1.1ProblemStatement ............................... 14 1.1.1TheoreticalBackground ........................ 14 1.1.2ChallengesinthePracticalSystem .................. 16 1.2ContributionsofThisDissertation ....................... 19 1.3StructureoftheDissertation .......................... 21 2PREDICTIONOFTRANSMISSIONDISTORTIONFORWIRELESSVIDEOCOMMUNICATION:ANALYSIS ........................... 23 2.1BackgroundonTransmissionDistortionPrediction ............. 23 2.2SystemDescription .............................. 28 2.2.1StructureofaWirelessVideoCommunicationSystem ....... 28 2.2.2ClippingNoise ............................. 29 2.2.3DenitionofTransmissionDistortion ................. 32 2.2.4LimitationsoftheExistingTransmissionDistortionModels ..... 34 2.3TransmissionDistortionFormulae ....................... 36 2.3.1OverviewoftheApproachtoAnalyzingPTDandFTD ....... 37 2.3.2AnalysisofDistortionCausedbyRCE ................ 39 2.3.2.1Pixel-leveldistortioncausedbyRCE ............ 39 2.3.2.2Frame-leveldistortioncausedbyRCE ........... 40 2.3.3AnalysisofDistortionCausedbyMVCE ............... 42 2.3.3.1Pixel-leveldistortioncausedbyMVCE ........... 42 2.3.3.2Frame-leveldistortioncausedbyMVCE .......... 43 2.3.4AnalysisofDistortionCausedbyPropagatedErrorPlusClippingNoise .................................. 44 2.3.4.1Pixel-leveldistortioncausedbypropagatederrorplusclippingnoise ......................... 44 2.3.4.2Frame-leveldistortioncausedbypropagatederrorplusclippingnoise ......................... 48 2.3.5AnalysisofCorrelationCausedDistortion .............. 50 2.3.5.1Pixel-levelcorrelationcauseddistortion .......... 50 2.3.5.2Frame-Levelcorrelationcauseddistortion ......... 55 5

PAGE 6

2.3.6Summary ................................ 55 2.3.6.1Pixel-Leveltransmissiondistortion ............. 55 2.3.6.2Frame-Leveltransmissiondistortion ............ 56 2.4RelationshipbetweenTheorem 2.2 andExistingTransmissionDistortionModels ..................................... 56 2.4.1Case1:Onlythe(k)]TJ /F3 11.955 Tf 10.6 0 Td[(1)-thFrameHasError,andtheSubsequentFramesareAllCorrectlyReceived .................. 57 2.4.2Case2:BurstErrorsinConsecutiveFrames ............. 57 2.4.3Case3:ModelingTransmissionDistortionasanOutputofanLTISystemwithPEPasinput ....................... 58 2.5PTDandFTDunderMulti-ReferencePrediction ............... 60 2.5.1Pixel-levelDistortionunderMulti-ReferencePrediction ....... 60 2.5.2Frame-levelDistortionunderMulti-ReferencePrediction ...... 61 3PREDICTIONOFTRANSMISSIONDISTORTIONFORWIRELESSVIDEOCOMMUNICATION:ALGORITHMANDAPPLICATION ............. 63 3.1ALiteratureReviewonEstimationAlgorithmsofTransmissionDistortion 63 3.2AlgorithmsforEstimatingFTD ........................ 67 3.2.1FTDEstimationwithoutFeedbackAcknowledgement ........ 67 3.2.1.1Estimationofresidualcauseddistortion .......... 67 3.2.1.2EstimationofMVcauseddistortion ............ 70 3.2.1.3Estimationofpropagationandclippingcauseddistortion 72 3.2.1.4Estimationofcorrelation-causeddistortion ........ 74 3.2.1.5Summary ........................... 75 3.2.2FTDEstimationwithFeedbackAcknowledgement ......... 75 3.3Pixel-levelTransmissionDistortionEstimationAlgorithm .......... 76 3.3.1EstimationofPTD ........................... 77 3.3.2Calculationof^E[eku] ........................... 78 3.3.3Calculationof^E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg]and^Dku(p) ............ 78 3.3.4Summary ................................ 79 3.4Pixel-levelEnd-to-endDistortionEstimationAlgorithm ........... 79 3.5ApplyingRMPC-PEEDAlgorithmtoH.264PredictionModeDecision ... 81 3.5.1Rate-distortionOptimizedPredictionModeDecision ........ 81 3.5.2ComplexityofRMPC-MS,ROPE,andLLNAlgorithm ........ 83 3.5.2.1RMPC-MSalgorithm ..................... 84 3.5.2.2ROPEalgorithm ....................... 85 3.5.2.3LLNalgorithm ........................ 86 3.6ExperimentalResults ............................. 87 3.6.1EstimationAccuracyandRobustness ................ 88 3.6.1.1Experimentsetup ...................... 88 3.6.1.2Estimationaccuracyofdifferentestimationalgorithms .. 89 3.6.1.3Robustnessofdifferentestimationalgorithms ....... 92 3.6.2R-DPerformanceofModeDecisionAlgorithms ........... 93 3.6.2.1Experimentsetup ...................... 94 6

PAGE 7

3.6.2.2R-Dperformanceundernointerpolationlterandnodeblockinglter ....................... 94 3.6.2.3R-Dperformancewithinterpolationlteranddeblockinglter .............................. 96 4THEEXTENDEDRMPCALGORITHMFORERRORRESILIENTRATEDISTORTIONOPTIMIZEDMODEDECISION ........................... 101 4.1AnOverviewonSubpixel-levelEnd-to-endDistortionEstimationforaPracticalVideoCodec ............................. 101 4.2TheExtendedRMPCAlgorithmforModeDecision ............. 103 4.2.1Subpixel-levelDistortionEstimation .................. 104 4.2.2ANewTheoremforCalculatingtheSecondMomentofaWeightedSumofCorrelatedRandomVariables ................ 106 4.2.3TheExtendedRMPCAlgorithmforModeDecision ......... 107 4.2.4MeritsandLimitationsofERMPCAlgorithm ............. 109 4.2.4.1Merits ............................. 109 4.2.4.2Limitations .......................... 110 4.3ExperimentalResults ............................. 110 4.3.1ExperimentSetup ........................... 110 4.3.2R-DPerformance ............................ 111 4.3.3subjectivePerformance ........................ 113 4.3.4Discussion ................................ 114 4.3.4.1Effectofclippingnoiseonthemodedecision ....... 114 4.3.4.2Effectoftransmissionerrorsonmodedecision ...... 115 5RATE-DISTORTIONOPTIMIZEDCROSS-LAYERRATECONTROLINWIRELESSVIDEOCOMMUNICATION ............................. 117 5.1AnLiteratureReviewonRateDistortionModelsinWirelessVideoCommunicationSystems ..................................... 117 5.2ProblemFormulation .............................. 122 5.3DerivationofBitRateFunction,QuantizationDistortionFunctionandTransmissionDistortionFunction ....................... 125 5.3.1DerivationofSourceCodingBitRateFunction ........... 125 5.3.1.1TheentropyofquantizedtransformcoefcientsforI.I.D.zero-meanLaplaciansourceunderuniformquantizer .. 125 5.3.1.2Improvewithrunlengthmodel ............... 126 5.3.1.3PracticalconsiderationofLaplacianassumption ..... 128 5.3.1.4Improvementbyconsideringthemodelinaccuracy .... 128 5.3.1.5SourcecodingbitrateestimationfortheH.264encoder 129 5.3.2DerivationofQuantizationDistortionFunction ............ 129 5.3.3DerivationofTransmissionDistortionFunction ............ 130 5.3.3.1TransmissiondistortionasafunctionofPEP ....... 130 5.3.3.2PEPasafunctionofSNR,transmissionrate,andchannelcodingrateinafadingchannel ............... 131 7

PAGE 8

5.3.3.3TransmissiondistortionasafunctionofSNR,transmissionrate,andchannelcodingrateinafadingchannel ..... 134 5.4Rate-DistortionOptimizedCross-layerRateControlandAlgorithmDesign 134 5.4.1OptimizationofCross-layerRateControlProblem .......... 134 5.4.2AlgorithmDesign ............................ 135 5.5ExperimentalResults ............................. 137 5.5.1ModelAccuracy ............................. 137 5.5.1.1Bitratemodel ........................ 137 5.5.1.2Quantizationdistortionmodel ................ 138 5.5.1.3PEPmodel .......................... 139 5.5.2PerformanceComparison ....................... 140 5.5.2.1Experimentsetup ...................... 141 5.5.2.2PSNRperformance ..................... 142 5.5.2.3Subjectiveperformance ................... 144 6CONCLUSION .................................... 147 6.1SummaryoftheDissertation ......................... 147 6.2FutureWork ................................... 148 APPENDIX APROOFSINCHAPTER2 .............................. 150 A.1ProofofLemma 1 ............................... 150 A.2ProofofProposition 1 ............................. 150 A.3ProofofLemma 2 ............................... 151 A.4ProofofLemma 3 ............................... 152 A.5ProofofLemma 4 ............................... 152 A.6Lemma 5 andItsProof ............................. 154 A.7Lemma 6 andItsProof ............................. 154 A.8ProofofCorollary 1 .............................. 156 BPROOFSINCHAPTER3 .............................. 157 B.1ProofofProposition 3 ............................. 157 B.2ProofofTheorem 3.1 ............................. 157 B.3ProofofProposition 4 ............................. 159 CPROOFSINCHAPTER4 .............................. 160 DPROOFSINCHAPTER5 .............................. 162 D.1ProofofEquation( 5 ) ............................ 162 D.2CalculationofEntropyforDifferentQuantizedTransformCoefcients ... 162 D.3ProofofProposition 5 ............................. 163 REFERENCES ....................................... 165 8

PAGE 9

BIOGRAPHICALSKETCH ................................ 170 9

PAGE 10

LISTOFTABLES Table page 2-1Notations ....................................... 32 2-2Anexamplethatshowstheeffectofclippingnoiseontransmissiondistortion. 36 3-1ComplexityComparison ............................... 87 3-2AveragePSNRgain(indB)ofRMPC-MSoverROPEandLLN ......... 96 3-3AveragePSNRgain(indB)ofRMPC-MSoverROPEandLLNunderinterpolationltering ........................................ 99 4-1AveragePSNRgain(indB)ofERMPCoverRMPC,LLNandROPE ...... 114 5-1RCPCencoderparameters ............................. 140 10

PAGE 11

LISTOFFIGURES Figure page 1-1Theoreticalsystemmodel. .............................. 15 1-2Theoreticalsystemmodelwithseparatesourcecodingandchannelcoding. .. 16 1-3Practicalsystemmodelofawirelessvideocommunicationsystem. ....... 18 2-1Systemstructure,whereT,Q,Q)]TJ /F8 7.97 Tf 6.59 0 Td[(1,andT)]TJ /F8 7.97 Tf 6.58 0 Td[(1denotetransform,quantization,inversequantization,andinversetransform,respectively. ............ 30 2-2Theeffectofclippingnoiseondistortionpropagation. .............. 48 2-3Temporalcorrelationbetweentheresidualsinonetrajectory. .......... 51 2-4TemporalcorrelationmatrixbetweenresidualandMVCEinonetrajectory. ... 51 2-5TemporalcorrelationmatrixbetweenMVCEsinonetrajectory. ......... 52 2-6Comparisonbetweenmeasuredandestimatedcorrelationcoefcients. .... 53 3-1TransmissiondistortionDkvs.frameindexkfor`foreman':(a)goodchannel,(b)poorchannel. ................................... 90 3-2TransmissiondistortionDkvs.frameindexkfor`stefan':(a)goodchannel,(b)poorchannel. ................................... 91 3-3TransmissiondistortionDkvs.PEPfor`foreman'. ................. 91 3-4TransmissiondistortionDkvs.PEPfor`stefan'. .................. 92 3-5TransmissiondistortionDkvs.frameindexkfor`foreman'underimperfectknowledgeofPEP:(a)goodchannel,(b)poorchannel. ............. 93 3-6TransmissiondistortionDkvs.frameindexkfor`stefan'underimperfectknowledgeofPEP:(a)goodchannel,(b)poorchannel. .................... 93 3-7PSNRvs.bitratefor`foreman',withnointerpolationlterandnodeblockinglter:(a)PEP=2%,(b)PEP=5%. .......................... 95 3-8PSNRvs.bitratefor`football',withnointerpolationlterandnodeblockinglter:(a)PEP=2%,(b)PEP=5%. .......................... 95 3-9PSNRvs.bitratefor`foreman',withinterpolationandnodeblocking:(a)PEP=2%,(b)PEP=5%. ..................................... 98 3-10PSNRvs.bitratefor`football',withinterpolationandnodeblocking:(a)PEP=2%,(b)PEP=5%. ..................................... 98 11

PAGE 12

3-11PSNRvs.bitratefor`foreman',withinterpolationanddeblocking:(a)PEP=2%,(b)PEP=5%. ..................................... 100 3-12PSNRvs.bitratefor`football',withinterpolationanddeblocking:(a)PEP=2%,(b)PEP=5%. ..................................... 100 4-1PSNRvs.bitratefor`foreman':(a)PEP=0.5%,(b)PEP=2%. .......... 111 4-2PSNRvs.bitratefor`mobile':(a)PEP=0.5%,(b)PEP=2%. ........... 112 4-3(a)ERMPCatthe84-thframe,(b)RMPCatthe84-thframe,(c)LLNatthe84-thframe,(d)ROPEatthe84-thframe,(e)ERMPCatthe99-thframe,(f)RMPCatthe99-th,(g)LLNatthe99-thframe,(h)ROPEatthe99-thframe. .. 115 4-4PSNRvs.bitratefor`foreman':(a)PEP=0.5%,(b)PEP=2%. .......... 115 4-5PSNRvs.bitratefor`mobile':(a)PEP=0.5%,(b)PEP=2%. ........... 116 5-1Channelmodel. .................................... 123 5-2Variancemodel. ................................... 127 5-3bppvs.Frameindex:(a)foreman,(b)mobile. ................... 138 5-4Quantizationvs.Frameindex:(a)foreman,(b)mobile. .............. 139 5-5PEPunderdifferentRCPCcodingrates. ...................... 141 5-6PSNRvs.averageSNR:(a)foreman,(b)mobile. ................. 142 5-7PSNRvs.bandwidth:(a)foreman,(b)mobile. ................... 144 5-8ArandomchannelsampleunderaverageSNR=10dBandbitrate=1000kbps:(a)ArandomSNRsample,(b)Distortionvs.Frameindexforforeman cifunderthischannel. ..................................... 145 5-9Forthe10-thframe:(a)original,(b)CLRC,(c)proposed-constant-PEP,(d)constant-PEP-QP-limit;forthe11-thframe:(e)original,(f)CLRC,(g)proposed-constant-PEP,(h)constant-PEP-QP-limit. ............................. 146 A-1Comparisonof2(x,y)andx2. .......................... 154 12

PAGE 13

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyVIDEODISTORTIONANALYSISANDSYSTEMDESIGNFORWIRELESSVIDEOCOMMUNICATIONByZhifengChenDecember2010Chair:DapengWuMajor:ElectricalandComputerEngineering Inthisdissertation,weaddresstheproblemofminimizingtheend-to-enddistortioninwirelessvideocommunication.Werstanalyticallyderivetransmissiondistortionasafunctionofvideostatistics,channelconditionsandsystemparametersforwirelessvideocommunicationsystems.Thenwedesignpracticalalgorithmstoestimatethesystemparametersandvideostatistics.Giventhechannelcondition,wemayaccuratelypredicttheinstantaneoustransmissiondistortionbyourformulaeandestimationalgorithms.Wealsoproveanewtheoremtoextendouralgorithmstosupportrate-distortionoptimizedmodedecisioninpracticalvideocodecs.Finally,wederiveamoreaccuratesourcebitratemodelandquantizationdistortionmodelthanexistingparametricmodels.Ourmodelshelpustodesignarate-distortionoptimizedcross-layerratecontrolalgorithmforminimizingtheend-to-enddistortionunderresourceconstraintsinwirelessvideocommunicationsystems.Ourresultsachieveremarkableperformancegainsoverexistingsolutions. 13

PAGE 14

CHAPTER1INTRODUCTION 1.1ProblemStatement Bothmultimediatechnologyandmobilecommunicationshaveexperiencedmassivegrowthandcommercialsuccessinrecentyears.Asthesetwotechnologiesconverge,wirelessvideo,suchasvideophonecallsandmobileTVin3G/4Gsystems,isexpectedtoachieveunprecedentedgrowthandworldwidesuccess.Therefore,howtoimprovethevideoqualityreproducedatthevideodecoderinawirelessvideocommunicationsystembecomesmorecompelling. 1.1.1TheoreticalBackground AtheoreticalsystemmodelforvideotransmissionoverawirelesschannelisshowninFig. 1-1 ,whereVnistheinputvideosequenceand~VnistheoutputvideosequenceafterVnpassingthroughthewirelesschannel.Thetargetofthetransmissionistoconveythevideoinformationattheinputsidetotheoutputsideasmuchaspossible.However,1)thevideosequenceisusuallyhighlyredundant,whichcausesthewasteofresourcesifitistransmittedwithoutremovinganyredundancy;and2)thesourcebitstreamisusuallynotwelldistinguishableattheoutputsideafterpassingthroughthechannel,whichcausesseriousdistortion.Therefore,toconveymaximumdistinguishablevideoinformationwhileconsumingminimumresourcesfortheinformationtobetransmittedfromthetransmittertothereceiver,weneed1)compressingtheinputusingasfewbitsaspossible,thatis,sourcecoding;and2)mappingthesourcebitstreamintoabetter,inthebiterrorsense,bitstreamfortransmission,thatis,channelcoding. Nowtheproblemsare1)whatistheminimumrequirementofresourcesforreliablytransmittingthegivensource?2)forthegivenchannel,howmuchinformationatmostcanbereliablytransmitted?and3)whatistheminimumdistortionwhichmayhappeniftheinformationcontainedinthegivensourceismorethantheinformationthechannel 14

PAGE 15

Figure1-1. Theoreticalsystemmodel. mayconvey?In1948,ShannonpublishedhisseminalworkAMathematicalTheoryofCommunicationinBellSystemsTechnicalJournal[ 1 ].Inthispaper,Shannonmathematicallydenesthemeasureofinformationbyentropy,whichisexpressedbytheaveragenumberofbitsneededforstorageorcommunication.Inthisseminalworktheanswersaregiven,forthersttime,tothersttwoaforementionedquestions,thatis,1)theminimumnumberofbitsrequiredforreliablytransmittingthegivensourceisitsentropy;2)themaximumnumberofbitscanbereliablytransmittedforthegivenchannelisthechannelcapacity.Althoughnotrigorouslyproved,theanswertothethirdquestionisalsopresentedinRef.[ 1 ](hisTheorem21).Thatis3)theminimumdistortionforthegivensourceandchannelistheminimumdistortionachievedbylossysourcecodingundertheconditionthattheencodedsourcerateislessthanthechannelcapacity. In1959,ShannonpublishedanotherfamousworkCodingTheoremsforaDiscreteSourceWithaFidelityCriterion[ 2 ],wheretherate-distortionfunctionisrstcoinedandthegreatestlowerboundofrateforagivendistortionisproved.ThejointsourcechannelcodingtheoremprovesthattheoptimalperformancecanbeachievedbythesourcechannelseparationtheoremasstatedinRef.[ 3 ]Thesourcechannelseparationtheoremshowsthatwecandesignthesourcecodeandthechannelcodeseparatelyandcombinetheresultstoachieveoptimalperformance.Basedonthesourcechannel 15

PAGE 16

Figure1-2. Theoreticalsystemmodelwithseparatesourcecodingandchannelcoding. separationtheorem,thetheoreticalsystemmodelistoseparatesourcecodingandchannelcodingseparatelyandsequentiallyasinFig. 1-2 However,thistheoremisderivedundertheconditionthatalltransmissionerrorscanbecorrectedbythechannelcodingtoanarbitrarilylowprobability.Thatis,itimplicitlyassumesthatthereisnodistortioncausedbythetransmissionerrorinthesystemmodel.Althoughdecreasingthechannelprotection,i.e.,redundantbitswillincreasethetransmissionerror,italsoreducesthedistortioncausedbylossysourcecodinggiventhesamechannelcapacity.Therefore,itisstillnotclearthatwhatistheminimumdistortionforthegivensourceandchanneliftherestrictionofarbitrarilylowprobabilityoftransmissionerrorislifted.Inaddition,thechannelcapacityisderivedbasedontheassumptionsofinniteblocklength,randomcodingandstationarychannel.Ontheotherhand,therate-distortion(R-D)boundisderivedbasedontheassumptionsofinniteblocklength,randomcoding,andstationarysources.TheseassumptionsinbothchannelcapacityandR-Dboundincurinnitedelay,innitelyhighcomplexity,andmismatchbetweentheoreticalandpracticalsourceandchannelmodels. 1.1.2ChallengesinthePracticalSystem Inapracticalwirelessvideocommunicationsystem,theresourcesareverylimited.Thereareusuallyfourkindsofresources,thatis,time,bandwidth,powerandspace,whichcanbeutilizedtoimprovewirelessvideoperformance.However,allofthese 16

PAGE 17

fourresourcesareusuallylimited.Specically,1)theend-to-enddelay,sumofsourcecodingdelayandtransmissiondelay,forthevideosignaltobereproducedbyvideodecoderisundercertaindelaybound;2)theachievabledatarate,sumofinformationrateandredundantrate,isundercertainbandwidthlimit;3)thetotalpowerconsumedbyvideoencodingandbytransmissionareundercertainconstraint;4)thechannelgaininawirelessfadingchannelstatisticallydependsonthegeographicalpositionandenvironment.Therefore,duetothelimitedresources,theprobabilityoftransmissionerrorcannotbearbitrarilylowinthepracticalsystem.Instead,amoredesirablesystemdesignistominimizetheend-to-enddistortionundertheresourceconstraintsbyallowingthetransmissionerroratacertainlevel. Inapracticalwirelesscommunicationsystem,modulationanderrorcontrolcodingaredesignedtomitigatethebiterrorduringtransmissionthroughanerror-pronechannel.Inapplicationlayer,error-resilientcodingattheencoderanderrorconcealmentatthedecoderaredesignedtoreducethedistortioncausedbysuchtransmissionerror.Wecallthedistortioncausedbythetransmissionerrorastransmissiondistortion,denotedbyDt.Inapracticalvideocodingsystem,thepredictivecoding,quantization,transform,andentropycodingareadoptedtogethertocompressthebits.Suchasourcecodingschemeproduceserrorduringquantization1.Wecallthedistortioncausedbyquantizationerrorasquantizationdistortion,denotedbyDq.Asaresult,thedistortionbetweentheoriginalvideoandthereconstructedvideoatthevideodecoderiscausedbyboththequantizationerrorandtransmissionerror.Wecallthemtogetherasend-to-enddistortion,denotedbyDete.ThepracticalsystemmodelisshowninFig. 1-3 Ontheonehand,thetransmissiondistortionisafunctionofthetransmissionerror,whichisagainafunctionofsignal-to-noiseratio(SNR),bandwidth,delayrequirement 1Inmodernvideocodec,e.g.H.264codec,thetransformissodesignedthatitisreversible. 17

PAGE 18

Figure1-3. Practicalsystemmodelofawirelessvideocommunicationsystem. andchannelprotectionparameters,e.g.,modulationorderandchannelcodingrate.Ontheotherhand,thequantizationdistortionisafunctionofavailablesourcedatarate,complexityrequirement,sourceencoderstructureandsourcecodingparameters,e.g.,theallowablenitesetforquantization.NowtheprobleminapracticalsystemcanbeformulatedbyGiventhesource,channel,resourcesandsystemstructure,howtotunethesystemparameterstominimizetheend-to-enddistortion. Thisproblemisverychallengingsince1)thestatisticalpropertiesofthevideosourceisunknownandthesourceisusuallynotstationary;2)thewirelesschannelistimevarying;3)allresourcesarelimited;4)thesystemisacomplexsystem,e.g.,non-linear;5)thesystemparametersindifferentlayersareusuallycoupled;forexample,increasingthethechannelcodingrateinthetransmitterwilldecreasethesourcedatarateforcompression. Totacklethiscomplexproblem,weneedtofollowthefollowingsteps:1)Findingstablevideostatisticsforquantizationdistortionandderivingthequantizationdistortionasafunctionofsourcerateconstraint(Rs),complexityconstraint(Cs),videocodecstructureandthosestablevideostatistics(~);2)Findingstablevideostatisticsfortransmissiondistortionandderivingthetransmissiondistortionasafunctionofpacketerrorprobability(PEP),codecstructureandthosestablevideostatistics(~);3)Deriving 18

PAGE 19

thePEPasafunctionofSNR,channelcodingrate(Rc),bandwidth,andtransmissiondelay(dt);4)Minimizingtheend-to-enddistortionundertheresourceconstraints.Thankstothesourcechannelseparationtheorem,thesourcecodingandchannelcodinghasbeenextensivelystudiedseparately.Inotherwords,therststephasbeenextensivelystudiedbythesourcecodingsocietyandthethirdstephasbeenextensivelystudiedbythecommunicationsocietyseparately.IntheOpenSystemInterconnectionReferenceModel(OSIReferenceModelorOSIModel),therststepbelongstotheapplicationlayerproblem,andthethirdstepbelongstothelowerlayers.Althoughtheyarerelativelyextensivelyresearched,theystillneedtobefurtherinvestigatedinordertodesignapracticalsystemwiththeminimumend-to-enddistortion.Ontheotherhand,thesecondstep,whichinfactisacross-layerproblem,haslongbeomittedbybothsocieties.Untillnow,thereisstillnowellacceptedtheoreticalanalysisforthiscross-layerproblem.Ifwecanndtransmissiondistortionasaclosed-formfunctionofPEP,wemaybeabletoanalyticallyderivetheminimumend-to-enddistortionformostexistingwirelessvideocommunicationsystems,whicharedesignedbasedonthesourcechannelseparationtheorem. 1.2ContributionsofThisDissertation Themajorcontributionsofourworkaresummarizedasfollows: 1. WeanalyticallyderivethetransmissiondistortionformulaeasafunctionofPEPandvideostatisticsforwirelessvideocommunicationsystems. 2. Withconsiderationofspatio-temporalcorrelation,nonlinearcodecandtime-varyingchannel,ourformulaeprovide,forthersttime,thefollowingcapabilities: supportofdistortionpredictionatdifferentlevels(e.g.,pixel/frame/GOPlevel). supportofmulti-referencepicturemotioncompensatedprediction. supportofslicedatapartitioning. supportofarbitraryslice-levelpacketizationwithFMOmechanism. beingapplicabletotime-varyingchannels. oneuniedformulaforbothI-MBandP-MB. 19

PAGE 20

supportofbothlowmotionandhighmotionvideosequences. 3. Besidesderivingthetransmissiondistortionformulae,wealsoidentiedtwoimportantpropertiesoftransmissiondistortionforthersttime: clippingnoise,producedbynon-linearclipping,causesdecayofpropagatederror. thecorrelationbetweenmotionvectorconcealmenterrorandpropagatederrorisnegative,andhasdominantimpactontransmissiondistortion,amongallthecorrelationsbetweenanytwoofthefourcomponentsintransmissionerror. 4. Wealsodiscussedtherelationshipbetweenourformulaandexistingmodels,andspecifytheconditions,underwhichthoseexistingmodelsareaccurate. 5. Wedesignalgorithmstoestimatecorrelationratioandpropagationfactor,whichfacilitatesthedesignoflowcomplexityalgorithmforestimatingtheframe-leveltransmissiondistortion(FTD). 6. Byusingtheformulaeanalyticallyderivedandtheparameterestimatedbystatistics,ourFTDestimationalgorithm,calledRMPC-FTD,ismoreaccurateandmorerobustthanexistingFTDalgorithms. 7. AnotheradvantageofourRMPC-FTDalgorithmisthatallparametersintheformulaecanbeestimatedbyusingtheinstantaneousvideoframestatisticsandchannelconditions,whichallowsthevideoframestatisticstobetime-varyingandthetransmissionerrorprocessestobenon-stationary.Asaresult,ourRMPC-FTDalgorithmismoresuitableforreal-timevideocommunication. 8. Wealsodesigntheestimationalgorithm,calledRMPC-PTD,forpixel-leveltransmissiondistortion(PTD)byutilizingtheknownvaluesoftheMVandcorrespondingresidualtofurtherimprovetheestimationaccuracyanddecreasetheestimationcomplexity. 9. WealsoextendRMPC-PTDtoestimatepixel-levelend-to-enddistortion(PEED)bythealgorithmcalledRMPC-PEED.OurRMPC-PEEDalgorithmprovidesnotonlymoreaccurateestimationbutalsolowercomplexityandhigherdegreeofextensibilitythantheexistingmethods. 10. WeapplyourRMPC-PEEDalgorithmtopredictionmodedecisioninH.264;theresultingalgorithmiscalledRMPC-MS.ExperimentalresultsshowthatourRMPC-MSalgorithmachievesmorethan1dBgainthanexistingalgorithms. 11. Tofacilitatethedesignofsubpixel-levelMeanSquareError(MSE)distortionestimationformodedecisioninH.264videoencoders,weproveageneraltheorem 20

PAGE 21

forcalculatingthesecondmomentofaweightedsumofcorrelatedrandomvariableswithouttherequirementoftheirprobabilitydistribution. 12. Weapplyourtheoremtothedesignofaverylow-complexityalgorithm,whichwecallERMPCalgorithm,formodedecisioninH.264.Experimentalresultsshowthat,ERMPCfurtherachieves0.25dBPSNRgainovertheRMPC-MSalgorithm. 13. Wederivemoreaccuratesourcebitratemodelandquantizationdistortionmodelthanexistingparametricmodels. 14. WeimprovetheperformanceboundforchannelcodingwithconvolutionalcodesandaViterbidecoder,andderiveitsperformanceunderRayleighblockfadingchannel. 15. WedesignaR-Doptimizedcross-layerratecontrol(CLRC)algorithmbyjointlychoosingquantizationstepsizeandchannelcodingratebasedonthegiveninstantaneouschannelcondition,e.g.,SNRandchannelbandwidth. 1.3StructureoftheDissertation InChapter 2 ,weanalyticallyderivethetransmissiondistortionformulaeasafunctionofPEPandvideostatisticsforwirelessvideocommunicationsystems.Weexplainthelimitationsinexistingtransmissiondistortionmodels,wherethesignicanteffectofclippingnoiseonthetransmissiondistortionhaslongbeenomitted.WethenderiveboththePTDandFTDwithconsideringtheclippingnoiseinthesystem.Wealsodiscussedtherelationshipbetweenourformulaandexistingmodels;wespecifytheconditions,underwhichthoseexistingmodelsareaccurate. InChapter 3 ,wedesignpracticalalgorithmstoestimatethesystemparameters,andfromtheestimatedparameters,wemaycalculatetheFTDbyusingtheformulaederivedinChapter 2 .ForPTD,weutilizetheknownvalues,e.g.residual,insomevideocodecreplacingthestatisticsofthecorrespondingrandomvariablestosimplifythePTDestimationanddesignalow-complexityandhigh-accuracyPTDestimationalgorithm.WealsoextendRMPC-PTDalgorithmtoestimatePEEDwithhighdegreeofextensibility.wethenapplyourRMPC-PEEDalgorithmformodedecisioninH.264toachievetheminimumR-Dcost.ThecomplexityandmemoryrequirementofourRMPC-MSalgorithmandexistingmodeselectionalgorithmsarecarefullycomparedin 21

PAGE 22

thischapter.Experimentalresultsaregiventocomparetheestimationaccuracy,robust,R-Dperformanceandextensibilitybetweenouralgorithmsandexistingalgorithms. InChapter 4 ,weextendourRMPC-MSalgorithmdesignedinChapter 3 tosupportsomeperformance-enhancedparts,e.g.interpolationlter,inH.264codec.Werstproveanewtheoremforcalculatingthesecondmomentofaweightedsumofcorrelatedrandomvariableswithouttherequirementoftheirprobabilitydistribution.Then,weapplythetheoremtoextendthedesignofpreviousRMPC-MSalgorithmtosupporttheinterpolationlteringinH.264.WecallthenewalgorithmasERMPCalgorithm.WealsodiscussthemeritsandlimitationsofourERMPCalgorithm.ExperimentalresultsaregiventocomparetheR-DperformanceandsubjectiveperformancebetweenERMPCandexistingalgorithms. InChapter 5 ,weaimtodesignarate-distortionoptimizedcross-layerratecontrol(CLRC)algorithmforwirelessvideocommunication.Tothisend,wederiveamoreaccuratesourcebitratemodelandquantizationdistortionmodelthanexistingparametricmodels.WealsoimprovetheperformanceboundofchannelcodingwithconvolutionalcodesandaViterbidecoder,andderiveitsperformanceunderRayleighblockfadingchannels.Giventheinstantaneouschannelcondition,i.e.SNRandbandwidth,wedesigntherate-distortionoptimizedCLRCalgorithmbyjointlychoosingquantizationstepsizeandchannelcodingrate.Experimentalresultsaregiventocomparethemodelsaccuracybetweenoursandexistingmodels.WealsocomparetheR-Dperformanceandsubjectiveperformancebetweenouralgorithmsandexistingalgorithmsinthischapter. Finally,Chapter 6 concludesthedissertationandprovidesanoutlookforourfuturework. 22

PAGE 23

CHAPTER2PREDICTIONOFTRANSMISSIONDISTORTIONFORWIRELESSVIDEOCOMMUNICATION:ANALYSIS Inthischapter,weanalyticallyderivedthetransmissiondistortionformulaeforwirelessvideocommunicationsystems.Wealsodiscussedtherelationshipbetweenourformulaandexistingmodels. 2.1BackgroundonTransmissionDistortionPrediction Transmissiondistortioniscausedbypacketerrorsduringthetransmissionofavideosequence,anditisthemajorpartoftheend-to-enddistortionindelay-sensitivewirelessvideocommunication1underhighpacketerrorprobability(PEP),e.g.,inawirelessfadingchannel.Thecapabilityofpredictingtransmissiondistortionatthetransmittercanassistindesigningvideoencodingandtransmissionschemesthatachievemaximumvideoqualityunderresourceconstraints.Specically,transmissiondistortionpredictioncanbeusedinthefollowingthreeapplicationsinvideoencodingandtransmission:1)modeselection,whichistondthebestintra/inter-predictionmodeforencodinganmacroblock(MB)withtheminimumrate-distortion(R-D)costgiventheinstantaneousPEP,2)cross-layerratecontrol,whichistocontroltheinstantaneouslyencodedbitrateforareal-timeencodertominimizetheframe-levelend-to-enddistortiongiventheinstantaneousPEP,e.g.,invideoconferencing,3)packetscheduling,whichchoosesasubsetofpacketsofthepre-codedvideototransmitandintentionallydiscardstheremainingpacketstominimizetheGOP-level(GroupofPicture)end-to-enddistortiongiventheaveragePEPandaverageburstlength,e.g.,instreamingpre-codedvideoovernetworks.Allthethreeapplicationsrequireaformulaforpredictinghowtransmissiondistortionisaffectedbytheirrespectivecontrolpolicy,inordertochoosetheoptimalmodeorencodingrateortransmissionschedule. 1Delay-sensitivewirelessvideocommunicationusuallydoesnotallowretransmissiontocorrectpacketerrorssinceretransmissionmaycauselongdelay. 23

PAGE 24

However,predictingtransmissiondistortionposesagreatchallengeduetothespatio-temporalcorrelationinsidetheinputvideosequence,thenonlinearityofboththeencoderandthedecoder,andvaryingPEPintime-varyingchannels.Inatypicalvideocodec,thetemporalcorrelationamongconsecutiveframesandthespatialcorrelationamongtheadjacentpixelsofoneframeareexploitedtoimprovethecodingefciency.Nevertheless,suchacodingschemebringsmuchdifcultyinpredictingtransmissiondistortionbecauseapacketerrorwilldegradenotonlythevideoqualityofthecurrentframebutalsothefollowingframesduetoerrorpropagation.Inaddition,aswewillseeinSection 2.3 ,thenonlinearityofboththeencoderandthedecodermakestheinstantaneoustransmissiondistortionnotequaltothesumofdistortionscausedbyindividualerrorevents.Furthermore,inawirelessfadingchannel,thePEPistime-varying,whichmakestheerrorprocessanon-stationaryrandomprocessandhence,asafunctionoftheerrorprocess,thedistortionprocessisalsoanon-stationaryrandomprocess. Accordingtotheaforementionedthreeapplications,theexistingalgorithmsforestimatingtransmissiondistortioncanbecategorizedintothefollowingthreeclasses:1)pixel-levelorblock-levelalgorithms(appliedtomodeselection),e.g.,RecursiveOptimalPer-pixelEstimate(ROPE)algorithm[ 4 ]andLawofLargeNumber(LLN)algorithm[ 5 6 ];2)frame-levelorpacket-levelorslice-levelalgorithms(appliedtocross-layerratecontrol)[ 7 11 ];3)GOP-levelorsequence-levelalgorithms(appliedtopacketscheduling)[ 12 16 ].Althoughtheexistingdistortionestimationalgorithmsworkatdifferentlevels,theysharesomecommonproperties,whichcomefromtheinherentcharacteristicsofwirelessvideocommunicationsystem,thatis,spatio-temporalcorrelation,nonlinearcodecandtime-varyingchannel.Inthischapter,weusethedivide-and-conquerapproachtodecomposecomplicatedtransmissiondistortionintofourcomponents,andanalyzetheireffectsontransmissiondistortionindividually.This 24

PAGE 25

divide-and-conquerapproachenablesustoidentifythegoverninglawthatdescribeshowthetransmissiondistortionprocessevolvesovertime. Stuhlmulleretal.[ 8 ]observedthatthedistortioncausedbythepropagatederrordecaysovertimeduetospatiallteringandintracodingofMBs,andanalyticallyderivedaformulaforestimatingtransmissiondistortionunderspatiallteringandintracoding.TheeffectofspatiallteringisanalyzedundertheimplicitassumptionthatMVsarealwayscorrectlyreceivedatthereceiver,whiletheeffectofintracodingismodeledasalineardecayunderanotherimplicitassumptionthattheI-MBsarealsoalwayscorrectlyreceivedatthereceiver.However,thesetwoassumptionsareusuallynotvalidinrealisticdelay-sensitivewirelessvideocommunication.Toaddressthis,thischapterderivesthetransmissiondistortionformulaundertheconditionthatbothI-MBsandMVsmaybeerroneousatthereceiver.Inaddition,weobserveaninterestingphenomenonthatevenwithoutusingthespatiallteringandintracoding,thedistortioncausedbythepropagatederrorstilldecays!Weidentify,forthersttime,thatthisdecayiscausedbynon-linearclipping,whichisusedtoclipthoseout-of-range2reconstructedpixelaftermotioncompensation;thisistherstofthetwopropertiesidentiedinthischapter.Whilesuchout-of-rangevaluesproducedbytheinversetransformofquantizedtransformcoefcientsisnegligibleattheencoder,itscounterpartproducedbytransmissionerroratthedecoderhassignicantimpactontransmissiondistortion. Someexistingworks[ 8 9 ]estimatetransmissiondistortionbasedonalineartime-invariant(LTI)systemmodel,whichregardspacketerrorasinputandtransmissiondistortionasoutput.TheLTImodelsimpliestheanalysisoftransmissiondistortion.However,itsacricesaccuracyindistortionestimationsinceitneglectstheeffectofcorrelationbetweennewlyinducederrorandpropagatederror.Liangetal.[ 16 ] 2Areconstructedpixelvaluemaybeoutoftherangeoftheoriginalpixelvalue,e.g.,[0,255]. 25

PAGE 26

studiedtheeffectofcorrelationandobservedthattheLTImodels[ 8 9 ]underestimatetransmissiondistortionduetothepositivecorrelationbetweentwoadjacenterroneousframes;however,theydidnotconsidertheeffectofmotionvector(MV)errorontransmissiondistortionandtheiralgorithmwasnottestedwithhighmotionvideos.Toaddresstheseissuesandndtherootcauseofthatunderestimation,thischapterclassiesthetransmissionreconstructederrorintothreeindependentrandomerrors,namely,ResidualConcealmentError(RCE),MVConcealmentError(MVCE),andpropagatederror;thersttwotypesoferrorarecallednewlyinducederror.Weidentify,forthersttime,thatMVCEisnegativelycorrelatedwithpropagatederrorandthiscorrelationhasdominantimpactontransmissiondistortion,amongallthecorrelationsbetweenanytwoofthethreeerrortypes,forhighmotionvideos;thisisthesecondofthetwopropertiesidentiedinthischapter.Forthisreason,aslongasMVtransmissionerrorsexistinhighmotionvideos,theLTImodelover-estimatestransmissiondistortion.Wealsoquantiestheeffectofindividualerrortypesandtheircorrelationsontransmissiondistortioninthischapter.Thankstotheanalysisofcorrelationeffect,ourdistortionformulaisaccurateforbothlowmotionvideoandhighmotionvideoasveriedbyexperimentalresults.AnothermeritofconsideringtheeffectofMVerrorontransmissiondistortionistheapplicabilityofourresultstovideocommunicationwithslicedatapartitioning,wheretheresidualandMVcouldbetransmittedunderUnequalErrorProtection(UEP). Refs.[ 4 5 10 11 ]proposedsomemodelstoestimatetransmissiondistortionundertheconsiderationthatbothMVandI-MBmayexperiencetransmissionerrors.However,theparametersinthelinearmodels[ 10 11 ]canonlybeacquiredbyexperimentallycurve-ttingovermultipleframes,whichforbidsthemodelsfromestimatinginstantaneousdistortion.Inaddition,thelinearmodels[ 10 11 ]stillassumethereisnocorrelationbetweenthenewlyinducederrorandpropagatederror.InRef.[ 4 ],theROPEalgorithmconsidersthecorrelationbetweenMVconcealmenterrorand 26

PAGE 27

propagatederrorbyrecursivelycalculatingthesecondmomentofthereconstructedpixelvalue.However,ROPEneglectsthenon-linearclippingfunctionandthereforeover-estimatesthedistortion.Inaddition,theextensionofROPEalgorithm[ 17 ]tosupporttheaveragingoperations,suchasinterpolationanddeblockinglteringinH.264,requiresintensivecomputationofcorrelationcoefcientsduetothehighcorrelationbetweenreconstructedvaluesofadjacentpixels,andtherebyprohibitingitfromapplyingtoH.264.InH.264referencecodeJM14.03,theLLNalgorithm[ 5 ]isadoptedsinceitiscapableofsupportingbothclippingandaveragingoperations.However,inordertopredicttransmissiondistortion,allpossibleerroreventsforeachpixelinallframesshouldbesimulatedattheencoder,whichsignicantlyincreasesthecomplexityoftheencoder.DifferentfromRef.[ 4 5 ],thedivide-and-conquerapproachinthischapterenablesourformulatoprovidenotonlymoreaccuratepredictionbutalsolowercomplexityandhigherdegreeofextensibility.ThemultiplereferencepicturemotioncompensatedpredictionextendedfromthesinglereferenceisanalyzedinSection 2.5 ,and,forthersttime,theeffectofmultiplereferencesontransmissiondistortionisquantied.Inaddition,thetransmissiondistortionformuladerivedinthischapterisuniedforbothI-MBsandP-MBs,incontrasttotwodifferentformulaeinRefs.[ 4 10 11 ]. Differentfromwiredchannels,wirelesschannelssufferfrommultipathfading,whichcanberegardedasmultiplicativerandomnoise.Fadingleadstotime-varyingPEPandbursterrorsinwirelessvideocommunication.Ref.[ 8 ]usesatwo-statestationaryMarkovchaintomodelbursterrors.However,evenifthechannelgainisstationary,packeterrorprocessisanon-stationaryrandomprocess.Specically,sincePEPisafunctionofthechannelgain[ 18 ],whichisnotconstantinawirelessfadingchannel,instantaneousPEPisalsonotconstant.Thismeanstheprobability 3http://iphome.hhi.de/suehring/tml/download/old jm/jm14.0.zip 27

PAGE 28

distributionofpacketerrorstateistime-varyinginwirelessfadingchannels,thatis,thepacketerrorprocessisanon-stationaryrandomprocess.HencetheMarkovchaininRef.[ 8 ]isneitherstationary,norergodicforwirelessfadingchannel.Asaresult,averagingtheburstlengthandPEPasinRef.[ 8 ]cannotaccuratelypredictinstantaneousdistortion.Toaddressthis,thischapterderivestheformulaforPixel-levelTransmissionDistortion(PTD)byconsideringnon-stationarityovertime.RegardingtheFrame-levelTransmissionDistortion(FTD),sincetwoadjacentMBsmaybeassignedtotwodifferentpackets,undertheslice-levelpacketizationandFMOmechanisminH.264[ 19 20 ],theirerrorprobabilitycouldbedifferent.However,existingframe-leveldistortionmodels[ 8 11 ]assumeallpixelsinthesameframeexperiencethesamechannelcondition.Asaresult,theapplicablescopeofthosemodelsarelimitedtovideowithsmallresolution.Incontrast,thischapterderivestheformulaforFTDbyconsideringnon-stationarityoverspace.Duetoconsiderationofnon-stationarityovertimeandoverspace,ourformulaprovidesanaccuratepredictionoftransmissiondistortioninatime-varyingchannel. Therestofthechapterisorganizedasfollows.Section 2.2 presentsthepreliminariesofthesystemunderstudytofacilitatethederivationsinthelatersections,andillustratesthelimitationsofexistingtransmissiondistortionmodels.InSection 2.3 ,wederivethetransmissiondistortionformulaasafunctionofvideostatistics,channelcondition,andcodecsystemparameters.Section 2.4 discussestherelationshipbetweenourformulaandtheexistingmodels.InSection 2.5 ,weextendformulaeforPTDandFTDfromsingle-referencetomulti-reference. 2.2SystemDescription 2.2.1StructureofaWirelessVideoCommunicationSystem Fig. 2-1 showsthestructureofatypicalwirelessvideocommunicationsystem.Itconsistsofanencoder,twochannelsandadecoderwhereresidualpacketsandMVpacketsaretransmittedovertheirrespectivechannels.IfresidualpacketsorMVpackets 28

PAGE 29

areerroneous,theerrorconcealmentmodulewillbeactivated.IntypicalvideoencoderssuchasH.263/264andMPEG-2/4encoders,thefunctionalblockscanbedividedintotwoclasses:1)basicparts,suchaspredictivecoding,transform,quantization,entropycoding,motioncompensation,andclipping;and2)performance-enhancedparts,suchasinterpolationltering,deblockingltering,B-frame,multi-referenceprediction,etc.Althoughtheup-to-datevideoencoderincludesmoreandmoreperformance-enhancedparts,thebasicpartsdonotchange.Inthischapter,weusethestructureinFig. 2-1 fortransmissiondistortionanalysis.Notethatinthissystem,bothresidualchannelandMVchannelareapplication-layerchannels;specically,bothchannelsconsistofentropycodingandentropydecoding,networkinglayers4,andphysicallayer(includingchannelencoding,modulation,wirelessfadingchannel,demodulation,channeldecoding).AlthoughtheresidualchannelandMVchannelusuallysharethesamephysical-layerchannel,thetwoapplication-layerchannelsmayhavedifferentparametersettings(e.g.,differentchannelcode-rate)fortheslicedatapartitioningunderUEP.Forthisreason,ourformulaobtainedfromthestructureinFig. 2-1 canbeusedtoestimatetransmissiondistortionforanencoderwithslicedatapartitioning. 2.2.2ClippingNoise Inthissubsection,weexaminetheeffectofclippingnoiseonthereconstructionpixelvaluealongeachpixeltrajectoryovertime(frames).Allpixelpositionsinavideosequenceformathree-dimensionalspatio-temporaldomain,i.e.,twodimensionsinspatialdomainandonedimensionintemporaldomain.Eachpixelcanbeuniquelyrepresentedbyukinthisthree-dimensionaltime-space,wherekmeansthek-thframeintemporaldomainanduisatwo-dimensionalvectorinspatialdomain.Thephilosophybehindinter-codingofavideosequenceistorepresentthevideosequencebyvirtualmotionofeachpixel,i.e.,eachpixelrecursivelymovesfrompositionvk)]TJ /F8 7.97 Tf 6.58 0 Td[(1topositionuk. 4Here,networkinglayerscanincludeanylayersotherthanphysicallayer. 29

PAGE 30

Figure2-1. Systemstructure,whereT,Q,Q)]TJ /F8 7.97 Tf 6.59 0 Td[(1,andT)]TJ /F8 7.97 Tf 6.59 0 Td[(1denotetransform,quantization,inversequantization,andinversetransform,respectively. Thedifferencebetweenthesetwopositionsisatwo-dimensionalvectorcalledMVofpixeluk,i.e.,mvku=vk)]TJ /F8 7.97 Tf 6.58 0 Td[(1)]TJ /F15 11.955 Tf 12.61 0 Td[(uk.Thedifferencebetweenthepixelvaluesofthesetwopositionsiscalledresidualofpixeluk,thatis,eku=fku)]TJ /F3 11.955 Tf 12.23 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku5.Recursively,eachpixelinthek-thframehasoneandonlyonereferencepixeltrajectorybackwardtowardsthelatestI-frame. Attheencoder,aftertransform,quantization,inversequantization,andinversetransformfortheresidual,thereconstructedpixelvaluemaybeout-of-rangeandshouldbeclippedas ^fku=\(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+^eku),(2) 5Forsimplicityofnotation,wemovethesuperscriptkofutothesuperscriptkoffwheneveruisthesubscriptoff. 30

PAGE 31

where\()functionisaclippingfunctiondenedby \(x)=8>>>>>><>>>>>>:L,xH,(2) whereLandHareuser-speciedlowthresholdandhighthreshold,respectively.Usually,L=0andH=255. TheresidualandMVatthedecodermaybedifferentfromtheircounterpartsattheencoderbecauseofchannelimpairments.DenotefmvkuandeekutheMVandresidualatthedecoder,respectively.Then,thereferencepixelpositionforukatthedecoderisevk)]TJ /F8 7.97 Tf 6.58 0 Td[(1=uk+fmvku,andthereconstructedpixelvalueforukatthedecoderis efku=\(efk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eeku).(2) Inerror-freechannels,thereconstructedpixelvalueatthereceiverisexactlythesameasthereconstructedpixelvalueatthetransmitter,becausethereisnotransmissionerrorandhencenotransmissiondistortion.However,inerror-pronechannels,weknowfrom( 2 )thatefkuisafunctionofthreefactors:thereceivedresidualeeku,thereceivedMVfmvku,andthepropagatederrorefk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku.Thereceivedresidualeekudependsonthreefactors,namely,1)thetransmittedresidual^eku,2)theresidualpacketerrorstate,whichdependsoninstantaneousresidualchannelcondition,and3)theresidualerrorconcealmentalgorithmifthereceivedresidualpacketiserroneous.Similarly,thereceivedMVfmvkudependson1)thetransmittedmvku,2)theMVpacketerrorstate,whichdependsoninstantaneousMVchannelcondition,and3)theMVerrorconcealmentalgorithmifthereceivedMVpacketiserroneous.Thepropagatederrorefk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvkuincludestheerrorpropagatedfromthereferenceframes,andthereforedependsonallsamplesinthepreviousframesindexedbyi
PAGE 32

Table2-1. Notations uk:Three-dimensionalvectorthatdenotesapixelpositioninanvideosequencefku:Valueofthepixelukeku:Residualofthepixelukmvku:MVofthepixelukku:Clippingnoiseofthepixeluk"ku:Residualconcealmenterrorofthepixelukku:MVconcealmenterrorofthepixelukku:TransmissionreconstructederrorofthepixelukSku:ErrorstateofthepixelukPku:ErrorprobabilityofthepixelukDku:TransmissiondistortionofthepixelukDk:Transmissiondistortionofthek-thframeVk:Setofallthepixelsinthek-thframejVj:NumberofelementsinsetV(cardinalityofV)k:Propagationfactorofthek-thframek:PercentageofI-MBsinthek-thframek:Correlationratioofthek-thframewk(j):pixelpercentageofusingframek)]TJ /F4 11.955 Tf 11.95 0 Td[(jasreferenceinthek-thframe Thenon-linearclippingfunctionwithinthepixeltrajectorymakesthedistortionestimationmorechallenging.However,itisinterestingtoobservethatclippingactuallyreducestransmissiondistortion.InSection 2.3 ,wewillquantifytheeffectofclippingontransmissiondistortion. Table 2-1 listsnotationsusedinthischapter.Allvectorsareinboldfont.Notethattheencoderneedstoreconstructthecompressedvideoforpredictivecoding;hencetheencoderandthedecoderhaveasimilarstructureforpixelvaluereconstruction.Todistinguishthevariablesinthereconstructionmoduleoftheencoderfromthoseinthereconstructionmoduleofthedecoder,weadd^ontothevariablesattheencoderandaddeontothevariablesatthedecoder. 2.2.3DenitionofTransmissionDistortion Inthissubsection,wedenePTDandFTDtobederivedinSection 2.3 .TocalculateFTD,weneedsomenotationsfromsettheory.Inavideosequence,allpixelpositionsinthek-thframeformatwo-dimensionalvectorsetVk,andwedenote 32

PAGE 33

thenumberofelementsinsetVkbyjVkj.So,foranypixelatpositionuinthek-thframe,i.e.,u2Vk,itsreferencepixelpositionischosenfromsetVk)]TJ /F8 7.97 Tf 6.58 0 Td[(1forsingle-reference.Usually,thesetVkinavideosequenceisthesameforallframek,i.e.,V1==Vkforallk>1.Hence,weremovetheframeindexkanddenotethesetofpixelpositionsofanarbitraryframebyV.NotethatinH.264,areferencepixelmaybeinapositionoutofpictureboundary;however,thesetofreferencepixels,whichislargerthantheinputpixelset,isstillthesameforallframek. Foratransmitterwithfeedbackacknowledgementofwhetherapacketiscorrectlyreceivedatthereceiver(calledacknowledgementfeedback),efkuatthedecodersidecanbeperfectlyreconstructedbythetransmitter,aslongasthetransmitterknowstheerrorconcealmentalgorithmusedbythereceiver.Then,thetransmissiondistortionforthek-thframecanbecalculatedbymeansquarederror(MSE)as MSEk=1 jVjXu2V[(^fku)]TJ /F5 11.955 Tf 11.87 3.15 Td[(efku)2].(2) Fortheencoder,everypixelintensityfkuoftherandominputvideosequenceisarandomvariable.Foranyencoderwithhybridcoding(seeFig. 2-1 ),theresidual^eku,MVmvku,andreconstructedpixelvalue^fkuarefunctionsoffku;sotheyarealsorandomvariablesbeforemotionestimation6.GiventheProbabilityMassFunction(PMF)of^fkuandefku,wedenethetransmissiondistortionforpixelukorPTDby Dku,E[(^fku)]TJ /F5 11.955 Tf 11.87 3.15 Td[(efku)2],(2) andwedenethetransmissiondistortionforthek-thframeorFTDby Dk,E[1 jVjXu2V(^fku)]TJ /F5 11.955 Tf 11.87 3.16 Td[(efku)2].(2) 6Inapplicationssuchascross-layerencodingratecontrol,distortionestimationforrate-distortionoptimizedbitallocationisrequiredbeforemotionestimation. 33

PAGE 34

ItiseasytoprovethattherelationshipbetweenFTDandPTDischaracterizedby Dk=1 jVjXu2VDku.(2) Ifthenumberofbitsusedtocompressaframeistoolargetobecontainedinonepacket,thebitsoftheframearesplitintomultiplepackets.Inatime-varyingchannel,differentpacketofthesameframemayexperiencedifferentpacketerrorprobability(PEP).Ifpixelukandpixelvkbelongtodifferentpackets,thePMFofefkumaybedifferentfromthePMFofefkvevenif^fkuand^fkvareidenticallydistributed.Inotherwords,DkumaybedifferentfromDkvevenifpixelukandpixelvkareintheneighboringMBswhenFMOisactivated.Asaresult,FTDDkin( 2 )maybedifferentfromPTDDkuin( 2 ).Forthisreason,wewillderiveformulaeforbothPTDandFTD,respectively.Notethatmostexistingframe-leveldistortionmodels[ 8 11 ]assumethatallpixelsinthesameframeexperiencethesamechannelconditionandsimplyuse( 2 )forFTD;howeverthisassumptionisnotvalidforhigh-resolution/high-qualityvideotransmissionoveratime-varyingchannel. Infact,( 2 )isageneralformfordistortionsofalllevels.IfjVj=1,( 2 )reducesto( 2 ).Forslice/packet-leveldistortion,Visthesetofthepixelscontainedinaslice/packet.ForGOP-leveldistortion,VisthesetofthepixelscontainedinaGOP.Inthischapter,weonlyshowhowtoderiveformulaeforPTDandFTD.Ourmethodologyisalsoapplicabletoderivingformulaeforslice/packet/GOP-leveldistortionbyusingappropriateV. 2.2.4LimitationsoftheExistingTransmissionDistortionModels Inthissubsection,weshowthatclippingnoisehassignicantimpactontransmissiondistortion,andneglectofclippingnoiseinexistingmodelsresultsininaccurateestimationoftransmissiondistortion.Wedenetheclippingnoiseforpixelukattheencoderas ^ku,(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku+^eku))]TJ /F3 11.955 Tf 11.95 0 Td[(\(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku+^eku),(2) 34

PAGE 35

andtheclippingnoiseforpixelukatthedecoderas eku,(efk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku+eeku))]TJ /F3 11.955 Tf 11.95 0 Td[(\(efk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku+eeku).(2) Using( 2 ),Eq.( 2 )becomes ^fku=^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+^eku)]TJ /F3 11.955 Tf 13 2.65 Td[(^ku,(2) andusing( 2 ),Eq.( 2 )becomes efku=efk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eeku)]TJ /F5 11.955 Tf 12.82 3.15 Td[(eku,(2) where^kuonlydependsonthevideocontentandencoderstructure,e.g.,motionestimation,quantization,modedecisionandclippingfunction;andekudependsonnotonlythevideocontentandencoderstructure,butalsochannelconditionsanddecoderstructure,e.g.,errorconcealmentandclippingfunction. Inmostexistingworks,both^kuandekuareneglected,i.e.,theseworksassume^fku=^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+^ekuandefku=efk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eeku.However,thisassumptionisonlyvalidforstoredvideoorerror-freecommunication.Forerror-pronecommunication,decoderclippingnoiseekuhasasignicantimpactontransmissiondistortionandhenceshouldnotbeneglected.Toillustratethis,Table 2-2 showsanexampleforthesysteminFig. 2-1 ,whereonlytheresidualpacketinthe(k)]TJ /F3 11.955 Tf 12.24 0 Td[(1)-thframeiserroneousatthedecoder(i.e.,^ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1viserroneous),andallotherresidualpacketsandalltheMVpacketsareerror-free.Supposethetrajectoryofpixelukinthe(k)]TJ /F3 11.955 Tf 11.09 0 Td[(1)-thframeand(k)]TJ /F3 11.955 Tf 11.09 0 Td[(2)-thframeisspeciedbyvk)]TJ /F8 7.97 Tf 6.58 0 Td[(1=uk+mvkuandwk)]TJ /F8 7.97 Tf 6.59 0 Td[(2=vk)]TJ /F8 7.97 Tf 6.59 0 Td[(1+mvk)]TJ /F8 7.97 Tf 6.58 0 Td[(1v.Since^ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1viserroneous,thedecoderneedstoconcealtheerror;asimpleconcealmentschemeistoleteek)]TJ /F8 7.97 Tf 6.59 0 Td[(1v=0.Fromthisexample,weseethatneglectofclippingnoise(e.g.,eku=45)resultsinhighlyinaccurateestimateofdistortion,e.g.,theestimateddistortion^Dku=2500(withoutconsideringclipping)ismuchlargerthanthetruedistortionDku=25.NotethatifanMViserroneousatthedecoder,thepixeltrajectoryatthedecoderwillbedifferentfromthetrajectoryat 35

PAGE 36

Table2-2. Anexamplethatshowstheeffectofclippingnoiseontransmissiondistortion. Encoder Transmitted ^fk)]TJ /F23 6.974 Tf 6.23 0 Td[(2w=250 ^ek)]TJ /F23 6.974 Tf 6.22 0 Td[(1v=)]TJ /F19 9.963 Tf 7.75 0 Td[(50(erroneous) ^eku=50 Reconstructed ^fk)]TJ /F23 6.974 Tf 6.23 0 Td[(2w=250 ^fk)]TJ /F23 6.974 Tf 6.23 0 Td[(1v=\(^fk)]TJ /F23 6.974 Tf 6.22 0 Td[(2w+^ek)]TJ /F23 6.974 Tf 6.22 0 Td[(1v)=200 ^fku=\(^fk)]TJ /F23 6.974 Tf 6.22 0 Td[(1v+^eku)=250 Received efk)]TJ /F23 6.974 Tf 6.23 0 Td[(2w=250 eek)]TJ /F23 6.974 Tf 6.22 0 Td[(1v=0(concealed) eeku=50 Decoder Reconstructed efk)]TJ /F23 6.974 Tf 6.23 0 Td[(2w=250 efk)]TJ /F23 6.974 Tf 6.23 0 Td[(1v=\(efk)]TJ /F23 6.974 Tf 6.22 0 Td[(2w+eek)]TJ /F23 6.974 Tf 6.22 0 Td[(1v)=250 efku=\(efk)]TJ /F23 6.974 Tf 6.22 0 Td[(1v+eeku)=255 Clippingnoise ek)]TJ /F23 6.974 Tf 6.23 0 Td[(2w=0 ek)]TJ /F23 6.974 Tf 6.23 0 Td[(1v=0 eku=45 Distortion Dk)]TJ /F23 6.974 Tf 6.23 0 Td[(2w=0 Dk)]TJ /F23 6.974 Tf 6.22 0 Td[(1v=(^fk)]TJ /F23 6.974 Tf 6.23 0 Td[(1v)]TJ /F25 9.963 Tf 9.89 2.63 Td[(efk)]TJ /F23 6.974 Tf 6.22 0 Td[(1v)2=2500 Dku=(^fku)]TJ /F25 9.963 Tf 9.89 2.63 Td[(efku)2=25 Prediction Received efk)]TJ /F23 6.974 Tf 6.23 0 Td[(2w=250 eek)]TJ /F23 6.974 Tf 6.22 0 Td[(1v=0(concealed) eeku=50 without Reconstructed efk)]TJ /F23 6.974 Tf 6.23 0 Td[(2w=250 efk)]TJ /F23 6.974 Tf 6.23 0 Td[(1v=efk)]TJ /F23 6.974 Tf 6.23 0 Td[(2w+eek)]TJ /F23 6.974 Tf 6.22 0 Td[(1v=250 efku=efk)]TJ /F23 6.974 Tf 6.23 0 Td[(1v+eeku=300 clipping Distortion ^Dk)]TJ /F23 6.974 Tf 6.23 0 Td[(2w=0 ^Dk)]TJ /F23 6.974 Tf 6.22 0 Td[(1v=(^fk)]TJ /F23 6.974 Tf 6.23 0 Td[(1v)]TJ /F25 9.963 Tf 9.89 2.63 Td[(efk)]TJ /F23 6.974 Tf 6.22 0 Td[(1v)2=2500 ^Dku=(^fku)]TJ /F25 9.963 Tf 9.89 2.63 Td[(efku)2=2500 theencoder;thentheresultingclippingnoiseekumaybemuchlargerthan45asinthisexample,andhencethedistortionestimationoftheexistingmodelswithoutconsideringclippingmaybemuchmoreinaccurate. Ontheotherhand,theencoderclippingnoise^kuhasnegligibleeffectonquantizationdistortionandtransmissiondistortion.Thisisduetotworeasons:1)theprobabilitythat^ku=0,isclosetoone,sincetheprobabilitythatL^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+^ekuH,isclosetoone;2)incasethat^ku6=0,^kuusuallytakesavaluethatismuchsmallerthantheresiduals.Since^kuisnegligible,theclippingfunctioncanberemovedattheencoderifonlyquantizationdistortionneedstobeconsidered,e.g.,forstoredvideoorerror-freecommunication.Since^kuisverylikelytobeaverysmallvalue,wewouldneglectitandassume^ku=0inderivingourformulafortransmissiondistortion. 2.3TransmissionDistortionFormulae Inthissection,wederiveformulaeforPTDandFTD.Thesectionisorganizedasbelow.Section 2.3.1 presentsanoverviewofourapproachtoanalyzingPTDandFTD.ThenweelaborateonthederivationdetailsinSection 2.3.2 throughSection 2.3.5 .Specically,Section 2.3.2 quantiestheeffectofRCEontransmissiondistortion;Section 2.3.3 quantiestheeffectofMVCEontransmissiondistortion;Section 2.3.4 quantiestheeffectofpropagatederrorandclippingnoiseontransmissiondistortion; 36

PAGE 37

Section 2.3.5 quantiestheeffectofcorrelations(betweenanytwooftheerrorsources)ontransmissiondistortion.Finally,Section 2.3.6 summarizesthekeyresultsofthischapter,i.e.,theformulaeforPTDandFTD. 2.3.1OverviewoftheApproachtoAnalyzingPTDandFTD ToanalyzePTDandFTD,wetakeadivide-and-conquerapproach.Werstdividetransmissionreconstructederrorintofourcomponents:threeindependentrandomerrors(RCE,MVCEandpropagatederror)basedontheirexplicitlydifferentrootcauses,andclippingnoise,whichisanon-linearfunctionofthosethreerandomerrors.Thiserrordecompositionallowsustofurtherdecomposetransmissiondistortionintofourterms,i.e.,distortioncausedby1)RCE,2)MVCE,3)propagatederrorplusclippingnoise,and4)correlationsbetweenanytwooftheerrorsources,respectively.Thisdistortiondecompositionfacilitatesthederivationofasimpleandaccurateclosed-formformulaforeachofthefourdistortionterms.Next,weelaborateonerrordecompositionanddistortiondecomposition. Denetransmissionreconstructederrorforpixelukbyeku,^fku)]TJ /F5 11.955 Tf 12.31 3.15 Td[(efku.From( 2 )and( 2 ),weobtain eku=(^eku+^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)]TJ /F3 11.955 Tf 13 2.66 Td[(^ku))]TJ /F3 11.955 Tf 11.95 0 Td[((eeku+efk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku)]TJ /F5 11.955 Tf 12.82 3.16 Td[(eku)=(^eku)]TJ /F5 11.955 Tf 11.81 .5 Td[(eeku)+(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)]TJ /F3 11.955 Tf 12.05 2.65 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku)+(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku)]TJ /F5 11.955 Tf 11.87 3.15 Td[(efk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku))]TJ /F3 11.955 Tf 11.95 0 Td[((^ku)]TJ /F5 11.955 Tf 12.81 3.15 Td[(eku).(2) DeneRCEe"kubye"ku,^eku)]TJ /F5 11.955 Tf 12.71 .5 Td[(eeku,anddeneMVCEekubyeku,^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)]TJ /F3 11.955 Tf 12.96 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku.Notethat^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku)]TJ /F5 11.955 Tf 12.26 3.16 Td[(efk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku=ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku,whichisthetransmissionreconstructederroroftheconcealedreferencepixelinthereferenceframe;wecallek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvkupropagatederror.AsmentionedinSection 2.2.4 ,weassume^ku=0.Therefore,( 2 )becomes eku=e"ku+eku+ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku+eku.(2) ( 2 )isourproposederrordecomposition. 37

PAGE 38

Combining( 2 )and( 2 ),wehave Dku=E[(e"ku+eku+ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eku)2]=E[(e"ku)2]+E[(eku)2]+E[(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku+eku)2]+2E[e"kueku]+2E[e"ku(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eku)]+2E[eku(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku+eku)].(2) DenoteDku(r),E[(e"ku)2],Dku(m),E[(eku)2],Dku(P),E[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eku)2]andDku(c),2E[e"kueku]+2E[e"ku(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eku)]+2E[eku(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eku)].Then,( 2 )becomes Dku=Dku(r)+Dku(m)+Dku(P)+Dku(c).(2)( 3 )isourproposeddistortiondecompositionforPTD.Thereasonwhywecombinepropagatederrorandclippingnoiseintooneterm(calledclippedpropagatederror)isbecauseclippingnoiseismainlycausedbypropagatederrorandsuchdecompositionwillsimplifytheformulae. Therearethreemajorreasonsforourdecompositionsin( 2 )and( 3 ).First,ifwedirectlysubstitutethetermsin( 2 )by( 2 )and( 2 ),itwillproduce5secondmomentsand10cross-correlationterms(assuming^ku=0);sincethereare8possibleerroreventsduetothreeindependentrandomerrors,thereareatotalof8(5+10)=120termsforPTD,makingtheanalysishighlycomplicated.Incontrast,ourdecompositionsin( 2 )and( 3 )signicantlysimplifytheanalysis.Second,eachtermin( 2 )and( 3 )hasaclearphysicalmeaning,andthereforecanbeaccuratelyestimatedwithlowcomplexity.Third,suchdecompositionsallowourformulaetobeeasilyextendedforsupportingadvancedvideocodecwithmoreperformance-enhancedparts,e.g.,multi-referencepredictionandinterpolationltering. ToderivetheformulaforFTD,from( 2 )and( 3 ),weobtain Dk=Dk(r)+Dk(m)+Dk(P)+Dk(c),(2) 38

PAGE 39

where Dk(r)=1 jVjXu2VDku(r),(2) Dk(m)=1 jVjXu2VDku(m),(2) Dk(P)=1 jVjXu2VDku(P),(2) Dk(c)=1 jVjXu2VDku(c).(2)( 2 )isourproposeddistortiondecompositionforFTD. Next,wepresentthederivationofaclosed-formformulaforeachofthefourdistortiontermsinSection 2.3.2 throughSection 2.3.5 2.3.2AnalysisofDistortionCausedbyRCE Inthissubsection,werstderivethepixel-levelresidualcauseddistortionDku(r).Thenwederivetheframe-levelresidualcauseddistortionDk(r). 2.3.2.1Pixel-leveldistortioncausedbyRCE WedenoteSkuasthestateindicatorofwhetherthereistransmissionerrorforpixelukafterchanneldecoding.NotethatasmentionedinSection 2.2.1 ,boththeresidualchannelandtheMVchannelcontainchanneldecoding;henceinthischapter,thetransmissionerrorintheresidualchannelortheMVchannelismeanttobetheerroruncorrectablebythechanneldecoding.TodistinguishtheresidualerrorstateandtheMVerrorstate,hereweuseSku(r)todenotetheresidualerrorstateforpixeluk.Thatis,Sku(r)=1if^ekuisreceivedwitherror,andSku(r)=0if^ekuisreceivedwithouterror.Atthereceiver,ifthereisnoresidualtransmissionerrorforpixelu,eekuisequalto^eku.However,iftheresidualpacketsarereceivedwitherror,weneedtoconcealtheresidualerroratthereceiver.DenoteekutheconcealedresidualwhenSku(r)=1,andwehave, eeku=8>><>>:eku,Sku(r)=1^eku,Sku(r)=0.(2) 39

PAGE 40

Notethatekudependson^ekuandtheresidualconcealmentmethod,butdoesnotdependonthechannelcondition.Fromthedenitionofe"kuand( 2 ),wehave e"ku=(^eku)]TJ /F3 11.955 Tf 11.99 0 Td[(eku)Sku(r)+(^eku)]TJ /F3 11.955 Tf 12 0 Td[(^eku)(1)]TJ /F4 11.955 Tf 11.95 0 Td[(Sku(r))=(^eku)]TJ /F3 11.955 Tf 11.99 0 Td[(eku)Sku(r).(2) ^ekudependsontheinputvideosequenceandtheencoderstructure,whileSku(r)dependsoncommunicationsystemparameterssuchasdelaybound,channelcodingrate,transmissionpower,channelgainofthewirelesschannel.UnderourframeworkshowninFig. 2-1 ,theinputvideosequenceandtheencoderstructureareindependentofcommunicationsystemparameters.Since^ekuandSku(r)aresolelycausedbyindependentsources,weassume^ekuandSku(r)areindependent.Thatis,wemakethefollowingassumption. Assumption1. Sku(r)isindependentof^eku. Assumption 1 meansthatwhether^ekuwillbecorrectlyreceivedornot,doesnotdependonthevalueof^eku.Denote"ku,^eku)]TJ /F3 11.955 Tf 12.2 0 Td[(eku;wehavee"ku="kuSku(r).DenotePku(r)astheresidualpixelerrorprobability(XEP)forpixeluk,thatis,Pku(r),PfSku(r)=1g.Then,from( 2 )andAssumption 1 ,wehave Dku(r)=E[(e"ku)2]=E[("ku)2]E[(Sku(r))2]=E[("ku)2](1Pku(r))=E[("ku)2]Pku(r).(2) Hence,ourformulaforthepixel-levelresidualcauseddistortionis Dku(r)=E[("ku)2]Pku(r).(2) 2.3.2.2Frame-leveldistortioncausedbyRCE Toderivetheframe-levelresidualcauseddistortion,theencoderneedstoknowthesecondmomentofRCEforeachpixelinthatframe.However,ifencoderknowsthecharacteristicsofresidualprocessandconcealmentmethod,theformulaewillbemuchsimplied.Onesimpleconcealmentmethodistoleteku=0forallerroneouspixels. 40

PAGE 41

Amoregeneralconcealmentmethodistousetheneighboringpixelstoconcealanerroneouspixel.Sowemakethefollowingassumption. Assumption2. Theresidual^ekuisstationarywithrespectto(w.r.t.)2Dvariableuinthesameframe.Inaddition,ekuonlydependsonf^ekv:v2NugwhereNuisaxedneighborhoodofu. Inotherwords,Assumption 2 assumesthat1)^ekuisa2Dstationarystochasticprocessandthedistributionof^ekuisthesameforallu2Vk,and2)ekuisalsoa2Dstationarystochasticprocesssinceitonlydependsontheneighboring^eku.Hence,^eku)]TJ /F3 11.955 Tf 10.12 0 Td[(ekuisalsoa2Dstationarystochasticprocess,anditssecondmomentE[(^eku)]TJ /F3 11.955 Tf 10.12 0 Td[(eku)2]=E[("ku)2]isthesameforallu2Vk.Therefore,wecandropufromthenotation,andletE[("k)2]=E[("ku)2]forallu2Vk. DenoteNki(r)asthenumberofpixelscontainedinthei-thresidualpacketofthek-thframe;denotePki(r)asPEPofthei-thresidualpacketofthek-thframe;denoteNk(r)asthetotalnumberofresidualpacketsofthek-thframe.Sinceforallpixelsinthesamepacket,theresidualXEPisequaltoitsPEP,from( 2 )and( 3 ),wehave Dk(r)=1 jVjXu2VkE[("ku)2]Pku(r) (2) =1 jVjXu2VkE[("k)2]Pku(r) (2) (a)=E[("k)2] jVjNk(r)Xi=1(Pki(r)Nki(r)) (2) (b)=E[("k)2]Pk(r). (2) where(a)isduetoPku(r)=Pki(r)forpixeluinthei-thresidualpacket;(b)isdueto Pk(r),1 jVjNk(r)Xi=1(Pki(r)Nki(r)). (2) Pk(r)isaweightedaverageoverPEPsofallresidualpacketsinthek-thframe,inwhichdifferentpacketsmaycontaindifferentnumbersofpixels.Hence,ourformulaforthe 41

PAGE 42

frame-levelresidualcauseddistortionis Dk(r)=E[("k)2]Pk(r).(2) 2.3.3AnalysisofDistortionCausedbyMVCE SimilartothederivationsinSection 2.3.2.1 ,inthissubsection,wederivetheformulaforthepixel-levelMVcauseddistortionDku(m),andtheframe-levelMVcauseddistortionDk(m). 2.3.3.1Pixel-leveldistortioncausedbyMVCE DenotetheMVerrorstateforpixelukbySku(m),anddenotetheconcealedMVbymvkuwhenSku(m)=1.Therefore,wehave fmvku=8>><>>:mvku,Sku(m)=1mvku,Sku(m)=0.(2) Here,weusethetemporalerrorconcealment[ 21 ]toconcealMVerrors.Denoteku,^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku)]TJ /F3 11.955 Tf 12.64 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku;wehaveeku=kuSku(m),wherekudependsontheaccuracyofMVconcealment,andthespatialcorrelationbetweenreferencepixelandconcealedreferencepixelattheencoder.DenotePku(m)astheMVXEPforpixeluk,thatis,Pku(m),PfSku(m)=1g.Wemakethefollowingassumption. Assumption3. Sku(m)isindependentofku. FollowingthesamederivingprocessinSection 2.3.2.1 ,wecanobtain Dku(m)=E[(ku)2]Pku(m).(2) NotethatkudependsonmvkuandtheMVconcealmentmethod,butdoesnotdependonthechannelcondition.Inmostcases,giventheconcealmentmethod,thestatisticsofkucanbeeasilyobtainedattheencoder.Fromtheexperiments,weobservethatkufollowsazero-meanLaplaciandistribution. 42

PAGE 43

NotethatinH.264specication,thereisnoslicedatapartitioningforaninstantaneousdecodingrefresh(IDR)frame[ 22 ];soSku(r)andSku(m)arefullycorrelatedinanIDR-frame,thatis,Sku(r)=Sku(m),andhencePku(r)=Pku(m).ThisisalsotrueforI-MB,andP-MBwithoutslicedatapartitioning.ForP-MBwithslicedatapartitioninginH.264,Sku(r)andSku(m)arepartiallycorrelated.Inotherwords,ifthepacketofslicedatapartitionA,whichcontainsMVinformation,islost,thecorrespondingpacketofslicedatapartitionB,whichcontainsresidualinformation,cannotbedecodedevenifitiscorrectlyreceived,sincethereisnosliceheaderintheslicedatapartitionB.Therefore,theresidualchannelandtheMVchannelinFig. 2-1 areactuallycorrelatediftheencoderfollowsH.264specication.Inthischapter,westudytransmissiondistortioninamoregeneralcasewhereSku(r)andSku(m)canbeeitherindependentorcorrelated.7 2.3.3.2Frame-leveldistortioncausedbyMVCE Toderivetheframe-levelMVcauseddistortion,wemakethefollowingassumption. Assumption4. Thesecondmomentofkuisthesameforallu2Vk. UnderAssumption 4 ,wecandropufromthenotation,andletE[(k)2]=E[(ku)2]forallu2Vk.DenoteNki(m)asthenumberofpixelscontainedinthei-thMVpacketofthek-thframe;denotePki(m)asPEPofthei-thMVpacketofthek-thframe;denoteNk(m)asthetotalnumberofMVpacketsofthek-thframe.FollowingthesamederivationprocessinSection 2.3.2.2 ,weobtaintheframe-levelMVcauseddistortionforthek-thframeasbelow Dk(m)=E[(k)2]Pk(m),(2) 7Toachievethis,wechangetheH.264referencecodeJM14.0byallowingresidualpacketstobeusedfordecoderwithoutthecorrespondingMVpacketsbeingcorrectlyreceived,thatis,^ekucanbeusedtoreconstructefkuevenifmvkuisnotcorrectlyreceived. 43

PAGE 44

wherePk(m),1 jVjPNkmi=1(Pki(m)Nki(m)),aweightedaverageoverPEPsofallMVpacketsinthek-thframe,inwhichdifferentpacketsmaycontaindifferentnumbersofpixels. 2.3.4AnalysisofDistortionCausedbyPropagatedErrorPlusClippingNoise Inthissubsection,wederivethedistortioncausedbyerrorpropagationinanon-lineardecoderwithclipping.Werstderivethepixel-levelpropagationandclippingcauseddistortionDku(P).Thenwederivetheframe-levelpropagationandclippingcauseddistortionDk(P). 2.3.4.1Pixel-leveldistortioncausedbypropagatederrorplusclippingnoise First,weanalyzethepixel-levelpropagationandclippingcauseddistortionDku(P)inP-MBs.Dku(P)dependsonpropagatederrorandclippingnoise;andclippingnoisedependsonpropagatederror,RCE,andMVCE.Hence,Dku(P)dependsonpropagatederror,RCE,andMVCE.Letr,m,pdenotetheeventofoccurrenceofRCE,MVconcealmenterrorandpropagatederror,respectively,andletr,m,pdenotelogicalNOTofr,m,prespectively(indicatingnoerror).Weuseatriplettodenotethejointeventofthreetypesoferror;e.g.,fr,m,pgdenotestheeventthatallthethreetypesoferrorsoccur,andukfr,m,pgdenotesthepixelukexperiencingnoneofthethreetypesoferrors. Whenweanalyzetheconditionthatseveralerroreventsmayoccur,thenotationcouldbesimpliedbytheprincipleofformallogic.Forexample,ekufr,mgdenotestheclippingnoiseundertheconditionthatthereisneitherRCEnorMVCEforpixeluk,whileitisnotcertainwhetherthereferencepixelhaserror.Correspondingly,denotePkufr,mgastheprobabilityofeventfr,mg,thatis,Pkufr,mg=PfSku(r)=0andSku(m)=0g.FromthedenitionofPku(r),themarginalprobabilityPkufrg=Pku(r)andthemarginalprobabilityPkufrg=1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pku(r).Thesame,Pkufmg=Pku(m)andPkufmg=1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pku(m). DeneDku(p),E[(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku+ekufr,mg)2];anddeneku,Dku(p) Dk)]TJ /F11 5.978 Tf 5.75 0 Td[(1u+mvku,whichiscalledpropagationfactorforpixeluk.Thepropagationfactorkudenedinthischapteris 44

PAGE 45

differentfromthepropagationfactor[ 11 ],leakage[ 8 ],orattenuationfactor[ 16 ],whicharemodeledastheeffectofspatiallteringorintraupdate;ourpropagationfactorkuisalsodifferentfromthefadingfactor[ 9 ],whichismodeledastheeffectofusingfractionofreferencedpixelsinthereferenceframeformotionprediction.NotethatDku(p)isonlyaspecialcaseofDku(P)undertheerroreventoffr,mgforpixeluk.However,mostexistingmodelsinappropriatelyusetheirpropagationfactor,obtainedundertheerroreventoffr,mg,toreplaceDku(P)ofallothererroreventsdirectlywithoutdistinguishingtheirdifference. TocalculateE[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eku)2]in( 2 ),weneedtoanalyzeekuinfourdifferenterroreventsforpixeluk:1)bothresidualandMVareerroneous,denotedbyukfr,mg;2)residualiserroneousbutMViscorrect,denotedbyukfr,mg;3)residualiscorrectbutMViserroneous,denotedbyukfr,mg;and4)bothresidualandMVarecorrect,denotedbyukfr,mg.So, Dku(P)=Pkufr,mgE[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+ekufr,mg)2]+Pkufr,mgE[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+ekufr,mg)2]+Pkufr,mgE[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+ekufr,mg)2]+Pkufr,mgE[(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku+ekufr,mg)2].(2) Notethattheconcealedpixelvalueshouldbeintheclippingfunctionrange,thatis,\(efk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku+eku)=efk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku+eku,soekufrg=0.AlsonotethatiftheMVchannelisindependentoftheresidualchannel,wehavePkufr,mg=Pku(r)Pku(m).However,asmentionedinSection 2.3.3.1 ,inH.264specication,thesetwochannelsarecorrelated.Inotherwords,Pkufr,mg=0andPkufr,mg=PkufrgforP-MBswithslicedatapartitioninginH.264.Insuchacase,( 2 )issimpliedto Dku(P)=Pkufr,mgDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+Pkufr,mgDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+PkufrgDku(p).(2) Inamoregeneralcase,wherePkufr,mg6=0,Eq.( 2 )isstillvalid.ThisisbecausePkufr,mg6=0onlyhappensunderslicedatapartitioningcondition,where 45

PAGE 46

Pkufr,mgPkufr,mgandE[(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku+ekufr,mg)2]E[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+ekufr,mg)2]underUEP.Therefore,thelasttwotermsin( 2 )isalmostequaltoPkufrgDku(p). NotethatforP-MBwithoutslicedatapartitioninginH.264,wehavePkufr,mg=Pkufr,mg=0,Pkufr,mg=Pkufrg=Pkufmg=Pku,andPkufr,mg=Pkufrg=Pkufmg=1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pku.Therefore,( 2 )canbefurthersimpliedto Dku(P)=PkuDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+(1)]TJ /F4 11.955 Tf 11.95 0 Td[(Pku)Dku(p).(2) AlsonotethatforI-MB,therewillbenotransmissiondistortionifitiscorrectlyreceived,thatis,Dku(p)=0.So( 3 )canbefurthersimpliedto Dku(P)=PkuDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku.(2) Comparing( 2 )with( 3 ),weseethatI-MBisaspecialcaseofP-MBwithDku(p)=0,thatis,thepropagationfactorku=0accordingtothedenition.ItisimportanttonotethatDku(P)>0forI-MB.Inotherwords,I-MBalsocontainsthedistortioncausedbypropagationerrorsincePku6=0.However,existingLTImodels[ 8 9 ]assumethatthereisnodistortioncausedbypropagationerrorforI-MB,whichunder-estimatesthetransmissiondistortion. Inthefollowingpartofthissubsection,wederivethepropagationfactorkuforP-MBandprovesomeimportantpropertiesofclippingnoise.Toderiveku,werstgiveLemma 1 asbelow. Lemma1. GiventhePMFoftherandomvariableek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkuandthevalueof^fku,Dku(p)canbecalculatedattheencoderbyDku(p)=E[2(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku,^fku)],where(x,y)iscallederrorreductionfunctionanddenedby (x,y),y)]TJ /F3 11.955 Tf 11.95 0 Td[(\(y)]TJ /F4 11.955 Tf 11.96 0 Td[(x)=8>>>>>><>>>>>>:y)]TJ /F6 11.955 Tf 11.95 0 Td[(L,y)]TJ /F4 11.955 Tf 11.96 0 Td[(xH.(2) 46

PAGE 47

Lemma 1 isprovedinAppendix A.1 .Infact,wehavefoundinourexperimentsthatinanyerrorevent,ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuapproximatelyfollowsLaplaciandistributionwithzeromean.Ifweassumeek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkufollowsLaplaciandistributionwithzeromean,thecalculationforDku(p)becomessimplersincetheonlyunknownparameterforPMFofek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkuisitsvariance.Underthisassumption,wehavethefollowingproposition. Proposition1. ThepropagationfactorforpropagatederrorwithLaplaciandistributionofzero-meanandvariance2isgivenby =1)]TJ /F3 11.955 Tf 13.15 8.09 Td[(1 2e)]TJ /F10 5.978 Tf 7.78 4.62 Td[(y)]TJ /F27 5.978 Tf 5.76 0 Td[(L b(y)]TJ /F6 11.955 Tf 11.96 0 Td[(L b+1))]TJ /F3 11.955 Tf 13.15 8.09 Td[(1 2e)]TJ /F27 5.978 Tf 7.78 4.62 Td[(H)]TJ /F10 5.978 Tf 5.76 0 Td[(y b(H)]TJ /F4 11.955 Tf 11.96 0 Td[(y b+1),(2) whereyisthereconstructedpixelvalue,andb=p 2 2. Proposition 1 isprovedinAppendix A.2 .Inthezero-meanLaplaciancase,kuwillonlybeafunctionof^fkuandthevarianceofek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku,whichisequaltoDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuinthiscase.SinceDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuhasalreadybeencalculatedduringthephaseofpredictingthe(k)]TJ /F3 11.955 Tf 12.26 0 Td[(1)-thframetransmissiondistortion,Dku(p)canbecalculatedbyDku(p)=kuDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuviathedenitionofku.ThenwecanrecursivelycalculateDku(P)in( 2 )sincebothDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuandDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkuhavebeencalculatedpreviouslyforthe(k)]TJ /F3 11.955 Tf 12.98 0 Td[(1)-thframe.( 3 )isveryimportantfordesigningalowcomplexityalgorithmtoestimatepropagationandclippingcauseddistortioninFTD,whichwillbepresentedinChapter 3 Next,weproveanimportantpropertyofthenon-linearclippingfunctioninthefollowingproposition. Proposition2. Clippingreducespropagatederror,thatis,Dku(p)Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku,orku1. Proof. First,fromLemma 5 ,whichispresentedandprovedinAppendix A.6 ,wehave2(x,y)x2foranyLyH.Inotherwords,thefunction(x,y)reducestheenergyofpropagatederror.Thisisthereasonwhywecalliterrorreductionfunction.WithLemma 1 ,itisstraightforwardtoprovethatwhateverthePMFofek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuis,E[2(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku,^fku)]E[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)2],thatis,Dku(p)Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku,whichisequivalenttoku1. 47

PAGE 48

Figure2-2. Theeffectofclippingnoiseondistortionpropagation. Proposition 2 tellsusthatifthereisnonewlyinducederrorsinthek-thframe,transmissiondistortiondecreasesfromthe(k)]TJ /F3 11.955 Tf 12.37 0 Td[(1)-thframetothek-thframe.Fig. 2-2 showstheexperimentalresultoftransmissiondistortionpropagationforbus cif.yuv,wheretransmissionerrorsonlyoccurinthethirdframe. Infact,ifweconsiderthemoregeneralcaseswheretheremaybenewerrorinducedinthek-thframe,wecanstillprovethatE[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eku)2]E[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku)2]usingtheproofforthefollowingcorollary. Corollary1. Thecorrelationcoefcientbetweenek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvkuandekuisnon-positive.Specif-ically,theyarenegativelycorrelatedundertheconditionfr,pg,anduncorrelatedunderotherconditions. Corollary 1 isprovedinAppendix A.8 .ThispropertyisveryimportantfordesigningalowcomplexityalgorithmtoestimatepropagationandclippingcauseddistortioninPTD,whichwillbepresentedinChapter 3 2.3.4.2Frame-leveldistortioncausedbypropagatederrorplusclippingnoise In( 2 ),Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku6=Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuduetothenon-stationarityoftheerrorprocessoverspace.However,boththesumofDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkuoverallpixelsinthe(k)]TJ /F3 11.955 Tf 12.92 0 Td[(1)-thframeand 48

PAGE 49

thesumofDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuoverallpixelsinthe(k)]TJ /F3 11.955 Tf 12.66 0 Td[(1)-thframewillconvergetoDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1duetotherandomnessofMV.Theformulaforframe-levelpropagationandclippingcauseddistortionisgiveninLemma 2 Lemma2. Theframe-levelpropagationandclippingcauseddistortioninthek-thframeis Dk(P)=Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1Pk(r)+Dk(p)(1)]TJ /F3 11.955 Tf 13.24 2.65 Td[(Pk(r))(1)]TJ /F6 11.955 Tf 11.95 0 Td[(k),(2) whereDk(p),1 jVjPu2VkDku(p)andPk(r)isdenedin( 2 );kisthepercentageofI-MBsinthek-thframe;Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1isthetransmissiondistortioninthe(k)]TJ /F3 11.955 Tf 11.96 0 Td[(1)-thframe. Lemma 2 isprovedinAppendix A.3 .Denethepropagationfactorforthek-thframek,Dk(p) Dk)]TJ /F11 5.978 Tf 5.75 0 Td[(1;thenwehavek=Pu2VkkuDk)]TJ /F11 5.978 Tf 5.76 0 Td[(1u+mvku Dk)]TJ /F11 5.978 Tf 5.76 0 Td[(1.NotethatDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkumaybedifferentfordifferentpixelsinthe(k)]TJ /F3 11.955 Tf 12.62 0 Td[(1)-thframeduetothenon-stationarityoferrorprocessoverspace.However,whenthenumberofpixelsinthe(k)]TJ /F3 11.955 Tf 12.47 0 Td[(1)-thframeissufcientlylarge,thesumofDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkuoverallthepixelsinthe(k)]TJ /F3 11.955 Tf 12.13 0 Td[(1)-thframewillconvergetoDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1.Therefore,wehavek=Pu2VkkuDk)]TJ /F11 5.978 Tf 5.75 0 Td[(1u+mvku Pu2VkDk)]TJ /F11 5.978 Tf 5.76 0 Td[(1u+mvku,whichisaweightedaverageofkuwiththeweightbeingDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku.Asaresult,Dk(p)Dk(P).Whenthenumberofpixelsinthe(k-1)-thframeissmall,Pu2VkkuDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkumaybelargerthanDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1althoughitsprobabilityissmallasobservedinourexperiments.However,mostexistingworksdirectlyuseDk(P)=Dk(p)inpredictingtransmissiondistortion.ThisisanotherreasonwhyLTImodels[ 8 9 ]under-estimatetransmissiondistortionwhenthereisnoMVerror.DetailswillbediscussedinSection 2.4.2 FromProposition 1 ,weknowthatkuisafunctionof^fku.So,kdependsonallsamplesof^fkuinthek-thframe.Sincethesamplesof^fkuusuallychangeoverframesduetothevideocontentvariation,thepropagationfactorkalsovariesfromframetoframeasobservedintheexperiments.Accuratelyestimatingkforeachframeisveryimportantforinstantaneousdistortionestimation.However,existingmodelsassume 49

PAGE 50

propagationfactorisconstantoverallframes,whichmakesthedistortionestimationinaccurate.WewilldiscusshowtoaccuratelyestimatekinrealtimeinChapter 3 2.3.5AnalysisofCorrelationCausedDistortion Inthissubsection,werstderivethepixel-levelcorrelationcauseddistortionDku(c).Thenwederivetheframe-levelcorrelationcauseddistortionDk(c). 2.3.5.1Pixel-levelcorrelationcauseddistortion WeanalyzethecorrelationcauseddistortionDku(c)atthedecoderinfourdifferentcases:i)forukfr,mg,bothe"ku=0andeku=0,soDku(c)=0;ii)forukfr,mg,eku=0andDku(c)=2E["ku(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku+ekufr,mg)];iii)forukfr,mg,e"ku=0andDku(c)=2E[ku(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+ekufr,mg)];iv)forukfr,mg,Dku(c)=2E["kuku]+2E["ku(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+ekufr,mg)]+2E[ku(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku+ekufr,mg)].FromSection 2.3.4.1 ,weknowekufrg=0.So,weobtain Dku(c)=Pkufr,mg2E["kuek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]+Pkufr,mg2E[ku(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku+ekufr,mg)]+Pkufr,mg(2E["kuku]+2E["kuek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]+2E[kuek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]).(2) Intheexperiments,wendthatinthetrajectoryofpixeluk,1)theresidual^ekuisapproximatelyuncorrelatedwiththeresidualinallotherframes^eiv,wherei6=k,asshowninFig. 2-3 ;and2)theresidual^ekuisapproximatelyuncorrelatedwiththeMVCEofthecorrespondingpixelkuandtheMVCEinallpreviousframesiv,wherei
PAGE 51

Figure2-3. Temporalcorrelationbetweentheresidualsinonetrajectory. Figure2-4. TemporalcorrelationmatrixbetweenresidualandMVCEinonetrajectory. Assumption 5 .Thus,( 2 )becomes Dku(c)=2PkufmgE[kuek)]TJ /F8 7.97 Tf 6.59 .01 Td[(1u+mvku]+2Pkufr,mgE[kuekufr,mg].(2) However,weobservethatinthetrajectoryofpixeluk,1)theresidual^ekuiscorrelatedwiththeMVCEiv,wherei>k,asseeninFig. 2-4 ;and2)theMVCEkuishighlycorrelatedwiththeMVCEivasshowninFig. 2-5 .Thisinterestingphenomenoncouldbeexploitedbyanerrorconcealmentalgorithmandissubjecttoourfuturestudy. 51

PAGE 52

Figure2-5. TemporalcorrelationmatrixbetweenMVCEsinonetrajectory. AsmentionedinSection 2.3.4.1 ,forP-MBswithslicedatapartitioninginH.264,Pkufr,mg=0.So,( 2 )becomes Dku(c)=2PkufmgE[ku(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku)]TJ /F5 11.955 Tf 11.87 3.16 Td[(efk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)].(2) NotethatinthemoregeneralcasethatPkufr,mg6=0,Eq.( 2 )isstillvalidsincekuisalmostuncorrelatedwithekufr,mgasobservedintheexperiment. ForI-MBsorP-MBswithoutslicedatapartitioninginH.264,sincePkufr,mg=Pkufr,mg=0andPkufr,mg=Pkufrg=Pkufmg=PkuasmentionedinSection 2.3.4.1 ,( 2 )canbesimpliedto Dku(c)=2Pku(2E["kuku]+2E["kuek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]+2E[kuek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku]).(2) UnderAssumption 5 ,( 3 )reducesto( 2 ). Deneku,E[kuefk)]TJ /F11 5.978 Tf 5.75 0 Td[(1u+mvku] E[ku^fk)]TJ /F11 5.978 Tf 5.75 0 Td[(1u+mvku];kuisacorrelationratio,thatis,theratioofthecorrelationbetweenMVCEandconcealedreferencepixelvalueatthereceiver,tothecorrelationbetweenMVCEandconcealedreferencepixelvalueatthetransmitter.kuquantiestheeffectofthecorrelationbetweentheMVCEandpropagatederrorontransmission 52

PAGE 53

Figure2-6. Comparisonbetweenmeasuredandestimatedcorrelationcoefcients. distortion.SincekuisastablestatisticsofMV,estimatingkuismuchsimplerandmoreaccuratethanestimatingE[kuefk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]directly,therebyresultinginmoreaccuratedistortionestimate.ThedetailsonhowtoestimatekuwillbepresentedinChapter 3 Althoughwedonotknowtheexactvalueofkuattheencoder,itsrangeis k)]TJ /F8 7.97 Tf 6.59 0 Td[(1Yi=1PiT(i)fr,mgku1,(2) whereT(i)isthepixelpositionofthei-thframeinthetrajectory,forexample,T(k)]TJ /F3 11.955 Tf 11.81 0 Td[(1)=uk+mvkuandT(k)]TJ /F3 11.955 Tf 11.7 0 Td[(2)=vk)]TJ /F8 7.97 Tf 6.58 0 Td[(1+mvk)]TJ /F8 7.97 Tf 6.59 0 Td[(1v.Theleftinequalityin( 2 )holdsintheextremecasethatanyerrorinthetrajectorywillcausekuandefk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkutobeuncorrelated,whichisusuallytrueforhighmotionvideo.Therightinequalityin( 2 )holdsinanotherextremecasethatallerrorsinthetrajectorydonotaffectthecorrelationbetweenkuandefk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku,whichisusuallytrueforlowmotionvideo. Usingthedenitionofku,( 2 )becomes Dku(c)=2Pkufmg(1)]TJ /F6 11.955 Tf 11.96 0 Td[(ku)E[ku^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku].(2) Inourexperiments,weobserveaninterestingphenomenonthatkuisalwayspositivelycorrelatedwith^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku,andnegativelycorrelatedwith^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku.Thisistheoretically 53

PAGE 54

provedinLemma 3 underAssumption 6 ;andthisisalsoveriedbyourexperimentsasshowninFig. 2-6 Assumption6. E[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)2]=E[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)2]. Assumption 6 isvalidundertheconditionthatthedistancebetweenmvkuandmvkuissmall;thisisalsoveriedbyourexperiments. Lemma3. UnderAssumption 6 ,E[ku^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku]=)]TJ /F7 7.97 Tf 10.49 5.69 Td[(E[(ku)2] 2andE[ku^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]=E[(ku)2] 2. Lemma 3 isprovedinAppendix A.4 .UnderAssumption 6 ,usingLemma 3 ,wefurthersimplify( 2 )asbelow. Dku(c)=(ku)]TJ /F3 11.955 Tf 11.95 0 Td[(1)E[(ku)2]Pku(m).(2) From( 3 ),weknowthatE[(ku)2]Pku(m)isexactlyequaltoDku(m).Therefore,( 2 )isfurthersimpliedto Dku(c)=(ku)]TJ /F3 11.955 Tf 11.95 0 Td[(1)Dku(m).(2) AsmentionedinSection 2.3.3.1 ,weobservethatkufollowsazero-meanLaplaciandistributionintheexperiment.Denotethecorrelationcoefcientbetweenkuand^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku.IfweassumeE[ku]=0,wehave=E[ku^fk)]TJ /F11 5.978 Tf 5.76 0 Td[(1u+mvku])]TJ /F7 7.97 Tf 6.59 0 Td[(E[ku]E[^fk)]TJ /F11 5.978 Tf 5.76 0 Td[(1u+mvku] ku^fku=)]TJ /F28 7.97 Tf 12.61 9.1 Td[(ku 2^fku.Similarly,itiseasytoprovethatthecorrelationcoefcientbetweenkuand^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkuisku 2^fku.ThisagreeswellwiththeexperimentalresultsshowninFig. 2-6 .Viathesamederivationprocess,onecanobtainthecorrelationcoefcientbetween^ekuand^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku,andbetween^ekuand^fku.Onepossibleapplicationofthesecorrelationpropertiesiserrorconcealmentwithpartialinformationavailable. 54

PAGE 55

2.3.5.2Frame-Levelcorrelationcauseddistortion DenoteVki(m)thesetofpixelsinthei-thMVpacketofthek-thframe.From( 2 ),( 2 )andAssumption 4 ,weobtain Dk(c)=E[(k)2] jVjXu2Vk(ku)]TJ /F3 11.955 Tf 11.95 0 Td[(1)Pku(m)=E[(k)2] jVjNk(m)Xi=1fPki(m)Xu2Vki(m)(ku)]TJ /F3 11.955 Tf 11.96 0 Td[(1)g.(2) Denek,1 jVjPu2Vkku;duetotherandomnessofmvku,1 Nki(m)Pu2Vkifmgkuwillconvergetokforanypacketthatcontainsasufcientlylargenumberofpixels.Byrearranging( 2 ),weobtain Dk(c)=E[(k)2] jVjNk(m)Xi=1fPki(m)Nki(m)(k)]TJ /F3 11.955 Tf 11.95 0 Td[(1)g=(k)]TJ /F3 11.955 Tf 11.95 0 Td[(1)E[(k)2]Pk(m).(2) From( 3 ),weknowthatE[(k)2]Pk(m)isexactlyequaltoDk(m).Therefore,( 2 )isfurthersimpliedto Dk(c)=(k)]TJ /F3 11.955 Tf 11.95 0 Td[(1)Dk(m).(2) 2.3.6Summary InSection 2.3.1 ,wedecomposedtransmissiondistortionintofourterms;wederivedaformulaforeachterminSections 2.3.2 through 2.3.5 .Inthissection,wecombinetheformulaeforthefourtermsintoasingleformula. 2.3.6.1Pixel-Leveltransmissiondistortion Theorem2.1. Undersingle-referenceprediction,thePTDofpixelukis Dku=Dku(r)+kuDku(m)+Pkufr,mgDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+Pkufr,mgDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+PkufrgkuDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku.(2) 55

PAGE 56

Proof. ( 2 )canbeobtainedbyplugging( 3 ),( 3 ),( 2 ),and( 2 )into( 3 ). Corollary2. Undersingle-referencepredictionandnoslicedatapartitioning,( 2 )issimpliedto Dku=Pku(E[("ku)2]+kuE[(ku)2]+Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)+(1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pku)kuDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku.(2) 2.3.6.2Frame-Leveltransmissiondistortion Theorem2.2. Undersingle-referenceprediction,theFTDofthek-thframeis Dk=Dk(r)+kDk(m)+Pk(r)Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(1+(1)]TJ /F3 11.955 Tf 13.24 2.66 Td[(Pk(r))Dk(p)(1)]TJ /F6 11.955 Tf 11.96 0 Td[(k).(2) Proof. ( 2 )canbeobtainedbyplugging( 3 ),( 3 ),( 2 )and( 2 )into( 2 ). Corollary3. Undersingle-referencepredictionandnoslicedatapartitioning,theFTDofthek-thframeisgivenby( 2 )8. 2.4RelationshipbetweenTheorem 2.2 andExistingTransmissionDistortionModels Asmentionedpreviously,someexistingworkshaveaddressedtheproblemoftransmissiondistortionprediction,andtheyproposedseveraldifferentmodels[ 8 ],[ 9 ],[ 16 ],[ 11 ]toestimatetransmissiondistortion.Inthissection,wewillidentifytherelationshipbetweenTheorem 2.2 andtheirmodels,andspecifytheconditions,underwhichthosemodelsareaccurate.Notethatinordertodemonstratetheeffectofnon-linearclippingontransmissiondistortionpropagation,wedisableintraupdate,thatis,k=0forallthefollowingcases. 8ThesameformulaforbothcasesisbecausebothmeanofDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkuandmeanofDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuconvergetoDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1whenthenumberofpixelsinthek-thframeissufcientlylarge,asseeninAppendix A.3 56

PAGE 57

2.4.1Case1:Onlythe(k)]TJ /F3 11.955 Tf 12.39 0 Td[(1)-thFrameHasError,andtheSubsequentFramesareAllCorrectlyReceived Inthiscase,themodelsproposedinRef.[ 8 ][ 11 ]statethatwhenthereisnointracodingandspatialltering,thepropagationdistortionwillbethesameforalltheframesafterthe(k)]TJ /F3 11.955 Tf 13.07 0 Td[(1)-thframe,i.e.,Dn(p)=Dn)]TJ /F8 7.97 Tf 6.59 0 Td[(1(8nk).However,thisisnottrueasweprovedinProposition 2 .Duetotheclippingfunction,wehaven1(8nk),i.e.,DnDn)]TJ /F8 7.97 Tf 6.58 0 Td[(1(8nk)incasethen-thframeiserror-free.Actually,fromAppendix A.6 ,weknowthattheequalityonlyholdsunderaveryspecialcasethat^fku)]TJ /F6 11.955 Tf 11.96 0 Td[(Hek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku^fku)]TJ /F6 11.955 Tf 11.95 0 Td[(Lforallpixelu2Vk. 2.4.2Case2:BurstErrorsinConsecutiveFrames InRef.[ 16 ],authorsobservethatthetransmissiondistortioncausedbyaccumulatederrorsfromconsecutiveframesisgenerallylargerthanthesumofthosedistortionscausedbyindividualframeerrors.ThisisalsoobservedinourexperimentwhenthereisnoMVerror.Toexplainthisphenomenon,letusrstlookatasimplecasethatresidualsinthek-thframeareallerroneous,whiletheMVsinthek-thframeareallcorrectlyreceived.Inthiscase,weobtainfrom( 2 )thatDk=Dk(r)+Pk(r)Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1+(1)]TJ /F3 11.955 Tf -436.55 -21.25 Td[(Pk(r))Dk(p),whichislargerthanthesimplesumDk(r)+Dk(p)asintheLTImodel;theunder-estimationcausedbytheLTImodelisduetoDk)]TJ /F3 11.955 Tf 13.05 0 Td[((Dk(r)+Dk(p))=(1)]TJ /F6 11.955 Tf 11.96 0 Td[(k)Pk(r)Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1. However,whenMViserroneous,theexperimentalresultisquitedifferentfromthatclaimedinRef.[ 16 ]especiallyforthehighmotionvideo.Inotherwords,theLTImodelnowcausesover-estimationforabursterrorchannel.Inthiscase,thepredictedtransmissiondistortioncanbecalculatedvia( 2 )inTheorem 2.2 asDk1=Dk(r)+kDk(m)+Pk(r)Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(11+(1)]TJ /F3 11.955 Tf 13.98 2.66 Td[(Pk(r))kDk)]TJ /F8 7.97 Tf 6.59 0 Td[(11,andbytheLTImodelasDk2=Dk(r)+Dk(m)+kDk)]TJ /F8 7.97 Tf 6.59 0 Td[(12.So,thepredictiondifferencebetweenTheorem 2.2 andthe 57

PAGE 58

LTImodelis Dk1)]TJ /F4 11.955 Tf 11.96 0 Td[(Dk2=(1)]TJ /F6 11.955 Tf 11.96 0 Td[(k)Pk(r)Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(11)]TJ /F3 11.955 Tf 11.96 0 Td[((1)]TJ /F6 11.955 Tf 11.96 0 Td[(k)Pk(m)E[(k)2]+k(Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(11)]TJ /F4 11.955 Tf 11.96 0 Td[(Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(12).(2) Atthebeginning,D01=D02=0,andDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1<
PAGE 59

DenotethesystembyanoperatorHthatmapstheerrorinputsequencefPkg,asafunctionofframeindexk,tothedistortionoutputsequencefDkg.SincegenerallyDk(p)isanonlinearfunctionofDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1,asaratioofDk(p)andDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1,kisstillafunctionofDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1.Asaresult,kisafunctionofDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1.ThatmeanstheoperatorHisnon-linear,i.e.,thesystemisnon-linear.Inaddition,sincekvariesfromframetoframeasmentionedinSection 2.3.4.2 ,thesystemistime-variant.Insummary,Hisgenerallyanon-lineartime-variantsystem. TheLTImodelassumesthat1)theoperatorHislinear,thatis,H(aPk1+bPk2)=aH(Pk1)+bH(Pk2),whichisvalidonlywhenkdoesnotdependonDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1;and2)theoperatorHistime-invariant,thatis,Dk+=H(Pk+),whichisvalidonlywhenkisconstant,i.e.,bothPk(r)andkareconstant.Underthesetwoassumptions,wehavei=,andweobtainQki=l+1i=()k)]TJ /F7 7.97 Tf 6.58 0 Td[(l.Leth[k]=()k,whereh[k]istheimpulseresponseoftheLTImodel;thenweobtain Dk=kXl=k)]TJ /F7 7.97 Tf 6.59 0 Td[(L[h[k)]TJ /F4 11.955 Tf 11.96 0 Td[(l](ElPl)].(2) FromProposition 2 ,itiseasytoprovethat01;soh[k]isadecreasingfunctionoftime.Weseethat( 2 )isaconvolutionbetweentheerrorinputsequenceandthesystemimpulseresponse.Actually,ifweleth[k]=e)]TJ /F28 7.97 Tf 6.58 0 Td[(k,where=)]TJ /F3 11.955 Tf 11.29 0 Td[(log,itisexactlytheformulaproposedinRef.[ 9 ].Notethat( 2 )isaveryspecialcaseof( 2 )withthefollowinglimitations:1)thevideocontenthastobeoflowmotion;2)thereisnoslicedatapartitioningorallpixelsinthesameframeexperiencethesamechannelcondition;3)kisaconstant,thatis,bothPk(r)andthepropagationfactorkareconstant,whichrequirestheprobabilitydistributionsofreconstructedpixelvaluesinallframesshouldbethesame.Notethatthephysicalmeaningofkisnottheactualpropagationfactor,butitisjustanotationforsimplifyingtheformula. 59

PAGE 60

2.5PTDandFTDunderMulti-ReferencePrediction ThePTDandFTDformulaeinSection 2.3 areforsingle-referenceprediction.Inthissection,weextendtheformulaetomulti-referenceprediction. 2.5.1Pixel-levelDistortionunderMulti-ReferencePrediction Ifmultipleframesareallowedtobethereferencesformotionestimation,thereconstructedpixelvalueattheencoderin( 2 )becomes ^fku=\(^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+^eku).(2) Forthereconstructedpixelvalueatthedecoderin( 2 ),itisabitdifferentasbelow. efku=\(efk)]TJ /F7 7.97 Tf 6.59 0 Td[(j0u+fmvku+eeku).(2) Ifmvkuiscorrectlyreceived,fmvku=mvkuandefk)]TJ /F7 7.97 Tf 6.59 0 Td[(j0u+fmvku=efk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku.However,ifmvkuisreceivedwitherror,theconcealedMVhasnodifferencefromthesingle-referencecase,thatis,fmvku=mvkuandefk)]TJ /F7 7.97 Tf 6.59 0 Td[(j0u+fmvku=efk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku. Asaresult,( 2 )becomes eku=(^eku+^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)]TJ /F3 11.955 Tf 13 2.66 Td[(^ku))]TJ /F3 11.955 Tf 11.95 0 Td[((eeku+efk)]TJ /F7 7.97 Tf 6.58 0 Td[(j0u+fmvku)]TJ /F5 11.955 Tf 12.82 3.16 Td[(eku)=(^eku)]TJ /F5 11.955 Tf 11.81 .5 Td[(eeku)+(^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(j0u+fmvku)+(^fk)]TJ /F7 7.97 Tf 6.58 0 Td[(j0u+fmvku)]TJ /F5 11.955 Tf 11.87 3.16 Td[(efk)]TJ /F7 7.97 Tf 6.59 0 Td[(j0u+fmvku))]TJ /F3 11.955 Tf 11.95 0 Td[((^ku)]TJ /F5 11.955 Tf 12.81 3.16 Td[(eku).(2) FollowingthesamederivingprocessfromSection 2.3.1 toSection 2.3.5 ,theformulaeforPTDundermulti-referencepredictionarethesameasthoseundersingle-referencepredictionexceptthefollowingchanges:1)MVCEeku,^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)]TJ /F3 11.955 Tf 12.3 2.66 Td[(^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(j0u+fmvkuandclippingnoiseeku,(efk)]TJ /F7 7.97 Tf 6.59 0 Td[(j0u+fmvku+eeku))]TJ /F3 11.955 Tf 12.38 0 Td[(\(efk)]TJ /F7 7.97 Tf 6.59 0 Td[(j0u+fmvku+eeku);2)Dku(m)andDku(c)aregivenby( 3 )and( 2 ),respectively,withanewdenitionofku,^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)]TJ /F3 11.955 Tf 12.96 2.65 Td[(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku;3)Dku(p),E[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg)2],ku,Dku(p) Dk)]TJ /F10 5.978 Tf 5.76 0 Td[(ju+mvkuand Dku(P)=Pkufr,mgDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+Pkufr,mgDk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+PkufrgDku(p),(2) 60

PAGE 61

comparedto( 2 ).ThegeneralizationofPTDformulaetomulti-referencepredictionisstraightforwardsincethemulti-referencepredictioncasejusthasalargersetofreferencepixelsthanthesingle-referencecase.Therefore,wehavethefollowinggeneraltheoremforPTD. Theorem2.3. Undermulti-referenceprediction,thePTDofpixelukis Dku=Dku(r)+kuDku(m)+Pkufr,mgDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+Pkufr,mgDk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+PkufrgkuDk)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku.(2) Corollary4. Undermulti-referencepredictionandnoslicedatapartitioning,( 2 )issimpliedto Dku=Pku(E[("ku)2]+kuE[(ku)2]+Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)+(1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pku)kuDk)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku.(2) 2.5.2Frame-levelDistortionunderMulti-ReferencePrediction Undermulti-referenceprediction,eachblocktypicallyisallowedtochooseitsreferenceblockindependently;hence,differentpixelsinthesameframemayhavedifferentreferenceframes.DeneVk(j),fuk:uk=vk)]TJ /F7 7.97 Tf 6.59 0 Td[(j)]TJ /F15 11.955 Tf 12.01 0 Td[(mvkug,wherej2f1,2,...,JgandJisthenumberofreferenceframes;i.e.,Vk(j)isthesetofthepixelsinthek-thframe,whosereferencepixelsareinthe(k)]TJ /F4 11.955 Tf 12.38 0 Td[(j)-thframe.Obviously,SJj=1Vk(j)=VkandTJj=1Vk(j)=?.Denewk(j),jVk(j)j jVkj.NotethatVkandVk(j)havethesimilarphysicalmeaningsbutonlythedifferentcardinalities. Dk(m)andDk(c)aregivenby( 3 )and( 2 ),respectively,withanewdenitionofk(j),fku:ku=^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)]TJ /F3 11.955 Tf 12.07 2.65 Td[(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkug.andk=PJj=1wk(j)k(j).DenethepropagationfactorofVk(j)byk(j),Pu2Vk(j)kuDk)]TJ /F10 5.978 Tf 5.75 0 Td[(ju+mvku Pu2Vk(j)Dk)]TJ /F10 5.978 Tf 5.76 0 Td[(ju+mvku.ThefollowinglemmagivestheformulaforDk(P). 61

PAGE 62

Lemma4. Theframe-levelpropagationandclippingcauseddistortioninthek-thframeforthemulti-referencecaseis Dk(P)=Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1Pkfr,mg+JXj=1(Pk(j)fr,mgwk(j)Dk)]TJ /F7 7.97 Tf 6.59 0 Td[(j)+(1)]TJ /F6 11.955 Tf 11.96 0 Td[(k)JXj=1(Pk(j)frgwk(j)k(j)Dk)]TJ /F7 7.97 Tf 6.59 0 Td[(j),(2) wherekisthepercentageofI-MBsinthek-thframe;Pk(j)fr,mgistheweightedaverageofjointPEPsofeventfr,mgforthej-thsub-frameinthek-thframe.Pk(j)frgistheweightedaverageofPEPofeventfrgforthej-thsub-frameinthek-thframe. Lemma 4 isprovedinAppendix A.5 .WithLemma 4 ,wehavethefollowinggeneraltheoremforFTD. Theorem2.4. Undermulti-referenceprediction,theFTDofthek-thframeis Dk=Dk(r)+kDk(m)+Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1Pkfr,mg+JXj=1(Pk(j)fr,mgwk(j)Dk)]TJ /F7 7.97 Tf 6.58 0 Td[(j)+(1)]TJ /F6 11.955 Tf 11.96 0 Td[(k)JXj=1(Pk(j)frgwk(j)k(j)Dk)]TJ /F7 7.97 Tf 6.59 0 Td[(j).(2) Proof. ( 2 )canbeobtainedbyplugging( 3 ),( 3 ),( 2 )and( 2 )into( 2 ). Itiseasytoprovethat( 2 )inTheorem 2.2 isaspecialcaseof( 2 )withJ=1andwk(j)=1.Itisalsoeasytoprovethat( 2 )inTheorem 2.3 isaspecialcaseof( 2 )withjVj=1. Corollary5. Undermulti-referencepredictionandnoslicedatapartitioning,( 2 )issimpliedto Dk=Dk(r)+kDk(m)+Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(1Pkfrg+(1)]TJ /F6 11.955 Tf 11.95 0 Td[(k)JXj=1(Pk(j)frgwk(j)k(j)Dk)]TJ /F7 7.97 Tf 6.59 0 Td[(j).(2) 62

PAGE 63

CHAPTER3PREDICTIONOFTRANSMISSIONDISTORTIONFORWIRELESSVIDEOCOMMUNICATION:ALGORITHMANDAPPLICATION Inthischapter,wedesignthealgorithmstoestimatethetransmissiondistortionbasedontheanalysisinChapter 2 .Wealsoapplythealgorithmintherate-distortionoptimizedmodedecisionproblemandachievearemarkableperformancegainthanexistingsolutions. 3.1ALiteratureReviewonEstimationAlgorithmsofTransmissionDistortion Transmittingvideooverwirelesswithgoodqualityorlowend-to-enddistortionisparticularlychallengingsincethereceivedvideoissubjecttonotonlyquantizationdistortionbutalsotransmissiondistortion(i.e.,videodistortioncausedbypacketerrors).Thecapabilityofpredictingtransmissiondistortioncanassistindesigningvideoencodingandtransmissionschemesthatachievemaximumvideoqualityorminimumend-to-endvideodistortion.InChapter 2 ,wehavetheoreticallyderivedformulaefortransmissiondistortion.Inthischapter,weleveragetheanalyticalresultsinChapter 2 todesignalgorithmsforestimatingtransmissiondistortion;wealsodevelopanalgorithmforestimatingend-to-enddistortion,andapplyittopredictionmodedecisioninH.264encoder. Toestimateframe-leveltransmissiondistortion(FTD),severallinearmodelbasedalgorithms[ 8 11 ]areproposed.Thesealgorithmsusethesumofthenewlyinduceddistortioninthecurrentframeandthepropagateddistortionfrompreviousframes,toestimatetransmissiondistortion.Thelinearmodelbasedalgorithmssimplifytheanalysisoftransmissiondistortionatthecostofsacricingthepredictionaccuracybyneglectingthecorrelationbetweenthenewlyinducederrorandthepropagatederror.Liangetal.[ 16 ]extendtheresultinRef.[ 8 ]byaddressingtheeffectofcorrelation.However,theydonotconsidertheeffectofmotionvector(MV)errorontransmissiondistortionandtheiralgorithmisnottestedwithhighmotionvideocontent.Underthiscondition,theyclaimthattheLTImodels[ 8 9 ]under-estimatetransmissiondistortion 63

PAGE 64

duetopositivecorrelationbetweentwoadjacenterroneousframes.InChapter 2 ,weidentifythattheMVconcealmenterrorisnegativelycorrelatedwiththepropagatederrorandthiscorrelationdominatesoverallothertypesofcorrelationespeciallyforhighmotionvideo.AslongasMVtransmissionerrorsexist,thetransmissiondistortionestimatedbyLTImodelsbecomesover-estimated.InChapter 2 ,wealsoquantifytheeffectsofthosecorrelationsontransmissiondistortionbyasystemparametercalledcorrelationratio.Ontheotherhand,noneofexistingworksanalyzestheimpactofclippingnoiseontransmissiondistortion.InChapter 2 ,weprovethatclippingnoisereducesthepropagatederrorandquantifyitseffectbyanothersystemparametercalledpropagationfactor.Inthischapter,wedesignalgorithmstoestimatecorrelationratioandpropagationfactor,whichfacilitatesthedesignofalowcomplexityalgorithmcalledRMPC-FTDalgorithmforestimatingframe-leveltransmissiondistortion.ExperimentalresultsdemonstratethatourRMPC-FTDalgorithmismoreaccurateandmorerobustthanexistingalgorithms.AnotheradvantageofourRMPC-FTDalgorithmisthatallparametersintheformuladerivedinChapter 2 canbeestimatedbyusingtheinstantaneousvideoframestatisticsandchannelconditions,whichallowstheframestatisticstobetime-varyingandtheerrorprocessestobenon-stationary.However,existingalgorithmsestimatetheirparametersbyusingthestatisticsaveragedovermultipleframesandassumethesestatisticsdonotchangeovertime;theirmodelsallassumetheerrorprocessisstationary.Asaresult,ourRMPC-FTDalgorithmismoresuitableforreal-timevideocommunication. Forpixel-leveltransmissiondistortion(PTD),theestimationalgorithmissimilartotheFTDestimationalgorithmsincethePTDformulaisaspecialcaseoftheFTDformulaasdiscussedinChapter 2 .However,insomeexistingvideoencoders,e.g.,H.264referencecodeJM14.01,motionestimationandpredictionmodedecisionare 1http://iphome.hhi.de/suehring/tml/download/old jm/jm14.0.zip 64

PAGE 65

separatelyconsidered.Therefore,theMVandcorrespondingresidualareknownfordistortionestimationinmodedecision.Insuchacase,thePTDestimationalgorithmcanbesimpliedwithknownvaluesoftheMVandcorrespondingresidual,comparedtousingtheirstatistics.Inthischapter,wedesignaPTDestimationalgorithm,calledRMPC-PTDforsuchacase;wealsoextendRMPC-PTDtoestimatepixel-levelend-to-enddistortion(PEED). PEEDestimationisimportantfordesigningoptimalencodingandtransmissionschemes.SomeexistingPEEDestimationalgorithmsareproposedinRefs.[ 4 5 ].InRef.[ 4 ],therecursiveoptimalper-pixelestimate(ROPE)algorithmisproposedtoestimatethePEEDbyrecursivelycalculatingtherstandsecondmomentsofthereconstructedpixelvalue.However,theROPEalgorithmneglectsthesignicanteffectofclippingnoiseontransmissiondistortion,resultingininaccurateestimate.Furthermore,theROPEalgorithmrequiresintensivecomputationofcorrelationcoefcientswhenpixelaveragingoperations(e.g.,ininterpolationlteranddeblockinglter)areinvolved[ 23 ],whichreducesitsapplicabilityinH.264videoencoder.Stockhammeretal.[ 5 ]proposeadistortionestimationalgorithmbysimulatingKindependentdecodersattheencodersideduringtheencodingprocessandaveragingthedistortionsoftheseKdecoders.ThisalgorithmisbasedontheLawofLargeNumber(LLN),i.e.,theestimateddistortionwillasymptoticallyapproachtheexpecteddistortionasKgoestoinnity.Forthisreason,wecallthealgorithminRef.[ 5 ]asLLNalgorithm.However,forLLNalgorithm,thelargernumberofdecoderssimulated,thehighercomputationalcomplexityandthelargermemoryrequired.Asaresult,LLNalgorithmisnotsuitableforreal-timevideocommunication.Toenhanceestimationaccuracy,reducecomplexityandimproveextensibility,inthischapter,weextendRMPC-PTDalgorithmtoPEEDestimation;theresultingalgorithmiscalledRMPC-PEED.ComparedtoROPEalgorithm,RMPC-PEEDalgorithmismoreaccuratesincethesignicanteffectofclippingnoiseontransmissiondistortionisconsidered.AnotheradvantageoverROPEalgorithmisthatRMPC-PEED 65

PAGE 66

algorithmismucheasiertobeextendedtosupportaveragingoperations,e.g.,interpolationlter.ComparedtoLLNalgorithm,thecomputationalcomplexityandmemoryrequirementofRMPC-PEEDalgorithmaremuchlowerandtheestimateddistortionhassmallervariance. Inexistingvideoencoders,predictionmodedecisionistochoosethebestpredictionmodeinthesenseofminimizingtheRate-Distortion(R-D)costforeachMacroblock(MB)orsub-MB.EstimationoftheMBlevelorsub-MBlevelend-to-enddistortionfordifferentpredictionmodesisneeded.Ininter-prediction,thereferencepixelsofthesameencodingblockmaybelongtodifferentblocksinthereferenceframe;therefore,PEEDestimationisneededforcalculatingR-Dcostinpredictionmodedecision.Inthischapter,weapplyourRMPC-PEEDalgorithmtopredictionmodedecisioninH.264;theresultingalgorithmiscalledRMPC-MS.Experimentalresultsshowthat,forpredictionmodedecisioninH.264encoder,ourRMPC-MSalgorithmachievesanaveragePSNRgainof1.44dBoverROPEalgorithmfor`foreman'sequenceunderPEP=5%;anditachievesanaveragePSNRgainof0.89dBoverLLNalgorithmfor`foreman'sequenceunderPEP=1%. Therestofchapterisorganizedasfollows.Section 3.2 presentsouralgorithmsforestimatingFTDundertwoscenarios:onewithoutacknowledgementfeedbackandonewithacknowledgementfeedback.InSection 3.3 ,wedevelopalgorithmsforestimatingPTD.InSection 3.4 ,weextendourPTDestimationalgorithmtoPEEDestimation.InSection 3.5 ,weapplyourPEEDestimationalgorithmtopredictionmodedecisioninH.264encoderandcompareitscomplexitywithexistingalgorithms.Section 5.5 showstheexperimentalresultsthatdemonstrateaccuracyandrobustnessofourdistortionestimationalgorithmandsuperiorR-Dperformanceofourmodedecisionschemeoverexistingschemes. 66

PAGE 67

3.2AlgorithmsforEstimatingFTD Inthissection,wedevelopouralgorithmsforestimatingFTDundertwoscenarios:onewithoutacknowledgementfeedbackandonewithacknowledgementfeedback,whicharepresentedinSections 3.2.1 and 3.2.2 ,respectively. 3.2.1FTDEstimationwithoutFeedbackAcknowledgement Chapter 2 derivesaformulaforFTDundersingle-referenceprediction,i.e., Dk=Dk(r)+Dk(m)+Dk(P)+Dk(c),(3) where Dk(r)=E[("k)2]Pk(r);(3) Dk(m)=E[(k)2]Pk(m);(3) Dk(P)=Pk(r)Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1+(1)]TJ /F6 11.955 Tf 11.95 0 Td[(k)(1)]TJ /F3 11.955 Tf 13.24 2.66 Td[(Pk(r))kDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1;(3) Dk(c)=(k)]TJ /F3 11.955 Tf 11.95 0 Td[(1)Dk(m);(3)"kistheresidualconcealmenterrorandPk(r)istheweightedaveragePEPofallresidualpacketsinthek-thframe;kistheMVconcealmenterrorandPk(m)istheweightedaveragePEPofallresidualpacketsinthek-thframe;kisthepercentageofencodedI-MBsinthek-thframe;boththepropagationfactorkandthecorrelationratiokdependonvideocontent,channelconditionandcodecstructure,andarethereforecalledsystemparameters;Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(1isthetransmissiondistortioninthek)]TJ /F3 11.955 Tf 12.04 0 Td[(1frame,whichcanbeiterativelycalculatedby( 3 ). Next,Sections 3.2.1.1 through 3.2.1.4 presentmethodstoestimateeachofthefourdistortiontermsin( 3 ),respectively. 3.2.1.1Estimationofresidualcauseddistortion FromtheanalysisinChapter 2 ,E[("k)2]=E[("ku)2]=E[(^eku)]TJ /F3 11.955 Tf 12.37 .01 Td[(eku)2]foralluinthek-thframe;^ekuisthetransmittedresidualforpixeluk;andekuistheconcealedresidual 67

PAGE 68

forpixelukatthedecoder.E[("k)2]canbeestimatedfromthenitesamplesof"kuinthek-thframe,i.e.,^E[("k)2]=1 jVjPu2Vk(^eku)]TJ /F5 11.955 Tf 14.47 3.84 Td[(beku)2,wherebekuistheestimateofeku. FromtheanalysisinChapter 2 ,Pk(r)=1 jVjPNk(r)i=1(Pki(r)Nki(r)),wherePki(r)isthePEPofthei-thresidualpacketinthek-thframe;Nki(r)isthenumberofpixelscontainedinthei-thresidualpacketofthek-thframe;Nk(r)isthenumberofresidualpacketsinthek-thframe.Pki(r)canbeestimatedfromchannelstatestatistics.DenotetheestimatedPEPby^Pki(r)foralli2f1,2,...,Nk(r)g;thenPk(r)canbeestimatedby^Pk(r)=1 jVjPNk(r)i=1(^Pki(r)Nki(r)).Asaresult,Dk(r)canbeestimatedby ^Dk(r)=^E[("k)2]^Pk(r)=1 (jVj)2Nk(r)Xi=1(^Pki(r)Nki(r))Xu2Vk(^eku)]TJ /F5 11.955 Tf 14.48 3.84 Td[(beku)2.(3) Next,wediscusshowto1)conceal^ekuatthedecoder;2)estimateekuattheencoder;and3)estimatePki(r)attheencoder. Concealmentof^ekuatthedecoder:Atthedecoder,if^ekuisreceivedwitherroranditsneighboringpixelsarecorrectlyreceived,itsneighboringpixelscouldbeutilizedtoconceal^eku.However,thisispossibleonlyifthepixelukisatthesliceboundaryandthepixelsattheothersideofthissliceboundaryiscorrectlyreceived.InH.264,mostpixelsinaslicedonotlocateatthesliceboundary.Therefore,ifonesliceislost,mostofpixelsinthatslicewillbeconcealedwithouttheinformationfromneighboringpixels.Ifthesamemethodisusedtoconcealekuofallpixels,itisnotdifculttoprovethattheminimumofE[("k)2]isachievedwheneku=E[^eku]. NotethatwhenekuisconcealedbyE[^eku]atthedecoder,E[("k)2]isthevarianceof^eku,thatis,E[("k)2]=2^eku.Inourexperiment,wendthatthehistogramof^ekineachframeapproximatelyfollowsaLaplaciandistributionwithzeromean.AsprovedinRef.[ 24 ],thevarianceofekdependsonthespatio-temporalcorrelationoftheinputvideosequenceandtheaccuracyofmotionestimation.Since^ekisafunctionofek,E[("k)2]alsodependsontheaccuracyofmotionestimation.So,foragivenvideosequence,moreaccurateresidualconcealmentandmoreaccuratemotion 68

PAGE 69

estimationproduceasmallerDk(r).Thiscouldbeusedasacriterionforthedesignoftheencodingalgorithmattheencoderandresidualconcealmentmethodatthedecoder. Estimationofekuattheencoder:Iftheencoderhasknowledgeoftheconcealmentmethodatthedecoderaswellasthefeedbackacknowledgementofsomepackets,ekucanbeestimatedbythesameconcealedmethodsatthedecoder.Thatmeansthemethodstoestimateekuofpixelsatthesliceboundaryaredifferentfromotherpixels.However,ifnofeedbackacknowledgementofwhichpacketsarecorrectlyreceived,thesamemethodmaybeusedtoestimateekuofallpixels,thatis,beku=1 jVjPu2Vk^eku.Notethatevenifthefeedbackacknowledgementofsomepacketsarecorrectlyreceivedbeforetheestimation,theestimateobtainedbythismethodattheencoderisstillquiteaccuratesincemostpixelsinaslicedonotlocateatthesliceboundary. Inmostcases,forastandardhybridcodecsuchasH.264,1 jVjPu2Vk^ekuapproximatelyequalszero2forP-MBsandB-MBs.Therefore,onesimpleconcealmentmethodistoletbeku=0asinmosttransmissiondistortionmodels.Inthischapter,westillusebekuincase1 jVjPu2Vk^eku6=0duetotheimperfectpredictivecoding,orinthegeneralcase,thatis,somefeedbackacknowledgementsmayhavebeenreceivedbeforetheestimation.Notethatwhenbeku=1 jVjPu2Vk^ekuattheencoder,^E[("k)2]isthesamplevarianceof^ekuandinfactabiasedestimatorof2^eku[ 25 ].Inotherwords,^E[("k)2]isasufcientstatisticofallindividualsamples^eku.Ifthesufcientstatistic^E[("k)2]isknown,theFTDestimatordoesnotneedthevaluesof^ekuofallpixels.Therefore,suchanFTDestimatorincursmuchlowercomplexitythanusingthevaluesof^ekuofallpixels. EstimationofPki(r):Inwiredcommunication,applicationlayerPEPisusuallyestimatedbyPacketErrorRate(PER),whichistheratioofthenumberofincorrectlyreceivedpacketstothenumberoftransmittedpackets,thatis,^Pki(r)=PERki(r).Inawirelessfadingchannel,instantaneousphysicallayerPEPisafunctionofthe 2Thisisactuallyanobjectiveofpredictivecoding. 69

PAGE 70

instantaneouschannelgaing(t)attimet[ 18 ],whichisdenotedbyp(g(t)).Atanencoder,therearetwocases:1)thetransmitterhasperfectknowledgeofg(t),and2)thetransmitterhasnoknowledgeofg(t)butknowstheprobabilitydensityfunction(pdf)ofg(t).ForCase1,theestimatedPEP^Pki(r)=p(g(t))sinceg(t)isknown.Notethatsincethechannelgainistimevarying,theestimatedinstantaneousPEPisalsotimevarying.3ForCase2,p(g(t))isarandomvariablesinceonlypdfofg(t)isknown.Hence,weshouldusetheexpectedvalueofp(g(t))toestimatePki(r),thatis,^Pki(r)=E[p(g(t))],wheretheexpectationistakenoverthepdfofg(t). 3.2.1.2EstimationofMVcauseddistortion FromtheanalysisinChapter 2 ,E[(k)2]=E[(ku)2]=E[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)]TJ /F3 11.955 Tf 12.32 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku)2]foralluinthek-thframe;mvkuisthetransmittedMVforpixeluk;andmvkuistheconcealedMVforpixelukatthedecoder.E[(k)2]canbeestimatedfromthenitesamplesofkuinthek-thframe,i.e.,^E[(k)2]=1 jVjPu2Vk(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)]TJ /F3 11.955 Tf 12.05 2.65 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+dmvku)2,wheredmvkuistheestimateofmvku. SimilartoSection 3.2.1.1 ,Pk(m)=1 jVjPNk(m)i=1(Pki(m)Nki(m)),wherePki(m)isthePEPofthei-thMVpacketinthek-thframe;Nki(m)isthenumberofpixelscontainedinthei-thMVpacketofthek-thframe;Nk(m)isthenumberofMVpacketsinthek-thframe.Pki(m)canbeestimatedfromchannelstatestatistics.DenotetheestimatedPEPby^Pki(m)foralli2f1,2,...,Nk(m)g;thenPk(r)canbeestimatedby^Pk(m)=1 jVjPNk(m)i=1(^Pki(m)Nki(m)).Asaresult,Dk(m)canbeestimatedby ^Dk(m)=^E[(k)2]^Pk(m)=1 (jVj)2Nk(m)Xi=1(^Pki(m)Nki(m))Xu2Vk(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+dmvku)2.(3) Next,wediscusshowto1)concealmvkuatthedecoder;2)estimatemvkuattheencoder;and3)estimatePki(m)attheencoder. 3Thisimpliesthatthepixelerrorprocessisnon-stationaryoverbothtimeandspace. 70

PAGE 71

Concealmentofmvkuatthedecoder:Differentfromresidual,MVarehighlycorrelatedinbothtemporalandspatialdomains.Hence,thedecodermayconcealtheMVbytemporallyneighboringblockifitsspatiallyneighboringblocksarenotavailable.Dependingonwhethertheneighboringblocksarecorrectlyreceivedornot,theremaybeseveraloptionsofMVerrorconcealmentmethodsforeachblock,oreachpixeltomakeitmoregeneral.Iftheneighboringblocksarecorrectlyreceived,mvkucanbeconcealedbythemedianoraverageofthoseneighboringblocks.InterestedreadersmayrefertoRef.[ 21 ],[ 26 ],[ 27 ]fordiscussionsondifferentMVconcealmentmethods.Inourexperiment,wealsoobservethatthehistogramofkinoneframeapproximatelyfollowsaLaplaciandistributionwithzeromean.Fordifferentconcealmentmethods,thevarianceofkwillbedifferent.Themoreaccurateconcealedmotionestimation,thesmallerDk(m). Estimationofmvkuattheencoder:IftheencoderknowstheconcealmentmethodsofcurrentblockandthePEPofneighboringblocks,wecanestimatetheMVcauseddistortionbyassigningdifferentconcealmentmethodswithdifferentprobabilitiesattheencoderasinRef.[ 4 ].However,iftheencoderdoesnotknowwhatconcealmentmethodsareusedbythedecoderornoneighboringblockscanbeutilizedforerrorconcealment(e.g.,bothtemporalandspatialneighboringblocksareinerror),asimpleestimationalgorithm[ 9 ],[ 10 ]istoletdmvku=0,thatis,usingthepixelvaluefromthesamepositionofthepreviousframe.Inthischapter,westillusedmvkutodenotetheestimateofconcealedmotionvectorforthegeneralcase. EstimationofPki(m):TheestimationofPki(m)issimilartotheestimationofPki(r).NotethatinH.264specication,thereisnoslicedatapartitioningforaninstantaneousdecodingrefresh(IDR)frame[ 22 ],soPki(r)=Pki(m)forallpixelsinanIDR-frame.ThisisalsotrueforI-MB,andP-MBwithoutslicedatapartitioning.ForP-MBwithslicedatapartitioninginH.264,theerrorstateofresidualandtheerrorstateofMVofthesamepixelarepartiallycorrelated.Tobemorespecic,iftheMVpacketislost,the 71

PAGE 72

correspondingresidualpacketcannotbedecodedevenifitiscorrectlyreceived,sincethereisnosliceheaderintheresidualpacket.Asaresult,Pki(rH.264)=Pki(r)+(1)]TJ /F4 11.955 Tf -439.12 -23.91 Td[(Pki(r))Pki(m). 3.2.1.3Estimationofpropagationandclippingcauseddistortion ToestimateDk(P),weonlyneedtoestimateksincePk(r)hasbeenestimatedinSection 3.2.1.1 .InChapter 2 ,wetheoreticallyderivethepropagationfactorkuofpixelukforpropagatederrorwithazero-meanLaplaciandistribution,i.e., =1)]TJ /F3 11.955 Tf 13.15 8.09 Td[(1 2e)]TJ /F10 5.978 Tf 7.78 4.63 Td[(y)]TJ /F27 5.978 Tf 5.76 0 Td[(L b(y)]TJ /F6 11.955 Tf 11.96 0 Td[(L b+1))]TJ /F3 11.955 Tf 13.15 8.09 Td[(1 2e)]TJ /F27 5.978 Tf 7.78 4.63 Td[(H)]TJ /F10 5.978 Tf 5.76 0 Td[(y b((H)]TJ /F4 11.955 Tf 11.96 0 Td[(y) b+1),(3) whereLandHareuser-speciedlowthresholdandhighthreshold,respectively;yisthereconstructedpixelvalue;b=p 2 2;andisthestandarddeviationofthepropagatederror. Here,weprovidethreemethodstoestimatethepropagationfactorkasbelow. Estimationofkbyku:AsdenedinChapter 2 ,k=Pu2VkkuDk)]TJ /F11 5.978 Tf 5.75 0 Td[(1u+mvku Pu2VkDk)]TJ /F11 5.978 Tf 5.76 0 Td[(1u+mvku.Therefore,wemayrstestimatekuby( 3 )andthenestimatekbyitsdenition.However,thismethodrequirestocomputeexponentiationsanddivisionsin( 3 )foreachpixel,andneedslargememorytostore^Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuforallpixelsinallreferenceframes. Estimatetheaverageofafunctionbythefunctionofanaverage:Ifweestimatekdirectlybytheframestatisticsinsteadofpixelvalues,boththecomputationalcomplexityandmemoryrequirementwillbedecreasedbyafactorofNVk.Ifonly^Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1insteadof^Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuisstoredinmemory,wemaysimplifyestimatingkby^k=Pu2Vk^ku^Dk)]TJ /F11 5.978 Tf 5.75 0 Td[(1 Pu2Vk^Dk)]TJ /F11 5.978 Tf 5.76 0 Td[(1=1 jVjPu2Vk^ku.Thisisaccurateifallpacketsinthesameframeexperiencethesamechannelcondition.Weseefrom( 3 )thatkuisafunctionofthereconstructedpixelvalue^fkuandthevarianceofpropagatederror2^fk)]TJ /F11 5.978 Tf 5.76 0 Td[(1u+mvku,whichisequaltoDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1inthiscase.Denoteku=g(^fku,Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1);wehavek=1 jVjPu2Vkg(^fku,Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(1).Onesimpleandintuitivemethodistousethefunctionofanaveragetoestimatetheaverageofafunction,thatis,^k=g(1 jVjPu2Vk^fku,^Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(1). 72

PAGE 73

Improveestimationaccuracybyusingthepropertyof( 3 ):Althoughtheabovemethoddramaticallyreducestheestimationcomplexityandmemoryrequirement,thatsimpleapproximationisonlyaccurateifkuisalinearfunctionof^fku.Inotherwords,suchapproximationcausesunderestimationfortheconvexfunctionoroverestimationfortheconcavefunction[ 28 ].Although( 3 )isneitheraconvexfunctionnoraconcavefunction,itisinterestingtoseethat1)kuissymmetricabout^fku=H+L 2;2)kuisamonotonicallyincreasingfunctionof^fkuwhenL<^fku>>>>><>>>>>>:y)]TJ /F6 11.955 Tf 11.95 0 Td[(L,y)]TJ /F4 11.955 Tf 11.96 0 Td[(xH.(3) 73

PAGE 74

OurexperimentalresultsinSection 5.5 showthattheproposedalgorithmprovidesaccurateestimate.Finally,itisstraightforwardtoestimateDk(P)by ^Dk(P)=^Pk(r)^Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1+(1)]TJ /F6 11.955 Tf 11.96 0 Td[(k)(1)]TJ /F3 11.955 Tf 13.24 2.66 Td[(^Pk(r))^Dk(p).(3) 3.2.1.4Estimationofcorrelation-causeddistortion ToestimateDk(c),theonlyparameterneedstobeestimatedisksinceDk(m)hasbeenestimatedinSection 3.2.1.2 .AsdenedinChapter 2 ,k=1 jVjPu2Vkku,whereku=E[kuefk)]TJ /F11 5.978 Tf 5.76 0 Td[(1u+mvku] E[ku^fk)]TJ /F11 5.978 Tf 5.76 0 Td[(1u+mvku].kudependsonthemotionactivityofthevideocontentaccordingtoChapter 2 Inourexperiment,wendthatkissmallwhentheaverageMVlengthoverthesetinthek-thframeislargerthanhalfoftheblocklength,andk1whentheaverageMVlengthinthek-thframeissmallerthanhalfoftheblocklength,orwhenthepropagatederrorfromthereferenceframesissmall.Anintuitiveexplanationforthisphenomenonisasbelow:1)iftheaverageMVlengthislargeandtheMVpacketsarereceivedwitherror,mostconcealedreferencepixelswillbeinsomeblockdifferentfromtheblockwherethecorrespondingtruereferencepixelslocate;2)iftheaverageMVlengthissmall,mostconcealedreferencepixelsandthecorrespondingtruereferencepixelswillstillbeinthesameblockeveniftheMVpacketisreceivedwitherror;3)sincethecorrelationbetweentwopixelsinsidethesameblockismuchhigherthanthecorrelationbetweentwopixelslocatedindifferentblocks,hencekissmallwhentheaverageMVlengthislargeandviceversa;4)ifthereisnopropagatederrorfromthereferenceframes,accordingtothedenition,itiseasytoprovethatk=1. Therefore,weproposealowcomplexityalgorithmtoestimatekbyvideoframestatisticsasbelow ^k=8>><>>:(1)]TJ /F3 11.955 Tf 13.24 2.66 Td[(Pk)]TJ /F8 7.97 Tf 6.58 0 Td[(1(m))(1)]TJ /F3 11.955 Tf 13.24 2.66 Td[(Pk)]TJ /F8 7.97 Tf 6.59 0 Td[(1(r)), jmvkj>block size 21,otherwise,(3) 74

PAGE 75

wherePk)]TJ /F8 7.97 Tf 6.59 0 Td[(1(r)isdenedin( 3 );Pk)]TJ /F8 7.97 Tf 6.59 0 Td[(1(m)isdenedin( 3 ); jmvkj=1 jVjPu2Vkjmvkuj,andjmvkujisthelengthofmvku.Asaresult, ^Dk(c)=(^k)]TJ /F3 11.955 Tf 11.95 0 Td[(1)^Dk(m).(3) 3.2.1.5Summary Withoutfeedbackacknowledgement,thetransmissiondistortionofthek-thframecanbeestimatedby ^Dk=^Dk(r)+^k^Dk(m)+^Pk(r)^Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(1+(1)]TJ /F6 11.955 Tf 11.96 0 Td[(k)(1)]TJ /F3 11.955 Tf 13.24 2.66 Td[(^Pk(r))^Dk(p),(3) where^Dk(r)canbeestimatedby( 3 );^Dk(m)canbeestimatedby( 3 );^Dk(p)canbeestimatedby( 3 );^kcanbeestimatedby( 3 );^Pk(r)canbeestimatedbytheestimatedPEPofallresidualpacketsinthek-thframeasdiscussedinSection 3.2.1.1 .Wecalltheresultingalgorithmin( 3 )asRMPC-FTDalgorithm. 3.2.2FTDEstimationwithFeedbackAcknowledgement Insomewirelessvideocommunicationsystems,thereceivermaysendthetransmitteranoticationaboutwhetherpacketsarecorrectlyreceived.ThisfeedbackacknowledgementmechanismcanbeutilizedtoimproveFTDestimationaccuracyasshowninAlgorithm 1 Algorithm1. FTDestimationatthetransmitterunderfeedbackacknowledgement. 1)Input:^Pki(r)and^Pki(m)foralli2f1,2,...,Nkg.2)Initializationandupdate.Ifk=1,doinitialization.Ifk>1,updatewithfeedbackinformation.Ifthereareacknowledgementsforpacketsinthe(k)]TJ /F3 11.955 Tf 11.96 0 Td[(1)-thframe,Forj=1:Nk)]TJ /F8 7.97 Tf 6.59 0 Td[(1ifACKforthej-thresidualpacketisreceived,update^Pk)]TJ /F8 7.97 Tf 6.59 0 Td[(1j(r)=0.ifNACKforthej-thresidualpacketisreceived,update^Pk)]TJ /F8 7.97 Tf 6.58 0 Td[(1j(r)=1. 75

PAGE 76

ifACKforthej-thMVpacketisreceived,update^Pk)]TJ /F8 7.97 Tf 6.59 0 Td[(1j(m)=0.ifNACKforthej-thMVpacketisreceived,update^Pk)]TJ /F8 7.97 Tf 6.59 0 Td[(1j(m)=1.EndUpdate^Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1.Else(neitherACKnorNACKisreceived),goto3).3)EstimateDkvia^Dk=^Dk(r)+^k^Dk(m)+^Pk(r)^Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(1+(1)]TJ /F6 11.955 Tf 11.96 0 Td[(k)(1)]TJ /F3 11.955 Tf 13.24 2.65 Td[(^Pk(r))^Dk(p),whichis( 3 ).4)Output:^Dk. Algorithm1hasalowcomputationalcomplexitysince^Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1isupdatedbasedonwhetherpacketsinthe(k)]TJ /F3 11.955 Tf 12.17 0 Td[(1)-thframearecorrectlyreceivedornot.Inamoregeneralcasethattheencodercantolerateafeedbackdelayofdframes,wecouldupdate^Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1basedonthefeedbackacknowledgementsforthe(k)]TJ /F4 11.955 Tf 11.28 0 Td[(d)-thframethroughthe(k)]TJ /F3 11.955 Tf 11.28 0 Td[(1)-thframe.However,thisrequiresextramemoryfortheencodertostoreallthesystemparametersfromthe(k)]TJ /F4 11.955 Tf 11.96 0 Td[(d)-thframetothe(k)]TJ /F3 11.955 Tf 11.96 0 Td[(1)-thframeinordertoupdate^Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1. 3.3Pixel-levelTransmissionDistortionEstimationAlgorithm ThePTDestimationalgorithmissimilartotheFTDestimationalgorithmpresentedinSection 3.2 .However,thevaluesofsomevariablesinthePTDformuladerivedinChapter 2 maybeknownattheencoder.Takingukasanexample,beforethepredictionmodeisselected,thebestmotionvectormvkuofeachpredictionmodeisknownaftermotionestimationisdone;hencetheresidual^ekuandreconstructedpixelvalue^fkuofeachmodearealsoknown.Insuchacase,theseknownvaluescouldbeusedtoreplacethestatisticsofthecorrespondingrandomvariablestosimplifythePTDestimation.Inthissection,wediscusshowtousetheknownvaluestoimprovetheestimationaccuracyandreducethealgorithmcomplexity. 76

PAGE 77

3.3.1EstimationofPTD Inthissection,weconsiderthecasewithnodatapartitioning;hence,Pku=Pku(r)=Pku(m).Forthecasewithslicedatapartitioning,thederivationprocessissimilartothatinthissection. FromChapter 2 ,weknowthatPTDcanbecalculatedby Dku=Dku(r)+Dku(m)+Dku(P)+Dku(c),(3) where Dku(r)=E[("ku)2]Pku(r);(3) Dku(m)=E[(ku)2]Pku(m);(3) Dku(P)=PkuDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+(1)]TJ /F4 11.955 Tf 11.95 0 Td[(Pku)Dku(p);(3) Dku(c)=2Pku(2E["kuku]+2E["kuek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]+2E[kuek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku]);(3) whereDku(p),E[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg)2]forj2f1,...,Jg;Jisthenumberofpreviousencodedframesusedforintermotionsearch;ekufr,mgdenotestheclippingnoiseundertheerroreventthatthepacketiscorrectlyreceived. Ifthevaluesformvku,^ekuand^fkuareknown,giventheerrorconcealmentattheencoder,thevaluesfor"ku=^eku)]TJ /F3 11.955 Tf 12.59 0 Td[(ekuandku=^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)]TJ /F3 11.955 Tf 12.65 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkuarealsoknown.Then,Dku(r)=("ku)2Pku,Dku(m)=(ku)2Pku,andDku(c)=Pku(2"kuku+2"kuE[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]+2kuE[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]).Hence,theformulaforPTDcanbesimpliedto Dku=E[(eku)2]=Pku(("ku+ku)2+2("ku+ku)E[ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku]+Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)+(1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pku)Dku(p).(3) Denote^D()theestimateofD(),anddenote^E()astheestimateofE().Therefore,Dkucanbeestimatedby^Dku=^Pku(("ku+ku)2+2("ku+ku)^E[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]+^Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)+(1)]TJ /F3 11.955 Tf 13.48 2.66 Td[(^Pku)^Dku(p),where^PkucanbeobtainedbythePEPestimationalgorithminSection 3.2 .^Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuistheestimateinthe(k)]TJ /F3 11.955 Tf 12.03 0 Td[(1)-thframeandisstoredforcalculating 77

PAGE 78

Dku.Therefore,theonlyunknownsare^E[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]and^Dku(p),whichcanbecalculatedbythemethodsinSections 3.3.2 and 3.3.3 3.3.2Calculationof^E[eku] Since^E[ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku]fromthe(k)]TJ /F3 11.955 Tf 12.32 0 Td[(1)-thframeisrequiredforcalculating^Dku,weshouldestimatetherstmomentofekuandstoreitforthesubsequentframe.FromChapter 2 ,weknoweku=e"ku+eku+ek)]TJ /F7 7.97 Tf 6.59 0 Td[(j0u+fmvku+eku.ForP-MBs,whenMVpacketiscorrectlyreceived,e"ku=eku=0andek)]TJ /F7 7.97 Tf 6.58 0 Td[(j0u+fmvku=ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku;whenMVpacketisreceivedwitherror,ek)]TJ /F7 7.97 Tf 6.59 0 Td[(j0u+fmvku=ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku,andsinceresidualandMVareinthesamepacket,ekufr,mg=ekufrg=0asprovedinChapter 2 .Therefore,therstmomentofkucanberecursivelycalculatedby E[eku]=Pku("ku+ku+E[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku])+(1)]TJ /F4 11.955 Tf 11.95 0 Td[(Pku)E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg].(3) Consequently,E[eku]canbeestimatedby^E[eku]=^Pku("ku+ku+^E[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku])+(1)]TJ /F3 11.955 Tf 13.42 2.66 Td[(^Pku)^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg]. ForI-MBs,whenthepacketiscorrectlyreceived,eku=0;whenMVpacketisreceivedwitherror,theresultisthesameasforP-MBs.Therefore,therstmomentofkucanberecursivelycalculatedby E[eku]=Pku("ku+ku+E[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]),(3) andE[eku]canbeestimatedby^E[eku]=^Pku("ku+ku+^E[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]). 3.3.3Calculationof^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg]and^Dku(p) FromChapter 2 ,weknowthatforI-MBs,Dku(p)=0;forP-MBs,Dku(p)=kuDk)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvkuanditcanbeestimatedby^Dku(p)=^ku^Dk)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku,where^kuisestimatedby( 3 )withy=^fkuand2=^Dk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku. However,suchcomplexityistoohightobeusedinpredictionmodedecisionsinceeverypixelrequiressuchacomputationforeachmode.Toaddressthis,weleveragethepropertyprovedinProposition 3 todesignalow-complexityandhigh-accuracyalgorithmtorecursivelycalculate^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg]and^Dku(p)forP-MBs. 78

PAGE 79

Proposition3. AssumeH=255andL=0.Thepropagatederrorek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvkuandtheclippingnoiseekufr,mgsatisfy ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg=8>>>>>><>>>>>>:^fku)]TJ /F3 11.955 Tf 11.95 0 Td[(255,ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku<^fku)]TJ /F3 11.955 Tf 11.95 0 Td[(255^fku,ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku>^fkuek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku,otherwise.(3) Proposition 3 isprovedinAppendix B.1 .UsingProposition 3 ,^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg]in( 4 )and^Dku(p)in( 4 )canbeestimatedunderthefollowingthreecases. Case1:If^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]<^fku)]TJ /F3 11.955 Tf 12.55 0 Td[(255,wehave^E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg]=^fku)]TJ /F3 11.955 Tf 12.55 0 Td[(255,and^Dku(p)=^E[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg)2]=(^fku)]TJ /F3 11.955 Tf 11.96 0 Td[(255)2. Case2:If^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]>^fku,wehave^E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg]=^fku,and^Dku(p)=(^fku)2. Case3:If^fku)]TJ /F3 11.955 Tf 11.97 0 Td[(255^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]^fku,wehave^E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg]=^E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku],and^Dku(p)=^E[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)2]. 3.3.4Summary PTDcanberecursivelyestimatedby( 4 )and( 4 )or( 3 );and^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg]and^Dku(p)canbecalculatedbythemethodsinSection 3.3.3 .TheresultingalgorithmiscalledRMPC-PTDalgorithm. 3.4Pixel-levelEnd-to-endDistortionEstimationAlgorithm Thepixel-levelend-to-enddistortion(PEED)foreachpixeluinthek-thframeisdenedbyDku,ETE,E[(fku)]TJ /F5 11.955 Tf 12.19 3.15 Td[(efku)2],wherefkuistheinputpixelvalueattheencoderandefkuisthereconstructedpixelvalueatthedecoder.Thenwehave Dku,ETE=E[(fku)]TJ /F5 11.955 Tf 11.87 3.15 Td[(efku)2]=E[(fku)]TJ /F3 11.955 Tf 12.06 2.66 Td[(^fku+^fku)]TJ /F5 11.955 Tf 11.87 3.16 Td[(efku)2]=E[(fku)]TJ /F3 11.955 Tf 12.06 2.66 Td[(^fku+eku)2]=(fku)]TJ /F3 11.955 Tf 12.05 2.65 Td[(^fku)2+E[(eku)2]+2(fku)]TJ /F3 11.955 Tf 12.06 2.65 Td[(^fku)E[eku]. (3) 79

PAGE 80

Wecallfku)]TJ /F3 11.955 Tf 12.15 2.66 Td[(^fkuquantizationerrorandekutransmissionerror.Whilefku)]TJ /F3 11.955 Tf 12.14 2.66 Td[(^fkudependsonlyonthequantizationparameter(QP),ekumainlydependsonthePEPandtheerrorconcealmentscheme.Ifthevalueof^fkuisknown,thentheonlyunknownsin( 3 )areE[(eku)2]andE[eku],whichcanbeestimatedbythemethodsinSection 3.3 .Wecallthealgorithmin( 3 )asRPMC-PEEDalgorithm. ComparedtoROPEalgorithm[ 4 ],whichestimatestherstmomentandsecondmomentofthereconstructedpixelvalueefku,wehavethefollowingobservations.First,RPMC-PEEDalgorithmestimatestherstmomentandthesecondmomentofreconstructederroreku;therefore,RPMC-PEEDalgorithmismucheasiertobeenhancedtosupporttheaveragingoperationsinH.264,suchasinterpolationlter.Second,estimatingtherstmomentandthesecondmomentofekuinRMPC-PEEDproduceslowerdistortionestimationerrorthanestimatingbothmomentsofefkuinROPE.Third,ourexperimentalresultsshowthatROPEmayproduceanegativevalueastheestimatefordistortion,whichviolatestherequirementthat(true)distortionmustbenon-negative;ourexperimentalresultsalsoshowthatthenegativedistortionestimateiscausedbynotconsideringclipping,whichresultsininaccuratedistortionestimationbyROPE. NotethatinChapter 2 ,weassumetheclippingnoiseattheencoderiszero,thatis,^ku=0.Ifweuse^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+^ekutoreplace^fkuin( 3 ),wemaycalculatethequantizationerrorbyfku)]TJ /F3 11.955 Tf 11.96 0 Td[((^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+^eku)andcalculatethetransmissionerrorby eku=(^fk)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+^eku))]TJ /F5 11.955 Tf 11.87 3.15 Td[(efku=(^fk)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+^eku))]TJ /F3 11.955 Tf 11.95 0 Td[((efk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku+eeku)]TJ /F5 11.955 Tf 12.81 3.16 Td[(eku)=e"ku+eku+ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eku,(3) whichisexactlytheformulafortransmissionerrordecompositioninChapter 2 .Therefore,^kudoesnotaffecttheend-to-enddistortionDku,ETEifweuse^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+^ekutoreplace^fkuincalculatingboththequantizationerrorandthetransmissionerror. 80

PAGE 81

3.5ApplyingRMPC-PEEDAlgorithmtoH.264PredictionModeDecision 3.5.1Rate-distortionOptimizedPredictionModeDecision InH.264specication,therearetwotypesofpredictionmodes,i.e.,interpredictionandintraprediction4.Ininterprediction,thereare7modes,i.e.,modesfor16x16,16x8,8x16,8x8,8x4,4x8,and4x4lumablocks.Inintraprediction,thereare9modesfor4x4lumablocksand4modesfor16x16lumablocks.Hence,thereareatotalof7+9+4=20modestobeselectedinmodedecision.ForeachMB,ourproposedError-ResilientRateDistortionOptimized(ERRDO)modedecisionconsistsoftwosteps.First,R-Dcostiscalculatedby J(!m)=DkETE(!m)+R(!m),(3) whereDkETE=Pu2VkiDku,ETE;Vkiisthesetofpixelsinthei-thMB(orsub-MB)ofthek-thframe;!misthepredictionmode,and!m(!m2f1,2,,20g);R(!m)istheencodedbitrateformode!m;isthepresetLagrangemultiplier.Then,theoptimalpredictionmodethatminimizestherate-distortion(R-D)costisfoundby ^!m=argmin!mfJ(!m)g.(3) IfDkETE(!m)in( 3 )isreplacedbysourcecodingdistortionorquantizationdistortion,wecallitSource-CodingRateDistortionOptimized(SCRDO)modedecision. Using( 3 )and( 3 ),wedesignAlgorithm 2 forERRDOmodedecisioninH.264;Algorithm 2 iscalledRMPC-MSalgorithm. Algorithm2. ERRDOModedecisionforanMBinthek-thframe(k1). 1)Input:QP,PEP.2)Initializationof^E[e0u]and^E[(e0u)2]forallpixelu. 4TherearetwootherencodingmodesforP-MBdenedinH.264,i.e.,skipmodeandI PCMmode.However,theyareusuallynotinvolvedinthePEEDestimationprocess. 81

PAGE 82

3)Formode=1:20(9+4intra,7inter).3a)Ifintramode,calculate^E[eku]by( 3 )forallpixelsintheMB,goto3b),Elseif^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]<^fku)]TJ /F3 11.955 Tf 11.95 0 Td[(255,^E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg]=^fku)]TJ /F3 11.955 Tf 11.96 0 Td[(255,^E[(ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg)2]=(^fku)]TJ /F3 11.955 Tf 11.95 0 Td[(255)2,Elseif^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]>^fku^E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg]=^fku,^E[(ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg)2]=(^fku)2,Else^E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg]=^E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku],^E[(ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg)2]=^E[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)2],Endcalculate^E[eku]by( 4 )forallpixelsintheMB,3b)calculate^Dkuby( 4 )forallpixelsintheMB,3c)estimateDku,ETEby( 3 )forallpixelsintheMB,3d)calculateR-Dcostvia( 3 )foreachmode,EndVia( 3 ),selectthemodewithminimumR-DcostastheoptimalmodefortheMB.5)Output:thebestmodefortheMB. UsingTheorem 3.1 ,wecandesignanotherERRDOmodedecisionalgorithmthatproducesthesamesolutionasthatofAlgorithm 2 ,asProposition 4 states. Theorem3.1. (DecompositionTheorem)Ifthereisnoslicedatapartitioning,end-to-enddistortioncanbedecomposedintoamode-dependenttermandamode-independent 82

PAGE 83

term,i.e., Dku,ETE(!m)=Dku, ETE(!m)+Cku.(3) whereCkuisindependentof!mand Dku, ETE(!m)=(1)]TJ /F4 11.955 Tf 11.95 0 Td[(Pku)f(fku)]TJ /F3 11.955 Tf 12.05 2.65 Td[(^fku)2+Dku(p)+2(fku)]TJ /F3 11.955 Tf 12.05 2.65 Td[(^fku)E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg]g.(3) Theorem 3.1 isprovedinAppendix B.2 UsingTheorem 3.1 ,weonlyneedtochangetwoplacesinAlgorithm 2 toobtainanewalgorithm,whichwecallAlgorithmA:rst,replaceStep3c)inAlgorithm 2 byestimateDku, ETEby( 3 )forallpixelsintheMB;second,replace3d)inAlgorithm 2 bycalculateR-DcostviaDk ETE(!m)+R(!m)foreachmode,whereDk ETE=Pu2VkiDku, ETE. Proposition4. Ifthereisnoslicedatapartitioning,AlgorithmAandAlgorithm 2 producethesamesolution,i.e.,^!m=argmin!mf^DkETE(!m)+R(!m)g=argmin!mf^Dk ETE(!m)+R(!m)g. Proposition 4 isprovedinAppendix B.3 NotethatDk ETEin( 3 )isnotexactlytheend-to-enddistortion;butthedecompositionin( 3 )canhelpreducethecomplexityofsomeestimationalgorithms,forexample,LLNalgorithm[ 29 ]. 3.5.2ComplexityofRMPC-MS,ROPE,andLLNAlgorithm Inthissubsection,wecomparethecomplexityofRMPC-MSalgorithmwiththatoftwopopularmodedecisionalgorithms,namely,ROPEalgorithmandLLNalgorithm,whicharealsobasedonpixel-leveldistortionestimation.Tomakeafaircomparison,thesameconditionsshouldbeusedforallthethreealgorithms.Assumeallthethreealgorithmsuseanerrorconcealmentschemethatconcealsanerroneouspixelbythe 83

PAGE 84

pixelinthesamepositionofthepreviousframe;then,eku=0andmvku=0;hence,"ku+ku=fku)]TJ /F3 11.955 Tf 12.05 2.65 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u. Here,thecomplexityisquantiedbythenumberofadditions(ADDs)andmultiplications(MULs)5.Ifasubroutine(orthesamesetofoperations)isinvokedmultipletimes,itiscountedonlyoncesincethetemporaryresultissavedinthememory;forexample,"ku+kuin( 4 )and( 4 )iscountedasoneADD.Asubstractioniscountedasanaddition.Weonlyconsiderpixel-leveloperations;block-leveloperations,forexampleMVaddition,areneglected.Weignorethecomplexityofthosebasicoperationssincetheircomplexityisthesameforallthethreealgorithms,suchasmotioncompensation. 3.5.2.1RMPC-MSalgorithm LetusrstconsiderthecomplexityofRMPC-MSalgorithm,i.e.Algorithm 2 ,forintermodes.InAlgorithm 2 ,theworstcaseis^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]<^fku)]TJ /F3 11.955 Tf 12.51 0 Td[(255.Underthiscase,thereisoneADDandonesquare,i.e.MUL.Theothertwocasesrequireonlytwocopyoperations,andsoareneglected.Notethat^fku)]TJ /F3 11.955 Tf 13.63 2.65 Td[(^E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku]255withhighprobability,thatis,^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]<^fku)]TJ /F3 11.955 Tf 12.35 0 Td[(255isrelativelyrare.Therefore,inmostcases,thereareonlytwocopyoperationsintheloop.Calculatingthesecondmomentofekuneeds4ADDsand4MULsasin( 4 ).Similarly,calculatingtherstmomentofekuneeds2ADDsand2MULsasin( 4 ).Finally,calculatingtheend-to-enddistortionneeds3ADDsand2MULsasin( 3 ).Hence,theworstcaseofcalculatingtheend-to-enddistortionforeachpixelis10ADDsand9MULs.Notethatinmostcases,thecomplexityis9ADDsand8MULsforintermodesasshowninTable 3-1 NotethatsincePkuisthesameforallpixelsinoneMB,wedonotneedtocalculate1)]TJ /F4 11.955 Tf 12.38 0 Td[(Pkuforeachpixel.Multiplyingby2canbeachievedbyashiftoperation;soitisnotcountedasoneMUL. 5Thoseminoroperations,suchasmemorycopy,shift,andconditionalstatement,areneglectedforallalgorithms. 84

PAGE 85

ForIntramodes,weknowthatek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg=0fromChapter 2 .Therefore,thecomplexityofintramodeisreducedto3ADDsand3MULsin( 4 ),1ADDsand1MULsin( 3 ).Asaresult,theend-to-enddistortionforeachpixelis7ADDsand6MULsforeachintramode. InH.264,thereare7intermodesand13intramodes;thereforethereareatotalof154ADDsand134MULsforeachpixelinmostcases.Intheworstcase,thereareatotalof161additionsand141MULsforeachpixel,wheretheadditionalcomputationcomesfromtheconsiderationofclippingeffect. MemoryRequirementAnalysis:Toestimatetheend-to-enddistortionbyAlgorithm 2 ,therstmomentandthesecondmomentofthereconstructederrorofthebestmodeshouldbestoredafterthemodedecision.Therefore,2unitsofmemoryarerequiredtostorethosetwomomentsforeachpixel.Notethattherstmomenttakesvaluesinf)]TJ /F3 11.955 Tf 15.28 0 Td[(255,)]TJ /F3 11.955 Tf 9.3 0 Td[(254,,255g,i.e.,8bitsplus1signbitperpixel,andthesecondmomenttakesvaluesinf0,1,,2552g,i.e.,16bitsperpixel. 3.5.2.2ROPEalgorithm InROPEalgorithm,themomentestimationformulaeforinterpredictionandintrapredictionaredifferent.Forintermodes,calculatingtherstmomentneeds2ADDsand2MULs;calculatingthesecondmomentneeds3ADDsand4MULs;calculatingtheend-to-enddistortionneeds2ADDsand2MULs.Forintramodes,calculatingtherstmomentneeds1ADDand2MULs;calculatingthesecondmomentneeds1ADDand3MULs.Hence,anintermodeneeds7ADDsand8MULs;anintramodeneeds4ADDsand7MULs.ForH.264,thetotalcomplexityforeachpixelis101ADDsand147MULs. NotethatwhenweimplementROPEinJM16.0,wendthatROPEalgorithmcausesout-of-rangevaluesforboththerstmomentandthesecondmomentduetotheneglectofclippingnoise.ExperimentalresultsshowthatROPEmayproduceanegativevalueastheestimatefordistortion,whichviolatestherequirementthat(true)distortion 85

PAGE 86

mustbenon-negative.Hence,inapracticalsystemtheestimatedresultfromROPEalgorithmneedstobeclippedintoalegitimatevalue;thiswillincurahighercomplexity. MemoryRequirementAnalysis:Toestimatetheend-to-enddistortionbyROPEalgorithm,therstmomentandthesecondmomentofthereconstructedpixelvalueofthebestmodeshouldbestoredafterthemodedecision.Therefore,2unitsofmemoryarerequiredtostorethetwomomentsforeachpixel.Therstmomenttakesvaluesinf0,1,,255g,i.e.,8bitsperpixel;thesecondmomenttakesvaluesinf0,1,,2552g,i.e.,16bitsperpixel.NotethatintheoriginalROPEalgorithm[ 4 ],thevaluesofthetwomomentsarenotboundedsincethepropagatederrorsareneverclipped. 3.5.2.3LLNalgorithm InJM16.0,LLNalgorithmusesthesamedecompositionmethodasTheorem 3.1 formodedecision[ 29 ].Insuchacase,forintermodes,reconstructingthepixelvalueinonesimulateddecoderneeds1ADD;calculatingtheend-to-enddistortionneeds1ADDandoneMUL.Forintramodes,thereisnoadditionalreconstructionforallsimulateddecoderssincethenewlyinducederrorsarenotconsidered;therefore,calculatingtheend-to-enddistortionneeds1ADDand1MUL.SupposethenumberofsimulateddecodersattheencoderisNd,thecomplexityforLLNalgorithmis27NdADDsand20NdMULs.ThedefaultnumberofsimulateddecodersinJM16.0is30,whichmeansthecomplexityforLLNalgorithmis810ADDsand600MULs.ThirtysimulateddecodersissuggestedinRef.[ 6 ].Inourexperiment,wendthatifthenumberofsimulateddecodersislessthan30,theestimateddistortionexhibitshighdegreeofrandomness(i.e.,havingalargevariance);however,ifthenumberofsimulateddecodersislargerthan50,theestimateddistortionisquitestable(i.e.,havingasmallvariance). NotethattheerrorconcealmentoperationsinLLNalgorithmarerequiredbutnotcountedinthecomplexitysinceitisdoneafterthemodedecision.However,evenwithoutconsideringtheextraerrorconcealmentoperations,thecomplexityofLLNalgorithmisstillmuchhigherthanRMPC-MSandROPE.Increasingthenumberof 86

PAGE 87

Table3-1. ComplexityComparison computationalcomplexity memoryrequirement intermode 9ADDs,8MULs (worst10ADDs,9MULs) RMPC-MS intramode 7ADDs,6MULs 25bits/pixel totalcomplexity 154ADDs,134MULs (worst161ADDs,141MULs) intermode 7ADDs,8MULs (morewithclipping) ROPE intramode 4ADDs,7MULs 24bits/pixel totalcomplexity 101ADDs,147MULs (morewithclipping) intermode 2NdADDs,NdMULs (morewitherrorconcealment) LLN intramode NdADDs,NdMULs 8Ndbits/pixel totalcomplexity 27NdADDs,20NdMULs (morewitherrorconcealment) simulateddecodersattheencodercanimproveestimationaccuracybutatthecostoflinearincreaseofcomputationalcomplexity. MemoryRequirementAnalysis:Toestimatetheend-to-enddistortionbyLLNalgorithm,foreachsimulateddecoder,eachreconstructedpixelvalueofthebestmodeshouldbestoredafterthemodedecision.Therefore,theencoderneedsNdunitsofmemorytostorethereconstructedpixelvalue.Areconstructedpixeltakesvaluesinf0,1,,255g,i.e.,8Ndbitsperpixel. Table 3-1 showsthecomplexityofthethreealgorithms. 3.6ExperimentalResults InSection 3.6.1 ,wecomparetheestimationaccuracyofRMPC-FTDalgorithmtothatoftheexistingmodelsunderdifferentchannelconditions;wealsocomparetheirrobustnessagainstimperfectestimateofPEP.InSection 3.6.2 ,wecomparetheR-D 87

PAGE 88

performanceofRMPC-MSandexistingmodedecisionalgorithmsforH.264;wealsocomparethemunderinterpolationlteranddeblockinglter. Tocollectthestatisticsandtestthealgorithms,allpossiblechannelconditionsshouldbetestedforeveryvideosequence6.However,estimatingtransmissiondistortionandcollectingthestatisticsofeachvideosequenceunderallpossibleerroreventsaretedioustaskssinceonlycommandlineinterfaceorcongurationleisavailableincurrentopensourceH.264referencecode,suchasJMandx264.Toanalyzethestatisticsandverifyouralgorithm,ourlabhasdevelopedasoftwaretool,calledVideoDistortionAnalysisTool(VDAT),whichprovidesafriendlyGraphicalUserInterface(GUI).VDATimplementschannelsimulator,supportsdifferentvideocodec,computesthestatistics,andsupportsseveraldistortionestimationalgorithms7.VDATisusedinalltheexperimentsinthissection. 3.6.1EstimationAccuracyandRobustness Inthissection,weuseAlgorithm 1 toestimateFTDandcompareitwithStuhlmuller'smodel[ 8 ]andDani'smodel[ 9 ].Toevaluateestimationaccuracy,wecomparetheestimateddistortionofdifferentalgorithmswithtruedistortionfor50framesunderthecaseofnoacknowledgementfeedback. 3.6.1.1Experimentsetup Toimplementtheestimationalgorithms,alltransmissiondistortionrelatedstatisticsshouldbecollectedforallrandomvariables,suchasresidual,motionvector,reconstructedpixelvalue,residualconcealmenterror,MVconcealmenterror,propagatederror,clippingnoise.Allsuchstatisticsarecollectedfromvideocodec 6http://trace.eas.asu.edu/yuv/index.html7InterestedreaderscandownloadalltheVDATsourcecodesat:http://users.ecel.u.edu/zhifeng/project/VDAT/index.htm. 88

PAGE 89

JM14.08.AlltestedvideosequencesareinCIFformat,andeachframeisdividedintothreeslices.Tosupporttheslicedatapartitioning,weusetheextendedproleasdenedinH.264specicationAnnexA[ 22 ].Toprovideunequalerrorprotection(UEP),weletMVpacketsexperiencelowerPEPthanresidualpackets.TherstframeofeachcodedvideosequenceisanI-frame,andthesubsequentframesareallP-frames.Intheexperiment,welettherstI-framebecorrectlyreceived,andallthefollowingP-framesgothroughanerror-pronechannelwithcontrollablePEP.WesetQP=28foralltheframes. EachvideosequenceistestedunderseveralchannelconditionswithUEP.Duetothespacelimit,weonlypresenttheexperimentalresultsforvideosequences`foreman'and`stefan'.Experimentalresultsforothervideosequencescanbefoundonline9.Foreachsequence,twowirelesschannelconditionsaretested:forgoodchannelcondition,residualPEPis2%andMVPEPis1%;forpoorchannelcondition,residualPEPis10%andMVPEPis5%.ForeachPEPsettingofeachframe,wedo600simulationsandtaketheaveragetomitigatetheeffectofrandomnessofsimulatedchannelsoninstantaneousdistortion. 3.6.1.2Estimationaccuracyofdifferentestimationalgorithms Fig. 3-1 showstheestimationaccuracyofRMPC-FTDalgorithm,Stuhlmuller'smodelinRef.[ 8 ]andDani'smodelinRef.[ 9 ]forsequence`foreman'.Fig. 3-2 showstheirestimationaccuracyforsequence`stefan'.WecanseethatRMPC-FTDalgorithmachievesthemostaccurateestimate.SincethesuperpositionalgorithminStuhlmuller'smodelneglectstheeffectofclippingnoiseandnegativecorrelationbetweenMVconcealmenterrorandpropagatederror,itover-estimatestransmissiondistortionasshowninFig. 3-2 .However,sincetheclippingeffectandthecorrelationcaused 8http://iphome.hhi.de/suehring/tml/download/old jm/jm14.0.zip9http://www.mcn.ece.u.edu/public/zhifeng/project/VDAT/journal/ 89

PAGE 90

distortionissmallforlowmotionsequenceunderlowPEPasprovedinChapter 2 ,linearmodelisquiteaccurateasshowninFig. 3-1 (a).Noticethatinforemansequenceundergoodchannel,theestimateddistortiondifferentfromgroundtruthisonlyaboutMSE=12afteraccumulatedwith50frameswithoutfeedback.InRef.[ 9 ],authorsclaimthatthelargerthefractionofpixelsinthereferenceframetobeusedasreferencepixels,thelargerthetransmissionerrorspropagatedfromthereferenceframe.However,duetorandomnessofmotionvectors,theprobabilitythatapixelwitherrorisusedasreferenceisthesameastheprobabilitythatapixelwithouterrorisusedasreference.Therefore,thenumberofpixelsinthereferenceframebeingusedformotionpredictionhasnothingtodowiththefadingfactor.Asaresult,thealgorithminRef.[ 9 ]under-estimatestransmissiondistortionasshowninFig. 3-1 andFig. 3-2 (a)(b) Figure3-1. TransmissiondistortionDkvs.frameindexkfor`foreman':(a)goodchannel,(b)poorchannel. Inourexperiment,weobservethat1)thehigherthepropagateddistortion,thesmallerthepropagationfactor;and2)thehigherpercentageofreconstructedpixelvaluesnear0or255,thesmallerthepropagationfactor.ThesetwophenomenaoncemoreverifythatthepropagationfactorisafunctionofallsamplesofreconstructedpixelvalueandsamplevarianceofpropagatederrorasprovedinChapter 2 .Thesephenomenacouldbeexplainedby( 3 )inthat1)isadecreasingfunctionofbfor 90

PAGE 91

(a)(b) Figure3-2. TransmissiondistortionDkvs.frameindexkfor`stefan':(a)goodchannel,(b)poorchannel. Figure3-3. TransmissiondistortionDkvs.PEPfor`foreman'. b>0;2)isanincreasingfunctionofyfor0y127andadecreasingfunctionofyfor128y255.Wealsonotethatalargersamplevarianceofpropagatederrorcausestobelesssensitivetothechangeofreconstructedpixelvalue,whilealargerdeviationofreconstructedpixelvaluefrom128causestobelesssensitivetothechangeofsamplevarianceofpropagatederror. Tofurtherstudyestimationaccuracy,wetesttheestimationalgorithmsundermanydifferentchannelconditions.Fig. 3-3 andFig. 3-4 showtheestimationaccuracyunderPEPvaryingfrom1%to10%.Inbothgures,RMPC-FTDalgorithmachievesthemostaccuratedistortionestimationunderallchannelconditions. 91

PAGE 92

Figure3-4. TransmissiondistortionDkvs.PEPfor`stefan'. 3.6.1.3Robustnessofdifferentestimationalgorithms InSection 3.6.1.2 ,weassumePEPisperfectlyknownattheencoder.However,inarealwirelessvideocommunicationsystem,PEPisusuallynotperfectlyknownattheencoder;i.e.,thereisarandomestimationerrorbetweenthetruePEPandtheestimatedPEP.Hence,itisimportanttoevaluatetherobustnessoftheestimationalgorithmsagainstPEPestimationerror.TosimulateimperfectPEPestimation,foragiventruePEPdenotedbyPtrue,weassumetheestimatedPEPisarandomvariableandisuniformlydistributedin[0,2Ptrue];e.g.,ifPtrue=10%,theestimatedPEPisuniformlydistributedin[0,20%]. Figs. 3-5 and 3-6 showtheestimationaccuracyofthethreealgorithmsfor`foreman'and`stefan',respectively,underimperfectknowledgeofPEPattheencoder.Fromthetwogures,itisobservedthatcomparedtothecaseunderperfectknowledgeofPEPattheencoder,forbothStuhlmuller'smodelandDani'smodel,imperfectknowledgeofPEPmaycauseincreaseordecreaseofthegapbetweentheestimateddistortionandthetruedistortion.Specically,forStuhlmuller'smodel,ifthePEPisunder-estimated,thegapbetweentheestimateddistortionandthetruedistortiondecreases,comparedtothecaseunderperfectknowledgeofPEP;forDani'smodel,ifthePEPisover-estimated,thegapbetweentheestimateddistortionandthetruedistortiondecreases,comparedtothecaseunderperfectknowledgeofPEP.Incontrast,RMPC-FTDalgorithmismore 92

PAGE 93

robustagainstPEPestimationerror,andprovidesmoreaccuratedistortionestimatethanStuhlmuller'smodelandDani'smodel. (a)(b) Figure3-5. TransmissiondistortionDkvs.frameindexkfor`foreman'underimperfectknowledgeofPEP:(a)goodchannel,(b)poorchannel. (a)(b) Figure3-6. TransmissiondistortionDkvs.frameindexkfor`stefan'underimperfectknowledgeofPEP:(a)goodchannel,(b)poorchannel. 3.6.2R-DPerformanceofModeDecisionAlgorithms Inthissubsection,wecomparetheR-DperformanceofAlgorithm 2 withthatofROPEandLLNalgorithmsformodedecisioninH.264.Tocompareallalgorithmsunderthemulti-referencepicturemotioncompensatedprediction,wealsoenhancetheoriginalROPEalgorithm[ 4 ]withmulti-referencecapability. 93

PAGE 94

3.6.2.1Experimentsetup JM16.0encoderanddecoderisusedintheexperiments.TosupportmoreadvancedtechniquesinH.264,weusethehighproledenedinH.264specicationAnnexA[ 22 ].Weconductexperimentsforveschemes,thatis,threeERRDOschemes,i.e.,RMPC-MS,LLN,ROPE;randomintraupdate;anddefaultSCRDOschemewithnotransmissiondistortionestimation.AllthetestedvideosequencesareinCIFresolutionwith30fps.EachcodedvideosequenceistestedunderdifferentPEPsettingsfrom0.5%to5%.Eachvideosequenceiscodedforitsrst100frameswith3slicesperframe.Theerrorconcealmentmethodusedistocopythepixelvalueinthesamepositionofthepreviousframe.Therstframeisassumedtobecorrectlyreceived. Theencodersettingisgivenasbelow.Noslicedatapartitioningisused;constrainedintrapredictionisenabled;thenumberofreferenceframesis3;B-MBsarenotincluded;only4x4transformisused;CABACisenabledforentropycoding;inLLNalgorithm,thenumberofsimulateddecodersis30. 3.6.2.2R-Dperformanceundernointerpolationlterandnodeblockinglter Intheexperimentsofthissubsection,boththedeblockinglterandtheinterpolationlterwithfractionalMVinH.264aredisabled.Duetothespacelimit,weonlyshowtheplotofPSNRvs.bitrateforvideosequences`foreman'and`football'underPEP=2%andPEP=5%,withratecontrolenabled.Figs. 3-7 and 3-8 showPSNRvs.bitratefor`foreman'and`football',respectively.TheexperimentalresultsshowthatRMPC-MSachievesthebestR-Dperformance;LLNandROPEachievessimilarperformanceandthesecondbestR-Dperformance;therandomintraupdatescheme(denotedby`RANDOM')achievesthethirdbestR-Dperformance;theSCRDOscheme(denotedby`NO EST')achievestheworstR-Dperformance. LLNhaspoorerR-DperformancethanRMPC-MS;thismaybebecause30simulateddecodersarestillnotenoughtoachievereliabledistortionestimatealthoughLLNwith30simulateddecodersalreadyincursmuchhighercomplexitythanRMPC-MS. 94

PAGE 95

(a)(b) Figure3-7. PSNRvs.bitratefor`foreman',withnointerpolationlterandnodeblockinglter:(a)PEP=2%,(b)PEP=5%. (a)(b) Figure3-8. PSNRvs.bitratefor`football',withnointerpolationlterandnodeblockinglter:(a)PEP=2%,(b)PEP=5%. ThereasonwhyRMPC-MSachievesbetterR-DperformancethanROPE,isduetotheconsiderationofclippingnoiseinRMPC-MS.Debugmessagesshowthat,withoutconsideringtheclippingnoise,ROPEover-estimatestheend-to-enddistortionforintermodes;henceROPEtendstoselectintramodesmoreoftenthanRMPC-MSandLLN,whichleadstohigherencodingbitrateinROPE;asaresult,thePSNRgainachievedbyROPEiscompromisedbyitshigherbitrate.Toverifythisconjecture,wetestall 95

PAGE 96

sequencesunderthesameQuantizationParameter(QP)settingswithoutratecontrol.weobservethatROPEalgorithmalwaysproduceshigherbitratethanotherschemes. Table 3-2 showstheaveragePSNRgain(indB)ofRMPC-MSoverROPEandLLNfordifferentvideosequencesanddifferentPEP.TheaveragePSNRgainisobtainedbythemethodinRef.[ 30 ],whichmeasuresaveragedistance(inPSNR)betweentwoR-Dcurves.FromTable 3-2 ,weseethatRMPC-MSachievesanaveragePSNRgainof1.44dBoverROPEfor`foreman'underPER=5%;anditachievesanaveragePSNRgainof0.89dBoverLLNfor`foreman'sequenceunderPEP=1%. Table3-2. AveragePSNRgain(indB)ofRMPC-MSoverROPEandLLN Sequence PEP RMPCvs.ROPE RMPCvs.LLN coastguard 5% 0.75 0.04 2% 0.26 0.23 1% 0.15 0.20 0.5% 0.16 0.17 football 5% 0.88 0.42 2% 0.26 0.22 1% 0.19 0.28 0.5% 0.22 0.15 foreman 5% 1.44 0.28 2% 0.74 0.53 1% 0.61 0.89 0.5% 0.30 0.65 mobile 5% 0.51 0.29 2% 0.14 0.23 1% 0.15 0.28 0.5% 0.10 0.19 3.6.2.3R-Dperformancewithinterpolationlteranddeblockinglter InH.264,interpolationlterprovidesnotableobjective(PSNR)gainanddeblockinglterprovidesnotablesubjectivegain.TosupporttheinterpolationlterwithfractionalMVinH.264[ 20 ],weextendAlgorithm 2 byusingthenearestneighbortoapproximate 96

PAGE 97

thereferencepixelpointedbyafractionalMV.Inaddition,deblockinglterisalsoenabledinJM16.0tocompareRMPC-MS,ROPEandLLNalgorithms. NotethatbothRMPC-MSandROPEarederivedwithoutconsideringlteringoperations.Duetohighspatialcorrelationbetweenadjacentpixels,theaveragingoperationinducedbyalterwillproducemanycross-correlationtermsforestimatingdistortioninasubpixelposition.Yangetal.[ 17 ]enhancetheoriginalROPEalgorithmwithinterpolationlterinH.264.However,theiralgorithmrequires1squarerootoperation,1exponentiationoperation,and2multiplicationoperationsforcalculatingeachcross-correlationterm.Sinceasix-tapinterpolationlterisusedinH.264forsubpixelaccuracyofmotionvector,thereare15cross-correlationtermsforcalculatingeachsubpixeldistortion.Therefore,thecomplexityoftheiralgorithmisveryhighandmaynotbesuitableforreal-timeencoding.Inthissubsection,weuseaverysimplebutR-Defcientmethodtoestimatesubpixeldistortion.Specically,wechoosethenearestintegerpixelaroundthesubpixel,andusethedistortionofthenearestintegerpixelastheestimateddistortionforthesubpixel.NotethatthissimplemethodisnotaimedatextendingRMPC-MSandROPEalgorithms,butjusttocomparetheR-DperformancesofthesetwoalgorithmsforH.264withfractionalMVformotioncompensation. WerstshowtheexperimentresultswithinterpolationlterbutwithnodeblockinglterasinFigs. 4-1 and 3-10 .FromFigs. 4-1 and 3-10 ,weobservethesameresultasshowninSection 4.3.2 :RMPC-MSachievesbetterR-DperformancethanLLNandROPEalgorithms.FromFigs. 4-1 and 3-10 ,wealsocanseethateachofthevealgorithmsachieveshigherPSNRthanitscorrespondingschemewithnointerpolationlter;thismeansthesimplemethodisvalid.WealsoobservefromTable 4-1 thatinthiscase,RMPC-MSachievesanaveragePSNRgainof2.97dBoverROPEforsequence`mobile'underPEP=0.5%;anditachievesanaveragePSNRgainof1.13dBoverLLNfor`foreman'underPEP=1%. 97

PAGE 98

(a)(b) Figure3-9. PSNRvs.bitratefor`foreman',withinterpolationandnodeblocking:(a)PEP=2%,(b)PEP=5%. (a)(b) Figure3-10. PSNRvs.bitratefor`football',withinterpolationandnodeblocking:(a)PEP=2%,(b)PEP=5%. WealsoshowtheexperimentresultswithbothinterpolationlteranddeblockinglterasshowninFigs. 3-11 and 3-12 .ItisinterestingtoseethateachofthevealgorithmswithinterpolationlteranddeblockinglterachievespoorerR-Dperformancethanthecorrespondingonewithinterpolationlterandnodeblockinglter.Thatis,addingdeblockinglterdegradestheR-Dperformanceofeachalgorithmsincetheirestimateddistortionsbecomelessaccurate.Inthiscase,ROPEsometimesperformsbetterthanRMPC-MS;thiscanbeseeninFig. 3-12 ,whichisalsotheonlycasewe 98

PAGE 99

Table3-3. AveragePSNRgain(indB)ofRMPC-MSoverROPEandLLNunderinterpolationltering Sequence PEP RMPCvs.ROPE RMPCvs.LLN coastguard 5% 0.49 0.23 2% 0.38 0.28 1% 0.43 0.38 0.5% 0.56 0.31 football 5% 0.45 0.27 2% 0.24 0.38 1% 0.25 0.35 0.5% 0.30 0.23 foreman 5% 1.51 0.56 2% 1.25 0.95 1% 1.20 1.13 0.5% 1.25 1.07 mobile 5% 0.92 0.30 2% 1.64 0.57 1% 2.58 0.35 0.5% 2.97 0.33 haveobservedthatROPEperformsbetterthanRMPC-MS.ThismaybebecauseRMPC-MShasahigherpercentageofintermodesthanROPE.SincethedeblockingoperationisexecutedaftertheerrorconcealmentasinJM16.0,forintraprediction,deblockinglteronlyaffectstheestimateddistortionifthepacketislost;forinterprediction,deblockinglteralwaysimpactstheestimateddistortion.Therefore,theestimationaccuracyforinterpredictionsuffersfromdeblockingltermorethanthatforintraprediction.Thus,itislikelythatmoreintermodesinRMPC-MScausehigherPSNRdropinFig. 3-12 99

PAGE 100

(a)(b) Figure3-11. PSNRvs.bitratefor`foreman',withinterpolationanddeblocking:(a)PEP=2%,(b)PEP=5%. (a)(b) Figure3-12. PSNRvs.bitratefor`football',withinterpolationanddeblocking:(a)PEP=2%,(b)PEP=5%. 100

PAGE 101

CHAPTER4THEEXTENDEDRMPCALGORITHMFORERRORRESILIENTRATEDISTORTIONOPTIMIZEDMODEDECISION Inthischapter,Werstproveanewtheoremforcalculatingthesecondmomentofaweightedsumofcorrelatedrandomvariableswithouttherequirementoftheirprobabilitydistribution.Then,weapplythetheoremtoextendtheRMPC-MSalgorithminChapter 3 tosupportthesubpixel-levelMeanSquareError(MSE)distortionestimation. 4.1AnOverviewonSubpixel-levelEnd-to-endDistortionEstimationforaPracticalVideoCodec Existingpixel-levelalgorithms,e.g.,theRMPCalgorithm,arebasedontheintegerpixelMVassumptiontoderiveanestimateofDku.Therefore,theirapplicationinstate-of-the-artencodersislimitedduetothepossibleuseoffractionalmotioncompensation.FortheRMPCalgorithms,iftheMVofoneblockforencodingisfractional,theMVhastoberoundedtothenearestinteger.ThisblockwillusethereferenceblockpointedtobytheroundedMVasareference.However,instate-of-the-artcodecs,suchasH.264[ 22 ]andHEVCproposals[ 31 ],aninterpolationlterisusedtointerpolateareferenceblockiftheMVisfractional.Therefore,thedistortionofnearestneighborapproximationisnotoptimalforsuchanencoder.Asaresult,weneedtoextendtheexistingRMPCalgorithmtooptimallyestimatethedistortionforblockswithinterpolationltering. Somesubpixel-levelend-to-enddistortionestimationalgorithmshavebeenproposedtoassistmodedecisionasinRef.[ 5 17 32 ].IntheH.264/AVCJMreferencesoftware[ 33 ],theLLNalgorithmproposedinRef.[ 5 ]isadoptedtoestimatetheend-to-enddistortionformodedecision.However,intheLLNalgorithmmoredecodersleadtohighercomputationalcomplexityandlargermemoryrequirements.AlsoforthesamevideosequenceandthesamePEP,differentencodersmayhavedifferentestimateddistortionsduetotherandomlyproducederroreventsateachencoder.InRef.[ 32 ],theauthorsextendROPEfortheH.264encoderbyusingtheupperbound, 101

PAGE 102

obtainedfromtheCauchy-Schwarzapproximation,toapproximatethecross-correlationterms.However,suchanapproximationrequiresveryhighcomplexity.Forexample,foranN-taplterinterpolation,eachsubpixelrequiresNintegermultiplications1forcalculatingthesecondmomentterms;N(N)]TJ /F3 11.955 Tf 12.72 0 Td[(1)=2oating-pointmultiplicationsandN(N)]TJ /F3 11.955 Tf 12.9 0 Td[(1)=2squarerootoperationsforcalculatingthecross-correlationterms;andN(N)]TJ /F3 11.955 Tf 13.04 0 Td[(1)=2+N)]TJ /F3 11.955 Tf 13.04 0 Td[(1additionsand1shiftforcalculatingtheestimateddistortion.Ontheotherhand,theupperboundapproximationisnotaccurateforpracticalvideosequencessinceitassumesthatcorrelationcoefcientis1,foranytwoneighboringpixels.InRef.[ 17 ],authorsproposesomecorrelationcoefcientmodelstoapproximatethecorrelationcoefcientoftwopixelsasfunctions,e.g.,anexponentiallydecayingfunction,oftheirdistance.However,duetotherandombehaviorofindividualpixelsamples,thestatisticalmodeldoesnotproduceanaccuratepixel-leveldistortionestimate.Inaddition,suchcorrelationcoefcientmodelapproximationsincurextracomplexitycomparedtotheCauchy-Schwarzupperboundapproximation,i.e.,theyneedadditionalN(N)]TJ /F3 11.955 Tf 12.95 0 Td[(1)=2exponentialoperationsandN(N)]TJ /F3 11.955 Tf 12.95 0 Td[(1)=2oating-pointmultiplicationsforeachsubpixel.Therefore,thecomplexityincurredisprohibitivelyhighforreal-timevideoencoders.Ontheotherhand,sinceboththeCauchy-Schwarzupperboundapproximationandthecorrelationcoefcientmodelapproximationneedtheoating-pointmultiplications,additionalround-offerrorsareunavoidable,whichfurtherreducetheirestimationaccuracy. InChapter 2 ,weproposeadivide-and-conquermethodtoquantifytheeffectsof1)residualconcealmenterror,2)MotionVector(MV)concealmenterror,3)propagationerrorandclippingnoise,and4)correlationsbetweenanytwoofthem,ontransmission 1Onecommonmethodtosimplifythemultiplicationofanintegervariableandafractionalconstantisasbelow:rstscaleupthefractionalconstantbyacertainfactor;rounditofftoaninteger;thendointegermultiplication;nallyscaledowntheproduct. 102

PAGE 103

distortion.Basedonourtheoreticalresults,weproposedtheRMPCalgorithminChapter 3 forrate-distortionoptimizedmodedecisionwithpixel-levelend-to-enddistortionestimation.Sincethecorrelationbetweenthetransmissionerrorsofneighboringpixelsismuchsmallerandmorestablethanthecorrelationbetweenthereconstructedvaluesofneighboringpixels,theRMPCalgorithmiseasierthanROPEtobeextendedforsupportingsubpixel-levelend-to-enddistortionestimation.Inthischapter,wersttheoreticallyderivethesecondmomentofaweightedsumofcorrelatedrandomvariablesasaclosed-formfunctionofthesecondmomentsofthoseindividualrandomvariables.Thenweapplythisresulttodesignaverylowcomplexitybutaccuratealgorithmformodedecision.ThisalgorithmisreferredtoasExtendedRMPC(ERMPC).TheERMPCalgorithmonlyrequiresNintegermultiplications,N)]TJ /F3 11.955 Tf 12.29 0 Td[(1additions,and1shifttocalculatethesecondmomentforeachsubpixel.Experimentalresultsshowthat,ERMPCachievesanaveragePSNRgainof0.25dBovertheexistingRMPCalgorithmforthe`mobile'sequencewhenPEPequals2%;andERMPCachievesanaveragePSNRgainof1.34dBoverthetheLLNalgorithmforthe`foreman'sequencewhenPEPequals1%. Therestofthischapterisorganizedasfollows.InSection 4.2 ,werstderivethegeneraltheoremforthesecondmomentofaweightedsumofcorrelatedrandomvariables,andthenapplythistheoremtodesignalow-complexityandhigh-accuracyalgorithmformodedecision.Section 5.5 showstheexperimentalresults,whichdemonstratesthebetterR-DperformanceandsubjectiveperformanceoftheERMPCalgorithmoverexistingalgorithmsforH.264modedecisioninerrorproneenvironments. 4.2TheExtendedRMPCAlgorithmforModeDecision Inthissection,werststatetheproblemofpixel-leveldistortionestimationinapracticalvideocodec.Thenwederiveageneraltheoremforanysecondmomentofaweightedsumofcorrelatedrandomvariablesforhelpingsolvetheproblem.Atlast,we 103

PAGE 104

applythetheoremindesigningalow-complexityandhigh-accuracydistortionestimationalgorithmformodedecision. 4.2.1Subpixel-levelDistortionEstimation InChapter 3 ,weknowthatE[eku]andE[(eku)2]canberecursivelycalculatedby E[eku]=Pku("ku+ku+E[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku])+(1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pku)E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg],(4) and E[(eku)2]=Pku(("ku+ku)2+2("ku+ku)E[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]+E[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)2])+(1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pku)E[(ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg)2],(4) where"ku,^eku)]TJ /F3 11.955 Tf 12.43 0 Td[(ekuistheresidualconcealmenterrorwhentheresidualpacketislost;ku,^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)]TJ /F3 11.955 Tf 12.38 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuistheMVconcealmenterrorwhentheMVpacketislost;E[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]andE[(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku)2]inthek)]TJ /F3 11.955 Tf 12.22 0 Td[(1-thframecanberecursivelycalculatedby( 4 )and( 4 ).Pkuisthepixelerrorprobability.Denote^E()astheestimateofE();E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg]andE[(ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg)2]canbeestimatedby ^E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg]=8>>>>>><>>>>>>:^fku)]TJ /F3 11.955 Tf 11.95 0 Td[(255,^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]<^fku)]TJ /F3 11.955 Tf 11.95 0 Td[(255^fku,^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]>^fku^E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku],^fku)]TJ /F3 11.955 Tf 11.96 0 Td[(255^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]^fku,(4) and ^E[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg)2]=8>>>>>><>>>>>>:(^fku)]TJ /F3 11.955 Tf 11.95 0 Td[(255)2,^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]<^fku)]TJ /F3 11.955 Tf 11.95 0 Td[(255(^fku)2,^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]>^fku^E[(ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku)2],^fku)]TJ /F3 11.955 Tf 11.95 0 Td[(255^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]^fku,(4) 104

PAGE 105

InH.264,theaccuracyofmotioncompensationisinunitsofonequarterofthedistancebetweenlumasamples.2Thepredictionvaluesathalf-samplepositionsareobtainedbyapplyingaone-dimensional6-tapFiniteImpulseResponse(FIR)lterhorizontallyandvertically.Thepredictionvaluesatquarter-samplepositionsaregeneratedbyaveragingsamplesatinteger-andhalf-samplepositions[ 20 ].Insuchacase,somevariablesinthepixel-leveldistortionestimationnowincludeafractionalMVandthosevariablesshouldbere-estimated.Asaresult,E[ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku],E[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)2],E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg]andE[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg)2]in( 4 )and( 4 )shouldbeestimatedbasedonthefractionalMV.Since^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg]canbecalculatedby^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]asin( 4 )and^E[(ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg)2]canbecalculatedby^E[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)2]asin( 4 ),weonlyneedtodeterminetherstmomentandsecondmomentofek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuandek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvkufromtheirneighboringintegerpixelpositions. Takeek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvkuforexample.Denotevk)]TJ /F7 7.97 Tf 6.58 0 Td[(j=u+mvkuandvisinasubpixelpositioninthek)]TJ /F4 11.955 Tf 12.29 0 Td[(j-thframe.Allneighboringpixelsintheintegerposition,usedtointerpolatethereconstructedpixelvalueatv,aredenotedbyuiandwithaweightwi,i21,2,...,N,whereN=6forthehalf-sampleinterpolation,andN=2forthequarter-sampleinterpolationinH.264.Therefore,theinterpolatedreconstructedpixelvalueattheencoderis ^fk)]TJ /F7 7.97 Tf 6.58 0 Td[(jv=NXi=1wi^fk)]TJ /F7 7.97 Tf 6.58 0 Td[(jui,(4) andtheinterpolatedreconstructedpixelvalueatthedecoderis efk)]TJ /F7 7.97 Tf 6.58 0 Td[(jv=NXi=1wiefk)]TJ /F7 7.97 Tf 6.58 0 Td[(jui.(4) 2NotethatconsideringthechromadistortiondoesnotalwaysimprovetheR-Dperformancebutinducesmorecomplexity.Therefore,weonlyconsiderlumacomponentsinthischapter. 105

PAGE 106

Asaresult,wehave E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(jv]=E[NXi=1wi(^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(jui)]TJ /F5 11.955 Tf 11.87 3.15 Td[(efk)]TJ /F7 7.97 Tf 6.59 0 Td[(jui)]=NXi=1(wiE[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(jui]),(4) and E[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(jv)2]=E[(NXi=1wi^fk)]TJ /F7 7.97 Tf 6.58 0 Td[(jui)]TJ /F7 7.97 Tf 17.3 14.94 Td[(NXi=1wiefk)]TJ /F7 7.97 Tf 6.58 0 Td[(jui)2]=Ef[NXi=1wi(^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(jui)]TJ /F5 11.955 Tf 11.87 3.15 Td[(efk)]TJ /F7 7.97 Tf 6.59 0 Td[(jui)]2g=E[(NXi=1wiek)]TJ /F7 7.97 Tf 6.59 0 Td[(jui)2].(4) Since^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(jui]havebeencalculatedbytheRMPCalgorithm,^E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(jv]canbeveryeasilycalculatedby( 4 ).However,calculating^E[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(jv)2]isnotstraightforwardsinceE[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(jv)2]in( 4 )isinfactthesecondmomentofaweightedsumofrandomvariables. 4.2.2ANewTheoremforCalculatingtheSecondMomentofaWeightedSumofCorrelatedRandomVariables TheMomentGeneratingFunction(MGF)canbeusedtocalculatethesecondmomentforrandomvariables[ 25 ].However,toestimatethesecondmomentofaweightedsumofrandomvariables,thetraditionalmomentgeneratingfunctionusuallyrequiresknowingtheirprobabilitydistributionandassumingtheyareindependent.However,mostrandomvariablesinvolvedintheaveragingoperationsinavideocodecarenotindependentandtheirprobabilitydistributionsareunknown.Therefore,someapproximations,suchastheCauchy-Schwarzupperboundapproximation[ 32 ]orthecorrelationcoefcientmodelapproximation[ 17 ],areusuallyadoptedtoapproximatethesecondmomentofacomplicatedrandomvariable.However,thoseapproximationsrequireveryhighcomplexity.Forexample,foreachsubpixel,withtheN-taplterinterpolation,theCauchy-SchwarzupperboundapproximationrequiresNintegermultiplicationsforcalculatingthesecondmomentterms,N(N)]TJ /F3 11.955 Tf 12.88 0 Td[(1)=2oating-pointmultiplicationsandN(N)]TJ /F3 11.955 Tf 9.3 0 Td[(1)=2squarerootoperationsforcalculatingthecross-correlation 106

PAGE 107

terms,andN(N)]TJ /F3 11.955 Tf 12.94 0 Td[(1)=2+N)]TJ /F3 11.955 Tf 12.93 0 Td[(1additionsand1shiftforcalculatingtheestimateddistortion.ThecorrelationcoefcientmodelrequiresanadditionalN(N)]TJ /F3 11.955 Tf 13.46 0 Td[(1)=2exponentialoperationsandN(N)]TJ /F3 11.955 Tf 12.17 0 Td[(1)=2oating-pointmultiplicationswhencomparedtotheCauchy-Schwarzupperboundapproximation. Inawirelessvideocommunicationsystem,thecomputationalcapabilityofthereal-timeencoderisusuallyverylimited,andoating-pointprocessingisundesirableespeciallyformobiledevices.Therefore,thequestionishowtodesignanewalgorithmtoaccuratelycalculatethesecondmomentin( 4 )viaonlyintegermultiplication,integeraddition,andshifts. wecandesignalow-complexityandhigh-accuracyalgorithmtoextendtheRMPCalgorithmthroughtheconsiderationofthefollowingtheorem. Theorem4.1. ForanyNcorrelatedrandomvariablesfX1,X2,...,XNgandwi2<,i2f1,2,...,Ng,thesecondmomentoftheweightedsumoftheserandomvariablesisgivenby E[(NXi=1wiXi)2]=NXi=1wiNXj=1[wjE(X2j)])]TJ /F7 7.97 Tf 11.95 14.94 Td[(N)]TJ /F8 7.97 Tf 6.59 0 Td[(1Xk=1NXl=k+1[wkwlE(Xk)]TJ /F4 11.955 Tf 11.95 0 Td[(Xl)2](4) Theorem 4.1 isprovedinAppendix C InH.264,mostaveragingoperations,e.g.,interpolation,deblocking,andbi-prediction,arethespecialcasesofTheorem 4.1 inthatPNi=1wi=1.In( 4 ),PNj=1[wjE(X2j)]istheweightedsumofE(X2j),whichhasbeenestimatedbytheRMPCalgorithm,andtheonlyunknownisPN)]TJ /F8 7.97 Tf 6.58 0 Td[(1k=1PNl=k+1[wkwlE(Xk)]TJ /F4 11.955 Tf 12.3 0 Td[(Xl)2].However,wewillseethatthisunknowncanbeassumedtobenegligibleforthepurposesofmodedecision. 4.2.3TheExtendedRMPCAlgorithmforModeDecision ReplacingXkandXlin( 4 )byekuiandekuj,weobtain ekui)]TJ /F5 11.955 Tf 12.61 3.16 Td[(ekuj=^fkui)]TJ /F5 11.955 Tf 11.87 3.16 Td[(efkui)]TJ /F3 11.955 Tf 11.96 0 Td[((^fkuj)]TJ /F5 11.955 Tf 11.87 3.16 Td[(efkuj)=(^fkui)]TJ /F3 11.955 Tf 12.06 2.65 Td[(^fkuj))]TJ /F3 11.955 Tf 11.95 0 Td[((efkui)]TJ /F5 11.955 Tf 11.87 3.15 Td[(efkuj)(4) 107

PAGE 108

In( 4 ),both^fkui)]TJ /F3 11.955 Tf 13.34 2.66 Td[(^fkujandefkui)]TJ /F5 11.955 Tf 13.16 3.16 Td[(efkujdependonthespatialcorrelationofthereconstructedpixelvaluesinpositionuianduj.Whenuiandujarelocatedinthesameneighborhood,theyareverylikelytobetransmittedinthesamepacket.Therefore,thedifferencebetween^fkui)]TJ /F3 11.955 Tf 12.41 2.66 Td[(^fkujandefkui)]TJ /F5 11.955 Tf 12.22 3.16 Td[(efkujisverysmallandhenceE[(ekui)]TJ /F5 11.955 Tf 12.98 3.16 Td[(ekuj)2]ismuchsmallerthanE[(ekui)2]andE[(ekuj)2].Ontheotherhand,distortionisestimatedforoneMBoronesub-MBasin( 3 )formodedecision.WhenthecardinalityjVkljislarge,Pv2VklPN)]TJ /F8 7.97 Tf 6.59 0 Td[(1i=1PNj=i+1[wiwjE(ekui)]TJ /F5 11.955 Tf 12.67 3.15 Td[(ekui)2]convergestoaconstantforallmodeswithhighprobabilityduetothesummationoverallpixelsinthatMB.Forsimplicity,wewillcallitnegligibleterminthefollowingsections.Therefore,in( 4 )onlythersttermintheright-handsideneedtobecalculated.SincePNi=1wi=1,weestimateE[(ekv)2]formodedecisionby ^E[(ekv)2]=NXi=1[wi^E(ekui)2].(4) With( 4 ),thecomplexityforestimatingthedistortionofeachsubpixel,withtheN-taplterinterpolation,isdramaticallyreducedtoonlyNintegermultiplications,N)]TJ /F3 11.955 Tf 12.18 0 Td[(1additions,and1shift.Here,weproposethefollowingalgorithmtoextendtheRMPCalgorithmformodedecision. Algorithm3. RatedistortionoptimizedmodedecisionforanMBinthek-thframe(k>=1). 1)Input:QP,PEP.2)Initializationof^E[e0u]and^E[(e0u)2]forallpixelu.3)LoopforallavailablemodesforeachMB.estimateE[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku]via( 4 )andE[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)2]via( 4 )forallpixelsintheMB,estimateE[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg]via( 4 )forallpixelsintheMB,estimateE[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg)2]via( 4 )forallpixelsintheMB,estimateE[eku]via( 4 )andE[(eku)2]via( 4 )forallpixelsintheMB,estimateDkuvia( 3 )forallpixelsintheMB, 108

PAGE 109

estimateR-DcostfortheMBvia( 3 ),EndVia( 3 ),selectthebestmodewithminimumR-DcostfortheMB.4)Output:thebestmodefortheMB. Inthischapter,Algorithm 3 isreferredtoasERPMC.NotethatifanMVpacketislost,theERMPCalgorithmconcealstheMVwithintegeraccuracytoreduceboththeconcealmentcomplexityandestimationcomplexity.Therefore,estimatingE[ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku]andE[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)2]in( 4 )and( 4 )doesnotrequire( 4 )andsavescomputationalcost3. 4.2.4MeritsandLimitationsofERMPCAlgorithm 4.2.4.1Merits SinceboththeCauchy-Schwarzupperboundapproximation[ 32 ]andthecorrelationcoefcientmodelapproximation[ 17 ]induceoating-pointmultiplications,round-offerrorisunavoidableinthosealgorithms.ThealgorithmbyYangetal.[ 17 ]needsextracomplexitytomitigatetheeffectofround-offerrorintheirdistortionestimationalgorithm.Incontrast,oneofthemeritsofTheorem 4.1 isthatitonlyneedsintegermultiplicationsandintegeradditions.Assumingwi(andwiwj)canbescaleduptobeanintegerwithoutanyround-offerror,wemaycompareallR-Dcostsbyscalingthemforallmodes.Therefore,round-offerrorisavoidedintheERMPCalgorithm. InRef.[ 8 ],theauthorsprovethatalow-passinterpolationlterwilldecreasetheframe-levelpropagatederrorundersomeassumptions.Infact,itiseasytoprovethatwhenPNi=1wi=1andjVkljislarge,thenegligibletermislargerthanorequaltozero.EvenintheMB-level,thenegligibletermislargerthanorequaltozerowithveryhigh 3Notethatmvkudenotestheconcealedmotionvectorforpixeluk,underthecasethatmvkuisreceivedwitherror. 109

PAGE 110

probability.From( 4 ),weseethattheblock-leveldistortiondecreases,withveryhighprobability,aftertheinterpolationlter. Oneadditionalbenetof( 4 )istoguidethedesignoftheinterpolationlter.Traditionalinterpolationlterdesignaimstominimizethepredictionerror.With( 4 ),wemaydesignaninterpolationlterbymaximizingPNk=1PNl=k+1[wkwlE(Xk)]TJ /F4 11.955 Tf 11.71 0 Td[(Xl)2]undertheconstraintofPNj=1[wjE(X2j)]. 4.2.4.2Limitations InAlgorithm 3 ,thesecondmomentofpropagatederrorE[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)2]isestimatedbyneglectingthenegligibletermtoreducethecomplexity.AmoreaccuratealternativemethodistoestimateE(ekui)]TJ /F5 11.955 Tf 12.87 3.15 Td[(ekui)2]recursivelybystoringthevalueinmemory.Thiswillbeconsideredinourfuturework. 4.3ExperimentalResults Inthissection,wecomparetheR-DperformanceandsubjectiveperformanceoftheERMPCalgorithmwiththatoftheLLNalgorithmformodedecisioninH.264.WealsocompareERMPCwithRMPCandROPEbyusingthenearestneighbortoapproximatethereferencepixelpointedbyafractionalMV.Tocompareallalgorithmsundermulti-referencepicturemotioncompensatedprediction,wealsoenhancetheoriginalROPEalgorithm[ 4 ]withmulti-referencecapability. 4.3.1ExperimentSetup TheJM16.0encoderanddecoderisusedintheexperiments.ThehighproleasdenedintheH.264specicationAnnexA[ 22 ]isused.AllthetestedvideosequencesareinCIFresolutionat30fps.EachcodedvideosequenceistestedunderdifferentPEPsettingsfrom0.5%to5%.Eachvideosequenceiscodedforitsrst100frameswith3slicesperframe.Theerrorconcealmentmethodusedforallalgorithmsistocopythepixelvalueinthesamepositionofthepreviousframe.Therstframeisassumedtobecorrectlyreceived. 110

PAGE 111

Theencodersettingisgivenasbelow.Constrainedintrapredictionisenabled;thenumberofreferenceframesis3;Bslicesarenotincluded;only4x4transformisused;CABACisenabledforentropycoding;intheLLNalgorithm,thenumberofsimulateddecodersis30. 4.3.2R-DPerformance Duetospacelimitations,weonlyshowtheplotsofPSNRvs.bitrateforvideosequences`foreman'and`mobile'underPEP=2%andPEP=5%.Figs. 4-1 and 4-2 showPSNRvs.bitratefor`foreman'and`mobile',respectively.TheexperimentalresultsshowthatERMPCachievesthebestR-Dperformance;RMPCachievesthesecondbestR-Dperformance;ROPEachievesbetterperformancethanLLNinsomecasessuchasathighrateinFig. 4-1 ,butworseperformancethanLLNinothercasessuchasinFig. 4-2 andatthelowrateinFig. 4-1 (a)(b) Figure4-1. PSNRvs.bitratefor`foreman':(a)PEP=0.5%,(b)PEP=2%. Itisinterestingtoseethatforsomesequencesandchannelconditions,ERMPCachievesanotablePSNRgainoverRMPC.Thisis,forexample,evidentwith`mobile'and`foreman'.Forsomeothercases,however,ERMPConlyachievesamarginalPSNRgainoverRMPC(e.g.,`coastguard'and`football').FromtheanalysisinSection 4.2.1 ,weknowthattheonlydifferencebetweenRMPCandERMPCistheestimateoftheerrorfromthereferencepixel,i.e.,propagatederror,undertheconditionthatthere 111

PAGE 112

(a)(b) Figure4-2. PSNRvs.bitratefor`mobile':(a)PEP=0.5%,(b)PEP=2%. isnonewlyinducederrorinthecurrentpixel.Therefore,theperformancegainofERMPCoverRMPConlycomesfromintermodes,sincetheybothuseexactlythesameestimatesforintramodes.Thus,ahigherpercentageofintramodesin`coastguard'and`football'mayresultinamarginalPSNRgainofERMPCoverRMPC. Formostsequencesandchannelconditions,weobservethatthehigherthebitrateforencoding,themorethePSNRgainofERMPCoverRMPC,suchasinFig. 4-1 andFig. 4-2 (a).In( 3 ),theend-to-enddistortionconsistsofbothquantizationdistortionandtransmissiondistortion.TheERMPCalgorithmgivesamoreaccurateestimationofpropagatederrorintransmissiondistortionthantheRMPCalgorithm.Whenthebitrateforsourceencodingisverylow,withratecontrolthecontrolledQPislarge,andhencethequantizationdistortionbecomesthedominantfactorintheend-to-enddistortion.Therefore,thePSNRgainofERMPCoverRMPCismarginal.Onthecontrary,whenthebitrateforsourceencodingishigh,thetransmissiondistortionbecomesthedominantpartintheend-to-enddistortion.Therefore,thePSNRgainofERMPCoverRMPCisnotable.However,thisisnotalwaystrueasobservedinFig. 4-2 (b).InJM16.0,theLagrangemultiplierin( 3 )isafunctionofQP.AhigherbitrateorsmallerQPalsocausesasmallerLagrangemultiplier.Therefore,theratecostin( 3 )becomessmaller,whichmayproduceahigherpercentageofintramodes.Insuchacase,the 112

PAGE 113

PSNRgainofERMPCoverRMPCdecreaseswhenthebitratebecomeshigher.Asaresult,differentsequencesgivedifferentresultsdependingonwhethermoreintramodesareselectedwhenbitrateincreases. LLNhaspoorerR-DperformancethanERMPC.Thismaybebecause30simulateddecodersarestillnotenoughtoachieveareliabledistortionestimatealthoughLLNwith30simulateddecodersalreadyincursmuchhighercomplexitythanERMPC.SincetheoriginalROPEdoesnotsupporttheinterpolationlteringoperationanditsextensions[ 17 32 ]inducemanyoating-pointoperationsandround-offerrors,weonlyusethesamenearestneighborapproximationtoshowhowitsR-DperformancediffersfromERMPC,RMPC,andLLN.Weseethatsuchanextensionisvalidforsomesequences,suchas`foreman'.However,thisapproximationgivespoorR-Dperformanceforsomeothersequences,suchas`mobile'.Therefore,RMPCiseasiertobeextendedthanROPEsincethenearestneighborapproximationforRMPCinallsequencesachievesgoodperformance. Table 4-1 showstheaveragePSNRgain(indB)ofERMPCoverRPMC,LLN,andROPEfordifferentvideosequencesanddifferentPEP.TheaveragePSNRgainisobtainedbythemethodinRef.[ 30 ],whichmeasuresaveragedistance(inPSNR)betweentwoR-Dcurves.FromTable 4-1 ,weseethatERMPCachievesanaveragePSNRgainof0.25dBoverRMPCforthesequence`mobile'underPEP=2%;itachievesanaveragePSNRgainof1.34dBoverLLNforthe`foreman'sequenceunderPEP=1%;anditachievesanaveragePSNRgainof3.18dBoverROPEforthe`mobile'sequenceunderPEP=0.5%. 4.3.3subjectivePerformance SincePSNRcouldbelessmeaningfulforerrorconcealment,amuchmoreimportantperformancecriterionisthesubjectiveperformance,whichdirectlyrelatestothedegreeofuser'ssatisfaction.Fig. 4-3 showsthesubjectivequalityofthe84-thframeandthe99-thframeof`foreman'sequenceunderaPERof1%andabitrateof 113

PAGE 114

Table4-1. AveragePSNRgain(indB)ofERMPCoverRMPC,LLNandROPE Sequence PEP ERMPCvs.RMPC ERMPCvs.LLN ERMPCvs.ROPE coastguard 5% 0.09 0.32 0.58 2% 0.08 0.36 0.46 1% 0.08 0.46 0.52 0.5% 0.06 0.37 0.62 football 5% 0.01 0.28 0.47 2% 0.01 0.39 0.25 1% 0.01 0.36 0.27 0.5% 0.03 0.26 0.33 foreman 5% 0.08 0.64 1.59 2% 0.13 1.07 1.37 1% 0.21 1.34 1.41 0.5% 0.17 1.24 1.42 mobile 5% 0.20 0.50 1.11 2% 0.25 0.82 1.89 1% 0.21 0.56 2.79 0.5% 0.21 0.54 3.18 250kbps.FromFig. 4-3 ,weseethesimilarperformanceresultasinSection 4.3.2 .Thatis,ERMPCachievesthebestperformance. 4.3.4Discussion 4.3.4.1Effectofclippingnoiseonthemodedecision Theexperimentsshowthatsinceitdoesnotconsiderclippingnoise,ROPEover-estimatestheend-to-enddistortionforintermodes.Hence,ROPEtendstoselectintramodesmoreoftenthanERMPC,RMPC,andLLN,whichleadstohigherencodingbitrates.Toverifythisconjecture,wetestedallsequencesunderthesameQuantizationParameter(QP)settingsfrom20to32withoutratecontrol.WeobservedthattheROPEalgorithmalwaysproducedahigherbitratethanotherschemesasshowninFig. 4-4 andFig. 4-5 114

PAGE 115

(a)(b)(c)(d) (e)(f)(g)(h) Figure4-3. (a)ERMPCatthe84-thframe,(b)RMPCatthe84-thframe,(c)LLNatthe84-thframe,(d)ROPEatthe84-thframe,(e)ERMPCatthe99-thframe,(f)RMPCatthe99-th,(g)LLNatthe99-thframe,(h)ROPEatthe99-thframe. (a)(b) Figure4-4. PSNRvs.bitratefor`foreman':(a)PEP=0.5%,(b)PEP=2%. 4.3.4.2Effectoftransmissionerrorsonmodedecision ComparedtotheregularRDOprocessinJM16.0withoutconsideringthetransmissionerror,ERMPC/RMPC/LLN/ROPEalgorithmsshowthreedistinctionsfromit.1)ThenumberofintraMBsincreasessincetransmissionerrorisaccountedforinthe 115

PAGE 116

(a)(b) Figure4-5. PSNRvs.bitratefor`mobile':(a)PEP=0.5%,(b)PEP=2%. modedecision;2)ThenumberofMBswithskipmodeincreasessincethetransmissionerrorwillincreasethetransmissiondistortioninallothermodesexcepttheskipmode;3)ifweallowtherstframetobeerroneous,thesecondframewillhavemanyintraMBssincethepropagatederrorfromtherstframeismuchhigherthanforotherframes.Thisisbecauseonlythevalueof128canbeusedtoconcealthereconstructedpixelvalueiftherstframeislost,whileifotherframesarelost,thecollocatedpixelinthepreviousframecanbeusedtoconcealthereconstructedpixelvalue.Therefore,thetransmissionerrorintherstframegivesmuchhigherpropagatederrorthanintheotherframes. 116

PAGE 117

CHAPTER5RATE-DISTORTIONOPTIMIZEDCROSS-LAYERRATECONTROLINWIRELESSVIDEOCOMMUNICATION Inthischapter,wederivemoreaccuratesourcebitratemodelandquantizationdistortionmodelthanexistingparametricmodels.WealsoimprovetheperformanceboundofchannelcodingwithconvolutionalcodesandaViterbidecoder,andderiveitsperformanceunderRayleighblockfadingchannel.Giventheinstantaneouschannelcondition,i.e.SNRandbandwidth,wedesignarate-distortionoptimizedcross-layerratecontrol(CLRC)algorithmbyjointlychoosingquantizationstepsizeandchannelcodingrate. 5.1AnLiteratureReviewonRateDistortionModelsinWirelessVideoCommunicationSystems Undertheprevalenceof3G/4Gnetworkandsmartphonesnowadays,real-timemobilevideoapplications,e.g.,videophonecalls,arebecomingmoreandmorepopular.However,transmittingvideoovermobilephonewithgoodqualityisparticularlychallengingsincethemobilechannelssubjecttomultipathfading,andthereforethechannelconditionchangesfromframetoframe.Giventheinstantaneouschannelcondition,e.g.,signalnoiseratio(SNR)andbandwidth,theminimumend-to-enddistortioncanbeachievedbyoptimallyallocatingthetransmissionbitratebetweensourcebitrateandredundantbitrate.Inapracticalwirelessvideocommunicationsystem,thiscanbeachievedbyjointlycontrolthesourceencodingparameters,e.g,quantizationstepsize,inthevideoencoder,andchannelencodingparameters,e.g.,channelcodingrate,inthechannelencoder.Sinceboththevideostatisticsandchannelconditionvarywithtime,weneedtodynamicallycontrolthoseparametersforeachframeinreal-timevideoencodingandpackettransmission.Therefore,weneedtoestimatethebitrateanddistortionforeachpossiblecombinationofparametersbeforeencodingeachframe.Asaresult,accuratebitratemodelanddistortionmodelwillbeveryhelpfultoachievetheminimumend-to-enddistortionwithlowcomplexity. 117

PAGE 118

Therearemanyworkstryingtoaddressthisproblemduringtheseyears.Whilemostofthemderivetheend-to-enddistortionasfunctionsofbitrateandpacketerrorrate[ 8 10 ],othersuseoperationalrate-distortion(R-D)functions[ 34 ].Theanalyticalmodelsaremoredesirablesinceitisverydifcultforthevideoencodertogetalloperationalfunctionsfordifferentvideostatisticsandchannelconditionsbeforerealencoding.However,theexistinganalyticalmodelsarestillnotaccurateenoughtoaccommodatethetime-varyingchannelcondition.Ontheotherhand,togettractableformulaeinthoseanalyticalmodels[ 8 10 ],authorsallassumethatblockcodes,i.e.,Reed-Solomoncodes,areadoptedastheforwarderrorcorrection(FEC)scheme.BasedonthatFECscheme,thedistortionisderivedasafunctionofchannelcodingrateandbiterrorrate.However,thisassumptionhastwolimitations:1)mostup-to-datevideocommunicationsystemsuseconvolutionalcodesormoreadvancedcodes,e.g.,turbocodes,forphysicallayerchannelcodingduetotheirexiblechoiceofchannelcodingratewithoutthechangethechannelcodingstructure;2)inthecross-layeroptimizationproblem,theselectionofsourcebitrateandredundantbitratebasedonagivenbiterrorrateissuboptimal,whiletheoptimalsolutioncanbeachievedbyjointlychoosingthembasedonthegiveninstantaneouschannelcondition,e.g.,SNRandchannelbandwidth.Inthischapter,weaimtosolvethecross-layeroptimizationproblembyderivingmoreaccuratebitratemodelandend-to-enddistortionmodel,whichconsistsoftwoparts,thatis,quantizationdistortionmodelandtransmissiondistortionmodel. Plentyofbitratemodelshavebeendevelopedinexistingliterature.Mostoftheexistingworksderivebitrateasafunctionofvideostatisticsandquantizationstepsize[ 35 38 ],whileothersmodelbitrateasafunctionofvideostatisticsandotherparameters[ 39 ].Ingeneral,thesemodelscomefromeitherexperimentalobservation[ 37 39 41 ]orparametricmodeling[ 38 42 43 ].However,bothofthemhavesomelimitations.Theexperimentalmodelingusuallyinducessomemodel 118

PAGE 119

parameterswhichcanonlybeestimatedfrompreviousframes.Therefore,themodelaccuracydependsnotonlyonthestatisticsandcodingparametersbutalsoontheestimationaccuracyofthosemodelparameters.However,intheory,theinstantaneousframebitrateshouldbeindependentofpreviousframesgiveninstantaneousvideoframestatisticsandcodingparameters.Inaddition,theestimationerrorofthosemodelparametersmayhaveasignicantimpactonthemodelaccuracy,whichcanbeobservedintheH.264/AVCJMreferencesoftware[ 33 ]andwillbeexplainedindetailintheexperimentalsectionofthischapter.Ontheotherhand,theparametricmodelinghasthefollowingtwolimitations:1)theassumedresidualprobabilitydistribution,e.g.,Laplaciandistribution,maydeviatesignicantlyfromthetruehistogram;2)theimplicitassumptionofalltransformcoefcientsbeingidenticallydistributedisnotvalidifrun-lengthcodingisconductedbeforetheentropycodingasinmostpracticalencoders.Sincethemodel-selectionproblemmayoftenbemoreimportantthanhavinganoptimizedalgorithm[ 44 ],simplyapplyingtheseparametricmodelstoarealencodermayresultinpoorR-Dperformance.Inthischapter,weimprovethebitratemodelbymodelingthecomponentofrun-levelmappingplusentropycodingastheprocessofchoosingdifferentcodebooksfordifferentquantizedtransformcoefcients.WealsocompensatethemismatchbetweenthetruehistogramandtheassumedLaplaciandistributionintheparametricmodelbyutilizingtheestimationdeviationofpreviousframes.Experimentalresultsshowthatourmethodachievesamoreaccurateestimateofbitratecomparedtoexistingmodels. QuantizationdistortioniscausedbyquantizationerrorunderlossysourcecodingandithasbeenextensivelyexploredsincetheseminalworkofShannon'sratedistortiontheoryrstproposedinRef.[ 1 ]andlaterprovedinRef.[ 2 ].Thequantizationdistortionarestudiedeitherasafunctionofbitrateandthesourceprobabilitydistribution,e.g.,theclassicalR-DfunctionforGaussiansource[ 28 45 ],orasafunctionoflevelnumberandthesourceprobabilitydistributiongivenacertainquantizer,e.g.,uniformscaler 119

PAGE 120

quantizerforGaussiansource[ 46 ].Inthecaseofminimummean-squaredigitizationofmemorylessGaussiansources,quantizerswithuniformlyspacedlevelshaveentropiesthatexceedtherate-distortionfunctionbyapproximately0.25bits/sampleatrelativelyhighrates[ 47 ].InRef.[ 48 ],theperformanceofoptimumquantizersforawideclassofmemorylessnon-Gaussiansourcesisinvestigated,anditisshownthatuniformthresholdquantizersperformaseffectivelyasoptimumquantizers.Forthisreason,theuniformquantizerisususallyadoptedinapracticalvideoencoder,e.g.H.264[ 22 ].Forauniformquantizer,thequantizationdistortionhavebeenderivedasafunctionofquantizationstepsize(orcorrespondingoperationpoint)andvideoframestatisticseitherfromexperimentalobservation[ 37 39 ]orbyparametricmodeling[ 38 42 ].Althoughtheparametricmodelinghasachievedquiteaccurateresult,itcanbefurtherimprovedduetothesourcedistributionmodelinaccuracy.Inthischapter,weimprovetheestimationaccuracyofquantizationdistortionbyutilizingthesimilarmethodinbitratemodel.Experimentalresultsshowthatourquantizationdistortionmodelismoreaccuratethanexistingmodels. Transmissiondistortioniscausedbytransmissionerrorundererror-pronechannels.Predictingtransmissiondistortionatthetransmitterposesagreatchallengeduetothespatio-temporalcorrelationinsidetheinputvideosequence,thenonlinearityofvideocodec,andvaryingPEPintime-varyingchannels.Theexistingtransmissiondistortionmodelscanbecategorizedintothefollowingthreeclasses:1)pixel-levelorblock-levelmodels(appliedtopredictionmodeselection)[ 4 6 ];2)frame-levelorpacket-levelorslice-levelmodels(appliedtocross-layerencodingratecontrol)[ 7 11 ];3)GOP-levelorsequence-levelmodels(appliedtopacketscheduling)[ 12 16 ].Althoughtheexistingtransmissiondistortionmodelsworkatdifferentlevels,theysharesomecommonproperties,whichcomefromtheinherentcharacteristicsofwirelessvideocommunicationsystem,thatis,spatio-temporalcorrelation,nonlinearcodecandtime-varyingchannel.However,noneofthoseworksanalyzedtheeffectof 120

PAGE 121

non-linearclippingnoiseonthetransmissiondistortion,andthereforecannotprovideaccuratetransmissiondistortionestimation.InChapter 2 ,weanalyticallyderive,forthersttime,thetransmissiondistortionformulaasaclosed-formfunctionofpacketerrorprobability(PEP),videoframestatistics,andsystemparameters;andtheninChapter 3 ,wedesigntheRMPCalgorithmtopredictthetransmissiondistortionwithlowcomplexityandhighaccuracy.Inthischapter,wewillfurtherderivePEPandtransmissiondistortionasfunctionsofSNR,transmissionrate,andchannelcodingrateforcross-layeroptimization. Channelcodingcanbeconsideredastheembeddingofsignalconstellationpointsinahigherdimensionalsignalingspacethanisneededforcommunications.Bymappingtoahigherdimensionalspace,thedistancebetweenpointsincreases,whichprovidesbettererrorcorrectionanddetectionperformance[ 18 ].Ingeneraltheperformanceofsoft-decisiondecodingisabout2-3dBbetterthanhard-decisiondecoding[ 18 ].Sinceconvolutionaldecodershaveefcientsoft-decisiondecodingalgorithms,suchasViterbialgorithm[ 49 ],wechooseconvolutionalcodesforphysicallayerchannelcodinginthischapter1.Inaddition,Rate-compatiblepuncturedconvolutional(RCPC)codescanadaptivelychangethecodingratewithoutchangingtheencoderstructure,whichmakesconvolutionalcodesanappropriatemethodinreal-timevideocommunicationoverwirelessfadingchannels.InthischapterweimprovetheperformanceboundofconvolutionalcodesbyaddingathresholdforlowSNRcase,andextendittosupportamoreexibleSNRthresholdfortransmitterswithchannelestimation.Fortransmitterswithoutchannelestimation,wealsoderivetheexpectedPEPasasimplefunctionofconvolutionalencoderstructureandchannelconditionunderRayleighblockfadingchannel. 1Ouralgorithmcanalsobeusedforotherchannelcodes,e.g.blockcodes,Turbocodes,andLDPCcodes,giventheirperformancefordifferentcodingrates. 121

PAGE 122

Giventhebitratefunction,quantizationdistortionfunctionandtransmissiondistortionfunction,minimizingend-to-enddistortionbecomesanoptimizationproblemunderthetransmissionbitrateconstraint.Inthischapter,wealsoapplyourbitratemodel,quantizationdistortionmodelandtransmissiondistortionmodeltocross-layerratecontrolwithrate-distortionoptimization(RDO).Duetothediscretecharacteristicsandthepossibilityofnon-convexityofdistortionfunction[ 50 ],thetraditionalLagrangemultipliersolutionforcontinuousconvexfunctionoptimizationisinfeasibleinavideocommunicationsystem.ThediscreteversionofLagrangianoptimizationisrstintroducedinRef.[ 51 ],andthenrstusedinasourcecodingapplicationinRef.[ 50 ].Duetoitssimplicityandeffectiveness,thisoptimizationmethodisdefactoadoptedbythepracticalvideocodec,e.g.,H.264referencecodeJM[ 33 ].Inthischapter,wewillusethesamemethodtosolveouroptimizationproblem. Therestofthischapterisorganizedasfollows.InSection 5.2 ,weformulatethecross-layeroptimizationproblem.InSection 5.3 ,wederiveourbitratemodel,quantizationdistortionmodel,andtransmissiondistortionmodel.InSection 5.4 ,weproposeapracticalcross-layerratecontrolalgorithmtoachieveminimumend-to-enddistortionunderthegivenSNRandchannelbandwidth.Section 5.5 showstheexperimentalresults,whichdemonstratesboththehigheraccuracyofourmodelsandthebetterperformanceofouralgorithmoverexistingalgorithms. 5.2ProblemFormulation Fig. 2-1 showsthestructureofatypicalwirelessvideocommunicationsystem.Itconsistsofanencoder,twochannelsandadecoderwhereresidualpacketsandMVpacketsaretransmittedovertheirrespectivechannels.Notethatinthissystem,bothresidualchannelandMVchannelareapplication-layerchannels.Fig. 5-1 showsthechanneldetailsforthesetwochannels. 122

PAGE 123

Figure5-1. Channelmodel. ThegeneralRDOprobleminawirelessvideocommunicationsystemcanbeformulatedas minDkETEs.t.RktRkcon,(5) whereDkETEistheend-to-enddistortionofthek-thframe,Rktisthetransmittedbitrateofthek-thframe,Rkcon(dependonthechannelcondition)isthebitrateconstraintofthek-thframe. Fromthedenition,wehave DkETE,E[1 jVkjXu2Vk(fku)]TJ /F5 11.955 Tf 11.87 3.16 Td[(efku)2],(5) whereVkisthesetofpixelsinthek-thframe;fkuistheoriginalpixelvalueforpixeluinthek-thframe;efkuisthereconstructedpixelvalueforthecorrespondingpixelatthedecoder; Denequantizationerrorasfku)]TJ /F3 11.955 Tf 12.31 2.65 Td[(^fkuandtransmissionerroras^fku)]TJ /F5 11.955 Tf 12.14 3.15 Td[(efku,where^fkuisthereconstructedpixelvalueforpixeluinthek-thframeattheencoder.Whilefku)]TJ /F3 11.955 Tf 12.41 2.65 Td[(^fkudependsonlyonthequantizationparameter(QP)2,^fku)]TJ /F5 11.955 Tf 12.57 3.15 Td[(efkumainlydependsonthePEPandtheerrorconcealmentscheme.Inaddition,experimentalresultsshowthat 2Intheratecontrolalgorithmdesign,quantizationoffsetisoftenxed. 123

PAGE 124

fku)]TJ /F3 11.955 Tf 12.97 2.66 Td[(^fkuiszero-mean,whichisalsoobviousintheoryforencodersdesignedunderMMSEcriterion.Therefore,wemakethefollowingassumption. Assumption7. fku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fkuand^fku)]TJ /F5 11.955 Tf 11.87 3.15 Td[(efkuareuncorrelated,andE[fku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fku]=0. UnderAssumption 7 ,from( 5 ),weobtain DkETE=E[1 jVkjXu2Vk(fku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fku)2]+E[1 jVkjXu2Vk(^fku)]TJ /F5 11.955 Tf 11.87 3.16 Td[(efku)2]+2 jVkjXu2VkE[(fku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fku)]E[(^fku)]TJ /F5 11.955 Tf 11.87 3.16 Td[(efku)]=E[1 jVkjXu2Vk(fku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fku)2]+E[1 jVkjXu2Vk(^fku)]TJ /F5 11.955 Tf 11.87 3.16 Td[(efku)2]=DkQ+DkT, (5) where,thersttermintheright-handsideiscalledframe-levelquantizationdistortion(FQD),i.e.,DkQ,E[1 jVkjPu2Vk(fku)]TJ /F3 11.955 Tf 12.34 2.66 Td[(^fku)2]andthesecondtermintheright-handsideiscalledframe-leveltransmissiondistortion(FTD),i.e.,DkT,E[1 jVkjPu2Vk(^fku)]TJ /F5 11.955 Tf 11.87 3.16 Td[(efku)2]. Inatypicalvideocodec,thespatialcorrelationandtemporalcorrelationisrstremovedbyintrapredictionandinterprediction.Thentheresidualistransformedandquantized.Giventheuniformquantizer,DkQonlydependsonthequantizationstepsizeQkandthevideoframestatisticskf.Therefore,wecanexpressDkQasafunctionofQkandkf,i.e.,DQ(Qk,kf),whereDQ()isindependentfromtheframeindexk.InChapter 2 ,wehavederivedDkTasafunctionofPEP,videoframestatisticskfandsystemparametersks,i.e.,DT(PEPk,kf,ks).SincePEPkdependsonSNR(t),transmissionbitrateRkt,andchannelcodingrateRkc,DkTalsodependsontheRkc.Thehigherchannelcodingrate,thehigherPEPkandthusthelargerDkT.However,underthesamebandwidthlimit,thehigherchannelcodingratealsomeansthefewerredundantbitsorthehighersourcebitrate,andthusthesmallerDkQ.InordertodesigntheoptimumQkandRkctoachievetheminimumDkETE,weneedtohavePEPkasafunctionofSNR(t),transmissionrateRkt,andRkc,i.e.,P((t),Rkt,Rkc).Denotekcthechannelstatistics,i.e.kc=f(t),Rktg,wecanexpressDkTasafunctionofRkc,kc,kf,andks,i.e.,DT(Rkc,kc,kf,ks).Ontheotherhand,Rkt=Rks RkcwhereRksdenotethesource 124

PAGE 125

bitrateanditisafunctionofthequantizationstepsizeQkandvideoframestatisticskf,i.e.,Rs(Qk,kf). Therefore,ifwecanderivetheclosed-formfunctionsforDQ(Qk,kf),DT(PEPk,kf,ks)andRs(Qk,kf),( 5 )canbesolvedbyndingthebestparameterpairfQk,Rkcg.Inotherwords,theproblemin( 5 )isequivalentto minDQ(Qk,kf)+DT(Rkc,kc,kf,ks)s.t.NkXi=1Rs(Qk,kf,i) Rkc,iRkt,(5) whereNkisthetotalnumberofpacketsinthek-thframe,andiisthepacketindex.Insummary,ourproblemin( 5 )isgiventhesystemstructureks,time-varyingvideoframestatisticskfandtime-varyingchannelstatisticskc,howtominimizeDkETEbyjointlycontrollingtheparameterspairfQk,Rkc,ig. 5.3DerivationofBitRateFunction,QuantizationDistortionFunctionandTransmissionDistortionFunction Inthissection,wederivethesourceratefunctionRs(Qk,kf),quantizationdistortionfunctionDQ(Qk,kf),andtransmissiondistortionfunctionDT(PEPk,kf,ks). 5.3.1DerivationofSourceCodingBitRateFunction 5.3.1.1TheentropyofquantizedtransformcoefcientsforI.I.D.zero-meanLaplaciansourceunderuniformquantizer FollowingthesimilarderivingprocessasinRef.[ 38 42 43 ],itiseasytoprovethatforindependentandidenticallydistributed(i.i.d.)zero-meanLaplaciansourceunderuniformquantizerwithquantizationstepsizeQandquantizationoffset2,theentropyofquantizedtransformcoefcientsis H=)]TJ /F4 11.955 Tf 9.3 0 Td[(P0log2P0+(1)]TJ /F4 11.955 Tf 11.95 0 Td[(P0)(1log2e 1)]TJ /F4 11.955 Tf 11.96 0 Td[(e)]TJ /F28 7.97 Tf 6.59 0 Td[(1)]TJ /F3 11.955 Tf 11.96 0 Td[(log2(1)]TJ /F4 11.955 Tf 11.95 0 Td[(e)]TJ /F28 7.97 Tf 6.59 0 Td[(1))]TJ /F6 11.955 Tf 11.95 0 Td[(12log2e+1),(5) where 1=p 2Q ;(5) 125

PAGE 126

Qisthequantizationstepsize;isthestandarddeviationoftheLaplaciandistribution;2isthequantizationoffset;P0=1)]TJ /F4 11.955 Tf 12.15 0 Td[(e)]TJ /F28 7.97 Tf 6.59 0 Td[(1(1)]TJ /F28 7.97 Tf 6.59 0 Td[(2)istheprobabilityofquantizedtransformcoefcientbeingzero.( 5 )isprovedinAppendix D.1 5.3.1.2Improvewithrunlengthmodel Inavideoencoder,thequantizedtransformcoefcientsareactuallynoti.i.d.AlthoughwemayassumetheDCTtransformorintegertransform[ 22 ]highlyde-correlatesthecorrelationamongneighboringpixels,differenttransformcoefcientshaveverydifferentvariancesinstatistics.Forexample,ina4x4integertransform,the16coefcientsshowadecreasingvarianceinthewell-knownzigzagscanorderasusedinH.264.Asaresult,thecoefcientswithhigherfrequencyhavehigherprobabilityofbeingzeroesafterquantization.Ontheotherhand,thecoefcientswithlowerfrequencyshowmorerandomnessindifferentlevelsevenafterquantization.Suchcharacteristicsareexploitedbytherun-levelmappingafterzigzagscantofurtherincreasethecompressibilityforentropycoding.Wemayregardthecomponentofrun-levelmappingplusentropycodingaschoosingdifferentcodebooksfordifferentquantizedtransformcoefcients.Frominformationtheory,weknowtheconcavityoftheentropyasafunctionofthedistribution(Theorem2.7.3inRef[ 28 ]).Therefore,notconsideringthemixtureof16coefcientswithdifferentvarianceswilloverestimatetheentropyofmixedtransformcoefcients. Toderivethejointentropyfor16coefcientswithdifferentvariances,weneedtomodelthevariancerelationshipamongthose16coefcients.Havingdoneextensiveexperiments,wendaninterestingphenomenon3,thatis,thevarianceisapproximately 3ThisphenomenonisfoundfromsamplesinoneframeoroneGOPforCIFsequences,i.e.,thenumberofsampleislargerthan101376. 126

PAGE 127

Figure5-2. Variancemodel. afunctionofpositioninthetwo-dimensionaltransformdomainasfollows 2(x,y)=2)]TJ /F8 7.97 Tf 6.59 0 Td[((x+y)20,(5) wherexandyisthepositioninthetwo-dimensionaltransformdomain,and20isthevarianceofthecoefcientatposition(0,0). With( 5 ),wecanderivethevariance2(x,y)forallpositionsgiventheaveragevariance2asinAppendix D.2 .Fig. 5-2 showsthetruevariancesandestimatedvariancesby( D )foralltransformcoefcientsbeforequantizationinthethirdframeof`foreman'sequencewithQP=34.Weonlyshowinterpredictionmodes8x8and4x4inFig. 5-2 .Theresultsofotherinterpredictionmodes[ 20 ]aresimilar.However,wealsonoticethatduetothehighcorrelationamongallcoefcientsinintrapredictionmodes,thetruevarianceofDCcomponentismuchlargerthanestimatedvarianceby( D ).ThemoreaccuratevariancemodelforDCcomponentinintramodeswillbeinvestigatedinourfuturework. 127

PAGE 128

Then,theestimatedjointentropyof16non-identicaltransformcoefcientsbycompensatingtherunlengthcodingmodelis Hrlc=1 163Xx=03Xy=0H(x,y),(5) whereH(x,y)istheentropyforcoefcientposition(x,y),andcanbecalculatedby( D ),( 5 )and( 5 )withtheirown2(x,y)and1(x,y).. 5.3.1.3PracticalconsiderationofLaplacianassumption Statisticallyspeaking,( 5 )isonlyvalidforsufcientlylargesamples.Whentherearenotenoughsamplesorthesamplevarianceisverysmall,e.g.,Q>3,theLaplacianassumptionforindividualcoefcientsisnotaccurate.Insuchcases,wemayusethemixeddistributionin( 5 )astheestimateinsteadof( 5 ).Thatis, Hk=8>><>>:estimatedby( 5 ),Q3estimatedby( 5 ),otherwise.(5) 5.3.1.4Improvementbyconsideringthemodelinaccuracy Theassumedresidualprobabilitydistribution,e.g.,Laplaciandistribution,maydeviatesignicantlyfromthetruehistogramespeciallywhenthenumberofsamplesarenotsufcient.Therefore,weneedtocompensatethemismatchbetweenthetrueresidualhistogramandassumedLaplaciandistributiontoobtainabetterestimate.DenoteHlastheentropyforthecasewithaLaplaciandistribution,Htastheentropyforthecasewiththetruehistogramand=Hl Ht.Inavideosequence,thechangesofresidualstatisticsandquantizationstepsizebetweenadjacentframeshavealmostthesameeffectonHlandHt.Therefore,wemayusethepreviousframestatisticstocompensatetheestimatedresultfrom( 5 ).AssumetheratiobetweenHklandHktapproximatek)]TJ /F8 7.97 Tf 6.58 0 Td[(1,wehaveHkl Hkt=Hk)]TJ /F11 5.978 Tf 5.76 0 Td[(1l Hk)]TJ /F11 5.978 Tf 5.76 0 Td[(1t.Asaresult,( 5 )canbefurthercompensatedas ^Hk=Hk)]TJ /F8 7.97 Tf 6.58 0 Td[(1tHk Hk)]TJ /F8 7.97 Tf 6.58 0 Td[(1l.(5) 128

PAGE 129

Althoughverysimple,( 5 )and( 5 )signicantlyimprovetheestimationaccuracyofresidualentropyasshowninFig. 5-3 5.3.1.5SourcecodingbitrateestimationfortheH.264encoder Forahybridvideocoderwithblock-basedcodingscheme,e.g.,H.264encoder,theencodedbitrateRsconsistsofresidualbitsRresi,motioninformationbitsRmv,predictionmodebitsRmode,andsyntaxbitsRsyntax.Thatis, Rks=^HkNresolutionNfps+Rkmv+Rkmode+Rksyntax,(5) whereNresolutionisthenormalizedvideoresolutionconsideringcolorcomponents,andNfpsmeansthenumberofframespersecond.ComparedtoRkresi,Rkmv,Rkmode,andRksyntaxarelessaffectedbyQ.Therefore,Rkmv,Rkmode,Rksyntaxcanbeestimatedfromthestatisticsinthepreviousframes. 5.3.2DerivationofQuantizationDistortionFunction Inthissubsection,weimprovetheestimationaccuracyofquantizationdistortionbyutilizingthesametechniquesinSection 5.3.1 .InRef.[ 38 42 ],authorsderivethedistortionforzero-meanLaplacianresidualdistributionunderuniformquantizeras DQ=Q2(1e21(2+1)]TJ /F3 11.955 Tf 11.96 0 Td[(221)+2)]TJ /F3 11.955 Tf 11.96 0 Td[(2e1) 21(1)]TJ /F4 11.955 Tf 11.95 0 Td[(e1),(5) Sincethecoefcientsaftertransformisnotidenticalindistribution,weneedtoderivetheoverallquantizationdistortionfunctionbyconsideringeachcoefcientindividually.Usingthevariancerelationshipamongcoefcientsin( 5 ),wehave Doverall=1 163Xx=03Xy=0D(x,y),(5) whereD(x,y)isthedistortionforcoefcientposition(x,y),andcanbecalculatedby( D ),( 5 )and( 5 )withtheirown2(x,y)and1(x,y). Whentherearenotenoughsamplesorthesamplevarianceisverysmall,e.g.,Q>3,theLaplacianassumptionforindividualcoefcientsisnotaccurate.Insuch 129

PAGE 130

cases,wemayusethemixeddistributionin( 5 )astheestimateinsteadof( 5 ).Thatis, DkQ=8>><>>:estimatedby( 5 ),Q3estimatedby( 5 ),otherwise.(5) Similarly,weneedtocompensatethemismatchbetweenthetrueresidualhistogramandassumedLaplaciandistributionforquantizationdistortionestimation.DenoteDQ,lasquantizationdistortionforthecasewithaLaplaciandistribution,DQ,tasquantizationdistortionforthecasewiththetruehistogramand=DQ,l DQ,t.( 5 )canbecompensatedas ^DkQ=Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1Q,tDkQ,l Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1Q,l,(5) whereDkQ,liscalculatedfrom( 5 ).( 5 )and( 5 )signicantlyimprovetheestimationaccuracyofquantizationdistortionasshowninFig. 5-4 5.3.3DerivationofTransmissionDistortionFunction Inthissubsection,wederivetheFTDasafunctionofSNR,transmissionrate,andchannelcodingrate. 5.3.3.1TransmissiondistortionasafunctionofPEP InChapter 2 ,wederivedtheFTDformulaundersingle-referencemotioncompensationandnoslicedatapartitioningas DkT=Pk(E[("k)2]+kE[(k)2]+Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1)+(1)]TJ /F3 11.955 Tf 13.24 2.65 Td[(Pk)kDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1(1)]TJ /F6 11.955 Tf 11.95 0 Td[(k).(5)PkistheweightedaveragePEPofallpacketsinthek-thframe;"kistheresidualconcealmenterror;kistheMVconcealmenterror;kisthepercentageofencodedI-MBsinthek-thframe;boththepropagationfactorkandthecorrelationratiokdependonvideostatistics,channelconditionandcodecstructure,andarethereforecalledsystemparameters;Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1isthetransmissiondistortionofthek)]TJ /F3 11.955 Tf 12.02 0 Td[(1frame,whichcanbeiterativelycalculatedby( 5 ). 130

PAGE 131

PkisdenedasPk,1 jVkjPNki=1(PkiNki),whereNkiisthenumberofpixelscontainedinthei-thpacketofthek-thframe;PkiisPEPofthei-thpacketofthek-thframe;Nkisthetotalnumberofpacketsofthek-thframe.TheothervideoframestatisticsandsystemparameterscanbeeasilyestimatedasdescribedinChapter 3 .WewilldescribehowtoestimatePEPinthefollowingsubsections. 5.3.3.2PEPasafunctionofSNR,transmissionrate,andchannelcodingrateinafadingchannel Below,weanalyzetheconditionalPEPforconvolutioncodingschemeunderwirelessfadingchannel,givenSNR.Sinceconvolutionalcodesarelinearcodes,theprobabilityoferrorcanbeobtainedbyassumingthattheall-zerosequenceistransmitted,anddeterminingtheprobabilitythatthedecoderdecidesinfavorofadifferentsequence[ 18 ].Theprobabilityofmistakingtransmittedsequencewithasequence,Hammingdistancedaway,iscalledpairwiseerrorprobability,anddenotedasP2(d).Withsoftdecision,ifthecodedsymbolsoutputfromtheconvolutionalencoderaresentoveranAWGNchannelusingcoherentBPSKmodulationwithenergyEc=RcEb,thenitcanbeshowthat P2(d)=Q(r 2Ecd N0)=Q(p 2d).(5) BeforecalculatingthePEP,weneedtoanalyzethersterrorprobability,whichisdenedastheprobabilitythatanotherpaththatmergeswiththeall-zeropathatagivennodehasametricthatexceedsthemetricoftheall-zeropathforthersttime[ 52 ].Accordingtothedenition,thersterrorprobabilitycanbeapproximatedbyitsupperbound,i.e.,theprobabilityofmistakingtheall-zeropathforanotherpaththroughthetrellis,as PfedmaxXd=dfreeWdP2(d),(5) whereWdistheweightspectrumofthespecicconvolutionalcode;dfreeisthefreedistanceofthespecicconvolutionalcode;dmaxisthemaximumdistancebetweenthe 131

PAGE 132

transmittedsequenceanddecodedsequence4.Asaresult,thePEPforablockofLdecodedbitsandforagivenSNRcanbeupper-boundedas[ 53 54 ] PEP()1)]TJ /F3 11.955 Tf 11.95 0 Td[((1)]TJ /F4 11.955 Tf 11.95 0 Td[(Pfe())LLPfe().(5) However,bothupperboundsin( 5 )and( 5 )areonlytightwhenislarge.Whenissmallsuchasinafadingchannel,theresultedboundmaybemuchlargerthan1,i.e.,LPfe()1.Fromourexperimentalresults,wendthatthePEP()followswaterfallshapewhenincrease,thatis,thereexistathresholdthsuchthat,when>th,theboundisquitetight,andwhen><>>:RtRc NkNfpsPdmaxd=dfreeWdP2(d,),th1,
PAGE 133

Inareal-timevideocommunicationsystem,iftheestimatedPEP()islargerthanathresholdvalue,i.e.PEP()>PEPth,transmittermaydiscardthispacketinsteadoftransmittingit.5Thebenetofdoingthisisthreefold:1)ifPEP()islarge,itisawasteofenergyandtimetotransmitthepacket;therefore,using( 5 )savestransmissionenergy;2)incross-layerratecontrol,sincevideoencoderhastheknowledgeofchannelcondition,videoencoderwillskipencodingcurrentframewhenthechannelgainisverylow,whichsavestheencodingenergy;3)Ifcurrentframeisskipped,videoencoderwillusepreviousencodedframesasreferencesforencodingthefollowingframes,whichreducethereferenceerrorpropagation. ( 5 )isderivedundertheconditionthatisknownatthetransmitterwithchannelestimation.Insomewirelesssystem,isunknownfortransmitter,e.g.,withoutfeedbackchannel.Insuchcase,theexpectedPEP,i.e.E[PEP],insteadofPEPisusedforestimatingtransmissiondistortiongiventheprobabilitydistributionofchannelgain.Proposition 5 givestheformulaofexpectedPEPunderaRayleighblockfadingchannel. Proposition5. UnderaRayleighblockfadingchannel,theexpectedPEPisgivenby E[PEP]=th e)]TJ /F27 5.978 Tf 7.79 4.62 Td[(th (1+1 dfreeth),(5) wherethisdenedby( 5 ). Proposition 5 isprovedinAppendix D.3 .Weseefrom( D )thatifth ,E[PEP]1)]TJ /F4 11.955 Tf 12.08 0 Td[(e)]TJ /F27 5.978 Tf 7.78 4.62 Td[(th 1)]TJ /F4 11.955 Tf 12.08 0 Td[(e)]TJ /F8 7.97 Tf 6.59 0 Td[(10.63.So,tocontrolthePEPunderareasonablelevel,thetransmittershouldsetitstransmissionpowersuchthat >>thbeforetransmittingthepacket. 5Insomedelay-insensitiveapplications,e.g.streamingvideo,thebufferisusedtoholdpacketswhenchannelconditionispoor.Insuchcases,thepacketwillbedroppedatthetransmitteronlywhenthequeuebufferisfullordelayboundisviolated,whichwilldecreasethePEP. 133

PAGE 134

5.3.3.3TransmissiondistortionasafunctionofSNR,transmissionrate,andchannelcodingrateinafadingchannel Incaseofadaptivemodulation,adaptivetransmissionpowerandadaptivebandwidth(subcarrier)allocation,P2(d)isafunctionofmodulationorderM,transmissionpowerPtandpassbandbandwidthB.Inthischapter,westudythecasethatmodulation,powerandbandwidthareallgivenduringthecross-layerratecontrol.Undersuchconditions,bothtransmissionbitrateRtandSNRareknownvalues.Forexample,withmodulationorderMandNyquistpulse-shaping,Rt=Blog2(M)and=PtTtg N0.Asaresult,bothPEPandDtdependonlyonthetuningparameterRc. 5.4Rate-DistortionOptimizedCross-layerRateControlandAlgorithmDesign Inthissection,weapplyourmodelsderivedinSection 5.3 tocross-layerratecontrolapplication.WeadoptthediscreteversionofLagrangemultiplierasusedinJM[ 33 ]toachievetheR-DoptimizedparameterpairfQk,Rkc,ig.Wealsodesignapracticalcross-layerratecontrolalgorithm. 5.4.1OptimizationofCross-layerRateControlProblem Tosolve( 5 ),wemayeitheruseLagrangianapproachesordynamicprogrammingapproaches[ 44 ].Intermsofcomplexity,theLagrangianapproachispreferable,sinceitcanberunindependentlyineachcodingunit,whereasdynamicprogrammingrequiresatreetobegrown.Notethatthecomplexityofthedynamicprogrammingapproachescangrowexponentiallywiththenumberofcodingunitsconsidered,whiletheLagrangianapproach'scomplexityonlygrowlinearly[ 44 ].ByusingthetheoreminRef.[ 50 51 ],wemayusetheLagrangianapproachforthei-thpacketinthek-thframeindependentlyas (Qki,Rkc,i)=argminfDQ(Qki)+DT(Rkc,i)+Rs(Qki) Rkc,ig,(5) whereisthepresetLagrangemultiplier,whichcanbedeterminedeitherbybi-sectionsearch[ 50 55 ]orbymodeling[ 33 56 ]. 134

PAGE 135

Forsomespecialcases,e.g.videoconference,theframesizeisusuallysmall.Insuchacase,eachframeistransmittedinonepacket,andtherefore,thebitallocationproblemcanbesimplied.Tobemorespecic,sinceallbitsareallocatedintoonepacket,givenandRt,everyRchaveacorrespondingRs;everyRshasacorrespondingQandthereforeDQ(by( 5 ),( 5 ),( 5 )and( 5 )).AsmentionedinSection 5.3.3.3 ,DTisalsoafunctionofRc.Inotherwords,theend-to-enddistortionDkETEonlydependsonRc.Therefore,thereexistsanoptimumRkc,suchthatDETEisminimized.Asaresult,Lagrangemultipliercanbeomitted.Thatis,theoptimumRkccanbeachievedbycomparingDETEforallpossibleRc,andtheoptimumQkcanbecalculatedbythecorrespondingRkc. 5.4.2AlgorithmDesign Inthissubsection,weproposeapracticalalgorithmforcross-layerrate-distortionoptimizationasfollowing. Algorithm4. Cross-layeroptimizedquantizationstepsizeQandchannelcodingrateRcdecisionforthek-thframe. 1)Input:Rt,,PEPth.2)InitializationofQ0andR0cfortherstframe,i.e.k=1.Ifk>1,goto3).3a)IfNk>1,i.e.,eachframeiscontainedinmorethanonepacket.Initializej=0bythemethodproposedinRef.[ 50 ]loopforj=0,1,...,,forpacketindexifrom1toNk>1,Foreachpacket,loopforallcombinationsoffQ,Rcgunderthegivenjcalculatethby( 5 ),estimatePkiforallpacketsby( 5 ),estimateDTby( 5 ),estimate1by( 5 ),estimateDQby( 5 ), 135

PAGE 136

calculateDETE(Q,Rc)by( 5 ),estimateRks,iby( 5 ),( 5 ),( 5 )and( 5 ),EndobtainthebestfQki(j),Rkc,i(j)g,i.e.,fQki(j),Rkc,i(j)gvia( 5 ),EndestimateRktbyRks,iandRkc,i,calculatej+1,EndobtainthebestfQki,Rkc,ig,i.e.,fQki(),Rkc,i()g,foreachpacket.3b)IfNk=1,i.e.,eachframeiscontainedinonepacket.loopforallchannelcodingrates.calculatethby( 5 ),estimatePEPforthek-thframeby( 5 ),estimateDkTby( 5 ),estimate^Hkby( 5 ),estimateQkby( 5 ),( 5 ),( 5 )and( 5 ),estimate^DkQby( 5 ),calculateDETE(Rc)by( 5 ),EndselectthebestRkcandcorrespondingQkwithminimumend-to-enddistortion.4)Output:thebestfQki,Rkc,ig. Algorithm 4 isreferredtoasCLRC.NotethatinAlgorithm 4 ,theiterationstoacquirethebestLagrangemultiplierusebi-sectionsearch[ 50 55 ].SinceloopforallcombinationsoffQ,RcgisexecutedforeachcandidateLagrangemultiplier,thecomplexityisveryhigh.Wemayalsousethemodelingmethod[ 33 56 ]insteadofbi-sectionsearchtodesignCLRC.Insuchacase,R-DoptimizedfQ,Rcgdecisionis 136

PAGE 137

similartotheR-DoptimizedmodedecisioninRef.[ 33 ]exceptthreedifferences:1)themodedecisionisreplacedbychannelcodingratedecisiongiventheLagrangemultiplier,2)thequantizationdistortionisreplacedbytheEnd-to-enddistortion,3)thesourcecodingbitrateisreplacedbythetransmissionbitrate.NotethatthemodelingmethodreducesthecomplexitytoestimatethebestfQki,Rkc,igatthecostofaccuracy. 5.5ExperimentalResults InSection 5.5.1 ,weverifytheaccuracyofourproposedmodels.TheninSection 5.5.2 ,wecomparetheperformancebetweenourCLRCalgorithmandexistingratecontrolalgorithms. 5.5.1ModelAccuracy Inthissubsection,wetestthebitratemodelproposedin( 5 ),distortionmodelproposedin( 5 ),andPEPmodelproposedin( 5 ). 5.5.1.1Bitratemodel TheJM16.0encoderisusedtocollectthetruedistortionandrequiredstatistics.Fig. 5-3 showsthetrueresidualbitrateandestimatedresidualbitratefor`foreman'and`mobile'fortherst20framesinordertomakedifferentcurvesdistinguishable.InFig. 5-3 ,`Truebpp'meansthetruebitperpixel(bpp)producedbytheJM16.0encoder;`withoutrlc'meansbppestimatedby( 5 );`withrlc'meansbppestimatedby( 5 );`withoutcompensation'meansbppestimatedby( 5 );`withcompensation'meansbppestimatedby( 5 )and( 5 );`Rho-domain'meansbppestimatedbyRefs.[ 10 57 ];`Xiang'smodel'meansbppestimatedbyRefs.[ 38 58 ]. Wecanseethattheestimationaccuracyisimprovedby( 5 )whentruebppisrelativelylarge.However,whentruebppissmall,`withoutrlc'giveshigherestimationaccuracy.Byutilizingthestatisticsofthepreviousframefrom( 5 ),theestimationaccuracyisfurtherimproved.Wealsondthat`Rho-domain'isaccurateatlowbpp;however,itisnotaccurateathighbpp.For`Xiang'smodel',theestimatedbppissmallerthanthetruebppinmostcases.Notethatwealsowanttocomparethebitratemodel 137

PAGE 138

(a)(b) Figure5-3. bppvs.Frameindex:(a)foreman,(b)mobile. usedinJM16.0.However,duetotheestimationerrorofitsmodelparameters,therstfewframesmayabnormallyunderestimatethequantizationstepsizeQ.Therefore,theratecontrolalgorithminJM16.0usethreeparameters,i.e.,RCMinQPPSlice,RCMaxQPPSliceandRCMaxQPChange,toreducetheeffectoftheestimationerror.Theirdefaultvaluesare8,42,4,respectively.However,webelieveagoodratecontrolalgorithmshoulddependmainlyonthemodelaccuracyratherthanthosemanuallychosenthresholds.Whenthoseparametersaresetas0,51,51,theestimatedQPcouldevenbe0intherstfewframes.Thatis,therstfewframesconsumemostoftheallocatedbits,andthereareonlyfewbitsavailablefortheremainingframesinJM.Therefore,wedonottestitsmodelaccuracyinthissubsection.Instead,wewillplottheR-DperformanceforitinSection 5.5.2 5.5.1.2Quantizationdistortionmodel Fig. 5-4 showsthecorrespondingquantizationdistortionofeachbitratecurveinFig. 5-3 .NotethatsinceRefs.[ 38 58 ]directlyuse( 5 )toestimatethequantizationdistortion,`withoutrlc'meansthequantizationdistortionestimatedbyboth( 5 )and`Xiang'smodel'.SimilartoFig. 5-3 ,wecanseethattheestimationaccuracyisimprovedby( 5 )when1issmall,i.e.,whenquantizationdistortionisrelativelysmall.However,whenquantizationstepsizeislarge,( 5 )ismoreaccuratethan( 5 ).Notethat,therelativityisforthesamevideosequence.Fordifferentvideosequences,sincethe 138

PAGE 139

residualvariancesaredifferent,inordertoachievethesamebitrate,sequenceswithlargervariance,e.g.,`mobile',willusehigherquantizationstepsizethansequenceswithlowervariance,e.g.,`foreman'.Differentfromthebitratemodel,whichdependsonlyon1,thequantizationdistortionmodeldependsonbothQand1.Therefore,wecannotusetheabsolutevalueofquantizationdistortionbetweentwosequencesforcomparingestimationaccuracyof( 5 )and( 5 ).AfternormalizedbythefactorQ2in( 5 )and( 5 ),theirrelativeaccuracyisvalidinmostcases.However,insomerarecases,( 5 )ismoreaccuratethan( 5 )evenwhenQ>3.Thiscanbeobservedforframeindexfrom14to17inforemansequence.Westillneedtoinvestigatethereasonbehindittofurtherimproveourmodelaccuracy.Forallcases,theestimationaccuracyisimprovedbyutilizingthestatisticsofthepreviousframefrom( 5 ).SimilartoFig. 5-3 ,`rho-domain'ismoreaccurateatlarge1,i.e.,lowbitrateorrelativelylargequantizationdistortion,thanatsmall1. (a)(b) Figure5-4. Quantizationvs.Frameindex:(a)foreman,(b)mobile. 5.5.1.3PEPmodel HereweverifytheaccuracyofPEPmodelderivedin( 5 ).WeusetheRCPCcodesfromTableI-VIinRef[ 59 ].Tobemorespecic,wechooseatypicalconvolutionalencoderstructurewithconstraintlength7,i.e.6memoryregisters,G1=133andG2=171.Thechannelcodingratesare2/3,3/4,4/5,5/6,6/7and7/8.Forcompleteness,weputallencoderparametersinTable 5-1 .Viterbialgorithmisusedtodecodethereceived 139

PAGE 140

bitswithnoise.BPSKmodulationisused.Eachpacketcontains2000informationbits.ForeachSNRandchannelcodingrate,thereare1000packetssimulatedtocollectthetruepacketerrorrate(PER). Table5-1. RCPCencoderparameters codingrate puncturingmatrix dfree weightspectrum 2/3 [11,10] 6 [1,16,48,158,642,2435, 9174,34701,131533,499312] 3/4 [11,10,01] 5 [8,31,160,892,4512,23297, 120976,624304,3229885,16721329] 4/5 [11,10,10,10] 4 [3,24,172,1158,7408,48706, 319563,2094852,13737566,90083445] 5/6 [11,10,01,10,01] 4 [14,69,654,4996,39677,314973, 2503576,19875546,157824160,1253169928] 6/7 [11,10,10,01,10,01] 3 [1,20,223,1961,18084,168982, 1573256,14620204,135966265,1264590899] 7/8 [11,10,10,10,01,10,01] 3 [2,46,499,5291,56137,598557, 6371293,67889502,723039772,7701832191] Fig. 5-5 showsthetruePERandestimatedPEPbytheupperboundin( 5 ).WecanseethattheestimatedPEPcurveisonlyabout1dBhigherthanthecorrespondingtruePERcurve.6 5.5.2PerformanceComparison Inthissubsection,weshowbothobjectiveperformanceandsubjectiveperformanceofCLRCalgorithm.Inordertoseethegainachievedbychannelestimation,wealsocomparetheperformanceachievedby( 5 )and( 5 ).Thisresultmayserveasaguidelineforsystemdesigntobalancetheperformanceandcost.Using( 5 ),wecanalsocomparetheperformancegainachievedbyourmodelsfromexistingmodels. 6Fromtheexperimentalresults,weobservethattheestimatedPEPcurveshowsanconstantoffsetfromthetruePERcurvegiventheRCPCencoderstructure,anddifferentRCPCencoderstructureshowsadifferentoffset.WemayutilizethisobservationtofurtherimprovethePEPmodelinourfuturework. 140

PAGE 141

Figure5-5. PEPunderdifferentRCPCcodingrates. 5.5.2.1Experimentsetup TheJM16.0encoderanddecoder[ 33 ]areusedintheexperiments.AllthetestedvideosequencesareinCIFresolutionat30fps.Eachvideosequenceisencodedforitsrst30frameswheretherstframeisanI-frameandthefollowingframesareP-frames.Theerrorconcealmentmethodistocopythepixelvalueinthesamepositionofthepreviousframe.Therstframeisassumedtobecorrectlyreceivedwithenoughchannelprotectionortimelyacknowledgementfeedback.Theencodersettingisgivenasbelow:Constrainedintrapredictionisenabled;thenumberofreferenceframesis5;Bslicesarenotincluded;only4x4transformisused;CABACisenabledforentropycoding;Forallratecontrolalgorithms,therstframeuseaxQP,i.e.,QP=28. Eachcodedvideosequenceistestedunderdifferentrayleighfadingchannels,i.e.,differentcombinationsofbandwidthfrom100Kbpsto1MbpsandaverageSNRfrom4dBto10dB.Foreachspecicchannelcondition,wesimulate300randompacketerrorprocessestomitigatetheeffectoferrorrandomnessoneachframe.RCPCcodesandmodulationarethesameasthoseinSection 5.5.1.3 141

PAGE 142

5.5.2.2PSNRperformance Figs. 5-6 showsY-componentPSNRvs.averageSNRfor`foreman'and`mobile'.InFig. 5-6 ,`proposed-constant-PEP'representtheperformanceachievedbyourmodelswithoutchannelestimation,i.e.,using( 5 ).`constant-PEP'representtheperformanceachievedbythedefaultratecontrolalgorithminJM16.0,i.e.,JVT-H017r3[ 33 60 61 ],withoutchannelestimation.Foreachalgorithm,wetesttwoparametersettingsof(RCMinQPPSlice,RCMaxQPPSlice,RCMaxQPChange),i.e.,(8,42,4)and(0,51,51)toseehowaccuratethosemodelsareunderdifferentmanuallysetthresholds.TheexperimentalresultsshowthatunderthesameQP-limitationrange,CLRCachievesupto5dBPSNRgainin`foreman'andupto4dBPSNRgainin`mobile'overnochannelestimation.Weobservethatboth`CLRC'and`proposed-constant-PEP'showverystableresultwhentheQP-limitationrangevaries,while`constant-PEP'showverydifferentresultsunderdifferentQP-limitationranges.Tobemorespecic,for`constant-PEP'thesmallerQP-limitationrangegivesupto3dBPSNRgainoverlargerQP-limitationrange.Thisphenomenonfurtherprovesthehigheraccuracyofourmodels. (a)(b) Figure5-6. PSNRvs.averageSNR:(a)foreman,(b)mobile. NotethatinRef.[ 10 ],authorsalsoproposebitratemodel,quantizationdistortionmodelandtransmissiondistortionmodelforsolvingjointsourcechannelratecontrolproblem.However,inboththatbitratemodelandquantizationdistortionmodel,onlythemodelparameter,i.e.,`rho',canbeestimatedfromagivenbitrateorquantization 142

PAGE 143

distortion.InordertoestimatedthequantizationstepsizeorQPbeforerealencoding,thosemodelsrequiresthepriorknowledgeofresidualhistogram[ 62 63 ].SinceH.263encodersusuallyusemeansquareerror(MSE)asacriterionformotionestimation,thiskindofpriorknowledgeisaccessibleinH.263aftermotionestimationandbeforequantization.However,itisnotavailableinH.264encoderssinceR-DcostinsteadofMSEisadoptedasacriterionformotionestimationandmodedecision.TheR-DcostfunctioninducesaLagrangemultiplier,whichcanonlybedeterminedafterQPisknown.Therefore,theirbitratemodelencountersachicken-and-eggproblemifonetriestoapplyitforestimatingquantizationstepsizeinH.264encoders.Duetothisreason,wedonotimplementthosemodelsinRef.[ 10 ]forcross-layerratecontrolintheH.264encoder[ 33 ].NotethatsincethemodelparametersinRef.[ 10 ]isattainableafterrealencoding,westillcomparetheirmodelaccuracyinSection 5.5.1 .FortheaccuracycomparisonbetweenourtransmissiondistortionmodelandthetransmissiondistortionmodelinRef.[ 10 ],pleaserefertoChapter 3 Fig. 5-7 showsY-componentPSNRvs.bandwidthfor`foreman'and`mobile'.WeseethesimilarresultsasinFig. 5-6 .Thatis,1)`CLRC'achievesthebestperformance;2)both`CLRC'and`proposed-constant-PEP'showverystableresultwhentheQP-limitationrangevaries,while`constant-PEP'showverydifferentresultsunderdifferentQP-limitationranges.However,wealsoobserveinourexperimentsthatPSNRshowsmorerandomnessforagivenSNRinFig. 5-7 thaninFig. 5-6 .Forexample,for`mobile'thePSNRat400kbpsisevenhigherthanPSNRatallotherbitrates.Afterinvestigation,wendthisisduetotherandomnessofproducedSNRsamplesequenceforagivenaverageSNRinafadingchannel.Inotherwords,therandomnessofSNRsamplesequencehasmoreimpactondistortionthanbitrateinafadingchannel.Tomitigatetheeffectoftherandomness,weshouldsimulatesufcientlylargenumberofSNRsamplesequencesforagivenaverageSNR.Unfortunately,thisisprohibitivelytimeconsumingandthereforeimpracticaltosimulate.Forexample,ifwesimulate1000 143

PAGE 144

SNRsamplesequenceswith30framespersequencesforaverageSNR8dB;foreachSNRsamplesequence,300randompacketerrorprocessesaresimulatedtomitigatetheeffectoferrorrandomnessoneachframe;InordertoplotPSNRvs.bandwidthfor6algorithmsandsettingswith4bitrates,weneed10003064framesencodingoperationsand10003030064framesdecodingoperations,whichatleastneeds16,000hoursbyusingJM16.0[ 33 ]inaPCwitha2.29GHzCPU. (a)(b) Figure5-7. PSNRvs.bandwidth:(a)foreman,(b)mobile. 5.5.2.3Subjectiveperformance SincePSNRcouldbelessmeaningfulforerrorconcealment,amuchmoreimportantperformancecriterionisthesubjectiveperformance,whichdirectlyrelatestothedegreeofuser'ssatisfaction.Byutilizingthechannelinformation,i.e.,SNRandbandwidth,ourCLRCalgorithmintelligentlychoosesthereferenceframeswhicharetransmittedunderthebestchannelconditionsandneglectsthosereferencesframeswhichexperiencepoorchannelconditions.Asaresult,thewell-knownerrorpropagationproblemisprohibitedevenduringtheencodingprocess. Toillustratethesubjectiveperformance,weplotfourframesfromtheforemansequence.Fig. 5-8 (a)showsarandomchannelsampleunderaverageSNR=10dBandbitrate=1000kbps;Fig. 5-8 (b)showsDistortionvs.Frameindexforforeman cifunderthischannel;Fig. 5-9 showsthecorrespondingsubjectivequalityofreconstructedframes.WeseethatduetoalowchannelSNRduringthetimeslotsofthe10-th 144

PAGE 145

(a)(b) Figure5-8. ArandomchannelsampleunderaverageSNR=10dBandbitrate=1000kbps:(a)ArandomSNRsample,(b)Distortionvs.Frameindexforforeman cifunderthischannel. frame,theencoderwithCLRCskipencodingthesethreeframestosaveencodingandtransmissionenergy.Sincetherearenopacketstransmitted,thereconstructedpictureofboththosethreeframesatthedecoderarethesameasattheencoder.Then,whenthechannelconditiongoeswellinthe11-thframe,encoderwithCLRCusethe9-thframeasreferencetoreconstructthe11-thfame.Sincethechannelconditionisgoodinthetimeslotofthe11-thframe,therearenotransmissiondistortionatthedecoder.Therefore,theerrorpropagationisprohibitedinthefollowingframes. Fortheencoderwithoutchannelestimation,the10-thframeisencodedandtransmitted.DuetothelowchannelSNRduringthetimeslotsofthe10-thframe,thepacketsarereceivedwitherroratthereceiverandtherefore,theresultedPSNRisalmostthesameasthatofencoderwithCLRC.However,withoutchannelinformation,theencoderstillusethe10-thframeasoneofthereferencesforencodingthe11-thframe.Therefore,althoughthe11-thframeiscorrectlyreceivedatthereceiverduetogoodchannelcondition,thereconstructederrorinthe10-thframearepropagatedintothe11-thframeatthedecoder,whichcausesbothlowersubjectivequalityandPSNRcomparingtotheencoderwithCLRC.InFig. 5-9 ,duetothespacelimit,weonlyshowthesubjectivequalityforencoderwith`constant-PEP'underdefaultQPlimitation 145

PAGE 146

range.Aswemayforesee,thesubjectivequalityforencoderwith`constant-PEP'undermaximumQPlimitationrangeistheworstamongallcases. (a)(b)(c)(d) (e)(f)(g)(h) Figure5-9. Forthe10-thframe:(a)original,(b)CLRC,(c)proposed-constant-PEP,(d)constant-PEP-QP-limit;forthe11-thframe:(e)original,(f)CLRC,(g)proposed-constant-PEP,(h)constant-PEP-QP-limit. 146

PAGE 147

CHAPTER6CONCLUSION 6.1SummaryoftheDissertation Inthiswork,weaddressedtheproblemofminimizingend-to-enddistortioninwirelessvideocommunicationsystem.InChapter 1 ,weexplainedthetheoreticalbackgroundandpracticalchallengesforsolvingthisproblem.WealsosummarizedourcontributionsinChapter 1 InChapter 2 ,weanalyticallyderivedthetransmissiondistortionasafunctionofvideostatistics,packeterrorprobabilityandsystemparameters.Withconsiderationofspatio-temporalcorrelation,nonlinearcodecandtime-varyingchannel,ourformulaeprovide,forthersttime,thefollowingcapabilities:1)supportofdistortionpredictionatdifferentlevels(e.g.,pixel/frame/GOPlevel),2)supportofmulti-referencepicturemotioncompensatedprediction,3)supportofslicedatapartitioning,4)supportofarbitraryslice-levelpacketizationwithFMOmechanism,5)beingapplicabletotime-varyingchannels,6)oneuniedformulaforbothI-MBandP-MB,and7)supportofbothlowmotionandhighmotionvideosequences.Besidesderivingthetransmissiondistortionformulae,inChapter 2 ,wealsoidentiedtwoimportantpropertiesoftransmissiondistortionforthersttime:1)clippingnoise,producedbynon-linearclipping,causesdecayofpropagatederror;2)thecorrelationbetweenmotionvectorconcealmenterrorandpropagatederrorisnegative,andhasdominantimpactontransmissiondistortion,amongallthecorrelationsbetweenanytwoofthefourcomponentsintransmissionerror.Wealsodiscussedtherelationshipbetweenourformulaandexistingmodels;wespecifytheconditions,underwhichthoseexistingmodelsareaccurate. InChapter 3 ,wedesignedRMPC-FTD,RMPC-PTD,RMPC-PEEDalgorithmsbasedontheanalysisinChapter 2 .Byvirtueofconsideringthenon-linearclippingnoiseandthenegativecorrelationbetweentheMVconcealmenterrorandthepropagatederror,RMPC-FTDalgorithmprovidesmoreaccurateFTDestimation 147

PAGE 148

thanexistingmodelsasveriedbyexperimentalresults.Inaddition,experimentalresultsalsoshowthatRMPC-FTDalgorithmismorerobustagainstinaccurateestimationofPEPthanexistingmodels.WealsodesignedRMPC-MSalgorithmformodedecisioninH.264.ExperimentalresultsshowthatourRMPC-MSalgorithmachievesanremarkableperformancegainthanexistingalgorithms. InChapter 4 ,weprovedanewtheoremforcalculatingthesecondmomentofaweightedsumofcorrelatedrandomvariableswithoutrequiringknowledgeoftheprobabilitydistributionsoftherandomvariables.Then,weappliedthetheoremtodesignaverylow-complexityalgorithmtoextendtheRMPCalgorithmtoperformmodedecision.Experimentalresultsshowthat,thenewalgorithm,ERMPC,achievesfurtherperformancegainovertheexistingRMPCalgorithm. InChapter 5 ,wederivedmoreaccuratesourcebitratemodelandquantizationdistortionmodelthanexistingparametricmodels.WealsoimprovedtheperformanceboundofchannelcodingwithconvolutionalcodesandaViterbidecoder,andderiveditsperformanceunderRayleighblockfadingchannels.Giventheinstantaneouschannelcondition,i.e.SNRandbandwidth,wedesignedarate-distortionoptimizedCLRCalgorithmbyjointlychoosingquantizationstepsizeandchannelcodingrate.ExperimentalresultsshowedthatourproposedR-DmodelsaremuchmoreaccuratethanexistingR-Dmodels.ExperimentalresultsalsoshowedthattheratecontrolalgorithmwithourmodelsachievessuperiorPSNRgainthantheexistingratecontrolalgorithminJM.TheothermoreimportantresultisthatthesubjectivequalityofourCLRCalgorithmismuchbetterthanexistingalgorithmsduetoitsintelligentreferenceframeselection. 6.2FutureWork First,wewillworkonmodelingtheLagrangemultiplier.CurrentRMPCandERMPCalgorithmsstillusethesameLagrangemultiplierasthatforsourcecodingRDO.However,inanerror-pronechannel,isafunctionofvideocontent,MV,mode,QP,PEP, 148

PAGE 149

errorconcealmentscheme,andconstrainedbitrate.Oneofourfutureresearchtopicsistoanalyticallyderivetheoptimalforwirelessvideotransmission,andthendesignanERRDOschemeforjointMV,mode,QPselection. Second,wewillalsoinvestigatetheeffectofdelayconstraintontheend-to-enddistortion.CurrentCLRCalgorithmdoesnotaddressthedelayconstraint.However,insomedelay-insensitiveapplications,e.g.streamingvideo,thebufferisusedtoholdpacketswhenchannelconditionispoor.Insuchcases,thepacketwillbedroppedatthetransmitteronlywhenthequeuebufferisfullordelayboundisviolated,whichwilldecreasethePEP.Ontheotherhand,thestringentencodingdelayconstraintisalsorelaxed,whichimprovesthevideoquality.Inotherwords,thetimeresourcecanbeutilizedtoreducetheend-to-enddistortion.Inourfuturework,wewillinvestigatehowtominimizetheend-to-enddistortionwithmoretimediversityinthewirelessvideocommunicationsystems. 149

PAGE 150

APPENDIXAPROOFSINCHAPTER2 A.1ProofofLemma 1 Proof. From( 2 )and( 2 ),weobtainefk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eeku=^fku)]TJ /F5 11.955 Tf 12.78 3.15 Td[(eku)]TJ /F5 11.955 Tf 12.35 0 Td[(e"ku)]TJ /F5 11.955 Tf 12.62 3.15 Td[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku, Togetherwith( 2 ),weobtain eku=(^fku)]TJ /F5 11.955 Tf 12.78 3.16 Td[(eku)]TJ /F5 11.955 Tf 12.35 0 Td[(e"ku)]TJ /F5 11.955 Tf 12.61 3.16 Td[(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku))]TJ /F3 11.955 Tf 11.95 0 Td[(\(^fku)]TJ /F5 11.955 Tf 12.78 3.16 Td[(eku)]TJ /F5 11.955 Tf 12.35 0 Td[(e"ku)]TJ /F5 11.955 Tf 12.61 3.16 Td[(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku).(A) So,ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku+eku=(^fku)]TJ /F5 11.955 Tf 12.78 3.15 Td[(eku)]TJ /F5 11.955 Tf 12.35 0 Td[(e"ku))]TJ /F3 11.955 Tf 11.95 0 Td[(\(^fku)]TJ /F5 11.955 Tf 12.78 3.15 Td[(eku)]TJ /F5 11.955 Tf 12.35 0 Td[(e"ku)]TJ /F5 11.955 Tf 12.61 3.15 Td[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku),and Dku(P)=E[(ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvku+eku)2]=E[2(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku,^fku)]TJ /F5 11.955 Tf 12.78 3.15 Td[(eku)]TJ /F5 11.955 Tf 12.35 0 Td[(e"ku)].(A) WeknowfromthedenitionthatDku(p)isaspecialcaseofDku(P)undertheconditionfr,mg,whichmeanseeku=^eku,i.e.e"ku=0,andfmvku=mvku,i.e.eku=0.Therefore,weobtain Dku(p)=E[2(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku,^fku)].(A) A.2ProofofProposition 1 Proof. TheprobabilitydensityfunctionoftherandomvariablehavingaLaplaciandistributionisf(xj,b)=1 2bexp)]TJ /F9 7.97 Tf 10.49 5.7 Td[(jx)]TJ /F28 7.97 Tf 6.59 0 Td[(j b.Since=0,wehaveE[x2]=2b2,andfrom( 3 ),weobtain E[x2])]TJ /F4 11.955 Tf 11.96 0 Td[(E[2(x,y)]=Z+1y)]TJ /F28 7.97 Tf 6.59 0 Td[(L(x2)]TJ /F3 11.955 Tf 11.95 0 Td[((y)]TJ /F6 11.955 Tf 11.95 0 Td[(L)2)1 2be)]TJ /F10 5.978 Tf 7.78 3.26 Td[(x bdx+Zy)]TJ /F28 7.97 Tf 6.59 0 Td[(H[x2)]TJ /F3 11.955 Tf 11.96 0 Td[((y)]TJ /F6 11.955 Tf 11.96 0 Td[(H)2]1 2bex bdx=e)]TJ /F10 5.978 Tf 7.78 4.62 Td[(y)]TJ /F27 5.978 Tf 5.75 0 Td[(L b((y)]TJ /F6 11.955 Tf 11.96 0 Td[(L)b+b2)+e)]TJ /F27 5.978 Tf 7.78 4.62 Td[(H)]TJ /F10 5.978 Tf 5.76 0 Td[(y b((H)]TJ /F4 11.955 Tf 11.95 0 Td[(y)b+b2).(A) Fromthedenitionofpropagationfactor,weobtain=E[2(x,y)] E[x2]=1)]TJ /F8 7.97 Tf -403.12 -19.2 Td[(1 2e)]TJ /F10 5.978 Tf 7.78 4.62 Td[(y)]TJ /F27 5.978 Tf 5.76 0 Td[(L b(y)]TJ /F28 7.97 Tf 6.58 0 Td[(L b+1))]TJ /F8 7.97 Tf 13.15 4.7 Td[(1 2e)]TJ /F27 5.978 Tf 7.79 4.62 Td[(H)]TJ /F10 5.978 Tf 5.76 0 Td[(y b(H)]TJ /F7 7.97 Tf 6.58 0 Td[(y b+1). 150

PAGE 151

A.3ProofofLemma 2 Proof. ForP-MBswithslicedatapartioning,from( 2 )and( 2 )weobtain Dk(P)=1 jVjXu2Vk(Pkufr,mgDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)+1 jVjXu2Vk(Pkufr,mgDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)+1 jVjXu2Vk(PkufrgDku(p)).(A) DenoteVkifr,mgthesetofpixelsinthek-thframewiththesameXEPPkifr,mg;denoteNkifr,mgthenumberofpixelsinVkifr,mg;denoteNkfr,mgthenumberofsetswithdifferentXEPPkifr,mginthek-thframe. Wehave 1 jVjXu2Vk(Pkufr,mgDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)=1 jVjNkfr,mgXi=1(Pkifr,mgXu2Vkifr,mgDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku).(A) ForlargeNkifr,mg,wehave1 Nkifr,mgPu2Vkifr,mgDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuconvergestoDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1,sothersttermintheright-handsidein( A )isDk)]TJ /F8 7.97 Tf 6.58 0 Td[(1Pkfr,mg,wherePkfr,mg=1 jVjPNkfr,mgi=1(Pkifr,mgNkifr,mg). Followingthesameprocess,weobtainthesecondtermintheright-handsidein( A )asDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1Pkfr,mg,wherePkfr,mg=1 jVjPNkfr,mgi=1(Pkifr,mgNkifr,mg);and 1 jVjXu2Vk(PkufrgDku(p))=1 jVjNkfrgXi=1(PkifrgXu2VkifrgDku(p)).(A) ForlargeNkifrg,wehave1 NkifrgPu2VkifrgDku(p)convergestoDk(p),sothethirdtermintheright-handsidein( A )isDk(p)(1)]TJ /F3 11.955 Tf 13.24 2.65 Td[(Pk(r)). NotethatPkifr,mg+Pkifr,mg=PkifrgandNkifr,mg=Nkifr,mg.So,weobtain Dk(P)=Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1Pk(r)+Dk(p)(1)]TJ /F3 11.955 Tf 13.24 2.66 Td[(Pk(r)).(A) ForP-MBswithoutslicedatapartitioning,itisstraightforwardtoacquire( A )from( 3 ).ForI-MBs,from( 2 ),itisalsoeasytoobtainDk(P)=Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(1Pk(r).So,togetherwith( A ),weobtain( 2 ). 151

PAGE 152

A.4ProofofLemma 3 Proof. SinceE[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)2]=E[(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku)2],wehaveE[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)2]=E[(ku+^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)2]andthereforeE[ku^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku]=)]TJ /F7 7.97 Tf 10.49 5.7 Td[(E[(ku)2] 2. Followingthesamederivingprocess,wecanproveE[ku^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]=E[(ku)2] 2. A.5ProofofLemma 4 Proof. ForP-MBswithslicedatapartioning,from( 2 )and( 2 )weobtain Dk(P)=1 jVjXu2Vk(Pkufr,mgDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)+1 jVjXu2Vk(Pkufr,mgDk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)+1 jVjXu2Vk(PkufrgDku(p)).(A) Thersttermintheright-handsidein( A )isexactlythesameasthersttermintheright-handsidein( A ),thatis,itequaltoDk)]TJ /F8 7.97 Tf 6.59 0 Td[(1Pkfr,mg,wherePkfr,mg=1 jVjPNkfr,mgi=1(Pkifr,mgNkifr,mg). DenoteVki(j)fr,mgthesetofpixelsusingthesamereferenceframek)]TJ /F4 11.955 Tf 12.53 0 Td[(jinthek-thframewiththesameXEPPki(j)fr,mg;denoteNki(j)fr,mgthenumberofpixelsinVki(j)fr,mg;denoteNk(j)fr,mgthenumberofsetswithdifferentXEPPki(j)fr,mgbutthesamereferenceframek)]TJ /F4 11.955 Tf 11.96 0 Td[(jinthek-thframe. Wehave 1 jVjXu2Vk(Pkufr,mgDk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)=1 jVjNk(j)fr,mgXi=1(Pki(j)fr,mgJXj=1Xu2Vki(j)fr,mgDk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku).(A) ForlargeNki(j)fr,mg,wehave1 Nki(j)fr,mgPu2Vki(j)fr,mgDk)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvkuconvergestoDk)]TJ /F7 7.97 Tf 6.59 0 Td[(j,so( A )becomes 1 jVjXu2Vk(Pkufr,mgDk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)=1 jVjNk(j)fr,mgXi=1(Pki(j)fr,mgJXj=1Nki(j)fr,mgDk)]TJ /F7 7.97 Tf 6.59 0 Td[(j).(A) Similartothedenitionin( 2 ),wedenetheweightedaverageoverjointPEPs,ofeventthatresidualisreceivedwitherrorandMVisreceivedwithouterror,forthesetof 152

PAGE 153

pixelsusingthesamereferenceframek)]TJ /F4 11.955 Tf 11.96 0 Td[(jinthek-thframeas Pk(j)fr,mg,1 jVk(j)jNk(j)fr,mgXi=1(Pki(j)fr,mgNki(j)fr,mg). (A) Wehave 1 jVjXu2Vk(Pkufr,mgDk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)=1 jVjJXj=1(Pk(j)fr,mgjVk(j)jDk)]TJ /F7 7.97 Tf 6.59 0 Td[(j)=JXj=1(Pk(j)fr,mgwk(j)Dk)]TJ /F7 7.97 Tf 6.59 0 Td[(j).(A) Followingthesameprocess,weobtain 1 jVjXu2Vk(PkufrgDku(p))=JXj=1(Pk(j)frgwk(j)k(j)Dk)]TJ /F7 7.97 Tf 6.59 0 Td[(j),(A) where Pk(j)frg,1 jVk(j)jNk(j)frgXi=1(Pki(j)frgNki(j)frg) (A) istheweightedaverageoverjointPEPs,ofeventthatresidualisreceivedwithouterror,forthesetofpixelsusingthesamereferenceframek)]TJ /F4 11.955 Tf 11.96 0 Td[(jinthek-thframe. Therefore,weobtain Dk(P)=Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(1Pkfr,mg+JXj=1(Pk(j)fr,mgwk(j)Dk)]TJ /F7 7.97 Tf 6.58 0 Td[(j)+JXj=1(Pk(j)frgwk(j)k(j)Dk)]TJ /F7 7.97 Tf 6.59 0 Td[(j).(A) ForP-MBswithoutslicedatapartitioning,Pkfr,mg=PkfrgandPk(j)fr,mg=0,thereforewehave Dk(P)=Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1Pkfrg+JXj=1(Pk(j)frgwk(j)k(j)Dk)]TJ /F7 7.97 Tf 6.58 0 Td[(j).(A) ForI-MBs,from( 2 ),itisalsoeasytoobtainDk(P)=Dk)]TJ /F8 7.97 Tf 6.58 0 Td[(1Pk(r).So,togetherwith( A ),weobtain( 2 ). 153

PAGE 154

FigureA-1. Comparisonof2(x,y)andx2. A.6Lemma 5 andItsProof ToproveProposition 2 ,weneedtousethefollowinglemma. Lemma5. Theerrorreductionfunction(x,y)satises2(x,y)x2foranyLyH. Proof. Fromthedenitionin( 3 ),weobtain 2(x,y))]TJ /F4 11.955 Tf 11.95 0 Td[(x2=8>>>>>><>>>>>>:(y)]TJ /F6 11.955 Tf 11.95 0 Td[(L)2)]TJ /F4 11.955 Tf 11.96 0 Td[(x2,x>y)]TJ /F6 11.955 Tf 11.96 0 Td[(L0,y)]TJ /F6 11.955 Tf 11.96 0 Td[(Hxy)]TJ /F6 11.955 Tf 11.95 0 Td[(L(y)]TJ /F6 11.955 Tf 11.95 0 Td[(H)2)]TJ /F4 11.955 Tf 11.95 0 Td[(x2,xy)]TJ /F6 11.955 Tf 12.02 0 Td[(L.Similarly,sinceyH,weobtain(y)]TJ /F6 11.955 Tf 12.27 0 Td[(H)2
PAGE 155

Denition1. IdealCodec:boththetrueMVandconcealedMVarewithinthesearchrange,andthepositionpointedbythetrueMV,i.e.,u+mvku,isthebestreferencepixel,undertheMMSEcriteria,for^fkuwithinthewholesearchrangeVk)]TJ /F8 7.97 Tf 6.59 0 Td[(1SR,thatis,v=argminv2Vk)]TJ /F11 5.978 Tf 5.76 0 Td[(1SRf(^fku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1v)2g. ToproveCorollary 1 ,weneedtousethefollowinglemma. Lemma6. Inanidealcodec,ekufpg=0,Inotherwords,ifthereisnopropagatederror,theclippingnoiseforthepixelukatthedecoderisalwayszeronomatterwhatkindoferroreventoccursinthek-thframe. Proof. Inanidealcodec,wehave(^eku)2=(^fku)]TJ /F3 11.955 Tf 12.51 2.65 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)2(^fku)]TJ /F3 11.955 Tf 12.51 2.65 Td[(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku)2.Duetothespatialandtemporalcontinuityofthenaturalvideo,wecanprovebycontradictionthatinanidealcodec^fku)]TJ /F3 11.955 Tf 12.06 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuand^fku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkuhavethesamesign,thatiseither ^fku)]TJ /F3 11.955 Tf 12.06 2.65 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku^eku0,or^fku)]TJ /F3 11.955 Tf 12.05 2.65 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku^eku0.(A) Ifthesignof^fku)]TJ /F3 11.955 Tf 12.17 2.65 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuand^fku)]TJ /F3 11.955 Tf 12.17 2.65 Td[(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkuisnotthesame,thenduetothespatialandtemporalcontinuityoftheinputvideo,thereexistsabetterpositionv2Vk)]TJ /F8 7.97 Tf 6.59 0 Td[(1betweenmvkuandmvku,andthereforewithinthesearchrange,sothat(^eku)2(^fku)]TJ /F3 11.955 Tf 12.63 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1v)2.Inthiscase,encoderwillchoosevasthebestreferencepixelwithinthesearchrange.Thiscontradictstheassumptionthatthebestreferencepixelisu+mvkuwithinthesearchrange. Therefore,from( A ),weobtain ^fku^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+^eku^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku,or^fku^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+^eku^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku.(A) Sinceboth^fkuand^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvkuarereconstructedpixelvalue,theyarewithintherangeH^fku,^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvkuL.From( A ),wehaveH^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+^ekuL,andthus\(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+^eku)=^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+^eku.Asaresult,weobtainekufr,m,pg=(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+^eku))]TJ /F3 11.955 Tf 11.95 0 Td[(\(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku+^eku)=0. Sinceekufr,m,pg=^ku=0,andfromSection 2.3.4.1 ,weknowthatekufr,pg=0,henceweobtainekufpg=0. 155

PAGE 156

Remark1. NotethatLemma 6 isprovedundertheassumptionofpixel-levelmotionestimation.Inapracticalencoder,block-levelmotionestimationisadoptedwiththecriterionofminimizingtheMSEofthewholeblock,e.g.,inH.263,orminimizingthecostofresidualbitsandMVbits,e.g.,inH.264.Therefore,somereferencepixelsintheblockmaynotbethebestreferencepixelwithinthesearchrange.Ontheotherhand,RateDistortionOptimization(RDO)asusedinH.264mayalsocausesomereferencepixelsnottobethebestreferencepixels.However,theexperimentresultsforallthetestvideosequencesshowthattheprobabilityofekufr,m,pg6=0isnegligible. A.8ProofofCorollary 1 Proof. From( A ),weobtainekufpg=(^fku)]TJ /F5 11.955 Tf 13.11 3.15 Td[(eku)]TJ /F5 11.955 Tf 12.67 0 Td[(e"ku))]TJ /F3 11.955 Tf 12.28 0 Td[(\(^fku)]TJ /F5 11.955 Tf 13.11 3.15 Td[(eku)]TJ /F5 11.955 Tf 12.67 0 Td[(e"ku).TogetherwithLemma 6 ,whichispresentedandprovedinAppendix A.7 ,wehaveL^fku)]TJ /F5 11.955 Tf 12.91 3.16 Td[(eku)]TJ /F5 11.955 Tf 12.48 0 Td[(e"kuH.FromLemma 5 inAppendix A.6 ,wehave2(x,y)x2foranyLyH;togetherwith( A ),itisstraightforwardtoprovethatE[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eku)2]E[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku)2].ByexpandingE[(ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvku+eku)2],weobtain E[ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+fmvkueku])]TJ /F3 11.955 Tf 23.11 8.09 Td[(1 2E[(eku)2]0.(A) Thephysicalmeaningof( A )isthatek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvkuandekuarenegativelycorrelatedifeku6=0.Sinceekufrg=0asnotedinSection 2.3.4.1 andekufpg=0asprovedinLemma 6 ,weknowthateku6=0isvalidonlyfortheerroreventsfr,m,pgandfr,m,pg,andeku=0foranyothererrorevent.Inotherwords,ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+fmvkuandekuarenegativelycorrelatedundertheconditionfr,pg,andtheyareuncorrelatedunderotherconditions. 156

PAGE 157

APPENDIXBPROOFSINCHAPTER3 B.1ProofofProposition 3 Proof. FromChapter 2 ,weknowthateku,(efk)]TJ /F7 7.97 Tf 6.59 0 Td[(j0u+fmvku+eeku))]TJ /F3 11.955 Tf 12.19 0 Td[(\(efk)]TJ /F7 7.97 Tf 6.58 0 Td[(j0u+fmvku+eeku).Ifthereisnonewlyinducederror,thatis,eeku=^ekuandfmvku=mvku,wehaveefk)]TJ /F7 7.97 Tf 6.59 0 Td[(j0u+fmvku+eeku=efk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+^eku=^fk)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku)]TJ /F5 11.955 Tf 12.61 3.16 Td[(ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+^eku=^fku)]TJ /F5 11.955 Tf 12.61 3.16 Td[(ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku.Therefore,wehave ekufr,mg=8>>>>>><>>>>>>:^fku)]TJ /F5 11.955 Tf 12.62 3.15 Td[(ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku)]TJ /F3 11.955 Tf 11.95 0 Td[(255,^fku)]TJ /F5 11.955 Tf 12.62 3.15 Td[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku>255^fku)]TJ /F5 11.955 Tf 12.62 3.15 Td[(ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku,^fku)]TJ /F5 11.955 Tf 12.62 3.15 Td[(ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku<00,otherwise.(B) Addingek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvkutotheleft-handsideandright-handsidein( B ),weobtain( 3 ). B.2ProofofTheorem 3.1 Proof. Sincethereisnoslicedatapartitioning,Dku,ETE=(1)]TJ /F4 11.955 Tf 12.24 0 Td[(Pku)Dku,ETEfr,mg+PkuDku,ETEfr,mg. First,ifthepacketislost,from( 4 )weobtain Dkufr,mg=("ku+ku)2+2("ku+ku)E[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku]+Dk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku=("ku+ku+E[ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku])2+2ek)]TJ /F11 5.978 Tf 5.75 0 Td[(1u+mvku,(B) andfrom( 4 )weobtain E[eku]fr,mg="ku+ku+E[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku].(B) Togetherwith( B ),( B )and( 3 ),weobtaintheend-to-enddistortionforthecasewherethepacketislostasbelow Dku,ETEfr,mg=(fku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fku+"ku+ku+E[ek)]TJ /F8 7.97 Tf 6.58 0 Td[(1u+mvku])2+2ek)]TJ /F11 5.978 Tf 5.76 0 Td[(1u+mvku.(B) 157

PAGE 158

Bydenition,wehave"ku=^eku)]TJ /F3 11.955 Tf 13.07 0 Td[(ekuandku=^fk)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku)]TJ /F3 11.955 Tf 13.13 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku.So,weobtain"ku+ku=^fku)]TJ /F3 11.955 Tf 12.05 2.65 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)]TJ /F3 11.955 Tf 11.99 0 Td[(eku,and Dku,ETEfr,mg=(fku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku)]TJ /F3 11.955 Tf 11.99 0 Td[(eku+E[ek)]TJ /F8 7.97 Tf 6.59 0 Td[(1u+mvku])2+2ek)]TJ /F11 5.978 Tf 5.76 0 Td[(1u+mvku.(B) Notethatiftheerrorconcealmentschemeistocopythereconstructedpixelvaluefromthepreviousframe,wehave"ku+ku=^fku)]TJ /F3 11.955 Tf 12.06 2.66 Td[(^fk)]TJ /F8 7.97 Tf 6.58 0 Td[(1u. Notethattheerrorconcealmentmethodisthesameforintramodeandintermodesincethereisnomodeinformationfordecoderifthepacketisreceivedinerror;hencemvkuandekuin( B )arethesameforbothintramodeandintermode.Ontheotherhand,thevalueoffkuisknownbeforethemodedecisionandallothervariablesin( B )comefromthepreviousframe.Therefore,theresultingend-to-enddistortioninthiscase,i.e.,Dku,ETEfr,mgwillalsobethesameforbothintramodeandintermode. Second,ifthepacketiscorrectlyreceived,from( 4 )weobtainDkufr,mg=Dku(p)andfrom( 4 )weobtainE[eku]fr,mg=E[ek)]TJ /F7 7.97 Tf 6.59 0 Td[(ju+mvku+ekufr,mg].From( 3 ),weobtaintheend-to-enddistortionas Dku,ETEfr,mg=(fku)]TJ /F3 11.955 Tf 12.05 2.65 Td[(^fku)2+Dku(p)+2(fku)]TJ /F3 11.955 Tf 12.05 2.65 Td[(^fku)E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg].(B) SincebothPkuandDku,ETEfr,mgarethesameforallmodes,wecandenotePkuDku,ETEfr,mgbyCku,whichisindependentofallmodes.LetDku, ETE=(1)]TJ /F4 11.955 Tf 13.19 0 Td[(Pku)Dku,ETEfr,mg;thenwehave Dku,ETE(!m)=Dku, ETE(!m)+Cku,(B) where Dku, ETE(!m)=(1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pku)Dku,ETEfr,mg (B) =(1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pku)f(fku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fku)2+Dku(p)+2(fku)]TJ /F3 11.955 Tf 12.05 2.66 Td[(^fku)E[ek)]TJ /F7 7.97 Tf 6.58 0 Td[(ju+mvku+ekufr,mg]g. (B) 158

PAGE 159

B.3ProofofProposition 4 Proof. From( 3 ),wehave argmin!mf^DkETE(!m)+R(!m)g=argmin!mfXu2Vki[Dku, ETE(!m)+Cku]+R(!m)g=argmin!mfXu2VkiDku, ETE(!m)+Xu2VkiCku+R(!m)g=argmin!mf^Dk ETE(!m)+R(!m)g=^!m.(B) Thisis,AlgorithmAandAlgorithm 2 producethesamesolution. 159

PAGE 160

APPENDIXCPROOFSINCHAPTER4 Proof. IfPNi=1wi6=0,letusdeneS,PNi=1wiandpi,wi S.Therefore,wehavewi=piS,PNi=1pi=1andE[(PNi=1wiXi)2]=E[(PNi=1piSXi)2]=S2E[(PNi=1piXi)2]. WefurtherdeneD,E[(PNi=1piXi)2].Thereforewehave E[(NXi=1wiXi)2]=S2D,(C) whereDcanbecalculatedby D=NXj=1[p2jE(X2j)]+NXk=1NXl=1(l6=k)[pkplE(XkXl)]=NXj=1[pjE(X2j)])]TJ /F7 7.97 Tf 17.3 14.95 Td[(NXj=1[pj(1)]TJ /F4 11.955 Tf 11.96 0 Td[(pj)E(X2j)]+NXk=1NXl=1(l6=k)[pkplE(XkXl)]=NXj=1[pjE(X2j)])]TJ /F7 7.97 Tf 17.3 14.94 Td[(NXj=1[pjNXj0=1(j06=j)pj0E(X2j)]+NXk=1NXl=1(l6=k)[pkplE(XkXl)]=NXj=1[pjE(X2j)])]TJ /F7 7.97 Tf 17.3 14.95 Td[(NXj=1NXj0=1(j06=j)[pjpj0E(X2j)]+NXk=1NXl=1(l6=k)[pkplE(XkXl)]=NXj=1[pjE(X2j)])]TJ /F7 7.97 Tf 11.95 14.94 Td[(N)]TJ /F8 7.97 Tf 6.58 0 Td[(1Xk=1NXl=k+1fpkpl[E(X2k)+E(X2l)]g+N)]TJ /F8 7.97 Tf 6.59 0 Td[(1Xk=1NXl=k+1[2pkplE(XkXl)]=NXj=1[pjE(X2j)])]TJ /F7 7.97 Tf 11.95 14.94 Td[(N)]TJ /F8 7.97 Tf 6.58 0 Td[(1Xk=1NXl=k+1[pkplE(Xk)]TJ /F4 11.955 Tf 11.95 0 Td[(Xl)2]. (C) From( C )and( C ),wehave E[(NXi=1wiXi)2]=SNXj=1[wjE(X2j)])]TJ /F7 7.97 Tf 11.96 14.95 Td[(N)]TJ /F8 7.97 Tf 6.59 0 Td[(1Xk=1NXl=k+1[wkwlE(Xk)]TJ /F4 11.955 Tf 11.96 0 Td[(Xl)2]=NXi=1wiNXj=1[wjE(X2j)])]TJ /F7 7.97 Tf 11.95 14.94 Td[(N)]TJ /F8 7.97 Tf 6.59 0 Td[(1Xk=1NXl=k+1[wkwlE(Xk)]TJ /F4 11.955 Tf 11.95 0 Td[(Xl)2]. (C) 160

PAGE 161

IfPNi=1wi=0,wehave wj=)]TJ /F7 7.97 Tf 17.78 14.95 Td[(NXj0=1(j06=j)wj0,(C) and E[(NXi=1wiXi)2]=NXj=1[w2jE(X2j)]+NXk=1NXl=1(l6=k)[wkwlE(XkXl)]=)]TJ /F7 7.97 Tf 16.64 14.94 Td[(NXj=1[wjNXj0=1(j06=j)wj0E(X2j)]+NXk=1NXl=1(l6=k)[wkwlE(XkXl)]=)]TJ /F7 7.97 Tf 11.3 14.95 Td[(N)]TJ /F8 7.97 Tf 6.59 0 Td[(1Xk=1NXl=k+1[wkwlE(Xk)]TJ /F4 11.955 Tf 11.96 0 Td[(Xl)2]. (C) Weseethat( C )isjustaspecialcaseof( C )underPNi=1wi=0.Therefore,foranywi2<,wehavethegeneralform( C ). 161

PAGE 162

APPENDIXDPROOFSINCHAPTER5 D.1ProofofEquation( 5 ) Proof. Fortransformcoefcientswithi.i.d.zero-meanLaplaciandistribution,theprobabilitydensityfunction(pdf)isp(x)=1 p 2ep 2jxj ,whereisthestandarddeviation.FortheuniformquantizerwithquantizationstepsizeQandquantizationoffset2,theprobabilityofzeroafterquantizationis P0=2ZQ(1)]TJ /F28 7.97 Tf 6.59 0 Td[(2)0p(x)dx=1)]TJ /F4 11.955 Tf 11.95 0 Td[(e)]TJ /F28 7.97 Tf 6.59 0 Td[(1(1)]TJ /F28 7.97 Tf 6.59 0 Td[(2),(D) andtheprobabilityoflevelnafterquantizationis Pn=ZQ(n+1)]TJ /F28 7.97 Tf 6.59 0 Td[(2)Q(n)]TJ /F28 7.97 Tf 6.58 0 Td[(2)p(x)dx=1 2(1)]TJ /F4 11.955 Tf 11.96 0 Td[(e)]TJ /F28 7.97 Tf 6.58 0 Td[(1)e12e)]TJ /F28 7.97 Tf 6.59 0 Td[(1n,(D) where1=p 2Q Asaresult, H=)]TJ /F4 11.955 Tf 9.29 0 Td[(P0log2P0)]TJ /F3 11.955 Tf 11.95 0 Td[(21Xn=1Pnlog2Pn=)]TJ /F4 11.955 Tf 9.29 0 Td[(P0log2P0+(1)]TJ /F4 11.955 Tf 11.95 0 Td[(P0)(1log2e 1)]TJ /F4 11.955 Tf 11.95 0 Td[(e)]TJ /F28 7.97 Tf 6.59 0 Td[(1)]TJ /F3 11.955 Tf 11.96 0 Td[(log2(1)]TJ /F4 11.955 Tf 11.95 0 Td[(e)]TJ /F28 7.97 Tf 6.58 0 Td[(1))]TJ /F6 11.955 Tf 11.96 0 Td[(12log2e+1).(D) D.2CalculationofEntropyforDifferentQuantizedTransformCoefcients Proof. Fora4x4integertransformwithaveragevariance2,thevarianceforeachtransformcoefcientcanbecalculateby( 5 )as 2=1 164Xx=04Xy=02(x,y)=225 102420.(D) Therefore,wehave 2(x,y)=2)]TJ /F8 7.97 Tf 6.58 0 Td[((x+y)1024 2252.(D) 162

PAGE 163

D.3ProofofProposition 5 Proof. InaRayleighfadingchannel,thereceivedsignalamplitudehastheRayleighdistribution,andreceivedsignalpowerhastheexponentialdistribution.Therefore,SNRinreceiverhastheexponentialdistribution[ 18 ],thatis, P()=1 e)]TJ /F27 5.978 Tf 7.79 3.69 Td[( ,(D) where =Pt g N0B;Ptistransmissionpower; gisthemeanofchannelgain;N0 2isnoisepowerspectraldensity;andBispassbandbandwidth. Byusingthewell-knownupperboundasapproximationforQfunction,i.e.,Q(x)1 2e)]TJ /F10 5.978 Tf 7.79 3.26 Td[(x2 2[ 64 ],from( 5 )wehave E[PEP]=Z10PEP()P()dZth01 e)]TJ /F27 5.978 Tf 7.78 3.69 Td[( d+Z1th(dmaxXd=dfree1 2LWde)]TJ /F28 7.97 Tf 6.59 0 Td[(d)1 e)]TJ /F27 5.978 Tf 7.78 3.69 Td[( d=1)]TJ /F4 11.955 Tf 11.96 0 Td[(e)]TJ /F27 5.978 Tf 7.78 4.62 Td[(th +dmaxXd=dfree1 2LWdZ1th1 e)]TJ /F28 7.97 Tf 6.59 0 Td[(d)]TJ /F27 5.978 Tf 7.78 3.69 Td[( d=1)]TJ /F4 11.955 Tf 11.96 0 Td[(e)]TJ /F27 5.978 Tf 7.78 4.63 Td[(th +dmaxXd=dfree1 2LWde)]TJ /F7 7.97 Tf 6.58 0 Td[(dth1 1+d e)]TJ /F27 5.978 Tf 7.78 4.63 Td[(th ,(D) wherethisdenedby( 5 ). letf(d)=1 2LWde)]TJ /F7 7.97 Tf 6.58 0 Td[(dthandPEPth=1,from( 5 )wehavePdmaxd=dfreef(d)=1.Ifweregardf(d)asapmffordandfurtherletg(d)=1 1+d e)]TJ /F27 5.978 Tf 7.78 4.62 Td[(th ,thethirdtermin( D )canberegardedasaexpectedvalueofg(d)withpmff(d).Sincef(d)decaysexponentiallywiththeincreaseofd,g(d)canbeapproximatedbyacloseupperbound1 1+dfree e)]TJ /F27 5.978 Tf 7.78 4.63 Td[(th .Therefore,( D )becomes E[PEP]1)]TJ /F4 11.955 Tf 11.96 0 Td[(e)]TJ /F27 5.978 Tf 7.78 4.62 Td[(th +e)]TJ /F27 5.978 Tf 7.78 4.63 Td[(th 1+dfree dmaxXd=dfree1 2LWde)]TJ /F7 7.97 Tf 6.58 0 Td[(dth=1)]TJ /F4 11.955 Tf 11.96 0 Td[(e)]TJ /F27 5.978 Tf 7.78 4.62 Td[(th +e)]TJ /F27 5.978 Tf 7.78 4.62 Td[(th 1+dfree ,(D) 163

PAGE 164

Inapracticalcommunicationsystem,dfree >>1.Ontheotherhand,since >>thasmentionedinSection 5.3.3.2 ande)]TJ /F7 7.97 Tf 6.59 0 Td[(x1)]TJ /F4 11.955 Tf 12.89 0 Td[(xe)]TJ /F7 7.97 Tf 6.59 0 Td[(xatsmallx,wemayapproximate1)]TJ /F4 11.955 Tf 11.96 0 Td[(e)]TJ /F27 5.978 Tf 7.78 4.63 Td[(th byth e)]TJ /F27 5.978 Tf 7.78 4.63 Td[(th .Therefore,wehave E[PEP]th e)]TJ /F27 5.978 Tf 7.78 4.62 Td[(th +e)]TJ /F27 5.978 Tf 7.79 4.62 Td[(th dfree =th e)]TJ /F27 5.978 Tf 7.78 4.62 Td[(th (1+1 dfreeth)(D) Notethatxe)]TJ /F7 7.97 Tf 6.59 0 Td[(xincreaseswhilexincreasesintheinterval0
PAGE 165

REFERENCES [1] C.E.Shannon,Amathematicaltheoryofcommunication,BellSyst.Tech.J.,vol.27,pp.379,623,1948. [2] ,Codingtheoremsforadiscretesourcewithadelitycriterion,IRENat.Conv.Rec.Part,vol.4,pp.142,1959. [3] T.BergerandJ.Gibson,Lossysourcecoding,IEEETransactionsonInformationTheory,vol.44,no.6,pp.2693,1998. [4] R.Zhang,S.L.Regunathan,andK.Rose,Videocodingwithoptimalinter/intra-modeswitchingforpacketlossresilience,IEEEJournalonSelectedAreasinCommunications,vol.18,no.6,pp.966,Jun.2000. [5] T.Stockhammer,M.Hannuksela,andT.Wiegand,H.264/AVCinwirelessenvironments,IEEETransactionsonCircuitsandSystemsforVideoTechnol-ogy,vol.13,no.7,pp.657,2003. [6] T.Stockhammer,T.Wiegand,andS.Wenger,Optimizedtransmissionofh.26l/jvtcodedvideooverpacket-lossynetworks,inIEEEICIP,2002. [7] M.Sabir,R.Heath,andA.Bovik,Jointsource-channeldistortionmodelingforMPEG-4video,IEEETransactionsonImageProcessing,vol.18,no.1,pp.90,2009. [8] K.Stuhlmuller,N.Farber,M.Link,andB.Girod,Analysisofvideotransmissionoverlossychannels,IEEEJournalonSelectedAreasinCommunications,vol.18,pp.1012,Jun.2000. [9] J.U.Dani,Z.He,andH.Xiong,Transmissiondistortionmodelingforwirelessvideocommunication,inProceedingsofIEEEGlobalTelecommunicationsConfer-ence(GLOBECOM'05),2005. [10] Z.He,J.Cai,andC.W.Chen,Jointsourcechannelrate-distortionanalysisforadaptivemodeselectionandratecontrolinwirelessvideocoding,IEEETransactionsonCircuitsandSystemforVideoTechnology,specialissueonwirelessvideo,vol.12,pp.511,Jun.2002. [11] Y.Wang,Z.Wu,andJ.M.Boyce,Modelingoftransmission-loss-induceddistortionindecodedvideo,IEEETransactionsonCircuitsandSystemsforVideoTechnol-ogy,vol.16,no.6,pp.716,Jun.2006. [12] J.Chakareski,J.Apostolopoulos,W.-T.Tan,S.Wee,andB.Girod,Distortionchainsforpredictingthevideodistortionforgeneralpacketlosspatterns,inProc.ICASSP,2004. 165

PAGE 166

[13] J.Chakareski,J.Apostolopoulos,S.Wee,W.-T.Tan,andB.Girod,Rate-distortionhinttracksforadaptivevideostreaming,IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.15,no.10,pp.1257,2005. [14] C.Zhang,H.Yang,S.Yu,andX.Yang,GOP-leveltransmissiondistortionmodelingformobilestreamingvideo,SignalProcessing:ImageCommunication,2007. [15] M.T.Ivrlac,L.U.Choi,E.Steinbach,andJ.A.Nossek,Modelsandanalysisofstreamingvideotransmissionoverwirelessfadingchannels,SignalProcessing:ImageCommunication,vol.24,no.8,pp.651,Sep.2009. [16] Y.J.Liang,J.G.Apostolopoulos,andB.Girod,Analysisofpacketlossforcompressedvideo:Effectofburstlossesandcorrelationbetweenerrorframes,IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.18,no.7,pp.861,Jul.2008. [17] H.YangandK.Rose,Advancesinrecursiveper-pixelend-to-enddistortionestimationforrobustvideocodinginH.264/AVC,IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.17,no.7,p.845,2007. [18] A.Goldsmith,WirelessCommunications.CambridgeUniversityPress,2005. [19] S.Wenger,H.264/AVCoverIP,IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.13,no.7,pp.645656,Jul.2003. [20] T.Wiegand,G.J.Sullivan,G.Bjontegaard,andA.Luthra,Overviewoftheh.264/AVCvideocodingstandard,IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.13,no.7,pp.560,Jul.2003. [21] D.Agraotis,D.R.Bull,andC.N.Canagarajah,Enhancederrorconcealmentwithmodeselection,IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.16,no.8,pp.960,Aug.2006. [22] ITU-TSeriesH:AudiovidualandMultimediaSystems,Advancedvideocodingforgenericaudiovisualservices,Nov.2007. [23] Y.Zhang,W.Gao,Y.Lu,Q.Huang,andD.Zhao,Jointsource-channelrate-distortionoptimizationforh.264videocodingovererror-pronenetworks,IEEETransactionsonMultimedia,vol.9,no.3,pp.445,Apr.2007. [24] B.Girod,Theefciencyofmotion-compensatingpredictionforhybridcodingofvideosequences,IEEEJournalonSelectedAreasinCommunications,vol.5,no.7,pp.1140,Aug.1987. [25] G.CasellaandR.L.Berger,StatisticalInference,2nded.DuxburyPress,2001. [26] J.Wu,X.Liu,andK.-Y.Yoo,Atemporalerrorconcealmentmethodforh.264/avcusingmotionvectorrecovery,IEEETransactionsonConsumerElectronics,vol.54,no.4,pp.1880,Nov.2008. 166

PAGE 167

[27] Y.WangandQ.-F.Zhu,Errorcontrolandconcealmentforvideocommunication:areview,inProceedingsofIEEE,vol.86,no.5,1998,pp.974. [28] T.M.CoverandJ.A.Thomas,ElementsofInformationTheory.Wiley-Interscience,1991. [29] T.Stockhammer,D.Kontopodis,andT.Wiegand,Rate-distortionoptimizationforJVT/H.26Lvideocodinginpacketlossenvironment,inProc.InternationalPacketVideoWorkshop.Citeseer,2002. [30] G.Bjontegaard,CalculationofaveragePSNRdifferencesbetweenRD-curves,13thVCEG-M33Meeting,Austin,USA,2001. [31] JCT-VC,TestModelunderConsideration(TMuC),Dresden,Apr.2010,JCTVC-A205,1stJCT-VCMeeting. [32] A.LeontarisandP.Cosman,Videocompressionforlossypacketnetworkswithmodeswitchingandadual-framebuffer,IEEETransactionsonImageProcessing,vol.13,no.7,pp.885,2004. [33] H.264/AVCreferencesoftwareJM16.0,Jul.2009.[Online].Available: http://iphome.hhi.de/suehring/tml/download [34] L.Kondi,F.Ishtiaq,andA.Katsaggelos,Jointsource-channelcodingformotion-compensatedDCT-basedSNRscalablevideo,IEEETransactionsonImageProcessing,vol.11,no.9,pp.1043,2002. [35] J.Ribas-CorberaandS.Lei,RatecontrolinDCTvideocodingforlow-delaycommunications,IEEETransactionsonCircuitsandSystemsforVideoTechnol-ogy,vol.9,no.1,pp.172,1999. [36] T.ChiangandY.Zhang,Anewratecontrolschemeusingquadraticratedistortionmodel,IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.7,no.1,pp.246,1997. [37] S.Ma,W.Gao,andY.Lu,Rate-distortionanalysisforH.264/AVCvideocodinganditsapplicationtoratecontrol,IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.15,no.12,p.1533,2005. [38] X.Li,N.Oertel,A.Hutter,andA.Kaup,LaplacedistributionbasedLagrangianratedistortionoptimizationforhybridvideocoding,IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.19,no.2,pp.193,2009. [39] Z.HeandS.Mitra,Optimumbitallocationandaccurateratecontrolforvideocodingvia-domainsourcemodeling,IEEEtransactionsonCircuitsandSystemsforVideoTechnology,vol.12,no.10,2002. 167

PAGE 168

[40] H.Lee,T.Chiang,andY.Zhang,ScalableratecontrolforMPEG-4video,IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.10,no.6,pp.878,2000. [41] K.Yang,A.Jacquin,andN.Jayant,Anormalizedrate-distortionmodelforH.263-compatiblecodecsanditsapplicationtoquantizerselection,inProceedingsofIEEEICIP,vol.2,1997,pp.41. [42] H.HangandJ.Chen,Sourcemodelfortransformvideocoderanditsapplication.I.Fundamentaltheory,IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.7,no.2,pp.287,1997. [43] F.Moscheni,F.Dufaux,andH.Nicolas,Entropycriterionforoptimalbitallocationbetweenmotionandpredictionerrorinformation,inSocietyofPhoto-OpticalInstrumentationEngineers(SPIE)ConferenceSeries,vol.2094,1993,pp.235. [44] A.OrtegaandK.Ramchandran,Rate-distortionmethodsforimageandvideocompression,IEEESignalProcessingMagazine,vol.15,no.6,pp.23,1998. [45] T.Berger,Ratedistortiontheory:Amathematicalbasisfordatacompression.Prentice-Hall,EnglewoodCliffs,NJ,1971. [46] J.ProakisandM.Salehi,Digitalcommunications,4thed.McGraw-hillNewYork,Aug2000. [47] I.QUANTIZERSandR.ROBERTS,AnalogSourceDigitization:AComparisonofTheoryandPractice,IEEETransactionsonInformationTheory,1967. [48] N.FarvardinandJ.Modestino,Optimumquantizerperformanceforaclassofnon-Gaussianmemorylesssources,IEEETransactionsonInformationTheory,vol.30,no.3,p.485,1984. [49] S.LinandD.J.Costello,ErrorControlCoding:FundamentalsandApplications.Prentice-Hall,1983. [50] Y.ShohamandA.Gersho,Efcientbitallocationforanarbitrarysetofquantizers.IEEETrans.Acoust.SpeechSignalProcess.,vol.36,no.9,pp.1445,1988. [51] H.EverettIII,GeneralizedLagrangemultipliermethodforsolvingproblemsofoptimumallocationofresources,OperationsResearch,vol.11,no.3,pp.399,1963. [52] J.G.Proakis,DigitalCommunications,4thed.McGraw-Hill,2001. [53] E.MalkamakiandH.Leib,Evaluatingtheperformanceofconvolutionalcodesoverblockfadingchannels,IEEETransactionsonInformationTheory,vol.45,no.5,p.1643,1999. 168

PAGE 169

[54] S.KallelandC.Leung,EfcientARQschemeswithmultiplecopydecoding,IEEETransactionsonCommunications,vol.40,no.3,pp.642,1992. [55] K.RamchandranandM.Vetterli,Bestwaveletpacketbasesinarate-distortionsense,IEEETransactionsonImageProcessing,vol.2,no.2,pp.160,1993. [56] G.SullivanandT.Wiegand,Rate-distortionoptimizationforvideocompression,IEEESignalProcessingMagazine,vol.15,no.6,pp.74,1998. [57] Z.He,Y.Kim,andS.Mitra,Object-levelbitallocationandscalableratecontrolforMPEG-4videocoding,inMPEG-4.2001ProceedingsofWorkshopandExhibitionon.IEEE,2002,pp.63. [58] X.Li,N.Oertel,A.Hutter,andA.Kaup,Rate-distortionoptimizedframelevelratecontrolforH.264/AVC,inEuropeanSignalProcess.Conf.(EUSIPCO),Lausanne,Switzerland,2008. [59] G.Begin,D.Haccoun,andC.Paquin,Furtherresultsonhigh-ratepuncturedconvolutionalcodesforViterbiandsequentialdecoding,IEEETransactionsonCommunications,vol.38,no.11,pp.1922,1990. [60] S.Ma,Z.Li,andF.Wu,Proposeddraftofadaptiveratecontrol,inJointVideoTeam(JVT)ofISO/IECMPEG&ITU-TVCEG,Doc.JVT-H017r3,8thMeeting,Geneva,2003,pp.20. [61] Z.Li,W.Gao,F.Pan,S.Ma,K.Lim,G.Feng,X.Lin,S.Rahardja,H.Lu,andY.Lu,AdaptiveratecontrolforH.264,JournalofVisualCommunicationandImageRepresentation,vol.17,no.2,pp.376,2006. [62] Y.Kim,Z.He,andS.Mitra,AnovellinearsourcemodelandauniedratecontrolalgorithmforH.263/MPEG-2/MPEG-4,inProceedingsofICASSP,vol.3.IEEE,2001,pp.1777. [63] Z.HeandS.Mitra,AlinearsourcemodelandauniedratecontrolalgorithmforDCTvideocoding,IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.12,no.11,pp.970,2002. [64] M.SimonandD.Divsalar,SomenewtwiststoproblemsinvolvingtheGaussianprobabilityintegral,IEEETransactionsonCommunications,vol.46,no.2,1998. 169

PAGE 170

BIOGRAPHICALSKETCH ZhifengChenreceivedtheB.E.degreefromtheEastChinaUniversityofScienceandTechnology,Shanghai,China,in2001,andtheM.S.andPh.D.degreesfromtheUniversityofFlorida,Gainesville,Florida,in2008and2010respectively.From2002to2003,hewasanengineerinEPSON(China),andfrom2003to2006,hewasaseniorengineerinPhilips(China),bothworkinginmobilephonesystemsolutiondesign.FromMay2009toAug2009,hewasaninterninDolby,Burbank,CA,wherehehadworkedinerrorresilientratedistortionoptimization.HejoinedthestrategicengineeringdepartmentatInterdigital,KingofPrussia,PA,in2010,whereheiscurrentlyastaffengineerworkinginthevideocodingresearch.Hisresearchinterestsincludelow-complexityvideoandimagecompression,perceptualvideocoding,Error-resilientvideocoding,rate-distortionoptimization,ratecontrol,cross-layerdesign,informationtheory,statistics,signalprocessing.Heistheauthorofseveraljournalandconferencepapersandhasbeenawardedfourpatents. 170