ComputerVisionandImageUnderstanding81,269–284(2001) doi:10.1006/cviu.2000.0888,availableonlineathttp://www.idealibrary.comon Estimating Anthropometry and Pose from a Single Uncalibrated Image CarlosBarro´nandIoannisA.Kakadiaris DepartmentofComputerScience,UniversityofHouston,4800Calhoun,Houston,Texas77204-3475 E-mail:[email protected],[email protected] AcceptedJuly29,2000 Inthispaper,wepresentafour-steptechniqueforsimultaneouslyestimatingahu- man’santhropometricmeasurements(uptoascaleparameter)andposefromasingle uncalibratedimage.Theuserinitiallyselectsasetofimagepointsthatconstitutethe projectionofselectedlandmarks.Usingthisinformation,alongwithaprioristatis- ticalinformationaboutthehumanbody,asetofplausiblesegmentlengthestimates isproduced.Inthethirdstep,asetofplausibleposesisinferredusingageometric methodbasedonjointlimitconstraints.Inthefourthstep,poseandanthropometric measurementsareobtainedbyminimizinganappropriatecostfunctionsubjectto theassociatedconstraints.Thenoveltyofourapproachistheuseofanthropometric statistics to constrain the estimation process that allows the simultaneous estima- tion of both anthropometry and pose. We demonstrate the accuracy, advantages, and limitations of our method for various classes of both synthetic and real input data. (cid:176)c 2001AcademicPress 1. INTRODUCTION Video-basedthree-dimensionalhumanmotiontrackingisanimportantandchallenging research problem. Its importance stems from numerous applications such as: (1) perfor- mancemeasurementforhumanfactorsengineering,(2)postureandgaitanalysisfortrain- ingathletesandphysicallychallengedpersons,(3)humanbody,hands,andfaceanimation, and(4)automaticannotationofhumanactivitiesinvideodatabases.Thechallengestoward thegeneralapplicabilityofavision-basedthree-dimensionaltrackingsystemonrealdata includethefollowing: † Data from one camera only: There are several applications for which the video recordings from only one view are available (e.g., for analyzing the motion of famous artistsinhistoricalrecordings).Inaddition,thecameramightbemoving,possiblyzooming inandout. 269 1077-3142/01$35.00 Copyright(cid:176)c 2001byAcademicPress Allrightsofreproductioninanyformreserved. 270 BARRO´NANDKAKADIARIS FIG.1. (a)Instanceofanimagethatcanbehandledbyouralgorithm.(b)Instanceofanimagethatcannot behandledbyouralgorithm. † Modelacquisition:Thereisnosuchthingasan“average”humanandthatmakes theselectionofageometricmodelformodel-basedtrackingdifficult. † Modeling:Thehumanmodelsthatarecurrentlyusedformotionestimationdonot incorporatestatisticalanthropometricinformation. Ourgoalistodevelopamodel-basedsystemfortrackinghumansfrommonocularimages. Inthispaper,wepresentatechniqueforsimultaneousanthropometryandposeestimation fromthefirstframeofanimagesequence.Theinputtothealgorithmistheimagecoordinates ofthevisiblelandmarksfromthehumansubject(asselectedbytheuser)intheimageunder examination (Fig. 6a). The output is the subject’s anthropometric measurements (up to a scale parameter) and his/her pose in the specific image (Fig. 6b). By the term “up to a scale,”werefertothefactthatfromasingleuncalibratedcamerawecannotinferabsolute lengths(like“upper-leg-length”and“shoulder-width”)butonlyratiosoflengths.Therefore, inthefollowingwhenwerefertotheestimationoftheanthropometricmeasurements,we implytheestimationofratiosoflengthslike“upper-leg-length”over“shoulder-width.”The noveltyofourapproachistheuseofanthropometricstatisticstoconstraintheestimation process. The impact of our method lies in the ability to semi-automate the initialization phaseformodel-basedhumantrackingmethodsfromasinglecamera.Aswillbeexplained in later sections, our method can handle images like the one depicted in Fig. 1a, but not imagesliketheonedepictedinFig.1b. The remainder of this paper describes our technique in more detail. In Section 2, we reviewpriorworkinthearea,whileinSection3weformulatetheproblemandweoffer adetailedanalysisofthegeometricandstatisticalrelationships.InSection4,wedescribe ourmethodindetail,andinSection5wepresentanumberofresultsfromoursystem. 2. PRIORWORK Twoofthechallengesinmodel-basedhumantrackingalgorithmsare:(1)theacquisition ofanaccuratehumanbodymodelthatwillbeemployedasthemodel,and(2)theinitializa- tionofthemodelinthefirstframeoftheimagesequence.Concerningmodelacquisition, existing approaches use either models of the human body whose parts are approximated withsimpleshapesandtheirdimensionshavebeenmanuallymeasured[10,20]ormodels whoseshapeand/ordimensionshavebeendeterminedbasedoncamerainputdata.Inthis ESTIMATINGANTHROPOMETRYANDPOSE 271 secondcategory,methodshavebeendevelopedtoobtainmodelsofhumanbodypartsfrom multiplecameras[11,12,15]orrangedata[8].Concerningpostureestimationandtrack- ing,methodshavebeenpresentedthatuseeitherone[4,6,17,21],ormultiplecameras[1, 7, 9, 13, 14, 16]. However, in most of the existing tracking approaches the user specifies anapproximatepositionandpostureforthehumanmodelatthefirstframeoftheimage sequence[6,14,19].Incontrast,BreglerandMalik[4],fortheinitializationstepoftheir humantrackingmethod,minimizeacostfunctionoverposition,angles,andbodydimen- sions. In particular, a user selects the 2D joint locations and then a 3D pose is found by minimizingthesumofthesquareddifferencesbetweentheprojectedmodeljointlocations andthecorrespondingmodeljointlocations.Theauthorsmentionthattheyhadgoodresults withaquasi-Newtonmethodandamixedquadraticandlinesearchprocedure.However,no informationisprovidedabouttheaccuracyandrepeatabilityoftheirmethod,norforwhat classofposturesandhumanbodydimensionsdoestheirmethodsucceed.Thecontribution ofourpaperisasystematicstudyandatechniquethattakesintoconsiderationstatistical anthropometricinformationtoconstraintheestimationprocess. 3. ANALYSIS In this section, first we formulate the problem, then we present a stick human body model(SM)thatincorporatesstatisticalanthropometricinformation,andfinallyweprovide adetailedanalysisofthegeometricalandstatisticalrelationshipsoftheSM’ssegments. 3.1. ProblemStatement Thehumanmusculosketelalsystemiscomposedofaseriesofjointedlinks,whichcanbe approximatedasrigidbodies.Humanmotionestimationisaimedatquantitativelydescrib- ingthespatialmotionofbodysegmentsandthemovementsofthejointsconnectingthose segments. A hallmark of the individuality of people we encounter daily results from the variationoftheiranthropometricmeasurements.Ifweassumethatwehavenoanthropo- metricinformationforthesubjectthatweareobserving,theproblemofanthropometryand poseestimationfromasingleimagecanbeformulatedasfollows:Givenasetofpointsin animagethatcorrespondtotheprojectionoflandmarkpointsofahumansubject,estimate boththeanthropometricmeasurements(uptoascale)ofthesubjectandhis/herposethat bestmatchtheobservedimage. 3.2. StickHumanBodyModel Forthepurposesofthisresearch,wehavedevelopedagenericstickhumanbodymodel (Fig.2)inspiredbythehumanbodymodelemployedattheCenterforHumanModeling andSimulationatUniversityofPennsylvania[3].Themodelconsistsofasetofsegments connected by joints. Specifically, a stick model is a tree (s;S;A), where S is a set of sites/landmarksandAisacollectionofedges(segments)withendpointsinS,ands 2S is the root. In our case, ADfHD, RY, LY, NK, UT, RC, LC, RUA, LUA, RLA, LLA, RHD, LHD, LT, RHP, LHP, RUL, LUL, RLL, LLL, RF, LFg as enumerated in Table 1, andthesetoflandmarksconsistsofasetofjointsJ Dfat,sp,la,lc,le,lh,lk,ls,lw,ra, rc, re, rh, rk, rs, rw, wtg (information about the SM’s joints is provided in Table 2) and other landmarks MDfry (right eye), ly (left eye), rhd (base of the right middle finger), 272 BARRO´NANDKAKADIARIS FIG.2. Stickhumanbodymodel(SM)anditsassociatedcoordinatesystems. lhd(baseoftheleftmiddlefinger),rf(tipoftherightfoot),if(tipoftheleftfoot)g(S D J [M). Alocalcoordinatesystemisattachedtoeachbodypart.Thekinematicsarerepresentedby atransformationtreewhoserootisthesubject’scoordinatesystemandwhoseleavesarethe coordinatesystemsofhead,hands,andfeet.Theoriginofthesubject’scoordinatesystem isthewaistjoint.Figure2depictsthelocalcoordinatesystemsofthestickhumanmodel, which corresponds to the joints listed in Table 2. Note that every joint has translational and rotational degrees of freedom. The joint’s translational degrees of freedom allow for segment scaling, and they are restricted by anthropometric proportionality constraints as explainedinSection4.3.Eachrotationaldegreeoffreedomhasanupperlimitandalower limitthatrestrictstheposeestimationtoplausiblehumanpostures.Thedefaultdataforthe jointsareextractedfrom[18]. 3.3. GeometricRelationships Inthissection,wewillexaminetheforeshorteningofthebodysegmentsintheimage,un- dertheassumptionofscaledorthographicprojection.LetcD[X ;Y ;Z ]>betheoriginof c c c thecamera(seeFig.3)andlet’sassumethattheimageplaneislocatedatZ alongtheZaxis im ofthecamera.Asknown,underscaledorthographicprojectionthepointP D[X ;Y ;Z ]> 1 1 1 1 ESTIMATINGANTHROPOMETRYANDPOSE 273 TABLE1 NamesoftheSM’sSegments ID Segment ID Segment HD Head NK Neck LY Lefteye RY Righteye LT Lowertorso UT Uppertorso LC Leftclavicle RC Rightclavicle LUA Leftupperarm RUA Rightupperarm LLA Leftlowerarm RLA Rightlowerarm LHD Lefthand RHD Righthand LHP Lefthip RHP Righthip LUL Leftupperleg RUL Rightupperleg LLL Leftlowerleg RLL Rightlowerleg LF Leftfoot RF Rightfoot (see Fig. 3) projects to the point p D[x ;y ;z ]> D[X C‚ (X ¡X );Y C‚ (Y ¡ 1 1 1 1 c 1 1 c c 1 1 Y );Z ]>, where ‚ D(Z ¡Z )=(Z ¡Z ). Similarly, the point P D[X ;Y ;Z ]> c im 1 im c 1 c 2 2 2 2 projects to the point p D[x ;y ;z ]>D[X C‚ (X ¡X );Y C‚ (Y ¡Y );Z ]>, 2 2 2 2 c 2 2 c c 2 2 c im where ‚ D(Z ¡Z )=(Z ¡Z ). If we assume that this point lies on the same plane 2 im c 2 c (normaltothecameraZ axis)asthepointP ,then‚ D‚ .Thus,foranypointontheline 1 1 2 P P ,itsprojectionisgivenbytheequation[x;y]> D‚ S[X;Y;Z]>,where 1 2 1 • ‚ 1 0 0 SD : 0 1 0 Similarly,foranypointonthelineP P ,itsprojectionisgivenbytheequation[x;y]> D 3 4 ‚ S[X;Y;Z]>,where‚ D(Z ¡Z )=(Z ¡Z )D(Z ¡Z )=(Z ¡Z )D‚ .Iffi is 3 3 im c 3 c im c 4 c 4 z TABLE2 InformationRelatedtotheJointsoftheStickModel ID Joint From To DOF PR at atlantooccipital NK HD Tz⁄Rz⁄Ry⁄Rx 3 sp solarplexus UT NK Tz⁄Ry⁄Rz⁄x 2 la leftankle LLL LF Tx⁄Rz⁄Rx⁄Ry 4 lc leftclavicle UT LC Tz⁄Rx⁄Ry 3 le leftelbow LUA LLA Tz⁄Ry 5 lh lefthip LT LUL Tz⁄Rz⁄Rx⁄Ry 2 lk leftknee LUL LLL Tz⁄R-y 3 ls leftshoulder LC LUA Tz⁄Rz⁄Rx⁄Ry 4 lw leftwrist LLA LHD Tz⁄Ry⁄Rx⁄Rz 6 ra rightankle RLL RF Tx⁄R-z⁄R-x⁄Ry 4 rc rightclavicle UT RC Tz⁄R-x⁄Ry 3 re rightelbow RUA RLA Tz⁄Ry 5 rh righthip LT RUL Tz⁄R-z⁄R-x⁄Ry 2 rk rightknee RUL RLL Tz⁄R-y 3 rs rightshoulder RC RUA Tz⁄R-z⁄R-x⁄Ry 4 rw rightwrist RLA RHD Tz⁄Ry⁄R-x⁄R-z 6 wt waist LT UT Tz⁄Ry⁄Rz⁄Rx 1 274 BARRO´NANDKAKADIARIS FIG.3. NotationpertainingtoProposition3.1. arealnumbersuchthat Z ¡Z (1Cfi )D 1 c; (1) z Z ¡Z 3 c then Z ¡Z D(Z ¡Z )=(1Cfi ) and ‚ D‚ (1Cfi ). Therefore, the scaled ortho- 3 c 1 c z 3 1 z graphic projection for the points of P P is given by [x;y]> D‚ (1Cfi )S[X;Y;Z]>. 3 4 1 z Let L DkP ¡P kandl Dkp ¡p k.Then 12 2 1 12 2 1 l12 D((x2¡x1)2C(y2¡y1)2)12 D‚1((X2¡X1)2C(Y2¡Y1)2)12 D‚1L12: Similarly, we can obtain thatl D‚ L , where L DkP ¡P k andl Dkp ¡p k. 34 3 34 34 4 3 34 4 3 Usingtherelation‚ D‚ (1Cfi ),weobtainthatl D‚ (1Cfi )L .Finally,theratio 3 1 z 34 1 z 34 ofl andl isgivenby 12 34 l ‚ L L 12 D 1 12 D 12 ; l ‚ (1Cfi )L (1Cfi )L 34 1 z 34 z 34 whichimpliestherelation L l 12 D(1Cfi ) 12; (2) z L l 34 34 whichsuggeststhefollowingproposition. PROPOSITION3.1. Forsegmentsthatlieinplanesalmostparalleltotheimageplane; theratioofsegmentlengthsin3Dissimilartotheratioofthelengthsofthecorresponding segmentsprojectedtotheimageplane. Proof. Since the segments lie in planes almost parallel to the image plane, fi is very z small.Thus,theresultisobtainedfromEq.(2). n 3.4. BuildingaCadreFamily Usingtheanthropometricmeasurementsin[18],webuildforourSMacadrefamily,also known as a boundary family [2]. The cadre family is a multivariate representation of the extremesofthepopulationdistribution.Ithastheabilitytospanthemultivariatespaceina systematicfashionandtocaptureasignificantamountofthevarianceinthespaceusinga smallnumberofsamplehumanmodels.Theprobabilitydensityfunctionofthemultivariate normaldistributionisdefinedby • ‚ 1 1 f(x)D p exp ¡ (x¡s)>6¡1(x¡s) ; (3) (2…)kj§j 2 ESTIMATINGANTHROPOMETRYANDPOSE 275 FIG.4. SamplemodelsfromthedistributionofSMs(7outof143). where k is the number of dimensions. In our case, the variables are the lengths of the 22 segmentsofourstickmodel(seeTable1),xisarandomvector,andsand§arethemean andthecovariancematricesofthepopulation,respectively. Thequadraticform Q(x)D(x¡s)>§(x¡s)definesahyperellispoidsurfacearounds, whoseshapedependson§.Wecomputetheprincipalcomponentsof§,andweselectseven E (i D1;:::;7)eigenvectorswiththelargesteigenvaluessuchthat‚ >‚ >¢¢¢>‚ . i 1 2 7 Note that each eigenvector corresponds to the original 22 variables associated with the P lengths as E D[e ;e ;:::;e ]>. In addition, all linear combinations 7 fi E Cs i i1 i2 i22 iD1 i i constrainedbytherelation X7 fi2 •1 (4) i iD1 lieintheinteriorofthehyperellispoidrelatedwith§ands.Byselectingthepositivenonzero coefficientsthatsatisfyEq.(4)(jfi jDjfi jD¢¢¢Djfi j6D0),wecanbuildafamilyofstick 1 2 7 modelsthatcorrespondtoallequidistantdiscretecombinationsoftheselectedeigenvectors: P 7 fl E Cs,wherefl 2f¡p1 ;p1 g.Furthermore,weaddinourcadrefamilysandthe iD1 i i i 7 7 axialpoints§E Cs;i D1;:::;7.ThetotalnumberofSMsproducedis143(ingeneralthis i proceduregives2n C2nC1components,wheren isthenumberofprincipalcomponent vectorskept).AsampleoftheseSMsisdepictedinFig.4. 3.5. DeterminingaCoveringSet In this section, we describe our algorithm for determining the set of anthropometric proportions that will be used by our iterative estimator to reach an anthropometrically plausiblesolution.Inordertoreducethenumberofvariables,weassumethattheleftand rightsidesofthehumanbodyaresymmetrical,andweonlyconsidertheleftside.Also,we donotemploythesegmentsassociatedwiththeeyes,thehands,andtheatlantooccipital joint.Thus,wefocusourstatisticalmodelingoneightsegmentsoftheSMasenumerated inTable3.Inthefollowing,letLDfl g8 bethesetofsegmentlengths,andRDfr g28 i iD1 k kD1 be the set of the corresponding ratios of segment lengths. These ratios are computed by dividingthelengthsofanytwodifferentsegmentsthatbelongtoL. TABLE3 TheSegmentsUsedforComputingtheCoveringSet l l l l l l l l 1 2 3 4 5 6 7 8 UTCLT LC LUA LLA LHP LUL LLL LF 276 BARRO´NANDKAKADIARIS TABLE4 SegmentLengthsandRatiosAssociatedwithOur CadreFamilyofSMs l1;1 l2;1 ... l8;1 ) r1;1 r2;1 ... r28;1 l1;2 l2;2 ... l8;2 ) r1;2 r2;2 ... r28;2 . . . . . . . . . . . . . . . . ... . . . . ... . l1;143 l2;143 ... l8;143 ) r1;143 r2;143 ... r28;143 Mean ) „(r ) „(r ) ... „(r ) 1 2 28 Variance ) (cid:190)(r ) (cid:190)(r ) ... (cid:190)(r ) 1 2 28 Weformthese(8)D28ratiosasfollows: 2 8 <lm;q if„(l )>„(l ) rk;q D:ln;q n m ln;q otherwise, lm;q where1•m <n •8,k D1;:::;28,andq D1;:::;143.Theindexformulathatrelates k withm andnisgivenby:k Dn¡mC(m¡1)(8¡ m). 2 InTable4therowsdepictthatthelengthsoftheeightselectedsegmentswillbeusedto producetheSM’sratios,andtheassociatedmeansandvariances.Thesegmentlengthsand theratiosaredenotedbyli;q andrk;q respectively,andq D1;:::;143denotestheindexof amemberofourcadrefamily(seeSection3.4).UsingthevaluesinTable4,wecompute thematrixCD[c ]oftheabsolutevalueoftheratiocorrelationas ij fl fl cij Dflflfl„[(ri;q ¡„(cid:190)((rri)))(cid:190)(r(rj;q)¡„(rj))]flflfl; i; j D1;:::;28: j j DEFINITION3.1. LetVD[vki]bea28£8matrix.Forallrk 2R,andli 2Lwedefine thefollowingfunctions: (cid:190)(r ) † weight(r )D k k „(r ) k Thevariance(cid:190)(r )isanindicationoftheprecisionofthestatisticalinformationconcerning k theratior .Therefore,whentheweightofaratioissmall,theratioismoreconstrained. k ( 1 if(r D lm ^(l Dl _l Dl )) † cover(r ;l )D k ln i m i n k i c otherwise; kb wherec Dmax fc g,and(r D ld ^(l Dl _l Dl ).Thevalueofcover(r ;l )mea- kb f kf f le i d i e k i surestowhatextenttheratior constrainsthelengthl . P k i † degree(r ;V)D v , wherev Dcover(r ;l ). k i ki ki k i Thedegreefunctionmeasuresthecorrelationofaratiowithallthesegments. degree(r ;V) † goodness(r ;V)D k . k weight(r ) k Thegoodnessfunctionisemployedindeterminingwhichratioswillbeusedtoconstrain theestimationprocessasexplainedinAlgorithm1. DEFINITION 3.2. A set B ‰R (the set of ratios) is a covering set of L (the set of segments),if8l 2L;9r Dl =l 2Rsuchthatl Dl _l Dl . i k m n i m i n ESTIMATINGANTHROPOMETRYANDPOSE 277 Ourobjectiveistofindasetofratiosthatconstrainallsegmentlengths.Thus,weformulate theproblemasasetcoveringproblemasfollows:IfListhesetofaSM’ssegmentlengths andRisthesetofthecorrespondingratios,findthecoveringsetBforL.Inthefollowing, weoutlinethestepsofthealgorithm. ALGORITHM1 (RATIOSELECTION). 1. B :D; 2. 8(r ;l )2(R£L);V[k;i]:Dcover(r ;l ) k i k i 3. 8i;l 2L;care(l ):D0 i i 4. while(True)do 5. 8j;r 2RnB;d[j]Ddegree(r ;V) j j 6. 8j;r 2RnB;g[j]Dgoodness(r ;V) j j 78.. mB ::DDBarg[frmag;xcj;arjr2eR[dnB]fCg[DjV]g[amn;ddr]m;cDarellde[e]CDV[m;e] m 9. if(care[i]‚1);8 ;l 2L i i 10. then 11. returnB 12. else 13. 8j;r 2RnB;V[j;d]:Dmaxf0;V[j;d]¡care[d]g j 14. 8j;r 2RnB;V[j;e]:Dmaxf0;V[j;e]¡care[e]g j 15. endwhile TheresultingsetisB Df LC ; LLA; LHP; LF ; LF g. UTCLT LUA LUA LUL LLL PROPOSITION 3.2. The Ratio Selection algorithm (outlined above) always returns a coveringset. Proof. The set R is the maximum covering set of L. In the worst case, the values of goodness(r ;V)couldallbeequal.However,sincestep7inAlgorithm1returnsoneindex, j the corresponding ratio is addedtoB anddiscarded from R, and therefore the algorithm alwaysreturnsacoveringset. n 4. ANTHROPOMETRYANDPOSEESTIMATION Our technique for simultaneously estimating the anthropometry and the pose from a singleuncalibratedimagehasthefollowingsteps[5]: ALGORITHM2 (ANTHROPOMETRYANDPOSEESTIMATION). Step1:Selectionofprojectedlandmarks Step2:Initialanthropometricestimates Step3:Initialposeestimates Step4:Iterativeminimizationoverlengthsandangles Inthefollowingsubsectionswedescribeeachstepindetail. 4.1. SelectionofProjectedLandmarks Wehavedevelopedasimpleuserinterfacethatallowstheusertoselecttheprojection of visible landmarks of the subject’s body (see Fig. 6a). In addition, the user marks the segmentswhoseorientationisalmostparalleltotheimageplane.Forexample,inFig.6a thegreendotsdepictprojectionoflandmarksassociatedwithsegmentswhoseorientation 278 BARRO´NANDKAKADIARIS isalmostparalleltotheimageplane,andthebluedotsdepictallotherselectedlandmarks. Althoughinformationfrombothtypesoflandmarkswillbeusedforposeestimation,initial lengthestimateswillbebasedontheprojectedlengthofthesegmentswhoseorientationis almostparalleltotheimageplaneonly. 4.2. InitialAnthropometricEstimates Ourbasicassumptionisthatthereisanumberofsegmentswhoseorientationisalmost paralleltotheimageplaneandthereforewecanobtaingoodapproximationratiosforthem usingProposition3.1.Thus,ouralgorithmcannothandleimagesliketheonedepictedin Fig. 1b, since one cannot locate segments that are almost parallel to the image plane to obtainreliableinitialanthropometricestimates. Leth betheprojectedlengthofasegmenti ontheimage,andletI ‰f1;:::;8gbethe i indexsetofthesesegmentswhoseorientationisalmostparalleltotheimageplane.Using themeasurementsh (i 2I),wecomputeallpossibleratioss thatcorrespondtotheratios i k ofSMsinTable4asfollows, 8 <hm if„(l )>„(l ) s D hn n m k : hn otherwise, hm wherek 2KDfk 2f1;:::;28gjk Dn¡m(m¡1)(8¡ m);m;n 2Ig.Basingourselec- 2 tion on these ratios, we select one SM from the family of 143 SMs whose length ratios closely match the ratios computed from the image using the Mahalanobis distance. To accomplishthisgoal,wedetermine ˆ ! X X q⁄ Dargmin (rk;q ¡sk) vkj(rj;q ¡sj) ; q k2K j2K whererk;q;rj;q aredefinedinTable4,q D1;:::;143; vkj D„[(rk;q ¡„(rk))(rj;q ¡„(rj))] arecovariancecoefficientsoftheratios,andk; j 2K. [v ]DO¡1 kj andOisthecovariancematrixoftheratiosfrk;qgk2•;jD1;:::;143. Thelengthmeasurementsoftheselectedq⁄stickmodelareusedasinitialsegmentlength estimates. Tofacilitatetheoverallunderstandingofouralgorithm,wefirstpresentthefourthstep inthenextsection,andthenwepresentthethirdstepinSection4.4. 4.3. IterativeMinimizationoverLengthsandAngles Thevariableswewanttoestimatearethelengthsofthebodysegmentsandtheirpose. Therefore,wewillsolveasystemofequationswherepriorinformationaboutthehuman body(e.g.,relationsbetweenlengthsofsegments)willprovideconstraintstoanoptimization thatminimizesthediscrepancybetweenthesynthesizedappearanceoftheSM(forthatpose) andtheimagedataofthesubjectinthegivenimage. Asmentionedearlier,theuserselectsasetofpointsontheimagethatcorrespondtothe projectionofthesitesofthestickmodel.Foreachofthesepoints,wesetupapoint-to-line