Chapter 2 Approximation and Interpolation Thepresentchapterisbasicallyconcernedwiththeapproximationoffunctions.The functionsin questionmaybefunctionsdefinedona continuum–typicallya finite interval–orfunctionsdefinedonlyonafinitesetofpoints.Thefirstinstancearises, forexample,inthecontextofspecialfunctions(elementaryortranscendental)that onewishestoevaluateasapartofasubroutine.Sinceanysuchevaluationmustbe reducedtoafinitenumberofarithmeticoperations,wemustultimatelyapproximate thefunctionbymeansofa polynomialorarationalfunction.Thesecondinstance is frequently encountered in the physical sciences when measurements are taken of a certain physical quantity as a function of some other physical quantity (such as time). In either case one wants to approximate the given function “as well as possible”intermsofothersimplerfunctions. The general scheme of approximation can be described as follows. We are giventhe functionf to be approximated,alongwith a class ˆ of“approximating functions”' anda“norm”k (cid:2) kmeasuringtheoverallmagnitudeoffunctions.We arelookingforanapproximation'O 2ˆoff suchthat kf (cid:3)'Ok(cid:4)kf (cid:3)'k forall ' 2ˆ: (2.1) Thefunction'O is calledthe bestapproximationto f fromthe classˆ, relativeto thenormk (cid:2) k. The class ˆ is called a (real) linear space if with any two functions ' , 1 ' 2 ˆ it also contains ' C' and c' for any c 2 R, hence also any (finite) 2 1 2 1 linear combination of functions ' 2 ˆ. Given n “basis functions” (cid:2) 2 ˆ, i j j D 1;2;:::;n,wecandefinealinearspaceoffinitedimensionnby 8 9 < Xn = ˆDˆ D ' W '.t/D c (cid:2) .t/; c 2R : (2.2) n : j j j ; jD1 Examplesoflinearspaces˚. 1.ˆDP :polynomialsofdegree(cid:4)m.Abasisfor m P is, forexample,(cid:2) .t/ D tj(cid:2)1, j D 1;2;:::;mC1, so thatn D m C 1. m j Polynomials are the most frequently used “general-purpose” approximants for W.Gautschi,NumericalAnalysis,DOI10.1007/978-0-8176-8259-0 2, 55 ©SpringerScience+BusinessMedia,LLC1997,2012 56 2 ApproximationandInterpolation dealing with functions on bounded domains (finite intervals or finite sets of points). One reason is Weierstrass’s theorem, which states that any continuous functioncanbe approximatedona finite intervalasclosely as onewishesby a polynomialofsufficientlyhighdegree. 2. ˆ DSk.(cid:3)/:(polynomial)splinefunctionsofdegreemandsmoothnessclassk m onthesubdivision (cid:3)W aDt1 <t2 <t3 <(cid:2)(cid:2)(cid:2)<tN(cid:2)1 <tN Db of the interval [a;b]. These are piecewise polynomials of degree (cid:4) m pieced togetheratthe“joints” t2;:::;tN(cid:2)1 in sucha way thatallderivativesupto and includingthektharecontinuousonthewholeinterval[a;b],includingthejoints: ˇ Sk.(cid:3)/Dfs 2CkŒa;b(cid:4)W sˇ 2P ; i D1;2;:::;N (cid:3)1g: m Œti;tiC1(cid:4) m We assume here 0 (cid:4) k < m; otherwise, we are back to polynomialsP (see m Ex.68).Wesetk D (cid:3)1ifweallowdiscontinuitiesatthejoints.Thedimension ofSk.(cid:3)/isnD.m(cid:3)k/(cid:2).N (cid:3)2/CmC1(seeEx.71),buttofindabasisisa m nontrivialtask;formD1,seeSect.2.3.2. 3. ˆ D T Œ0;2(cid:2)(cid:4): trigonometricpolynomialsof degree (cid:4) m on [0, 2(cid:2)]. These m arelinearcombinationsofthebasicharmonicsuptoandincludingthemthone, thatis, (cid:2) .t/Dcos .k(cid:3)1/t; k D1;2;:::;mC1I k (cid:2)mC1Ck.t/Dsinkt; k D1;2;:::;m; where now n D 2m C 1. Such approximants are a natural choice when the functionf to be approximatedis periodicwith period2(cid:2). (If f hasperiod p, onemakesapreliminarychangeofvariablest 7!t (cid:2)p=2(cid:2).) 4. ˆD En:exponentialsums.Forgivendistinct˛j >0,onetakes(cid:2)j.t/ De(cid:2)˛jt, j D 1;2;:::;n. Exponential sums are often employed on the half-infinite intervalRC:0(cid:4)t <1,especiallyifoneknowsthatf decaysexponentiallyas t !1. Notethattheimportantclassofrationalfunctions, ˆDR Df' W ' Dp=q; p 2P ; q 2P g; r;s r s isnotalinearspace.(Whynot?) Possible choices of norm – both for continuous and discrete functions – and the type of approximation they generate are summarized in Table 2.1. The continuouscase involvesaninterval[a;b] anda “weightfunction”w.t/ (possibly w.t/(cid:5)1)definedon[a;b]andpositiveexceptforisolatedzeros.Thediscretecase involvesa setofN distinctpointst ,t ;:::;t alongwithpositiveweightfactors 1 2 N 2 ApproximationandInterpolation 57 Table2.1 Typesofapproximationandassociatednorms Continuousnorm Approximation Discretenorm kuk1D maxju.t/j L1 kuk1 D max ju.ti/j a(cid:2)t(cid:2)b 1(cid:2)i(cid:2)N Uniform Chebyshev Z b XN kuk D ju.t/jdt L kuk D ju.t /j 1 1 1 i aZ iD1 b XN kuk D ju.t/jw.t/dt WeightedL kuk D wju.t /j 1;w 1 1;w i i aZ b !12 i DX1N !12 kuk D ju.t/j2w.t/dt WeightedL kuk D wju.t /j2 2;w 2 2;w i i a iD1 Leastsquares w ;w ;:::;w (possibly all equal to 1). The interval [a;b] may be unboundedif 1 2 N the weightfunctionwis such thatthe integralextendedover[a;b],whichdefines thenorm,makessense. Hence,wemaytakeanyoneofthenormsinTable2.1andcombineitwithanyof theprecedinglinearspacesˆtoarriveatameaningfulbestapproximationproblem (2.1).Inthecontinuouscase,thegivenfunctionf,andthefunctions' oftheclass ˆ,ofcourse,mustbedefinedon[a;b]andsuchthatthenormkf (cid:3)'kmakessense. Likewise,f and' mustbedefinedatthepointst inthediscretecase. i Notethatifthebestapproximant'O inthediscretecaseissuchthatkf (cid:3)'OkD0, then 'O.t / D f.t / for i D 1;2;:::;N. We then say that 'O interpolates f at i i the points t and we refer to this kind of approximationproblem as an interpola- i tionproblem. Thesimplestapproximationproblemsaretheleastsquaresproblemandthein- terpolationproblem,andtheeasiestspaceˆtoworkwiththespaceofpolynomials of givendegree.These are indeedthe problemswe concentrateon in this chapter. In the case of the least squares problem,however,we admit generallinear spaces ˆ of approximants,and also in the case of the interpolation problem, we include polynomialsplinesinadditiontostraightpolynomials. Beforewestartwiththeleastsquaresproblem,weintroduceanotationaldevice that allows us to treat the continuous and the discrete case simultaneously. We define,inthecontinuouscase, 8 ˆˆˆˆˆˆ<Z0 tif t <a .whenever (cid:3)1<a/; w.(cid:6)/d(cid:6) if a (cid:4)t (cid:4)b; (cid:5).t/D (2.3) ˆˆˆˆˆˆ:Zab w.(cid:6)/d(cid:6) if t >b .whenever b <1/: a 58 2 ApproximationandInterpolation Thenwecanwrite,forany(say,continuous)functionu, Z Z b u.t/d(cid:5).t/D u.t/w.t/dt; (2.4) R a since d(cid:5).t/ (cid:5) 0 “outside” [a;b], and d(cid:5).t/ D w.t/dt inside. We call d(cid:5) a continuous(positive)measure.Thediscretemeasure(alsocalled“Diracmeasure”) associatedwiththepointsetft ;t ;:::;t gisameasured(cid:5)thatisnonzeroonlyat 1 2 N thepointst andhasthevaluew there.Thus,inthiscase, i i Z XN u.t/d(cid:5).t/D w u.t /: (2.5) i i R iD1 (AmoreprecisedefinitioncanbegivenintermsofStieltjesintegrals,ifwedefine (cid:5).t/tobeastepfunctionhavingjumpw att .)Inparticular,wecandefinetheL i i 2 normas (cid:2)Z (cid:3) 1 2 kuk D ju.t/j2d(cid:5).t/ ; (2.6) 2;d(cid:5) R andobtainthecontinuousorthediscretenormdependingonwhether(cid:5)istakento beasin(2.3),orastepfunction,asin(2.5). We callthesupportofd(cid:5)–anddenoteitbysuppd(cid:5)–theinterval[a;b]inthe continuouscase (assuming w positive on [a;b] exceptfor isolated zeros), and the setft ;t ;:::;t ginthediscretecase.Wesaythatthesetoffunctions(cid:2) .t/in(2.2) 1 2 N j islinearlyindependentonthesupportofd(cid:5)if Xn c (cid:2) .t/(cid:5)0 forall t 2suppd(cid:5) implies c Dc D(cid:2)(cid:2)(cid:2)Dc D0: (2.7) j j 1 2 n jD1 ExampleW thepowers(cid:2) .t/Dtj(cid:2)1,j D1;2;:::;n. j Xn Here cj(cid:2)j.t/ D pn(cid:2)1.t/isapolynomialofdegree(cid:4) n(cid:3)1.Suppose,first, jD1 that suppd(cid:5) = [a;b]. Then the identity in (2.7) says that pn(cid:2)1.t/ (cid:5) 0 on [a;b]. Clearly, this implies c D c D (cid:2)(cid:2)(cid:2) D c D 0, so that the powers are linearly 1 2 n independentonsuppd(cid:5)= [a;b].If,on theotherhand,suppd(cid:5) D ft ;t ;:::;t g, 1 2 N thenthepremisein(2.7)saysthatpn(cid:2)1.ti/ D 0,i D 1;2;:::;N;thatis,pn(cid:2)1 has NQdistinctzerosti. Thisimpliespn(cid:2)1 (cid:5) 0 onlyifN (cid:6) n. Otherwise, pn(cid:2)1.t/ D NiD1.t (cid:3)ti/ 2 Pn(cid:2)1 wouldsatisfy pn(cid:2)1.ti/ D 0, i D 1;2;:::;N, withoutbeing identicallyzero.Thus,wehavelinearindependenceonsuppd(cid:5)Dft ;t ;:::;t gif 1 2 N andonlyifN (cid:6)n. 2.1 LeastSquaresApproximation 59 2.1 LeastSquares Approximation Wespecializethebestapproximationproblem(2.1)bytakingasnormtheL norm 2 (cid:2)Z (cid:3) 1 2 kuk D ju.t/j2d(cid:5).t/ ; (2.8) 2;d(cid:5) R whered(cid:5)iseitheracontinuousmeasure(cf.(2.3))oradiscretemeasure(cf.(2.5)), andbyusingapproximants' fromann-dimensionallinearspace 8 9 < Xn = ˆDˆ D ' W '.t/D c (cid:2) .t/; c 2R : (2.9) n : j j j ; jD1 Herethebasisfunctions(cid:2) areassumedlinearlyindependentonsuppd(cid:5)(cf.(2.7)). j Wefurthermoreassume,ofcourse,thattheintegralin(2.8)ismeaningfulwhenever uD(cid:2) oruDf,thegivenfunctiontobeapproximated. j The solution of the least squares problem is most easily expressed in terms of orthogonalsystems(cid:2) relativetoanappropriateinnerproduct.Wethereforebegin j withadiscussionofinnerproducts. 2.1.1 InnerProducts Givenacontinuousordiscretemeasured(cid:5),asintroducedearlier,andgivenanytwo functionsu;vhavingafinitenorm(2.8),wecandefinetheinnerproduct Z .u;v/D u.t/v.t/d(cid:5).t/: (2.10) R (Schwarz’sinequalityj.u;v/j(cid:4)kuk (cid:2)kvk ,cf.Ex.6,tellsusthattheintegral 2;d(cid:5) 2;d(cid:5) in (2.10)is welldefined.)Theinnerproduct(2.10)hasthe followingobvious(but useful)properties: 1. symmetry:.u;v/D.v;u/; 2. homogeneity:.˛u;v/D˛.u;v/,˛ 2R; 3. additivity:.uCv;w/D.u;w/C.v;w/;and 4. positive definiteness:.u;u/ (cid:6) 0, with equalityholdingif andonlyif u (cid:5) 0 on suppd(cid:5). Homogeneityandadditivitytogethergivelinearity, .˛ u C˛ u ;v/D˛ .u ;v/C˛ .u ;v/ (2.11) 1 1 2 2 1 1 2 2 60 2 ApproximationandInterpolation Fig.2.1 Orthogonalvectorsandtheirsum in the first variableand, by symmetry,also in the second. Moreover,(2.11)easily extendstolinearcombinationsofarbitraryfinitelength.Notealsothat kuk2 D.u;u/: (2.12) 2;d(cid:5) Wesaythatuandvareorthogonalif .u;v/D0: (2.13) Thisisalwaystriviallytrueifeitheruorvvanishesidenticallyonsuppd(cid:5). Itisnowasimpleexercise,forexample,toprovetheTheoremofPythagoras: if .u;v/D0; then kuCvk2 Dkuk2Ckvk2; (2.14) where k (cid:2) k D k (cid:2) k . (From now on we use this abbreviated notation for the 2;d(cid:5) norm.)Indeed, kuCvk2 D.uCv;uCv/D.u;u/C.u;v/C.v;u/C.v;v/ Dkuk2C2.u;v/Ckvk2 Dkuk2Ckvk2; where the first equality is a definition, the second follows from additivity, the third from symmetry, and the last from orthogonality. Interpreting functions u;v as “vectors,”we can picture the configurationof u;v (orthogonal)and uCv as in Fig.2.1. Moregenerally,wemayconsideranorthogonalsystemsfu gn : k kD1 .u ;u / D0 if i ¤j; u 6(cid:5)0 onsuppd(cid:5)I i j k i;j D1;2;:::;nI k D1;2;:::;n: (2.15) 2.1 LeastSquaresApproximation 61 ForsuchasystemwehavetheGeneralizedTheoremofPythagoras, ˇˇ ˇˇ ˇˇˇˇXn ˇˇˇˇ2 Xn ˇˇ ˛ u ˇˇ D j˛ j2ku k2: (2.16) ˇˇ k kˇˇ k k kD1 kD1 Theproofisessentiallythesameasbefore.Animportantconsequenceof(2.16)is thateveryorthogonalsystemislinearlyindependentonthesupportofd(cid:5).Indeed, if the left-hand side of (2.16)vanishes, then so does the right-handside, and this, sinceku k2 >0byassumption,implies˛ D˛ D(cid:2)(cid:2)(cid:2)D˛ D0. k 1 2 n 2.1.2 TheNormal Equations We are now in a position to solve the least squares approximation problem. By (2.12),wecanwritetheL error,orratheritssquare,intheform: 2 E2Œ'(cid:4)WDk'(cid:3)fk2 D.'(cid:3)f;'(cid:3)f/D.';'/(cid:3)2.';f/C.f;f/: Inserting' herefrom(2.9)gives 0 1 0 1 Z 2 Z Z Xn Xn E2Œ'(cid:4)D @ c (cid:2) .t/Ad(cid:5).t/(cid:3)2 @ c (cid:2) .t/Af.t/d(cid:5).t/C f2.t/d(cid:5).t/: j j j j R R R jD1 jD1 (2.17) The squared L error, therefore, is a quadratic function of the coefficients c , 2 1 c ;:::;c of'.TheproblemofbestL approximationthusamountstominimizing 2 n 2 a quadratic function of n variables. This is a standard problem of calculus and is solvedbysettingallpartialderivativesequaltozero.Thisyieldsasystemoflinear algebraic equations. Indeed, differentiating partially with respect to c under the i integralsignin(2.17)gives 0 1 Z Z Xn @ E2Œ'(cid:4)D2 @ c (cid:2) .t/A(cid:2) .t/d(cid:5).t/(cid:3)2 (cid:2) .t/f.t/d(cid:5).t/; j j i i @ci R jD1 R and setting this equal to zero, interchanging integration and summation in the process,weget Xn .(cid:2) ;(cid:2) /c D.(cid:2) ;f/; i D1;2;:::;n: (2.18) i j j i jD1 These are called thenormalequationsforthe least squaresproblem.Theyform a linearsystemoftheform Ac Db; (2.19) wherethematrixAandthevectorbhaveelements 62 2 ApproximationandInterpolation A DŒa (cid:4); a D.(cid:2) ;(cid:2) /I bDŒb (cid:4); b D.(cid:2) ;f/: (2.20) ij ij i j i i i Bysymmetryoftheinnerproduct,Aisasymmetricmatrix.Moreover,Aispositive definite;thatis, Xn Xn xTAx D a x x >0 if x ¤Œ0;0;:::;0(cid:4)T: (2.21) ij i j iD1jD1 Thequadraticfunctionin(2.21)iscalledaquadraticform(sinceitishomogeneous of degree 2). Positive definiteness of A thus says that the quadratic form whose coefficients are the elements of A is always nonnegative, and zero only if all variablesx vanish. i To prove(2.21),allwe haveto dois insertthe definitionofthe a anduse the ij elementaryproperties1–4oftheinnerproduct: (cid:4) (cid:4) Xn Xn Xn Xn (cid:4)(cid:4)Xn (cid:4)(cid:4)2 xTAx D x x .(cid:2) ;(cid:2) /D .x (cid:2) ;x (cid:2) /D(cid:4) x (cid:2) (cid:4) : i j i j i i j j (cid:4) i i(cid:4) iD1jD1 iD1jD1 iD1 P Thisclearlyisnonnegative.Itiszeroonlyif n x (cid:2) (cid:5)0onsuppd(cid:5),which,by iD1 i i theassumptionoflinearindependenceofthe(cid:2) ,impliesx Dx D(cid:2)(cid:2)(cid:2)Dx D0. i 1 2 n Now itisa well-knownfactof linearalgebrathatasymmetricpositivedefinite matrixA isnonsingular.Indeed,itsdeterminant,aswellasallitsleadingprincipal minordeterminants,arestrictlypositive.Itfollowsthatthesystem(2.18)ofnormal equations has a unique solution. Does this solution correspond to a minimum of EŒ'(cid:4)in(2.17)?Calculustellsusthatforthistobethecase,theHessianmatrixH D Œ@2E2=@c @c (cid:4) has to be positive definite. But H D 2A, since E2 is a quadratic i j function.Therefore,H,withA,isindeedpositivedefinite,andthesolutionofthe normalequationsgives us the desired minimum. The least squaresapproximation problemthushasauniquesolution,givenby Xn 'O.t/D cO (cid:2) .t/; (2.22) j j jD1 wherecO DŒcO ;cO ;:::;cO (cid:4)T isthesolutionvectorofthenormalequation(2.18). 1 2 n Thiscompletelysettlestheleastsquaresapproximationproblemintheory.How aboutinpractice? Assumingageneralsetof(linearlyindependent)basisfunctions,wecanseethe followingpossibledifficulties. 1. The system (2.18) may be ill-conditioned. A simple example is provided by suppd(cid:5) D Œ0;1(cid:4), d(cid:5).t/ D dt on [0,1], and (cid:2) .t/ D tj(cid:2)1, j D 1;2;:::;n. j Then Z 1 1 .(cid:2) ;(cid:2) /D tiCj(cid:2)2dt D ; i;j D1;2;:::;nI i j i Cj (cid:3)1 0 2.1 LeastSquaresApproximation 63 thatis,thematrixAin(2.18)ispreciselytheHilbertmatrix(cf.Chap.1,(1.60)). The resulting severe ill-conditioning of the normal equations in this example is entirely due to an unfortunatechoice of basis functions– the powers. These becomealmostlinearlydependent,moresothelargertheRexponent(cf.Ex.38). Another source of degradationlies in the elementb D 1(cid:2) .t/f.t/dt of the j 0 j right-hand vector b in (2.18). When j is large, the power (cid:2) D tj(cid:2)1 behaves j verymuchlikeadiscontinuousfunctionon[0,1]:itispracticallyzeroformuch of the interval until it shoots up to the value 1 at the right endpoint. This has the unfortunate consequence that a good deal of information about f is lost whenoneformstheintegraldefiningb .Apolynomial(cid:2) thatoscillatesrapidly j j on [0,1] would seem to be preferable from this point of view, since it would “engage”thefunctionf morevigorouslyoveralloftheinterval[0,1]. 2. The seconddisadvantageis the factthat all coefficientscO in (2.22)dependon j n; that is, cO D cO.n/, j D 1;2;:::;n. Increasing n, for example, will give an j j enlargedsystemofnormalequationswithacompletelynewsolutionvector.We refertothisasthenonpermanenceofthecoefficientscO . j Bothdefects1and2canbeeliminated(oratleastattenuatedinthecaseof1) inonestroke:selectforthebasisfunctions(cid:2) anorthogonalsystem, j .(cid:2) ;(cid:2) /D0 if i ¤jI .(cid:2) ;(cid:2) /Dk(cid:2) k2 >0: (2.23) i j j j j Then the system of normalequationsbecomesdiagonaland is solved immedi- atelyby .(cid:2) ;f/ j cO D ; j D1;2;:::;n: (2.24) j .(cid:2) ;(cid:2) / j j Clearly, each of these coefficients cO is independent of n, and once com- j puted, remains the same for any larger n. We now have permanence of the coefficients. Also, we do not have to go through the trouble of solv- ing a linear system of equations, but instead can use the formula (2.24) directly. This does not mean that there are no numerical problems associ- ated with (2.24). Indeed, it is typical that the denominators k(cid:2) k2 in (2.24) j decrease rapidly with increasing j, whereas the integrand in the numera- tor (or the individual terms in the case of a discrete inner product) are of the same magnitude as f. Yet the coefficients cO also are expected to de- j crease rapidly. Therefore, cancellation errors must occur when one computes the inner productin the numerator.The cancellationproblem can be alleviated somewhatbycomputingcO inthealternativeform j ! Xj(cid:2)1 1 cO D f (cid:3) cO (cid:2) ;(cid:2) ; j D1;2;:::;n; (2.25) j k k j .(cid:2) ;(cid:2) / j j kD1 where the empty sum (when j D 1) is taken to be zero, as usual. Clearly, by orthogonality of the (cid:2) , (2.25) is equivalent to (2.24) mathematically, but not j necessarilynumerically. 64 2 ApproximationandInterpolation AnalgorithmforcomputingcO from(2.25),andatthesametime'O.t/,isas j follows: s D0; 0 for j D1;2;:::;n do 2 1 4cOj D k(cid:2) k2 .f (cid:3)sj(cid:2)1;(cid:2)j/ j sj Dsj(cid:2)1CcOj(cid:2)j.t/: ThisproducesthecoefficientscO ;cO ;:::;cO aswellas'O.t/Ds . 1 2 n n Any system f(cid:2)O g that is linearly independent on the support of d(cid:5) can be j orthogonalized (with respect to the measure d(cid:5)) by a device known as the Gram1–Schmidt2procedure.Onetakes (cid:2) D(cid:2)O 1 1 and,forj D2;3;::: ,recursivelyforms Xj(cid:2)1 .(cid:2)O ;(cid:2) / j k (cid:2) D(cid:2)O (cid:3) c (cid:2) ; c D : j j k k k .(cid:2) ;(cid:2) / kD1 k k Theneach(cid:2) sodeterminedisorthogonaltoallprecedingones. j 2.1.3 Least SquaresError;Convergence We have seen in Sect. 2.1.2 that if the class ˆ D ˆ consists of n functions (cid:2) , n j j D 1;2;:::;n,thatarelinearlyindependentonthesupportofsomemeasured(cid:5), thentheleastsquaresproblemforthismeasure, minkf (cid:3)'k Dkf (cid:3)'Ok ; (2.26) 2;d(cid:5) 2;d(cid:5) '2ˆn 1Jo´rgen Pedersen Gram (1850–1916) was a farmer’s son who studied at the University of Copenhagen. After graduation, he entered an insurance company as computer assistant and, moving up the ranks, eventually became itsdirector. He was interested in series expansions of specialfunctionsandalsocontributedtoChebyshevandleastsquaresapproximation.The“Gram determinant”wasintroducedbyhiminconnectionwithhisstudyoflinearindependence. 2Erhard Schmidt (1876–1959), astudent of Hilbert,became aprominent member of theBerlin SchoolofMathematics,wherehefoundedtheInstituteofAppliedMathematics.Heisconsidered one of the originators of Functional Analysis, having contributed substantially to the theory of Hilbertspaces.Hisworkonlinearandnonlinearintegralequationsisoflastinginterest,asishis contributiontolinearalgebraicsystemsofinfinitedimension.Heisalsoknownforhisproofofthe Jordancurvetheorem.Hisprocedureoforthogonalizationwaspublishedin1907andtodayalso carriesthenameofGram.Itwasknown,however,alreadytoLaplace.