Q�H�J�Q�L�Q�R�U�*�I�R�\�W�L�V�U�H�Y�L�Q�8� J�Q�L�W�W�H�6�H�W�L�Q�L�I�H�G�Q�,�Q�D�Q�L�V�F�L�S�R�7�G�H�W�D�O�H�5�G�Q�D�U�X�K�F�6�L�D�V�V�,�I�R�Q�R�L�W�D�P�U�R�I�V�Q�D�U�7�H�K�7� �+�U�H�J�Q�D�/�$�D�P�V�N�M�L�'�'�\�D�S�O�$� �Q�L�G�H�K�V�L�O�E�X�3� (�/�7�,�7�.�2�2�%�6�7�1�,�5�3�(� P�R�U�I�H�W�L�F�R�W�K�V�L�Z�X�R�\�I�L�)�'�3�V�U�H�K�V�L�O�E�X�S�Q�R�L�V�U�H�Y�V�U�H�K�V�L�O�E�X�S�H�K�W�W�O�X�V�Q�R�F�R�W�G�H�V�L�Y�G�D�H�U�D�X�R�<�(�7�2�1�7�1�$�7�5�2�3�0�,� �Z�R�O�H�E�Q�R�L�V�U�H�Y�W�Q�H�P�X�F�R�G�H�K�W�N�F�H�K�F�H�V�D�H�O�3�W�L� Q�R�L�V�U�H�9�W�Q�H�P�X�F�R�'� G�U�R�F�H�U�I�R�Q�R�L�V�U�H�9�V�D�Q�Z�R�Q�N�R�V�O�D�)�'�3�V�U�H�K�V�L�O�E�X�3� �H�W�D�G�Q�R�L�W�D�F�L�O�E�X�3� � H�V�D�E�D�W�D�G�K�F�U�D�H�V�H�U�*�&�0�8�Q�H�J�Q�L�Q�R�U�*�I�R�\�W�L�V�U�H�Y�L�Q�8�Q�L�Q�R�L�W�D�F�L�O�E�X�S�R�W�N�Q�L�/� �$�3�$�Q�R�L�V�U�H�Y�G�H�K�V�L�O�E�X�S�U�R�I�Q�R�L�W�D�W�L�&� Alpay, D., Dijksma, A., & Langer, H. (2007). The Transformation of Issai Schur and Related Topics in an Indefinite Setting. In EPRINTS-BOOK-TITLE University of Groningen, Johann Bernoulli Institute for Mathematics and Computer Science. W�K�J�L�U�\�S�R�&� H�K�W�I�R�W�Q�H�V�Q�R�F�H�K�W�W�X�R�K�W�L�Z�W�L�I�R�W�U�D�S�U�R�W�[�H�W�H�K�W�H�W�X�E�L�U�W�V�L�G�G�U�D�Z�U�R�I�R�W�U�R�G�D�R�O�Q�Z�R�G�R�W�G�H�W�W�L�P�U�H�S�W�R�Q�V�L�W�L�H�V�X�O�D�Q�R�V�U�H�S�\�O�W�F�L�U�W�V�U�R�I�Q�D�K�W�U�H�K�W�2� �V�Q�R�P�P�R�&�H�Y�L�W�D�H�U�&�H�N�L�O�H�V�Q�H�F�L�O�W�Q�H�W�Q�R�F�Q�H�S�R�Q�D�U�H�G�Q�X�V�L�N�U�R�Z�H�K�W�V�V�H�O�Q�X�V�U�H�G�O�R�K�W�K�J�L�U�\�S�R�F�U�R�G�Q�D�V�U�R�K�W�X�D� \�F�L�O�R�S�Q�Z�R�G�H�N�D�7� \�O�H�W�D�L�G�H�P�P�L�N�U�R�Z�H�K�W�R�W�V�V�H�F�F�D�H�Y�R�P�H�U�O�O�L�Z�H�Z�G�Q�D�V�O�L�D�W�H�G�J�Q�L�G�L�Y�R�U�S�V�X�W�F�D�W�Q�R�F�H�V�D�H�O�S�W�K�J�L�U�\�S�R�F�V�H�K�F�D�H�U�E�W�Q�H�P�X�F�R�G�V�L�K�W�W�D�K�W�H�Y�H�L�O�H�E�X�R�\�I�,� �P�L�D�O�F�U�X�R�\�H�W�D�J�L�W�V�H�Y�Q�L�G�Q�D� H�K�W�V�Q�R�V�D�H�U�O�D�F�L�Q�K�F�H�W�U�R�)�O�D�W�U�R�S�K�F�U�D�H�V�H�U�O�Q�J�X�U�Z�Z�Z�S�W�W�K�H�U�X�3�H�V�D�E�D�W�D�G�K�F�U�D�H�V�H�U�*�&�0�8�Q�H�J�Q�L�Q�R�U�*�I�R�\�W�L�V�U�H�Y�L�Q�8�H�K�W�P�R�U�I�G�H�G�D�R�O�Q�Z�R�'� �P�X�P�L�[�D�P�R�W�G�H�W�L�P�L�O�V�L�H�J�D�S�U�H�Y�R�F�V�L�K�W�Q�R�Q�Z�R�K�V�V�U�R�K�W�X�D�I�R�U�H�E�P�X�Q� �H�W�D�G�G�D�R�O�Q�Z�R�'� Operator Theory: Advances and Applications, Vol. 176, 1–98 ⃝c 2007 Birkh¨auser Verlag Basel/Switzerland The Transformation of Issai Schur and Related Topics in an Indefinite Setting D. Alpay, A. Dijksma and H. Langer Abstract. We review our recent work on the Schur transformation for scalar generalized Schur and Nevanlinna functions. The Schur transformation is de- fined for these classes of functions in several situations, and it is used to solve corresponding basic interpolation problems and problems of factorization of rational J-unitary matrix functions into elementary factors. A key role is played by the theory of reproducing kernel Pontryagin spaces and linear re- lations in these spaces. Mathematics Subject Classification (2000). Primary 47A48, 47A57, 47B32, 47B50. Keywords. Schur transform, Schur algorithm, generalized Schur function, gen- eralized Nevanlinna function, Pontryagin space, reproducing kernel, Pick ma- trix, coisometric realization, self-adjoint realization, J-unitary matrix func- tion, minimal factorization, elementary factor. Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Classical Schur analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Generalized Schur and Nevanlinna functions . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3 Reproducing kernel Pontryagin spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.4 The general scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.5 Outline of the paper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2 Kernels, classes of functions, and reproducing kernel Pontryagin spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.1 Reproducing kernel Pontryagin spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 D. Alpay thanks the Earl Katz family for endowing the chair which supports his research and NWO, the Netherlands Organization for Scientific Research (grant B 61-524). The research of A. Dijksma and H. Langer was in part supported by the Center for Advanced Studies in Mathematics (CASM) of the Department of Mathematics, Ben–Gurion University. 2 D. Alpay, A. Dijksma and H. Langer 2.2 Analytic kernels and Pick matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.3 Generalized Schur functions and the spaces P(s) . . . . . . . . . . . . . . . . . . . . 19 2.4 Generalized Nevanlinna functions and the spaces L(n) . . . . . . . . . . . . . . . 23 2.5 Additional remarks and references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3 Some classes of rational matrix functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.1 Realizations and McMillan degree of rational matrix functions . . . . . . . 26 3.2 J-unitary matrix functions and the spaces P(Θ): the line case . . . . . . . 27 3.3 J-unitary matrix functions and the spaces P(Θ): the circle case . . . . . 32 3.4 Factorizations of J-unitary matrix functions . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.5 Additional remarks and references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4 Pick matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.1 Generalized Schur functions: z1 ∈ D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.2 Generalized Schur functions: z1 ∈ T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 + 4.3 Generalized Nevanlinna functions: z1 ∈ C . . . . . . . . . . . . . . . . . . . . . . . . . . 47 4.4 Generalized Nevanlinna functions: z1 = ∞ . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.5 Additional remarks and references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 5 Generalized Schur functions: z1 ∈ D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.1 The Schur transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.2 The basic interpolation problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 z1 5.3 Factorization in the class U . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 c 5.4 Realization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 5.5 Additional remarks and references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 6 Generalized Schur functions: z1 ∈ T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 6.1 The Schur transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 6.2 The basic boundary interpolation problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 z1 6.3 Factorization in the class U . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 c 6.4 Additional remarks and references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 + 7 Generalized Nevanlinna functions: z1 ∈ C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 7.1 The Schur transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 7.2 The basic interpolation problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 z1 7.3 Factorization in the class U . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 ℓ 7.4 Realization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 7.5 Additional remarks and references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 8 Generalized Nevanlinna functions with asymptotic at ∞ . . . . . . . . . . . . . . . . . . 82 8.1 The Schur transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 8.2 The basic boundary interpolation problem at ∞ . . . . . . . . . . . . . . . . . . . . . 85 ∞ 8.3 Factorization in the class U . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 ℓ 8.4 Realization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 8.5 Additional remarks and references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 The Transformation of Issai Schur 3 1. Introduction The aim of this survey paper is to review some recent development in Schur analysis for scalar functions in an indefinite setting and, in particular, to give an overview of the papers [7], [8], [9], [10], [11], [15], [16], [17], [18], [125], and [126]. 1.1. Classical Schur analysis In this first subsection we discuss the positive definite case. The starting point is a function s(z) which is analytic and contractive (that is, |s(z)| ≤ 1) in the open unit disk D; we call such functions Schur functions. If |s(0)| < 1, by Schwarz’ lemma, also the function 1 s(z) − s(0) ŝ(z) = (1.1) ∗ z 1 − s(z)s(0) ∗ is a Schur function; here and throughout the sequel denotes the adjoint of a matrix or an operator and also the complex conjugate of a complex number. The transformation s(z) →↦ ŝ(z) was defined and studied by I. Schur in 1917–1918 in his papers [116] and [117] and is called the Schur transformation. It maps the set of Schur functions which are not identically equal to a unimodular constant into the set of Schur functions. If ŝ(z) is not a unimodular constant, the transformation (1.1) can be repeated with ŝ(z) instead of s(z) etc. In this way, I. Schur associated with a Schur function s(z) a finite or infinite sequence of numbers ρj in D, called Schur coefficients, via the formulas s0(z) = s(z), ρ0 = s0(0), and for j = 0, 1, . . . , 1 sj(z) − sj(0) sj+1(z) = ŝj(z) = , ρj+1 = sj+1(0). (1.2) ∗ z 1 − sj(z)sj(0) The recursion (1.2) is called the Schur algorithm. It stops after a finite number of steps if, for some j0, |ρj 0 | = 1. This happens if and only if s(z) is a finite Blaschke product: n ∏ z − aℓ s(z) = c ∗ , |c| = 1, and |aℓ| < 1, ℓ = 1, . . . , n, 1 − za ℓ=1 ℓ with n = j0, see [116] and [117]. ( ) a b If for a 2 × 2 matrix M = and v ∈ C we define the linear fractional c d transform TM(v) by av + b TM(v) = , cv + d the transform ŝ(z) in (1.1) can be written as ŝ(z) = TΦ(z)(s(z)), 4 D. Alpay, A. Dijksma and H. Langer where ( ) 1 1 −s(0) Φ(z) = √ . ∗ 2 −zs(0) z z 1 − |s(0)| Then it follows that ( )( ) 1 1 s(0) z 0 −1 Φ(z) = √ , (1.3) ∗ 2 s(0) 1 0 1 1 − |s(0)| and s(0) + zŝ(z) s(z) = TΦ(z)−1(ŝ(z)) = . (1.4) ∗ 1 + zŝ(z)s(0) ( ) 1 0 −1 The matrix polynomial Φ(z) in (1.3) is Jc-inner with Jc = , that 0 −1 is, { ≤ 0, |z| < 1, −1 −∗ Jc − Φ(z) JcΦ(z) = 0, |z| = 1. −1 Note that Θ(z) = Φ(z) Φ(1) is of the form ( ) ∗ uu Jc 1 Θ(z) = I2 + (z − 1) u∗Jcu, u = s(0)∗ . (1.5) −1 Of course Φ(z) in (1.4) can be replaced by Θ(z), which changes ŝ(z). Later, see Theorem 5.10, we will see that the matrix function Θ(z) given by (1.5) is elementary in the sense that it cannot be written as a product of two nonconstant Jc-inner matrix polynomials. A repeated application of the Schur transformation leads to a representation of s(z) as a linear fractional transformation a(z)s˜(z) + b(z) s(z) = , (1.6) c(z)s˜(z) + d(z) where s˜(z) is a Schur function and where the matrix function ( ) a(z) b(z) Θ(z) = c(z) d(z) is a Jc-inner matrix polynomial. In fact, this matrix function Θ(z) can be chosen a finite product of factors of the form (1.5) times a constant Jc-unitary factor. To see this it is enough to recall that the linear fractional transformations TM have the semi-group property: TM 1M2(v) = TM1(TM2(v)), if only the various expressions make sense. A key fact behind the scene and which hints at the connection with interpo- lation is the following: Given a representation (1.6) of a Schur function s(z) with a Jc-inner matrix polynomial Θ(z) and a Schur function s˜(z), then the matrix polynomial Θ(z) depends only on the first n = deg Θ derivatives of s(z) at the origin. (Here deg denotes the McMillan degree, see Subsection 3.1.) To see this we The Transformation of Issai Schur 5 it n use that det Θ(z) = e z with some t ∈ R and n = deg Θ, see Theorem 3.12. It follows that ( ) a(z)d(z) − b(z)c(z) s˜(z) TΘ(z)(s˜(z)) − TΘ(z)(0) = ( ) c(z)s˜(z) + d(z) d(z) −2 (det Θ(z))s˜(z)d(z) n = ( ) = z ξ(z) −1 c(z)s˜(z)d(z) + 1 with it −2 e s˜(z)d(z) ξ(z) = ( ). −1 c(z)s˜(z)d(z) + 1 −1 For any nonconstant Jc-inner matrix polynomial Θ(z) the function d(z) is ana- −1 lytic and contractive, and the function d(z) c(z) is analytic and strictly contrac- tive, on D, see [79], hence the function ξ(z) is also analytic in the open unit disk and therefore n TΘ(z)(s˜(z)) − TΘ(z)(0) = O(z ), z → 0. These relations imply that the Schur algorithm allows to solve recursively the Carath´eodory–Fej´er interpolation problem: Given complex numbers σ0, . . . , σn−1, find all (if any) Schur functions s(z) such that n−1 n s(z) = σ0 + zσ1 + · · · + z σn−1 + O(z ), z → 0. The Schur algorithm expresses the fact that one needs to know how to solve this problem only for n = 1. We call this problem the basic interpolation problem. The basic interpolation problem: Given σ0 ∈ C, find all Schur functions s(z) such that s(0) = σ0. Clearly this problem has no solution if |σ0| > 1, and, by the maximum mod- ulus principle, it has a unique solution if |σ0| = 1, namely the constant function s(z) ≡ σ0. If |σ0| < 1, then the solution is given by the linear fractional transfor- mation (compare with (1.4)) σ0 + zs˜(z) s(z) = , (1.7) ∗ 1 + zs˜(z)σ 0 where s˜(z) varies in the set of Schur functions. Note that the solution s(z) is the inverse Schur transform of the parameter s˜(z). If we differentiate both sides of (1.7) and put z = 0 then it follows that s˜(z) satisfies the interpolation condition σ1 ′ s˜(0) = , σ1 = s (0). 2 1 − |σ0| Thus if the Carath´eodory–Fej´er problem is solvable and has more than one solution (this is also called the nondegenerate case), these solutions can be obtained by repeatedly solving a basic interpolation problem (namely, first for s(z), then for s˜(z), and so on) and are described by a linear fractional transformation of the form (1.6) for some Jc-inner 2 × 2 matrix polynomial Θ(z). 6 D. Alpay, A. Dijksma and H. Langer The fact that the Carath´eodory–Fej´er interpolation problem can be solved it- eratively via the Schur algorithm implies that any Jc-inner 2×2 matrix polynomial can be written in a unique way as a product of Jc-inner 2 × 2 matrix polynomials of McMillan degree 1, namely factors of the form ( )( ) 1 1 ρ z 0 √ (1.8) ∗ 2 ρ 1 0 1 1 − |ρ| with some complex number ρ, |ρ| < 1, and a Jc-unitary constant. These factors of McMillan degree 1 are elementary, see Theorem 5.10, and can be chosen nor- malized: If one fixes, for instance, the value at z = 1 to be I2, factors Θ(z) of the form (1.5) come into play. Note that the factor (1.8) is not normalized in this sense when ρ ≠ 0. Furthermore, the Schur algorithm is also a method which yields this Jc-minimal factorization of a Jc-inner 2 × 2 matrix polynomial Θ(z) into elemen- tary factors. Namely, it suffices to take any number τ on the unit circle and to apply the Schur algorithm to the function s(z) = TΘ(z)(τ); the corresponding se- quence of elementary Jc-inner 2× 2 matrix polynomial gives the Jc-inner minimal factorization of Θ(z). Schur’s work was motivated by the works of Carath´eodory, see [53] and [54], and Toeplitz, see [122], on Carath´eodory functions which by definition are the analytic functions in the open unit disk which have a nonnegative real part there, see [116, English transl., p. 55]. A sequence of Schur coefficients can also be associated with a Carath´eodory function; sometimes these numbers are called Verblunsky coefficients, see [86, Chapter 8]. Carath´eodory functions φ(z) play an important role in the study of the trigonometric moment problem via the Herglotz representation formula ∫ 2π it ∫ 2π ∑∞ ∫ 2π e + z ℓ −iℓt φ(z) = ia + dµ(t) = ia + dµ(t) + 2 z e dµ(t), it e − z 0 0 0 ℓ=1 where a is a real number and dµ(t) is a positive measure on [0, 2π). A function φ(z), defined in the open unit disk, is a Carath´eodory function if and only if the kernel ∗ φ(z) + φ(w) Kφ(z, w) = (1.9) ∗ 1 − zw is nonnegative. Similarly, a function s(z), defined in the open unit disk, is a Schur function if and only if the kernel ∗ 1 − s(z)s(w) Ks(z, w) = ∗ 1 − zw is nonnegative in D. In this paper we do not consider Carath´eodory functions with associated kernel (1.9), but functions n(z) which are holomorphic or meromorphic in the + upper half-plane C and for which the Nevanlinna kernel ∗ n(z) − n(w) Ln(z, w) = ∗ z − w The Transformation of Issai Schur 7 + has certain properties. For example, if the kernel Ln(z, w) is nonnegative in C , then the function n(z) is called a Nevanlinna function. The Schur transformation (1.1) for Schur functions has an analog for Nevanlinna functions in the theory of the Hamburger moment problem and was studied by N.I. Akhiezer, see [4, Lemma 3.3.6] and Subsection 8.1. To summarize the previous discussion one can say that the Schur transfor- mation, the basic interpolation problem and Jc-inner factorizations of 2×2 matrix polynomials are three different facets of a common object of study, which can be called Schur analysis. For more on the original works we refer to [82] and [83]. Schur analysis is presently a very active field, we mention, for example, [75] for scalar Schur functions and [74] and [84] for matrix Schur functions, and the references cited there. The Schur transform (1.1) for Schur functions is centered at z1 = 0. The Schur transform centered at an arbitrary point z1 ∈ D is defined by 1 s(z) − s(z1) 1 s(z) − σ0 ŝ(z) = ≡ , ∗ ∗ bc(z) 1 − s(z)s(z1) bc(z) 1 − s(z)σ 0 where bc(z) denotes the Blaschke factor related to the circle and z1: z − z1 bc(z) = . ∗ 1 − zz 1 This definition is obtained from (1.1) by changing the independent variable to ζ(z) = bc(z), which leaves the class of Schur functions invariant. In this paper we consider the generalization of this transformation to an indefinite setting, that is, to a transformation centered at z1 of the class of generalized Schur functions with z1 ∈ D and z1 ∈ T, and to a transformation centered at z1 of the class of + generalized Nevanlinna functions with z1 ∈ C and z1 = ∞ (here also the case z1 ∈ R might be of interest, but it is not considered in this paper). We call this generalized transformation also the Schur transformation. 1.2. Generalized Schur and Nevanlinna functions In the present paper we consider essentially two classes of scalar functions. The first class consists of the meromorphic functions s(z) on the open unit disc D for which the kernel ∗ 1 − s(z)s(w) Ks(z, w) = , z, w ∈ hol (s), ∗ 1 − zw has a finite number κ of negative squares (here hol (s) is the domain of holomorphy of s(z)), for the definition of negative squares see Subsection 2.1. This is equivalent to the fact that the function s(z) has κ poles in D but the metric constraint of being not expansive on the unit circle T (in the sense of nontangential boundary values from D), which holds for Schur functions, remains. We call these functions s(z) generalized Schur functions with κ negative squares. The second class is the 8 D. Alpay, A. Dijksma and H. Langer set of generalized Nevanlinna functions with κ negative squares: These are the + meromorphic functions n(z) on C for which the kernel ∗ n(z) − n(w) Ln(z, w) = , z, w ∈ hol (n), ∗ z − w has a finite number κ of negative squares. We always suppose that they are ex- ∗ ∗ tended to the lower half-plane by symmetry: n(z ) = n(z) . Generalized Nevan- linna functions n(z) for which the kernel Ln(z, w) has κ negative squares have at + most κ poles in the open upper half-plane C ; they can also have ‘generalized poles of nonpositive type’ on the real axis, see [99] and [100]. Note that if the kernels Ks(z, w) and Ln(z, w) are nonnegative the functions s(z) and n(z) are automatically holomorphic, see, for instance, [6, Theorem 2.6.5] for the case of Schur functions. The case of Nevanlinna functions can be deduced from this case by using Mo¨bius transformations on the dependent and independent variables, as in the proof of Theorem 7.13 below. Generalized Schur and Nevanlinna functions have been introduced indepen- dently and with various motivations and characterizations by several mathemati- cians. Examples of functions of bounded type with poles in D and the metric constraint that the nontangential limits on T are bounded by 1 were already con- sidered by T. Takagi in his 1924 paper [121] and by N.I. Akhiezer in the maybe lesser known paper [3] of 1930. These functions are of the form p(z) s(z) = , n ∗ ∗ z p(1/z ) where p(z) is a polynomial of degree n, and hence are examples of generalized Schur functions. Independently, functions with finitely many poles in D and the metric constraint on the circle were introduced by Ch. Chamfy, J. Dufresnoy, and Ch. Pisot, see [55] and [78]. It is fascinating that also in the work of these authors there appear functions of the same form, but with polynomials p(z) with integer coefficients, see, for example, [55, p. 249]. In related works of M.-J. Bertin [41] and Ch. Pisot [105] the Schur algorithm is considered where the complex number field is replaced by a real quadratic field or a p-adic number field, respectively. In none of these works any relation was mentioned with the Schur kernel Ks(z, w). The approach using Schur and Nevanlinna kernels was initiated by M.G. Krein and H. Langer in connection with their study of operators in Pontryagin spaces, see [94], [95], [96], [97], [98], and [99]. Their definition in terms of kernels allows to study the classes of generalized Schur and Nevanlinna functions with tools from functional analysis and operator theory (in particular, the theory of reproducing kernel spaces and the theory of operators on spaces with an indefinite inner prod- uct), and it leads to connections with realization theory, interpolation theory and other related topics. 1.3. Reproducing kernel Pontryagin spaces The approach to the Schur transformation in the indefinite case in the present paper is based on the theory of reproducing kernel Pontryagin spaces for scalar The Transformation of Issai Schur 9 and matrix functions, associated, for example, in the Schur case with a Schur function s(z) and a 2 × 2 matrix function Θ(z) through the reproducing kernels ( ) ∗ ∗ 1 − s(z)s(w) Jc − Θ(z)JcΘ(w) 1 0 Ks(z, w) = , KΘ(z, w) = , Jc = ; 1 − zw∗ 1 − zw∗ 0 −1 these spaces are denoted by P(s) and P(Θ), respectively. In the positive case, they have been first introduced by L. de Branges and J. Rovnyak in [50] and [49]. They play an important role in operator models and interpolation theory, see, for instance, [20], [79], and [81]. In the indefinite case equivalent spaces were introduced in the papers by M.G. Krein and H. Langer mentioned earlier. We also consider the case of generalized Nevanlinna functions n(z) and cor- responding 2 × 2 matrix functions Θ(z), where the reproducing kernels are of the form ( ) ∗ ∗ n(z) − n(w) Jℓ − Θ(z)JℓΘ(w) 0 1 Ln(z, w) = , KΘ(z, w) = , Jℓ = . z − w∗ z − w∗ −1 0 We denote by L(n) the reproducing kernel space associated with the first kernel and by P(Θ) the reproducing kernel space associated to the second kernel. This space P(Θ) differs from the one above, but it should be clear from the context to which reproducing kernel KΘ(z, w) it belongs. The questions we consider in this paper are of analytic and of geometric nature. The starting point is the Schur transformation for generalized Schur functions centered at an inner point z1 ∈ D or at a boundary point z1 ∈ T, and for generalized Nevanlinna functions centered at an inner point + z1 ∈ C or at the boundary point ∞. Generalized Schur and Nevanlinna functions are also characteristic functions of certain colligations with a metric constraint, and we study the effect of the Schur transformation on these underlying colligations. We explain this in more detail for generalized Schur functions and an inner point z1 ∈ D. By analytic problems we mean: • The basic interpolation problem for generalized Schur functions, that is, the problem to determine the set of all generalized Schur functions analytic at a point z1 ∈ D and satisfying s(z1) = σ0. The solution depends on whether |σ0| < 1, > 1, or = 1, and thus the basic interpolation problem splits into three different cases. It turns out that in the third case more data are needed to get a complete description of the solutions in terms of a linear fractional transformation. • The problem of decomposing a rational 2 × 2 matrix function Θ(z) with a ∗ single pole in 1/z 1 and Jc-unitary on T as a product of elementary factors with the same property. Here the Schur algorithm, which consists of a repeated application of the Schur transformation, gives an explicit procedure to obtain such a factorization. The factors are not only of the form (1.8) as in Subsection 1.1 but may have a McMillan degree > 1. These new types of factors have first been exhibited by Ch. Chamfy in [55] and by Ph. Delsarte, Y. Genin, and Y. Kamp in [63].