FACULTY OF ENGINEERING Department of Fundamental Electricity and Instrumentation Pleinlaan 2 - 1050 Brussels Some practical applications of the best linear approximation in nonlinear block-oriented modelling Thesis submitted in fulfilment of the requirements for the degree of Doctor in Engineering (Doctor in de Ingenieurswetenschappen) by Lieve Lauwers Chair: Prof. Dr. ir. Gert Desmet (Vrije Universiteit Brussel) Vice chair: Prof. Dr. ir. Herman Terryn (Vrije Universiteit Brussel) Secretary: Dr. Kurt Barbé (Vrije Universiteit Brussel) Adviser: Prof. Dr. ir. Johan Schoukens (Vrije Universiteit Brussel) Jury: Em. Prof. Dr. ir. Keith Godfrey (University of Warwick) Prof. Dr. ir. Joos Vandewalle (KULeuven) Prof. Dr. Steve Vanlanduit (Vrije Universiteit Brussel) © 2011 (cid:47)(cid:76)(cid:72)(cid:89)(cid:72)(cid:3)(cid:47)(cid:68)(cid:88)(cid:90)(cid:72)(cid:85)(cid:86) 2011 Uitgeverij University Press Leegstraat 15 B-9060 Zelzate Tel +32 9 342 72 25 E-mail: [email protected] www.universitypress.be ISBN 978-94-9069-56(cid:28)(cid:16)(cid:28) Legal Deposit D/2011/11.161/01(cid:28) All rights reserved. No parts of this book may be reproduced or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the author. To the ones I love... Acknowledgements I would like to thank everyone that has contributed to this work in one way or another. I thank Johan Schoukens for giving me the opportunity to pursue my PhD at the ELEC department. I wish to thank all my colleagues for the pleasant work environment, especially my office mates: Koen, Mohamed and Charles. I know I was not always easy to share the office with... Aspecialwordofthanksgoestomythreemusketeers: Joke, WendyandKurt. Thanksalotforbelievinginmeandforgivingmethestrengthtogoon. Ienjoyed ourrelaxingtalksandthefuntimewespentatthedepartmentandatconferences. Mostofall,Iamgratefultomyfamilyfortheirendlessloveandsupport. Thank you for encouraging me all the way! I am who I am, because of you! I fall short of words when it comes to thank my soulmate, Joke. Thank you for loving me for who I am, for your patience and understanding. Without you, I could not have done it! Lieve Lauwers May 2011 Contents Operators and Notational Conventions xi Symbols xiii Abbreviations xv 1 Introduction 1 1.1 Mathematical Models . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Linear versus Nonlinear Systems . . . . . . . . . . . . . . . . . . 2 1.3 System Identification Process . . . . . . . . . . . . . . . . . . . . 4 1.3.1 Collect data . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3.2 Select a model . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3.3 Fit the model to the data . . . . . . . . . . . . . . . . . . 6 1.3.4 Validate the model . . . . . . . . . . . . . . . . . . . . . 7 1.3.5 Iterative process . . . . . . . . . . . . . . . . . . . . . . . 7 1.3.6 Some important questions . . . . . . . . . . . . . . . . . 7 1.4 Outline of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . 9 1.5 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.6 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 I Modelling using the Best Linear Approximation 15 2 Some Block-oriented Systems 17 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.2 Basic Building Blocks . . . . . . . . . . . . . . . . . . . . . . . . 18 2.2.1 Linear time-invariant systems . . . . . . . . . . . . . . . . 18 2.2.2 Static and dynamic nonlinearities . . . . . . . . . . . . . . 19 2.3 Specific Block Structures . . . . . . . . . . . . . . . . . . . . . . 20 v Contents 2.3.1 Wiener systems . . . . . . . . . . . . . . . . . . . . . . . 20 2.3.2 Hammerstein systems . . . . . . . . . . . . . . . . . . . . 21 2.3.3 Wiener-Hammerstein systems . . . . . . . . . . . . . . . . 22 2.3.4 Hammerstein-Wiener systems . . . . . . . . . . . . . . . . 22 2.3.5 Nonlinear feedback systems . . . . . . . . . . . . . . . . . 23 2.3.6 Nonlinear parallel systems . . . . . . . . . . . . . . . . . . 24 2.4 Parallel block-oriented models in a broader context . . . . . . . . 25 2.5 Identification Issues . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.5.1 How to select a model structure? . . . . . . . . . . . . . . 28 2.5.2 How to initialise the model? . . . . . . . . . . . . . . . . 29 3 The Best Linear Approximation 31 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.2 Approximation Criterion . . . . . . . . . . . . . . . . . . . . . . . 32 3.3 Defining the Best Linear Approximation . . . . . . . . . . . . . . 33 3.4 Class of Excitation Signals . . . . . . . . . . . . . . . . . . . . . 36 3.4.1 Gaussian noise . . . . . . . . . . . . . . . . . . . . . . . . 36 3.4.2 Random phase multisine . . . . . . . . . . . . . . . . . . 37 3.5 Properties of the Best Linear Approximation . . . . . . . . . . . . 39 3.5.1 Existence . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.5.2 Representation . . . . . . . . . . . . . . . . . . . . . . . . 40 3.6 Estimation of the Best Linear Approximation. . . . . . . . . . . . 42 3.6.1 Periodic data . . . . . . . . . . . . . . . . . . . . . . . . 42 3.6.2 Nonperiodic data . . . . . . . . . . . . . . . . . . . . . . 45 3.6.3 Summary of notation . . . . . . . . . . . . . . . . . . . . 46 4 Model Structure Selection Method 47 4.1 Existing Approaches . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.2 Model Structure Selection Method . . . . . . . . . . . . . . . . . 48 4.2.1 Two-step nonparametric approach . . . . . . . . . . . . . 49 4.2.2 Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 4.2.3 Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 50 4.3 Behaviour of the Best Linear Approximation . . . . . . . . . . . . 50 4.3.1 Simulation setup. . . . . . . . . . . . . . . . . . . . . . . 50 4.3.2 Static nonlinearity . . . . . . . . . . . . . . . . . . . . . . 51 4.3.3 Dynamic nonlinearity: NFIR system . . . . . . . . . . . . 51 4.3.4 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.3.5 Wiener-Hammerstein system . . . . . . . . . . . . . . . . 59 4.3.6 WH-NFIR system . . . . . . . . . . . . . . . . . . . . . . 61 4.3.7 Nonlinear feedback system . . . . . . . . . . . . . . . . . 65 vi Contents 4.3.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 65 4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.5 Practical Applications . . . . . . . . . . . . . . . . . . . . . . . . 68 4.5.1 Measurement setup . . . . . . . . . . . . . . . . . . . . . 69 4.5.2 Case study 1: Silverbox . . . . . . . . . . . . . . . . . . . 69 4.5.3 Case study 2: Crystal Detector . . . . . . . . . . . . . . . 72 4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Appendix 4.A Calculation of the Uncertainty Bounds . . . . . . . . . . 76 5 On the Initialisation of Wiener-Hammerstein Models 79 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 5.2 Initialisation Algorithm . . . . . . . . . . . . . . . . . . . . . . . 80 5.2.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.2.2 Decomposing the WH structure . . . . . . . . . . . . . . 81 5.2.3 Extracting the poles and zeros from GBLA . . . . . . . . 82 5.2.4 Constructing the basis functions . . . . . . . . . . . . . . 83 5.2.5 Solving a problem linear-in-the-parameters . . . . . . . . . 84 5.2.6 Composing the initial estimates . . . . . . . . . . . . . . . 86 5.2.7 Numerical issues . . . . . . . . . . . . . . . . . . . . . . . 87 5.2.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 88 5.2.9 Derived approach . . . . . . . . . . . . . . . . . . . . . . 89 5.3 Nonlinear Optimisation . . . . . . . . . . . . . . . . . . . . . . . 90 5.4 Simulation Study . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.5 Practical Application . . . . . . . . . . . . . . . . . . . . . . . . 97 5.5.1 Description of the DUT . . . . . . . . . . . . . . . . . . . 97 5.5.2 Description of the data . . . . . . . . . . . . . . . . . . . 97 5.5.3 Best linear approximation . . . . . . . . . . . . . . . . . . 98 5.5.4 Initial estimates . . . . . . . . . . . . . . . . . . . . . . . 99 5.5.5 Nonlinear optimisation . . . . . . . . . . . . . . . . . . . 102 5.5.6 Benchmark results . . . . . . . . . . . . . . . . . . . . . . 103 5.5.7 Comparison with other approaches . . . . . . . . . . . . . 106 5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Appendix 5.A Study of the Delay Terms . . . . . . . . . . . . . . . . . 110 Appendix 5.B Model Equivalence Proof . . . . . . . . . . . . . . . . . 112 Appendix 5.C Hinge Functions . . . . . . . . . . . . . . . . . . . . . . 113 II Modelling using Prior Knowledge 115 6 Parameter Estimation of the Rice Distribution 117 vii Contents 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 6.2 Bayesian Framework . . . . . . . . . . . . . . . . . . . . . . . . . 118 6.2.1 Prior knowledge . . . . . . . . . . . . . . . . . . . . . . . 118 6.2.2 Methodology. . . . . . . . . . . . . . . . . . . . . . . . . 119 6.3 Rice Estimation Problem . . . . . . . . . . . . . . . . . . . . . . 120 6.3.1 Rice distribution . . . . . . . . . . . . . . . . . . . . . . . 120 6.3.2 Problem statement . . . . . . . . . . . . . . . . . . . . . 121 6.3.3 Existing approaches . . . . . . . . . . . . . . . . . . . . . 122 6.4 Bayesian Approach . . . . . . . . . . . . . . . . . . . . . . . . . 128 6.4.1 Prior density . . . . . . . . . . . . . . . . . . . . . . . . . 128 6.4.2 Maximum A Posteriori estimator . . . . . . . . . . . . . . 130 6.5 Comparison of Estimators . . . . . . . . . . . . . . . . . . . . . . 132 6.5.1 Dependency of starting values . . . . . . . . . . . . . . . 132 6.5.2 Performance as a function of SNR . . . . . . . . . . . . . 133 6.5.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 136 6.6 Study of ε . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 6.6.1 Interpretation . . . . . . . . . . . . . . . . . . . . . . . . 138 6.6.2 Parameter uncertainty . . . . . . . . . . . . . . . . . . . . 140 6.6.3 Sensitivity analysis. . . . . . . . . . . . . . . . . . . . . . 142 6.6.4 User guideline . . . . . . . . . . . . . . . . . . . . . . . . 143 6.7 Practical Application to fMRI Data . . . . . . . . . . . . . . . . . 143 6.7.1 Background to fMRI . . . . . . . . . . . . . . . . . . . . 143 6.7.2 Processing fMRI data . . . . . . . . . . . . . . . . . . . . 145 6.7.3 Description of the data . . . . . . . . . . . . . . . . . . . 147 6.7.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 6.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Appendix 6.A Property on the Prior Density . . . . . . . . . . . . . . . 152 Appendix 6.B Existence of a Normalising Constant . . . . . . . . . . . 153 Appendix 6.C Uniqueness of the MAP Solution . . . . . . . . . . . . . 157 Appendix 6.D Explicit Expressions for the Jacobian . . . . . . . . . . . 165 7 Conclusions 167 7.1 Overall Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 168 7.2 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 7.2.1 Extending the model structure selection method . . . . . . 169 7.2.2 Automated distinction of model structures . . . . . . . . . 169 7.2.3 Reducing the number of model parameters. . . . . . . . . 169 7.2.4 Designing optimal statistical tests . . . . . . . . . . . . . 170 7.2.5 Modelling the dynamics in fMRI signals . . . . . . . . . . 170 viii