Signals, Processes, and Systems Ulrich Karrenberg Signals, Processes, and Systems An Interactive Multimedia Introduction to Signal Processing Third edition Ulrich Karrenberg Düsseldorf, Germany ISBN 978-3-642-38052-5 ISBN 978-3-642-38053-2 (eBook) DOI 10.1007/978-3-642-38053-2 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2013938956 © Springer-Verlag Berlin Heidelberg 2007, 2013 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, speci(cid:191) cally the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on micro(cid:191) lm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a speci(cid:191) c statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) V This book is dedicated to Claude E. Shannon, a pioneer of modern communications tech- nology. He died on 25 February 2001. Only few people know in depth the fundamental content of his 48-page book "Mathematical Theory of Communication" published in 1948. This, however, does not diminish his genius and the uniqueness of his findings. They have changed the world more than any other discovery because communication is now the keyword in our society and in life itself. His work will be accomplished once his the- ory has been integrated into modern physics thus leading to a greater understanding of the central principles of nature. This is a task which has not yet been fulfilled. • If you want to build a ship, don't round up people to procure wood, don't allocate the different tasks and jobs, but arouse their longing for the open sea which stretches to infinity! (Antoine de Saint-Exupery) • The success of language in conveying information ist vastly overrated, especially in learned circles. Not only is language highly elleptical, but also nothing can supply the defect of first-hand experience of types cognate to the things explicitly mentioned. ..... First-hand knowledge is the ultimate basis of intellectual life. ..... The second-handedness of learned world is the secret of its mediocrity. It is tame because it never has been scared by facts. (Alfred North Whitehead) • People ought to be ashamed who take the miracles of science and techno- logy for granted without understanding more about them than a cow does about the botanical principles behind the plants it happily munches. (Albert Einstein at the Berlin Funkausstellung – telecommunications exhibition – in 1930) • Real problems ignore the fact that education has been arbitrarily divided into different school subjects. (the author) • The purpose of computing is insight, not numbers! (R. W. Hamming) • Information and uncertainty find themselves to be partners. (Warren Weaver) VI Preface to the 3rd edition The 3rd English edition is to appear five years after the 2nd English edition. In the meantime, the 6 German edition has been published, which serves as the reference for this edition. The 1st international English-language version of this "learning system" has had a good reception worldwide and was awarded the German Educational Software Prize DIGITA 2003. At the end of 2008, the first licensed Chinese edition was published at Tsinghua University Press in Beijing. Multimedia interactive learning systems in both print and electronic form – that is, books which can be "brought to life" and which make possible learning through research and experimentation – are very much in keeping with the spirit of the times. The changes compared with the 2nd edition pertain mainly to the following points: • DASYLab S is now integrated in this learning system in Version 11. Incidentally, with 25,000 licenses conferred, DASYLab is one of the most sold configurable software for measurement data acquisition worldwide! • The essential link to the outside world for the purpose of the acquisition and output of real signals in the S Version is still the soundcard – i.e. the two high precision analog inlets and outlets. • Again, many of the meanwhile 280 DASYLab experiments have been extended, revised and optimised. Illustrative materials, an essential part of the learning system, have been revised and extended. • Additional contents have been added: the fundamentals of wavelets in Chapter 3 and the Discrete Wavelet Transformation DWT as the method for the recognition of patterns, compression and noise elimination in Chapter 11. Now Chapter 14 on neural networks and Chapter 15 with the mathematical modelling of signals - processes - systems complete this edition. As the many written reactions and requests for licences from universities and academies show, this learning system has been received positively, indeed enthusiastically. The idea of being able to conduct exciting experiments on a (home) computer, without the material equipment of expensive laboratories, appears to have caught on. I must again thank the DASYLab team from National Instruments in Aachen, the measX company and the team at the Springer publishing company. All have worked together superbly with me and were always open to my special wishes! A special thanks to Dr. Joachim Neher from the Fraunhofer Institut IPA in Stuttgart. The modules for neural networks developed by him have been modified for my purposes and supplemented, so that I could realize my educational concept. His best gift to me was the new 3D- FOURIER-module. A learning system of this kind grows, blossoms and flourishes from the interaction between users and the author. For this reason I am always happy to take up any concrete suggestions, which will be given careful consideration and included if possible. I look forward to hearing your ideas. Düsseldorf, (cid:45)(cid:68)(cid:81)(cid:88)(cid:68)(cid:85)(cid:92), 201(cid:22) Ulrich Karrenberg Introduction Page VII Introduction ............................................................................................(cid:17)(cid:17)(cid:17)(cid:17).................. 1 A science that people can understand ................................................................. 1 Target groups ....................................................................................................... 2 Graphic programming .......................................................................................... 3 The digital Edition ............................................................................................... 4 Chapter 1 .................................................................................................................... 11 The concept: methods – content – objectives .............................................. 11 Everything under one roof .................................................................................. 12 Hardware: Systems on a chip ............................................................................. 12 The software is the instrument ............................................................................ 13 A case for up-to-date education .......................................................................... 14 On the unity of theory and practice ...................................................................... 14 Multimedia and interactive learning .................................................................... 14 Science and Mathematics .................................................................................... 15 In search of other "tools" ..................................................................................... 17 Physics as the point of departure ......................................................................... 22 Clarification of Objectives .................................................................................. 24 Preliminary conclusions: the concept takes on clearer contours .......................... 28 Exercises on Chapter 1 ........................................................................................ 30 Chapter 2 ..................................................................................................................... 33 Signals in the time and frequency domain .................................................... 33 The FOURIER Principle ...................................................................................... 33 Periodic oscillations ............................................................................................. 34 Our ear as a FOURIER-analyzer .......................................................................... 35 FOURIER transformation: from the time domain to the frequency domain and back .................................................................................. 42 Important periodic oscillations/signals ................................................................ 47 Comparison of signals in the time and frequency domain ................................... 48 The confusing phase spectrum ............................................................................. 50 Interference: nothing to be seen although everything is there .............................. 50 Opposites which have a great deal in common: sine and δ-pulse .......................... 52 Non-periodic and one-off signals ......................................................................... 56 Pure randomness: stochastic noise ...................................................................... 57 Noise and information ........................................................................................ 58 Exercises for Chapter 2 ....................................................................................... 61 Chapter 3 .................................................................................................................... 65 The Uncertainty Principle ................................................................................... 65 A strange relationship between frequency and time and its practical consequences ............................................................................. 65 Sinusoidal signal and δ-pulse as a limiting case of the Uncertainty Principle ...... 69 Why ideal filters cannot exist .............................................................................. 70 Frequency measurements in the case of non-periodic signals ............................. 74 Near-periodic signals .......................................................................................... 80 Tones, sounds and music ..................................................................................... 81 Limiting uncertainty: Short Time FFT and Wavelets ......................................... 85 Exercises on Chapter 3 ......................................................................................... 96 Page VIII Introduction Chapter 4 ................................................................................................................... 99 Language as a carrier of information .......................................................... 99 How speech, tones and sounds are generated and perceived .............................. 107 Case study: a simple system for voice recognition ............................................. 115 Refinement and optimisation phase ................................................................... 120 Pattern recognition ........................................................................................... 123 Exercises on Chapter 4 ...................................................................................... 125 Chapter 5 ................................................................................................................. 127 The Symmetry Principle .................................................................................... 127 For reasons of symmetry: negative frequencies ................................................. 127 Proof of the physical existence of negative frequencies ..................................... 127 Periodic spectra ................................................................................................. 135 Inverse FOURIER transformation and GAUSSian plane .................................. 138 Exercises on Chapter 5 ...................................................................................... 150 Chapter 6 ................................................................................................................... 151 System analysis ..................................................................................................... 151 Sweep ................................................................................................................ 153 Modern test signals ........................................................................................... 158 The δ-pulse ....................................................................................................... 160 The transfer function as a locus curve .............................................................. 164 The step function .............................................................................................. 167 The GAUSSian pulse ........................................................................................ 173 The GAUSSian oscillation pulse ...................................................................... 175 The Burst signal................................................................................................. 175 The Si-function and the Si-oscillation pulse ...................................................... 176 Noise ................................................................................................................. 179 Transients in systems ........................................................................................ 183 Exercises on Chapter 6 ..................................................................................... 188 Chapter 7.................................................................................................................... 191 Linear and non-linear processes ..................................................................... 191 System analysis and system synthesis ............................................................... 191 Measuring a process to reveal whether it is linear or non-linear ......................... 191 Line and space .................................................................................................. 192 Inter-disciplinary significance ........................................................................... 192 Mirroring and projection ................................................................................... 193 A complex component: the transistor ............................................................... 195 There are only few linear processes .................................................................. 195 Multiplication of a signal by a constant ............................................................ 196 Addition of two or more signals ........................................................................ 197 Delay ................................................................................................................. 197 Differentiation ................................................................................................... 199 Integration ......................................................................................................... 206 Introduction Page IX Malicious functions or signal curves ................................................................ 213 Filters ................................................................................................................. 215 Non-linear processes ......................................................................................... 220 Multiplication of two signals ........................................................................... 221 Formation of the absolute value ...................................................................... 225 Quantization ..................................................................................................... 227 Windowing ....................................................................................................... 230 Exercises on Chapter 7 ..................................................................................... 231 Chapter 8 .................................................................................................................. 233 Classical modulation procedures .................................................................... 233 Transmission media ........................................................................................... 233 Modulation with sinusoidal carriers ................................................................. 233 Modulation and demodulation in the traditional sense ....................................... 234 Amplitude modulation and demodulation AM .................................................. 235 Wasting energy: double sideband AM with carrier ........................................... 242 Single sideband modulation without a carrier ................................................... 243 Frequency multiplex ......................................................................................... 251 Mixing ............................................................................................................... 254 Frequency modulation FM ................................................................................ 256 Demodulation of FM-signals ............................................................................. 266 The phase locked loop PLL ............................................................................... 266 Phase modulation .............................................................................................. 272 Immunity to interference of modulation processes ........................................... 274 Practical information theory .............................................................................. 277 Exercises on Chapter 8 ...................................................................................... 278 Chapter 9 ................................................................................................................... 281 Digitalisation ......................................................................................................... 281 Digital technology does not always mean the same thing .................................. 281 Digital processing of analog signals ................................................................. 281 The gateway to the digital world: the A/D converter ........................................ 283 Principle of a D/A converter ............................................................................. 285 Analog pulse modulation processes .................................................................. 288 DASYLab and digital signal processing ........................................................... 290 Digital signals in the time and frequency domain ............................................. 292 The period length of digital signals .................................................................... 293 The periodic spectrum of digital signals ............................................................ 302 The Sampling Principle ..................................................................................... 304 Retrieval of the analog signal ........................................................................... 309 Non-synchronicity ........................................................................................... 311 Signal distortion as a result of signal windowing ............................................. 314 Check list .......................................................................................................... 315 Exercises on Chapter 9 ..................................................................................... 317 Page X Introduction Chapter 10 ................................................................................................................ 319 Digital filters .......................................................................................................... 319 Hardware versus software ................................................................................. 319 How analog filters work .................................................................................... 319 FFT filters .......................................................................................................... 322 Digital filtering in the time domain .................................................................... 327 Convolution ...................................................................................................... 331 Case study: Design and application of digital filters ........................................... 333 Avoiding ripple content in the conducting state region ....................................... 336 The transfer function of digital filters ................................................................ 343 Exercises on chapter 10 .................................................................................... 345 Chapter 11 ................................................................................................................ 347 Digital transmission technology l: source encoding ................................. 347 Encoding and decoding of digital signals and data ........................................... 349 Compression .................................................................................................... 349 Low-loss and lossy compression ...................................................................... 351 RLE encoding .................................................................................................. 352 HUFFMAN encoding ...................................................................................... 352 LZW encoding ................................................................................................. 353 Source encoding of audio signals ...................................................................... 356 Delta encoding or delta modulation .................................................................. 356 Sigma-delta modulation or encoding (S-D-M) .................................................. 361 Noise shaping and decimation filter ................................................................. 363 Exploiting psycho-acoustic effects (MPEG) .................................................... 363 Quadrature Mirror Filter (QMF) ...................................................................... 367 Discrete Wavelet Transformation and Multi-Scale Analysis MSA.................... 373 Exploiting psycho-acoustic effects (MPEG) ...................................................... 380 Encoding and physics ........................................................................................ 386 Exercises on Chapter 11 .................................................................................... 387 Chapter 12 ................................................................................................................ 389 Digital transmission technology ll: channel encoding ............................. 389 Error protection encoding for the reduction of bit error probability ................... 389 Distance ........................................................................................................... 390 HAMMING codes and HAMMING distance ................................................... 392 Convolutional encoding .................................................................................. 394 VITERBI decoding .......................................................................................... 398 Hard and soft decision ..................................................................................... 400 Channel capacity .............................................................................................. 402 Exercises on Chapter 12 .................................................................................... 404 Chapter 13 ................................................................................................................. 405 Digital Transmission Techniques III: Modulation ................................... 405 Keying of discrete states .................................................................................... 408 Amplitude Shift Keying (2-ASK) ...................................................................... 408