Snehashish Chakraverty · Deepti Moyi Sahoo · Nisha Rani Mahato Concepts of Soft Computing Fuzzy and ANN with Programming Concepts of Soft Computing Snehashish Chakraverty (cid:129) Deepti Moyi Sahoo Nisha Rani Mahato (cid:129) Concepts of Soft Computing Fuzzy and ANN with Programming 123 SnehashishChakraverty DeeptiMoyi Sahoo Department ofMathematics Department ofMathematics, National Institute ofTechnology Rourkela Schoolof Science Rourkela, Odisha, India O.P. JindalUniversity Punjipathra, India NishaRani Mahato Department ofMathematics National Institute ofTechnology Rourkela Rourkela, Odisha, India ISBN978-981-13-7429-6 ISBN978-981-13-7430-2 (eBook) https://doi.org/10.1007/978-981-13-7430-2 LibraryofCongressControlNumber:2019935543 ©SpringerNatureSingaporePteLtd.2019 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar methodologynowknownorhereafterdeveloped. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained hereinorforanyerrorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregard tojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore Preface In the early 1990s, soft computing (SC) gained popularity due to its efficiency in handling problems with imprecision, approximation, decision making, and uncer- tainty. Unlike earlier hard computing techniques which dealt with precision, soft computing techniques consider imprecision to obtain rigorous, computationally efficient,androbustresults.Thefascinatingabilityofsoftcomputingtechniquesto deal with human intelligence, reasoning, analysis, learning, imprecision, and mul- titasking has made it favorable in various scientific and technological disciplines, viz.computerscience,mathematics,controltheory,structuralengineering,medical, management, psychology, etc. Generally, the principal constituents of soft com- puting include techniques focusing on fuzzy system, evolutionary computation, machine learning, and probabilistic reasoning. In this regard, this book starts with anintroductorychapterforabriefknowledgeaboutsoftcomputinganditsvarious constituents. Among the various constituents discussed above, fuzzy systems and machine learninghaveacquiredmuchattentioninsolvingreal-worldproblemsdealingwith imprecisionorapproximations.Thereexistseveralbooksthatdealwithoneormore constituents of soft computing. But, two major concepts, viz. fuzzy set theory and neural network, form the basic platforms offuzzy systems and machine learning, respectively. Inthisregard,thecomputational programmingparadigm ofthefuzzy settheoryconceptsandartificialneuralnetwork(ANN)modelsisessentialfortheir functioning in various science and engineering disciplines. But as per the best of ourknowledge,bookscoveringthebasicconceptsoffuzzysettheoryandANNina detailed as well as systematic manner along with programming are scarce. Accordingly, the authors realized the need for such a book that contains a concise and systematic description of various concepts in fuzzy set theory and ANN along with step-by-step C and/or MATLAB codes addressing simple solved example problemsforabetterunderstanding.Anotherkeyfeatureofthisbookisthatitalso provides partial attention to handle interval uncertainty. As regards, the book is mainly divided into two parts focusing on major soft computing techniques, viz. fuzzy set theory and ANN. The fuzzy set theory part comprises nine chapters, and ANN comprises the rest of the chapters. As a whole, v vi Preface the book consists of 14 chapters giving basic knowledge of fuzzy set theory and ANN. Before we incorporate the details of the book, the authors assume that the readers have prerequisite knowledge of calculus, set theory, and linear algebra. In fuzzy set theory, a fuzzy number may be approximately represented in terms of closed intervals using a-cut approach. As such, Chap. 1 comprises basic prelimi- naries related to intervals, interval arithmetic and interval matrices, accompanied withCandMATLABprograms.Then,Chap.2introducesthefundamentalsrelated to fuzzy sets with respect to the extension of crisp (classical) sets based on its membership function. Chapter 3 discusses various types of fuzzy numbers, and particularly, conversions of such fuzzy numbers to intervals have been done using a-cut approach. Chapter 4 emphasizes other properties offuzzy sets such as rela- tions and compositions on fuzzy relations. Chapter 5 elaborates different types of fuzzy functions along with C or MATLAB codes related to fuzzy functions. Chapter6providestheextensionofclassicaldifferentiationandintegrationtofuzzy differentiation and integration. Chapter 7 includes the methodology of the trans- formationoffuzzysetstocrispsetsintermsofdefuzzificationmethods.Chapters8 and 9 discuss special importance of applications using interval computation in the system of equations and eigenvalue problems, respectively, that may be further extended to fuzzy number with respect to the parameter a. Chapters 7–10 include MATLAB programs only for most of the solved problems. Chapters 10–14 introduce a computing paradigm known as ANN. Chapter 10 emphasizes mainly on the basic foundation of ANN. Chapter 11 contains McCulloch–Pitts model. Chapters 12 and 13 contain different learning algorithms. Further, Chap. 14 includes delta learning rule and backpropagation methods. Chapters 10–14 present MATLAB programming codes for most of the ANNtitles for variety of problems. In order to emphasize the importance of 14 chapters mentioned above, few unsolved problems have also been included at the end of some chapters for self-validation of the topics. Moreover, corresponding bibli- ographies are given at the end of each chapter for ease and better referencing. Thisbookismainlywritten forundergraduateandgraduatecoursesalloverthe world.Itcoverstopicsrelatedtothebasicsofsoftcomputing,fuzzysettheory,and neural networks. This book may also be used by researchers, industry, faculties, etc., for understanding the two challenging soft computing techniques, viz. fuzzy set theory and ANN, along with programming techniques. Further, the idea for solving the uncertain system of equations and eigenvalue problems occurring in science and engineering disciplines has also been incorporated. Rourkela, India Snehashish Chakraverty Punjipathra, India Deepti Moyi Sahoo Rourkela, India Nisha Rani Mahato Acknowledgements The first author greatly appreciates the support, encouragement, and patience pro- vided by hisfamily members, in particular his wife Shewli and daughters Shreyati andSusprihaa.Further,thebookmaynothavebeenpossiblewithouttheblessings of his beloved parents late Sh. Birendra K. Chakraborty and Smt. Parul Chakraborty. The second author would like to express her sincere gratitude to her family members, especially Sh. Daya Nidhi Sahoo, Smt. Tilottama Sahoo, Trupti Moyi Sahoo, and Deepak Kumar Sahoo, for their love and blessings. Last but not least, the author also wants to thank her beloved son Nishit, husband Mr. Soumyaranjan Sahoo, father-in-law Sh. Duryodhan Sahoo, and mother-in-law Smt. Rukmini Sahoo for showing their faith and giving liberty to do all the things. Finally, the third author’s warmest gratitude goes to her family members for their continuous support and motivation, especially Sh. Devendra Mahato, Smt. Premshila, Tanuja, Devasish, and Satish. Alsosecondandthirdauthorsgreatlyappreciatetheinspirationofthefirstauthor for his support and inspiration which helped a lot to learn new things. This work would not have been possible without his guidance, support, and encouragement. Authors sincerely acknowledge the reviewers for their fruitful suggestions and appreciations of the book proposal. Further, all the authors do appreciate the sup- portandhelpofthewholeteamofSpringer.Finally,authorsaregreatlyindebtedto the authors/researchers mentioned in the Bibliography section given at the end of each chapter. Snehashish Chakraverty Deepti Moyi Sahoo Nisha Rani Mahato vii Contents Part I Fuzzy Set Theory 1 Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Preliminaries of Intervals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.1 Addition and Subtraction of Intervals . . . . . . . . . . . . . 5 1.1.2 Multiplication of Intervals. . . . . . . . . . . . . . . . . . . . . . 7 1.1.3 Division and Inverse of Intervals. . . . . . . . . . . . . . . . . 8 1.2 Interval Matrices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.2.1 Arithmetic Operations on Interval Matrices . . . . . . . . . 18 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2 Fuzzy Sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.1 Preliminaries of Fuzzy Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.2 Generating Membership Function . . . . . . . . . . . . . . . . . . . . . . 27 2.3 Support, Height, Core, and Boundary of Fuzzy Set. . . . . . . . . . 31 2.4 Intersection, Union, and Complement. . . . . . . . . . . . . . . . . . . . 35 2.5 Other Operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 2.6 Extension Principle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3 Fuzzy Numbers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.1 Preliminaries of Fuzzy Numbers . . . . . . . . . . . . . . . . . . . . . . . 53 3.2 Types of Fuzzy Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.2.1 Triangular Fuzzy Number (TFN). . . . . . . . . . . . . . . . . 55 3.2.2 Trapezoidal Fuzzy Number (TrFN) . . . . . . . . . . . . . . . 58 3.2.3 Gaussian Fuzzy Number (GFN) . . . . . . . . . . . . . . . . . 61 3.3 Conversion of Fuzzy Number to Interval Form Using a-Cut . . . 65 3.3.1 Triangular Fuzzy Number a-Cut Decomposition . . . . . 65 3.3.2 Trapezoidal Fuzzy Number a-Cut Decomposition . . . . 66 3.3.3 Gaussian Fuzzy Number a-Cut Decomposition . . . . . . 66 ix x Contents 3.4 Fuzzy Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 3.4.1 Arithmetic Operations of Fuzzy Numbers . . . . . . . . . . 67 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4 Fuzzy Relations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.1 Preliminaries of Fuzzy Relations . . . . . . . . . . . . . . . . . . . . . . . 71 4.2 Intersection, Union, and Complement. . . . . . . . . . . . . . . . . . . . 72 4.3 First, Second, and Total Projection . . . . . . . . . . . . . . . . . . . . . 78 4.4 Composition of Fuzzy Relations . . . . . . . . . . . . . . . . . . . . . . . 83 4.5 Fuzzy Equivalence Relation. . . . . . . . . . . . . . . . . . . . . . . . . . . 89 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5 Fuzzy Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 5.1 Preliminaries of Crisp Function . . . . . . . . . . . . . . . . . . . . . . . . 95 5.2 Preliminaries of Fuzzy Function . . . . . . . . . . . . . . . . . . . . . . . 96 5.3 Types of Fuzzy Function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 5.3.1 Type I Fuzzy Function . . . . . . . . . . . . . . . . . . . . . . . . 96 5.3.2 Type II Fuzzy Function . . . . . . . . . . . . . . . . . . . . . . . 99 5.3.3 Type III Fuzzy Function. . . . . . . . . . . . . . . . . . . . . . . 101 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6 Fuzzy Differentiation and Integration . . . . . . . . . . . . . . . . . . . . . . . 105 6.1 Fuzzy Differentiation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.1.1 Fuzzifying Function at Crisp Point . . . . . . . . . . . . . . . 106 6.1.2 Crisp Function at Fuzzy Point. . . . . . . . . . . . . . . . . . . 108 6.2 Fuzzy Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.2.1 Fuzzifying Function Over Crisp Limits . . . . . . . . . . . . 110 6.2.2 Crisp Function Over Fuzzy Limits. . . . . . . . . . . . . . . . 112 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 7 Defuzzification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 7.1 Defuzzification Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 7.1.1 Aggregation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 7.1.2 Max-Membership Method. . . . . . . . . . . . . . . . . . . . . . 121 7.1.3 Centroid Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 7.1.4 Weighted-Average Method . . . . . . . . . . . . . . . . . . . . . 126 7.1.5 Mean–Max Method . . . . . . . . . . . . . . . . . . . . . . . . . . 126 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 8 Interval System of Linear Equations. . . . . . . . . . . . . . . . . . . . . . . . 129 8.1 System of Linear Equations. . . . . . . . . . . . . . . . . . . . . . . . . . . 129 8.2 Interval System of Equations. . . . . . . . . . . . . . . . . . . . . . . . . . 130 8.2.1 Methods for Solving ISLEs. . . . . . . . . . . . . . . . . . . . . 130 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Contents xi 9 Interval Eigenvalue Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 9.1 Eigenvalue Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 9.1.1 Standard Eigenvalue Problems . . . . . . . . . . . . . . . . . . 142 9.1.2 Generalized Eigenvalue Problems . . . . . . . . . . . . . . . . 143 9.2 Interval Eigenvalue Problems . . . . . . . . . . . . . . . . . . . . . . . . . 145 9.2.1 Vertex Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Part II Artificial Neural Network 10 Artificial Neural Network Terminologies. . . . . . . . . . . . . . . . . . . . . 153 10.1 Artificial Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 10.2 Network Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 10.2.1 Single Layer Neural Network . . . . . . . . . . . . . . . . . . . 154 10.2.2 Multilayer Neural Network . . . . . . . . . . . . . . . . . . . . . 154 10.2.3 Competitive Layer Neural Network. . . . . . . . . . . . . . . 155 10.3 Different Training Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 10.3.1 Supervised Training . . . . . . . . . . . . . . . . . . . . . . . . . . 156 10.3.2 Unsupervised Training . . . . . . . . . . . . . . . . . . . . . . . . 156 10.4 Activation Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 10.4.1 Some Common Activation Function . . . . . . . . . . . . . . 158 10.4.2 Weights. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 10.4.3 Bias. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 10.4.4 Threshold . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 10.5 Computing Net Input Using Matrix Multiplication Method . . . . 160 10.6 Matlab Programs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 10.6.1 Write a Matlab Program to Generate the Following Activation Functions That Are Used in Neural Networks: (a) Identity Function, (b) Binary Step Function with Threshold Value 1 and 0, (c) Binary Sigmoid Function, and (d) Bipolar Sigmoid Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 10.6.2 Write a Program in Matlab if the Net Input to an OutputNeuronIs0.38,CalculateItsOutputWhenthe Activation Function Is (a) Binary Sigmoid Function and (b) Bipolar Sigmoid Function . . . . . . . . . . . . . . . . 164 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 11 McCulloch–Pitts Neural Network Model. . . . . . . . . . . . . . . . . . . . . 167 11.1 McCulloch–Pitts Neural Network Model . . . . . . . . . . . . . . . . . 167 11.2 McCulloch–Pitts Neuron Architecture . . . . . . . . . . . . . . . . . . . 167 11.3 Matlab Programs for McCulloch–Pitts Neuron Model. . . . . . . . 168 11.3.1 Write a Matlab Program to Generate the Output of Logic AND Function by McCulloch–Pitts Neuron Model. The Threshold on Unit Is 2. . . . . . . . . . . . . . . 168