Artificial Intelligence Frontiers in Statistics ArtifIinctiealll igence FrontiineS rtsa tistics Ala nsdt atIilslt ics Edibtye d D.HJa.n d ProfeosfSs toart istics ThOep eUnn iveMrisliKtteoyyn,n UeKs , Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Group, an informa business First published by Chapman & Hall 1993 CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 1993 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works ISBN-13: 9781003059875 (hbk) This book contains information obtained from authentic and highly regarded sources. Reason- able efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www. copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organiza- tion that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com A catarleocgfouo retrd sh b iooki asv aifrloamtb hlBeer iLtiibsrha ry LibrofaC royn gCraetsasl oging-diant-aP ublication Artiifnitceilafllri ogneitnnstic aeetr isAs Iat nisdtca st:i stics II/Ie dibtyeD d.H Ja.n -d1.se td . p. cm. Incliunddeesx . ISB0N- 412-40710-8 1.Artiifnitceilall2i .Sg teantcie3s..Et xipcsesyr.stt ems (CompsuctieerIn H.ca en)Dd .,J . Q335.5.1A979836 519.5'0285'63-dc20 92-34777 CIP Contents List of contributors ix Introduction D.]. Hand xv PART ONE Statistical expert systems 1 1 DEXPERT: an expert system for the design of experiments 3 T.J. Lorenzen, L.T. Truss, W.S. Spangler, W.T. Corpus and A.B. Parker 2 Inside two commercially available statistical expert systems 17 J.F.M. Raes 3 AMIA: Aide aIa Modelisation par l'Inteiligence Artificieile (expert system for simulation modeiling and sectoral forecasting) 31 M. Ollivier, R. Arrus, M.-A. Durillon, S. Robert and B. Debord 4 An architecture for knowledge-based statistical support systems 39 A Prat, E. Edmonds, J.M. Catot, ]. Lores, ]. Calmes and P. Fletcher 5 Enhancing explanation capabilities of statistical expert systems through hypertext 46 P. Hietala 6 Measurement scales as metadata 54 D.]. Hand PART TWO Belief networks 65 7 On the design of belief networks for knowledge-based systems· 67 B. Abramson 8 Lack-of-information based control in graphical belief systems 82 vi Contents 9 Adaptive importance sampling for Bayesian networks applied to filtering problems 90 AR. Runnalls 10 Intelligent arc addition, belief propagation and utilization of parallel processors by probabilistic inference engines 106 A Ranjbar and M. McLeish 11 A new method for representing and solving Bayesian decision problems 109 P.P. Shenoy PART THREE Learning 139 "1.2 Inferring causal structure in mixed populations 141 C. Glymour, P. Spirtes and R. Scheines 13 A knowledge acquisition inductive system guided by empirical interpretation of derived results 156 K. Tsujino and S. Nishida 14 Incorporating statistical techniques into empirical symbolic learning systems 168 F. Esposito, D. Malerba and G. Semeraro 15 Learning classification trees 182 W. Buntine 16 An analysis of two probabilistic model induction techniques 202 S.L. Crawford and M. Fung PART FOUR Neural networks 215 17 A robust back propagation algorithm for function approximation 217 D.S. Chen and R.C. Jain 18 Maximum likelihood training of neural networks 241 H. Gish 19 A connectionist knowledge acquisition tool: CONKAT 256 A Ultsch, R. Mantyk and G. Halmans 20 Connectionist, rule-based, and Bayesian decision aids: an empirical comparison 264 S. Schwartz, J. Wiles, I. Gough and S. Phillips PART FIVE Text manipulation 279 21 Statistical approaches to aligning sentences and identifying word correspondences in parallel texts: a report on work in progress 281 W.A. Gale and K.W. Church Contents vii 22 Probabilistic text understanding 295 R.P. Goldman and E. Chamiak 23 The application of machine learning techniques in subject classification 312 I. Kavanagh, C. Ward and J. Dunnion PART SIX Other areas 325 24 A statistical semantics for causation 327 J. Pearl and T.S. Verma 25 Admissible stochastic complexity models for classification problems 335 P. Smyth 26 Combining the probability judgements of experts: statistical and artificial intelligence approaches 348 L.A. Cox 27 Randomness and independence in non-monotonic reasoning 362 E. Neufeld 28 Consistent regions in probabilistic logic when using different norms 370 D. Bouchaffra 29 A decision theoretic approach to controlling the cost of planning 387 L. Hartman Index 401 Contributors B. Abramson W. Buntine Department of Computer Science RIACS & NASA Ames Research Center University of Southern California Mail Stop 269-2 Los Angeles Moffet Field CA 90089-0782 CA 94035 USA USA R. Almond ].M. Catot Department of Statistics GN-22 Departamento de Estadistica University of Washington E.T.S.E.I.B. Seattle Diagonal 647 WA 98195 08028-Barcelona USA Spain R. Arrus E. Charniak CRISS Department of Computer Science (Centre de Recherche en Informatique Brown University Appliquee aux Sciences Sociales) Box 1910 Universite des Sciences Providence 38040 Grenoble Cedex RI 02912 France USA D. Bouchaffra D.S. Chen CRISS Artificial Intelligence Laboratory (Centre de Recherche en Informatique 162 ATL Appliquee aux Sciences Sociales) The University of Michigan B.P. 47 X Ann Arbor 38040 Grenoble Cedex MI 48109-2110 France USA