ADAPTIVE INTELLIGENT SYSTEMS Proceedings of the BANKAI Workshop Brussels, Belgium, 12-14 October 1992 Edited by Society for Worldwide Interbank Financial Telecommunication S.C. 1993 ELSEVIER SCIENCE PUBLISHERS AMSTERDAM · LONDON · NEW YORK · TOKYO ELSEVIER SCIENCE PUBLISHERS B.V. Sara Burgerhartstraat 25 P.O. Box 211, 1000 AE Amsterdam, The Netherlands ISBN: 0 444 89838 7 ©1992 Elsevier Science Publishers B.V. All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher, Elsevier Science Publishers B.V., Copyright & Permissions Department, P.O. Box 521, 1000 AM Amsterdam, The Netherlands. Special regulations for readers in the U.S.A.-This publication has been registered with the Copyright Clearance Center Inc. (CCC), Salem, Massachusetts. Information can be obtained from the CCC about conditions under which photocopies of parts of this publication may be made in the U.S.A. All other copyright questions, including photo- copying outside of the U.S.A., should be referred to the copyright owner, Elsevier Science Publishers B.V., unless otherwise specified. No responsibility is assumed by the publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein. This book is printed on acid-free paper. Printed in The Netherlands V PREFACE The BANKAI Workshops provide a forum for focused discussion and information sharing on specific advanced I.T. technologies and financial applications. Attendance is limited to those actively involved in the development and practical introduction of these technologies into banking and financial organizations. The Third BANKAI Workshop on Adaptive Intelligent Systems was held in Brussels in October 1992. An international group of system developers and managers from academia, the financial industry and their suppliers met to discuss the practical concerns and directions which affect their tasks in building intelligent systems which can either adapt themselves or be easily adapted to changing financial environments. The programme committee was comprised of: Robert Phelps S.W.I.F.T. s.c. Luc Steels Vrije Universiteit Brüssel Jane Nyakairu National Westminster Bank Richard Flavell Imperial College The ambiance of these Workshops differs from most A.I. conferences in concentrating on real experience and present needs rather than theoretical possibilities or limited prototype applications. It is hoped that these proceedings will give better insight into advanced I.T. practice and potential as it exists today. As well as the formal presentations included here, which covered a wide range of technologies and approaches to ensuring adaptivity, an important feature of the Workshop was the inclusion of three major debate sessions. These focused on neural networks, classical software engineering techniques and rule-based systems. These debates have been edited and are included in this volume to give direct insight into the practical concerns which affect and motivate today's developers. Adaptive Intelligent Systems, S.W.I.F.T. (ed.) © 1993 Elsevier Science Publishers B.V. All rights reserved. 1 Artificial Neural Networks and ARIMA-Models within the Field of Stock Market Prediction - A Comparison Dipl.-Kfm. Thomas Lohrbach, Prof. Dr. Matthias Schumann University Goettingen, School of Management, Information Systems Department, Platz der Goettinger Sieben 7, D-3400 Goettingen, Federal Republic of Germany, Phone: +49-551/39-4433 and 39-4442, Fax: +49-551/39-9679 Abstract: Within the field of stock market prediction a controversial discussion between technicians and fundamentalists concerning the qualification of these different methods has taken place. On the one hand, experts use so-called charts to extract those formations they regard to be significant for the future development of stock prices. This procedure requires extensive experience in recognizing and interpreting the patterns and can also contain many sources of error. On the other hand, the fundamentalists have to decide which information, even regarding other influences, they consider. Therefore, it is intended to link both perspectives. Some analysts use statistical methods (i.e. moving averages or auto-regressive models) in order to indicate important clues concerning future trends in stock prices. The ARIMA-Model combines the abilities of these two methods. Another problem-solving approach uses Artificial Neural Networks (ANN). They are in a loose sense based on concepts derived from research into the nature of the brain [16]. Particularly the ANN's ability of filtering 'noisy' influences, which may be caused by differential behaviour of various investors seems to predetermine this approach. Our intention for both approaches is a short-term prediction (the following day's stock price). In spite of that this will be extended to a medium-term prediction (a monthly forecast). 1. CHARACTERIZATION OF THE STOCK MARKET PREDICTION For predicting stock market prices, it is necessary to discuss whether an influence exists between the information of the past and the prospective development of the prices. Presuming that the development of prices depends on the decisions of potential investors, who only can take those values of the past into their consideration, a certain slope must exist. For a prediction it is necessary that the stock market is not an information-efficient market. Such a market is characterized by Fama [3] in the following way: " A market in which prices always 'fully reflect' available information is called efficient." As a 2 T. Lohrbach and M. Schumann result information influencing the market must not allow any profits because the market itself reacts on proclaimed 'news' with an immediate adaptation of the prices. Various interpretations concerning the presence of information efficiency exist [7, 27 and 15] Thus, analysing the possibilities of predicting stock market prices also implies the denial of information efficiency at a first glance. The next step will show which information might be important for price changes. There are various issues that come into account such as overall economic development and situation on the capital market etc. [9] or merely the course of the shares on their own respectively mathematical transformations of these. On the one hand the technicians maintain that all factors which influence the price level are still considered in the quotation, since it represents the supply and demand on the stock market [2 and 6]. On the other hand the fundamentalists even regard external terms like interest rates and economic policy etc. separately [9]. They argue that such influences are not implicitly regarded in the prices. Both ideas are reflected upon in this paper. In addition, a period must be fixed for which the prognosis has to be investigated. Often, one tries to identify a long-term trend (i.e. one year) for the price development. According to a fast reaction on changes in the stock market a shorter period (i.e. one day/one month) may be more interesting. 2. DATA MATERIAL AND METHODS FOR PROGNOSIS Although quotations of various shares, indices and other so-called 'external' information were available (period from 12-31-82 till 12-31-91), within this article it is only referred to the German Hochtief-share and Deutscher Aktienindex (DAX). The presentation is bounded to these two values, for the main conclusions of our investigations, this limitation is of no significance. External information (daily quoted) according to: the number of all traded shares, the number of shares that increased, remained at the same level and decreased in the Frankfurt stock market and Wall Street, Dow- Jones-, Nikkei-, Financial-Times- and Westbau-lndex (index, referring to shares within the field of construction trade), money market rates, exchange rates, gold price as well as oil price and external information (monthly quoted) according to: Consumer price index, money stock, unemployment rate, incoming orders within the field of construction trade, capital goods industry, consumer goods industry and processing industry, production within the field of construction trade, capital goods industry, consumer goods industry and processing industry was available. The next question concerns methods, used for prediction. Because of the (supposed) influence of stochastic elements on those time series, it is necessary to use a method which is able to filter such un de si red elements. One method with such filtering abilities is the ARIMA-method. The virtue of ARIMA is well characterized by Vandaele [28]: "... can be viewed as an approach by which time series data are sifted through a series of progressively finer sieves ..." The aim of sifting some components is to identify so-called 'white-noise-processes' (merely stochastic influences on the time series). Another approach with such capabilities is ANN. ANN consist of many simple elements (units, neurons, processing elements, PE) which are interconnected [21]. Their way of working can be described as the parallel interaction of these simple elements where several operations are performed at the Artificial Neural Networks and ARIMA-Models 3 same time. ANN are not programed but trained with a large amount of examples. ANN do not store their 'information' locally but all units are responsible for working correctly [11]. This results in a major advantage since the loss of some elements or incomplete input does not automatically lead to a wrong answer. These abilities predetermine ANN for the prediction of stock price development. 3. STOCK PREDICTION WITH ANN 3.1. Describing of the ANN'S configuration Within this investigation, a Counterpropagation Network (CPG) and the Software NWorks (using an IBM RS 6000) was used. The CPG consists of four layers (Input-, Normalization-, Kohonen- and Output-Layer) and selects from a set of exemplars by allowing the neurons to compete amongst each other [16,18 and 5]. Some problems need consideration. No detailed instructions exist concerning the dimension of the Kohonen Layer. One might suggest utilizing two elements, since it deals with the prediction of whether a stock price increases or decreases. But it is questionable if the complete data set only consists of two exactly defined classes. Various patterns might exist which are too different to be represented by a single neuron only. Thus, one can propose to utilize as many neurons as there are training examples. This would cause complex nets which might not be able to extract reliable information but merely memorize all training data. The configuration of the Kohonen Layer, in order to find a compromise between memorization and generalization, needs variation. One has to create stationary time series, because when using original values, identified structures of the past can not be transposed into the future due to their different spread of values. The question remains how long a once-trained net is able to predict. Testing various possibilities is necessary. First of all within the daily prediction, two proportions (prop. 1: 2089 training data, 50 testing data; prop. 2: 209 training data, 50 testing data) were investigated. For the fundamental analysis a third proportion (1604 training data, 535 testing data) was regarded. In the case of the medium-term prediction 78 training data and 26 testing data were used (prop. 4). With regard to the number of training steps, two different approaches are possible. The training volume might be determined by the user, but this procedure seems to be very arbitrary. Therefore, the error of the determined and existing output during the training is used as a convergence criterion [18]. Training is terminated if this output error reaches a very low level or if it does not change during a large number of training steps. Fig. 1 shows which output classification was investigated: Output layer Output Output layer Output PE pos. neg. PE pos. neg. 1 price increases the following day 1 0 1 price increases the following month 1 0 2 price decreases the following day 1 0 2 price decreases the following month 1 0 3 price increases the following day 1 0 3 price increases the following month 1 0 significant (>0.5 %) significant (>1.5 %) price decreases the following day 1 0 price decreases the following month 1 0 significant (>0.5 %) l· significant (>1.5 %) Fig. 1 : Description of the Output (Daily vs. Monthly Prediction) 4 T. Lohrbach and M. Schumann Because the output of the ANN is between 0 and 1, it must be interpreted. A result near to 0 or 1 will be regarded as price decreases or increases. Three methods for the interpretation are used (see fig. 2). A statistical auxiliary means (cross-correlation analysis) should show which of the time series is influential for the further develop- ment of the investigated shares and therefore was used as input data. Unfortunately, this procedure only works with respect to the short-term prediction. The analysis does not identify influences within different monthly time series because the significance level is not touched as a result of reducing the number of examples (only 78). Fig. 2: Methods for Interpretation of the Output 3.2. Results using ANN for prediction 3.2.1. Short-term prediction 3.2.1.1. Technical approach The first tests used the last 40 quotations as input for the ANN. Each was presented as the relative change of two following days (continuously coding). The figures below show the results (in percent) of correctly classified output, according to each output neuron. Beyond that, the notation pos./neg. explain the number of correctly recognized output values 1/0 (see fig. 1). The notation class, describes the number of classified values (see fig. 2). I ^ 1 2 3 4 "1 I method pos. neg. class. pos. neg. class. pos. neg. class. pos. neg. class. 1 in% in% in% in% in% in% ' in% in% in% in% in% in% A 66.67 47.73 100 47.73 66.67 100 100 79.17 100 0 79.59 100 I B 100 0 4 0 100 4 100 92.31 28 0 84.62 52 | C 0 0 0 0 0 0 0 100 36 0 86.36 44 I Fig. 3: Results and Number of Classifications in Percent (DAX), 'Pure' Time-series, Prop. 1 Such a spreading of the results seems to be necessary, since only one digit alone is not satisfactory for a differentiation. An example can explain the necessity of such a differentiation on the outcome. Simply counting the 'correct' prognosis without any reference to whether it dealt with an increasing or decreasing output, using PE1 attains 25 'correct' answers (50%) whereas PE3 makes 40 'correct' statements (80%). Depending on the choice of either PE1 or PE3, one has to interpret the Artificial Neural Networks and ARIMA-Models 5 results either as 'bad' or 'good'. Thus, merely regarding such a single digit does not allow clear conclusions concerning the ability for recognizing structures. 1 PE 1 2 3 4 1 [method pos. neg. class. pos. neg. class. pos. neg. class. pos. neg. class. I in% in% in% in% in% in% in% in% in% in % in% in% A 43.75 61.76 100 61.76 43.75 100 44.44 75.61 100 100 61.22 100 B 0 0 0 0 0 0 0 100 2 0 64.71 34 c L 0 0 0 0 0 0 0 0 0 0 60 10 | Fig. 4: Results and Number of Classifications in Percent (Hochtief-share), 'Pure' Time-series, Prop. 1 The high results of PE 3 and PE4 for the Hochtief-share are remarkable. But it should be noted that the networks only recognized decreasing developments with such high results. Although the main trend of prices also decreased, it is questionable whether a sufficient generalization has been attained. Aditionally, prop. 2 is investigated. One could expect that the results improve, the more recent data is used. In spite of this assumption, the results arranged contrary. In this case , the smaller temporal distance of training and testing data did not lead to an advantage. Subsequently, so-called indicators are investigated (see fig. 5 and 6). As opposed to the above-described procedure not only the time-series itself but transformations of these are used. Five of those will be taken into consideration. The Trend-Oscillator (TO) [12], the Relative-Strongness (RS) [17], the Momentum (MM) [10], the Relative- Strongness-lndex (RSI) and the Overbought/Oversold Indicator (OBOS) [10]. Regarding all indicators, a cross-correlation analysis (used as a statistical auxiliary means) showed the highest influence of MM and OBOS for the DAX and of RS and RSI for the Hochtief-share. Additionally, it is to allude that the correlation of all ex- tracted indicators only was close-fitting the significance level. I PE 1 2 3 4 | I method pos. neg. class. pos. neg. class. pos. neg. class. pos. neg. class. I in% in% in% in% in% in% in% in% in% in% in% in% A 58.62 57.14 100 55.56 59.38 100 18.75 76.47 100 25 78.57 100 B 71.43 61.54 54 60 68.75 52 33.33 72.73 56 33.33 75.76 72 | C 100 75 20 100 87.5 18 25 85.71 22 25 73.68 38 I Fig. 5: Results and Number of Classifications in Percent (DAX), Using Indicators, Prop. 1 τ~ PE 1 ι 3 4 method pos. neg. class. pos. neg. class. DOS. neg. class. pos. neg. class. in% in% in% in% in% in% in% in% in% in% in% in% A 45 63.33 100 66.67 51.43 100 38.89 81.25 100 45.45 61.54 100 B 44.44 84.62 44 100 50 34 60 81.3 56 66.67 57.89 44 C 100 85.71 16 100 66.67 14 100 85.71 30 60 20 Fig. 6: Results and Number of Classifications in Percent (Hochtief-share), Using Indicators, Prop. 1 6 T. Lohrbach and M. Schumann Subsequently, prop. 2 was investigated. As opposed to the analysis of the 'pure time-series' the tests of indicators lead to nearly the same results for the DAX. Indeed, the number of classifications concerning the hard criteria increased. Only the results of the DAX are mentioned (see fig. 7) because those of Hochtief worsened. 1 ^ 1 2 3 4 | 1 method pos. neg. class. pos. neg. class. pos. neg. class. pos. neg. class. I in % in% in% in% in% in% in % in% in% in% in% in% A 75 55.26 100 58.33 65.38 100 30 80 100 20 77.5 100 B 75 54.84 86 63.16 66.67 86 16.67 82.86 82 25 78.95 84 C 75 54.84 86 63.16 66.67 86 16.67 82.86 41 25 78.95 42 | Fig. 7: Results and Number of Classifications in Percent (DAX), Using Indicators, Prop. 2 On the one hand, the results using indicators (see fig. 5 and 6), referring to method A, do not seem to differ significantly to the results using 'pure time-series'. On the other hand, there are differences in the case of the hard criteria method B and C (see fig. 2). At a first glance, a comparison between the results by using 'pure' time-series and the results by using indicators is difficult because the number of classifications is almost zero (using 'pure' time-series, method B and C, see fig. 4) and so no prognosis took place in the first case. Taking into account that in both series of tests (pure time-series vs. indicators) identical variations for determining the best number of neurons in the Kohonen Layer are used, one can draw the conclusion to prefer indicators. As mentioned in 3.1., only a hard criterion is able to identify relevant patterns. Thus, the results using indicators are regarded as being better because more 'hard selected' samples have been recognized. An improvement in this case might be the usage of pruning during training. Pruning can be understood as a method that attempts minimizing both, network complexity and error over the learning data set. An ANN with minimal complexity which does well on a learning data set will generalize for the future better than a more complex network. The reduction of complexity will be attained by removing those small weights whose influence in gaining a good result is neglectable [8]. Then, all relevant indicators could be identified because the ANN itself judges whether a piece of information is necessary for classification or not. This will be our next topic of further research. 3.2.1.2. Fundamental approach In a first step, the cross-correlation (as a statistical auxiliary means) between the above-mentioned shares and all daily available time-series was investigated. Those with the highest correlation were used as input. Doing so, the results turned out to be extremely worse. An analysis of the data material showed that the structure of the proportions, as opposed to the technical approach, was not suited. An example may illustrate this. The Dow-Jones showed the highest correlation with the DAX. Whereas the development (prop. 1 and prop. 2) of the DAX followed that of the Dow-Jones in 60,49% with regard to the whole data material, the analysis of the testing data merely points to a quota of 52%. Therefore, a third prop. (prop. 3, 1.604 learning data and 535 testing data) was built which beared this fact in mind. At first the DAX will be regarded. Beginning with all daily available information as input (22 time-series), the results were not satisfying. Therefore, the input needed Artificial Neural Networks and ARIMA-Models 7 further analysis. 7 of the 22 time-series refer to stock statistics, i.e. number of all increased/decreased/not-changed shares at Frankfurt Stock Market etc. Their influence was reduced, since it seemed to be sufficient to regard only the number of increased shares. Decreased and not-changed shares certainly will be correlated to the increased. An elimination of some Exchange Rates and Money Market Rates took place, too. This lead to the remaining 13 time-series (see fig. 8). Tests were much better than those of the bigger input vector. I PË 1 2 3 4 | I method pos. neg. class. pos. neg. class. pos. neg. class. pos. neg. class. I in% in% in% in% in% in% in% in% in% in% in% in% A 49.8 55.94 100 55.43 56.41 100 53.85 72.08 100 65.00 72.04 100 B 87.5 68.18 8.6 55.56 91.67 7.85 83.33 79.41 26.54 66.67 71.21 25.23 I C 55.56 66.67 5.05 55.56 77.78 5.05 100 86.67 5.98 66.67 86.21 5.98 | Fig. 8: Results and Number of Classifications in Percent (DAX), Using Fundamental Information, Prop. 3 A further reduction of the information (only seven time-series) worsened the re- sults. On the one hand, the change from 22 to 13 input elements showed that too much information can cause a fitting of the noise [29]. On the other hand, the loss of relevant information produces bad results. This points out that such a 'manual' pruning is not satisfactory because it is too inexact, and therefore confirms again the importance of pruning. The same way was used for predicting the DAX as applied for the Hochtief-share (see fig. 9). The conclusions coincided with those of the DAX. PE 1 I I 3 iI « I method pos. neg. class. pos. neg. class. pos. neg. class. pos. neg. class. in% in% in% in% in% in% in% in% in% in% in% in% A 51.11 59.15 100 54.63 55.74 100 41.18 66.22 100 36.59 62.75 100 B 52.63 61.22 21.87 50 57.32 22.43 36.84 69.44 30.47 33.33 67.65 41.5 C 53.85 59.09 10.65 44.83 53.45 16.26 38.46 64.15 12.34 33.33 66.25 18.32 Fig. 9: Results and Number of Classifications in Percent (Hochtief-share), Using Fundamental Information, Prop. 3 Overall, the outcome concerning the identification of relevant patterns did not improve. Especially when one regards the DAX, using indicators, prop. 2, the results of the fundamental approach are to be considered as worse (i.e. with regard to the number of classified cirteria B and C). But one has to take into account that the number of classifications of prop. 3 refer to a higher number of testing data than i.e. those of prop. 1. The probability that relevant patterns were identified might therefore be higher for the greater proportion (prop. 3) than for the smaller one (prop. 2). 3.2.1.3. Combining technical and fundamental approach The next test consists of not only analyzing either technical or fundamental information but to combine both approaches. Indeed, only the best input concerning the technical as well as the fundamental approach has been combined using prop.3.