UNIVERSITY OF TÜBINGEN WILHELM-SCHICKARD-INSTITUTE FOR COMPUTER SCIENCE Department of Computer Architecture JavaNNS Java Neural Network Simulator User Manual, Version 1.1 Igor Fischer, Fabian Hennecke, Christian Bannes, Andreas Zell Contents 1. Introduction..................................................................................................................4 1.1. How to read this manual.......................................................................................4 2. Licensing and Acknowledgements..............................................................................5 2.1. License Agreement...............................................................................................5 2.2. Acknowledgments................................................................................................6 3. Installation....................................................................................................................8 3.1. Running JavaNNS................................................................................................8 4. A Quick Tour of JavaNNS...........................................................................................9 4.1. Starting JavaNNS.................................................................................................9 4.2. Loading Files........................................................................................................9 4.3. View Network.......................................................................................................9 4.4. Training Network.................................................................................................9 4.5. Analyzing Network.............................................................................................10 4.6. Creating a Network.............................................................................................10 4.7. Graphical Network Display................................................................................10 4.8. Training and Validation Pattern Sets..................................................................11 5. Network Creation and Editing...................................................................................12 5.1. Network View and Display Settings..................................................................12 5.2. Tools for Creating Networks..............................................................................13 5.3. Editing Units.......................................................................................................14 6. Pattern Management...................................................................................................15 7. Training and Pruning Networks.................................................................................16 8. Analyzing Networks...................................................................................................18 8.1. Projection Panel..................................................................................................18 8.2. Weights Panel.....................................................................................................18 8.3. Analyzer..............................................................................................................19 9. Loading, Saving and Printing.....................................................................................20 Appendix A:Kernel File Interface.................................................................................21 A.1.The ASCII Network File Format........................................................................21 A.2.Form of the Network File Eries..........................................................................21 A.3.Grammar of the Network Files...........................................................................22 A.4.Grammar of the Pattern Files..............................................................................27 Appendix B: Example Network File..............................................................................30 B.1.Example 1:..........................................................................................................30 B.2.Example 2:..........................................................................................................32 4 1. Introduction Java Neural Network Simulator (JavaNNS) is a simulator for neural networks developed at the Wilhelm-Schickard-Institute for Computer Science (WSI) in Tübingen, Germany. It is based on the Stuttgart Neural Network Simulator (SNNS) 4.2 kernel, with a new graphical user interface written in Java set on top of it. As a consequence, the capabilities of JavaNNS are mostly equal to the capabilities of the SNNS, whereas the user interface has been newly designed and -- so we hope -- become easier and more intuitive to use. Some complex, but not very often used features of the SNNS (e.g. three-dimensional display of neural networks) have been left out or postponed for a later version, whereas some new, like the log panel, have been introduced. Besides the new user interface, a big advantage of JavaNNS is its increased platform indepen- dence. Whereas SNNS was developed with primarily Unix workstations in mind, JavaNNS also runs on PCs, provided that the Java Runtime Environment is installed. As of writing of this manual JavaNNS has been tested on: (cid:127) Windows NT (cid:127) Windows 2000 (cid:127) RedHat Linux 6.1 (cid:127) Solaris 7 (cid:127) Mac OS X with more to follow soon. 1.1. How to read this manual Because of large similarities between SNNS and JavaNNS, this manual covers only the differ- ences between the two. It should be read as a companion to the SNNS User Manual, available from the WSI web site: http://www-ra.informatik.uni-tuebingen.de/SNNS/ We suggest that you first read the SNNS Manual, in order to become acquainted with the the- ory of neural networks, the way they are implemented in the SNNS kernel and to get a basic idea of the SNNS graphical user interface. If you are already familiar with SNNS, you can skip this step and start directly with this manual. In the next chapter, you will find the license agreement. Please read it carefully and make sure that it is acceptable for you before installing and using JavaNNS. The installation process dif- fers slightly for Windows and Unix machines and is described separately for each case. After installing, we suggest that you follow our quick tour through the simulator to get the first impression of how it is organized and used. The rest of the manual covers in more detail creat- ing, manipulation and analyzing neural networks. You can skim it in the first reading and use it later as a reference. 5 2. Licensing and Acknowledgements JavaNNS is Copyright (c) 1996-2001 JavaNNS Group, Wilhelm-Schickard-Institute for Com- puter Science (WSI), University of Tübingen, Sand 1, 72076 Tübingen, Germany. It uses the kernel of SNNS (Stuttgart Neural Network Simulator), which is Copyright (c) 1990-95 SNNS Group, Institute for Parallel and Distributed High-Performance Systems (IPVR), University of Stuttgart, Breitwiesenstrasse 20-22, 70565 Stuttgart, Germany. Currently, JavaNNS is distributed by the University of Tübingen and only as a binary. Although it is distributed free of charge, please note that it is NOT PUBLIC DOMAIN. The JavaNNS License is gives you the freedom to give away verbatim copies of the JavaNNS distribution (which include the license). We do not allow modified copies of our software or software derived from it to be distributed. You may, however, distribute your modifications as separate files along with our unmodified JavaNNS software. We encourage users to send changes and improvements which would benefit many other users to us so that all users may receive these improvements in a later version. The restriction not to distribute modified copies is also useful to prevent bug reports from someone else's modifications. For our protection, we want to make certain that everyone understands that there is NO WAR- RANTY OF ANY KIND for the JavaNNS software. 2.1. License Agreement 1. This License Agreement applies to the JavaNNS program and all accompanying programs and files that are distributed with a notice placed by the copyright holder saying it may be distributed under the terms of the JavaNNS License. 'JavaNNS', below, refers to any such program or work, and a 'work based on JavaNNS' means either JavaNNS or any work con- taining JavaNNS or a portion of it, either verbatim or with modifications. Each licensee is addressed as 'you'. 2. You may copy and distribute verbatim copies of the JavaNNS distribution as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty, keep intact all the notices that refer to this License and to the absence of any warranty, and give any other recipients of JavaNNS a copy of this license along with JavaNNS. 3. You may modify your copy or copies of JavaNNS or any portion of it only for your own use. You may not distribute modified copies of JavaNNS. You may, however, distribute your modifications as separate files (e.,g. new network or pattern files) along with our unmodified JavaNNS software. We also encourage users to send changes and improve- ments which would benefit many other users to us so that all users may receive these improvements in a later version. The restriction not to distribute modified copies is also useful to prevent bug reports from someone else's modifications. 4. If you distribute copies of JavaNNS you may not charge anything except the cost for the media and a fair estimate of the costs of computer time or network time directly attribut- able to the copying. 5. You may not copy, modify, sublicense, distribute or transfer JavaNNS except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, distribute or transfer JavaNNS is void, and will automatically terminate your rights to use JavaNNS under this License. However, parties who have received copies, or rights to use copies, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 6 6. By copying, distributing or modifying JavaNNS (or any work based on javaNNS) you indicate your acceptance of this license to do so, and all its terms and conditions. 7. Each time you redistribute JavaNNS (or any work based on JavaNNS), the recipient auto- matically receives a license from the original licensor to copy, distribute or modify JavaNNS subject to these terms and conditions. You may not impose any further restric- tions on the recipients’ exercise of the rights granted herein. 8. Because JavaNNS is licensed free of charge, there is no warranty for it, to the extent per- mitted by applicable law. The copyright holders and/or other parties provide JavaNNS ’as is’ without warranty of any kind, either expressed or implied, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. The entire risk as to the quality and performance of JavaNNS is with you. Should the program prove defective, you assume the cost of all necessary servicing, repair or correction. 9. In no event will any copyright holder, or any other party who may redistribute JavaNNS as permitted above, be liable to you for damages, including any general, special, incidental or consequential damages arising out of the use or inability to use JavaNNS (including but not limited to loss of data or data being rendered inaccurate or losses sustained by you or third parties or a failure of JavaNNS to operate with any other programs), even if such holder or other party has been advised of the possibility of such damages. 2.2. Acknowledgments JavaNNS is a joint effort of a large number of people, computer science students, research assistants as well as faculty members at the Institute for Parallel and Distributed High Perfor- mance Systems (IPVR) at University of Stuttgart, the Wilhelm Schickard Institute of Com- puter Science at the University of Tübingen, and the European Particle Research Lab CERN in Geneva. The project to develop an efficient and portable neural network simulator which later became SNNS was lead since 1989 by Dr. Andreas Zell, who designed SNNS and acted as advisor for more than two dozen independent research and Master's thesis projects that made up SNNS, JavaNNS and some of its applications. Over time the source grew to a total size of now 5MB in 160.000+ lines of code. Research began under the supervision of Prof. Dr. Andreas Reuter and Prof. Dr. Paul Levi. We are grateful for their support and for providing us with the neces- sary computer and network equipment. The following persons were directly involved in the SNNS project. They are listed in the order in which they joined the SNNS team. Table 1: SNNS and JavaNNS project members Andreas Zell Design of the SNNS simulator, SNNS project team leader. Niels Mache SNNS simulator kernel (really the heart of SNNS), parallel SNNS kernel on MasPar MP-1216. Tilman Sommer original version of the graphical user interface XGUI with integrated network editor, PostScript printing. Ralf Hübner SNNS simulator 3D graphical user interface, user interface development (version 2.0 to 3.0). Thomas Korb SNNS network compiler and network description language Nessus. 7 Table 1: SNNS and JavaNNS project members Michael Vogt Radial Basis Functions. Together with Günter Mamier imple- mentation of Time Delay Networks. Definition of the new pat- tern format. Günter Mamier SNNS visualization and analyzing tools. Implementation of the batch execution capability. Together with Michael Vogt implementation of the new pattern handling. Compilation and continuous update of the user manual. Maintenance of the ftp server. Bugfixes and installation of external contributions. Michael Schmalzl SNNS network creation tool Bignet, implementation of Cas- cade Correlation, and printed character recognition with SNNS. Kai-Uwe Herrmann ART models ART1, ART2, ARTMAP and modification of the BigNet tool. Artemis Hatzigeorgiou documentation about the SNNS project, learning procedure Backpercolation 1. Dietmar Posselt ANSI-C translation of SNNS. Sven Döring ANSI-C translation of SNNS and source code maintenance. Implementation of distributed kernel for workstation clusters. Tobias Soyez Jordan and Elman networks, implementation of the network analyzer. Tobias Schreiner Network pruning algorithms. Bernward Kett Redesign of C-code generator snns2c. Jens Wieland Design and implementation of batchman. Jürgen Gatter Implementation of TACOMA and some modifications of Cas- cade Correlation. Igor Fischer Java user interface design and development. Fabian Hennecke Java user interface development. Christian Bannes Java user interface support Hans Rudolph JavaNNS kernel port for Mac OS X There are a number of important external contributions by: Martin Reczko, Martin Riedmiller, Mark Seemann, Marcus Ritt, Jamie DeCoster Jochen Biedermann, Joachim Danz, Christian Wehrfritz Randolf Werner, Michael Berthold and Bruno Orsier. 8 3. Installation To be able to use JavaNNS, you have to have Java Runtime Environment (or JDK, which con- tains it) installed. JavaNNS has been tested to work with Java 1.3, Java 1.2.2 might also work, though problems with file management have been reported in certain environments. JavaNNS for Windows platforms is distributed as the zip file JavaNNS-Win.zip, and as gzipped tar archive, like JavaNNS-LinuxIntel.tar.gz and JavaNNS-Solaris.tar.gz for other operating systems. Unzip (unpack) the file into a folder of your choice. You should get: 1. JavaNNS.jar - the Java archive file containing the JavaNNS user interface classes 2. examples - folder with example networks, patterns etc. 3. manual - folder containing this manual JavaNNS needs the kernel library in order to work properly. If you run JavaNNS the first time on your machine a dialog will appear to ask you where to install the library. JavaNNS then rememberes the location of the library by storing it in the file JavaNNS.prop- erties, which is generated and placed into your home directory (Personal in Windows termi- nology). If you which to change properties, use Properties editor in the JavaNNS View menu. You can delete the library or the JavaNNS.properties file any time you want. In that case the library installation dialog will appear the next time you start JavaNNS. 3.1. Running JavaNNS That’s all! Now you can start JavaNNS by typing: java -jar JavaNNS.jar from the command prompt or by clicking the JavaNNS.jar file from the graphical user inter- face. 9 4. A Quick Tour of JavaNNS JavaNNS is a simulator for artificial neural networks, i.e. computational models inspired by biological neural networks. It enables you to use predefined networks or create your own, to train and to analyze them. If any of these terms is unknown to you, please refer to a book about neural networks or to the SNNS User Manual - this manual describes only the usage of JavaNNS. 4.1. Starting JavaNNS To begin the tour, let’s start JavaNNS, as described in "Installing": type java -jar JavaNNS.jar or, if using Windows, click the JavaNNS.bat file. After starting the program, its main window opens. As we have started the program no parameters in the command line, the window is empty, containing only the usual menu bar. Also, no network files have been loaded. 4.2. Loading Files Use File/Open menu to open an example file: navigate to the examples directory and open the files xor_untrained.net, and xor.pat - a simple network and a corresponding pattern file. 4.3. View Network The main window still remains empty, so choose View/Network to display the network. You should see a new window appearing, schematically showing a network, consisting of 4 units (neurons) and links between them, in its main part. Neurons and links have different colors, representing different values of unit activations and link weights. The colored bar on the left edge of the window shows which color corresponds to which value and can be used as reminder. The colors - and appearance in general - can be adjusted through View/Display Set- tings, which corresponds to the Display/Setup window in SNNS. 4.4. Training Network Let us now train the network - reprogram its weights, so that it gives the desired output when presented an input pattern. For that purpose, open the Control Panel in the Tools menu. The Control Panel is, as in the SNNS, the most important window in the simulator, because almost all modifications and manipulations of the network are done through it. We shall also open the Error Graph window, in order to watch the training progress. Finally, to receive some textual and numerical information, we can open the Log window. Both are accessible through the View menu. A sample screen shot with the windows open is shown in Figure 1. The Control Panel is, contrary to the one in SNNS, divided into six tabs, each containing con- trols for specific purpose. For this introduction, let us switch directly to the learning tab. Here, the user can choose the learning function, set its parameters, number of learning cycles and update steps and finally perform network initialization and learning. The classic Backpropaga- tion (equals Std_Backpropagation in SNNS) is the default learning function. As you can see, for each learning function default parameters are provided. Learning is performed by pressing one of the buttons: "Learn current" which performs training with the currently selected pattern, and "Learn all", which trains the network with all patterns from the pattern set. During learning, the error graph displays the error curve - the type of error to be drawn is set on the left edge of the window. The error is also written into the log window. 10 Figure 1: JavaNNS with XOR network, error graph, control and log panel 4.5. Analyzing Network For analyzing the network and its performance, tools like Analyzer (in the Tools menu) and Projection (in the View menu), already familiar to SNNS users, can be used. For Projection, two input units and a hidden or output unit have to be selected in order for the menu item to become enabled. The Projection Panel than displays the activation of the hidden or output unit as a function of the two inputs. The activation is represented by color, so a colored rectangle is obtained. Analyzer is used to show output or activation of a unit as a function of other unit’s activation or of the input pattern. Its usage is similar to the Analyze panel in the SNNS. 4.6. Creating a Network Now let’s create a network of our own. Choose File/New to remove the current network from the simulator. Then, choose Create/Layers from the Tools menu. A window resembling the Bignet tool of SNNS appears. Choose width and height "1", unit type "Input" and click "Cre- ate" to create a new layer. For the next layer, set height to five and the unit type to "Hidden" and click "Create" again. Finally, create the output layer with the height of one and unit type "Output" and close the window. To connect the created units, use Create/Connections from the Tools menu. Simply choose "Connect feed-forward" and click "Connect". Doing that, you have created a simple feed-forward neural network, with one input, five hidden and one output unit. You can now close the Connections window, too. 4.7. Graphical Network Display You can arrange units on the display manually, by clicking and dragging them with the mouse. In fact, clicking a unit selects it, and dragging moves all selected units. To deselect a unit, press the CTRL-Key on the keyboard and click it while still holding the key pressed. Using