AMPS Java Development Guide AMPS Java Development Guide 5.2 Publication date Jun 26, 2017 Copyright © 2017 All rights reserved. 60East, AMPS, and Advanced Message Processing System are trademarks of 60East Technologies, Inc. All other trademarks are the property of their respective owners. Table of Contents 1. Introduction ................................................................................................................................. 1 1.1. Prerequisites ...................................................................................................................... 1 1.2. Upgrading from a Previous Version of the AMPS Java Client ..................................................... 1 2. Installing the AMPS Client ............................................................................................................. 2 2.1. Prerequisites ...................................................................................................................... 2 2.2. Obtaining the Client ............................................................................................................ 2 2.3. Test Connectivity to AMPS .................................................................................................. 2 3. Your First AMPS Program ............................................................................................................. 3 3.1. About the Client Library ..................................................................................................... 3 3.2. Connecting to AMPS .......................................................................................................... 3 3.3. Connection Strings ............................................................................................................. 6 3.4. Connection Parameters ........................................................................................................ 7 3.5. Next Steps ........................................................................................................................ 8 4. Subscriptions ................................................................................................................................ 9 4.1. Subscribing ....................................................................................................................... 9 4.2. Asynchronous Message Processing Interface ......................................................................... 11 4.3. Unsubscribing .................................................................................................................. 12 4.4. Understanding Messages .................................................................................................... 13 4.5. Understanding Threading and Message Handlers .................................................................... 13 4.6. Advanced Messaging Support ............................................................................................. 14 4.7. Next Steps ....................................................................................................................... 15 5. Error Handling ............................................................................................................................ 16 5.1. Exceptions ....................................................................................................................... 16 5.2. Disconnect Handling ......................................................................................................... 18 5.3. Unexpected Messages ........................................................................................................ 21 5.4. Unhandled Exceptions ....................................................................................................... 21 5.5. Detecting Write Failures .................................................................................................... 22 6. State of the World ....................................................................................................................... 24 6.1. Performing SOW Queries ................................................................................................... 24 6.2. SOW and Subscribe .......................................................................................................... 25 6.3. Setting Batch Size ............................................................................................................ 27 6.4. Client-Side Conflation ....................................................................................................... 28 6.5. Managing SOW Contents ................................................................................................... 29 7. Using Queues ............................................................................................................................. 30 7.1. Backlog and Smart Pipelining ............................................................................................. 30 8. Delta Publish and Subscribe .......................................................................................................... 33 8.1. Introduction ..................................................................................................................... 33 8.2. Delta Subscribe ................................................................................................................ 33 8.3. Delta Publish ................................................................................................................... 33 9. High Availability ......................................................................................................................... 35 9.1. Reconnection with HAClient .............................................................................................. 35 9.2. Choosing Store Durability .................................................................................................. 36 9.3. Connections and the ServerChooser ..................................................................................... 37 9.4. Heartbeats and Failure Detection ......................................................................................... 40 9.5. Considerations for Publishers .............................................................................................. 40 9.6. Considerations for Subscribers ............................................................................................ 42 9.7. Conclusion ...................................................................................................................... 45 10. AMPS Programming: Working with Commands ............................................................................. 46 10.1. Understanding AMPS Messages ........................................................................................ 46 10.2. Creating and Populating the Command ............................................................................... 46 10.3. Using execute ................................................................................................................. 47 iii AMPS Java Development Guide 10.4. Command Cookbook ....................................................................................................... 48 11. Utilities .................................................................................................................................... 69 11.1. Composite Message Types ................................................................................................ 69 11.2. NVFIX Messages ............................................................................................................ 70 11.3. FIX Messages ................................................................................................................. 72 12. Advanced Topics ....................................................................................................................... 75 12.1. C# Client Compatibility ................................................................................................... 75 12.2. Rebuilding the Client ....................................................................................................... 75 12.3. Transport Filtering ........................................................................................................... 75 12.4. Working with Messages & Byte Buffers .............................................................................. 76 12.5. Providing SSL Certificates to the AMPS Java Client ............................................................. 77 13. Performance Tips and Best Practices ............................................................................................. 81 13.1. Measure Performance and Set Goals .................................................................................. 81 13.2. Simplify Message Format and Contents .............................................................................. 82 13.3. Use Content Filtering Where Possible ................................................................................. 82 13.4. Use Asychronous Message Processing ................................................................................ 82 13.5. Use Hash Indexes Where Possible ..................................................................................... 83 13.6. Use a Failed Write Handler and Exception Listener ............................................................... 83 13.7. Reduce Bandwidth Requirements ....................................................................................... 83 13.8. Limit Unnecessary Copies ................................................................................................ 84 13.9. Manage Publish Stores ..................................................................................................... 85 13.10. Work with 60East as Necessary ....................................................................................... 85 A. Exceptions ................................................................................................................................. 86 Index ............................................................................................................................................ 88 iv Chapter 1. Introduction This document explains how to use the Java client for AMPS. Use this document to learn how to install, configure, develop applications using AMPS. 1.1. Prerequisites Before reading this book, it is important to have a good understanding of the following topics: • Developing in Java. To be successful using this guide, you will need to possess a working knowledge of the Java language. Visit http://java.oracle.com for resources on learning Java. • AMPS concepts. Before reading this guide, you will need to understand the basic concepts of AMPS, such as topics, subscriptions, messages, and SOW. Consult the AMPS Users' Guide to learn more about these topics before proceeding. You will need an installed and running AMPS server to use the product as well. You can write and compile programs that use AMPS without a running server, but you will get the most out of this guide by running the programs against a working server. 1.2. Upgrading from a Previous Version of the AMPS Java Client The AMPS Java client is designed to maintain binary compatibility for hotfix versions, unless otherwise specified in the release notes. For versions that introduce or change features, as indicated by a change to the major or minor version number, 60East cannot guarantee binary compatibility. An upgrade that changes either a major or minor version number may require a recompile from source. Changes that do not affect source compatibility are not generally noted in upgrade documents when the major or minor version numbers change. For example, an upgrade from X.Y.1.0 to X.Y.1.1 of the Java client would be a drop-in replacement unless otherwise noted, as would an upgrade from X.Y.1.1 to X.Y.2.7. However, an upgrade from X.Y.2.7 to X.Z.0.0 may require that you recompile your application from source. 1 Chapter 2. Installing the AMPS Client 2.1. Prerequisites Before proceeding with this guide, make sure that the following programs are installed and functioning properly on your development machine. • Java Development Kit version 1.7 or greater. • Java Runtime Environment version 1.7 or greater. • optional Apache Ant version 1.8 or greater. 2.2. Obtaining the Client The AMPS Java client source code and pre-compiled JARs are available in the installed AMPS instance. From the directory where AMPS is installed, the Java client can be found in the api/client/java/ directory. The pre-compiled JAR files are located in the api/client/java/dist/lib/ directory, which contains the following JAR files: • amps_client.jar contains the com.crankuptheamps.client package, which includes the classes necessary to build an AMPS client. This JAR and its contents will be discussed throughout this Developer Guide. • amps_client-sources.jar contains the source code files used to create the Java implementation of the AMPS client libraries. This file can be included in an IDE to assist in debugging. It can also be used to rebuild the AMPS client libraries if any custom changes are necessary. See Section 12.2 for instructions on how to recompile the AMPS Java client source. • amps_client-javadoc.jar, like the amps_client-sources.jar, can be included in an IDE to pro- vide the javadoc annotations for the implemented classes and methods. 2.3. Test Connectivity to AMPS Before writing programs using AMPS, make sure connectivity to an AMPS server from this computer is working. Launch a terminal window and change the directory to the AMPS directory in your AMPS installation, and use spark to test connectivity to your server, for example: ./bin/spark ping -type fix -server 192.168.1.2:9004 If you receive an error message, verify that your AMPS server is up and running, and work with your systems administrator to determine the cause of the connectivity issues. Without connectivity to AMPS, you will be unable to make the best use of this guide. 2 Chapter 3. Your First AMPS Program In this chapter, we will learn more about the structure and features of the AMPS Java client library, and build our first Java program using AMPS. 3.1. About the Client Library The AMPS client is packaged as a single JAR file, amps_client.jar. You can find amps_client.jar in the dist/lib directory of the AMPS Java client installation. Every Java application you build will need to reference this JAR file, and the JAR file must be deployed along with your application in order for your application to function properly. 3.2. Connecting to AMPS Let’s begin by writing a simple program that connects to an AMPS server and sends a single message to a topic: import com.crankuptheamps.client.Client; import com.crankuptheamps.client.exception.AMPSException; public class TestPublisher { public static void main(String[] args) { Client client = new Client("TestPublisher-Client"); try { client.connect("tcp://127.0.0.1:9007/amps/json"); client.logon(); client.publish("messages", "{ \"message\" : \"Hello, world!\" } "); } catch (AMPSException aex) { System.err.println("TestListener caught exception."); } finally { client.close(); } } } Example 3.1. Connecting to AMPS In the preceding Example 3.1, we show the entire program; but future examples will isolate one or more specific portions of the code. The next section describes how to build and run the application and explains the code in further detail. Build and Run with a Java Command Line To build this program, you can run the following javac command, substituting the path to the amps_client.jar with the path where you have installed the AMPS Java Client: 3 Your First AMPS Program javac -classpath /opt/AMPS/api/client/java/dist/lib/amps_client.jar \ ./TestPublisher.java This will create the TestPublisher.class file. To run the class file and send your first message to AMPS, you can issue the following command: java -cp .:/opt/AMPS/api/client/java/dist/lib/amps_client.jar TestPublisher If the message is published successfully, there is no output to the console. We will demonstrate how to create a subscriber to receive messages in Chapter 4. Build and Run in Eclipse To include the JAR file in a standalone Eclipse environment: 1. Right-click the project to which you would like to add amps_client.jar and select Properties to open the Properties configuration window. 2. Click the Java Build Path item in the left menu, and then select the Libraries tab. 3. Click the Add External JARS button and choose the location of your amps_client.jar. 4. Click OK. Your Eclipse build path should now include the amps_client.jar. To run the project: 1. Select the menu option Run ▸ Run Configuration, which opens the Run Configuration window. 2. Right-click Java Application and select New from the menu. 3. Fill in the Name field (for this example, you could use “Test Publisher”). 4. Click the Search... button for the Main class field. 5. Select the TestPublisher class. 6. Finally, click the Run button to run the application. If the message is published successfully, there is no output to the console. We will demonstrate how to create a subscriber to receive messages in Chapter 4. Examining the Code Let us now revisit the code we listed earlier. 4 Your First AMPS Program import com.crankuptheamps.client.Client; import com.crankuptheamps.client.exception.AMPSException; public class TestPublisher { public static void main(String[] args) { Client client = new Client("TestPublisher-Client"); try { client.connect("tcp://127.0.0.1:9007/amps/json"); client.logon(); client.publish("messages", "{ \"message\" : \"Hello, world!\" } "); } catch (AMPSException aex) { System.err.println("TestListener caught exception."); } finally { client.close(); } } } Example 3.2. Connecting to AMPS The import statements add names into reference for convenience in typing later on in the code. These import the names from the AMPS namespaces: • com.crankuptheamps.client.Client —contains the methods for interacting with AMPS • com.crankuptheamps.client.exception.AMPSException — the package contains the ex- ception classes thrown by AMPS when errors occur. When you use AMPS in your programs, you will be using classes from these namespaces. This line creates a new Client object. Client encapsulates a single connection to an AMPS server. Methods on Client allow for connecting, disconnecting, publishing, and subscribing to an AMPS server. The argument to the Client constructor, “TestPublisher-Client”, is a name chosen by the client to identify itself to the server. Errors relating to this connection will be logged with reference to this name, and AMPS will use this name to help detect duplicate messages. AMPS enforces uniqueness for client names when a transaction log is configured, and it is good practice to always use unique client names. Here, we open a try block that concludes with catch (AMPSException aex). This statement declares a connection to AMPS with the provided URI. The URI consists of the transport, the address, and the protocol to use for the AMPS connection. In this case, the transport is tcp, the address is 127.0.0.1:9007, and the protocol is amps. This connection will be used for JSON messages. Check with the person who manages the AMPS instance to get the connection string to use for your programs. The AMPS logon() command connects to AMPS and creates a named connection. If we had provided logon credentials in the URI, the command would pass those credentials to AMPS. Without credentials, the client logs on to AMPS anonymously. AMPS versions 5.0 and later require a logon() command in the default configuration. Here, we publish a single message to AMPS on the messages topic, containing the data { "message" : "Hello, World!" }. This data is placed into an XML message and sent to the server. Upon successful completion of this function, the AMPS client has sent the message to the server, and subscribers to the messages topic will receive this “Hello world” message. All caught exceptions in AMPS derive from AMPSException. We close out the example with a finally block that closes the Client connection and releases all accompanying resources, making the connection eligible for garbage collection. 5 Your First AMPS Program 3.3. Connection Strings The AMPS clients use connection strings to determine the server, port, transport, and protocol to use to connect to AMPS. When the connection point in AMPS accepts multiple message types, the connection string also specifies the precise message type to use for this connection. Connection strings have a number of elements. Figure 3.1. elements of a connection string As shown in the figure above, connection strings have the following elements: • Transport defines the network used to send and receive messages from AMPS. In this case, the transport is tcp. For connections to transports that use the Secure Sockets Layer (SSL), use tcps. • Host address defines the destination on the network where the AMPS instance receives messages. The format of the address is dependent on the transport. For tcp and tcps, the address consists of a host name and port number. In this case, the host address is localhost:9007. • Protocol sets the format in which AMPS receives commands from the client. Most code uses the default amps protocol, which sends header information in JSON format. AMPS supports the ability to develop custom protocols as extension modules, and AMPS also supports legacy protocols for backward compatibility. • MessageType specifies the message type that this connection uses. This component of the connection string is required if the protocol accepts multiple message types and the transport is configured to accept multiple message types. If the protocol does not accept multiple message types, this component of the connection string is optional, and defaults to the message type specified in the transport. Legacy protocols such as fix, nvfix and xml only accept a single message type, and therefore do not require or accept a message type in the connection string. As an example, a connection string such as tcp://localhost:9007/amps/json would work for programs connecting from the local host to a Transport configured as follows: <AMPSConfig> ... <!-- This transport accepts any known message type for the instance: the client must specify the message type. --> <Transport> <Name>any-tcp</Name> <Type>tcp</Type> <InetAddr>9007</InetAddr> <ReuseAddr>true</ReuseAddr> <Protocol>amps</Protocol> 6