AMPS Configuration Reference Guide AMPS Configuration Reference Guide 5.0 Publication date Apr 01, 2016 Copyright © 2016 All rights reserved. 60East, AMPS, and Advanced Message Processing System are trademarks of 60East Technologies, Inc. All other trademarks are the property of their respective owners. Table of Contents 1. AMPS Configuration Basics ........................................................................................................... 1 1.1. AMPS Configuration File Special Characters .......................................................................... 2 1.2. Using Units in the Configuration .......................................................................................... 5 1.3. Environment Variables in AMPS Configuration ....................................................................... 6 2. Generating a Configuration File ...................................................................................................... 8 3. Instance Level Configuration .......................................................................................................... 9 3.1. SOW Statistics Interval ...................................................................................................... 10 3.2. Slow Client Policies .......................................................................................................... 11 3.3. Minidump Directory .......................................................................................................... 12 3.4. Configuration Validation .................................................................................................... 12 3.5. Tuning ............................................................................................................................ 12 4. Admin Server ............................................................................................................................. 14 5. Modules .................................................................................................................................... 15 6. Message Types ........................................................................................................................... 18 7. Transports .................................................................................................................................. 22 8. Logging ..................................................................................................................................... 27 9. State-of-the-World (SOW) Features ................................................................................................ 29 9.1. SOW/Topic ...................................................................................................................... 29 9.2. SOW/Queue ..................................................................................................................... 33 9.3. SOW/View ...................................................................................................................... 37 9.4. SOW/ConflatedTopic ......................................................................................................... 38 10. Replication Destination ............................................................................................................... 40 11. Transaction Log ........................................................................................................................ 44 12. Authentication ........................................................................................................................... 46 13. Entitlement ............................................................................................................................... 48 14. Actions .................................................................................................................................... 49 14.1. Running an Action on a Schedule ...................................................................................... 49 14.2. Running an Action in Response to a Signal ......................................................................... 49 14.3. Running an Action on Startup or Shutdown ......................................................................... 50 14.4. Running an Action on Client Connection ............................................................................ 51 14.5. Running an Action on Message Delivery ............................................................................ 51 14.6. Running an Action on Message Publish .............................................................................. 52 14.7. Running an Action on OOF Message ................................................................................. 53 14.8. Running an Action on Minidump ....................................................................................... 53 14.9. Running an Action on Offline Start or Stop ......................................................................... 54 14.10. Rotate Log Files ........................................................................................................... 54 14.11. Manage the Statistics Database ........................................................................................ 55 14.12. Manage Journal Files ..................................................................................................... 55 14.13. Removing Files ............................................................................................................. 56 14.14. Deleting Messages from SOW ......................................................................................... 57 14.15. Querying a SOW Topic .................................................................................................. 58 14.16. Manage Security ........................................................................................................... 59 14.17. Enable and Disable a Transport ....................................................................................... 59 14.18. Publishing Messages ...................................................................................................... 60 14.19. Manage Replication ....................................................................................................... 60 14.20. Extract Values ............................................................................................................... 61 14.21. Increment Counter ......................................................................................................... 62 14.22. Executing System Commands .......................................................................................... 62 14.23. Debugging AMPS ......................................................................................................... 63 14.24. Creating a Minidump ..................................................................................................... 63 14.25. Shut Down AMPS ......................................................................................................... 64 iii AMPS Configuration Reference Guide 14.26. Action Configuration Examples ....................................................................................... 64 A. Obsolete Configuration Parameters ................................................................................................ 67 Index ............................................................................................................................................ 68 iv Chapter 1. AMPS Configuration Basics If you have not become familiar with the AMPS User Guide, in particular the Getting Started chapter, please start there before reading this guide. The easiest way to create a custom XML configuration file for AMPS is to start with the sample configuration file produced by the --sample-config flag to AMPS. Example 1.1 shows a simplified sample configuration file. <?xml version="1.0" encoding="UTF-8"?> <!-- Sample AMPS configuration This file defines an AMPS instance that provides publish and subscribe, topic filtering, and content filtering for JSON messages. The instance provides messaging services on port 9007 of the server. This configuration also provides an adminstrative interface on port 8085, and logs serious messages (error and higher severity) to stdout. This sample file does not configure State of the World (SOW) Topics, Transaction Logs, Aggregation and Views, Historical Query, Replication, Authentication and Entitlement, Conflating Topic Replicas, or other features of AMPS. More details for the features available and how to configure them are provided in the AMPS User Guide and the AMPS Configuration Reference. Both are available at http://crankuptheamps.com/documentation/ --> <AMPSConfig> <!-- Name of the AMPS instance --> <Name>AMPS-Sample</Name> <!-- Configure the administrative HTTP server on port 8085 This HTTP server provides admin functions and statistics for the instance --> <Admin> <InetAddr>localhost:8085</InetAddr> </Admin> <!-- Configure a transport for JSON messages over TCP on port 9007 --> <Transports> <Transport> <Name>json-tcp</Name> <Type>tcp</Type> 1 AMPS Configuration Basics <InetAddr>9007</InetAddr> <MessageType>json</MessageType> <Protocol>amps</Protocol> </Transport> </Transports> <!-- Log messages of severity 'error' and higher to stdout --> <Logging> <Target> <Protocol>stdout</Protocol> <Level>error</Level> </Target> </Logging> </AMPSConfig> Example 1.1. Simple AMPS configuration file The AMPS configuration XML file is defined first by wrapping the config file with an AMPSConfig tag to identify it as a configuration file. Next, the instance is given a name using the <Name> tag. Once our instance has a name, it is good to define the connection target for the administration port. By default, the administration port can be found by pointing a browser to http://localhost:8085, but if a different port or host name is desired, then that is defined in the Admin and InetAddr tags. The Admin port is discussed more in Table 4.1. Next we describe how to get messages into AMPS. There are several different transport types which can be parsed by AMPS, all of which are discussed in greater detail in the Transports chapter, but for this sample, we keep things simple by focusing on JSON messages over tcp. In AMPS, each key to defining each transport is to give them a unique InetAddr port and specify the type of message AMPS will process on that port using MessageType tag. The MessageType tells AMPS how to parse the incoming messages on a specific port. In the above example, messages are coming in on port 9007, and AMPS uses the JSON parser to parse the body of the message. AMPS also requires a Protocol tag for the Transport, which specifies the format of the commands to AMPS. In this case, we use the standard amps protocol. (Older AMPS applications and AMPS installations may require a different protocol format, such as fix or xml. There's no functional difference between these protocols, but the AMPS server and the clients need to use the same protocol format to successfully exchange messages.) The last portion of the configuration is Logging. In the above example, the Logging tag defines only one log target, but it's quite common to have one or more Logging targets. Again referring to the example, all logging messages that are at error level and above will be logged to the logging Protocol of stdout. In other words, these messages will be logged to the terminal, and not to a file. AMPS supports a robust set of logging features and configurations, all of which are covered in more detail in the Logging chapter in the AMPS User Guide and in Chapter 8 of this reference. 1.1. AMPS Configuration File Special Characters In AMPS there are a few special characters that you should be aware of when creating your configuration file. These characters can provide some handy short cuts and make configuration creation easier, but you should also be aware of them so as not to introduce errors. 2 AMPS Configuration Basics State of the World File Name When specifying the file for a State of the World database, using the %n string in the file name specifies that the AMPS server will use the message type and topic name in that position to create a unique filename. Example 1.2 shows how to use this in the AMPS configuration file. <SOW> <Topic> <Topic>Customers</Topic> <FileName>./sow/%n.sow</FileName> <MessageType>json</MessageType> <Key>/customerId</Key> </Topic> </SOW> Example 1.2. SOW file name tokens used in configuration file Log Rotation Name When specifying an AMPS log file which has RotationThreshold specified, using the %n string in the log file name is a useful mechanism for ensuring the name of the log file is unique and sequential. Example 1.3 shows a file name token replacement in the AMPS configuration file. <Logging> <Target> <Protocol>file</Protocol> <Level>info</Level> <FileName>log/log-%n.log</FileName> <RotationThreshold>2G</RotationThreshold> </Target> </Logging> Example 1.3. Log file name tokens used in configuration file In the above example, a log file will be created in the AMPSDIR/log/ directory. The first time this file is created, it will be named log-1.log. Once the log file reaches the RotationThreshold limit of 2G, the previous log file will be saved, and the new log file name will be incremented by one. Thus, the next log file will be named AMPSDIR/log/log-2.log. Dates AMPS allows administrators to use date-based file names when specifying the file name in the configuration, as demonstrated in Example 1.4. <Logging> <Target> 3 AMPS Configuration Basics <Protocol>file</Protocol> <Level>info</Level> <FileName> log/log-%Y-%m-%dT%H%M%S.log </FileName> <RotationThreshold>2G</RotationThreshold> </Target> </Logging> Example 1.4. Date tokens used in configuration file In the above example, a log file will be created in the $AMPSDIR/log named 2011-01-01-120000.log if the log was created at noon on January 1, 2011. AMPS provides full support for the date tokens provided by the standard strftime function, with the exception of %n, as described above. The following table shows some of the most commonly used tokens: Table 1.1. Commonly Used Date and Time Tokens Token Provides Example %a Short weekday name Fri %A Full weekday name Friday %b Short month name Feb %B Full month name February %c Simple date and time Fri Feb 14 17:25:00 2014 %C Century 20 %d Day of the month (leading zero if necessary) 05 %D Short date format (MM/DD/YY) 02/20/14 %e Day of the month (leading space if necessary) 5 %F Short date format (YYYY-MM-DD) 2014-02-20 %H Hour (00-23) 17 %I Hour (00-12) 05 %j Day of the year (001-366) 051 %m Month (01-12) 02 %p AM or PM PM %r Current time, 12 hour format 05:25:00 pm %R Current time, 24 hour format 17:25 %T ISO 8601 Time format 17:25:00 %u ISO 8601 day of the week (1-7, Monday = 1) 5 %V ISO 8601 week number (00-53) 07 %y Year, last two digits 14 %Y Year, four digits 2014 %Z Timezone name or abbreviation (blank if undetermined) PST 4 AMPS Configuration Basics 1.2. Using Units in the Configuration To make configuration easy, AMPS permits the use of units to expand values. For example, if a time interval is measured in seconds, then the letter s can be appended to the value. For example, the following SOW topic definition used the Expiration tag to set the record expiration to 86400 seconds (one day). <SOW> <Topic> ... <Expiration>86400s </Expiration> ... </Topic> </SOW> Example 1.5. Expiration Using Seconds An even easier way to specify an expiration of one day is to use the following Expiration: <SOW> <Topic> ... <Expiration>1d</Expiration> ... </Topic> </SOW> Example 1.6. Expiration Using Days Table 1.2 shows a listing of the time units AMPS supports in the configuration file. Table 1.2. AMPS Configuration - Time Units Units Description ns nanoseconds us microseconds ms milliseconds s seconds m minutes h hours d days w weeks AMPS configuration supports a similar mechanism for byte-based units when specifying sizes in the configuration file. Table 1.3 shows a listing of the byte units AMPS supports in the configuration file. Table 1.3. AMPS Configuration - Byte Units Units Description kb kilobytes mb megabytes gb gigabytes 5 AMPS Configuration Basics Units Description tb terabytes Dealing with large numbers in AMPS configuration can also be simplified by using common exponent values to handle raw values. This means that instead of having to input 10000000 to represent ten million, a user can input 10M. Table 1.4 contains a list of the exponents supported. Table 1.4. AMPS Configuration - Numeric Units Units Description k 103 - thousand M 106 - million To make it easier for users to remember the units, AMPS interval and byte units are not case sensitive. 1.3. Environment Variables in AMPS Configura- tion AMPS configuration also allows for environment variables to be used as part of the data when specifying a config- uration file. If a global system variable is commonly used in an organization, then it may be useful to define this in one location and re-use it across multiple AMPS installations or applications. AMPS will replace any token wrapped in ${} with the environment variable defined in the current user operating system environment. Example 1.7 demonstrates how the environment variable ENV_LOG is used to define a global environment variable for the location of the host logging. <Logging> <Target> <Protocol>file</Protocol> <FileName>${ENV_LOG}</FileName> <Level>info</Level> <RotationThreshold>2G</RotationThreshold> </Target> </Logging> Example 1.7. Environment Variable Used in Configuration Internal Environment Variables In addition to supporting custom environment variables, AMPS includes a configuration variable, AMPS_CONFIG_DIRECTORY, which can be used to reference the directory in which the configuration file used to start AMPS is located. For example, assume that AMPS was started with the following command at the command prompt: 6