University of Leicester Department of Engineering Statistical Arbitrage: Opportunity Spotting for Financial gain in Financial Markets Thesis submitted for the degree of M.Phil. by John Holme Supervised by Dr. F. S. Schlindwein and Prof. N. B. Jones June 2010 Statistical Arbitrage: Opportunity Spotting for Financial gain in Financial Markets John Holme Abstract The project sought to identify anomalies in the price-time relationship of historically highly correlated company stocks, and to exploit these anomalies by trading both sets of stocks in a manner so as to yield a profit independently from financial market movement. The stock positions taken upon each opportunity are those of a zero investment strategy (i.e. the same value of one stock is bought as is sold in another stock – with a net of zero outlay). The idea being that the bought stock rises and/or the sold stock falls. Either way makes money. The aim of the work was to engineer this Statistical Arbitrage system, which spots real-time opportunities, and capitalize upon the event for profit. The application has indeed been engineered, and to this end this aspect part of the work has been realized. While significant annualised percentage gains of between 6.0% and 44.1% have been achieved in later simulations, this could have be due to factors present in the market at the time and/or as yet unconsidered influences. Poor or inconsistent performance in falling and level market conditions leave, at least myself, unwilling to invest in the strategy, without more work being undertaken. While the overall outcome of this work does not bode well for a totally infallible alchemist dream, I still believe that somewhere in this method is a holy grail, and would urge other individuals to complement this work if at all possible. Page 2 of 191 Declaration of Originality It is hereby certified that this thesis is the author‘s original work, except where otherwise stated. References are given stating sources where applicable. The thesis, wholly or partly, has not been submitted for any other degree either to the University of Leicester or any other Institution of Education. John Holme 4th June 2010 Page 3 of 191 Acknowledgements I would like to thank the following people to whom I owe a great debt in the production of this thesis Prof N B Jones My original supervisor Dr Schlindwein My supervisor Prof Mike Warrington & Dr Chris Thomas For getting me into the position to do this work Dr Capildeo For improving the quality of life I enjoy now Dr Sharif For saving my life during this work Mrs Blanca Cecilia Rodriguez Martinez de Holme My wife Marisol, JC, Dan & Tash My children Page 4 of 191 Table of Contents CHAPTER 1 FOREWARD: Big Bang ................................................ 12 CHAPTER 2 BACKGROUND ........................................................... 14 2.1 Share trading in general and terms frequently used ....................... 14 2.2 Day trading, Strategies and Program Trading ................................. 19 2.3 Objectives ...................................................................................... 22 2.4 Financial Terminology ................................................................... 26 2.5 What Is Pairs Trading? .................................................................. 29 2.5.1 Evidence of Profitability ........................................................... 30 2.6 An Example Using Futures Contracts ............................................ 32 2.7 An Example Using Options ............................................................ 32 2.7.1 SHORTING A STOCK ............................................................... 33 CHAPTER 3 RELATED WORK ......................................................... 38 3.1 Further Background ...................................................................... 38 3.2 Market Neutral .............................................................................. 39 3.3 Interesting for the reader ............................................................... 54 3.4 Technical Analysis ......................................................................... 56 3.4.1 Hedge funds ............................................................................ 62 3.5 When things go wrong ................................................................... 63 3.5.1 Long Term Cap ........................................................................ 64 3.5.2 Tiger Management ................................................................... 64 3.5.3 A risky strategy ........................................................................ 65 3.5.4 Bailey Coates Cromwell Fund .................................................. 66 3.5.5 Marin Capital .......................................................................... 67 3.5.6 Worldcom ................................................................................ 68 3.5.7 The Credit Crunch: No quick end in sight ................................ 68 CHAPTER 4 TECHNOLOGY AND DESIGN ...................................... 70 4.1 ANALYSIS ...................................................................................... 70 4.2 Requirements ................................................................................ 70 4.3 Design Details based upon Requirements ...................................... 72 4.3.1 Detailed List of Functional Requirements ................................. 73 4.3.2 Database Design ...................................................................... 73 4.3.3 The Process (functional) matrix ................................................ 77 4.3.4 Process diagram ...................................................................... 77 4.3.5 Class diagram .......................................................................... 83 4.4 Technology .................................................................................... 84 4.5 Creation of a Simulation ................................................................ 86 4.5.1 A trading simulation ................................................................ 86 4.6 Data .............................................................................................. 86 Page 5 of 191 CHAPTER 5 EXPERIMENTS & RESULTS ........................................ 88 5.1 Introduction .................................................................................. 88 5.2 Proof of the code and concept ........................................................ 89 5.2.1 Testing .................................................................................... 90 5.2.2 Simulation Results .................................................................. 93 5.3 Parameters .................................................................................... 94 5.4 Simulation over historical periods of time and search for the holy grail .............................................................................................. 101 5.4.1 List of experiments ................................................................ 102 5.4.2 RISING Market ...................................................................... 103 5.4.3 FALLING Market .................................................................... 109 5.4.4 LEVEL Market ....................................................................... 115 5.5 Performance in the real time market ............................................ 121 5.6 Real Time .................................................................................... 122 5.6.1 Data ...................................................................................... 122 5.6.2 Number of trades ................................................................... 123 5.6.3 Final values vs ValueOnDay .................................................. 125 5.6.4 Breakdown ............................................................................ 128 5.6.5 Profit per trade ...................................................................... 128 5.6.6 Reports .................................................................................. 129 5.7 Pseudo Real Time ........................................................................ 137 5.7.1 Data ...................................................................................... 137 5.7.2 Number of trades ................................................................... 137 5.7.3 Pseudo Real Time Replay ....................................................... 138 5.7.4 Profit per trade ...................................................................... 139 5.8 Live Pseudo Replay ...................................................................... 141 5.8.1 Number of trades ................................................................... 142 5.8.2 Varying simulation start date and code trigger parameters ..... 142 5.8.3 Per trade breakdown .............................................................. 145 5.8.4 Profit per trade ...................................................................... 147 5.8.5 Annualised Profit ................................................................... 149 CHAPTER 6 FURTHER WORK ...................................................... 151 6.1 Rising, falling and level simulations ............................................. 151 6.2 Real time trading ......................................................................... 151 6.3 Parameter wrapping .................................................................... 152 6.4 Changing regression values ......................................................... 153 6.5 Profit and Loss ............................................................................ 154 CHAPTER 7 CONCLUSIONS ......................................................... 156 7.1 Rising, Falling and Level Market Movement Simulations .............. 156 7.2 Testing ........................................................................................ 156 7.3 Performance in a Real time scenario ............................................ 157 7.4 Trading Reports ........................................................................... 158 7.5 General feeling ............................................................................ 158 APPENDIX 1. Proof of code ............................................................. 160 Page 6 of 191 7.6 Testing and Setup ....................................................................... 160 7.6.1 Regression Data ..................................................................... 162 7.7 Simulation 1................................................................................ 164 7.7.1 Simulation data ..................................................................... 164 7.7.2 Final day Reports ................................................................... 166 7.8 Simulation 2................................................................................ 167 7.8.1 Simulation data ..................................................................... 167 7.9 Final day Reports ........................................................................ 169 7.10 Simulation 3................................................................................ 171 7.10.1 Simulation data ..................................................................... 171 7.10.2 Final day Reports ................................................................... 173 APPENDIX 2. Database .................................................................. 175 7.11 General Relationships described .................................................. 175 7.12 Entity Model ................................................................................ 176 7.13 Main tables ................................................................................. 177 7.13.1 Table: GEN_TRD_basis_HIST ................................................. 177 7.13.2 Table: SuggestedPositionArchive ............................................ 178 7.13.3 * Table: WTB_exclude_pairs */ ............................. 179 7.13.4 * Table: WTB_Params */ ................................... 179 7.13.5 * Table: WTBCorpActionsExceptions */ .................... 179 7.13.6 * Table: WTBSuspectedCorpActions */ .................... 179 7.13.7 * Table: NASDAQ_company */ ............................. 180 7.13.8 * Table: NASDAQ_equity_price */ .......................... 180 7.14 Create table and views ................................................................. 180 7.15 Table last_load ............................................................................ 180 7.16 Full Database List ....................................................................... 180 Bibliography .................................................................................... 181 Page 7 of 191 Table of Tables Table 1 Requirements of the opportunity spotting application ............ 73 Table 2: The trading parameters table (TRD_Params) ......................... 95 Table 3: trading parameter field meanings .......................................... 96 Table 4: Analysis periods ................................................................... 97 Table 5: Trading Parameters ............................................................ 100 Table 6: Simulation period summary ................................................ 103 Table 7: Rising data excerpt ............................................................. 104 Table 8: Rising Period profit & loss ................................................... 107 Table 9: Falling data extract ............................................................. 110 Table 10: Falling tabulated results ................................................... 112 Table 11: Level Simulation Results ................................................... 116 Table 12: Level Simulation Data ....................................................... 119 Table 13: Live data (first trades) ....................................................... 122 Table 14: Live data (final trades) ....................................................... 122 Table 15: Live trade cumulative count .............................................. 124 Table 16: Live profit per trade .......................................................... 126 Table 17: Live breakdown of pairs .................................................... 128 Table 18: Live Pseudo cumulative number of trades ......................... 137 Table 19: Live Pseudo final values and value of day of positions ....... 138 Table 20: Live Pseudo profit per trade .............................................. 139 Table 21: Live pseudo replay simulation data extract ....................... 141 Table 22: Live Pseudo Replay number of trades ................................ 142 Table 23: Comprehensive assessment .............................................. 143 Table 24: Profit per trade ................................................................. 147 Page 8 of 191 Table of figures Figure 1: Investor, broker, exchange relationship ............................... 16 Figure 2: Perfect Share Price Correlation ............................................ 23 Figure 3: Highly correlated share prices ............................................. 24 Figure 4: Main database tables .......................................................... 74 Figure 5: Database Design ................................................................. 76 Figure 6: Functional matrix ............................................................... 77 Figure 7: High level process flows ....................................................... 78 Figure 8: Low level process flows ........................................................ 79 Figure 9: Class Diagram ..................................................................... 83 Figure 10: Market movements .......................................................... 102 Figure 11 : Meanings/Definition of the columns in the simulation results ............................................................................................. 103 Figure 12: RISING Market graphic ................................................... 106 Figure 13: Meanings/Definition of the columns in the simulation results ............................................................................................. 109 Figure 14: FALLING Market graphic ................................................. 111 Figure 15: Meanings/Definition of the columns in the simulation results ............................................................................................. 115 Figure 16: LEVEL Market graphic .................................................... 118 Figure 17: Live Final amount and valueOnDay ................................. 127 Figure 18: Live breakdown of pairs ................................................... 128 Figure 19: Live profit per trade ......................................................... 129 Figure 20: Live Open position suggestion GOOG-NTAP ..................... 130 Figure 21: Live on-going position GOOG-NTAP ................................. 131 Page 9 of 191 Figure 22: Live Suggested closure GOOG-NTAP ................................ 132 Figure 23: A view of the relative performance of the stock and any unusual trading volumes in a 1 Year period ..................................... 133 Figure 24: A view of the relative performance of the stock and any unusual trading volumes in a 3 Month period .................................. 133 Figure 25: A view of the relative performance of the stock and any unusual trading volumes in a 3 month period .................................. 134 Figure 26: 3 Months Stock Graph of first party in the pair to show unusual volume or price movement ................................................. 134 Figure 27: 3 Months Stock Graph of second party in the pair to show unusual volume or price movement ................................................. 135 Figure 28: News from each component of the pair ............................ 136 Figure 29: Live Pseudo final values and value of day of positions ...... 139 Figure 30: Live Pseudo profit per trade ............................................. 140 Figure 31: Comprehensive assessment ............................................. 144 Figure 32: Per trade breakdown ....................................................... 146 Figure 33: Profit per trade ................................................................ 148 Figure 34: Annualised profit ............................................................. 149 Figure 35: Historical price graphs for simulated stocks .................... 161 Figure 36: Simulation data regression values on the initial day ........ 163 Figure 37: Simulation 1 - Continuation of trend data ....................... 165 Figure 38: Simulation 1 - Final day Report ....................................... 166 Figure 39: Simulation 2 - Days price files ......................................... 168 Figure 40: Simulation 2 - Final day report ........................................ 170 Figure 41: Simulation 3 - Days price files ......................................... 172 Page 10 of 191