STRATEGIES IN SKILL ACQUISITION: RECONCILING CONTINUOUS MODELS OF THE LEARNING CURVE WITH ABRUPT STRATEGY SHIFTS JEROMY KYRAM ANGLIM Submitted in total fulfilment of the requirements of the degree of Doctor of Philosophy March 2011 PSYCHOLOGICAL SCIENCES THE UNIVERSITY OF MELBOURNE ii Abstract How does task completion time change with practice and what processes underlie this change? Despite over 100 years of scientific research (e.g., Bryan & Harter, 1899) no wholly satisfactory answer has yet emerged. After analysing many skill acquisition datasets Newell and Rosenbloom (1981) influentially declared that the relationship between practice and task completion time was best represented by a power function labelling the relationship the Power Law of Practice. Use of the term ‘law’ might suggest that the case was closed, yet several recent findings have chal- lenged the Power Law’s ‘legal’ status. First, Heathcote, Brown, and Mewhort (2000) concluded that the Power Law of Practice was an arte- fact of aggregation. Across a large number of skill acquisition datasets they showed that when analysed at the individual-level the exponen- tial function tended to provide superior fit. Second, several researchers have suggested that strategy shifts may even cause discontinuities in the learning curve (e.g., Delaney, Reder, Staszewski, & Ritter, 1998; Haider & Frensch, 2002; Rickard, 1997). In summary, findings suggest that the power function is an analytical artefact and that learning may in some instances involve discontinuities. In response to these challenges, this thesis had three aims. The first aim was to develop and test mathematical models of the relationship between practice, strategy use and performance. The second aim was to assesstheroleofindividualdifferences,includingpriorexperience,ability, iii iv and personality in predicting strategy use and performance. The third aim was to model the differential effects of instructed versus self-initiated strategy shift on strategy use and performance. Collectively, the aims were designed to provide a multifaceted explanation of the relationship between practice, strategy use, and performance. To achieve these aims three studies were conducted. In each study participants completed a set of trials on a text editing task. On each trial strategy sophistication and task completion time were measured. Final sample sizes in the three studies were n = 63, n = 154, and n = 154. 1 2 3 Each study also measured a selection of individual difference variables including prior experience, demographics, personality, and ability. Text editing was chosen as the criterion task because strategy use is impor- tant to task performance and strategy use could readily be measured. The text editing task was developed to enable trial-level measurement of strategy use. Strategy sophistication was operationalised as the pro- portion of key presses used that were classified as sophisticated (e.g., using control and right cursor keys to move between words) as opposed to simple (e.g., using just the right cursor to move between characters). In Studies 1 and 2 all participants received the same instructions. In Study 3 participants were randomly assigned to one of three conditions with varying instructions. In a No Training condition practice preceded withoutinterruption, inaTrainingconditionadditionalinstructionswere presented halfway through practice, and in a Control condition a filler task was presented halfway through practice. Aims 1 and 2 were assessed by Study 1 and 2 and the No Training condition of Study 3. Aim 3 was assessed by comparing the conditions in Study 3. With regard to Aim 1 results from the three studies told a consis- tent story. Results reiterated the importance of analysing data at the individual-level. While at the group-level, a three parameter power func- tion provided superior fit, at the individual-level a three parameter ex- v ponential function was significantly better in two out of three studies. Similarly, at the group-level, strategy sophistication was a continuously increasing, monotonicallydeceleratingfunctionofpractice, wellmodelled by a three parameter Michaelis–Menten function. In contrast, at the individual-level, the functional form of the relationship varied dramati- cally between individuals with a variety of often discontinuous functions providing good fit. Although abrupt strategy shifts did occur, meaningful discontinuities in the relationship between practice and task completion time were rare. Findings supported a model that explained how abrupt strategy shifts can co-occur with continuous learning curves. These findings were that: (a) strategy shifts were more likely to occur early in practice when other learning was occurring; (b) trial-to-trial variance in task completion time was often large relative to the benefits of the strategy shift; and (c) strategy shifts often took several trials to be fully realised. These and other factors combined to generally smooth out the discontinuous effects of strategy shift on performance. In relation to the second aim, concerning individual differences, abil- ity and prior experience consistently emerged as moderate to strong pre- dictors of task performance, whereas self-reported Big 5 personality was unrelatedtotaskperformance. Similarbutgenerallyweakerrelationships were found between individual differences and strategy sophistication. A model that proposed that the effect of ability and prior experience on task performance was mediated by strategy sophistication was not sup- ported. Findings were broadly consistent with cognitive correlates and skill transfer models of individual differences. In relation to the third aim, looking at differences between instructed strategy shift and self-initiated strategy shift, hypotheses were partially supported. In summary, relative to self-initiated strategy shifts, in- structed strategy shifts were more abrupt. Performance also tended to vi decline sharply immediately following the instructed strategy shift. After additional practice, performance was similar to groups that had not re- ceivedinstructedstrategyshift. Thestudyhighlightedhowthedynamics of instructed strategy shift differ from self-initiated strategy shift with regards to discontinuities. Taken together the results tell an interconnected story regarding the relationshipbetweenpractice, strategyuse, andperformance. Thisthesis contributes to skill acquisition research through a unique combination of features including trial-level measurement of strategy use, individual- level modelling, and the use of nonlinear and discontinuous functions. It is hoped that future research will build on this approach using other samples, tasks, and contexts. vii Declaration I, Jeromy Anglim, certify that (i) thisthesiscomprisesonlymyoriginalworktowardsthePhDexcept where indicated in the Preface*, (ii) due acknowledgement has been made in the text to all other ma- terial used, (iii) this thesis is less than 100,000 words in length, exclusive of tables, maps, bibliographies and appendices. Signature: Date: viii Preface The dataset presented as Study 1 of this thesis comes from a collabo- ration between Niloufar Mahdavi, Janice Langan-Fox (my own principal supervisor), and myself. The study formed part of Niloufar Mahdavi’s honours thesis who was supervised by Janice Langan-Fox. The overall design in terms of measurement of abilities and use of a criterion text editing task was based on a series of studies that Janice Langan-Fox had completed with various research students in the preceding years. Nilou- far Mahdavi’s use of the dataset was distinct from my use. I was involved in some design decisions, most notably the measurement of strategy use, the choice of text editing keys, and the passage of text to be edited. I programmed the experimental task, designing how strategy use and performance data were recorded and setting out the algorithms by which strategywasextractedfromkeylogs. Allanalysespresentedinthisthesis were conducted from raw data and represent my own work. Initial analyses of Study 1 were presented in the following conference proceedings • Anglim, J., Langan-Fox, J., & Mahdavi, N. (2005). Modeling the RelationshipbetweenStrategies, AbilitiesandSkilledPerformance. CogSci 2005, 27th Annual Meeting of the Cognitive Science Society, July 21–23 Stresa, Italy. ix Acknowledgements I would like to thank: • Janice Langan-Fox, my principal supervisor, for inspiring me to pursue a PhD, keeping me on track, setting high standards, and guiding me during the critical moments. • Alex Wearing for his wisdom, support, and encouragement. • Yoshi Kashima for his collegiality, suggestions, and encouragement. • Richard Bell for his helpful suggestions and encouragement. • Niloufar Mahdavi for kindly and professionally collaborating with me. • Philip Smith, Paul Dudgeon, Murray Aitkin, and Garry Robins for their instruction and support. • Phillip Ackerman and Ruth Kanfer for sharing some of their raw data with me. Although this data was not incorporated into the final thesis, it helped me develop ideas about modelling learning curves at the individual-level. • Richard Moulding for reading an early draft and providing useful suggestions. • The many individuals who developed the various open source tools—R, LaTeX, Sweave, git, make, bibtex, and more—used in the creation of this thesis. • The participants for giving up their time to be involved in my research. • Corrine for her love and support, the way that she listened, her gentle encouragement, and her practical advice. x • Jasmine for her love and support over the course of my life, helping me get over the hurdles and continue my journey in psychology, and reminding me from time to time of John C. King’s prophetic words of encouragement.