International Phytoplankton Intercomparison proficiency test in the abundance and composition of marine microalgae 2016 report PHY-ICN-16-MI1 VR 1.0 Rafael Salas1 & Jacob Larsen2 1 Marine Institute, Rinville, Oranmore, Co.Galway, Ireland 2 IOC Science and Communication center on harmful algae Department of Biology, University of Copenhagen, Øster Farimagsgade 2D 1353 Copenhagen K. Denmark 1 Table of Contents: 1. Summary of results Pages 4-5 2. Introduction Pages 5-7 3. Materials and Methods Pages 7-11 3.1 Sample preparation, homogenisation and spiking Pages 7-8 3.2 Culture material, treatments and replicates Page 8-9 3.3 Cell concentration Page 9 3.4 Sample randomization Page 9 3.5 Forms and instructions Pages 9 3.6 Statistical analysis Page 9-10 3.7 IPI Ocean teacher online HAB quiz Pages 10-11 4. Results Pages 11-25 4.1 Homogeneity and stability study Pages 11-12 4.2 Outliers and missing values Pages 12-13 4.3 Analysts’ data Pages 13-14 4.4 Assigned value and its standard uncertainty Page 14 4.5 Comparison of the assigned value Page 15 4.6 Calculation of performance statistics Page 16 4.6.1 Z-scores Page 16 4.7 Combined performance statistics Pages 17 4.7.1 RLP and RSZ Page 17 4.7.2 Plots of repeatability standard deviation Page 17 4.8 Qualitative data pages 17-18 4.9 Ocean teacher online HAB quiz Pages 18-25 5. Discussion Pages 26-32 2 Annex I: Form 1: Return slip and checklist Page 33 Annex II: Form 2: Enumeration and identification results log sheet Page 34 Annex III: Test Instructions Pages 35-45 Annex IV: Workshop Agenda Page 46-47 Annex V: Participating laboratories Page 48 Annex VI: Statement of performance certificate Page 49-50 Annex VII: Homogeneity and stability test Pages 51-66 Annex VIII: Analysts results Pages 67-70 Annex IX: Robust mean + SD iteration ISO13528 pages 71-78 Annex X: Summary of Z-scores for all measurands Pages 79-82 Annex XI Performance statistics for the test Page 83 Annex XII: Summary of statistical parameters and laboratory means pages 84-87 Annex XIII: Graphical summary of results Pages: 88-95 Annex XIV: Mandel’s h and k statistics Pages 96-97 Annex XV: RLP + RSZ for all measurands Page 98 Annex XVI: Charts of repeatability standard deviations Pages 99-106 Annex XVII: Ocean teacher online HAB quiz Pages: 107-122 3 Annex XVIII: HABs Ocean teacher analyst results Pages 123-126 1. Summary of results • 82 analysts from 43 laboratories took part in this intercomparison exercise. 81 analysts returned sample results and 79 completed the online Hab quiz. There were 69 participants from laboratories across Europe, 5 from South America, 2 in Australia, 1 in New Zealand and 5 in Africa. • Ten species were used in this test. These were the dinoflagellates Alexandrium ostenfeldii (Paulsen) Balech & Tangen, Prorocentrum triestinum J.Schiller, Karenia selliformis A.J.Haywood, K.A.Steidinger & L.MacKenzie, Karlodinium veneficum (D.Ballantine) J.Larsen, Dinophysis acuta Ehrenberg and the diatoms Pseudo-nitzschia australis Frenguelli, Guinardia delicatula (Cleve) Hasle, Chaetoceros didymus Ehrenberg, Coscinodiscus wailesii Gran & Angst and Thalassiosira gravida Cleve. • The cell counts of the species Karlodinium veneficum which did not past the minimum requirements for homogenization and stability were discounted for statistical purposes and also Karenia selliformis which did not preserve well in the samples was not used here. All the other species counts were used. • The average and confidence limit for each test item was calculated using the robust algorithm in annex C of ISO13528 which takes into account the heterogeneity of the samples and the between samples standard deviation from the homogeneity and stability test. ISO 13528 is only valid for quantitative data. We have used the consensus values from the participants. • All measurands passed the F-test except for K.veneficum. Only A.ostenfeldii passed the homogeneity test according to ISO13528 but they all passed the expanded criterion except for K.veneficum . The stability test was passed by 5 out of the 9 measurands but failed K.veneficum, D.acuta, T.gravida and P.australis. All measurands passed the stability test according to the expanded 13528:2015 except for K.veneficum. • The consensus values new Standard deviation (STD) was used for all measurands regardless of the Pass/Fail flags from the homogeneity test. • There were a small number of action signals across all measurands. 9 Red flags in total (1.4% of results), 22 (3.4%) yellow flags and 6 (0.93%) orange flags (Non-Ids) from 648 scores is evidence of good performance overall. Eight analysts did not pass the full test with a below 80% score. There is evidence of method bias on low cell density measurands due to the volume analysed. 4 • The Ocean teacher online HAB quiz results suggests a high rate of proficiency. 68% of analysts achieved a score over 90% (Proficient). Another 21.5% of analysts above 80%, 8% between 70 and 80% and 2.5% needs improvement. • There was good consensus on the various identifications of diatom species from images in questions 1 to 3. Although the images of T.mobiliensis and C.densus were the most difficult organisms to identify from these images, results suggest a good performance overall. In Questions 4 to 6, there were good overall marks on flagellate identification based on depictions. Q7-9 Good scores on Peridinioid terminology but difficulties with the lesser known Suessiaceae group. Q10-12 Problems identifying T.macroceros group (Q10) worst score(68.8% correct). Q12-15 Theory based on 1’ and 2a plate for identification of Protoperidinium is understood but difficult to execute using images. 2. Introduction The Intenational Phytoplankton Intercomparison or IPI (formerly known as Bequalm) study in 2016 was designed to test the ability of analysts to identify and enumerate correctly marine phytoplankton species in lugol’s preserved water samples. As in previous years, samples have been spiked using laboratory cultures. Initially, there were ten species of interest in this intercomparison exercise. These were; the dinoflagellates Alexandrium ostenfeldii (Paulsen) Balech & Tangen, Prorocentrum triestinum J.Schiller, Karenia selliformis A.J.Haywood, K.A.Steidinger & L.MacKenzie, Karlodinium veneficum (D.Ballantine) J.Larsen, Dinophysis acuta Ehrenberg and the diatoms Pseudo-nitzschia australis Frenguelli, Guinardia delicatula (Cleve) Hasle, Chaetoceros didymus Ehrenberg, Coscinodiscus wailesii Gran & Angst and Thalassiosira gravida Cleve. The collaboration between the Marine Institute in Ireland and the IOC UNESCO Centre for Science and Communication of Harmful algae in Denmark on the IPI exercise commenced in 2011. This collaboration involves the use of algal cultures from the Scandinavian Culture Collection of Algae and Protozoa in Copenhagen, the elaboration of a marine phytoplankton taxonomy quiz using the online platform ‘Ocean Teacher’ Global academy hosted by the IODE (International Oceanographic Data and information Exchange) office based in Oostende, Belgium, a project office of the IOC and the organization of a training workshop which is held annually to discuss the results of the intercomparison exercise and to provide training on phytoplankton taxonomy. This workshop has become an important forum for phytoplankton taxonomists working on phytoplankton monitoring programmes from around the world to convene and be able to discuss taxonomical matters 5 related to monitoring, new advances and finds, taxonomical nomenclature changes, looking at samples from different geographical areas and listen to relevant stories from other laboratories about harmful algal events in their regions of relevant ecological importance. This workshop has been held in various locations in previous years but over the last 3 years, it has taken the format of a 3 days training workshop with at least 2 days dedicated to lectures on algal groups in rooms equipped with microscopes and using live cultures and preserved samples from participants and from locations across the globe (See Workshop agenda: Annex IV). This year, 82 analysts from 43 laboratories took part in this intercomparison exercise. 81 analysts returned sample results and 79 completed the online Hab quiz. There were 69 participants from laboratories across Europe, 5 from South America, 2 in Australia, 1 in New Zealand and 5 in Africa. The list of participating laboratories can be found in Annex V and a breakdown of participation from each country in figure 1 below. Figure 1: Breakdown participation per country of the Phytoplankton intercomparison exercise IPI 2016 This intercomparison exercise has been coded in accordance with defined protocols in the Marine Institute, for the purposes of quality traceability and auditing. The code assigned to the current study is PHY-ICN-16- MI1. PHY standing for phytoplankton, ICN for intercomparison, 16 refers to the year 2016, MI refers to the Marine Institute and 1 is a sequential number of intercomparisons for the year. So, 1 indicates the first intercomparison for the year 2016. 6 AAAsss fffiiiggguuurrreee 222 iiinnndddiiicccaaattteeesss ttthhheee nnnuuummmbbbeeerrr ooofff IIIPPPIII pppaaarrrtttiiiccciiipppaaannntttsss hhhaaasss iiinnncccrrreeeaaassseeeddd aaapppppprrreeeccciiiaaabbblllyyy sssiiinnnccceee 222000000555 aaannnddd ttthhheee iiinnnfffllluuueeennnccceee ooofff ttthhheee ttteeesssttt hhhaaasss aaalllsssooo bbbeeeeeennn wwwiiidddeeennneeeddd tttooo aaallllll cccooonnntttiiinnneeennntttsss... IIInnn ttthhheee lllaaasssttt tttwwwooo yyyeeeaaarrrsss ttthhheee nnnuuummmbbbeeerrr ooofff participants have ppllaatteeaauu oouutt iinn aanndd aarroouunndd tthhee 8800 pplluuss mmaarrkk aanndd wwhhiillee tthhee mmaajjoorriittyy ooff llaabboorraattoorriieess come from European countries (84%), a sizeable 1166%% iiss mmaaddee uupp ffrroomm llaabboorraattoorriieess iinn AAffrriiccaa,, SSoouutthh AAmmeerriiccaa aanndd OOcceeaanniiaa.. FFiigguurree 22:: IIPPII ppaarrttiicciippaattiioonn iinn tthhee last 10 years 3. Materials and Methods 3.1 SSaammppllee pprreeppaarraattiioonn,, hhoommooggeenniizzaattiioonn aanndd ssppiikkiinngg All samplleess wweerree pprreeppaarreedd ffoolllloowwiinngg tthhiiss protocol: TThhee sseeaawwaatteerr uusseedd iinn tthhiiss eexxppeerriimmeenntt wwaass nnaattuurraall ffiieelldd water collected aatt BBaallllyyvvaauugghhaann ppiieerr,, GGaallwwaayy bbaayy, Ireland, filtered through 47mm GGFF//CC WWhhaattmmaannnn ffiilltteerrss (WhatmannTM,,, KKKeeennnttt,,, UUUKKK))),,, aaauuutttoooccclllaaavvveeeddd (((SSSyyysssttteeeccc VVV111000000,,, WWWeeetttttteeennnbbbeeerrrggg ,,, GGGeeerrrmmmaaannnyyy))) aaannnddd ppprrreeessseeerrrvvveeeddd uuusssiiinnnggg neutral Lugol’s iodine solution (Clin-tteecchh,, DDuubblliinn,, IIrreellaanndd)). The centrifuge tubes ((5500mmll vvoolluummee)) were made up to tthhee rreeqquuiirreedd vvoolluummee wwiitthh sstteerriillee ffiilltteerreedd sseeaawwaatteerr ccoonnttaaiinniinngg neutral lugol’s iodine.. TThhiiss wwaass ccaarrrriieedd oouutt using aann aauuttoommaattiicc eeppppeennddoorrff mmuullttiippiippeettttee XXssttrreeaamm ((00-5500mmll)) ((EEppppeennddoorrff,, HHaammbbuurrgg,, GGeerrmmaannyy)) and the volume weighted in a calibrated balannnccceee (((MMMEEE444111444SSS SSSaaarrrtttooorrriiiuuusss,,, AAAGGG GGGooottttttiiinnngggeeennn,,, GGGeeerrrmmmaaannnyyy)))... TTThhheee dddeeennnsssiiitttyyy ooofff ssseeeaaawwwaaattteeerrr wwwaaasss cccooonnnsssiiidddeeerrreeeddd fffooorrr ttthhhiiisss pppuuurrrpppooossseee tttooo bbbeee 111...000222555ggg///mmmlll... TTThhheee fffiiinnnaaalll vvvooollluuummmeee ooofff eeeaaaccchhh sample was 45 ml approximately before spiking. 7 A stock solution for each of the ten species was prepared using 50ml glass screw top bottles (Duran®, Mainz, Germany). Then, a working stock containing the ten species to the required cell concentration was prepared using a measured aliquot from each stock solution into a 2l Schott glass bottle. Then, the working stock was homogenized and sub-divided into five replicate working stocks containing 400ml each. These working stocks were then inverted 100 times to homogenise the samples and 5ml aliquots were pipetted out after each 100 times inversion using a calibrated 5ml pipette (Gilson, Middleton, USA) with 1-10ml pipette tips (Eppendorf, Hamburg, Germany) The 5ml aliquots were dispensed into the 50ml centrifuge tubes (Sardstedt, Nümbrech, Germany) containing 45ml seawater. Samples were capped and labeled. Parafilm was used around the neck of the centrifuge tube to avoid water loss through evaporation or leaking, placed in padded envelopes and couriered via TNT or DHL couriers for a one day delivery across the world, in order for all the laboratories to have approximately the same arrival time. 3.2 Culture material, treatments and replicates. Most of the laboratory cultures used in the 2016 exercise have been collected in Galway bay and Bantry bay during the months of February and May 2016 except for the A.ostenfeldii culture from the CCMP culture collection in Scotland, the Karlodinium veneficum culture from the SCCAP culture collection in Denmark and Dinophysis acuta culture from the IEO, Vigo, Spain. The diatom cultures were isolated from samples collected using the micro-pipette technique into unialgal cultures. Most species were identified through light microscopy techniques using an inverted microscope Olympus IX-51 and a compound research Olympus microscope BX-53 (Olympus, UK) except for Pseudo-nitzschia australis which was confirmed to species level using qPCR (Roche Lightcycler) species specific gene probes. A total of 500 samples were produced for the enumeration and identification study. Each participant was sent a set of four samples, three for analysis plus one spare for a total of 328 samples to 43 laboratories. Another 15 samples were used by the expert laboratory to carry out the homogeneity and stability test. The data generated by this laboratory was used to test the homogeneity and stability of the samples. A minimum of 10 samples (50ml volume) were necessary for the homogeneity test and a minimum of 3 samples for the stability test. Samples had to be divided in two portions of 25ml each. 8 A time delay between the homogeneity test and the stability test is required. ISO 13528 indicates that this delay should be similar to that experienced by the participants in the test. As analysts have a month to return results from sample receipt, it was decided that this time delayed should be of one month as well. 3.3 Cell concentrations Preliminary cell counts from the original stock solutions were made to establish the cell concentration of each species and this was carried out using a glass Sedgewick-Rafter cell counting chamber (Pyser-SGI, Kent, UK) to ascertain an approximation of the cell concentration of each species in the samples. 3.4 Sample randomization All samples were allocated randomly to the participants using Minitab® Statistical Software Vr16.0 randomization tool. 3.5 Forms and instructions A set of instructions and forms required were sent via e-mail to all the analysts to complete the exercise including their unique identifiable laboratory and analyst code. Form 1 (Annex I) to confirm the receipt of materials; number and condition of samples and correct sample code. Form 2 (Annex II) in an Excel spreadsheet format to input species composition and calculate abundance for each species. Form 2 was used for the identification and enumeration part of the exercise. All analysts were asked to read and follow the instructions (Annex III) before commencing the test. At the end of the exercise and with the publication of this report, analysts will be issued with a statement of performance certificate (See Annex VI) which is tailored specifically for each test. This is an important document for auditing purposes and ongoing competency. 3.6 Statistical analysis Statistical analysis was carried out using PROlab Plus version 2.14, dedicated software for the statistical analysis of intercalibration and proficiency testing exercises from Quodata, Minitab® Statistical Software Vr16.0 and Microsoft office Excel 2007. 9 We followed the standard ISO normative 13528 which describes the statistical methods to be used in proficiency testing by interlaboratory comparisons. Here, we use this standard to determine and assess the homogeneity and stability of the samples, what to do with outliers, determining assigned values and calculating their standard uncertainty. Comparing these values with their standard uncertainty and calculating the performance statistics for the test through graphical representation and the combination of performance scores. The statistical analysis of the data and final scores generated from this exercise has been carried out using the consensus values from the participants. The main difference with previous years is that by using ISO13528, the consensus values from the participants must undergo several transformations before they can be used to generate Z-scores. The main transformation is the use of iteration to arrive at robust averages and standard deviations for each test item. This process allows for outliers and missing values to be dealt with, and it also allows for the heterogeneity of the samples to be taken into consideration when calculating these values. 3.7 IPI Ocean teacher online HAB quiz. The online HAB quiz was organized and set up by Jacob Larsen (IOC UNESCO, Centre for Science and Communication on Harmful Algae, Denmark) and Rafael Salas (Marine Institute, Ireland). The exercise was prepared in the web platform ‘Ocean teacher’. The Ocean teacher training facility is run by the IODE (International Oceanographic Data and information Exchange) office based in Oostende, Belgium. The IODE and IOC organize some collaborative activities among them, the IOC training courses on toxic algae and the IPI online HAB quiz. The online quiz uses the open source software Moodle Vr2.0 (https://moodle.org ). First time participants had to register in the following web address: http://classroom.oceanteacher.org/ before allowed to access the quiz content, while analysts already registered from previous years, could go directly to the login page. Once registered, participants could login into the site and using a password, able to access the quiz. Three months time was given to analysts to register, complete and submit the online quiz. The course itself was found under the courses tab in the main menu page. Analysts could link to the International Phytoplankton Intercomparison and quiz IPI 2016 HAB quiz content from here. The test itself consisted of 15 questions (see Annex XVII). Most questions used in this quiz this year were ‘matching type’ Q1 to 15 except for Q9 which was Multiple choice. Matching questions have dropdown 10