Florida Law Review Volume 66|Issue 3 Article 10 February 2015 Freedom of Information Act and Federal Licensing Procedures: Invoking Exemption 7(F) to Protect Examination Materials Karl Gruss Follow this and additional works at:http://scholarship.law.ufl.edu/flr Part of theLegislation Commons Recommended Citation Karl Gruss,Freedom of Information Act and Federal Licensing Procedures: Invoking Exemption 7(F) to Protect Examination Materials, 66 Fla. L. Rev. 1403 (2015). Available at: http://scholarship.law.ufl.edu/flr/vol66/iss3/10 This Note is brought to you for free and open access by UF Law Scholarship Repository. It has been accepted for inclusion in Florida Law Review by an authorized administrator of UF Law Scholarship Repository. For more information, please [email protected]. Gruss: Freedom of Information Act and Federal Licensing Procedures: Invo FREEDOM OF INFORMATION ACT AND FEDERAL LICENSING PROCEDURES: INVOKING EXEMPTION 7(F) TO PROTECT EXAMINATION MATERIALS Karl Gruss* Abstract The United States Supreme Court’s 2011 decision in Milner v. Department of the Navy shut the door on an expansive interpretation of one of the nine enumerated exemptions to the public disclosure requirements mandated under the Freedom of Information Act. No longer can federal agencies seek cover behind the judicially crafted interpretation of Exemption 2 known as the “High 2” that permitted agencies to withhold documents from the public eye solely because disclosure of the information contained therein could risk circumvention of an individual agency’s regulations or statutes. However, Justice Alito’s concurring opinion in Milner hinted at the Court’s possible acceptance of an alternative option available to federal agencies that previously employed the High 2 to rebuff public requests for information. This Note focuses on federal agencies involved in the licensing and certification of individuals in public safety- sensitive positions—principally the U.S. Coast Guard and the Federal Aviation Administration—that prior to Milner either actively used or could have used the High 2 exemption to protect information relating to examination questions and answers used to evaluate license and certificate applicants’ competencies. This Note first examines the rise of the High 2 interpretation through the circuit courts, its application by federal agencies, and the Supreme Court decision sounding the High 2’s death knell. This Note then argues that courts should embrace an interpretation of another Freedom of Information Act exemption, Exemption 7(F), to permit federal agencies involved in the licensing and certification of individuals in public safety-sensitive positions to withhold information relating to examination questions and answers. INTRODUCTION .................................................................................... 1404 I. AGENCY DRIVE FOR THE PROTECTION OF EXAMINATION Q&AS ........................................................... 1409 A. Rote Learning and the Impact on Public Safety ................................................................. 1409 * J.D., May 2014, University of Florida Levin College of Law. I would like to sincerely thank all of my colleagues on the Florida Law Review, including those who dedicated their time to assisting me with this Note. I would also like to specifically thank our Editor in Chief Clark Splichal for his dedication to the Review over the past year. 1403 Published by UF Law Scholarship Repository, 2015 1 Florida Law Review, Vol. 66, Iss. 3 [2015], Art. 10 1404 FLORIDA LAW REVIEW [Vol. 66 B. The Federal Aviation Administration as a Mixed-Function Licensing Agency Case Study .................................................................... 1410 II. OPERATING IN THE CONTEXT OF FOIA’S WATCHFUL EYE .................................................................... 1413 A. FOIA and the Presupposition of Disclosure ................ 1414 B. Rise of the High 2 Exemption ....................................... 1415 C. The High 2 Exemption’s Subsequent Fall .................... 1417 D. Refinement of Exemption 7 ........................................... 1420 III. APPLICATION OF EXEMPTION 7(F) ........................................ 1422 A. Exemption 7(F)’s First Prong and Mixed-Function Agencies ........................................................................ 1423 1. The Law Enforcement Hurdle ............................... 1423 2. Meeting the Compiled Information Mandate ................................................................. 1425 B. Exemption 7(F)’s Second Prong and Mixed-Function Agencies ............................................. 1426 CONCLUSION ....................................................................................... 1428 INTRODUCTION Imagine if individuals could fly aircraft or navigate commercial vessels without first establishing the minimum competencies necessary to do so in a skillful manner. The operation of aircraft and commercial vessels itself entails inherent risks to public safety, even in the hands of experienced controllers,1 let alone in the hands of novice members of the general public. Given the potential for disastrous results if such activities went unregulated, the Federal Aviation Administration (FAA) and the U.S. Coast Guard require that individuals demonstrate particular skills prior to obtaining authorization to command certain aircraft and marine vessels, 1. According to the Federal Aviation Administration (FAA), roughly 80% of general aviation aircraft fatalities are directly attributable to human error, with loss of control in flight identified as the number one cause of fatal general aviation accidents during 2001–2011. Fact Sheet – General Aviation Safety, FAA (Jun. 19, 2012), https://www.faasafety.gov/files/events/EA/ EA27/2013/EA2748590/FAA_Fact_Sheet_%E2%80%93_General_Aviation_Safety.pdf; see also DOUGLAS A. WIEGMANN & SCOTT A. SHAPPELL, FED. AVIATION ADMIN., A HUMAN ERROR ANALYSIS OF COMMERCIAL AVIATION ACCIDENTS USING THE HUMAN FACTORS ANALYSIS AND CLASSIFICATION SYSTEM (HFACS) 1 (2001), available at http://www.faa.gov/data_research/research/ med_humanfacs/oamtechreports/2000s/media/0103.pdf (noting the slow rate of decline of aviation accidents attributable to human error when compared to the rate of decline of accidents attributable to mechanical failure). http://scholarship.law.ufl.edu/flr/vol66/iss3/10 2 Gruss: Freedom of Information Act and Federal Licensing Procedures: Invo 2014] UTILIZING EXEMPTION 7(F) TO PROTECT EXAMINATION MATERIALS 1405 respectively.2 The FAA and the U.S. Coast Guard represent a category of federal agencies, termed “mixed-function licensing agencies” in this Note,3 charged with the licensing and certification of individuals in public safety- sensitive positions that adopt procedures to examine applicants’ qualifications.4 Examination techniques employed by these mixed-function licensing agencies may include, among other methods, classroom-based written examinations.5 These written examinations are designed to test license applicants’ knowledge of concepts necessary to successfully operate in environments where their duties and responsibilities directly influence individuals’ security and welfare.6 Though mixed-function licensing agencies publish study guides and other similar reference materials, they often seek to protect the actual examination questions and answers (examination Q&As) from public view to maintain the integrity of the examination process.7 Shielding actual examination Q&As prior to the administration of a test 2. Depending on the type of aircraft an individual wishes to pilot, the FAA requires that the license applicant pass both a written and practical examination. See 14 C.F.R. §§ 61.3, .33–.39, .43 (2013); Recreational Pilot and Private Pilot Knowledge Tests, FAA, http://www.faa.gov/pilots/ become/knowledge (last visited June 16, 2014) (providing an overview of knowledge testing procedures for recreational and private pilots); Recreational Pilot and Private Pilot Practical Tests, FAA, http://www.faa.gov/pilots/become/practical (last visited June 16, 2014) (describing practical testing requirements facing recreational and private pilots). In similar fashion, the Coast Guard requires that individuals seeking to hold certain positions on commercial vessels obtain a Merchant Mariner Credential (MMC) with endorsements detailing the scope of the credential’s authorization by demonstrating proficiencies through a written examination. See 46 C.F.R §§ 10.201, .209 (2012) (describing general procedures to apply for and obtain an MMC); 46 C.F.R. § 11.903 (listing endorsements requiring passage of an examination prior to issuance); Examinations, U.S. COAST GUARD, NAT’L MAR. CTR., http://www.uscg.mil/nmc/training/exams/default.asp (last visited June 16, 2014) (discussing the process of passing written examinations to obtain MMC endorsements). 3. The term is used to reflect the fact that these agencies tackle a broad range of functions beyond the licensing and certification of private individuals. The term “mixed-function agency” has been used by courts previously to describe agencies embracing both administrative and law enforcement missions. See, e.g., Living Rivers, Inc. v. U.S. Bureau of Reclamation, 272 F. Supp. 2d 1313, 1318–19 (D. Utah 2003) (determining that the U.S. Bureau of Reclamation qualified as a mixed-function agency because of the dual roles played by the agency in light of its statutory authorization). 4. See supra note 2. 5. As part of the agency’s licensing process, the FAA administers Airman Knowledge Tests tailored to the specific license individuals would like to obtain. Airman Knowledge Test Guides, FAA, https://www.faa.gov/training_testing/testing/test_guides (last visited June 16, 2014). The U.S. Coast Guard likewise administers Deck and Engineering exams depending on the position a license applicant wishes to hold. U.S. COAST GUARD, NAT’L MAR. CTR., GUIDE FOR ADMINISTRATION OF MERCHANT MARINER EXAMINATIONS (DECK & ENGINEERING GUIDE) 1, 2, 22, 54 (2011), available at http://www.uscg.mil/nmc/training/pdfs/deck_and_engineering_guide.pdf. 6. See About Us, NAT’L MAR. CTR., U.S. COAST GUARD, http://www.uscg.mil/nmc/about_us/ default.asp (last visited June 16, 2014). 7. See discussion infra Part I. Published by UF Law Scholarship Repository, 2015 3 Florida Law Review, Vol. 66, Iss. 3 [2015], Art. 10 1406 FLORIDA LAW REVIEW [Vol. 66 may seem natural.8 However, all federal agencies operate under the watchful eye of the Freedom of Information Act (FOIA),9 which, at its core, mandates that federal agencies provide public access to official government records, subject to a set of nine limited exemptions.10 Mixed- function licensing agencies often assemble large databases of examination Q&As from which samples are pulled for each test.11 These examination databases constitute official agency records.12 Therefore, mixed-function licensing agencies intent on protecting examination Q&As from the public eye must employ one of the nine enumerated FOIA exemptions. Courts’ interpretations of FOIA’s Exemption 2, which excludes official government documents “related solely to the internal personnel rules and practices of an agency” from the statute’s disclosure mandate,13 for years enabled agencies to protect examination Q&As from publication.14 Based on a broad view of the exemption established by the D.C. Circuit Court in Crooker v. Bureau of Alcohol, Tobacco & Firearms,15 courts in the Second,16 Seventh,17 and Ninth Circuits18 embraced an interpretation of the 8. See discussion infra Part I (discussing the prevalence of rote learning in the examination process and studies documenting the negative impact on information comprehension). 9. Freedom of Information Act, 5 U.S.C. § 552 (2012). 10. Id. § 552(b)(1)–(9); see also David S. Levine, Secrecy and Unaccountability: Trade Secrets in Our Public Infrastructure, 59 FLA. L. REV. 135, 157 (2007) (stating that, in light of passage of FOIA, “[s]ecrecy [in a democratic government] is the exception, rather than the norm”). 11. See, e.g., Airman Knowledge Test Question Bank: Private Pilot, FAA (Jan. 13, 2014), https://www.faa.gov/training_testing/testing/test_questions/media/pvt.pdf (providing a glimpse into the private pilot knowledge examination question bank); Frequently Asked Questions, MYWRITTENEXAM.COM, http://www.mywrittenexam.com/mwe/faq (last visited June 16, 2014) (explaining that the testing company has access to the over 900 questions that appear in the FAA’s private pilot examination questions bank); U.S. COAST GUARD, NAT’L MAR. CTR., supra note 2 (providing access to deck and engineering examination questions); Dom Yanchunas, Coast Guard to Resume Posting Full Archive of Mariner Exam Questions, PROF. MARINER (Aug. 28, 2012, 11:53 AM), http://www.professionalmariner.com/Web-Exclusive-2012/Coast-Guard-to-resume-posting- full-archive-of-mariner-exam-questions (detailing the U.S. Coast Guard’s policy of posting all examination questions). 12. Title 5 of the U.S. Code does not specifically define what qualifies as an “agency record,” but the U.S. Supreme Court has articulated a two-part test stating that an (1) “agency record” must consist of information created or obtained by the agency, and (2) the agency must control the information at the time of the FOIA request. See U.S. Dep’t of Justice v. Tax Analysts, 492 U.S. 136, 144–45 (1989). 13. 5 U.S.C. § 552(b)(2). 14. See discussion infra Section II.C (providing a look at the U.S. Coast Guard’s response to the Court’s narrowing of Exemption 2). 15. 670 F.2d 1051, 1074 (D.C. Cir. 1981) (holding that “if a document for which disclosure is sought meets the test of ‘predominant internality,’ and if disclosure significantly risks circumvention of agency regulations or statutes, then Exemption 2 exempts the material from mandatory disclosure”), overruled by Milner v. Dep’t of the Navy, 131 S. Ct. 1259 (2011). 16. Massey v. FBI, 3 F.3d 620, 622 (2d Cir. 1993) (stating that “internal agency information may be withheld . . . if the material is of public interest, and ‘the government demonstrates that disclosure of the material would risk circumvention of lawful agency regulations’” (quoting Buffalo http://scholarship.law.ufl.edu/flr/vol66/iss3/10 4 Gruss: Freedom of Information Act and Federal Licensing Procedures: Invo 2014] UTILIZING EXEMPTION 7(F) TO PROTECT EXAMINATION MATERIALS 1407 exemption that shielded “predominantly internal” documents whose disclosure would “significantly risk[] circumvention of agency regulations or statutes.”19 Referred to as the “High 2” exemption by courts following the Crooker precedent,20 the exemption provided mixed-function licensing agencies justification for protecting examination materials.21 The judicially crafted High 2 exemption crumbled in 2011 following the Supreme Court’s decision in Milner v. Department of the Navy.22 In Milner, the Court brought an end to the Crooker construction, holding that a plain reading of FOIA does not permit such an expansive application of the exemption.23 Beyond simply drawing the curtains on the High 2 concept, the Court’s decision signaled a return to a plain-language reading of FOIA under which the nine exemptions “are ‘explicitly made exclusive’ and must be ‘narrowly construed.’”24 Against this backdrop, mixed-function licensing agencies no longer enjoy access to the broad protections offered by the now-defunct High 2 exemption in efforts to protect licensing examination Q&As.25 But while Milner may have foreclosed the use of the High 2 exemption, several potential options have emerged to fill the void. The first method of bypassing FOIA’s disclosure requirement entails securing codification of the High 2 concept by amending the statute itself to either rewrite Exemption 2 or create a new exemption embodying the language articulated in the Crooker decision.26 The second method entails utilizing Exemption 3, which permits agencies to exempt specific documents from disclosure by statute.27 Evening News v. U.S. Border Patrol, 791 F. Supp. 386, 391 (W.D.N.Y. 1992))), overruled by Milner, 131 S. Ct. 1259. 17. Kaganove v. EPA, 856 F.2d 884, 889 (7th Cir. 1988) (embracing the Crooker interpretation after embarking on its own review of FOIA’s legislative history), overruled by Milner, 131 S. Ct. 1259. 18. Milner v. U.S. Dep’t of the Navy, 575 F.3d 959, 965 (9th Cir. 2009), rev’d, 131 S. Ct. 1259. 19. Crooker, 670 F.2d at 1073–74. 20. See Milner, 575 F.3d 959 passim. 21. See discussion infra Section II.C (providing a look at the Coast Guard’s response to the Court’s narrowing of Exemption 2). 22. 131 S. Ct. 1259, 1271 (2011). 23. Id. (“Exemption 2, consistent with the plain meaning of the term ‘personnel rules and practices,’ encompasses only records relating to issues of employee relations and human resources.” (quoting 5 U.S.C. 552(b)(2) (2006))). 24. Id. at 1262 (citation omitted) (quoting EPA v. Mink, 410 U.S. 73, 79 (1973) and FBI v. Abramson, 456 U.S. 615, 630 (1982)). 25. See infra notes 118–121 and accompanying text. 26. For a discussion of such proposed language, see Daniel J. Metcalfe, Amending the FOIA: Is it Time for a Real Exemption 10?, ADMIN. & REG. L. NEWS, Summer 2012, at 16, 16–18. 27. 5 U.S.C. § 552(b)(3) (requiring a statute contain language that explicitly either “(i) requires that the matters be withheld from the public in such a manner as to leave no discretion on Published by UF Law Scholarship Repository, 2015 5 Florida Law Review, Vol. 66, Iss. 3 [2015], Art. 10 1408 FLORIDA LAW REVIEW [Vol. 66 Neither of these options, however, presents an ideal solution to the problem that mixed-function licensing agencies face. Modification of Exemption 2 or creation of an entirely new exemption codifying the High 2 standard fails to guarantee the Court will read the language as covering federal licensing materials.28 Furthermore, reliance on a statutory amendment to FOIA subjects agencies to shifting political winds. Similarly, relying on individual statutory exclusions under Exemption 3 will force agencies to appeal to Congress on a case-by-case basis.29 Such reliance creates an onerous burden on mixed-function licensing agencies by removing their ability to efficiently protect information from public disclosure in the face of evolving circumstances. At the same time, congressional approval of an agency’s request for an individual statutory exemption eliminates the ability of courts to examine whether the government is justified in withholding information, potentially exposing the use of the exemption to abuse.30 This Note argues that a third option, the utilization of Exemption 7(F),31 presents mixed-function licensing agencies a concrete means to lawfully protect licensing examination materials from public disclosure. Part I discusses why mixed-function licensing agencies seek to withhold examination Q&As in the first place. Part II tracks FOIA’s development and evolution, beginning with the statute’s basic premise that presupposes information should be made public. Part II analyzes the development by Congress of various qualifications to this premise through amendments to the original FOIA, and how agencies successfully justified withholding information under Exemption 2. Part III addresses how, in narrowing the scope of longstanding interpretations of Exemption 2, the Supreme Court opened the door for mixed-function licensing agencies to employ Exemption 7(F) to protect information relating to the certification and licensing of select individuals. This Note concludes that mixed-function licensing agencies should protect the integrity of their competency assessment tools by using Exemption 7(F) to continue to withhold the issue; or (ii) establishes particular criteria for withholding or refers to particular types of matters to be withheld” to qualify under this exemption). 28. The Court has not explicitly stated that the High 2 construction, if constituting a permissible interpretation of Exemption 2’s scope, would cover mixed-function licensing agency examination Q&As. 29. See supra note 27. 30. It is difficult to project how Congress may respond to individual requests for statutory exemptions to FOIA’s public disclosure requirement, but it is possible to foresee agencies applying to Congress for individual exemptions with full knowledge that the material the agencies seek to withhold falls far outside that which FOIA’s nine enumerated exemptions are designed to protect. 31. 5 U.S.C. § 552(b)(7)(F) (stating that FOIA’s disclosure mandate does not apply to “records or information compiled for law enforcement purposes, but only to the extent that the production of such law enforcement records or information . . . could reasonably be expected to endanger the life or physical safety of any individual”). http://scholarship.law.ufl.edu/flr/vol66/iss3/10 6 Gruss: Freedom of Information Act and Federal Licensing Procedures: Invo 2014] UTILIZING EXEMPTION 7(F) TO PROTECT EXAMINATION MATERIALS 1409 examination Q&As from public disclosure. I. AGENCY DRIVE FOR THE PROTECTION OF EXAMINATION Q&AS Rote learning undermines agencies’ competency assessment tools and increases the public-safety risk of licensing unqualified individuals to operate aircraft and commercial vessels. This drives mixed-function licensing agencies to protect examination Q&As from public disclosure. Based on government reports documenting competency concerns centering on rote learning, this Part addresses the analytical basis for applying Exemption 7(F) to agency examination Q&As. A. Rote Learning and the Impact on Public Safety Common sense appears to dictate that mixed-function licensing agencies would seek to withhold examination Q&As from the public eye; after all, providing test takers with not only all of the questions they will encounter on an exam but also all of the answers runs counter to traditional educational principles. But in the face of FOIA’s presupposition of public disclosure32 and the risk of litigation posed by improperly invoking one of the nine enumerated exemptions to the disclosure mandate,33 mixed- function licensing agencies must establish a firmer ground to deny access than a general appeal to common sense. Studies conducted on the effect of rote learning or rote memorization, defined as “mechanical or unthinking routine or repetition,”34 provide one form of empirical support for protecting examination Q&As.35 It is critical to focus on what is at stake here. Mixed-function licensing agencies are not concerned with providing individuals the go-ahead to operate in a manner where their actions have the potential to affect only themselves. Instead, mixed-function licensing agencies arm individuals with licenses to man the controls of machines36 that in an instant can inflict widespread damage on the public. It is in recognition of these licenses’ inherent power that mixed-function licensing agencies look to employ exemptions to FOIA’s disclosure mandate in the examination of license 32. See infra Section II.A. 33. See infra notes 59–60. 34. MERRIAM-WEBSTER’S COLLEGIATE DICTIONARY 1084 (11th ed. 2011). 35. See, e.g., STEPHEN M. CASNER ET AL., NASA AMES RESEARCH CTR., FAA PILOT KNOWLEDGE TESTS: LEARNING OR ROTE MEMORIZATION? 10–12 (2004), available at http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20040034190.pdf; KEIKO MOEBUS, MOEBUS AVIATION CONSULTING, IMPACT ASSESSMENT OF THE PUBLICATION OF QUESTIONS OF THEORETICAL EXAMINATIONS FOR PART 66 AND PART FCL § 1 (2009), available at http://www.easa.europa.eu/ rulemaking/docs/research/Final%20Report%20on%20publication%20assessment.pdf. 36. For example, the FAA grants individuals commercial and recreational private licenses and the U.S. Coast Guard grants individuals commercial vessel licenses. See supra note 2 and accompanying text. Published by UF Law Scholarship Repository, 2015 7 Florida Law Review, Vol. 66, Iss. 3 [2015], Art. 10 1410 FLORIDA LAW REVIEW [Vol. 66 applicants. B. The Federal Aviation Administration as a Mixed-Function Licensing Agency Case Study Access to an agency’s justification for withholding information from the public is, in itself, largely covered by an exemption to FOIA protecting intra-agency memorandums not available by law to a party other than another agency in litigation.37 However, in response to concerns raised over the potential impact on general aviation safety of rote memorization versus “genuine conceptual understanding” due to misalignment in airman testing, curriculum development, and industry standards,38 the FAA is currently engaged in creating an agency working group tasked, in part, with evaluating the agency’s test management practices.39 The FAA administers airman knowledge tests as part of the agency’s airman certification procedure.40 These examinations consist of objective, multiple-choice questions covering a range of issues related to the certification the test taker hopes to acquire,41 and the agency draws from a database of questions and answers to assemble its exams.42 Prior to the FAA’s investigation, the National Aeronautics and Space Administration (NASA)43 and the European Aviation Safety Agency (EASA)44 analyzed the effectiveness of the FAA’s testing system on critical concept comprehension. To conduct its study, NASA reworded existing examination questions from the FAA’s database, shuffled the answers, and created new figures for problems that required test takers to examine diagrams in formulating answers.45 NASA also created its own set of unique questions not available 37. 5 U.S.C. 552(b)(5) (2012) (“[FOIA’s disclosure mandate] does not apply to matters that are . . . inter-agency or intra-agency memorandums or letters which would not be available by law to a party other than an agency in litigation with the agency . . . .”). 38. AIRMEN TESTING STANDARDS AND TRAINING WORKING GRP., A REPORT FROM THE AIRMEN TESTING STANDARDS AND TRAINING WORKING GROUP TO THE AVIATION RULEMAKING ADVISORY COMMITTEE 6 (2013), available at http://www.faa.gov/regulations_policies/rulemaking/committees/ documents/media/Airmen.Testing.Standards.Recommendation.Report.9.30.2013.PDF. 39. Aviation Rulemaking Advisory Committee, New Task Assignment for the ARAC: Establishment of Airman Certification System Working group, 79 Fed. Reg. 4800 (proposed Jan. 29, 2014). The working group, consisting of a wide range of stakeholders and tackling a variety of issues related to the FAA’s airman certification system, must submit its findings no later than December 31, 2015. Id. 40. Airman Knowledge Test Guides, supra note 5. 41. For an example of an examination offered by the FAA, see Airman Knowledge Test Question Bank: Commercial Pilot, FAA (Jan. 13, 2014), http://www.faa.gov/training_testing/ testing/test_questions/media/com.pdf. 42. CASNER ET AL., supra note 35, at 1. 43. See generally id. 44. See generally MOEBUS, supra note 35. 45. CASNER ET AL., supra note 35, at 5–7. http://scholarship.law.ufl.edu/flr/vol66/iss3/10 8 Gruss: Freedom of Information Act and Federal Licensing Procedures: Invo 2014] UTILIZING EXEMPTION 7(F) TO PROTECT EXAMINATION MATERIALS 1411 through the FAA’s examination database concerning required pilot knowledge,46 and it used unaltered questions from the FAA’s database as a control set.47 NASA administered the revised examination to forty-eight private pilot applicants that recently completed the standard FAA knowledge test to determine “if pilot applicants’ knowledge of the required aeronautical knowledge was based on understanding, or on a more superficial review of the questions appearing in the FAA [Q&A database].”48 NASA found a statistically significant drop in the percentage of questions answered correctly versus the control when figures diverging from those in the FAA database were inserted into questions,49 and an even greater drop when students faced entirely unique questions.50 Coupled with data provided by the FAA indicating many test takers were able to complete standard examinations in less time than it would take the average person to read the question prompts,51 NASA ultimately concluded that, in the case of the FAA, releasing examination Q&As in advance undermined the viability of the examination process as a competency assessment tool.52 In similar fashion to the NASA study, the EASA study analyzed the effects of rote learning on meaningful technical information comprehension by examining the ability of university students to learn and recall 136 Q&As from a flight training school examination question bank in a twenty-four-hour period.53 On the first day of the study, the EASA tested students on the exact same 136 questions provided in advance, along with 136 reworded versions of those same questions.54 One week later, the same students were tasked with memorizing another 136 questions, and one day later were presented with an exam consisting of an exact copy of those same questions, 136 reworded versions of those questions and the same 272 questions from the previous week’s exam.55 The EASA found that the average test taker could score above 75%, considered a passing score, though the test takers scored marginally worse on the reworded questions.56 Based on the study, the EASA concluded that aviation examinations designed to test technical comprehension were susceptible to rote learning and shallow understanding if the full set of examination Q&As was provided in advance.57 The EASA report qualified its finding 46. Id. at 7. 47. Id. at 5–6. 48. Id. at 5. 49. Id. at 10. 50. Id. 51. Id. at 2–4. 52. Id. at 10. 53. MOEBUS, supra note 35, § 1. 54. Id. 55. Id. 56. Id. § 8. 57. Id. Published by UF Law Scholarship Repository, 2015 9