Several sources were consulted to identify the set of terms to be included in the glossary. Existing glossaries in introductory measurement textbooks were reviewed, and web searches for existing glossaries were conducted. In addition, score reports and interpretive materials used with tests from K-12 testing, higher education admissions and outcomes testing, and national (NAEP) and international (PISA) testing were reviewed.
An attempt was made to identify new terms learned by those just beginning their study of educational measurement and to determine which terms appear most frequently on score reports obtained by test takers (or their parents) who are not experienced in interpreting test scores and using them for personal decision-making.
An advisory group was created for use in the development stages to offer advice to the glossary editor about terms to be included and term definitions to be modified. In view of the broad scope of terms to be included, the advisory group needed to be representative of a broad range of work contexts and experience with lay audiences. Individuals considered included introductory textbook authors, test score-report developers, providers of professional development in assessment for lay audiences, researchers, higher education professionals, and individuals with a broad “big picture” measurement exposure. It was possible to assemble an advisory group that met all criteria posed, and several members of the advisory panel met several of these criteria.
Initially, the advisory group provided reactions to the editor regarding a draft list of terms to be included in the glossary. Members of the group were asked to delete terms that, in their judgment, should not be included and add terms that should be there but were not on the draft list. (Terms without definitions were provided for review.) After the editor developed the definitions, the advisory group independently reviewed the list of terms and definitions and offered suggestions for revisions to the editor. Once the first set of revisions was completed, the advisory group was given another opportunity to offer suggestions for change. The editor made final decisions in cases in which disagreement occurred.
Members of the development team for this glossary were:
Editor, David A. Frisbie, University of Iowa and Iowa Testing Programs
Advisor, Michael Beck, Queststar Assessment
Advisor, Jennifer Beimers, Pearson Education
Advisor, Susan Brookhart, Brookhart Enterprises and Duquesne University
Advisor, Valerie Moody, University of Iowa