Organized by the NCME Artificial Intelligence in Measurement & Education (AIME) SIGIMIE
Large language models are increasingly used in educational and psychological measurement activities. Their rapidly evolving sophistication and ability to detect language semantics make them viable tools to supplement subject matter experts and their reviews of large amounts of text statements, such as educational content standards.
This paper presents an application of text embeddings to find relationships between different sets of educational content standards in a process termed content mapping. This content mapping process is routinely used by state education agencies and is often a requirement of the United States Department of Education peer review process. We discuss the educational measurement problem, propose a formal methodology, demonstrate an application of our proposed approach, and provide measures of its accuracy and potential to support real-world activities.
Presenters:Harold Doran & Reese Butterfuss, Human Resources Research Organization
National Council on Measurement in Education