News@NCME

Annual Meeting

ITEMS Portal

NCMEntoring

NCME SIGMIEs

Become an NCME Member

Empowering Insights in Educational Measurement

Welcome to News@NCME, your dedicated source for the latest developments, insights, and discussions in the world of educational measurement. As a vibrant community of professionals, educators, researchers, and students, we are committed to advancing the field through sharing knowledge, fostering collaboration, and celebrating achievements.

At News@NCME, we understand the importance of staying informed and connected. Our platform offers a dynamic range of articles, updates, and resources tailored to enrich your understanding and engagement with educational measurement. Whether you are delving into complex research, seeking best practices, or exploring innovative approaches, our content is curated to support your journey.

We invite you to explore our latest news, participate in upcoming events, and engage with our diverse community. Your insights and contributions are what make NCME a leading voice in educational measurement. Let's continue to learn, share, and grow together.

Editor of News@NCME: Cheng Hua, Ph. D., University of Montevallo

 

NCME President: Dr. Andrew Ho

Harvard Graduate School of Education

Bird with Tree

President's Message

                        Updated on Oct. 1st, 2024

Dear NCME Community,

September was a productive month for NCME members, as our community was busy, e.g., 1) preparing and submitting proposals for our annual meeting in Denver, 2) attending our NCME Special Conference on Classroom Assessment in Chicago, and 3) advancing NCME’s scholarship and professional standards. For my October update, here are a few of our many goings-on.

1)  Congratulations to our annual meeting program team! Proposal submissions for our Denver meeting in 2025 increased in all categories this year, including a 50% increase in graduate student submissions. This is a sign of a healthy and growing field. Please pass along appreciation to Katherine Castellano and Scott Monroe (program co-chairs), Qiwei Britt He and Matt Madison (training session co-chairs), and Catherina Villafuerte and Nicolas Mireles (Graduate Student Issues Committee co-chairs) for their great work ensuring an informed and smooth proposal submission process.

2)  The spirit of service continues among our over 300 reviewers who volunteered to serve on symposium review panels and to review individual papers. Over 100 reviewers attended “reviewer meetups” that I hosted to engage our reviewers around the dual goals of formative and evaluative ratings. To me, the NCME mission requires both discernment (in selecting the most engaging scholarship and practice to present at the annual meeting) and generosity (in providing feedback that advances the work of all who submitted, regardless of the selection decision).

3)  I brought our NCME Board of Directors to the fifth NCME Special Conference on Classroom Assessment at the beautiful Palmer House Hotel in Chicago. What an event! My thanks to conference organizers from the University of Illinois Chicago (Paul Zavitkovsky), NCIEA (Carla Evans, Caroline Wiley, and Scott Marion), and the Chicago Public Schools (Peter Leonard). I particularly enjoyed seeing so many CPS teachers and staff members attending. I hope all NCME conferences can work to engage local agencies. Please help us to do this in Denver, as well.

4)  The theme of our recent board meeting in Chicago was “strategic stewardship, fiscal prudence with ambitious goals.” Although the financial position of NCME is strong, it is our responsibility to balance our budget. This will require some difficult decisions, especially given our ambitions around open access to our scholarship and professional guidance. We appreciate your engagement in our deliberations through our committees and SIGIMIEs as well as directly to us as elected board members.

5)  Our working dinner discussion topic at our Board of Directors meeting in Chicago was about how Artificial Intelligence is changing the way we work, in general, and, if applicable, specifically around educational measurement. We asked what role NCME should play in advancing expertise in Artificial Intelligence in Educational Measurement (AIME). We are discussing this with Chris Ormerod and John Whitmer as co-chairs of the AIME SIGIMIE, and we hope all who are interested will engage us there and in related sessions at our future meetings.

Thank you, as always, for the many ways that you contribute selflessly to this organization to advance our shared mission.

2025 NCME Annual Meeting Call for Proposals

The Annual Meeting will be held in Denver, Colorado, from April 23 (training session day) through April 26, 2025. Please consider volunteering to be a session chair, discussant and/or a reviewer. We look forward to receiving your submissions and hope to see you all in Denver. If you have questions, comments, or suggestions, please do not hesitate to contact us at NCMEProgramChairs@ncme.org.

Call for Authors to Publish in ITEMS!

Do you want to share your expertise with a broader audience in an interactive format? Are you interested in publishing in Educational Measurement: Issues and Practice? If so, consider developing an ITEMS module!

What Are ITEMS Modules?
ITEMS modules are instructional resources designed to introduce key topics in educational measurement. With our user-friendly templates, you can get started without needing special instructional design skills or prior teaching experience.

Benefits of Developing an ITEMS Module:

  • Self-Paced Publication Cycle: Complete the process in less than a year.
  • Publication in a Reputable Journal: Gain visibility in Educational Measurement: Issues and Practice.
  • Enhance Communication skills: Improve your communication and instructional skills by creating a module on a topic of your interest.
  • Support Colleagues and Learners: ITEMS serves as a freely accessible learning resource for a diverse audience, including practitioners, graduate students, scholars, and the public. You can help them become familiar with important topics!

Topics:
ITEMS are currently open to any topics related to educational measurement. Preferred topics may include, but are not limited to, process data, learning analytics, machine learning techniques in educational measurement, and advanced psychometric methods for evaluating bias and fairness. This is a fantastic opportunity for graduate students and early career professionals eager to share emerging topics in educational measurement!

 

If you’re interested, please fill out the following form: Click Here

If you have any questions, don’t hesitate to reach out!

 

Stella Kim

stella-kim@charlotte.edu

Incoming Editor, Instructional Topics in Educational Measurement Series

NCME launches Presidential Task Force on Educational Measurement and Civil Rights

          On May 30, the Presidential Task Force on the Role of Educational Measurement in the Protection and Advancement of Civil Rights held its inaugural meeting. The Task force, which was announced in March by Immediate Past President Michael Walker, will be chaired by Dr. Britte Haugan Cheng (Menlo Education Research, LLC).

          Cheng is joined on the Task Force by Co-chairs Dr. Sarah Beach (Education Rights Institute, University of Virginia School of Law), Dr. Juan D’Brot (Center for Assessment), and Dr. Howard Everson (Graduate Center, City University of New York). The eleven other Task Force members represent university faculty, graduate students, departments of education, school districts, consulting firms, testing organizations, and other nonprofits. Together, they will:

  • Look at the history of educational measurement in civil rights litigation and document the good and the bad.
  •      Identify specific elements of civil rights protections that may possibly be supported by educational measurement.
  •        Look at educational measurement programs at the federal, state, and local levels that claim to capture educational progress and document what purposes they serve fairly.
  •      Address common misunderstandings that get in the way of good legislation, policy, and practice that could ensure equal rights.
  •      And finally, write a report that reflects the consensus of the field of educational measurement, and indicating areas where greater understanding and consensus-building may be needed.

        The Task Force plans to involve other NCME members as well as representatives from different communities who interact with educational measurement in various capacities. 

        President Andrew Ho attended the inaugural meeting to offer his congratulations on the start of this important work. He encouraged the Task Force to present its findings at the 2025 Annual Meeting. Task Force members appreciated his remarks. “President Ho’s comments set a lovely stage for our conversation,” said Cheng afterward. She added, “I am excited and honored to be working on this task force with so many great people.”

        “I am thrilled to be working with the Task Force, and I can’t wait to get started,” said Walker. “I am confident that the final product will be a valuable asset for test score users.”

Upcoming Events

1. Call for Nominations

        The National Assessment Governing Board (NAGB) is currently seeking qualified individuals to serve as Board members to help govern the National Assessment of Educational Progress (NAEP). One of the three seats designated explicitly for Testing and Measurement Experts will become available this year. Current NAGB and NCME member Scott Marion is eligible for reappointment, but NAGB and Secretary Cardona welcome new applicants. Board members serve a four-year term, with the potential for reappointment for a second term.

        In 2019, as part of an NCME initiative to increase its impact on educational policy, the NCME Board began nominating members for consideration for the Testing and Measurement Expert seat on NAGB. To date, NCME members Suzanne Lane, Scott Marion, and Guillermo Solano-Flores have served in this role.

        The NCME Board will be nominating candidates again this year. NCME members who are interested in being considered for nomination should send their vita to ncme@ncme.org by September 30, 2024. Letters of support are welcome but not required.

        Nominations for NAGB are due by November 1. Those interested in self-nominating or nominating others independently can find additional details by visiting NAGB's website.

        For more information, please contact the NCME Board at ncme@ncme.org.

2. Enhancing Automated Scoring Engines with Generative AI: A Dual-Study

October 9th, 2024 | 4:00 PM ET
Presenters: Justin O. Barber and Edward W. Wolfe, Pearson

Automated Scoring Engines (ASEs) are revolutionizing the scoring of constructed response (CR) assessment items through artificial intelligence. A major challenge in developing ASEs, however, is the limited number of training examples, particularly for infrequent score categories, driven by high scoring costs. Data augmentation, particularly using generative AI, is emerging as a solution to this issue. In our first study, we investigated multiple data augmentation methods and their impact on ASE performance. Simulated CR responses were created using these techniques to expand training datasets. The second study specifically focused on generative AI's effectiveness for data augmentation, evaluating ASE performance using metrics such as score point recall and quadratic weighted kappa to measure agreement between human and ASE scores. Our findings show that ASEs trained with generative AI-augmented datasets can achieve performance equal to or even surpass human raters. This presentation will delve into the methodologies, key results, and implications of both studies, highlighting how generative AI can enhance the reliability and accuracy of automated scoring systems.

Join us for the AIME meeting on October 9th at 4:00 PM ET!
Zoom link: Join here
Meeting ID: 817 2844 3502 | Passcode: 649132

To learn more about AIME, visit ncme-aime.org or sign up at NCME.org. We hope to see you there!

Chris Ormerod, Co-Chair | John Whitmer, Co-Chair | Maggie Beiting-Parish, Secretary

NCME Committee Updates

Announcement from the Classroom Assessment Committee

 

Submit Classroom Assessment Scholarship to NCME Annual Conference

“If we could have a superpower to change the field of psychometrics, we would ensure it originated from what students learn in the classroom rather than from “experimental psychology and psychophysics laboratories” (Sireci, 2020, p.1). The present moment is the best time for us to bring educational measurement closer and make it more useful to the classroom. How can we do that? The answer is simple and straightforward: submit your work on classroom assessment at the annual conference of NCME. Next year’s conference will be held from April 23 to 25, 2025, in the breathtaking city of Denver, Colorado. You can find more information about the conference here. 

 

The Classroom Assessment Committee within NCME will work closely with the conference program co-chairs to ensure we have ample space in the conference program for high-quality sessions on this important topic. We look forward to receiving and reviewing your proposals on classroom assessment for the annual conference in 2025. We strongly believe that together we can increase the presence of research on classroom assessment at NCME, taking the first steps to bring our field closer to the classroom.

 

Register for the NCME Special Conference on Classroom Assessment

We are pleased to announce that the 5th NCME Special Conference on Classroom Assessment will be held in person this year on September 19-20, 2024, at the Palmer House Hotel in Chicago, IL. The conference theme is “Reclaiming the Promise of Balanced Assessment Systems: Achieving Deeper Learning at Scale for Both Students and Adults.” This year’s special conference will be co-hosted by the University of Illinois at Chicago and the Chicago Public Schools, with support from the Center for Assessment. These hosts continue NCME’s longstanding commitment to improving assessment practices that privilege deeper learning for all, and that happen directly between teachers and their students. Details and registration at https://www.ncme.org/event/classroom-assessment/upcoming-meeting 

 

Sign Up to be a Friend of the NCME Classroom Assessment Committee

The NCME Classroom Assessment Committee is updating our “Friends of the Committee” list – and we’d like to hear from you! Friends of the Committee include faculty, graduate students, and other colleagues interested in providing ad hoc support of the Committee’s work to enhance equitable and high-quality practice of classroom assessment in and beyond the NCME community. Friends of the committee support our work in classroom assessment based on their interests, passions, and areas of expertise. If you’re interested in volunteering, please fill out this short (2 minute!) survey to get started. If you have questions about the Classroom Assessment Committee or how you can get involved, please reach out to Co-Chairs Dr. Dustin Van Orman (vanormd2@wwu.edu) and Heidi Andrade (handrade@albany.edu).

 

Access Equity and Assessment for Learning Resources

In the fall of 2023, the Classroom Assessment Committee held a webinar series titled “Equity and Assessment for Learning.” Bringing together researchers, practitioners, and experts in culturally sustaining assessment, this two-part series explored the role of classroom assessment in promoting equity and how to develop professional learning for more equitable formative assessment. Access the recordings Here https://www.ncme.org/event/classroom-assessment/upcoming-meeting191 and review the resources that were generated through this collaboration.

Announcement from the Committee on Diversity Issues and Testing 

Committee Chair: Dr. Merve Sarac

        As the new chair of CODIT, I am excited to share that the NCME Pipeline Scholarship program has seen a surge in applications for 2024. This program provides travel scholarships to doctoral students from underrepresented ethnic minority groups, enabling them to present papers at the NCME annual meeting. The growing interest in the scholarship program is inspiring, and we are committed to further supporting talented scholars in the years ahead.

        Looking ahead, the CODIT committee will again collaborate with the Graduate Student Issues Committee to organize a reception for Historically Marginalized Groups at the 2025 NCME annual meeting. This session aims to capture diverse perspectives within the measurement community and foster innovation in our field.

Announcement from the Mission Fund Committee

Committee Chair: Dr. Jonathan Rubright & Dr. Stella Kim

1. Selected Projects Showcase at 2024 NCME Annual Meeting

        During the 2024 NCME Annual Meeting, a dedicated session celebrated the outstanding individuals who have received our Mission Fund and Honorable Mention awards over the past two years. Four funded projects showcased their remarkable work, which have embodied our steadfast dedication to excellence and innovation. The showcased projects included:

  • "Developing and Evaluating Score Reports Adapted for Parents Whose Primary Language is not English" by Samuel Dale Ihlenfeldt (University of Minnesota),
  • "A Guiding Framework for Effective, Equitable Classroom Assessment for Teacher Practice and Student Success" by Dustin Van Orman (Western Washington University), Erin Riley-Lepo (The College of New Jersey), and James McMillan (Virginia Commonwealth University),
  • "Validation and Implementation of a School-Based Gender Identity Screener for Children" by Matthew Diemer (University of Michigan),
  • "Beyond Group Comparison: Accounting for Intersectional Sources of Bias in International Survey Measures" by Rujun Xu and James Soland (University of Virginia).

        These projects not only significantly advanced the field of education but also demonstrated a profound commitment to addressing pressing challenges and fostering inclusivity.

2. 2024 Mission Fund Recipient Announcement

        The Mission Fund Committee is excited to announce the recipient of the 2024 Mission Fund award: Dr. Brian Leventhal at James Madison University, for his project titled “Mirrors and Windows: Creating Videos to Promote Recruitment in the Measurement Profession.” This project will use qualitative narrative research techniques to capture the diverse stories of individuals and ideas within the field, transcending the confines of conventional norms.

        The primary objective of this project is to amplify the voices of professionals whose experiences may diverge from the traditional narrative. Through in-depth interviews, approximately 20 participants will share how their unique journeys have influenced their work in measurement and their efforts to foster inclusivity and social justice within the profession. These interviews will be conducted in person at the 2025/2026 Annual Meetings of NCME. The culmination of this endeavor will be a dedicated webpage hosting the interview videos, complemented by resources for students, including a directory of measurement-focused graduate programs.

         Join us in anticipating the insights that this project will unearth and share with our NCME community. Stay tuned for updates on its progress and impact!

Educational Resources

Review the Meeting Recording: NLP for Enhancing Student Learning Outcomes

On May 8, 2024, Rose E. Wang from Stanford University presented on "Natural Language Processing for Improving Student Learning Outcomes," organized by the NCME Artificial Intelligence in Measurement & Education (AIME) SIGIMIE.

Dr. Wang highlighted her empirical language-based approach using NLP to enhance educational language at scale. She discussed Bridge, a method using cognitive task analysis to translate expert thought processes into decision-making models for remediation. Bridge-guided large language models (LLMs) outperformed novice tutors and LLMs alone in blind tests.

Dr. Wang also introduced Tutor CoPilot, a real-time decision aid using Bridge, which showed promising results in an ongoing randomized trial. Students working with Tutor CoPilot-equipped tutors reported significantly more positive tutoring experiences.

A recording of the event is available for those who missed it or wish to review the content. Access the meeting recording here.

Mastering the Job Market: A Guide for Educational Measurement Graduates on Resumes, Networking and Interviews

Effective job hunting for graduate students in Educational Measurement involves strategic preparation, from leveraging internships at organizations like Cognia, NCSBN, and ACT, to honing interview skills and managing a robust online presence. This guide offers comprehensive insights into creating impactful resumes, utilizing recommendation letters, and maximizing LinkedIn profiles and personal websites for academic and professional success. Embrace the journey with resources and tips to navigate the unique academic job market, underpinned by a systematic approach that mirrors the precision of your field. For further details on each aspect of your job search, click here.

NCME Announces Updated Listings of Graduate Programs in Educational Measurement

The National Council on Measurement in Education (NCME) has updated its comprehensive listing of educational measurement graduate programs, a crucial tool for guiding students and connecting employers with potential candidates. Originally established in 1990, the latest 2019 revision continues to catalog programs across the United States and Canada, aiding in the growth and recruitment in this critical educational sector.

Download the Program Book.

Call for Papers - CEJEME Journal
 

CEJEME Special Issue on AI and Machine Learning in Measurement

The Chinese/English Journal of Educational Measurement and Evaluation (CEJEME) is issuing a CALL FOR PAPERS for the upcoming special issue on AI and machine learning in measurement.

CEJEME is a newly established, bilingual journal of the National Council on Measurement in Education (NCME), co-sponsored by Beijing Normal University, China (ISSN: 2837-0899). CEJEME publishes articles on cutting-edge topics in educational measurement, evaluation, and policy. The journal aims to cater to a broad, applied audience and provides a platform for intellectual exchange on a wide range of methodological, ethical, and policy issues within an international and technology-driven context.

Each article is published online in two languages (Chinese and English) at no expense to the authors and is freely available to the public. All published articles have DOIs and are therefore immediately visible in the research publication ecosystem. Manuscripts can be submitted in either Chinese or English and are peer-reviewed in the language of submission. We guarantee a peer-review decision within four months from the date of submission.

You may browse all past issues on the CEJEME journal website, for example, the inaugural issue and issues published last year.

Special Issue on AI and Machine Learning in Educational Measurement

CEJEME is inviting submissions of articles that discuss topics related to AI and/or machine learning in educational measurement. We welcome a variety of article types, including typical research papers, commentaries, software exchange notes, or other innovative formats.

Please complete the Form of Interest by July 31, 2024 to provide brief information on your intended submission. We will reach out to invite full paper submissions by August 5, 2024. Full paper submissions are due October 14, 2024. The special issue will be published online by the end of December 2024.

We look forward to receiving your form of interest for this special issue of CEJEME. For further information on this special issue or your questions, please feel free to contact the co-editors, Dr. Yi Zheng at yi.isabel.zheng@asu.edu or Dr. Okan Bulut at bulut@ualberta.ca.

To be continued...

Upcoming Events

1. Examining Automated Writing Evaluation Tools

July 10th, 2024, 4:00 pm ET

Presentation by Corey Palermo from Measurement Incorporated and Joshua Wilson from the University of Delaware

Join us for an enlightening session where Corey Palermo from Measurement Incorporated and Joshua Wilson from the University of Delaware will delve into the realm of Automated Writing Evaluation (AWE) tools. This session will commence with a comprehensive overview of various AWE research lines. Subsequently, the speakers will summarize the results of two focal studies: one investigating Elementary English learners' engagement with automated feedback and another examining the causal impact of AWE on student writing outcomes. The session will conclude with implications and recommendations for researchers and developers of AWE tools, followed by an opportunity for questions and discussion.

To attend the AIME meeting on July 10th at 4 pm EST, follow the Zoom link.

Meeting ID: 817 2844 3502

Passcode: 649132

To learn more about AIME, visit ncme-aime.org or to sign up, please visit NCME.org.

2. NCME Special Conference on Classroom Assessment

Registration for the 2024 Classroom Assessment Conference is now open!

Date: September 19-20, 2024

Location: Chicago, IL

        The 5th NCME Special Conference on Classroom Assessment is scheduled for September 19-20, 2024, at the Palmer House Hotel in Chicago, IL. Themed "Reclaiming the Promise of Balanced Assessment Systems: Achieving Deeper Learning at Scale for Both Students and Adults," the event is co-hosted by the University of Illinois at Chicago and Chicago Public Schools, with support from the Center for Assessment. The conference upholds NCME's commitment to enhancing assessment practices focused on deeper learning.

        This event continues the tradition of the NCME Special Conference on Classroom Assessment, previously hosted by the University of Kansas, University of Colorado, Boulder, University of Virginia, George Mason University, Virginia Commonwealth University, and College of William and Mary.

Registration Fees

On or Before August 16, 2024:

  • NCME members: $300
  • Non-NCME members: $425

After August 16, 2024:

  • NCME members: $325
  • Non-NCME members: $450

For questions about your registration, contact the NCME Office via email at ncme@ncme.org.

About NCME

Explore the National Council on Measurement in Education (NCME), a community dedicated to advancing educational measurement theory and practice; learn more about our mission and goals.

Suggest News

If you have a newsworthy topic to share, don't hesitate to get in touch with News@NCME at news@ncme.org.