Diane Gaa, WELEARN
I was recently introduced to NCME by a colleague (Sara S. Hennings, PhD), and I was immediately intrigued. An entire organization devoted to educational measurement – as a learning leader, I am always on the lookout for organizations, individuals, and resources that advocate for the measurement of the impacts and effectiveness of learning.
I come from a corporate learning and development background. As a corporate learning and development leader within a centralized learning function for a global organization with over 40,000 employees, my team and I were responsible for guiding investment decisions for licensed content, and what technologies we use to support our efforts. How do we roll this out to our employees across the globe? – and when it was all said and done – we had to answer the question “What did we achieve? have?” In other words, how do we measure the impact?
Corporate Learning and Development, the Next Frontier for Educational Measurement
If you are not familiar with the space of corporate learning (or training and development), let me share with you some basic facts. Corporate learning is broadly defined as those activities which an organization undertakes to build the skills and capabilities of their employees. There is always a significant amount of debate regarding what is the role of the organization versus the role of the educational system in the development of skills. However, that is not the subject of this article.
While the level of spending fluctuates from year to year, according to Training Industry, Inc., it is estimated that organizations in the US will spend approximately $172 billion a year in corporate learning in 2020. In addition to this, approximately $20 billion will be spent in corporate-sponsored tuition assistance (educational assistance for employees to get their degrees). Despite all of this spending, it is very hard for organizations to measure the effectiveness of the learning they provide to their employees.
So, you may ask, why is that?
Many may be asking how organizations can spend that much money and not know the effectiveness of the programs. That is, of course, the $192 billion question– and as you might suspect, it is complicated.
The longstanding measure of the effectiveness of learning spending has been the Kirkpatrick Model. Invented in the 1950s by Donald Kirkpatrick. (Kurk, 2016) The Kirkpatrick Model proposes that there are four levels to evaluate the results of training and the impact of learning.
- Reaction (Did they like it?)
- Learning (Knowledge Assessment)
- Behavioral Change (Are they doing something differently?)
- Results (What happened as the result of behavior change?)
As you can imagine, trying to measure across all four levels of learning is intensive and can be intimidating for many learning and development organizations. Add to the complexity of this is the need to be able to isolate all other factors to determine the exact impact of learning.
What is the Challenge for the NCME Members?
There are qualitative ways to summarize the overall impact of the training. However, creating a measurement model that could quantify individual learning from training would have a far-reaching impact on the field of training and development in corporations.
Organizations have data about their programs and the performance of their employees, yet there are no clear measurement models applied to the data that have been used to understand the effectiveness of the training programs.
The challenge I give you is to create a Special Interest Group focused on the measurement of learning in corporations and to invite employers to the table with their data to create the analysis process and to provide insights that could lead to new opportunities for organizations to support their employees through learning.
Kurt, S. “Kirkpatrick Model: Four Levels of Learning Evaluation,” in Educational Technology, October 24, 2016. Retrieved from https://educationaltechnology.net/kirkpatrick-model-four-levels-learning-evaluation/