Evolving Learning Professionals and Evaluation Practices

Q: In your view, how have the roles of learning professionals in organizations evolved in recent years, especially in the context of the learning team?

A: I’ll answer generally—it’s evolved in a way that has been very positive and much needed. The pandemic drove that change. We’re accustomed to what my dad did – in person, stand up training. So that was a big evolution, the shift to virtual work and virtual training. We’re still doing remote and blended learning, along with in-person. In-person learning can still build relationships that have more heart. But it’s not essential to do only in-person training.

The slower evolution is that learning professionals are becoming more strategic partners. I’m using the word slow evolution for that. I think our industry is still holding out hope that if we train learners better then learning professionals will eventually win the day. We’re a bit slower to accept the fact that we need to be performance professionals as well as learning professionals in order to get to those strategic people results.

Q: What do you think is stopping people from tracing their effectiveness all the way to the impact of the business?

A: I picture our level three—behavior, the application on the job—as having barbed wire around it. And there are signposts on the wire that say: “Danger, keep out, mind your own business.” There may be minefields there and electrical shock risks, so I think we’re hesitant for some really good reasons.

I’ve encountered line managers and supervisors who seem to think we’re there to poke around in their business, find out what they’re not doing well and make them look bad. They’d rather keep the learning team out. On the other hand, we’re also having senior leaders saying to focus on results and get to the results quickly.

So, we can honor that desire to focus on results by rooting our efforts in the organization’s mission and strategic plan. It’s certainly not as easy as pushing buttons to send out surveys, but it leads to much greater value.

Q: What do you say to learning professionals who tell you they want to evaluate more, but their leadership just says, “We know it works; just do your thing.” It’s like they just want to skip it under the assumption that things are good.

A: I’m glad you use the word assumption. Because when leaders say, “we’re fine, just keep doing what you’re doing,” unfortunately what is happening is that eventually senior leaders determine that the training is not adding enough value for the cost.

It’s almost a given that training will be the first to get cut when the budgets need to tighten. But it doesn’t have to be. When we get cut, at least some of the time, it is because we are not working in the world of performance like the rest of senior leadership who are demonstrating value. I think they see cost center cost center, that the learning is disconnected from performance and the results. I think they fail to understand that feeling good after a training session is not the value that you’re aiming for.

Here’s an example. I work with some people at a major corporation out west, who the other day said they went before the board of directors, and the board wanted to see how learning was adding value. The learning team came in and talked about their corporate university and shared smile sheet scores and test scores. They explained that their assessments were improved and they could show gains between pretest and posttest. And, the board of directors asked, “Well, what does that have to do with the performance on the job?” The learning team said “Well, that’s not our job.” Then the board asked them “What does your training have to do with business?” The learning team responded “Well, that’s not our job either. If you want to know about that, why don’t you interview the supervisors and the senior leaders? That’s their job.”

If you find yourself shaking your head then you know why it’s easy for leadership to see the learning function as having costs that exceed your value, and that makes learning a prime target when it’s time to make significant cuts. Now, learning leaders are incorporating numbers and talking to senior leadership themselves. As an industry, we even name ourselves for our activities (learning) instead of our outcomes (performance). We really need to shift our thinking.

Q: Every so often, I notice from the instructional design perspective that someone comes out with a “brand new” instructional model. And if you look at it, it is literally just ADDIE with some new words being used. Do you find people trying that with Kirkpatrick?

A: I was on a plane with a senior salesperson from a major electronics retailer not too long ago, and he was talking about training and learning and he said, “We can call that lipstick.” I asked him to explain that. He said, “well, we keep kind of dialing it up and giving it a new look. It used to be training and then learning, then competencies, and now upskilling. But it’s still the same thing and it’s still disconnected from us.”

In the world of evaluation, there are variations that build off Katzell’s work, but they make it more complex. We have found that the complex does not execute very well. They may be using different names for the levels. What it still comes down to is that most of the world are doing smile sheets, your Level One. They do pre- and post-testing for Level Two. They do a 90-day survey for Level Three. And, really, Level Four is kind of out of their reach. And they call those levels all different names. But I can tell you that it’s not the Kirkpatrick model. That may be like my dad’s thinking in 1954 but it has since then other practitioners have turned into something that; it just is ridiculous to think that you’ll be able to get Level Two to Four data and information from a smile sheet.

Q: Or you’re still missing the connection to “Did it have the intended effect?”

A: You mentioned the word intent. I think that is really a key word is that we want to find out what the intent of senior leaders are line managers. And then what we want to do is instead of just measure it, a lot of people think evaluation is measuring something that has happened after class 90 days down the road. But what we try and do particularly with Level Three on the job behavior, the key word is influence we want variety of methods to influence the behavior so that we have noticeable results at 30-60-90 days.

Better than a smile sheet, which usually have low response rates, and on which people can lie to “be nice” to us, these other approaches yield worthwhile results. So that really is the key to the bridge. What we’re trying to do is transform learning professionals to learning and performance professionals. And influencing is more impactful than relying on human nature and tradition because we’re not that adaptable.

Q: I’ve seen in different organizations sometimes that just learning is just its own thing, but I’ve seen in other organizations where learning is treated like a change management and a marketing initiative. And they think ahead about what behavior change they’re looking for. And they consider their learners like stakeholders and try to track where their thinking is and where their hearts and minds are. What has been your experience?

A: That is an incredibly positive evolution. What learning leaders are doing now is spending time talking to line managers, building those relationships, and finding out what are the performance gaps and performance opportunities and plan to look at Level Four results. This reminds me of Stephen Covey – that we should seek to understand before we seek to be understood. By building a good trusting relationship, which is the foundation for becoming partners with business leaders.

It’s especially helpful when you have a leader who’s at the top and they understand the value of this and they bring everybody else up with them. You still see learning leaders who feel pressured to produce volumes of learning content. I like to encourage them with what I call PLLI – Protect and Leverage the Learning Investment. What we’re basically saying is, you keep shelling out millions of dollars for training. The research clearly says if people don’t apply it soon after training, it’s going to be forgotten. You’d be better off writing a check to a charity.

How do we protect that learning investment? Protect it by making sure that when people are done with training, that they don’t just go back to work and hope for the best but there’s somebody or something waiting for them when they get back from these training experiences. We want their leaders and coaches to welcome them back and work with the learners to start using what they’ve learned right away. It’s an important part of the change for leaders to pull it through.

I have some usual questions around this. One is how do we show value? The other is how do we get buy in? That word buy in is so viral. And a lot of people have different definitions of it. Paying for the training is not enough to show buy in.

The Kirkpatrick definition of buy in is that it is shown as action –the senior leaders are part of the solution. They help remove barriers and then champion people that are doing well and challenge the leaders who are not doing what they’re supposed to be doing. And the same with line managers. We want to get them to be part of the team that is influencing the behavior. I would even encourage use of language like “sign on.” They sign on because they know this is a mission critical (leadership, onboarding, whatever it is) initiative and we help them and guide them on how they can be part of the solution. Both line managers and senior leaders are important and recruiting them is part of the overall change management strategy.

Q: How does the new world Kirkpatrick model differ from the original and how has this transformed the approach toward training evaluation?

A: We take another message from Stephen Covey. “Start with the end in mind”. What are the performance expectations? And then let’s start putting our training and our learning together. The other thing is what I’ve mentioned before, is Kirkpatrick Level Three is not about measuring what has happened. It is about influencing what has happened and certainly tracking what is happening.

The other thing we talk about a lot is continuous improvement. Consider that if people do a smile sheet at Level One, a test at Level Two after 90 days, a survey at Level Three and hope for the best at Level Four—how can there be continuous improvement if there’s no continuous evaluation along the way? We’re really monitoring and adjusting, with our fingers on the pulse during the performance.

I think one of the hallmarks of the New World Kirkpatrick Model is spending time on the other side of the bridge, working and listening to the people leaders, speaking their language (performance metrics), not using the L word (learning) with the line managers and senior leaders, but talking about performance improvement.

Q: You mention building and leveraging business partnerships. Can you elaborate on why these partnerships are crucial for the success of learning initiatives?

A: It’s these partnerships that are crucial in life. If you build trusting relationships, where you can be honest with each other, you can build a mutual understanding –from which you can achieve a lot. Key to that is resisting the temptation to start peddling your wares. Mirroring what you’ve heard, like you’ve done during this conversation, helps validate the clear understanding build that trust even more.

What we’re saying is we’re involved in a conversation. You trust me, I trust you. And once that trust is developed, the topics and the content and the issues become easy. Because we have already developed the kind of relationship that can deal with whatever content comes up.

The data will tell the story of the value to the business. That’s much more impactful than anyone trading on their name to assert the training was valuable. Another thing that we do is work out the formula of actions that will contribute to the desired outcomes. The learning experience is just one part. There’s mentoring, coaching, and providing the needed resources for success. Just like attorneys we work with at the Department of Justice. They win their cases, not by asserting their view, but by creating a chain of evidence that shows how it came to be. You can’t expect people to just believe you. You’d have to show them the connections in a compelling manner. And we never say we’re proving anything. Proof is a scary term to many people. We talked about a preponderance of evidence, which is a civil criterion for success, and that we’re never the sole cause of it. We contribute. Then, if the people don’t do the behavior change, it doesn’t happen.

When we’re approached by someone who has a definitive solution in mind (specific kind of training), as a trusted partner we can share with them the tradeoffs of what they will or won’t get from the approach. We suggest other options they should consider. This is especially important when a new trend is sweeping people away and a learning leader thinks that using this modern technology or approach will impress top leaders.

Q: How should organizations adapt their learning strategies to use AI tools and enhance learning outcomes?

A: Organizations should embrace technology, especially AI, for its ability to quickly provide data and aid in making quick decisions. They need to focus on integrating technology not just for training, but also for performance, accountability, and support. Learning professionals should evolve into learning and performance experts, leveraging technology to support and encourage employees. The emphasis should be on using technology for quick, targeted, and practical learning solutions. For example, the shift from lengthy classroom training to concise, performance-based videos that address immediate needs. This approach has been adopted in various sectors, including the military, where learning is made more relevant and contextual, like the US Navy’s ‘Ready relevant learning’ initiative.

Q: How do you foresee the landscape of Workplace Learning and Development evolving over the next five or ten years?

A: In the next five to ten years, technology will likely reduce the need for expensive human interaction in training and development. The focus should be on building relationships and trust, something technology cannot do at this point. The roles in Learning and Development will shift towards those who can build partnerships and understand the nuances of human interaction and trust-building. Traditional job roles in this field might significantly change or become obsolete. Also, there might be a contraction in the number of content producers due to the acceleration of content creation by AI tools. However, professionals need to exercise discernment, as AI, while efficient, can still reflect human flaws in its outputs.

Q: If you could go back in time and give your younger self career advice, what advice would you give?

A: I would advise my younger self to have more conversations about career choices and personal strengths. Instead of jumping from one degree to another, seeking feedback and guidance would have been more beneficial. This approach would have helped in understanding what contributions I could make to people’s lives and finding the right path more efficiently. Feedback, often viewed negatively, should be sought out actively for personal and professional growth. It’s also important to explore different professions and opportunities before committing to a specific educational path, as understanding what a field truly entails is crucial before making significant investments in education. Employers are increasingly looking at abilities and performance rather than just degrees, making practical experience and skills more relevant.