The first article in this three-part series on artificial intelligence (AI) use cases in learning and development (L&D) explored using AI for content development. Here, we’ll discuss using AI to assess learners’ skills before, during and after training.
Assessing employees’ skill levels has always been an important part of delivering targeted training programs that close key gaps. However, accurately (and efficiently) assessing skills and competencies isn’t easy — especially for learning leaders working in large organizations.
Training Industry’s L&D career research shows that less than 10% of training professionals consider themselves excellent at either assessing needs or measuring impact, and more than 40% of training professionals say they need to improve their skills in both areas in order to advance their career.
Assessments are not a one-and-done activity. They need to be delivered at every state of the learning journey to effectively identify and close skills gaps. In other words, the assessment process doesn’t end with pre-training assessments. That’s only the first step. Learning leaders must also gauge learners’ progress and understanding of key concepts during the learning experience and utilize post-training assessments to determine if the learning intervention was successful in meeting learning objectives and bridging the initial skills gaps identified.
Thus, there’s three key types of assessments that training professionals need to create and deliver:
- Pre-training assessments.
- Mid-training assessments or knowledge checks.
- Post-training assessments.
Let’s explore how AI can help simplify the assessment process and drive better training (and business) outcomes.
Pre-Training Skills Assessments
Pre-training assessments are an important tool for determining learners’ current skill levels. However, learning leaders — and learners, themselves — may struggle to connect assessment results to relevant courses and programs that can help improve the identified areas for improvement, says Jessi Schue, market research analyst at Training Industry. AI can help pinpoint where learners should focus their skill building through custom learning paths. “AI has the power to understand learners’ assessment results on a broader scale, helping to guide learners on what content they should work with to build these skills based on the individual’s learning preference. AI can also provide feedback on key areas for improvement.” Ultimately, AI has the power to see skill strengths that we may miss, helping learners and their managers have a deeper understanding of the individual’s skill set and potential for growth, Schue says.
There are many different tools with AI-driven assessment capabilities. For example, Acorn, a performance learning management system (PLMS), uses AI to assess key skills and capabilities, generate reports, conduct performance assessments and to create and evaluate competency models. Doing this manually takes “a lot of administrative work,” says Keith Metcalfe, Acorn’s president.
For large organizations, evaluating and tracking skills and competencies takes time and can quickly lead to multiple Excel sheets and tabs. Although many organizations want to leverage competency models for more impactful learning, learning leaders can quickly become overwhelmed by the process of assessing employees’ skills, competencies and behaviors, which can deter them from getting started. AI can help automate this initial assessment process, which is a critical first step in developing effective competency models.
Metcalfe says that AI makes assessing and tracking learners’ skill and capability levels “much smoother and easier” from a user perspective. He also says that Acorn isn’t focused on leveraging AI for AI’s sake. Rather, they’re looking at real-world challenges for companies — like creating competency assessments — and seeing how AI can help.
Writing learning objectives:
AI can also help write learning objectives based on the skills gaps identified through pre-training assessments.
For instance, say that you’re creating a leadership development course and have identified that giving feedback is a challenge for most managers who will be taking the course. Simply write a generative AI prompt like: “Write a learning objective around improving managers’ ability to provide objective and actionable feedback for a leadership development course I’m developing.”
When plugged into ChatGPT, the response for this prompt was:
Learning Objective: By the end of this course, managers will demonstrate an enhanced ability to provide objective and actionable feedback to their direct reports, resulting in improved employee performance and professional development outcomes.
Mid-Training Assessments and Knowledge Checks
Mid-training assessments help determine learners’ understanding of the course material they’ve consumed so far and can provide helpful feedback to help them course correct if they need to. AI can help create these assessments in the form of multiple-choice or short-answer quizzes, gamified assessments, role-plays and more.
For example, a salesperson completing a course on a newly launched product might complete a role-play with an AI-generated avatar acting as a customer, where they have to answer frequently asked questions (FAQs) about the product they’re learning about. Based on the interaction, AI can deliver helpful feedback on their performance and highlight any areas they might have missed.
Apratim Purakayastha, Skillsoft’s chief technology officer, says AI is fueling more interactive and experiential assessments. Skillsoft’s CAISY, for example, is designed to simulate real-world conversations using AI-generated digital agents.
Skillsoft has always been a strong believer of verified learning, Purakayastha says, and uses objective assessments before, during and after training to ensure its content is effective in targeting key skills. For instance, the platform can assess learners’ progress during a course by delivering flash cards right to their mobile phone. Or it might use a multiple-choice or matching-style quiz.
In the past, Skillsoft’s assessments were created by humans, Purakayastha says. However, “For the past two years, we’ve been using various types of generative AI technologies to generate assessments. We feed it our transcripts [and] other course metadata, learning objectives and outcomes assessments.” That said, they still “keep a human in the loop.”
By using AI to generate assessments, learning leaders can focus more on improving their quality. AI can help with this process, too. For example, AI can flag when over 80% of learners get certain assessment questions wrong, Purakayastha says. Often, when a large percentage of learners receive low assessment results, it’s because the assessment is misleading or low-quality. AI can be used “as a feedback loop” to judge the quality of assessments by tracking usage data and providing meaningful insights to L&D professionals.
Post-Training Assessments
Post-training assessments help learning leaders determine whether training was successful in bridging the skills gaps identified during the pre-training assessment phase.
Here are some examples of post-training assessments that AI can help generate:
- Open-ended questions with AI scoring: AI can design open-ended questions that assess beyond recall. It can analyze learners’ responses based on relevance to the prompt, use of research and examples, depth of understanding, clarity and other parameters.
- Post-training surveys: AI-generated post-training surveys allow learners to provide feedback on the learning experience and help you gauge their confidence and ability to apply what they’ve learned.
- Tests and exams: For more comprehensive programs, such as certifications, exams can assess learners’ understanding of the material covered in the certification program, such as key concepts, frameworks and models and best practices for application. AI can help by writing test questions in multiple formats (i.e., multiple-choice, short-answer, matching definitions, etc.).
Parting Thoughts
In conclusion, “AI’s ability to give adaptive guidance and offer a broader understanding of an individual’s skill level helps organizations utilize this feedback more effectively and efficiently,” Schue says.
With AI-powered assessments, organizations can better identify skills gaps and address them through targeted training programs. This benefits learners, who receive more personalized development opportunities, and the business, which benefits from a more skilled and adaptive workforce.
