You have probably heard of data mining and how analyzing your training data can make you a better training professional. I’m sure you’ve used Kirkpatrick’s levels of evaluation, Phillips’ return on investment (ROI) or similar training evaluation methods. Evaluation is a vital part of an effective learning and development (L&D) program — no argument there.
However, even the best evaluation methods suffer a significant disadvantage: You often measure training’s effect after delivering the program. You could use a survey in the middle of a course or quick surveys throughout the course delivery, or you could observe participants in a simulation to measure the effect of training on behavior change.
What else can give you real-time feedback on how your participants progress through your training courses?
The Event Logs Goldmine
If you use online training platforms such as a learning management system (LMS), you have a goldmine of information from the event logs the LMS collects. Event logs are used for security, but the same data can also offer valuable insights into user behavior.
How do you analyze the event logs’ data? Through process mining.
How Process Mining Works
The LMS event logs’ formats vary, but the information they collect is mostly the same. By seeing when learners access a course, which parts of the course they access, how long they work with activity and when they exit the course, you can see their paths through the course. Using process mining software, you can create a picture of how multiple learners interact with the course.
According to research by Alejandro Bogarin, Rebeca Cerezo and Cristobal Romero, education professionals have used educational process mining to:
1. Detect Problems With Course Design and Flow
Maybe the process mining diagrams show bottlenecks in the course flow, or there is the pass rate for one assessment is significantly lower than the rates for other assessments. You can use your process mining charts to pinpoint difficulties in the course and correct the issues in real time.
2. Map How Learners Interact With Each Other
In analyzing course event logs, you can see the days and times when most of the learners interact in discussion boards. Event logs can also show you which learners are discussion leaders by counting how many replies they receive. This data can be useful in designing collaborative assignments by balancing group dynamics.
3. Test Course Topics and Assignments
Measuring how long learners take to complete an assignment can help you adjust the assignment’s difficulty level. For example, I changed the length of my course videos based on how much time my students spent viewing them. I used to record videos that were 20 to 30 minutes but found that the students only watched the first five or six minutes. Based on this analysis, I created more videos but capped their length at six minutes or less. I covered the same concepts but in a shorter, more accessible format.
The Power of Process Mining
There are many ways to teach yourself about process mining. A quick search on YouTube brings up many process mining tutorials, and there are several process mining software platforms with a free beginner’s plan. It is also easy to extract event logs from many LMSs.
Data mining has given L&D professionals significant insights into the value of training for participants and organizations. Process mining takes data mining one step further, providing more ways to make learning better.
