Learning design
How to measure training effectiveness: metrics, formulas and a one-page plan
Training has worked when employees can do the task without help and the work shows it. This guide gives six measures for software training, each with a formula, a worked sum and the place the data comes from.
Updated
Research sources checked on 21 November 2026. The rollout figures are made-up examples, not benchmarks.
The usual frame is the Kirkpatrick model, which sorts evidence into four levels from reaction to results. That page explains the levels and the ROI sum. This page is the working version for software training: which numbers to collect, how to calculate them, and whose system they come from. In Kirkpatrick's terms the task check is level 2; errors, tickets and adoption are level 3; the business figure they feed is level 4.
The training effectiveness formula people look for
There is no single formula. Any "training effectiveness index" you come across is a weighted mix of the measures below, and the weights are somebody's opinion. What you can calculate honestly is a small set of ratios, each answering one question.
| Measure | Formula | Where the data comes from |
|---|---|---|
| Completion rate | People who finished ÷ people assigned × 100 | LMS report |
| Pass rate on an unaided task check | People who passed ÷ people who attempted × 100 | Scored simulation, marked exercise or observed check |
| Time to proficiency | Working days from start date to the day the standard is met | Workload reports and team lead sign-off |
| Error or rework rate | Transactions corrected or reversed ÷ transactions completed × 100 | A report from the system itself |
| How-to tickets per 100 users | How-to tickets ÷ users × 100 | Service desk, by category |
| Process adoption | Work done the new way ÷ all work of that kind × 100 | A report from the system itself |
The worked sums below follow one invented rollout: 400 finance and procurement employees moving to SAP S/4HANA, with go-live on 1 March.
Six training effectiveness metrics, with worked sums
1. Completion rate
372 of the 400 assigned users finished the course before go-live. 372 ÷ 400 × 100 = 93%.
Completion measures reach. It shows who the training got to and, more usefully, the 28 it missed. Report it by team so that each manager has names to chase. It carries no information about ability, so never report it alone.
2. Pass rate on an unaided task check
A task check asks the person to carry out the task with no hints and scores the result against a pass mark. Of the 372 who attempted it, 301 passed first time and 356 had passed by go-live.
- First-attempt pass rate: 301 ÷ 372 × 100 = 81%
- Pass rate by go-live: 356 ÷ 372 × 100 = 96%
- Assigned users proven able: 356 ÷ 400 × 100 = 89%
Give the steering group the last figure, because it counts the people who never turned up. The first-attempt figure is for you: a task with a low first-attempt rate needs better teaching or a simpler process. The check can be a scored simulation, a marked exercise in a training client, or a supervisor watching with a checklist. Show me, try me, test me covers the format, and there is a buyer's guide to scored software evaluations.
3. Time to proficiency
Define proficient in figures a report can show. For an accounts payable clerk that might be 20 invoices posted a day with under 2% reversed, held for five working days. Time to proficiency is the count of working days from go-live, or from a new starter's first day, to the day that run is complete.
Five clerks got there in 9, 11, 12, 14 and 31 working days. The median is 12. The mean is (9 + 11 + 12 + 14 + 31) ÷ 5 = 15.4. Report the median, so that one person who was off sick does not move the headline, and look at the 31 separately.
4. Error or rework rate
Pick a correction that the system records by itself: reversed invoices, cancelled orders, reopened records. In week 2 after go-live, 341 of 6,200 invoices were reversed: 341 ÷ 6,200 × 100 = 5.5%. In week 8 it was 97 of 6,450: 97 ÷ 6,450 × 100 = 1.5%.
Split the rate by task and by team. A rate that is high in one team points to a training gap. A rate that is high everywhere usually points to the process or the screen design, and more training will not fix that.
5. How-to tickets per 100 users
Agree a "how do I" category with the service desk before go-live, so these tickets can be told apart from faults and access requests. In the first 30 days the 400 users raised 520 of them: 520 ÷ 400 × 100 = 130 per 100 users. In days 31 to 60 they raised 168: 168 ÷ 400 × 100 = 42 per 100.
Read the ticket subjects as well as the rate. The five most common are next month's refresher, and the next edit to the training manual.
6. Adoption of the process
Training can leave people able to use the new process and still not using it. Measure the share of work that went the intended way. Suppose the project wanted a purchase order raised before every invoice, and 1,930 of the 2,270 invoices in March had one. 1,930 ÷ 2,270 × 100 = 85%.
Low adoption beside a high pass rate points away from training and towards habit, incentives or a manager who still accepts the old way. There is more in user adoption.
A fair before and after
A figure with nothing to compare it to proves little. Three things make the comparison fair.
- Take a baseline. Before go-live, record the same measure in the old system over the same length of time: the old reversal rate, the old ticket volume.
- Compare the same period. Week 8 always looks better than week 2, because people learn by doing the job. That improvement is not evidence for the course.
- Find a comparison group. Rollouts usually go in waves or by site, and an earlier wave that had less training is a natural one. The ROI Institute lists control groups and trend lines among the standard ways to isolate what a programme did.
Wave 1 in Leeds, 180 users, had slides and a demo. They raised 266 how-to tickets in their first 30 days, which is 148 per 100 users. Wave 2 in Rotterdam, 220 users, also did scored practice sessions for six tasks. They raised 211, which is 96 per 100.
The gap is 52 tickets per 100 users. For 220 users that is 52 × 2.2 = 114 fewer tickets. At an example 25 minutes of service desk and user time per ticket, 114 tickets is 47.5 hours.
State the caveat in the report: wave 2 also had a more stable system and the lessons of wave 1. The practice sessions probably helped, and this comparison cannot say by how much.
Popular numbers that mislead
Three figures appear in most training reports because they are easy to collect and usually flattering.
Shows whether people liked the session. A 1997 meta-analysis by Alliger and colleagues found that ratings of how useful training was related more strongly to learning and use on the job than ratings of how much people enjoyed it.
Shows that people reached the last screen. It cannot show whether they could post an invoice the next morning.
A quiz taken minutes after the content measures short-term memory. Roediger and Karpicke (2006) found that the study method that scored best after five minutes scored worse a week later.
Satisfaction has a second problem. In a 2019 study by Deslauriers and colleagues, physics students taught through active sessions learned more than students given lectures, yet felt they had learned less. A feedback form can therefore mark down the training that worked. If you keep the survey, ask how useful the session was for named tasks, and treat the answers as leads to follow up. What a learner can do unaided is the better test, which is the argument for hands-on training with a check at the end.
A one-page measurement plan
Write the plan before the training is built, as one page inside the software training plan, and get the system owner and the service desk to agree to supply their reports. This is the plan for the example rollout.
| Measure | Target | Source | When | If it misses |
|---|---|---|---|---|
| Completion | 95% of assigned users by go-live | LMS | Weekly from 4 weeks before | Managers get names. Catch-up sessions |
| Unaided task check | 90% of assigned users passed by go-live | Scored simulations, results in the LMS | Weekly from 4 weeks before | Coaching and a second attempt. A floor-walker for that team in week 1 |
| Time to proficiency | Median of 15 working days | Workload report, team lead | Day 30 and day 60 | Pair slow starters with an experienced colleague |
| Invoice reversal rate | Under 2% by week 8 (old system: 1.8%) | S/4HANA report | Weekly | Refresher on the task with the most reversals |
| How-to tickets | Under 100 per 100 users in the first 30 days | Service desk category | Weekly | Top five subjects become quick guides |
| Purchase order before invoice | 90% by day 90 | S/4HANA report | Monthly | Raise it with the procurement lead as a policy matter |
Set against the sums above, this rollout met its targets for proficiency and reversals, narrowly missed on completion and the task check, and missed on tickets. That is a useful report, because it names where the next month's effort should go.
- 6 weeks beforeAgree the plan
Targets, sources and report owners signed off. Ticket category created.
- 4 weeks beforeTake the baseline
Error rate and ticket volume in the old system over a matching period.
- Go-live weekFreeze the training figures
Completion and task check results by team, sent to managers.
- Day 30First work figures
Tickets per 100 users, error rate, first count of time to proficiency.
- Day 60Second reading
The same measures, compared with day 30 and with the baseline.
- Day 90Report and close
Adoption figure, the comparison between waves, and what to change for the next rollout.
Six measures is the upper limit for one page. If a number has no owner, no source and no action attached when it misses, leave it off.
The unaided task check, built from a screen recording
Recourse turns a screen recording of a task into a simulation with an Evaluate mode: the learner does the task on a replica with no hints and is scored against a pass mark. On Pro, $299 a month for up to three seats, the result reaches your LMS through a SCORM 1.2 package. That covers measure 2 and nothing else. Recourse does not read error reports or service desk data, and the replica is not the live system, so a pass shows that someone can follow the recorded path, not how they handle an odd case.
Start freeQuestions people ask
What is the formula for training effectiveness?
What are the best training effectiveness metrics?
How do you measure training effectiveness using the Kirkpatrick model?
How do you measure training effectiveness for employees on new software?
How soon after training should you measure?
What is time to proficiency and how is it calculated?
How do you calculate training ROI?
Sources
- Alliger, Tannenbaum, Bennett, Traver and Shotland, A meta-analysis of the relations among training criteria, Personnel Psychology, 1997.
- Deslauriers and others, Measuring actual learning versus feeling of learning, PNAS, 2019.
- Roediger and Karpicke, Test-enhanced learning, Psychological Science, 2006.
- ROI Institute, ROI Methodology, for the ways to isolate the effect of a programme.
Keep reading
Last reviewed 21 November 2026. Recourse publishes these guides for learning, customer education and change teams. Details drawn from public sources can change.