Resources How-to's Start with good data: making sure performance ratings and scores are well calibrated


Start with good data: making sure performance ratings and scores are well calibrated

The performance ratings and scores your managers give their employees on competencies and goal attainment are some of the key inputs you use for a variety of HR metrics. So making sure those ratings and scores are fair and accurate is critically important. Yet because evaluations are by their nature subjective, you're bound to find some variability in the numbers. So it's important to take steps to ensure inter-rater reliability across the organization. Here are some HR metrics and steps you can use to do that.

1. Start by comparing results

Begin by looking at your overall ratings and scores. Are they what you expected? Next, look at scores by department/division/group. Do any groups have higher or lower than expected ratings? Finally, compare results by manager. Do any managers seem to assign higher or lower scores or ratings than others?

What you're looking for are trends, any data or distribution that doesn't seem to fit with your expectations or the average. If you spot any of these, it's time to dig deeper.

2. Identify the source of potentially miscalibrated data

If you spot any scores or ratings that are not what you expected, or that seem to be out of the norm, try to identify or isolate the sources of the data. Do these ratings/scores come from any particular parts of the organization? Can you trace them to a particular manager or group of managers?

3. Identify the root cause of potentially miscalibrated data

Once you've identified the source of the potentially miscalibrated data, it's time to get at the root cause. Ratings/scores that are out of sync with the rest of the organization could indicate things like:

  • A manager who rates all their employees higher or lower than the norm
  • A group or individual that had particularly challenging or particularly easy goals this period
  • A group or individual that was unable to achieve their goals for a particular reason
  • A group that has a significant number of high or low performers
  • A group that has a significant number of new or highly experienced employees
  • A group or individual who was assessed on competencies that didn't really apply to their jobs
  • A group or manager with a high number of incomplete appraisals
  • A group or manager who don't understand the intended meaning of the various ratings and levels of performance.

You can identify whether in fact ratings/scores are poorly calibrated, and the root cause of the miscalibration by talking to the managers and employees involved, as well as their peers. Frame your questions and discussion in a positive light, making sure everyone knows the goal is to improve the accuracy and fairness of your review process, as well as the results and decisions based on those results.

4. Take action to ensure ratings and scores are well calibrated going forward

Once you've discover the root cause of the miscalibration, you're ready to take action. The specific actions you take will depend on the root causes identified, but often include some of the following:

  • Provide managers and employees with training on the meaning and use of your rating scale and scores to properly set expectations and established shared standards
  • Review your rating scales to make sure they're allowing you to collect the performance data you need. You may want to adjust your rating scales or take other action to avoid "grade creep".
  • Put a process in place to review managers' ratings before they are finalized. You can setup meetings where peer managers review and adjust each other's ratings, have more senior managers review and adjust ratings as needed, or have someone in HR review ratings.
  • Ensure any performance issues (either high or low performance) are being addressed appropriately.

5. Verify that your actions have been effective

Finally, you also need to verify that your actions have been effective. So compare ratings and scores from review period to review period, looking in particular for changes in the areas where you identified miscalibration. If your actions don't seem to have been effective, look again for the root cause, and define a new action plan.

6. Repeat steps 1 through 5

And repeat all these steps on a regular basis to ensure your performance ratings and scores remain well calibrated.

Supporting employee engagement, satisfaction, and better HR metrics

It's well worth the effort to regularly analyze your performance review ratings and scores, looking for trends that indicate miscalibration, and taking action to correct it. Your performance reviews should be fairer and more consistent, fostering employee satisfaction and engagement. You'll also have better quality performance data to support your other HR metrics analyses and talent management programs.

As part of our Certain to Succeed implementation programs, our implementation consultants work with our customers each year to review the metrics from their processes, including the calibration of ratings and scores, and formulate an action plan for continuous improvements. Contact us to find out how we can help you be brilliant at talent management.

Success with Talent Management made easy. Experience it for yourself.

Get Started