2018 Research: Chapter 3 - Measuring Impact

CoachSource is pleased to provide the fourth in a series of seven blog posts featuring chapters from our 2018 Executive Coaching Research Study.

In this post, we feature Chapter 3 of the report, "Measuring Impact," in its entirety. It includes the following subsections:

  • Evaluating Satisfaction with a Coach
  • Rating Results of a Typical Engagement
  • Means of Measuring Coaching Impact
  • Linking Coaching to Business Results
  • Emotional Perceptions of Coaches

​We will continue releasing the full report, chapter by chapter, over the course of 2018-2019. To read prior chapters, please visit our blog. To be informed of the release of future chapters, please subscribe to our newsletter by entering your email address below:

Newsletter Sign Up




Please note: This material is copyrighted by CoachSource, LLC.

Chapter 3: Measuring Impact

Evaluating Satisfaction with Coach

Evaluating a leader’s satisfaction with the coach is a “Level 1” business metric that is rather easy to implement. We asked respondents, “In what manner and how often are leaders asked to evaluate satisfaction with the coach?” since such wording might uncover both the way and the frequency in which satisfaction with the coach is measured.

In the 2013 study, we found organizations (74%) and external coaches (71%) selected HR/LD to informally check in with leaders as the highest used method to evaluate satisfaction with the coach. Leaders (43%) and internal coaches (57%), however, rated Leader completes a satisfaction survey at the end of coaching as the most typically used tool to gauge satisfaction with the coach.

This year, organizations (75%), leaders (34%), and internal coaches (65%) rated Leader completes a satisfaction survey at the end of coaching the most frequently. Contrary to the 2013 survey, figures for this evaluation metric increased significantly for organizations (38% of organizations in 2013 – a large increase). The next most frequently used evaluation tool for organizations (73%), leaders (31%), and internal coaches (49%) was HR/LD informally checks in with the leader.

The perception difference between leaders and the other groups continues: i.e. Leader completes a satisfaction survey at the end of coaching (34% of leaders compared to 75% of organizations), HR/LD to informally check in with leaders (31% leaders vs 73% organizations), and Leader’s manager asked about satisfaction with the coach (26% vs. 45%). Leaders selected We generally don’t ask about satisfaction with the coach at 31% - An increase since 2013, where only 21% of leaders selected this statement. Figure 21 displays the overall responses for evaluating a leader’s satisfaction by rater group.

 

Rating Results of a Typical Engagement

The 2013 study asked all four groups the novel question, “How would you rate the results of a typical coaching engagement?” Though the 2017 survey asked this same effectiveness question, the response options were changed from a 5-point Likert-type response scale to a 7-point scale to enhance the diversity of results.

Across all four groups, results skewed mostly positive. Eighty-six percent of organizations, 95% of leaders, 89% of internal coaches, and 97% of external coaches selected Very effective, Mostly effective, or Effective. These results reflect 2013 interpretations, with 84% of organizations, 88% of leaders, 98% of internal coaches, and 91% of external coaches having reported interactions as either Very effective or Moderately effective (the two options for recording an effective engagement in 2013). This year, the most frequently selected options were 68% of leaders noted typical coaching engagements were Very effective whereas internal coaches (52%) and external coaches (49%) noted it as Mostly effective.

Perhaps the most encouraging statistic from this section is that leaders selected Very effective 68% of the time whereas the other groups were all below 39% for this choice – at least a 30-point difference.

Also, although most groups reported effective interactions, 10% of organizations still reported engagements as Mostly ineffective. Comparing this stat to 2013 results, 9% of organizations rated engagements as Moderately ineffective. Since these statistics are similar, it might be interesting to further press organization contacts as to why they still believe some engagements to be ineffective. See Figure 22 for full results.

Means of Measuring Coaching Impact

An incredibly common concern for coaching practice managers centers around how to measure the impact of coaching. Therefore, as in both the 2005 and 2013 surveys, we asked participants to record how they measured the impact of coaching.

Similar to 2013 results, participants selected the top two methods of measurement as: Leader self-report of progress (69-81% across groups) with a large drop to #2: Assessment by leader’s boss (21-55% across groups). Across groups, the third most-used method was to conduct a Brief survey to check progress (10-51%). Other methods garnered fewer selections, such as Leader’s promotion to higher level of responsibility (18-40%), business impact (24-42%), or Follow up comprehensive 360 (10-29%).

Figure 23 shows respondent selections for the various methods used to measure the impact/effectiveness of the coaching.

Upon comparing 2017 figures to the 2005 and 2013 results, the more subjective methods (e.g., self-report, boss’ report) are once again used with greater frequency than objective metrics (e.g., brief follow-up survey, business impact, repeat 360).

Regarding organizations, most respondents agreed that a leader’s self-report of his or her progress was the main method of measuring coaching impact/effectiveness (almost 30% higher than the second most used method). Additionally, organizations selected We did not do any measurement (6%) less than had been reported both in 2013 (16%) and 2005 (33%). This may be a reassuring statistic as it might suggest that organizations are focusing more on the impact of executive coaching. As usual, external coaches selected most options with greater frequency than organizations. However, unlike 2013, internal coaches selected most choices with greater frequency than external coaches.

Among the Other responses, a common qualitative theme was “follow-up”. Respondents indicated that an organization utilized progress evaluations during and after engagements to determine the effectiveness of coaching interactions. Engagement effectiveness was determined through follow-up surveys, informal assessments and/or interviews. Additionally, another common theme was that behavioral changes of leaders were analyzed. Such behavioral changes ranged from working better with fellow employees to noticeable improvements to the knowledge, skills, and abilities (KSAs) in leaders.

Linking Coaching to Business Results

In 2005, only 7% of organizations reported that they had linked business results to coaching. In 2013, 51% of organizations indicated they had linked coaching to business results (selecting either Regularly or Occasionally – Note this scale was changed from 2005). Although the 2017 study used the same scale as 2013, only 32% of organizations reported having linked coaching to results (Regularly or Occasionally). So the trend improved greatly between 2005 and 2013, but has faded since 2013 to the present.

In 2013, 40% of organizations selected the option that they did not link such executive coaching to business results and that they would like to link such a relationship. This year it is now 63% - a large increase.

Leaders had a very different view in this area. 68% of leaders said their coaching work was regularly or occasionally linked to business results (compared to 32% of organizations). Additionally, leaders selected Yes, regularly more than both types of coaches (46% internal coaches, 65% external coaches). Essentially, leaders feel coaching is more often linked to business results than other rater groups do by a fairly large margin.

Mirroring organizations with their responses, internal coaches maintained an 8% or less difference in their responses when compared to organizations. Possibly, internal coaches and organizations might share similar understandings of how executive coaching is measured in organizational settings.

Across the board, however, 2017 results indicated a noticeable decline in linking executive coaching to business results as both Yes, regularly and Yes, occasionally response choices were less than 2013 Yes percentages. Figure 24 displays the breakdown of each group’s responses.

We asked organizational representatives to provide qualitative responses on how executive coaching is linked in their organization and they stated the following:

  • "We did a study with our People Analytics group linking coaching to compensation. We found that those that were coached made about $30K more annually than those that were not coached."
  • "Depending on what the leader is focused on developing, we might be able to draw a straight line to business results. It could be increasing the effectiveness of a leadership team that is driving certain projects, aligning stakeholders or building relationships for key initiatives, ability to implement a strategic plan, etc."
  • "Qualitative feedback, plus mini-survey, then conversation about how areas of focus impact performance of leader and the team."
  • "We link it with action learning projects and ask participants to work with their coaches to work on some long-standing projects to realize ROI of coaching for their organization."
  • "Leader's effectiveness in achieving results/turnaround/promotion."
  • "Achieving budget or project."
  • "The coaching targets link with strategic business needs."

Emotional Perceptions of Interactions

We added a new question to our 2017 survey asking about the emotional interactions between a coach and coachee. We asked each rater group to what extent they felt their interactions were positive or negative with their coach or leader. We provided participants with a 9-point scale to rate interactions from Very negative to Very positive.

On average, organizations found interactions to be Mostly positive (52%), whereas leaders more frequently found interactions to be Very positive (68%). This is favorable insight for coaches. It appears that organizations may not have crystal clear insight into the interactions between a coach and coachee, since the coaching interactions appear to be more positive than organizations assumed them to be.

Likewise, internal coaches most frequently ranked their interactions as Mostly positive (45%), but external coaches rated their interactions as Very positive (53%). Internal coaches were more likely to rank their interactions as Positive (16%) in comparison to higher ratings external coaches gave themselves.

It would be interesting to examine in future studies how a leader perceives their interactions to be with an internal coach versus an external coach. Overall, more than any other rater group leaders ranked their interactions as Very positive most frequently (68%), demonstrating that coaches and organizations may not realize just how positive their interactions are with their leaders. See Figure 25.

Please note: This material is copyrighted by CoachSource, LLC.

Our next blog post will be Chapter 4: Coach Selection, which includes:

  • Locating Coaches
  • Coach Selection Criteria
  • Coach Biographies
  • Certification of Coaches
  • Educational/Training Backgrounds of Coaches
  • Turning Down Assignments
  • Coaching Supervision

And remember, we plan on releasing the full report, chapter by chapter, over the course of 2018-2019. To read prior chapters, please visit our blog. To be informed of the release of future chapters, please subscribe to our newsletter by entering your email address below:

Newsletter Sign Up



Please feel free to post a comment below.

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.
Go to top