The Call That Made Me Rethink the Coaching Industry
A founder I work with runs a mid-sized coaching company in Southeast Asia. Last quarter she called me with a problem that sounded like a success.
She had certified forty new coaches in eighteen months. Revenue was up. Demand was outpacing supply. Her certification program had become a real business.
Then the complaints started.
Clients said Coach A was transformational. Coach B felt like a motivational playlist on repeat. Coach C was technically correct but somehow made people smaller. The founder had hired all three from the same training, passed the same exams, received the same certificates.
The certificate told her nothing about what happened in the room.
She asked me the question that more founders are starting to ask: "How do I know if my coaches are actually doing what we trained them to do?"
I did not have a satisfying answer. Credentialing is a snapshot. Exams test recall. Client testimonials are biased by rapport and personality. Supervision is expensive and episodic. By the time you notice a problem, the damage is already in the market.
A few months later, a research team at Cambridge published a study that changed the terms of the conversation.
The Study That Turns Coaching Language Into Data
In April 2025, Psychological Medicine published a paper titled "Capitalizing on natural language processing (NLP) to automate the evaluation of coach implementation fidelity in guided digital cognitive-behavioral therapy (GdCBT)."
The study is straightforward in design and radical in implication.
Researchers collected over thirteen thousand coach-to-client messages from a six-month guided digital CBT program. Three thousand three hundred eighty-one users were supported by coaches through text-based interactions. Human coders, trained in CBT fidelity, rated the coaches on whether they followed protocol. Then the researchers used NLP to extract linguistic features from the messages and trained machine-learning models to predict those human fidelity ratings.
The best model hit an AUC of 76.06%.
That number is not perfect. It is not replacing clinical supervisors this year. But it proves something that ought to make every coach, trainer, and founder paying for development stop and look up.
Language is now readable as a quality signal.
Not charisma. Not credentials. Not the client's satisfaction score. The actual pattern of language the coach uses when guiding someone through change.
The same skills that NLP practitioners have studied for fifty years, calibration to physiology, precision in questioning, tracking of subjective experience, can now be partially modeled by algorithms. The study focused on CBT, but the underlying pattern applies wherever one person is paid to help another person change: leadership coaching, executive development, sales enablement, founder mentorship, even internal management conversations.
If your language pattern deviates from the protocol, AI can flag it. If you drift from the model, the model can notice.
What Implementation Fidelity Actually Means
Implementation fidelity is the boring term that sits behind every broken training program in the world.
It means: did the person actually do what they were trained to do?
Not did they attend the workshop. Not did they pass the quiz. Did they, in the wild, with a real client, under real pressure, deliver the intervention the way it was designed?
In the Cambridge study, fidelity meant things like using positive sentiment words, avoiding prohibited responses, and adhering to the CBT protocol in the message thread. Human coders could agree on it. That agreement is important. It means fidelity is not a matter of taste. It has structure.
For founders, this is the missing link in every leadership development budget.
You send your managers to a communication course. You pay for a coach for your executives. You buy a culture program. Then everyone comes back, certificates in hand, and resumes doing exactly what they were doing before, with slightly better vocabulary.
The program did not fail because the content was bad. It failed because no one measured whether the behavior changed in the actual conversation.
Implementation fidelity closes that loop. It says: here is the model. Here is what faithful delivery looks like. Now let us look at the real language and see if the model survived contact with reality.
What the 76.06% Really Tells Us
A 76.06% AUC sounds technical. It is worth translating.
AUC stands for area under the receiver operating characteristic curve. In plain language, it measures how well a model separates high-fidelity coaches from low-fidelity coaches based on language alone. A score of 76.06% means the model is significantly better than chance and already useful as a screening tool.
The researchers did not hand the algorithm a list of coaching rules. They gave it thousands of messages and the human ratings. The algorithm found the patterns. Sentiment mattered. Linguistic structure mattered. The presence or absence of certain word classes mattered.
This is the part that matters to anyone running a coaching or training business. The model did not need to understand CBT. It learned to read fidelity from the text.
There are obvious limits. The study was in guided digital CBT, a text-based environment with a clear protocol. Face-to-face coaching is richer. Voice tone, physiology, and presence all carry signal that text misses. A 76.06% score leaves plenty of room for the human element.
But the trajectory is clear. As multimodal AI improves, it will incorporate voice, facial expression, and physiological markers. The fidelity score will climb. The coaches who understand this now will be the ones who shape the standards rather than scramble to meet them later.
Why This Is a Threat to the Credential Economy
The coaching and training industry has operated on a credential economy for decades.
Certification equals competence. More letters after the name equals more trust. A longer training program equals deeper expertise. The market has very few ways to verify whether any of that translates into real client results.
AI is about to create a second market.
In the second market, what matters is not where you trained. It matters whether your language pattern, in actual sessions, matches the protocol that produces outcomes. A junior coach with high fidelity may outperform a senior coach with low fidelity. A certificate may become a starting point, not a signal.
This is not theoretical. The Cambridge study already shows that sentiment analysis and linguistic feature extraction can predict fidelity with acceptable performance. The next generation of tools will not stop at sentiment. They will track Meta Model precision, ecology checks, presuppositions, calibration markers, and state-shifting language in real time.
For the good coaches, this is liberation. They have been competing against credential inflation for years. Language-based fidelity measurement gives them an objective scoreboard.
For the mediocre coaches, this is exposure. They have built careers on confidence, testimonials, and networking. When the language pattern becomes visible, the gap between performance and reputation becomes measurable.
The Three Things NLP Can Already Read in a Coach's Messages
The Cambridge study used a broad set of NLP features. For founders and leaders, the practical translation comes down to three markers you can already train yourself to notice.
1. Specificity
Weak coaching lives in abstraction. "You need to believe in yourself." "Stay positive." "Trust the process."
These phrases feel supportive and do nothing. They do not tell the nervous system what to do.
Strong coaching moves toward specificity. "What do you see when you imagine the meeting going well?" "Where in your body do you feel the resistance?" "What exactly did they say, and what did you say back?"
NLP can measure this. Abstract language has different patterns than sensory-specific language. The algorithm does not need to understand the content to detect whether the coach is guiding the client toward concrete internal experience or staying in the clouds.
2. Ecology
Every intervention has side effects. A coach who pushes a client to set a bigger goal without asking what else changes in their life is not being bold. They are being careless.
The ecology check asks: if this change happens, what else changes?
In text, ecology shows up as questions about context, relationships, identity, and downstream consequences. A coach with high fidelity will ask about the system around the client. A coach with low fidelity will treat the client like a standalone problem to be solved.
NLP can flag this by tracking whether the conversation stays narrowly focused on the stated issue or expands to include the client's larger system. This is one reason the Cambridge researchers found that fidelity is not a subjective preference. It has linguistic structure.
3. Forward Movement
A session can feel profound and produce nothing. Insight without action is entertainment.
High-fidelity coaching creates forward movement. It leaves the client with a next step, an experiment, a behavioral commitment. The language moves from exploration to decision.
NLP can detect this by analyzing the direction of the conversation. Is the coach closing loops? Is the client making commitments? Is the language shifting from past difficulty to future action?
These three markers, specificity, ecology, and forward movement, are not CBT-specific. They are universal to any change conversation. A founder can use them to evaluate a business coach. A CEO can use them to evaluate an executive coach. A team lead can use them to evaluate their own one-on-ones.