Case Study AI Deployment B2B SaaS, Cybersecurity

Deploying Fin AI without breaking the trust I'd spent years building.

A walkthrough of how I rolled out Intercom's Fin AI at a B2B SaaS company to handle the majority of inbound support autonomously, while sustaining strong CSAT and never letting a sensitive conversation reach the AI in the first place.

High
Autonomous resolution rate, sustained
100%
Sustained CSAT since launch
24/7
Coverage across all time zones

The problem I was actually trying to solve

Fin AI wasn't deployed because we were drowning. It was deployed because the team was spending too many hours on the same handful of repetitive, low-complexity tickets, while the genuinely interesting and important work, including incident triage, complex enterprise escalations, and deep customer enablement, was getting squeezed.

There was a second problem too. The business had been growing into US and non-UK time zones, and customers raising issues outside UK working hours were waiting until the following morning for an initial response. For an enterprise SaaS platform with global users, that was unacceptable.

So the brief I set myself was specific: handle the repetitive volume autonomously, give every customer some form of immediate response regardless of time zone, and free up the team to do the work only humans can do.

Making the case before writing a single workflow

The first resistance was financial. Adding another tool, particularly an AI-priced one, prompted reasonable questions about whether the cost made sense. I built two analyses to answer that:

To remove ongoing risk, I implemented budget notifications and hard limits from day one, so Fin's spend would scale with our actual business need rather than running away with usage.

Lesson

If you're proposing AI tooling at any scale, lead with the financial case before the operational one. Founders and finance leads care about cost predictability more than capability promises.

The hardest part wasn't the AI

The platform I was deploying on has a relatively unusual structure: a meaningful proportion of customers operate on both sides of the platform, taking on different roles depending on the workflow they're in.

The terminology overlaps across both sides. The product flows are different. And without strict, clear audience rules, an AI looking at an ambiguous question has no way to know which side of the platform the user is asking from, and the correct answer is genuinely different in each case.

A question that sounded almost identical from two different users could need two completely different answers. Fin couldn't be trusted to guess.

Solving this meant treating audience segmentation as the foundation of the entire deployment. Every snippet, every workflow, every internal article had to be tagged and routed by audience first, and answered second. The AI was only as good as the structure I gave it.

How I actually rolled it out

I didn't switch Fin on for everyone at once. The deployment ran in four deliberate phases.

1. Two months of batch testing, no live customers

Before any real customer saw a Fin response, I ran extensive batch tests of how Fin handled real historical questions. I reviewed every response personally. Where Fin was wrong, I either rewrote source content, restructured the workflow, or added an explicit escalation rule. Anything sensitive, including security questions, billing, and enterprise-level issues, was set to escalate automatically and never attempt an answer.

2. Single-audience trial

The simpler audience went live first, with fewer dual-side users and more standardised questions. Fin was rolled out for that audience initially, with continuous review of every conversation in the first weeks.

3. Continuous auditing and content rewriting

Since launch I've personally rewritten or added a substantial number of internal knowledge snippets and articles based on what Fin was struggling with, what customers were actually asking, and where the gaps in coverage emerged. The deployment never sat still.

4. Expansion with full audience controls

Only once Fin was performing reliably for a single audience did I expand to the dual-sided users, with the audience routing logic now battle-tested.

Why CSAT has been 100% from day one

This is the part that surprises people. Fin has never dropped below 100% CSAT, but that's not because Fin is perfect. It's because I didn't let Fin into conversations where it could fail.

The escalation rules were set before launch. Anything that wasn't a clear match to a confident knowledge source escalated to a human. Anything sensitive, including account, security, and contract issues, never reached Fin in the first place. The AI's job was to handle the questions where it would be confidently right, and to gracefully hand off everything else.

Operating principle

An AI that handles 85% of tickets brilliantly is far more valuable than one that attempts 100% and damages 15% of customer relationships in the process.

What I'd do differently next time

If I were doing this again at a different company, three things would change:


What this looks like in numbers

Since launch, Fin has handled the majority of inbound conversations autonomously. The team has been freed from repetitive work and is now focused on enterprise enablement, deeper analysis of recurring failure patterns, and proactive customer outreach. Out-of-hours customers receive an immediate response. CSAT has been sustained at 100%, with hard cost predictability built in.

None of that is luck. It's the result of treating AI deployment as a product problem, with audience segmentation, escalation policy, financial modelling, and continuous content investment all treated as first-class concerns from the start.

Want to talk about scaling support?

Whether you're hiring, scoping a deployment, or just thinking through what good looks like, I'd love to hear from you.

hello@kianpace.me
← Back to portfolio Connect on LinkedIn ↗