How I Built a QA Process Nobody Asked For

There was no ticket for it. No project brief, no roadmap item, no manager who pulled me aside and said this needs to exist. The QA process I built for our AI-generated responses started the way a lot of the most important work I’ve done started: I noticed something was wrong and couldn’t stop thinking about it.

The something was this: our AI was handling the majority of customer interactions, and nobody was systematically checking whether it was doing that well.

The most valuable work I’ve ever done was work nobody assigned me

At the time I noticed this, I was working as a CX Operations Specialist at a wellness app. AI handled an enormous volume of our customer tickets automatically, which was by design. The automation rate was high, the resolution metrics looked good, and leadership was satisfied with the numbers.

But I was reading the tickets. And what I was seeing didn’t always match what the dashboard was telling me.

Template errors that sent customers information about the wrong subscription tier. Duplicate replies that made it look like our system was broken. Automated responses that referenced data fields that hadn’t populated correctly, so the customer received a message that was personalized with a blank where their name should have been. None of these were catastrophic individually. Collectively, across thousands of automated interactions, they were a quiet erosion of trust.

The problem wasn’t that the AI was failing dramatically. It was that it was failing in small, subtle ways that were easy to miss if you weren’t paying close attention. And nobody had been assigned to pay close attention.

So I decided to do it myself.

Building something from nothing is harder when nobody knows it’s missing

The first challenge was practical: how do you build a review process when there’s no existing infrastructure for it, no dedicated time for it, and no one who has asked for it?

I started small. At the end of each shift I’d spend time reading through the AI-generated responses from that day, not to resolve anything but to audit. I built a simple tagging system to categorize what I was finding: template errors, data population failures, tone mismatches, duplicate sends, incorrect resolutions. I kept a running log.

Within a few weeks the log started telling a story. Certain error types were recurring. Certain triggers were reliably producing bad outputs. The AI wasn’t failing randomly. It was failing in patterns, which meant the failures were fixable if someone knew where to look.

The second challenge was getting anyone to look. This is the part that doesn’t get talked about enough when people discuss building new processes: the work of convincing people that the problem exists is often harder than the work of solving it. When something has been invisible for long enough, its invisibility starts to look like evidence that it isn’t there.

I started bringing specific examples to engineering. Not “the AI is making mistakes” but “here are seven instances of this specific template error in the last two weeks, here is what triggered each one, and here is what the customer received.” Specificity changed the conversation. Vague concerns get acknowledged and forgotten. Documented patterns with examples get investigated.

Within a month of starting that practice I had escalated over ten distinct defects to the backend team. Most of them got fixed. The fix rate on the types of errors I was tracking improved measurably. And the process that had started as something I did quietly at the end of my shift became something the team recognized as valuable.

Ownership without authority is its own kind of leadership

I want to be honest about something: building this process was not always comfortable. There’s a particular vulnerability in taking ownership of something that wasn’t assigned to you, especially when the implicit message from the organization is that things are fine. You’re not just doing extra work. You’re making an argument, with your time and your attention, that something important is being missed.

That argument can land badly. It can read as criticism of decisions that other people made. It can make colleagues defensive. It can put you in the position of having to prove a problem exists before anyone will help you solve it.

What I learned is that ownership without authority requires a specific kind of discipline. You have to be rigorous about documentation because your observations only carry weight if they’re specific and reproducible. You have to be patient about adoption because people don’t change processes quickly even when the case is clear. And you have to stay focused on the outcome, better experiences for customers, rather than the recognition, which may or may not come.

In my case the recognition did eventually come. The process was adopted. The defects I flagged were fixed. The quality of the AI outputs improved in ways that touched thousands of customer interactions. But I’ve thought about what I would have done if none of that had happened, if the work had stayed invisible, if the patterns I documented had been dismissed.

I think I would have kept doing it anyway. Not because I’m selfless, but because I couldn’t un-see what I was seeing. Once you know a problem exists and you have the capacity to address it, the cost of ignoring it is higher than the cost of doing the work.

That’s what ownership without authority actually feels like from the inside. Not heroic. Just necessary.


I’ve been thinking lately about what it means to build things that nobody asked for. The QA process was one example. The knowledge base restructure was another. The reporting framework I put together for leadership was a third. None of them started with a brief. All of them started with a gap I couldn’t leave alone.

I don’t think that instinct is unique to me. I think a lot of people in CX ops work this way, quietly building the infrastructure that makes everything else possible, without a mandate and often without recognition. This blog is partly my attempt to make that kind of work visible, to give it language and legitimacy.

If you’ve ever built something nobody asked for and wondered whether it counted, it did. It does. The fact that nobody assigned it to you doesn’t make it less real. It might actually make it more so.

Leave a Reply

Discover more from Beyond the Queue

Subscribe now to keep reading and get access to the full archive.

Continue reading