Recently, a seasoned OSC reached out to me because she knows I do mystery shops. She also knows about the Annual Mystery Shop Report we conduct every year to get a pulse on the state of the industry when it comes to customer experience and the customer journey.

She had been mystery shopped. On a Saturday. Her day off. Actually, it was the beginning of a four-day weekend for her. And as a solo OSC, she needs to take time off. Gone are the days when we should expect an OSC to be available 24/7. That’s not dedication. That’s a recipe for burnout.

But there was more to this particular shop that bothered me.

When we mystery shop a builder who hires us, we’re trying to get an accurate picture of what actually happens when a real prospect reaches out. That means we try to behave like a real prospect. Well, sort of.

There’s no doubt that real people shop during off-hours and on weekends. But when we’re hired to mystery shop an OSC, we’re first testing the skills and follow-up of the OSC, not the shortcomings of their coverage hours. As consultants, we look at the program holistically. The mystery shop itself, however, is designed to evaluate the OSC’s performance: speed, tenacity, diversity of messaging and how consistently they continue outreach over a 30-day period before giving up.

With that in mind, we submit the inquiry during normal business hours when the OSC is supposed to be working. We use a real person’s name. We use an email address that looks like something a real person would actually use. And we ask a legitimate question on the form. We follow the same standards with all 50 builders in our Annual Mystery Shop Report, regardless of their coverage.

We’re not trying to trick the OSC into failing. We’re trying to evaluate the experience a real buyer is likely to have.

That’s not what happened here.

The email address used for this shop looked something like [email protected]. That’s not the actual address, but you get the idea. There also wasn’t an actual question submitted with the inquiry.

So now put yourself in the OSC’s shoes.

A strange-looking lead comes in while you’re off for a four-day weekend. There’s no question from the prospect. The email address looks suspicious. When she returned from her days off, she still attempted to contact the lead six times before eventually deciding it was probably spam.

Could she have done some things differently? Yes.

In fact, when we talked through what happened, the shop uncovered a couple of legitimate opportunities for improvement. It also exposed a bigger operational gap: What happens to incoming leads when a solo OSC is off?

That’s worth talking about.

When the Shop Becomes the Sales Pitch

What happened next is where this really became a problem for me. This wasn’t a mystery shop her builder commissioned. A company conducted the shop, created a report showing where she supposedly fell short, contacted her with the results and then tried to sell her their software.

That’s not mystery shopping designed to improve performance. That’s a gotcha tactic.

And here’s the irony: the OSC’s immediate reaction wasn’t, “Maybe my company needs better coverage or technology when I’m off.” It was, “I guess I’m going to have to start checking things on the weekends.”

No.

That is exactly the wrong lesson. A seasoned OSC who works hard all week should not walk away from a mystery shop believing she can never truly take a day off because someone might be testing her.

If you’re a solo OSC, there does need to be a plan for what happens when you’re unavailable. Maybe that’s backup coverage. Maybe it’s AI. Maybe it’s live chat. Maybe it’s automation that acknowledges the inquiry and sets appropriate expectations until the OSC returns. There are plenty of ways technology can help fill that gap.

I’m not opposed to software helping an OSC when they’re off. Quite the opposite. Let the technology help the unicorn out.

But if you’re selling that technology, creating a shop that seems designed to test the gaps in coverage as much as the skills of the OSC, grading someone on how they handled it while they weren’t working, and then showing up to tell them that your software could have prevented their “failure,” probably isn’t the best way to earn their trust.

Ironically, this OSC probably should consider some technology options to help cover the gaps when she’s off. But this experience didn’t make her interested in the solution being offered. It did the opposite.

This scenario is not really about whether the OSC “passed” or “failed” the shop. It’s about the difference between mystery shopping designed to improve the customer journey and mystery shopping designed to manufacture a failure so you can sell the solution.

Mystery shopping should uncover opportunities. It should help us understand the customer journey, identify gaps, coach our people and improve our processes.

Sometimes the problem will be the OSC. Sometimes it will be the process. Sometimes it will be staffing. Sometimes it will be technology. And sometimes the shop itself isn’t a fair representation of the customer journey we’re supposedly trying to measure.

You Don’t Know What You Don’t Know

There’s another potential problem with shopping as a way to generate business: You don’t know what you don’t know.

And yes, you could argue that we mystery shop 50 builders every year for our Annual Mystery Shop Report without knowing the full context of every company we shop. That’s true. The difference is what we do with that information.

Our annual report looks for patterns across the industry. We apply the same standards to all 50 builders and report on what we find collectively. We don’t take an unsolicited individual shop, show a company where their OSC fell short and then use those results to sell our services.

Now, would I potentially reach out to a company that doesn’t have an OSC and show them where they might have an opportunity to compete? Sure. But I wouldn’t show up to a company that already has an OSC and say, “Hey, your OSC sucks!” That’s just not how I roll.

Once you start using an individual’s performance as the sales pitch, the lack of context matters.

You don’t know how many leads that OSC is juggling. You don’t know if they’ve already been asking leadership for additional coverage or technology. You don’t know if they’re fighting the same battle your report just uncovered.

And what if that unsolicited report lands in front of leadership? Does it help make the OSC’s case for additional resources? Or does it make a hardworking OSC look like they’re simply not doing their job?

You don’t know. And that’s the problem. You could inadvertently get someone fired.

That’s the law of unintended consequences. And it’s something worth thinking about before you use someone’s performance as a sales tool.

Mystery shopping can be an incredibly valuable tool. But when it’s being used to generate business, we need to think about the unintended consequences for the person on the other side of that report.

These unicorns can be sensitive. Don’t break the unicorns.

A good mystery shop should make people curious about what they can improve.

It shouldn’t make a great OSC afraid to take Saturday off.