Call Monitoring Methodology

Traditionally, almost all call centers & operations units…somehow, someway…monitor & measure the quality of their people’s work.

Was it accurate?

Were the customer’s needs met?Exceeded?

Was the employee courteous?

Were all appropriate policies adhered to, processes followed & steps taken?

Did the customer fully understand the outcome?

Was he satisfied?

Normally, these organizations employ a separate “quality assurance team”, or a call monitoring group, to measure how well, or how poorly, the customer’s request was handled.

Most places use a checklist (each item is worth an appropriate amount of “points”) to rate the employee’s performance on a scale of 1 to 100.

Just like when you went to school & took a test and then received a grade.

Several years ago, Citi migrated to a new way of monitoring & grading their incoming customer service calls.

Instead of using the traditional 1-100 methodology, they instead started to use 5 classifications: “Exceptional”, “Outstanding”, “Par”, “Needs Improvement” & “Unsatisfactory”.

Does this look familiar to you? Personally?

What about at YOUR annual appraisal?

Yes, it mirrors the same exact methodology & terminology that companies employ in rating & ranking their employees semi-annually/annually.

(Pls note: I’ve written a whole ‘nother piece regarding the philosophical debate between “rating” & “ranking” and how we say 1 thing, but actually do the other!)

Throughout my career, I’ve seen so many monitored transactions/calls which received a “100” score…but we really nothing to write home about.

There were “no mistakes made”.

No points off for mistakes or omissions.

Usually, simple calls simply executed.

But the “absence of errors” does NOT necessarily mean that the employee did an exceptional job!

Or even an outstanding one.

In my mind, a “100 score” should equate to perfection, or, at least, as close as you can get to it!

In a TEST, “no errors” = “perfection” (except when essay questions are involved).

But that’s not the case when it’s a human interaction!!!

There are almost an infinite number of variables to consider when you’re evaluating this interaction…AND IT JUST DOESN’T FIT NICELY INTO A NUMERICAL SCORING SYSTEM!!!

In my mind (and, trust me, that’s a scary place to be)…

Not only were no errors made, but the customer’s needs were met…and exceeded!

It should WOW the customer.

It should cover other options available to the customer, helpful recommendations on how to avoid issues in the future or transact in an easier method, and the customer should show their appreciation for the help they just received.

But with most 100-scored calls, that’s not necessarily what happened. No mistakes were made, no points were taken away, the employee receives a 100 score.

I believe you should be able to take EVERY 100-scored call (100 = perfection) & without actually listening to it, present it to your CEO as a fine example of the quality work that your team produces & provides to your customers.

But it’s pretty obvious that the strict “1-100 scoring methodology” doesn’t have the proper flexibility to truly reflect the quality of the customer’s experience.

No customer walks around & says, “Boy, I just had a 100 score experience with that company”, or, for that matter, says, “Damn, that was a 64 level service I just received”!

If the 5 grade system (Exceptional, Outstanding, Par/Average/Good, Needs Improvement, Unsatisfactory) works for rating your employee’s overall performance, it certainly makes sense to use the same protocol for measuring & grading their individual work units.

It also allows the organization to maintain high standards for quality.

At Citi, when we transitioned our monitoring methodology, we used to give out red ribbons for all calls rated “Outstanding” & blue ribbons for “Exceptional” calls.

Employees would proudly display these ribbons & you could quickly see who the “best” employees were.

And again, “not making any mistakes” alone would NOT earn you an Outstanding nor an Exceptional rating.

The basic philosophy is that most calls will/should “begin” at “Par” (average, good).

A very difficult call…because of the complexity of the situation/issue itself or, perhaps, the mindset of the customer, e.g., an extremely irate caller…will allow the call to actually begin at “Outstanding”.

(In my mind, effectively & efficiently handing a complicated call or one where the customer is clearly agitated “deserves a better score” than similarly handing a routine inquiry, with all other things relatively the same!)

With this new “scoring philosophy”, the employee gets to “move the needle”, either up or down, with what he says & does (or for that matter, what he doesn’t say or doesn’t do).

Of course, the accuracy factor plays an important role as does compliance with policy & regulation.

But so does “customer satisfaction”!!!

Was the caller happy?

Did he thank the employee?

Did he express how he felt, directly or indirectly, with how the call proceeded?

And regarding the employee…

Did the employee offer options or alternatives that could help the customer in the future?

For example, educate the customer on self-service tools that are available 24×7 or are more convenient to use?

Inform the customer on practices, products or service that could prevent issues in the future, e.g., overdraft protection for bounced checks, auto deduct for monthly bills, on-line bill payment, electronic funds transfers, balance/transaction alerts, etc.?

Teach the customer on “how the process works”, e.g., don’t depend on the balance provided at the ATM, but instead, be aware of other transactions that are scheduled to post that will affect the balance & help avoid “insufficient funds” situations, etc.?

With “Exceptional-rated” items, you should feel COMPLETELY comfortable in having your CEO listen to the recorded call or examine the transaction…without first listening or reviewing the item yourself!

You’d be happy to put it right on the front cover of your company’s book or in a commercial…without first listening to it!

And with “Outstanding” items, you truly feel that it deserves special recognition.

With Exceptional items (and just s little less with Outstanding), you’d feel perfectly fine playing it in front of a new hire training class & say, “I’d like you to grow up to be like this employee!”

These calls need to be “special”.

There needs to be specific examples of where your employee went “above & beyond” or resolved a very difficult situation.

Customer reaction to whole experience must be positive.

You can ram all the official policies & procedures down a customer’s throat & be “100% accurate”, but if the customer really doesn’t understand the concept or doesn’t feel any more comfortable after the call than before it, then it isn’t “special”.

Somehow, someway, you want to be able to “hear the customer smile” on the phone call.

That can only happen when you’ve done the right thing & the customer fully accepts the outcome.

You can even make a minor mistake on a call & still receive a “Par” (or even, Outstanding or Exceptional rating) when you look at the call IN ITS ENTIRETY & you examine the customers reaction.

If the employee forget to remind the customer about one small feature of the product/service (that did not have a significant negative impact on the customer), but otherwise did a really great job, don’t be afraid to take everything into account & provide an Exceptional or Outstanding rating.

With the old 1-100 methodology, it would be impossible to earn a 100 as points would be deducted < < < depending on how the form was constructed!

Lemme try & illustrate the basic philosophy behind this “new” way of monitoring phone calls…

Once, when I was running a Customer Service business for Lehman Brothers’ mortgage servicing subsidiary, I had a joint monitoring session with the entire Quality Assurance Unit.

To help prove a point.

We all listened to one of the best calls I’d heard in a very long time.

We actually “live-monitored” the call…it was going on right now & wasn’t one we pulled from our database of recorded calls.

The customer was absolutely infuriated when she got on the phone.

&$@?%#

Her monthly mortgage payment suddenly increased by ~$500…and she was furious! And we “never warned her nor told her why!!!”

(Supposedly.)

My rep handled the situation wonderfully right upfront by letting the customer speak & blow off steam…being empathetic…assuring her that there was probably a very valid reason for the “sudden increase…and that she would find it for, and explain it to, the caller.

After researching the account, my rep discovered that there was a new $500 escrow amount (in addition to the normal principal & interest payment) that suddenly increased the total payment due.

“Escrow” usually involves property taxes for the home/property paid to the county, by the bank, on behalf of the customer. It helps to “protect the bank’s interest in the property” by ensuring that all appropriate taxes are paid & that the county will not foreclose on the property for non-payment of taxes.

Escrow could also include payments for “forced-place insurance” if the customer fails to have, or provide tangible evidence of, a current casualty insurance policy on the home.

In these cases, the bank would pay the property taxes directly to the county &/or provide “in house insurance” themselves as the customer couldn’t/didn’t comply with several requests to substantiate the fact that the property was properly covered by casualty insurance.

The escrow part of a monthly mortgage payment “reimburses” the bank for monies they are spending (actually will spend) on behalf of the customer.

County property taxes are normally due ONCE A YEAR (usually a few months before or after the new year).

If you agreed that the mortgage company would pay this bill (on your behalf), then they will “charge your escrow” monthly (say, from Dec forward) to accumulate enough funds to pay the entire amount due in November.

If your account is NOT escrowed for property taxes, then you, yourself, must fork over the entire amount due every November. (The bank still “monitors” whether you’ve done this if they still hold your mortgage as they don’t want your “jointly-owned” property to be seized by the county in a tax sale!)

Well, what happened with this particular customer is that the county had granted her a “tax abatement” on the new house she had purchased 5 years earlier.

Property taxes for the 5-year abatement program were -0-.

Counties have these types of programs to encourage new hone construction in their jurisdiction.

“Hey, come build your new home in XYZ County & you won’t pay any property taxes for x years!!!”

(Very similar to what governments will do to attract corporations to build or expand…like SanAntonio did with Citi back in the early ’90s!)

Well, the customer’s home was 5 years old now, the abatement program expired & the county started to assess annual property taxes. Her loan stipulated that we would pay the property taxes on her behalf.

As such, the bank calculates what the payments are & then spreads that amount over the year so the customer can “pay their property taxes in monthly installments”. In reality, the bank pays the county all at once, but instead of having the customer reimburse them for the entire annual amount at one time, it spreads these “repayments to the bank” (escrow payments) over the entire year.

It’s similar in concept to being on a budget plan with your power company where you pay a steady, set monthly payment instead of having wildly-fluctuating payments (like during the summer with A/C) during certain times of the year.

As it turned out, the annual property taxes were now set at $6000 by the county.

Spreading that out over 12 months created the new escrow amount of $500/month.

It was certainly a legitimate part of the monthly payment due…but it came as a total surprise to this customer. (In reality, we had already sent a letter to the customer letting her know of this upcoming escrow payment of $500/month, but obviously she didn’t read it or didn’t understand it. It never should have been a surprise to the customer!)

My rep explained the whole situation in plain English & didn’t drown the customer with lots of technical terms & bank jargon.

The customer eventually understood exactly what was happening, did admit to receiving a letter the prior month & actually apologized to my rep for her attitude earlier in the call!

She was incredibly satisfied with the call & REPEATEDLY thanked the rep for a truly-wonderful experience.

You’ve never heard a customer change their tune like this. She simply showered the rep with profuse praise.

The call is now over.

Everyone in the room is lauding the rep for the great call she handled. One person even said it was the best call “since sliced bread”!

Now…

…we start to go around the room to first provide our individual scores & substantiate why we rated it as we did. (We were still on the 1-100 scoring system, but I was already actively campaigning to transition to what I helped create at Citi.)

They wanted me to go first, but I said that I’d like to go last (I never wanna show my hand until everyone’s bets have been placed) so around the room we go.

The first monitor gave the call a score of 94%.

She deducted 3 points for not using the standard closing (“Is there anything else that I may help you with?”) and then another 3 points off for “talking over the customer”. At one point in the call, there was some silence as the rep was checking on some information. The customer started to say something & so did my rep about a nanosecond later. To any reasonable person, simultaneously.

It wasn’t a big deal at all…the customer was not displeased nor annoyed in the least. My rep actually apologized for the very minor mishap (even though it was unavoidable & really, no one’s “fault”).

Yes, technically, she spoke over the customer with about 1/1000th of a word, but it wasn’t a case of truly “interrupting the customer” so, in reality, NO PLINTS should’ve been deducted!

Note: It does serve to illustrate the fact that the Monitoring Unit always maintained a “Gotcha!” attitude, so damned hungry to “catch the rep” doing something.

You guys know the type.

And their Director would cry to me on why “no one liked her people”!

Really? 🤔)

94 was the final score with these two 3-point infractions.

All the other Quality Assurance people “miraculously” had the same exact rating! (“Oh, what a co-inky-dink! A miracle!”)

*bows*

*does the Catholic “genuflect” thing* (It’s like a “1/2 kneel” thingie…down & up!)

I’m sure they may have slightly adjusted their scores (up or down) to “present a united, unified front” to me.

(Note: I had been trying for months to get them to change their scoring mechanism from the 1-100 scale (which, in my opinion, is viewed as “punitive” & “negative” to the reps) to the 5-category methodology that I had brought along from Citi with me.)

Now, it was my turn to reveal “my score”!

But before I did, I got up & said, “Pls give me a couple of minutes to handle something before I share my score with your guys.”

They looked at me a little puzzled.

“I hafta go out there & yell at “Suzy” about this call. Don’t worry…it shouldn’t take too long!”

They went crazy!!!

“But, Mike, what are you talking about?!? That was one of the best calls we’ve heard!”

“She completely turned the customer around!”

“Someone even commented that it was the greatest thing since sliced bread!”

I just stood there.

“I don’t care! I don’t give a damned!

“The call was rated a 94, right? Our standard for acceptable calls is 95 & above.

“I’m going out there to lay the lumber on her. She failed a call & I’m not gonna stand for it! Not in MY organization! No way, José!

“That’s totally unacceptable!”

Total silence.

I let the “absurdity” of their call scores fill the room.

“Everyone here commented how great a call this was. I even said so myself.

“But because of your rigid scoring methodology, you just hadda deduct 3 points for not using our standard closing, right?

“Were you listening to the same call that I heard? The customer was so ecstatic about the situation & finally understanding what escrow meant. It was pretty obvious why the customer called. Not using the standard closing was a minor error.

“And you guys were incredibly picky about the talking-over-the-customer thingie! It was an accident. She wasn’t trying to “talk over the customer” to get a point iacrosd while the customer was speaking. They both started talking at almost the same exact time after a short period of silence. The rep even apologized & the customer didn’t seem to mind at all.

“You actually FAILEDNa tremendously well-handled call. I’d have no problem having Terry (my boss’s boss) listen to this call. I’d love to play it for a training class as an example of how to handle the irate caller.

“This was an exemplary call when you take a step back & look at it in its entirety. All the great stuff she did, and how she went about it, greatly, greatly outweighs those insignificant, and even questionable, errors you pointed out.

“And no one took the customer’s obvious satisfaction with the service into consideration as a factor in rating this call…because you can’t!

“Your 1-100 tool doesn’t allow for it.

“I’m NOT trying to accuse you guys of improperly scoring this call.

“Rather, I want to demonstrate how a rigid, black-and-white scoring mechanism is simply not appropriate for rating customer interactions such as a phone call.

“The rep wasn’t completing a form. She was working with a human being, trying to figure out what the customer believed to be a terrible mistake. And the customer started out furiously & you just listened to how she sounded at the end of the call.

“That’s exactly why I’m pushing the new scoring method that I’m promoting.

“Think about this. Had this been a very simple 30-second call about a payment being received & no mistakes were made, you would have given that call a 100! Oh, assuming you used the official closing.

“To me, that would’ve been just an average call. At best! She SHOULD have advised the customer of 5 easier ways to make a payment than mailing us a check!

“This call we listened to, on the other hand, was truly exceptional. You know it & I know it!

“And whatever scoring system we use must be able to recognize the difference between the two & provide a comprehensive score that truly reflects the job that the rep did & the acceptance of that by the customer!”

I saw a few faces that reflected a degree of acceptance, acknowledgement, of what I’d just said. And, of course, there was some of the old guard who felt that they had just been one-upped.

If your organization is still using the archaic 1-100 scale (no matter how wonderful “they” think it works because they built in a gazillion “call/experience enhancers” into the scoring tables), perhaps you’ll want to discuss this topic with them.

Using 5 categories…Exceptional, Outstanding, Par, Needs Improvement, Unsatisfactory…provides you with the flexibility necessary to truly rate the call accurately.

It better reflects a fluid situation of one human being speaking with another human being where things like “feelings” & “perception” are important elements in measuring & grading how that customer experience transpired.

And your company probably already employs a similar rating/ranking system for annual performance appraisals & reviews.

Why not use similar terms & grouping for individual incidents that you already use for evaluating overall performance?

And it’s not punitive based. You’re not highlighting the mistakes they made.

Rather, you’ll be highlighting various opportunities they may have had to enhance the customer’s service experience.

And some very simple, basic inquiries may have very few opportunities for improvement, but you’ll never see them unless you get away from “scoring calls” to “valuations experiences”!

You should have to do something out of the ordinary to receive an Outstanding or Exceptional grade.

100% means “no mistakes”. Yes, you may “require” your people to go above & beyond on every call, but I seriously doubt it.

And even if you did, things tend to get lost if you have 5,000 different things the rep should have done on every phone call or with processing a widget.

Make it more of a positive experience for all involved.

No organization would dare celebrate every 100%-rated call it monitored…they would lose their importance, simply water things down & not really reflect the level of exceptional service you’re trying to provide.

Using this “new” scoring mechanism (probably 20 years old already), you should have no reservations whatsoever about making a really-big deal when Betty handles an Exceptional call.

Nor would you have any issue recognizing an Outstanding call.

One more thing…

We used numerical values to measure the quality of the call…then we’d find out what category they fell into.

Exceptional is 5.0

No other score. And if there were any minor errors/oversights, they would have to be pretty insignificant, have little-to-no impact on the result or to the customer & be tremendously outweighed by enhancements made to the call by the rep.

Outstanding is 4.0 to 4.9.

Par (Average, Good) is 3.0 up to 3.9. BTW, you decide how much an enhancement is “worth” or how much you should deduct for a mistake or an omission.

I’d recommend down to 1/10th of a point.

“Needs Improvement” is 2.0 to 2.9.

And pulling up the rear, “Unsatisfactory” is 1.0 to 1.9.

No scores lower than 1.0…no scores higher than 5.0

You’ll see that the Exceptional rating is very narrow…5 on the head.

(Alternate method:

Exceptional is 4.5 to 5.0

Outstanding is 3.5 to 4.4

“Par/Average/Good” is 2.5 to 3.4

“Needs Improvement” is 1.5 to 2.4

“Unsatisfactory” is 1.4 & under to 1.0)

It has to do more with “attitude” & “the customer’s perception” than mathematics.

At a minimum, it’ll make you think…

 

Thank you again for listening. If you’re interested, you can provide any feedback whatsoever (including requests) & I’ll be sure to help you in any way possible.