Innumeracy among the numerati

(5-minute read)

Convergys Consulting is a company that sells data analytics services to call centers, for the purpose of identify operating cost reduction opportunities at those call centers. The company asks prospective new-hire analysts to do a take-home analysis of 120 rows of call center data. Now, I am 20 years past being a lowly analyst, but a networking acquaintance nonetheless encouraged me to apply for the role, given that she believed her firm would love to have a quantitative expert on staff. And I would have been extraordinarily happy to have the work.

I asked this acquaintance if I should dumb myself down to get the job, but she assured me that this firm is an ego-less meritocracy and therefore I did not need to strategically self-censor. So, as my deliverable for the take-home interview exercise, I made a ten-page PowerPoint–with data visualizations so slick that I’ve since used them as samples of great data visualizations. It featured a storyboard structure, trend summary (by call center agent, agent tenure, day of week, over time, by topic, etc), hypothesized causal relationships to investigate outside the data, Tufte-style clean design, and concrete next steps to pitch a meaty engagement to the fictive prospective client. I hit this thing so far out of the proverbial park it that landed in the next stadium.

In addition, in my deliverable, I explained that all inferences were necessarily preliminary, because the low sample size provided translated into large margins of error—even at an aggressively low confidence level. In fact, there were no statistically significant differences in the data Convergys provided for the exercise! None. In other words, all of the blips in their 120 rows of data had a strong chance of merely being random noise.

This is the kind of situation where one’s insights are such a wake-up call that the company reacts with one of two extremes: either they (a) haul you in to fix how they do things, or else they (b) reject you in terror at realizing they don’t understand what they’re doing as well you do. In Convergys’ case, I just never heard back. Not even a “thanks-but-no-thanks”. And that networking acquaintance never again returned my emails or calls. 

Ego over truth

Twenty-ish years ago, after I got the Golden Ticket of an MBA summer internship in the Goldman Sachs M&A group, I discovered a calculation error in their Excel valuation model. This was the model that the most elite department inside the most elite Wall Street firm used to value every single multi-billion-dollar, front-page-WSJ transaction. They were proud of the model, and they asked us eight carefully-selected interns from the top business school programs to check the model in our spare time – issuing a challenge they believed was ironclad. I was the only one of us eight who found something. I wrote a gentle, respectful email, n direct response to the error-checking request. I explained the issue I had discovered and–in obsequiously non-threatening, self-deprecating female fashion–gushed that this was not a material error in almost any case but was an error nonetheless.

This didn’t make me particularly popular. My boss frostily told me to “go shopping”, since there were no deals available for me to be staffed on.  

That same summer in Manhattan, at a restaurant, I got a glass of lemonade with a floating piece of lemon that still had its sticker attached. Meaning they hadn’t washed the lemons to make the lemonade, and so not likely also the fruit for the fruit salad. Arguably, since produce stickers are food-grade and edible, it’s not a material error. But it nonetheless tells you something about the level of care and competence in the kitchen, and it makes you wonder what more egregious issues you didn’t catch. (And, finding paper in a mouthful of food is just gross.)

But, the waiter was pissed at my request for new lemonade.

The next year, I visited my new fiancee’s parents’ house for the holidays. He had been a Harvard math whiz. He thought it would be fun to get out one of his old statewide math competition tests and have the two of us compete to do the test. I got the same score as he did . . . despite the fact that he was a math major (I was literature), he went to Harvard (I to one of the “lesser” Ivies), and he had done the test in high school (it was sight unseen to me).

In a flash of rage at the outcome, he threw his pencil across the room.

Truth over ego

Once, I did get hired by a consulting firm explicitly to revamp their numeric capabilities–after I pointed out that their Fortune 50 pharmaceutical client was making global business decisions on data “trends” in fact indistinguishable from random noise. Neither the consultancy nor their clients knew what a margin of error was or how to calculate one. I brought in several new tools to improve decision quality and lower the cost of decision analysis: Bayesian methods, lower confidence levels, better survey design, reliably automated calculations, hypothesis testing to replace data mining, and framing research questions within business strategy. My boss was embarrassed. But at the same time, he was eager enough for a competitive edge in the consulting services market to ride through his own discomfort. It was only my own loyalty that faltered in this case: when I found him cooking the books to his subcontractors, I blew the whistle.

Then there was the time, at the idealistic young age 26, I explained Monte Carlo simulation to an insecure, hot-headed, community college-educated boss, and he literally punched a wall in frustration that he couldn’t follow me. But the CEO (my boss’s boss) sought me out directly, bringing little old me to the Board of Directors meeting to explain my point directly to the receptive Board. I was thanked for my precociousness with a promotion and massive bonus.

Impressive leaders don’t feel they need to know more than their quant analysts. But, such leaders are rare exceptions. In twenty-some years, I’ve never again met someone so unthreatened by a smart employee as that particular CEO. (I would have eagerly defined my career by following that CEO, but a few months later he tragically died of cardiac arrest while jogging.)

Conclusion

The story at hand–of not getting a callback from Convergys–has many themes. It is a story about the consequences of being overqualified for a position and of being smarter than the boss, a story of a would-be academic mind forced to grovel for acceptance in the daft corporate world, a story about the psychological barriers to innovation and learning, a story about humans projecting destructive competition onto situations that should be about productive collaboration, and a story most concretely about innumeracy among self-proclaimed numerati.

Falling over oneself to apologize for one’s own competence is not usually enough to mitigate other people’s terror at confronting their own incompetence. However, hiding your competence will not get you a good job, a promotion, or a satisfying career either. The solution–as with most issues in life–is threading the needle, finding a strategic middle ground.

In my experience, many firms using and selling data analytics mess up basic statistics. Yet the world keeps turning.

– January 2017