Book excerpt: Bonuses and promotions causing bad incentives

(This is an excerpt from my book, "Algorithms and Misinformation: Why Wisdom of the Crowds Failed the Internet and How to Fix It")

Bonuses are a powerful incentive. Technology companies are using them more than ever. Most technology companies cap salaries and instead use bonuses and stock grants as most of their compensation for employees.

These bonuses are often tied to key metrics. For example, imagine that if you deploy a change to the recommendation algorithms that boosts revenue by a fraction of a percent, you would get the maximum bonus, a windfall of a million dollars, into your pocket.

What are you going to do? You’re going to try to get that bonus. In fact, you’ll do anything you can to get that bonus.

The problem comes when the criteria for what gets the bonus isn’t exactly correct. It doesn’t matter if it is mostly correct — increasing revenue is mostly correct as a goal — what matters is if there is any way, any way at all, to get that bonus in a way that doesn’t help the company and customers.

Imagine you find a way to increase revenue by biasing the recommendations toward outright scams, snake oil salesmen selling fake cures to the desperate. Just a twiddle to the algorithms and those scams show up just a bit more often, and that nudges the revenue just that much higher, at least when you tested it for a couple days.

Do you roll out this new scammy algorithm to everyone? Should everyone see more of these scams? And what happens to customers, and the company, if people see all these scams?

But that bonus. That tasty, tasty bonus. $1 million dollars. Surely, if you weren’t supposed to do this, they wouldn’t give you that bonus. Would they? This has to be the right thing. Isn’t it?

People working within technology companies have to make decisions like this every day. Examples abound of ways to generate more revenue that ultimately are harmful to the company, including increasing the size and number of paid promotions, salacious or otherwise inappropriate content, deceptive sales pitches, promoting lower quality items where you receive compensation, spamming people with takeover or pop-up advertising, and feeling strong emotions such as hatred.

As an article in Wired titled “15 Months of Fresh Hell Inside Facebook” described, this is a real problem. There easily can be “perverse incentives created by [the] annual bonus program, which pays people in large part based on the company hitting growth targets.”

“You can do anything, no matter how crazy the idea, as long as you move the goal metrics,” added Facebook whistleblower Francis Haugen. If you tell people their bonus depends on moving goal metrics, they will do whatever it takes to move those goal metrics.

This problem is why some tech companies reject using bonuses as a large part of their compensation. As Netflix founder and CEO Reed Hastings explained, “The risk is that employees will focus on a target instead of spotting what’s best for the company in the present moment.”

When talking about bonuses in our interview, a former executive who worked at technology startups gave the example of teams meeting their end-of-quarter quotas by discounting, which undermines pricing strategy and can hurt the long-term of the company. He also told of an executive who forced a deal through that was bad for the company because signing the deal ensured he hit his quarterly licensing goal and got his bonus. This other executive, when challenged by the CEO, defended his choice by saying he was not given the luxury of long-term thinking.

“We learned that bonuses are bad for business,” Netflix CEO Reed Hastings said. “The entire bonus system is based on the premise that you can reliably predict the future, and that you can set an objective in any given moment that will continue to be important down the road.”

The problem is that people will work hard to get a bonus, but it is hard to set a criteria for bonuses that cannot be abused in some way. People will try many, many things seeking to find something that wins the windfall the company is dangling in front of them. Some of the innovations might be real. But others may actually cause harm, especially over long periods of time.

As Reed Hastings went on to say, what companies need to be able to do is “adapt direction quickly” and have creative freedom to do the right thing for the company, not to focus on what “will get you that big check.” It’s not just how much you pay people, it’s also how you pay them.

Similarly, the people working on changing and tuning algorithms want to advance in their careers. How people are promoted, who is promoted, and for what reason creates incentives. Those incentives ultimately change what wisdom of the crowd algorithms do.

If people are promoted for helping customers find and discover what they need and keeping customers satisfied, people inside the company have more incentive to target those goals. If people are promoted for getting people to click more regardless of what they are clicking, then those algorithms are going to get more clicks, so more people get those promotions.

In the book An Ugly Truth, the authors found Facebook “engineers were given engagement targets, and their bonuses and annual performance reviews were anchored to measurable results on how their products attracted more users or kept them on the site longer.” Performance reviews and promotions were tied with making changes that kept people engaged and clicking. “Growth came first,” they found. “It’s how people are incentivized on a day-to-day basis.”

Who gets good performance reviews and promotions determines which projects get done. If a project that reduces how often people see disinformation from adversaries is both hard and gets poor performance reviews for its team, many people will abandon it. If another project that promotes content that makes people angry gets its team promoted because they increased engagement, then others will look over and say, that looks easy, I can do that too.

In the MIT Technology Review article “How Facebook Got Addicted to Spreading Misinformation,” Karen Hao described the incentives: “With their performance reviews and salaries tied to the successful completion of projects, employees quickly learned to drop those that received pushback and continue working on those dictated from the top down.”

The optimization of these algorithms is a series of steps, each one a small choice, about what people should and shouldn’t do. Often, the consequences can be unintended, which makes it that much more important for executives to check frequently if they are targeting the right goals. As former Facebook Chief Security Officer Alex Stamos said, “Culture can become a straightjacket” and force teams down paths that eventually turn out to be harmful to customers and the company.

Executives need to be careful of the bonus and promotion incentives they create for how their algorithms are tuned and optimized. What the product does depends on what incentives teams have.