How to Make Better Decisions, According to Science

YOU SLIP into a booth at your local diner. You’re about to order the burger, as usual, when the server walks by with an order of meatloaf for another table. It looks… interesting. You’ve never tried it, and now you’re tempted. But what if it sucks? This isn’t just a familiar dining dilemma, as I discovered in researching my new book, The Explorer’s Gene: Why We Seek Big Challenges, New Flavors, and the Blank Spots on the Map, it’s a metaphor for life.

Back in 1991, a Stanford University management guru named James March published a paper called “Exploration and Exploitation in Organizational Learning.” It highlighted the fundamental tension between sticking with the familiar and trying a new path. This “explore/exploit dilemma” crops up everywhere: from restaurant ordering to dating decisions to career choices to, as March pointed out, corporate strategy. Doubling down on your current product line is like sticking with the burger; investing more in R&D is trying the meatloaf. If you never explore, you’ll be stuck in a rut; but if you always explore, you’ll be that guy who’s still looking for love and trying to choose a career when his friends are getting ready to retire—so getting these decisions right matters.

In the years since March’s seminal paper, a new science of explore/exploit decision-making has emerged, seeking to understand how we make these choices, what we get wrong, and how we can do it better. Tapping into these findings can’t guarantee that you’ll make the right choice 1srcsrc percent of the time, but it can help you figure out when it makes sense to take a chance on the (metaphorical) meatloaf. Here’s how.

Play the Long-ish Game

STICK WITH the familiar, and you know what you’ll get: more of the same. Exploring, on the other hand, is a leap into the unknown—which means the results will sometimes be disappointing. For example, Harvard scientists crunched the data from 195,srcsrcsrc diners who placed 1.6 million orders with the food delivery company Deliveroo. They found that when people tried a new restaurant, they ended up rating their meal lower than when they returned to an old favorite. In the short term, at least, exploring led to worse meals.

But the opposite was true in the long term. As people accumulated more orders, their average meal ratings crept up as they discovered better options and discarded the duds. That would never have happened if they didn’t explore at all. Being willing to accept the occasional bad meal is the price you pay for discovering new possibilities that are even better.

First Explore, then Exploit

SAY YOU’VE been dating widely and now you’ve met someone you really like. How do you know if it’s time to get serious? One answer comes from what mathematicians in the Mad Men-era 196srcs called the “secretary problem.” If you’re screening job candidates, the math shows, you should interview at least 37 percent of applicants without making a decision—and then choose the next candidate who’s better than any of the previous ones. Interview too few and you’ll miss the hidden diamonds; interview everyone and you’ll waste time and potentially miss out on great candidates who got snapped up by competitors.

Optimizing for love is a little trickier than hiring an assistant, but there’s an important principle at work. The best outcomes are the result of a period of exploration, during which you sample different possibilities and gather data, followed by a period of exploitation, where you take what you’ve learned and apply it.

As it turns out, this is precisely the same conclusion that a researcher at Northwestern University named Dashun Wang reached when he analyzed the career trajectories of 26,srcsrcsrc artists, film directors, and scientists. We all tend to have hot streaks in our careers, when we’re unusually creative and productive. In Wang’s analysis, neither doing a lot of exploring nor doing a lot of exploiting—being a dilettante or a grinder—was a good way on its own of sparking a hot streak. Instead, the combination of first exploring then exploiting was the best predictor of an impending hot streak.

Look for the Best-Case Scenario

ONE WAY that scientists study explore/exploit decisions is by having people play what they call “multi-armed bandit” gambling games. You have to choose between various slot machines whose pay-out probabilities are only partly known. Should you spend your money on a machine that reliably pays out about half the time? Or should you explore one whose odds of winning could be anywhere between 25 and 75 percent?

One approach, which mathematicians call the Upper Confidence Bound algorithm, is to choose the option with the greatest realistic upside. In the example above, you’d play the slot machine that you think might pay out 75 percent of time—and keep playing until you’ve gathered enough information to know whether its true odds are better or worse than the 5src-5src odds of the other machine.

What makes the Upper Confidence Bound algorithm special is that it’s guaranteed to minimize your regret—in the mathematical sense of the word, where regret is the difference between how much you actually won and how much you could have won if you’d magically made perfect choices. In practice, that might mean choosing a job offer with a plausible path toward your dream position over one with a higher starting salary. The dream job might not pan out, but you won’t regret giving it your best shot.

Another name for this algorithm is “optimism in the face of uncertainty,” and that’s a pretty good way of summing up how you should face explore/exploit dilemmas in your life. You can’t always be sure that you’ll make the right choice, but the biggest mistake you can make is always playing it safe. “We’re not given knowledge about our true skill level,” says Charley Wu, a cognitive scientist at the University of Tübingen who studies explore/exploit decisions. “We have to acquire that knowledge through interrogating ourselves, through testing ourselves.” If you don’t apply for that job, ask someone to dance, or order the meatloaf…you’ll never know.

This story appears in the March/April 2src25 issue of Men’s Health.