Why does pessimism sound so smart?

"For reasons I have never understood, people like to hear that the world is going to hell," historian Deirdre N. McCloskey told The New York Times earlier this year.

It's hard to argue. Despite the record of things getting better for most people most of the time, pessimism isn't just more common than optimism, it also sounds smarter. It's intellectually captivating, and paid more attention to than the optimist, who is often viewed as an oblivious sucker.

It's always been this way. John Stuart Mill wrote 150 years ago: "I have observed that not the man who hopes when others despair, but the man who despairs when others hope, is admired by a large class of persons as a sage."

Matt Ridley wrote in his book "The Rational Optimist": "If you say the world has been getting better you may get away with being called naive and insensitive. If you say the world is going to go on getting better, you are considered embarrassingly mad. If, on the other hand, you say catastrophe is imminent, you may expect a MacArthur genius award or even the Nobel Peace Prize."

In investing, a bull sounds like a reckless cheerleader, while a bear sounds like a sharp mind who has dug past the headlines -- despite the record of the S&P 500 rising 18,000-fold over the last century. Wharton professor Jeremy Siegel is often chided as a perma-stock-bull, blindly cheering for a higher market every time he goes on TV. But he's done it since the early 1980s, a period in which the market increased in value 40 times over. Alas, few care about past results when someone else is warning about the Next Great Depression.

This goes beyond investing. Harvard professor Teresa Amabile shows that those publishing negative book reviews are seen as smarter and more competent than those giving positive reviews of the same book. "Only pessimism sounds profound. Optimism sounds superficial," she wrote.

Why?

There's clearly more at stake with pessimism. Daniel Kahneman won the Nobel Prize for showing that people respond stronger to loss than gain. It's an evolutionary shield: "Organisms that treat threats as more urgent than opportunities have a better chance to survive and reproduce," Kahneman once wrote. Here are a few other reasons I've observed for why pessimism gets so much attention:

1. Optimism appears oblivious to risks, so by default pessimism looks more intelligent. But that's a wrong way to view optimists. Most optimists will tell you things will get ugly, that we'll have recessions, bear markets, wars, panics and pandemics. But they remain optimistic because they set themselves up in portfolio, career and disposition to endure those downsides. To the pessimist, a bad event is the end of the story. To the optimist, it's a slow chapter in an otherwise excellent book. The difference between an optimist and a pessimist often comes down to endurance and time frame.

2. Pessimism shows that not everything is moving in the right direction, which helps you rationalize the personal shortcomings we all have. Misery loves company, as they say. Realizing that things outside your control could be the cause of your own problems is a comforting feeling, so we're attracted to it.

3. Pessimism requires action, whereas optimism means staying the course. Pessimism is "SELL, GET OUT, RUN," which grabs your attention because it's an action you need to take right now. You don't want to read the article later or skim over the details, because you might get hurt. Optimism is mostly, "Don't worry, stay the course, we'll be all right," which is easy to ignore since it doesn't require doing anything.

4. Optimism sounds like a sales pitch, while pessimism sounds like someone trying to help you. And that's often the truth. But in general, most of the time, optimism is the correct default setting, and pessimism can be as big a sales pitch as anything -- especially if it's around emotional topics like money and politics.

5. Pessimists extrapolate present trends without accounting for how reliably markets adapt. That's important, because pessimistic views often start with a foundation of rational analysis, so the warning appears as reasonable as it is scary.

In 2008, environmentalist Lester Brown wrote: "By 2030 China would need 98 million barrels of oil a day. The world is currently producing 85 million barrels a day and may never produce much more than that. There go the world's oil reserves."

He's right; we'll run out of oil in that scenario. But that's not how markets work. A shortage pushed up oil prices, high prices incentivized producers to come up with new drilling techniques, and now we have more oil than we know what to do with. World oil production last year was 96 million barrels -- already way above what he thought was the high mark. Failing to account for markets' ability to adapt is the cause of death of most pessimist forecasts.

Should you ever listen to pessimists? Of course. They're the best indication of what's unsustainable, and thus probably about to change, and thus the soil of what's to be optimistic about.

Morgan Housel, a columnist at The Motley Fool, is a two-time winner, Best in Business award, Society of American Business Editors and Writers and Best in Business 2012, Columbia Journalism Review.