March, 2008, Volume 12, Number 5

CONTENTS  

Wharton Leadership Conference: Register Now

Spring is almost here, and the 12^th annual Wharton Leadership Conference is only three months away. Speakers include Johnson & Johnson CEO William Weldon, American Express CEO Kenneth Chenault, and Southwest Airlines president Colleen Barrett, among many others. Please join us for this one-day event on June 18, 2008, at The Wharton School in Philadelphia: You can register today.
 

EXPERT POLITICAL OPINION: On Foxes, Hedgehogs and the Pitfalls of Punditry  

By Mark Hanna 

The drawn-out political contest among presidential contenders this past year has forced journalists and political observers to work overtime, as they have been called upon to predict polling results and sketch out future scenarios. But the political pundits have scored poorly in this regard, as when Hillary Clinton was wrongly predicted to lose the New Hampshire primary, or when Mike Huckabee was consigned to irrelevancy early on – before he won the Iowa caucus and finished as the Republican runner-up.  

With such a patchy track record, then, it is natural to ask: Can experts really predict anything? 

Philip E. Tetlock, a professor of leadership at the University of California, Berkeley, and a psychologist by training, asked – and attempted to answer – the same question in his award-winning 2005 book, Expert Political Opinion: How Good Is It? How Can We Know? In the book, he explored the follies of expert predictions in a variety of fields, with a special focus on politics, and discovers which styles of thinking lead to more successful forecasting.  

Judging Judgment

According to Tetlock, good judges should pass two kinds of tests: 

Correspondence tests rooted in empiricism. How well do the judges’ private beliefs map into onto the publicly observable world? Good judges see the world as it is or will be.  

Coherence and process tests rooted in logic. Are their beliefs internally consistent? And do they update those beliefs in response to evidence?  

In other words, good judges should both “get it right” and “think the right way.” 

First, let’s examine the correspondence issue as it relates to forecasting accuracy. Tetlock wanted to study the predictive prowess of serious political experts using a regional forecasting exercise. He defined political experts as professionals who “make his or her livelihood by commenting or offering advice on political and economic trends.” He tracked the accuracy of 284 experts from dozens of countries on a wide range of topics, including the transitions to democracy and capitalism, economic growth, interstate violence, and nuclear proliferation. 

The selected experts were mostly male (76 percent) with an average age of 43 and approximately 12 years of relevant work experience. A little over half had doctoral degrees (52 percent), and almost all had postgraduate training (96 percent). They came from a variety of disciplines, including international relations, economics, national security, journalism, diplomacy, international law, and a number of specific area studies. They worked in academia, government, think tanks, foundations, international institutions, and the media.  

To judge their forecasting accuracy, Tetlock used the criteria of calibration and discrimination. Experts were said to be well-calibrated if their estimates of subjective probability corresponded with the objective reality (e.g., events estimated to occur 70 percent of the time should actually occur 70 percent of the time). Experts were said to be “perfectly discriminating” if they forecast a probability of one to events that actually happened and a probability of zero to events that did not. 

Tetlock looked at average calibration and discrimination scores for 27,451 forecasts, comparing the experts’ predictions with those of “minimalist performance benchmarks” – dilettantes, dart-throwing chimps, minimally-briefed undergraduates, simple base rates, crude extrapolation algorithms, and a sophisticated form of statistical forecasting known as autoregressive distributed lag modeling.  

When he examined the performance of the experts against the minimalist benchmarks, he found little evidence that expertise translates into greater ability to make either well-calibrated or discriminating forecasts. The experts’ performance was, in fact, so bad that they barely beat the chimps, winning on one variable (discrimination) but losing on the other (calibration). The extrapolation algorithms routinely outperformed human judges, and the hand-down winner of the forecasting contest was the sophisticated statistical model. 

Drilling Down: Foxes and Hedgehogs 

Looking at these results, some observers might conclude that good political judgment by human experts is merely a matter of good luck – no better than flipping coins or examining goat entrails. Tetlock was not one of them. He knew that some experts were clearly better than others, and he wanted to find out why. He drilled down further into his data, looking for distinguishing factors.  

What he found was startling. Who experts were – professional background, status, etc. – scarcely mattered. (One interesting finding here was that contact with major media was negatively correlated with accuracy.) Nor did what experts thought – whether they were liberals or conservatives, realists or idealists, optimists or pessimists. But one factor stood out as significant: how experts thought – their style of reasoning. 

Tetlock saw that the low and high variable loadings on cognitive style bore a remarkable resemblance to intellectual historian Sir Isaiah Berlin’s famous distinction between hedgehogs and foxes. Echoing the thoughts of ancient Greek poet Archilochus, Berlin observed that many of the world’s greatest thinkers can be classified as either hedgehogs, who know one big thing, or as foxes, who know many things. According to Isaiah Berlin, “Dante belongs to the first category, Shakespeare to the second; Plato, Lucretius, Pascal, Hegel, Dostoevsky, Nietzsche, Ibsen, Proust are, in varying degrees, hedgehogs; Herodotus, Aristotle, Montaigne, Erasmus, Molière, Goethe, Pushkin, Balzak, Joyce are foxes.” 

Tetlock fills out the distinction between his political expert hedgehogs and foxes and their forecasting accuracy this way: 

Low scorers look like hedgehogs: thinkers who “know one big thing,” aggressively extend the explanatory reach of that one big thing into new domains, display bristly impatience with those who “do not get it,” and express considerable confidence that they are already pretty proficient forecasters, at least in the long term.  High scorers look like foxes: thinkers who know many small things (tricks of the trade), are skeptical of grand schemes, see explanation and prediction not as deductive exercises but rather as exercises in flexible “ad hocery” that require stitching together diverse sources of information, and are rather diffident about their own forecasting, and…rather dubious that the cloudlike subject of politics can be the object of a clocklike science. 

In terms of Tetlock’s “correspondence tests rooted in empiricism” criterion, open-minded foxes were both better calibrated and more discriminating than the close-minded hedgehogs. Furthermore, foxes enjoyed their most decisive victories in long-term exercises inside their own domains of expertise. If foxes had any weakness at all, it was that they were prone to over-assign likelihoods to multiple future scenarios, in effect, violating the rule that all probabilities in a mutually exclusive, exhaustive set events must sum to one. Tetlock observed that at times, the foxes’ trendy open-mindedness started to look like good old-fashioned confusion. 

Regarding the “coherence and process tests rooted in logic” criterion, Tetlock found once again that foxes outperformed hedgehogs. Hedgehogs were slower than they should have been in revising the “one big idea” behind their forecasts. They weren’t good updaters like the foxes. Hedgehogs also tended to be very defensive about the incorrect forecasts, saying they were “only off on the timing,” or that they almost got it right, or that they would have been right had it not been for some unforeseen, low-probability event.  Tetlock saw that for the hedgehogs, “bad luck proved a vastly more popular explanation for forecasting failure than good luck proved for forecasting success.”  

Hedgehogs also seemed to have a double standard for weighing evidence. Their defiant attitude was, “I win if the evidence breaks in my direction” but, “if the evidence breaks the other way, the methodology must be suspect.” 

In Defense of Hedgehogs 

Lest the foxes get an overweening view of themselves, Tetlock points out that sometimes it helps to be a close-minded hedgehog. Based on fifty years of social science research, Tetlock concludes in part that hedgehogs have a: 

·         Tough negotiating posture that protect them from exploitation by competitive adversaries;

·         Willingness to take responsibility for controversial decisions guaranteed to make them enemies;

·         Determination to stay the course with policies, presumed sound, that run into temporary difficulties;

·         Capacity to inspire confidence by projecting a decisive, can-do presence.

Tetlock then devotes an entire chapter to addressing the numerous objections that hedgehog advocates raise in defense of themselves. In the end, each objection gets a fair hearing, but not a free pass. According to Tetlock, “hedgehogs made just too many mistakes spread across too many topics,” and thus, “qualified forms of the earlier indictments remain standing.” 

Final Observations 

Reading this book, one concludes that good judgment is a kind of precarious balancing act. Executing this balancing act requires skills of a higher order, in effect, thinking about how to think. As Tetlock says, “We need to cultivate the art of self-overhearing, to learn how to eavesdrop on the mental conversations we have with ourselves as we struggle to strike the right balance between preserving our existing worldview and rethinking core assumptions. This is no easy art to master.” 

Tetlock also has some policy recommendations for objectifying standards of judgment. He suggests holding pundits, politicians, and analysts systematically accountable to reasonable standards of evidence. He also suggests applying some of his performance metrics to real-world political controversies. Participants in political debates would be encouraged to translate vague claims into testable predictions that could be subsequently scored for empirical accuracy and logical defensibility.  

Until that happens, which isn’t likely to be soon, it’s best to cast a cold eye on many of the media’s political pundits, who tend to be hedgehogs. Additionally, one must understand the hedgehog’s strength, but keep that inner fox actively and critically engaged.   

Author’s Notes:  Mark Hanna is a freelance business researcher and writer based in Cedar Rapids, Iowa. He can be reached at markhanna@mchsi.com 

GLOBALIZATION and THE WORLD ECONOMIC FORUM: Q+A with Rakesh Khurana

On the final day of the World Economic Forum’s annual meeting in Davos, Switzerland, this January, Wharton Leadership Digest editor, Michael Useem, sat down with Rakesh Khurana to make sense of the Forum’s evolution over the years and the special role it plays in globalization. Khurana, a professor of management at the Harvard Business School, is author of the new book, From Higher Aims to Hired Hands: The Social Transformation of American Business Schools and the Unfulfilled Promise of Management as a Profession, a critical look at the rise and professionalization of U.S. business schools over the last 100 years.

Michael Useem: In your view, Rakesh, what does globalization mean when it’s used here at Davos?

Rakesh Khurana:  The way you phrase the question is exactly right – “What does it mean in the context of the World Economic Forum?” – because globalization as a term right now is used with significant plasticity. There are contested views about its consequences, whether it leads to positive or negative effects, on average, for individuals and nations and societies.

My impression is, at the Forum, most attendees typically use it in the context of economics: the increased levels of trade, and the interchange of services, and the interlinking of economies that were at one time isolated from another. So the dominant view here largely concerns economic exchange, in terms of the movement of capital, people and resources across the globe.

Underlying that, though, is a cultural definition of globalization, which I would call a convergence model of the world. It’s the view that what defines us as humanity matters more than the things that set us apart; that our views of others are often based on misperceptions; and that a greater understanding of people’s perspectives leads to respect and better relations, and, ultimately, a world focused on our commonalities. That is a very optimistic view of the world, rooted in the idea of neo-liberal markets, democratic institutions, and a humanistic vision.

Useem:  This year is your fifth annual meeting of the World Economic Forum.  Looking back over the last four or five years, how has use of the term “globalization” changed?

Khurana:  When I first attended five years ago, which was not long after September 11th, I noticed, for the first time, people pausing to wonder whether the convergence toward globalization was going to be as smooth as they had imagined. They took more seriously the idea that not only was economics going to be determine globalization, but also culture and people’s beliefs and perceptions. Since then, the conversation has become more sophisticated and nuanced, drawing on sociological concepts, such as the meanings people find in, and the attachments they have to, their ethnicities, religions and cultures.

The inequality issues associated with globalization have also become much more important, along with the recognition that interdependencies, while often creating a great deal of value, also challenge traditional notions of the nation-state’s autonomy – the term itself includes the word “dependency.” These issues are a legitimate part of the discussion in a way they were not in the pre-9/11, panglossian view of globalization.

Useem:  Yesterday, Rakesh, I attended a session you moderated, “Long‑term Value in a Short‑term World.” In the audience were attendees from Sweden, Italy, Australia, France, and beyond. On the way to this interview, after leaving a talk by the prime minister of Japan, I passed the finance minister of Egypt and Rupert Murdoch, not to mention Joseph Nye, former dean of Harvard’s Kennedy School. Given this incredibly diversity, do you think the Forum not only reflects the globalization of the economy and the wider world, but also contributes to that process?

Khurana:  Yes, there is a reflective component at work. Individuals who participate in the Forum not only represent their institutions but are also return to those organizations having been influenced by the Davos experience.   

From a participant’s perspective, that’s part of the attraction of Davos. It’s an opportunity to learn across disciplines, as well as hear from academics and be exposed to a huge range of cultures. This offers participants alternative ways to frame problems they are struggling with, especially those that cut across boundaries. The issue of how to manage for both the long-term and the short-term, for example, is an increasing challenge felt across industries and countries.

Davos also offers an opportunity for people to collect their wits. In a very busy world, it’s a break that allows people to review what they think and have their views challenged in a non-threatening way, and it gives them an opportunity to unfreeze a lot of their priors.

Useem:  When it comes to prominent political or corporate leaders attending the Forum’s annual meeting, some come, and some opt out. How do you explain those decisions to attend or not attend?

Khurana:  One typically sees four groups represented here. The first is business, which is probably the largest group, and if you are the CEO of a major company, you’re not just here as an accomplished individual, but as a representative of your institution. Having your institution “seen” at Davos can signal a global orientation, which is an important draw.

For political leaders, the second group, their attendance signals not only the aspirations of their countries, but also the major roles they may already play in globalization. It’s about showing the size, scale, and prestige of your country in the economic and political landscape of the world.

The third group is thought leaders. I use that term in a sociological sense, meaning the intelligentsia: academic and cultural leaders who are brought in to help frame and describe the issues.

The fourth group is media leaders, who emphasize the importance of transparency when you have political and business leaders brought together. Media players also amplify the effect of the meeting beyond the 2,500 people attending. If you open the pages of The Financial Times or The Wall Street Journal or The New York Times this week, you will see stories rooted in the debates and emerging consensus found here about certain problems around the world.  

Useem:  At the 1998 meeting – the first one I myself attended – the stars of the moment were people like Bill Gates of Microsoft and Andy Grove of Intel. There were very few representatives here from China and even India. Is Davos, then, a lagging economic indicator or a leading one?

Khurana:  When Bill Gates was the star in 1998, his affiliation was Microsoft. Gates was a star again this year, but in a very different role. He has shifted his attention and his agenda toward the problems of globalization, which he addresses through his philanthropy, the Gates Foundation. He sees an opportunity to help develop that agenda here.

At this meeting, he’s called for “creative capitalism,” in which success is defined not just by economic returns but by social achievements as well. He called gpt innovative thinking about ways to serve underserved markets, using both the powers of capitalism and markets and the power of people who want to have an impact and make a difference. So you see a single individual using the same platform over time to further very different agendas. 

India and China have both used the Davos platform in a similar way. In 1998, for example, India was aspiring to become a world-stage player, and its leaders came to learn what policies they should pursue. Over the last five years, however, India has become a major player, and I have seen India sharing the stage with countries who have “already arrived.” Similarly, we see countries and economies represented here who are trying to win greater attention and signal their willingness to open up.  

Useem:  As I mentioned at the outset, Rakesh, you’ve attended the Forum for the last five years running. Thinking about this very personally, how has the experience affected you?   

Khurana:  I feel very fortunate to be invited and to participate. As someone trained in organizational sociology, I have begun to understand different perspectives and how problems might be framed outside my own discipline. The experience has also allowed me to look with a critical eye at some things we all take for granted. You realize, “Sustainability might be more important to me than I thought.” On the other hand, it can be frustrating to see that how individuals interact and how their political leaders interact are two very different things.  

On a more personal note, I was born in India, raised in Queens, New York, and I have been able to travel around the world since then. Having that diaspora or outsider’s perspective, it is interesting to meet individuals who are also not focused solely on a particular nation or small region, but rather are thinking about how all these places integrate together. In conversation here, I find myself talking about how to find a sense of comfort and home in one place, while still looking for integration and commonalities across boundaries.
 

THE HEART OF LEADERSHIP: Why Some Marines Need Violins, and Some Executives Need Lower Pay 

By Russell Palmer 

In his new book, Ultimate Leadership: Winning Execution Strategies for Your Situation, Russell Palmer demonstrates that knowing the right principles of leadership is not enough: these ideals must be uniquely tailored to individual situations. Palmer draws on his own experience as former CEO of Touche Ross (now Deloitte & Touche,) former dean of the Wharton School, and now CEO of the private investment firm, The Palmer Group, and also brings in insights from leaders as diverse as author and executive Larry Bossidy; former Marine Corps commandant General P.X. Kelley; and John McKernan, the former Governor of Maine and chairman of Education Management Corp. In these three excerpts from chapter 10 of his book, Palmer explores how focusing on individuals, trimming executive offices, and linking pay to strategic metrics can advance a leader’s ultimate goals.     

Motivate Individuals 

Marcus Buckingham, who once headed the strengths management practice for The Gallup Organization, said good managers (and leaders) differ from bad managers (and leaders) in one basic respect: bad managers play checkers, while good managers play chess. “The good manager knows that not all employees work in the same way,” he said. “They know that if they are to achieve success, they must put their employees in a situation where they will be able to use their strengths. Great managers know that they don’t have 10 salespeople working for them – they have 10 individuals working for them.... A great manager is brilliant at spotting the unique differences that separate each person and then capitalizing on them.” 

Gen. P.X. Kelley, the former commandant of the U.S. Marine Corps and a member of the Joint Chiefs of Staff, tells about an experience he had that vividly illustrates the need for a leader to understand the individual concerns of every subordinate. When General Kelley was in Vietnam, one of the Marines under his charge had the reputation of being a troublemaker. Other officers had tried to correct the offender, but with little positive effect. Determined to understand the source of the Marine’s problems, General Kelley called the young man aside one day and began to ask questions about his home and background. To his surprise, he learned that the young Marine had trained as a classical musician playing the violin. 

Without saying anything to the Marine, General Kelley planned a surprise. He asked the officer responsible for ordering supplies to pick up a violin when he next flew out to get supplies for his unit. The officer was baffled, but orders are orders – and he did what he was told. 

Soon after the violin arrived, General Kelley called the Marine into his office. He had placed the violin behind the door, so the Marine couldn’t see it when he entered the room. “Son, how long has it been since you’ve played the violin,” the general asked. The Marine said it had been a very long time. General Kelly responded, “Then why don’t you turn around, and play me a tune?” 

The Marine looked behind him and saw the violin, and tears came to his eyes. As soon as the young man picked up the instrument, the general saw he was an experienced violinist. He played a melody for the general, and it seemed as though his world had changed. That Marine went on to become one of the most popular young men in his unit, constantly in demand for his musical talent at gatherings and parties. He became a model of discipline and good conduct. 

Keep the Executive Office Lean 

Today, in some organizations that are highly centralized, decision making gets moved more and more to the corporate offices. This is often a mistake when more and more decision making should be taking place in the divisions. The only real way to guard against this problem is to keep a lean executive office. As the CEO of Touche Ross, about every three years I made a 5% or 10% or 15% cut in the executive office staff. We didn’t necessarily fire the people – we moved many back out to the field. In addition, we never, ever took on more space for executive office functions. I’ve known leaders who have moved the corporate offices from one city to the next primarily so that they could cut the executive office staff in half. 

As a part of the impulse to augment the executive office, there is the tendency to overpay executive office people and underpay the line staff. If you look at those who are contributing to your revenues and profits, who are meeting the customers, who are in charge of developing the people, you will almost invariably come up with line managers. But often if you compare their pay to that of some in the executive office, it seems out of whack. To admit this is not to demean the executive office’s functions or the functional organizations that are needed in today’s Sarbanes-Oxley regulated world. But nothing can substitute for top leaders out in the field where the rubber meets the road getting the job done. It’s a terrific motivator for people to have the feeling that they are out there where it is happening, that they are making most of the decisions based on firsthand knowledge, that there isn’t some bureaucratic morass keeping them from moving ahead and getting the job done, and that they are being recognized for what they accomplish. 

At AEC, a Palmer Group acquisition, after the first CEO we hired didn’t work out, I hired Bill Brooks from National Education Corporation, where he headed the Spartan Aviation Training Division, to come and be the CEO. You might ask why he would leave a big organization and a very prestigious job and join a much smaller operation where he didn’t have the staff and corporate resources that he had in his previous position. The answer was that he wanted to run his own show. He didn’t want bureaucracy. Brooks was confident that he could do a better job making decisions from his vantage point rather than hearing them from some corporate officer. 

The first time we met to go over the following year’s budget, he submitted the budget and operating cash flow numbers. We looked through the plans and I asked, “What about your capital expenditure plan?” 

Brooks replied, “I haven’t put that together because I didn’t know how you did that here.” 

I asked, “Well, how did you do it at NEC?” 

“We submitted our plan early in the year for the next year’s capital expenditures budget,” he said. “We then got it back with some comments. We resubmitted it. We then had someone tell us we needed to cut it by 10% or 15%. We resubmitted it and then toward the end of the year they sent us what they thought our capital budget should be, and we moved ahead.” 

I asked him what he had in mind next year for AEC. He pulled out a sheet of paper and went over various things. I asked him a couple of questions and made a suggestion or two because of facts we knew that he would need to know. We then went on to other issues. 

At the end of the meeting he asked, “Now what do I need to put together to get my capital budget approved?” 

I said, “We just did that.” 

“That’s why I like working here,” he said. Top people, the kind you want working for your organization, want to be in charge of their area. They want to take responsibility, they want to make decisions, and they want to be able to move ahead with some speed as opposed to waiting for approval on everything from somewhere. This is a powerful motivator. A lean executive suite can help you achieve this. 

Tie Pay to the Strategic Plan 

After discussing nonrational incentives, I end with a very rational motivator: pay. It is a very important part of motivating workers. Today, pay and promotion are used more frequently to motivate employees than maintain a secure long-term relationship with the company in which the employee feels a part of what’s going on. However, pay has unfortunately become tied more and more to a percentage figure of profit in an organization. Profits are important, but they are only one element of myriad goals to be achieved in most strategic plans. And in the long term, the overall strategic goals are more important than short-term profits. 

You may hear the cry, “How can we measure all these soft items such as developing people, introducing new products, putting in a new marketing program, reducing turnover?” Further complicating this metric is that few managers want to be measured on growth as a factor as important as profit. Yet growth of the enterprise is at least as important in the long run as profits. These things and other more macro items in the strategic plan can be measured. It’s not easy, but it can be done. An effective incentive pay system has to be tied to the overall strategic plan. If you give one person responsibility for the strategic plan and someone else responsibility for the pay system, and they are not connected, who do you think is going to win? Obviously, it will be the person with the pay system.  

Let me end this section on the topic of CEO pay, which many feel is a tremendous problem. I don’t believe that the problem is in paying top leaders/CEOs whatever it takes to adequately reward them. The problems I see are in the number of CEOs who aren’t doing a good job or are doing a very bad job and getting paid as if they were doing an outstanding job. Basically, if you have absolutely outstanding CEOs/leaders, you can’t pay them too much; but for every one of those who are being paid a lofty amount, there is certainly more than one who is doing a mediocre or bad job who should be paid much less. It is extremely demotivating to the troops when they see their wages frozen, and people being laid off because the company isn’t making enough money, and yet the leader seems to be making an inordinate amount of money. Having said all that, I feel there are some CEOs who should try to hold their compensation down not because they don’t deserve it but so their action will provide a motivation and incentive to the troops who, for whatever reason, aren’t doing as well comparatively as the CEO.  

When I was at Touche Ross, every individual partner was listed in a book as to how much he or she made each year. By and large, the partners were happy until they got the book and saw how they were doing compared to others that they knew. Everyone wanted to be on the “first page” of the book, and as I remember in the early days, this page included 25 people. I told our people to make the type smaller so that we could get 50 people on the first page, and I think this was a plus. On the other hand, on several occasions I suggested to the people who were deciding my salary that I did not want the number to be over $X because that would put too much of a gap between me and certain of the other top people in the firm, and that would be demotivating.