The BBVA Foundation Frontiers of Knowledge Awards in Economics, Finance and Management has gone in this eighteenth edition to Charles F. Manski for his pioneering contributions to the measurement of uncertainty in economic research and its application to public policy analysis. The professor at Northwestern University (Chicago, United States) is described by the committee as a “foundational figure” in the development of modern methods that have transformed how economists infer conclusions from data, report degrees of uncertainty in their models, and evaluate public policies in the face of incomplete evidence.
His work over the course of five decades “has profoundly influenced empirical research across education, health policy, labor markets, industrial policy, and social programs, by encouraging economists to rely on credible, transparent inference,” regarding the assumptions on which they base their research.
“The methods he developed assess the degree of confidence we can have in empirical measurement,” the citation continues, enshrining him as “a critical conscience of measurement in the social sciences.”
Manski’s research “has uncovered some of the erroneous assumptions we economists are prey to, which make our predictions and understanding of behavior quite fragile,” says committee member Sir Richard Blundell, David Ricardo Professor of Political Economy at University College London (United Kingdom). “He has taught us to look carefully at the assumptions underpinning our analysis and to base both our predictions and understanding of behavior on evidence that we can really believe in.”
Manuel Arellano, Professor of Economics in the Center for Monetary and Financial Studies (CEMFI) of Banco de España, and secretary to the committee, hails Manski as “a major innovator in methods of empirical measurement in economics and social sciences, who has also made fundamental contributions to methods for measuring the range of possible outcomes that be conclusively posited in policy analysis under conditions of uncertainty.”
Prof. Arellano regards Manski as especially influential in the measurement of economic agents’ expectations: “Firms, households, individuals…, we all make decisions or hold off from making them depending on how sure we feel about our future circumstances. If I can be confident of what my income will be next year, I will opt for consumption decisions that I would make differently if I had serious doubts.”
Manski, he continues, has been at the forefront in advocating the quantification of agent uncertainty: “This means, for example, using surveys to measure the probabilities people assign to the price of their home or their income increasing, or to being unemployed at some point in the future. Manski has been an innovator in measuring such expectations in surveys and in how to use them in economic analysis. This activity was initially greeted with skepticism, but now central banks like Banco de España, Banca d’Italia and the New York Federal Reserve conduct regular surveys of agent expectations, relying largely on the ideas launched by Charles Manski.”
“Most economists,” says Manski, “don’t deal with uncertainty. They would prefer to get firm answers to questions. And this is particularly the case in studying public policy.” The public, he explains, “wants to have answers to know if a policy is good or not, and economists like to provide them.” The result is that conclusions in economics are frequently characterized by what he calls “incredible certitude,” with figures and percentages that lack robust empirical support.
“On those occasions when economists do explicitly deal with uncertainty, they do so by stating that there is a 10% or 20% chance that a policy will have a given effect. But my own work focuses on more difficult situations where you can’t put probabilities on things.”
Manski has pioneered the development of econometric methods to study precisely these situations of “deep uncertainty,” with this uncertainty factored into the analysis: “These are difficult public policy problems, and what you really need to do is to quantify the uncertainty. What that implies is that instead of providing a point estimate of some quantity, like what the tax revenue will be under certain income tax policies, I might give a bound, an interval, to say it will be between this and that level. And the width of that bound is going to express how much uncertainty there is. The bound may be very narrow, meaning we know a lot, or very wide, meaning we know relatively little.”
In the late 1980s, during a term directing the Institute for Research on Poverty at the University of Wisconsin-Madison, Manski identified a methodological gap that would change the course of his academic career and ultimately transform modern econometrics.
A colleague of his was conducting a study into trajectories of homelessness, which had run into a critical problem of missing data. Its design was that of a longitudinal study following the lives of homeless people, but of the original sample only around 60% could be located one year later. The main obstacle to data collection was the absence of a fixed address for as many as 40% of the reporting subjects.
Manski explains: “The standard procedure at the time was to assume that when people leave a sample, they do so at random, which implies that the subjects the team couldn’t locate after a year were just like the subjects that they could. In other words, if the people that you can’t find are essentially the same as those you are able to interview for a second or the third time, then you don’t need to worry about the missing data.”
But he was not convinced. His reading was that the homeless individuals who could not subsequently be found were likely to be experiencing systematically worse conditions than those who reappeared, precisely because they had gone off grid. And this would invalidate any conclusion based on random missingness.
This problem, which Manski has explored throughout his career, led him to the concept of partial identification, one of his most notable contributions to the field of econometrics. His proposal – as developed in Identification for Prediction and Decision (Harvard University Press, 2007) – was to replace the misleading certainty of a specific value with the use of intervals or bounds; a fundamental methodological contribution whereby, rather than engineering a point estimate, i.e., a single numerical value, a range of outcomes is offered that takes data uncertainty into account.
“The idea is that you cannot arrive at definitive conclusions or definitive numbers, but must always work with intervals. Ideally, you would like the intervals to be as small as possible, but then there’s a trade-off. You can tighten the interval by making more assumptions, but there is a danger in that,” he warns. For, in effect, “you can draw strong conclusions if you make strong assumptions, but then they won’t be believable. As a kind of a shorthand, I call this process the law of decreasing credibility.”
For Iván Fernández Val, Professor of Economics at Boston University, currently visiting at CEMFI, this contribution of Manski’s has been “foundational and transformative,” because it “has changed the way economists think and how we look at data.” To illustrate the importance of the partial identification concept, Fernández Val gives the example of an electoral poll where only two parties are standing: “Some people are going to respond, but others will decline. So even if you interview the entire electorate, you can never know exactly which percentage will vote each way. All you can say is that there is a range of possible proportions for each of the parties, consistent with the observed responses. This observation fundamentally changes the way data is analyzed. Instead of using methods that seek to estimate a single number, what you get is a range of possible values. This may complicate the exercise, but it offers a more realistic picture of the uncertainty inherent in estimates.”
Charles Manski’s methodological contributions have found practical expression in public policy design, especially in education. The economist’s interest in the field dates back to his PhD thesis, in which he analyzed decision-making at the pre-university stage, focusing on how pupils in their last year of secondary school came to decide whether or not to go to college and, in the affirmative case, which college they would choose.
“I wanted to understand college-going decisions,” says Manski, adding that his interest was driven by a policy question: “In the early 1970s, the federal government began a grant policy, a scholarship scheme. The idea was to try to get more low-income students to go to college, because tuition was expensive.”
His response was to develop a counterfactual hypothesis – an approach that sets out alternative scenarios and compares them to the real outcome – seeking to understand how students take decisions under uncertainty and to assess the impact of the economic assistance offered. “The problem was that we had data on the numbers attending university at the time, but what we wanted to know was how many more 18- or 19-year-olds would choose to go to college and what kinds of colleges they would choose,” if the new grant policy was implemented.
Manski wrote up his conclusions in a paper co-authored with David Wise of the Kennedy School at Harvard, which would lead on to their 1983 book College Choice in America and culminate in one of the awardee’s landmark works, Public Policy in an Uncertain World: Analysis and Decision (2013).
When designing public policies that involve social dynamics, as in the education field, it is important to know how individuals influence each other. It was to address this need that Manski formalized what is known as the “reflection problem,” mathematically analyzing the complex question of whether an individual is acting under the influence of their immediate peers or whether the influence is reciprocal. Among its other uses, this approach can elucidate how the make-up of a class influences the way students learn and interact with each other.
“If I belong to a class made up of excellent students, is it me who is good or am I reflecting how good my classmates are? To what extent can I separate the influence of my classmates’ high ability from the fact that I am with them? And what is my own influence?” explains José García Montalvo, Professor of Economics and Business at Pompeu Fabra University. “If my classmates influence me but I also influence them, how do we separate those two things? In most circumstances we can’t.”
In this scenario, García Montalvo continues, Manski proposes carrying out observations at various points over a period of time. This may not yield a precise value for the influence at work but will at least give “an uncertainty interval that is not too wide.”
Another area of research enriched by Manski’s contributions is the measurement of uncertainty in health and medical decision-making, as described in his book Patient Care under Uncertainty (Princeton University Press, 2019).
“You might think that medical treatment and clinical decision-making would be a strange topic for an economist to study,” he observes, “but actually there’s a lot in common with studying public policy. A medical doctor is basically acting as what we call in economics a social planner on behalf of the patient. The doctor is trying to do the best thing possible for the patient, whether they suffer from heart disease or cancer or diabetes or whatever, but there’s uncertainty everywhere in medicine.
Manski contends that the same methodological tools developed for measuring uncertainty in economic research and public policy analysis can be applied to help physicians, health authorities, and patients themselves make better decisions in the presence of incomplete or ambiguous evidence. His book analyzes the considerable uncertainty that exists regarding issues like a patient’s real state of health, their potential response to a given treatment, or the evidence derived from clinical trials on a drug’s real effectiveness.
All too often he says, we have cases where the results of clinical trials conducted on a limited group of patients are erroneously extrapolated to the general population, statistical errors occur when opting for one therapy over others, and claims about a drug’s effectiveness do not stand up to analysis of the real-world evidence. For all these reasons, the awardee is a strong advocate of incorporating the case-by-case measurement of real uncertainty into healthcare and medical practice in order to make better evidence-based decisions.
“Doctors do all they can to help their patients,” says Manski, “but it’s difficult for them to deal with uncertainty, and often they have these situations of ambiguity or deep uncertainty where they can’t really put probabilities on competing treatment options. So it turns out that the same methodological research that I do on public policy is also applicable to medical decision-making. Health issues seem to me hugely important, which is why I have a whole set of collaborations with medical researchers and health economists to address decision-making problems in this domain.”
Manski declares himself proud to be described by the Frontiers of Knowledge committee as “a critical conscience of measurement in the social sciences,” since he “cares deeply” about the credibility of the assumptions economists and other specialists bring to bear in their work. In this respect, he describes himself as healthily skeptical.
“There’s a lot of hype in research, there’s a lot of marketing that goes on, where economists try to draw very strong conclusions that are actually not that believable. And what econometricians like myself do is to say, wait a minute, you have no real basis to draw that conclusion. This, of course, is particularly important with public policy, because public policy is innately heavily political, and people are going to look at it from different positions. It’s essential therefore that the research is robust and credible so people will believe it, and not think somebody’s just making it up.”
At the age of 77, Manski combines a busy teaching schedule with his continuing efforts to refine measurement of uncertainty, with the aim of optimizing decision-making based on the best possible evidence, even among non-econometricians: “I’m currently working on a practical project to turn all these ideas into a web application that even someone without mathematical knowledge can use to optimize decision-making in healthcare or any other field.”
Charles F. Manski (Boston, Massachusetts, United States, 1948) received his bachelor’s degree (1970) and PhD (1973), both in economics, from the Massachusetts Institute of Technology. He spent the first twenty-five years of his academic career at Carnegie Mellon University (1973-1980), The Hebrew University of Jerusalem (1979-1983) and, on his return to the United States, the University of Wisconsin-Madison (1983-1998), where he held a series of professorships in the Department of Economics, as well as leading its Institute for Research on Poverty. Since 1997, he has been Board of Trustees Professor in Economics at Northwestern University (Evanston, Illinois), where he also chaired the Department of Economics from 2007 to 2010. Manski is the author of numerous research papers and nine books, including Discourse on Social Planning under Uncertainty and Identification for Prediction and Decision . He has served as Chair of the Board of Overseers of the Panel Study of Income Dynamics (1994-1998) and as Chair of the National Research Council Committee on Data and Research for Policy on Illegal Drugs (1998-2001). His editorial service includes terms as editor of the Journal of Human Resources , co-editor of the Econometric Society Monograph Series, and a member of the Editorial Board of the Annual Review of Economics .
A total of 82 nominations were received in this edition, comprising 70 candidates. The awardee researcher was nominated by Thierry Magnac , Professor of Economics at the University of Toulouse (France), and Richard J. Smith , Emeritus Professor of Econometric Theory and Economic Statistics at the University of Cambridge (United Kingdom).
The committee in this category was chaired by Eric S. Maskin , Adams University Professor in the Department of Economics at Harvard University (United States) and 2007 Nobel Laureate in Economic Sciences, with Manuel Arellano , Professor of Economics in the Center for Monetary and Financial Studies (CEMFI) of Banco de España acting as secretary.
Remaining members were Sir Richard Blundell , David Ricardo Professor of Political Economy at University College London (United Kingdom) and 2014 BBVA Foundation Frontiers of Knowledge Laureate in Economics, Finance and Management; Antonio Ciccone , Professor of Economics at the University of Mannheim (Germany); Pinelopi Koujianou Goldberg , William Nordhaus Professor of Economics and Global Affairs at Yale University (United States); Andreu Mas-Colell , Professor Emeritus of Economics at Pompeu Fabra University and the Barcelona School of Economics (Spain) and 2009 BBVA Foundation Frontiers of Knowledge Laureate in Economics, Finance and Management; Lucrezia Reichlin , Professor of Economics at the London Business School (United Kingdom); and Fabrizio Zilibotti , Tuntex Professor of International and Development Economics at Yale University (United States).
The evaluation support panel was coordinated by Elena Cartea , Deputy Vice-President for Scientific-Technical Areas at the Spanish National Research Council (CSIC), and Joan Llull Cabrer , Research Professor at the Institute for Economic Analysis (IAE, CSIC). Its members were: Manuel José García Santana , Professor in the Department of Economics and Business at Pompeu Fabra University (UPF); Inés Macho Stadler , Professor of Economics in the Economic Sciences Faculty at the Universitat Autònoma de Barcelona (UAB); Xavier Ramos Morilla , Professor in the Department of Applied Economics at the Universitat Autònoma de Barcelona (UAB); and Virginia Sánchez Marcos , Professor of Fundamentals of Economic Analysis in the Department of Economics of the University of Cantabria (UNICAN).