>> Welcome back everybody and good afternoon. I know a number of people have joined us and there are some watching online and so I wanted to say again that I am Susan Collins, the Joan and Sanford Weill dean of the Gerald R. Ford School of Public Policy and we're delighted to have all of you here with us on the occasion of both the 100th anniversary of the Federal Reserve opening its doors for business and also the 100th anniversary of public policy at the University of Michigan, the Ford School centennial. Well today's keynote speaker is Betsey Stevenson who is a highly accomplished economist who is currently on leave from the University of Michigan serving as a member of President Obama's Council of Economic Advisors and I'm really proud and honored to have her as a member of the Ford School faculty. She previously served as chief economist at the Labor Department and a research associate at the with the National Bureau of Economic Research, an affiliate there. Her widely read research and public discussions on the impact of public policies on the labor market have been well recognized and have had a large policy impact as well and I know that's one of the topics that she is spending some of her time at the CEA looking at and she has focused extensively on the role of data, subjective well-being and in a variety of other areas as well for how we think about public policy. Her topic today as you'll learn more about it in just a few moments is the role of research for this Council of Economic Advisors and that seems a particularly appropriate topic given the broader context of our conference. I did with want to mention that we have set aside a little bit of time for questions following her remarks and if you have questions we do have staff who can bring you a microphone. Please wait for the microphone and for those of you who are watching online I invite you to tweet your questions. Please use the hashtag Ford School Gramlich and one of our staff will read your questions to our speaker and with no further ado it is my great pleasure to invite Betsey Stevenson to the podium. [Applause]

>> Thank you so much. It is a tremendous pleasure to be here and I was just saying to Marena she was asking how I was liking the world of the Council of Economic Advisors and it is one of the most thrilling, exciting places I've ever been and I think part that is I think I have to move so rapidly from very, very disparate issues and that is both, it's challenging and it's excitement so what I wanted to do was to take this talk as a chance to step back a little bit and think about the role of the Council of Economic Advisors and it gave me an excuse to read what a lot of former CEA chairs have written about the Council of Economic Advisors and I thought after I'd read it that it was both very helpful and I was somewhat glad I hadn't read it before I came in because I think it's allowed me to go in with fresh thinking and but I think there are some ways in which they all paint the a same picture and without having read their pieces I would have said the exact same things that they said in the sense that they all paint the same picture of the Council as providing the President and senior officials with objective advice based on the principles and tools of economics. The other thing I found really interesting is that many of them mentioned the fact that the Council of Economic Advisors looks after the national interest by considering how policy impacts societal welfare in a process dominated by the views of interest usually called Special Interests and mistaken remarked how the Council of Economic Advisors Special Interests was the national interest. What I found interesting about this is that it helped me understand actually why I sometimes faced challenges in my job that I hadn't equate recognized before because it always seemed obvious to me but the national interest is the interest and I am learning over time that there are many interests but I fully stand behind the job of the Council of Economic Advisors as trying to figure out what is really the best thing to do in terms of maximizing societal welfare. So that description of the Council remains true nearly 20 years later and of course many of you I know have the actually spent time at the Council so the structure has really stayed the same. The Council is comprised of a chair and the two members, staff of senior economists, professional statisticians, a director of forecasting and many junior economists. The Council members and senior economists typically come for a short spell in government from academia or a think tank, a model that former chair Joe Stiglitz described as "Citizen bureaucrats." While sometimes clunky this citizen bureaucrat model does facilitate the Council's role as an effective conduit between the research community and the policy making process. And the Council continues to elevate decisions or to evaluate decision in terms of the overall impact on welfare. So what has changed? I think the biggest thing that's changed is technology and that's not just a change in the Council rather that's the broader change that we've seen in our profession and the Council has of course reflected this change. The economics profession is in the midst of a revolution. We're become a more empirical field. Rather than just theorize about the effects of policy we turn to hard data to tell us about how the world really works. This revolution is the result of technological change that's given us an explosion of data and importantly the computing power with which to analyze it. In short, there are now opportunities to analyze public policy in a way that our predecessors could have only dreamed about. These opportunities make academic research more connected to reality and policy. In turn, our deeper connection to reality makes us as economists more accessible, more relevant and actually more central to the policy debate. Implicit in this shift is the hope that empirical evidence will improve our understanding of government programs and the economy. Now while this change is impacting all of economics I think it's a particular boom for micro-economics. And in reading many of the reviews of what the Council has done micro has sort of been shall we say the Cinderella in the family, and I say Cinderella on purpose as in the sort of ignored stepchild who I think is rising in prominence with our ability to do more analysis. Now, hopefully many of you will disagree with that and say that the former chairs had a bias when they wrote up their reviews of the Council of Economic Advisors but I do think that what definitely has occurred is a notable shift in the amount of applied work that we're doing. The work and the staff of the Council has shifted, along with the economics profession. Our junior economists are no longer only advanced graduate students but include computer savvy undergraduate economics majors following the full-time RA model that many empirically-oriented economists are using to crunch literally millions of data points. And our staff economists have noted ruefully that they're expected to conduct research at the Council that a few decades ago would have easily earned them their dissertation and now they're expected to do it in a few short weeks, a month or two if they're lucky. This type of sophisticated analysis of large scale data sets in brief time is I think one of the advantages for a graduate student who is spending time at the Council. Not only are you exposed to an enormous range of ideas but you actually learn how to get them done and the get them done quickly. The explosion of empirical economics has also increased the number of competing analysis that are relevant to any particular public policy problem and this explosion represents both an opportunity and a caution for policy makers in the public. The caution is becoming obvious there are conflicting studies on many important policy issues and I think we see this across a range of fields not just economics. In a field where I know very little, health, I'm still confused about whether coffee is good for me or not good for me given the number of studies that come out and so the most important advice for policy makers is to understand that a given research result should not be thought of as right or wrong but as a guide that should shift one's thinking. The question is how far? We're informal Bazian and to us, the CEA the question of how far to shift our views depends on both the precision of your priors, that is how much we already knew about the issues and the precision of the new signal which is largely shaped by your interpretation of the credibility of the research design. The challenge for the Council and for others trying to inform policy makers about research is that policy makers tend not to be Bazians. The Council's job is therefore to bring Bazian updating to policy makers. The opportunity is that the proliferation of studies truly does improve the evidence-base available to us making it easier for us to make the case that policy should be guide by robust research. For the Council of Economic Advisors the explosion empirical research and the tools available to scholars has deepened our role I believe as a conduit of research to White House senior advisors and to the President. So let me turn now to giving you some concrete examples of what CEA does and how I think this revolution has changed or caused us to tweak a little bit about what we do. So the CEA as many of you know is required by statute to write an economic report of the President each year and what it says is in the statute is that report should set forth current and foreseeable trends in the levels of employment, production and purchasing power and a program for carrying out the policy, to promote conditions under which there will be afforded useful employment for those able, willing and seeking to work. So in most years that I have looked through the he ERP has attempted to do more than that and has attempted to be broader and to speak more to the broader policy debate that is going on and I think one of the things that's changed is that the rapid timeline under which things are moving and the technology available to us means that we can release reports on a more timely basis and feed into the policy making process in a way that provides more timely analysis on economic policy. So we've recently begun to reconsider how we should best produce the economic report and if you look at the most, the 2014 economic report of the President you'll notice that several of the chapters have been previously released as reports that fed into a more timely aspect of where the policy cycle was. I think the ERP to me serves two really important roles. One is it provides a record for what the administration was thinking and doing at the time but it also provides, it also feeds into realtime policy process and this more continuous model of releasing chapters I believe better serves that latter purpose. So to give you an example of one of those chapters this past January marked the 50th anniversary of the announcement of the War on Poverty, something Sheldon made sure I understood clearly as I went to the Council this past summer. And walking into the Council with a President who was interested in thinking about poverty and inequality we saw one of our jobs as making sure that senior advisors understood that there was a broader historical framework to think about and that there would be questions naturally raised as we got closer to that anniversary about how good of a job has our government programs done? Now the challenge that we faced is that you'd think that 50 years later it would be an easy question to answer whether or not our government policies have reduced or not succeeded in reducing poverty. It turns out that is not actual an easy question to answer. It's not easy for what is either a simple or arcane perhaps reason which is that our official poverty measure excludes almost every poverty-fighting program that we have. So if you look at the official poverty rate and use that as a tool to measure or as a statistics to measure our tools in fighting poverty you're failing to understand that measure in fact excludes our very tools. So CEA, what we really wanted was something that seemed like a simple concept. What would poverty be if there was no government, market-based poverty? And what is poverty once we put the government programs on top? And we turned to our staff. We worked with our senior economists. We said, "How are we going to answer this question?" A very, very smart senior economist who graduated from the Ford school, Jordan Matsudaira said, "I cannot answer this question on my own in the next court of law months. This is a big idea question." And he went out and canvassed and talked to academics who worked on this issue and he discovered a team of academics who were trying to extend the supplemental poverty measure which does more to include our policy programs as a measure back in time. But Jordan said to them, "That's great. I'm really glad you're doing this work. The problem is if we want to say something about how effective our programs are over time supplemental poverty measures quasi-relevant so as you extend it back in time you're shifting the goal post and if you use your measure to compare how poverty, as measured by the SPM is today compared to 50 years ago I'm still going to get something that doesn't tell me did our programs work or not? So he worked with them and came up with the idea of creating an anchored version of the SPM that's not quasi-relevant. These researchers came in, presented to the overall administration. They worked with our team and they released a paper that I hope they felt was better targeted at how they could influence the policy debate and we released a chapter which was able to explicitly try to answer the question of how much has public policies reduce poverty? And we were to be able to release that in time for the 50th anniversary so we could feed directly into the debate that was ongoing. That is I think a great model and let me say that was one of the things we did where we did actually spend months doing it, not weeks doing it. But we still spent way less time than academics would typically spend. Let me give you one other example that I think really illustrates the modern era of economics. The policy team and senior officials inside the administration were grappling with whether the President should raise the minimum wage for federal contractors and there were a lot of questions. What would happen if we raised the minimum wage for federal contractors? How would this impact employment? How would this impact productivity? How would it impact the cost to the taxpayer? And I think why I think this example illustrates the modern era is because of course the first thing we did is what CEA would have always done. We canvassed the literature and we had many people who were reading a full range of literature not just on minimum wage changes, living wage ordinances, changes in federal contracting pay and providing a summary of what do we know. We took a look at, in terms of the employment effects, at all of the many, many studies that had been done to sort of see where does the literature seem to suggest what are the employment effects on raising the minimum wage and then we did our own empirical research. It turns out there are federal dollars spent in every single state and every state has raised its minimum wage. What happened to spending on procurement in each state as they raised their minimum wage? Now is where I talk about something that not that long ago would have helped someone make progress on their dissertation, a very nice piece of work which showed that there was zero effect on procurement costs in a state following its increase in the minimum wage. So that pulling together existing research, doing our own primary research and doing our own detective work on the data, one thing I should mention is that a natural question you might have is how many people are there working for federal contractors who earn the minimum wage? That data doesn't actually exist. We don't collect that data so that's when I talk about data detective work, developing models and looking at downloading federal contracts trying to figure out what kind of work these people do, what city they're in and what's the likelihood that they're being paid the minimum age given other people doing that kind of work in that kind of town? This sort of overall analysis, doing primary research, investigating the current research this is the kind of thing that is really only possible in the modern era where data, millions of data points are at our fingertips, computing tools, enormous amounts of computing power at our fingertips and frankly our students have all been better trained coming out of undergraduate programs, coming out of MPP programs, coming out of first or second year graduate school to use econometric programs to do the kind of analysis to help us shed light on these issues. Now beyond these issues that are directly feeding into policy decisions the Council keeps abreast of the new economic literature for topics that might be important for policy making in the future and where the profession stands on critical issues. Maintaining a close connection to academia is useful here too as we're able to offer advice on new policy suggestions that is grounded in cutting edge work. For instance we're regularly looking at what is the latest working paper coming out of MBBR and is there information in there that we want to make sure policy makers know about not related to any particular policy but just in the back of their head as they're trying to understand how the world is changing? We also invite prominent scholars to discuss their work and in recent weeks have enjoyed sessions with Thomas Pikety, Bob Hall, [inaudible], Amir Sufi to discuss their prominent work in person and I think it's really important to emphasize that the White House is always interested in policy recommendations even if they can't be readily implemented because policy ideas that may seem unrealistic at first can, the policy recommendations that flow out of research can eventually become commonsense approaches to policy. So for instance as many others have noted, tradable permit inspection options were once only academic exercises and the Council of Economic Advisors was at the forefront of saying no, these may actually be real live policy solutions. By continuously beating that drum the world eventually moved towards that. I think the other thing that's happening of course as the world is changing there is greater potential value and certainly greater opportunity for a CEA to engage not only inside the administration but to engage with policy makers outside the administration and to reach a more general audience whether it's journalists or influential advocacy groups, so CEA now engages talking to the broader public much more directly and I think again, technology holds the key. We're making fuller use of modern media platforms. Both Jason and I have Twitter accounts and I have many times after the Employment Report was released whatever way we possibly can. Now the last thing I wanted to say and I think that this is important for people who take a look at the empirical revolution in economics and sometimes mistakenly think that the shift toward a more empirically grounded economics means theory is less important and I think this is just wrong and so I always when I talk about this want to make sure that I have a chance to get that out because what I think has happened instead is that I think that theory is less important but that the role of theory has changed. When facts were rare we used theory to fill in for facts. Today facts are abundant. We have too many facts and we use theory to help us weed out facts and to give us a set of tools to identify empirical research results that are unlikely to be true. So let's return to the example of research on the minimum wage. When I said that we took a look at as many of the results in the literature that we can I think one of the lessons for how theory weeds out facts is really apparent when you look at a scatter plot of the impact of the minimum wage on employment because it has a huge left tail and no right tail and this is evidence of publication bias. It's probably [inaudible] publication bias. If I produce a result that tells you that the greatest way that I can increase employment in my community would be to raise the minimum wage and that would enormously boost employment no journal editor is likely to believe that is true. Now if instead I produce a result that says minimum wage has a negative impact on employment those results are more likely end up getting published. But I think this again reminds us of the role of the Council and other academics within government, other economists and empirical researchers reminding people again about that Bazian updating, thinking about what are the results you have in front of you and what they mean. And every time we as a CEA produce that scatter plot I always get a few people on Twitter who say to me, "But yes, look at all the, there's still all these negative effects on employment. Look at these estimates on the left-hand side and there's none the right hand side." And it's useful to understand that those have been sensibly I think weeded out. But I think in addition to thinking about the fact that observations are thrown away because of the fact that they don't match our theory I think that more important or perhaps most important role now interaction between theory and empirics is the growth of behavioral economics because the growth of behavioral economics depended on data to reveal to economists and psychologists the systematics ways in which people make mistakes. Without evidence economists had known that people make mistakes. I think sometimes people forget this. Economists did know people made mistakes. We just assumed these mistakes were random and we didn't have any reason to believe they were systematic and so on average people made calculated rational decisions. What data is helping us understand is that there are ways in which people made systematic mistakes and this revolution in theory coming out of the revolution in empirical science I think is very important for policy makers and something for the CEA needs to help policy makers understand what does this mean and what is the role of policy in helping people in particular areas where we now know that they make mistakes? And I think this has opened up a whole new type of research for us to be paying attention to and a whole new way for us to be thinking about how we can inform the policy process. So let me conclude by saying this administration and I should say it's really the only one I know because I am kind of young, really has been dedicated to building a smarter more innovative, more accountable government and has been focused on integrating research into policy. In fact, I just left a meeting where the President of the United States attended and one of the Cabinet members talked very excitedly about working with the Chief Statistician of the United States to make sure that we were going to get the data that we needed and so this idea that we have to know what we're trying to do and be able to measure it is at every level of government now and many agencies and offices in the White House, including the Council of Economic Advisors are helping to make Research more prominent in the minds of policy makers and helping to set an evidence-based agenda to improve government's performance. The Council's role and this is a continuation of a long standing model that's been successful across many administrations and while our approach is changing somewhat with the times the fundamental goal looking out for the national interest by bringing rigorous economic thinking to matters of national importance is resolute. The revolution in economics has increased both the opportunity and the challenges for economists in policy and the Council of Economic Advisors is happily embracing both. [Applause]

>> I've got a question that has some, is a little unfair. It's self-interest but it seemed to me, first of all it's a meeting about policy and academia and bridging that gap as Ned Gramlich did better than anyone I know, but you said a couple of things about policy that I'm not sure we prepare our students for namely, the use of simulation and simulation techniques and also behavioral economics. So when you come back in a year would you have an interest or would you recommend that maybe Econ Departments in Policy Schools begin to train people to use those tools?

>> I think that that is a great question and I do think that if we think about the revolution in empirical economics and what it means for policy makers it does have implications for how we train students and the types of skills we need them to have. At its core, a grounding in economic theory is essential. If you're going to be an economist advising in policy makers or playing a role in economic policy you need to understand the role of incentives. That hasn't changed and that's been true since before the Council was set in place in 1946 but there is this sense in which our theories are changing. Our understanding of the ways in which people make mistakes are changing and mistakes, people making systematic mistakes is perhaps--if I think about where is the role of government greatest it's where there's market failure, asymmetric information. When the market's doing well the market's doing well. When people are making mistakes are there things we can do to help them make fewer mistakes? And when there are problems with information are there things we can do to remove those blockages? And we certainly I think that is the direction in which we should be making sure our students are trained to focus on why is there a role for government? What would happen if there was no government and to do that today I think we do need to have a grounding in the types of mistakes people make and then to tools to be able as you said to turn to the data and to consider potential counterfactuals and to think of the most creative way to say if we enact this policy what do I think is likely to happen?

>> Are you at all concerned that the empirical--it sounds terrific with more empirically-based policy analysis and randomized control trials but the most obvious implication for those are on short-run reactions and short-run problems. Does it mean that we have less emphasis maybe or how do we take that method and apply it to the long-run issues like deficits are going 20 years and environmental degradation, global warming, changes in increase distribution over long periods of time? How do we, does it apply to that as well or is it mostly kind of short-term things and if it isn't long-term things then how do we address those analytically, those long-run problems?

>> So I actually spend a lot of time trying think about this because if you, I do think one of the important roles of the Council of Economic Advisors is that because we don't--we're all going back to our academic jobs and I should have said I do think this is actually--many other people have pointed this out but this idea that in some sense we really don't care what people think of us because we have a real job to go back to is actually one of the things that gives us a degree of independence that's kind of important. You know there's not career positioning. It's all fun and when it's not you can take your ball and bat and go home and so the Council thinks a lot about long-run and I'm an applied micro-economist and I actually spend a lot of time thinking about long-run and in fact--but I think that there are things we learn from the empirical data that can help inform us about long-run issues. I will say that there is empirical work on this relationship between debt and growth that I think is very interesting and should actually inform policy makers but we won't dig into that but it's not like those issues don't have empirical research. But for things that I think about because I focus mostly on social policy, I think that we have a couple of big opportunities for long-run growth ahead of us right now. One of them is doing more to promote minority men in the labor market and I think that's grounded in empirical research which shows that when we invest more in boys in preschool they end up having better performance being more likely to work in the labor force. Someone in academics, we had a bunch of academics to talk about work/family issues and with senior officials just this week and one of them told me about a study that showed that you can see the effects of paid maternity leave on the wages of children whose moms stayed home at age 30. So I haven't looked at the paper but it's on my list of things to do this weekend because you can look for empirical results and what we have to do is then say well, what does this mean for the long-run? I think that it is important that we keep asking that question and I think the nature of the Council is designed to think about the long-run but for me I think if you look at women's labor force participation I think it was the growth in women's labor force participation was extremely important for economic growth in the last half of the 20th century and what we do to support it in this century is going to be important for long-run economic growth, the same thing for minority men. Immigration is something where we think had a lot about what's the effect on long-run growth, so I do think that there is a role to aggregate up the micro results and what do they mean for the long-run.

>> Betsey, having been a member of the CEA more than 40 years ago I was fascinated by your account of how things have changed over that period both for the CEA and for our profession. And I heartedly endorse what you said at the beginning about the mission of the CEA and I would include paying attention to the long-run but there's one thing you didn't mention and this is something that was first raised to me by Walter Heller and that since then I've thought a lot about and that is another goal which is to some extent intention with all the ones you mentioned and that is that when all is said and done and the internal fights have been won or lost by the CEA you are also the defenders of the administration's economic policy and certainly times do arise when that defense is not exactly congruent with the best possible advise you might give as an economist. And I wondered if that had yet concerned you and how you would deal with it?

>> Well, see one of the ways I dealt with it is I didn't address it today. I find that things that aren't nice to talk about we just don't talk about. So that's obviously true, I mean nobody labors under the impression that--it wouldn't make sense for the President and senior advisors to take the advice of the economists all the time because we should not be giving advice that makes any, that is conditional on making political sense. We should give the best economic advice and sometimes that advice is ridiculous given the political situation and so it's obviously not going to be taken all the time and also as you said there are, people have differing views. I think that it's important that we continue to give the best advice that we can and to look for and to follow the policy that if you don't have anything nice to say don't say anything at all. I guess there's one last thing I'll say on that. I think for me the hardest thing is what I did say in the talk which is that I think about the world as an uncertain place and politics does not really allow you to talk about uncertainty. We talk about point estimates and that is very unnatural for me. I think about 90 percent confidence intervals and I think basically everyone in the administration who wasn't an economist would tell me that not only does that open you up to trouble but it's boring, boring, boring. So it doesn't serve any purpose but I think that doesn't impact the advice that we give and it just means that in our public engagement we have to choose carefully.

>> And not get questions from people like me. Let me just say that the problem you asserted about confidence intervals is not confined to the government. I confronted exactly the same problem in the corporate world. Nobody wanted to hear about confidence intervals. They wanted point estimates.

>> I'm really glad you said that because actually I had a little bit, I did a brief stint myself in the corporate world and I found the exact same thing and it's actually what drove me back to academia, so I think it's exactly right. It is true that people don't grow up comfortable with statistics. It would be good if more of our preschoolers were comfortable with statistics more of our elementary school students but that's just not the world that we currently live in and I think that's why it isn't just about government. The public, the world they want a point estimate.

>> Betsey, thank you. [Applause]

[ Background conversations ]