Cultivating socially responsible engineers

0:00:00.7 Shobita Parthasarathy: My name is Shobita Parthasarathy, I am a professor and director of the Science Technology and Public Policy Program, known as STPP here at the Ford School of Public Policy. For those of you who don't know, STPP is an interdisciplinary university-wide program dedicated to training students conducting cutting edge research and informing the public and policy makers on issues at the intersection of technology, society, equity, science and public policy. If you'd like to learn more about it, you can do so at our website, stpp.fordschool.umich.edu. I'd like to thank everybody for joining us here today, both in person and online. We're entering the third year of our pandemic, however, this is the first in-person event that we've had in two years, and so it's very exciting to be able to gather all of you and to see some people that I've only seen on Zoom, and I look forward to sharing ideas, both online and face-to-face. This is our last public event of the year, but we're planning exciting events for next year, and you can find out more about those on our website, on our Twitter, STPP_UM.

0:01:15.2 SP: And our newsletter, which you can sign up for, also on our website. And now for today's event, we're hosting an important and exciting conversation about how we might rethink engineering education to center equity and justice, and the role that universities and public policy makers might play. STPP has long had an interest in these issues, as many of you know, we serve students from across the university with most coming from engineering and the natural and physical sciences, and our courses, graduate certificate program and applied research projects, center questions about how we can make science and technology more equitable and just. So we're particularly excited to convene this event today, to push this conversation further, today's panel features leaders in academia and government and is appropriately diverse.

0:02:06.6 SP: Let me introduce them. Dr. Amy Ko is a professor at University of Washington's Information School and director of the Code & Cognition Lab. She studies computer science education, human computer interaction and humanity's individual and collective struggle to understand computing and harness it for equity and justice. Dr. José Zayas-Castro is division director for the National Science Foundation's division of Engineering Education and Centers. Prior to joining NSF, he served as a department chair and associate dean for research at University of South Florida's College of Engineering and in various leadership positions at the USF Center for Entrepreneurship.

0:02:50.6 SP: Dr. Johanna Okerlund is a post-doctoral fellowship, is a post-doctoral fellow, sorry, here at STPP and the Ford School. She's a Human Computer Interaction researcher. Dr. Okerlund is developing ways to bring ethics and justice into the computer science curriculum and has worked with STPP's technology assessment project on the social and equity dimensions of large language models, a type of artificial intelligence. Dr. Tim McKay is Arthur F. Thurnau Professor of Physics and Astronomy, Associate Dean for Undergraduate Education in the College of LS and A, and Professor of Education in the School of Education here at the University of Michigan. Dr. McKay's team has applied observational and experimental data science methods to astrophysics, cosmology and education, and since 2008 have been using classroom and institutional data to make higher education more equitable, inclusive and effective. Last but not least, our moderator is Dr. Alec Gallimore, Robert J. Vlasic Dean of Engineering here at University of Michigan. Dr. Gallimore is a rocket scientist and a member of the Applied Physics faculty and serves as Richard F and Eleanor A. Towner Professor of Engineering and Arthur F. Thurnau Professor in the Department of Aerospace Engineering, and he's also a founder and co-director of the Plasma-dynamics and Electric Propulsion Laboratory.

0:04:19.2 SP: Finally, and most relevant to this panel, he has a long-standing commitment to equity-centered engineering, you can find even more detailed biographies of our panelists on the STPP event page. Before we begin, I wanna thank the Public Interest Technology University Network, whose generous grant has made this event possible. We also have a number of co-sponsors for this event, including the Michigan Institute for Data Science, the Nuclear Engineering and Radiological Sciences Department, the College of Engineering, the Engineering Education Research Program, the Center for Ethics, Society and Computing, Women in Science and Engineering, ADVANCE Sloan Equity and Inclusion in STEM Introductory Courses and BLUELab. I also wanna thank our STPP and Ford School staff who worked tirelessly to bring this panel together, Molly Kleinman, Mariam Negaran, Kristin Burgard, Erin Flores and Katee Cole. The panel discussion will run for about an hour, and then we'll have time for audience questions. For those of you with us in person, you got note cards and pencils when you came in, you can write them down, and Monica and Natalia, do you wanna raise your hands, Monica and Natalia, will come around to collect them. If you need more note cards, you can just flag one of them down. For those of you watching online, you can ask your questions in the chat and Molly will pass them on to me. Without further ado, I'll turn it over to you, Dean Gallimore.

0:05:47.4 Dean Gallimore: Thank you. Can everyone hear me? Can they hear me online? Okay, great. It's really an honor and a pleasure to be here, and I also like to acknowledge my colleague, Dean Michael Barr for inviting me. I think the reason I'm here is because I'm probably the closest engineering dean that the school could find, so I figure that's fair that I come and join...

0:06:10.0 SP: I thought you might be closest intellectually also.

0:06:12.6 DG: Oh, there's that too as well, and really, it's a pleasure serving as your moderator on this great panel. Let me also show this... This is a material that we send out to prospective faculty members and students, and what it says, it says, "Serving the common good." And when I think about serving the common good, I think about serving people. Not just some people, but all people. So this notion of equity-centered engineering is about how do we do a better job serving all people? And when we talk about equity-centered engineering, the College of Engineering, we say, "How do we equip our learners in a way so they understand the role they play in either widening the gaps, hopefully inadvertently, but also uplifting people in closing those gaps, by making the pie bigger and providing more resources, especially to people who are traditionally marginalized? And lest you think that this is something that doesn't apply to engineering, let me give you some examples in which when we didn't think about this notion of justice and equity at the forefront of engineering technology, what can happen.

0:07:18.0 DG: No one would argue, perhaps, of the importance of the US highway system. It's allowed the relatively inexpensive flow of people and goods. But what if I told you that that was a tool that was weaponized to break apart middle class, African-American and Latinx families in the United States? What if I told you that we've had facial recognition systems, for example, that not only do not discern people of color, but they can actually be used as a way of giving false positives and has been used by the police, for example, to literally here in Detroit, hold an innocent man against his will because a facial recognition system said he fit the usual description? What if I talked about closer to the pandemic, blood ox systems that are developed but do not work as well with people of certain skin hues, if you will? And I can talk about the fact that, for example, artificial hearts do not fit many women's bodies because they've been traditionally designed with the male body in mind, similar with automobile system. Women are less likely to survive crashes because up until recently, engineers who design cars did not use bodies that not only represented women bodies but represented many different body types.

0:08:41.1 DG: What do those things have in common? What they have in common was a failure of imagination of our engineering teams to think about how do we serve all people. Not just people who look like me, or look like him or him or him, but what about her? What about them? And so what we're trying to do in the College of Engineering at Michigan is we're trying to introduce our students, first year undergraduate students, Master's students, even first year PhD students for this way of thinking. Is it about ethics? You bet. Is it about mission? You bet. Is it about business and improving the business products, if you will? You also bet. They're not at odds with each other. And so we are not experts in this area here in engineering, so we gladly partner with people who are more knowledgeable, so for example, Tim McKay's Unit, Literature, Science and Arts. We were just talking about how they borrow some of our faculty members, so we're hoping [chuckle] to get a repay for this work, if you will. And so without further ado, I'd like to go ahead and start, if you will, the moderated panel session, and I'm gonna ask a question that I'd like each of our panelists to answer as a starter, and then after that, we can just pick and choose whoever wants to jump in. Does that sound reasonable? 

0:10:00.9 Speaker 3: Sounds good.

0:10:03.8 DG: Alright, so maybe just moving down, is that okay to start with you? 

0:10:06.7 Johanna: Sure.

0:10:08.3 DG: Okay, so it's clear that all of you are dedicated to the notion of equity and justice considerations in STEM education, and we're talking particularly engineering here, if you will. Can you each take about a minute or two to indicate how you think about equity, justice in STEM education, perhaps even focusing a little bit about Engineering, Computer Science, if you will, and maybe talk a little bit about how your work is involved in this area? 

0:10:35.1 Johanna: Sure. So I kind of come at this from two different perspectives. My background is in Human Computer interaction and Computer science. So I come at this as a technologist, and I've had sort of my own journey as a technologist trying to be as ethical and just as I can, if you will. I am sort of on this journey, and this journey has led me to STPP. And as a Post-doc here, one of the things that I've been doing is learning from the public policy people, how do they approach these things? They approach technology from a very different perspective. Sometimes it's a very top-down perspective, sometimes they're talking about technology development, but not the specifics of the technology, is just, "Okay, what mechanisms need to be in place for this technology to be more equitable and just?"

0:11:18.0 Johanna: So one perspective I come at this is just sort of reflecting on my own sort of personal learning journey as I try to become a more equitable and justice-minded technologist, if you will. Another thing that I've been doing as a Post-doc here is thinking about Computer science curriculum specifically, and how can some of the lessons that I'm learning over here in public policy be embedded into Computer science curriculum. And I think a number of us on the panel are thinking about that kind of question, curricular updates in different sorts of ways. I've taken it in a very sort of low-level approach, how does the practice of computing need to change at a content level? If we look at what is the content that computer scientists are learning, how can that be broadened and expanded in different ways in order to bring in some of these lessons about equity and justice? 

0:12:04.0 DG: Thank you.

0:12:05.4 José: Thank you. First of all, good afternoon, and it's a real pleasure to be here, and I wanna thank Shobita and all the team for the invitation. I'm gonna come in on two hats. One is called the National Science Foundation hat, which is the one I'm representing here.

0:12:26.6 José: Since the late 1960s and the early 1970s, the National Science Foundation has been funding efforts to increase the diversity in engineering and the sciences, broadening participation, is the word. And there have been a number of programs initiated over the years. So as it was mentioned, I have the responsibility of working with a group of people in the Division of Engineering Education and Centers, where we have a host of programs to promote diversity, equity and inclusion, broadening participation story, how that can be done better? Whose story is that? The researcher. There for those of you that may not be aware, the National Science Foundation is an independent agency of the federal government that sponsors, it is a grantee institution, sponsors provide resources to researchers of the community like University of Michigan and universities across the country. And they come up with the ideas. At the same time, people come to the National Science Foundation, like me, what they call rotators, we come from Academia and we put that together and seek feedback from the community. So there are a number of programs and initiatives that go in... From various units in Engineering, Computer Science, Education and Human Resources and social and behavioral economics.

0:13:57.6 José: And they come up together. Now, one thing that I'm sure Dean Gallimore is well aware is that the Dean, the Deans of Engineering, sign the pledge to diversity and inclusion. And that has been growing. And our AD, Susan Margulies and others, we interact with the deans representing the university to try to be in tune and where were things have been going on. There have been historically a number of efforts in the engineering community, working, improving the participation of under-represented folks, and you see it in professional societies. National Society of Black Engineers, Society of Women Engineers, Society of Hispanic Professional Engineers, and they come together and they are supported by a number of the grantees that the National Science Foundation has. My other hat is my hat as professor. And over the years, I have had the opportunity of working with folks that are interested in increasing their recruitment, retention, graduation and locational work of under-represented communities, at the undergraduate and at the graduate level. And that has been funded for various organizations. And this is all team work, okay? These are all team-based. This is not an individual that can carry the torch by herself or himself. So we have had that opportunity.

0:15:34.5 José: I have been fortunate of working in three distinct universities over the years, all of them public universities. And that has given a different perspective in three very different locations. And going back to the National Science Foundation effort, that's also team-based and it is across the foundation. Our division is what we call, is a horizontal trans-virtual division because we serve the whole engineering community, not by discipline, any discipline come together, and the whole aspect of broadening participation, we are looking at it from that point of view.

0:16:14.8 Amy: Hey, thanks José. I think like Johanna and José, I come at this from many different perspectives in my background. So one is, as a professor doing research and teaching in this space, and that's I think probably one of the more familiar ones for many of those in the audience coming from academic contexts. I also come at it from the perspective of an entrepreneur though, somebody that's been a co-founder and chief technology officer of a startup that was creating technologies that had these problems. So having to think about that from a direct engineering perspective. And I also come at it from the perspective of somebody who's often oppressed by these technologies as a transgender person, constantly having to make space for myself in technologies that don't really make space for me, that's something that I've experienced oppression with every single day.

0:17:01.2 Amy: So when I think about this problem intellectually, it really brings together, multiple different lenses. One is sort of the individual experience of these. Another one is sort of the technical perspectives, like Johanna was mentioning, around how do some of those individual experiences boil down to the technical ideas underlying computing or engineering? And then of course, the systems perspectives that I think a lot of people in Public Policy will often think about. Who are the decision makers? What are the incentives? What are the structures that we have historically that lead to some of those individual decisions, and therefore some of those individual experiences that people have with technology? 

0:17:40.0 Tim: Great. So I think like José, I'm gonna talk about two branches of what I wanna emphasize. I'm gonna speak mostly as an educator and in particular, as a physicist who is an educator in a liberal arts college, and I really hold both of those identities very, very close to my heart. As a physicist, I wanna say that we bear some guilt in generating engineers who think that the world is full of problems that are simple and that have correct answers. When you take an introductory Physics course, we will show you abstracted systems that have right and wrong answers in a way that begins to convince people that the world might actually have right and wrong answers like that, that are that simple. And that carries forward in education in a way that I think is really problematic. It's not that there's nothing that is that simple, there are a few things that are that simple, but it's very, very few. And I think we need to reconsider the way we frame what the nature and values of science and engineering are at a very early stage in that kind of way.

0:18:41.0 Tim: I also wanna talk a little bit about the process of educating engineers, the engineering education system itself, because I think if we want engineers to become socially responsible people, it starts in the process of educating them and in the way we do that. A lot of my work today goes into trying to make our education system here at the University of Michigan and in other places, more equitable and inclusive in the places where it is least, and that often is in large introductory courses, for example, on our campuses. This is the perfect place for engineers to learn how systems of privilege and oppression work in a way that is very familiar, that they're kind of expert in to see what that's like to explore what can be done to counter those things.

0:19:25.0 Tim: And I hope to recognize their responsibility to do something about it. A really key element of creating socially responsible engineers is convincing them that they have great agency in what they do, they're not driven toward what they do, and they have full responsibility for the work they do and that's stuff that we really need to get across to them. To create... To succeed in cultivating ethical engineers, I think we need to make additional forms of knowledge expected of engineers, it's not enough that we train them in the ways that we have, we have to make those additional forms of knowledge expected for them, and that's a place where we look for collaboration with our engineering colleagues. I know as a physicist that there are many things about how science plays out in the world that I was not trained in, and so I turned to my colleagues and programs like this to learn more about that and make sure that our students do the same.

0:20:18.7 DG: Great, thank you. That was inspiring. Actually, very nice. All of it. Now we'll switch to questions that one or two of you can answer and feel free to jump in. Before I switch to the next question, I wanted to also acknowledge the important role the funding agencies like NSF has played, NSF through career grants which are given to junior faculty members, all the way up to these major multi-million dollar center grants, all have this notion of broadening participation in, and that has done, I would argue, as much, if not more than anything else in terms of transforming engineering education and making it clear, it's imperative that we incorporate not only diversity and inclusion, equity inclusion, but justice into it. So with that, I gave some examples, but are there other cases or examples that you're aware of, and this could be in the government or academia, or industry, that has been successful or might be even applied to help engineering students learn about equity and justice. Who would like to go first? I'm smiling but I had to ask...

0:21:28.0 Amy: I'll jump in on that one.

0:21:28.4 DG: Please. Thank you.

0:21:30.6 Amy: Yeah, so in some ways, I think I reject the premise of the question, the framing of the question that there is just a particularly effective example that's universal to all engineering disciplines. I think it's not surprising. I think all of our experiences are that we resonate with different examples, I'll give some theoretical grounding here, there's an activist and educator from the '60s, Paulo Freire, who talked a lot about limiting situations and critical consciousness about those limiting situations, and in some ways what we want to do is help engineers understand their own limiting situations, but also others and help them build critical consciousness about them.

0:22:05.0 Amy: So for example, when I work with refugee communities and the South Puget Sound, often what we're talking about is examples like the algorithms and data that enabled them to get access to the United States, what were the systems that underlied their ability to get here and pursue opportunity here. And in contrast, when I'm teaching, let's say, wealthy kids from Bellevue who's parents work at Microsoft, I'm often talking about, Well, why did their schools have computer science and other schools didn't have Computer Science, and all of the underlying algorithms and data that are part of property tax systems and other types of estimates like that that drive which kinds of schools get revenue and which ones don't, so those examples, the ones that are really close to home for students, that really help them understand their own context in their own communities. Those are the ones I think are often most effective.

0:22:56.9 José: If I may share... Let me give you a couple of examples. I'm gonna... There are multiple programs, as I mentioned in the Foundation, one is called revolutionizing engineering departments better known as for the community as RED, which is a joint effort between engineering and the Directorate of Education and Human Resources. I'm gonna give an example that we picked up that may be at the heart of Johanna because this is an effort in the... And again, this is an example, I'm not saying that this is better than others, it's just we have limited time, it's an effort the College of Computing and Informatics at the University of North Carolina share a lot. And they're working on that and they're completing data, by the way, if you are interested in knowing about these projects, the abstract of all the grants that NSF provides, are public, public domain, so all those are available. Now I'm gonna come back, if I may, to something I mentioned, which is the impact of the engineering research centers as well as others. In a synopsis, the engineering research centers have moved what we call generations right now we're in generation four.

0:24:18.7 José: If you look at the diagram of the generation four what we're seeing is research, workforce development, innovation that brings industry partners, diversity, equity and inclusion, they need all to come together, and that took that to another level, and this was provided on the efforts that the foundation has been listening to the community, to their needs, obviously to the federal administration, and if you go to NSF, you're gonna see a phrase, that the director concocted or put together when he was appointed, we need to work very hard in reaching the missing millions.

0:25:03.4 DG: That's great, thank you. I'll pivot to the next question, but first, a little bit of acknowledgement of our mechanical engineering department, the changes that you talked about in the gen four was done by a combination of Professor Dawn Tilbury, who used to run the engineering directorate and her former boss, Professor Kon-Well Wang, who was in charge of revamping the ERC, so Michigan engineers made a big change at NSF, which I'm really proud of. How do you think now about engineering education... In terms of engineering education, should we change to incorporate equity and justice? Perhaps even in the course level. Who would like to start us off? 

0:25:44.9 José: I will submit that one of the efforts that the foundation sponsors is that, we need the community to participate more and more in the programs that are available because there are programs for... For example, you may have a faculty member in engineering that has not worked with the social sciences. But there is a program to initiate those individuals in joining engineering education with the social sciences, and apply those efforts and move that together. Across the computer, information systems and engineering aspect, there are efforts also, which is called CISE, in the Computer Science Community, and broadening participations, engineering education and aspects that you are relatable... We were talking before in workforce environment, which you can relate to something that is called Research Experiences For Undergraduates and Research Experiences For Teachers. I remember experience... Knowing a person that was using Research Experiences For Teachers to take it to teachers in public school in under-served communities. As I said before, and I said put the opportunities out there, we need more and more people to continue participating and creating alliances and collaborations that have been shared before.

0:27:13.4 SP: Great. Johanna, would you feel comfortable answering the question? Thank you.

0:27:17.1 Johanna: Sure. Yeah. So I think one of the most fundamental changes that needs to happen has already been kind of mentioned, that students need to stop perceiving technical content as purely technical, and instead learn to perceive it as inescapably entangled with social and political factors. Another thing that I think needs to happen sort of from the beginning is there needs to be room for students to form different identities relative to technology. So oftentimes, the sort of identity that attracts people to computing or engineering is this problem-solving, hacking, creative, it is this curiosity-driven. It's that kind of technical identity and that's great. There needs to be room for that. But there also needs to be room for other sorts of identities. There needs to be room for students to form a political identity relative to technology or a personal identity, or be able to relate what they're learning in the classroom to things that are happening in their community. So part of that is related to the broadening participation in computing, but I also think it's related to this issue of, how do we bring ethics and justice into the classroom? It is that there needs to be room for these other identities to form and other ways of learning computing. So intro computing is often taught in a very abstract way, as opposed to allowing people to play and explore and learn the different dimensions of it in ways that make sense to them.

0:28:37.4 DG: That's great. We're starting a new robotics program, and we rejected the notion of starting with calculus. And in fact, we're starting with Linear Algebra where the first year undergraduates can start programming robots, this notion of play-based education from the very beginning is something that we're starting with. So this all sounds great. I'm sure there's more we could be saying. Why aren't we doing this? What are the challenges? How do we... You know, our faculty is all on board for doing these changes, right? 

[laughter]

0:29:07.4 Tim: I can tell you one challenge that stands in the way, which is that in the natural sciences and in engineering, we have a model of student understanding, which you could compare to a brick wall. There's a kind of sense in faculty member's minds that when you take first semester physics, you put that first brick in place and it's fully in place, right? And then you put the second brick in place, and then the third brick in place. And so we have this sense that the only way you can learn science is through this complete list of absolutely everything. And if you take anything away from it, you won't know anything. This is clearly not true, and educators understand that nobody gets the same first brick even, much less the rest of it. So we need to be thinking about the flexibility that actually exists in our curriculum to encourage, allow, even require students to engage with aspects of engineering, which are unfamiliar like its history and its social impact. And those need to be central parts of an engineer's education. Disciplines like engineering define who they are by what they do. And when our students look at our engineering faculty and don't see them think about or engage with these kinds of things, they recognize that and they know that the real Brass Ring is they're ignoring these features.

0:30:29.8 Tim: So there's a way in which we, old people like me, need to change and display that change to students as part of what we're doing in order to get them to this kind of place. So I think we need to create flexibility.

0:30:43.3 DG: Amy? 

0:30:43.7 Amy: Yeah. I'd love to follow up on that. I often come from these problems from an empathetic perspective, and I try to figure out who are the people that don't feel capable of acting? And I think in a lot of ways, I see it coming down to faculty. And when you approach faculty empathetically about some of these problems, what I hear from faculty in computer science and other engineering disciplines is, I might be able to make space for these topics in my classes, but I don't know how. I don't have professional development contexts in higher education to learn how, I don't have time to learn how, I don't know who to learn from, I don't know who to trust to learn from, there's not even, in most cases, a scholarly literature that teaches them what works and what doesn't. We're only at the beginning of building out some of those discoveries around what works in education research in disciplinary ways. And so when we think about bringing some of that flexibility to the table and saying, "How about we teach this to faculty?" Often times the response is, entirely appropriately, "I can't do that."

0:31:47.4 Amy: And it goes back to sort of Johanna's point about making space for those other identities and the kind of epistemological pluralism, as Seymour Papert would have called it. In those learning contexts, how do we make space for other faculty identities and other faculty epistemologies for conversations about technology and engineering that aren't strictly technical.

0:32:09.5 DG: So that's one challenge. Those are faculty members potentially who understand the importance of this and they just... They need help and support. How do we reach those faculty members who say, There's no way we need to be doing this, we can't possibly be adding any more material into the curriculum and why do we need to do this stuff? Engineering is hard. Come on. How do we reach those people? [laughter]

0:32:35.8 Amy: I'm just gonna respond to that one very briefly, in some of the activist spaces that I live in, you don't waste your time with those people.

0:32:44.3 DG: Well that's...

0:32:45.8 Amy: You start with those early adopters who are excited about making progress and just don't know how... I'm not gonna go and try to convince a bunch of transphobes to not be transphobes, I'm gonna work with the people who are allies, who need the right resources to help us make change happen.

0:33:00.9 José: If I may wear now my professorial hat a minute here.

0:33:04.4 DG: Please.

0:33:04.9 José: I think we need to start in the doctoral programs because the way we educate or train or both the doctoral students is critically important. So a few colleagues and I have done it, now, it's a very small scale, it's anecdotal, and then break a little bit a few barriers, for example, who invented the notion that everything has to be a new course. That is a multiple of three? 

0:33:42.7 DG: That's right.

0:33:46.0 José: Why? Where does it come... Why cannot be modularized. The questions still arises, which may be a research question for many. Now, perhaps for a reason we didn't want a pandemic, the whole aspect of the use of information technology to learn accelerated. I'm not saying it's perfect, I'm not saying it's there, but accelerated dramatically, so a question may remain, as I was conversing with a couple of colleagues a few minutes ago, here in Michigan is we need to ask those questions the researchers in a collaborative approach, as was mentioned in one of the previous questions. So what is gonna be that environment of the future? And how do we sign for the future without being tied to that rope of the past? 

0:34:43.8 DG: Johanna, I have a question for you. As an administrator, I wanna do all these things, right? So what advice do you have for the role that administrators can play, Universities can play in terms of seeing this kind of change happen? 

0:35:02.1 Johanna: Yeah, I think there needs to be top-down support, so the people at the top and need to make it clear that this kind of stuff is not just supported but is a priority, so that involves reflecting on mission and branding and making sure to just promote whatever is happening as much as possible, so the top-down messaging, but the change really comes from the bottom, that there needs to be grassroots efforts, but the top can support those efforts, can make sure to make space... Make sure that there's space for people to learn to collaborate and take the time, and I don't know, maybe there's a lunch group that's meeting, that's talking about the stuff that the administration can make sure to support those grassroots efforts.

0:35:42.2 DG: Anyone else? I need help.

0:35:47.2 Amy: An example of a top-down thing that I think is a particularly tough nut, but an important one is honestly tenure and promotion criteria, until tenured faculty feel like this is valued, it's not gonna be something that they do prior to tenure. And after tenure, I think that we have to re-create that culture of viewing tenure as a privilege that the public has granted, especially at public universities like this one, that really says that you're charged with that responsibility of figuring out what is the right thing to do, without security of employment, what's the right thing to do to speak out about because if you don't, what is it for exactly? Why do we... You don't use it? Why do we get it? 

0:36:31.2 DG: Yes.

0:36:32.1 Tim: I think there are a lot of things that we need to do as administrators. One of the things we do is deploy the resources of the university, and an example of that is that all of our faculty have expectations about the amount of teaching that they will do, but we account for that in all different kinds of ways, for example, we don't do very much to encourage or support collaborative teaching, that's not been the model. Here we have a case where collaborative teaching may be an essential element of what we wanna do, even more complicated, it may happen across colleges. So I think it is on us to think about if we want to foster new kinds of courses, new kinds of learning for students, we need to make sure that those are practical for the faculty to do. Most of the faculty I know are very happy to meet the teaching expectations that they have, but totally unable to go beyond that, so if we can't figure out a way to actually make it count, and I'm pretty sure we can. Then it won't happen. I know so many well-intentioned people who cannot do what they wanna do because of the system they're in.

0:37:43.0 José: If we look at then at the programs I've mentioned before at NSF, there are opportunities to do that thing from a research point of view, and from a broadening participation point of view, there is one grantee here in industrial and operations engineering, and she's doing wonderful work, she's a grantee NSF and there are others across the nation. So those opportunities do exist to look at them as important research questions that intersect the science, the technology, the social sciences, ADVANCE and then do implementation with the proper support from the university.

0:38:26.5 DG: So when I meet with my advisory board, they always say, "How do you get anything done?" [chuckle] Because doing things top-down is a challenge in higher ed, and we use this term, buy-in, "We need to get buy-in". Which maybe we do, maybe we don't, to some degree. How do you think about that? How do you balance the notion of top-down? We talked a little bit about the benefit of it. But how much importance should we play in this notion of buy-in and how do we think about getting buy-in? 

0:39:06.4 José: There are parallelism to National Science Foundation. We have a director of the foundation, we have assistant directors, we have division directors and then program directors. In that recruitment and hiring process, they try to bring people that have that buy-in already and they help them make those changes. So you have a parallel situation I would say, that in university, so it's not on the dean or those other, it goes and it flows.

0:39:47.7 Amy: Buy-in is interesting. I sit in a unique administrative position as a full professor who oversees an undergraduate program, I often find myself trying to build coalitions out of faculty and students simultaneously, to get the laggards to catch up. So if you get those core champions in your school and you get a whole community of students all demanding the same thing, anybody that's resistant, it's really hard to ignore both your colleagues and all of the students in your school at the same time, and that's often led to a lot of change. And I don't know if that's any different really, than the kind of coalition building that we do in a democracy to make change, it's very similar, it's just internal to an institution.

0:40:31.1 DG: Next question is... Thank you. Is measurement. How do we think about measurement regarding the infusion of justice in the curriculum? How do we know when we've made progress? 

0:40:52.5 Tim: That's a great question. I'm thinking about some work we've been doing recently to try and understand in almost engineering sense, how our students are engaging with our curriculum. In the college of LS and A, we have a traditional liberal arts curriculum that emphasizes both breadth and depth. We have very diverse students in our college, who pursue those things in different ways. For example, we might have a physics student who really loves physics and is not as excited about the breadth part of our college and finds the narrowest possible ways to meet the breadth requirements, right? We've been actually analyzing the way students engage with the curriculum, to try and understand better, which students are doing what, and to think about what that means for us in terms of how we might deploy the curriculum and the incentives associated around it, to ensure that more students are really meeting the intent of the curriculum that we have. So there are ways of examining what people are doing within the system that we've created to get a sense of whether...

0:42:00.4 Tim: What we find, and what we will always find is that some students do these things very well and richly, and others do it in much narrower ways, sometimes because of constraints we've placed on them, but sometimes because we need to speak to them in different ways, we need to convince them of different kinds of things. So I think there are ways of looking at what's happening in a measurement kind of way, they don't get quite directly to the spirit of what we're trying to get at here, a real sense that an engineer would understand that their work is done always in social and historical context, how do we quite get that? But we can see what they're engaging in, at least.

0:42:35.0 Johanna: Yeah.

0:42:35.9 DG: Go ahead.

0:42:37.4 Johanna: I have thought a bit about learning goals for computer science students specifically, and it's really hard to pinpoint kind of what do we want them to get out of this? And I think of, if I was in industry and I wanted to hire a engineer or a computer scientist that had the ethics stamp of approval, what would I expect from them? To some extent, I would expect some knowledge, I would expect them to know... To have a certain knowledge base. But even more than that, I would expect them to have certain sensibilities. I would expect them to be kind of humble about what technology could do, I would expect them to approach it as continually learning as well. It's not... I don't think it's a check box that it's like, "Oh, I learned these things and I can go and be an ethical engineer forever." I think it's that they have to continually be learning and listening to communities and figuring out what are the new issues that are continually being uncovered? And approach it in a much more dynamic way, and those things are really hard to pinpoint and hard to measure.

0:43:28.2 José: If we go to this statement of reaching the missing millions, one... Which is not as granular is... We can count them and we can dissect. Where are they coming from? And how many? And how are we progressing over time? And that's... And at least, that shows some data. Okay. It's moving. It's moving in the right direction, and then what the other panels have mentioned, go more granular. Many years ago, a mathematician, Sheila Tobias, wrote a little book. And that book was, They're Not Dumb, They're Just Different. And that needs to be well captured. But if the environment is created, they are gonna come because others will share that experience and now that many things go viral, I think they know how to share good experiences.

0:44:27.9 DG: That's a great segue to my next question, and just reminds me of this phrase that, talent may be distributed uniformly, but opportunity is not. So how do you think the changes we're talking about, not if, but once we're able to implement them in engineering, will affect the manner in which we're able to recruit students, faculty members and staff members for that matter? 

0:44:58.6 Amy: The undergraduate program that I oversee is unique. It's not a computer science degree, it's an informatics degree in an information school, and in so many ways, what we do in our undergraduate program already reflects a lot of these ideas. It's one that brings together in conversation information, technology, people, equity, justice. They often go take the same jobs as computer science grads, but here's the key difference. When you look at our diversity numbers relative to computer science, it's 50% women, all of the groups typically underrepresented in computer science, it's not the case in informatics. And in some ways, I think that programs like that, that really have fully embraced a lot of these ideas, they're a predictor of what we might see if we were to be successful in engineering and computing more broadly. They're a way of glimpsing into what it looks like to make room for all of those other identities in learning and teaching. I'm really optimistic, I'm really confident [chuckle] when we create this, that's what it'll looks like. We'll have that diverse representation if we can address all of the other inequities outside the institution to make sure that they can successfully join our institutions and learn.

0:46:14.0 Tim: I wonder sometimes whether expressing an attitude that's really different from the usual attitude of engineers, which is a great attitude, but the idea that everything is a problem that you solve. I think that a lot of communities see that statement, we solve problems and they watch problems not get solved but created, and it really undermines the credibility of the field, and so I think a more critical engineering could be much more effective at bringing everyone into it. Because it's only the people who actually think everything can be solved as a problem, who are drawn to it now, and everyone else worries about that.

0:46:56.8 Amy: There's a phrase we use in our undergraduate program that every solution is a problem. [laughter] Sometimes to be solved, but also...

0:47:05.8 Tim: We have a similar thing that says, "Data never answers questions, it only raises questions."

0:47:09.7 Amy: Exactly.

0:47:11.3 DG: That's true. I resonate with what you're saying, Amy. Our Director of AI, Professor Rada Mihalcea, she told me that half of the admitted PhD students in AI are women. And our robotics program is equally diverse. I do think we're going to make some serious progress as we move along here. Are the challenges though we're talking about really peculiar to engineering, or do other fields have similar problems and might we learn from those fields or might they learn from us? 

0:47:44.9 Tim: I can speak to some other fields that definitely have these problems, many of the natural sciences, particularly physics as an example, has very similar challenges and problems and mental attitudes that create it. Another that I see a lot and that you have lots of people here and for that are involved in is economics. Where there is a sense that everything, we just have to do the analysis and it'll tell you the optimal solution, and that whole sense that optimal solutions exist in meaningful and useful ways is risky in a lot of social circumstances, so I definitely think there are related fields. Those fields share so many things, right down to the way they think about and run their introductory courses. That's the list of fields where many people believe not everyone can learn this. And they project that pretty much, they kind of tell our students sometimes explicitly, but often, implicitly, "A lot of you aren't gonna be successful, and I'm kind of okay with that. That's the way the class will be, a lot of people won't be successful." That's not true in a lot of disciplines.

0:48:44.5 DG: Let me react to that, and I encourage others to chime in on that one. One of the takeaways I have so far is this identify a problem, solve problem, focus could be a challenge, but how is that different from medicine, which doesn't seem to have the same challenges in terms of bringing in at least women. I don't know how diverse it is, or is it the same or am I missing it completely.

0:49:13.7 Tim: That's a great example.

0:49:17.2 Amy: Yeah, it's the same.

[laughter]

0:49:20.5 Amy: Plus more, 'cause you have not only the pre-med gauntlet that students go through and all of the natural sciences they encounter there, plus the med school ones too, and the cost of med school, and so there's just layers upon layers of inequity there that result in a lack of diversity in that profession. And I think about computer science a lot, and it has its own unique problems, but had a great conversation with one of the postdocs here in nuclear engineering, and we were just comparing notes and we're like, "Yep, we're about 95% the same, computer science and nuclear engineering." There are some disciplinary differences, like the fact that computer science is on the front page of the New York Times every morning in a negative light. That creates pressure that doesn't exist in nuclear engineering until there's a failure.

0:50:08.8 Tim: And war.

0:50:09.6 Amy: Exactly. Those kinds of conversations around what makes each discipline unique, I think are valuable because they're often, they reveal resources and opportunities for making change, but it's also important to talk about the ways in which they're all really similar in some ways. They're all coming from the same big words that we often don't talk about, like capitalism, and all of the sources of power in society that ended up shaping all of them in really similar ways.

0:50:45.7 DG: The notion of... We talked about the role of funding agencies affecting change, positive change, and so on, but engineering is accredited, like some many areas as well, so what do you think about the role of accreditation, ABET, and if you had the leaders of ABET, what would you tell those individuals that they should be doing? 

0:51:09.1 José: Well, I can tell you that if you wanna tell something to ABET, you can do it until June 15th, because right now, ABET is open to comments about proposed changes of the ABET criteria, rewrite it to diversity, equity and inclusion. So, if there is an opportunity in the community, you have the possibility of sharing. I don't know if it's just limited to the engineering community, so that opportunity exists, and it has to be shared and...

0:51:48.5 DG: Oh, let me follow that a little bit, and then go back to the question. A number of engineering deans did send a letter to ABET, saying that we believe DEI is foundational engineering education. And needs to be, essentially, mandated in engineering curriculum, just like the other areas, like working in teams and so on and so forth. So, going back to the question. That's great. What should they put as their requirements? How should they be thinking about, in terms of what to look for? 

0:52:23.9 Tim: I guess I would encourage ABET to give people some space to experiment. It's a pretty restrictive kind of system, and there's some virtue to that, but of course, but they need to recognize that if they want that something new to emerge, it may not be clear what it should be yet. And they should create space for engineering schools to experiment. That can be part of the system, right? Accreditation for us in the liberal arts college is very different. It is not a long checklist, it is principles and affordances. Not rules.

0:53:03.0 Amy: I love Johanna's idea of let's just put one word on there. Let's just put "humility" on there, and see what people do, right? 

[chuckle]

0:53:12.1 Amy: You go figure out how to assess that, how to teach it, and we'll work out later whether or not you wanna harden that idea. But I think that's probably the core of so much of it, because either students come in with an outsized amount of overconfidence and solutionism, or they leave with it, even if they didn't start with it, and they learn it from faculty in engineering. So, what would it look like to counter that? 

0:53:36.4 DG: So, what do you think? 

0:53:38.0 Johanna: Yeah, I like that. I like that idea of just a box that says humility...

[laughter]

0:53:40.9 Amy: Your idea. Your discovery.

0:53:45.2 Johanna: Yeah, 'cause it is... It does run the risk of sort of hardening it, and you don't wanna put something that schools would just take it at a surface level, right? To treat it as a checkbox.

0:53:57.1 Tim: Yeah.

0:53:57.9 Johanna: But making it more kind of open-ended, and more of a sensibility or more of a value as opposed to a measurable skill, and just let them... Universities try to figure out, "What does that look like?" And "What might that mean?" And experiment with it a bit.

0:54:13.9 José: I wanna... Just in case that the... All the audience is not aware and this is José Zayas-Castro, this is not NSF, right? 

[laughter]

0:54:23.6 DG: Stop the recording.

0:54:24.3 José: Let it be clear that ABET is not an abstract organization sitting in Washington. ABET is the engineering community and their professional societies. So, it's when we are with our professorial or industry, it's awesome, because I'm a faculty and in engineering like Argus is. And that comes about how do we go? There were some changes in the ABET 2000 that removed somewhat the checklist approach, but still, there are potential... And universities can create our own objectives and programs and work around that. But again, it's us, the community, who... It's enabled the indirectly or directly. And obviously, what the deans did is important, because they are representing the programs that are being reviewed.

0:55:22.6 DG: Is anyone aware of any models that we should be looking at, that we can adopt to engineering? 

0:55:32.4 Tim: I think what Amy said about information schools is pretty important. I've seen this... Because information schools mostly didn't exist not so long ago and have grown rapidly, they have recruited a faculty which represents, in their own research, this broad array of interests and orientations. The students see that, they participate in it as well. So, I think there's something really interesting going on in that space that is relevant for all of us.

0:56:02.2 Amy: Yeah, I couldn't agree more, obviously.

[laughter]

0:56:09.2 Amy: But I'll say that... As an insider to it, right? I don't think that we've done a great job of sharing our story, right? We've tried. We've tried to be visible, we've tried to make sure those conversations are there. But I haven't written some white paper that describes our undergraduate program to even all of our peers, let alone the rest of academia. So, there's a lot of work to do there, I think, to help share some of those insights and knowledge and where they came from, that we haven't done yet.

0:56:36.5 Johanna: Another model might be the Science and Technology Public Policy program, the STPP. Shameless plug. How they... It offers a certificate for graduate students, and these graduate students come from all sorts of different disciplines. And a lot of them are looking for that social aspect in some way or another, and that's what STPP offers to all of them. So, it exists in a different school, but it's interdisciplinary and open to lots of different disciplines.

0:57:05.1 José: I don't have a motive, but I would, again, encourage everybody that is interested to track down the ORDs of the National Science Foundation in these programs, because they are starting to collect data and see where is that going. Is it multi-year, if we're going from this directorate, and that may stimulate other conversations and participation in submitting other type of proposals and since some of them are taking different shapes. It may be a good thing to keep watching.

0:57:37.8 DG: I'll tell you one thing, just following up on what Tim and Amy said in particular. We're building a new facility on North Campus, called the Leinweber Computer Science and Information building. And we're super excited about it, because it's going to be expansion space for our Computer Science and Engineering program, and it's gonna be the new home of the School of Information. Now, don't tell my faculty colleagues this, from Engineering, but one of the things I'm hoping will happen will be this creative cross-pollination, if you will. So, some of the... Really in some respects, the advances that School of Information have been able to make in the infro space can find their way to computer science. We spilt a series of these community teams in the summer of 2020, after the murder of George Floyd, and others. And that gave us a shot in the arm through a very tragic episode that provided energy for momentum in this change, the notion of, "If not now, when?" As time progresses, I worry that we may actually lose momentum potentially. So how do we continue making this change? How do we not lose this momentum? And what do we do if the culture winds do shift? Let's say in two years, or three years, maybe after election or something like that? What do we do to maintain that momentum so we keep progress going? 

0:59:14.7 José: I'll give an example from a grantee is let's keep watching E4USA, Engineering For Us All, which is a program at high schoolers is being run as a pilot that may explode. And they're doing... And may go into other areas or disciplines like Tim was mentioning before.

0:59:44.0 Amy: I think that when I think about the communities that have the same goal that underlies this conversation, for example, we're surrounded by hundreds, thousands of students who want the same thing, a subset of faculty who want the same thing, sometimes leaders who want the same thing, but often power isn't configured to make that happen. And so I think we really have to reckon with the idea that sometimes if we want these things to happen, we just have to shift power around a little bit, and then they will. And until we do that, it probably means not making progress. 'Cause the people with the power, if they're not passionate about making that change, it's not gonna be what they lead with.

1:00:29.3 José: Then all solicitations that we put out called broadening participation, it added two tracks, Track Three and Track Four. Track Three is what is called, IM hubs. So it's to create hubs to bring folks together and see what the proposals come, but Track Four is basically a center-based approach to broadening participations where the requirement is that the PIs are the deans of engineering. Let's see what that brings. Those are tracks to be looked at, in the years to come.

1:01:07.7 DG: Johanna, what do you think? 

1:01:11.4 Johanna: Sorry I forgot the question.

1:01:14.8 DG: No, it's okay. How do we do... Let me...

1:01:15.8 Johanna: How do we keep it going.

1:01:17.0 DG: Or maybe even, Are you optimistic that the momentum will continue? 

1:01:21.7 Johanna: I'm pretty optimistic that the momentum will continue, I think there's a little bit of a pendulum. So we get a little bit of momentum in this direction. And then there might be a little bit of backlash, where people get tired or people get lazy, but that just... We need to capitalize on that and get the pendulum swinging back again, and get the energy up and make sure that it keeps shifting in the direction we want it to.

1:01:41.2 DG: For what it's worth, I remain optimistic.

1:01:46.2 José: Well, I'll use... This is Jose again, not the NSF. I'll use a quote that I've been using the last several years since watching Professor Cornel West, "We have to be prisoners of hope, with action." We have to act. But we gotta be prisoners of hope, because... And we need to continue going. We need to continue going. Great discoveries in technology, and to keep using engineering precise, they didn't come about because somebody started and made it the first time, there were so many failures even what Alec mentioned at the beginning some of the failures but before that, they were even many more failures and people kept going. And so yes, there is hope as we continue acting, participating, putting proposals, formulating research ideas, collaborating, and seeking. And keep in mind that probably the only place that the shortest path between two points is a straight line is in basic geometry and when you're crossing the street and even that. But in life, there are many curves, hills, and valleys, so you keep being excited because from you we expect... See a lot of young people here, not me obviously because I'm probably one of the oldest, if not the oldest here. The expectation of, no researchers no people doing cross-collaboration, no faculty members that will help us diversify much more of the face of our colleagues.

1:03:29.8 DG: I'd like to thank my panel and then invite Shobita. I think there are questions from the audience.

1:03:35.6 SP: There are lots of questions, I'm really grateful for all of the audience engagement. And in the interest of time, I won't be able to ask them all individually, but I'll do my best to bring them all together in ways that might be useful. So one set of questions brings together I think, something that Tim said with some of Amy and Johanna's comments, and that's around the role of the humanities and social sciences and their expertise in addressing, these are long considerations, of course, and complex ones in those fields, but of course, one of the challenges in addition to the one that you talked about, Tim, in terms of just incentives for teaching, is the way we think about... The way, and by that 'we' I guess I'm talking about the natural and physical sciences and engineering and you talked a little bit about this as well, in terms of solutionism.

1:04:30.9 SP: It's not just that those are open-ended and critical, they surface critique without a solution, but also more fundamental assumptions that I think engineering and natural and physical science and economics tend to make about objectivity, about the central role of quantification, and also about rigor. So assumptions about who is rigorous and who is not. And I'm just... And thinking about that in connection to what Johanna and Amy were saying about fostering different relationships with technology, not just as inventors but also as people and political agents, subjects, political subjects. How do we manage that challenge in practice? So how do we cultivate those kinds of multiple identities when engineers and scientists are taught that objectivity is the way? Quantification and extreme rigor is the way. How do we manage that challenge to really get the benefits of... And foster this kind of interdisciplinarity and collaboration? 

1:05:50.1 Amy: Such a challenging problem because so much of what creates it is sort of arbitrary. The origins of academia did not involve a whole bunch of separate units with people not having conversations with each other. It was the exact opposite of that. It was a bunch of people that didn't really have labels for their disciplines yet, just sharing ideas. This is why we all have PhDs. There aren't PhDs in something, there are just PhDs. I'm fortunate to have been in an institution that has that interdisciplinary at its core. So there's 60 faculty in our information school, 40 different disciplines of PhDs in one unit. And so when somebody like me comes in with maybe some positivist, objective, quantitative view of something like I might have 14 years ago, I was a computer scientist, there was somebody from English, in my department, that called me out on it.

1:06:46.1 Amy: Or when my colleague in English would say things like, "Maybe technology works this way." I'd say, "No, it doesn't actually work that way. It's not actually magic, and it's really actually much less capable than you might imagine." So those kinds of conversations that I've had over the past 14 years across those 40 disciplines, that's what we have figure out how to do at scale across all of academia, and the challenge there is, when? Like I have these at lunch meetings before faculty meetings built into it, but when does it happen across campus, between units? I think that's a really exciting challenge for campus leaders to figure out.

1:07:24.1 Tim: Yeah, I've watched it happen in such interesting ways through my career too, because of course, we teach people in disciplines, and especially when we teach graduate students, we are disciplining them, that's what we're doing, we're showing them what it is to be a physicist, to be a mechanical engineer. And there's reason for that because we've built up a system around that that is very effective in certain ways, but it gets this label of exclusivity of uniqueness that is really problematic. And so what I've seen happen for a lot of scholars that I know, is that their career goes through this, they study the liberal arts, they go to graduate school and they become very disciplined and they get good at it, and they become faculty members, and some of them have the good fortune to break out of that later in their career.

1:08:09.5 Tim: For me, I had this unbelievable good opportunity to run the Honors Program for the College of Literature, Science and the Arts. That's like running a mini version of the whole college, where students study everything you can imagine, and they do it really well. So there's just no way, if you encounter that in that kind of way that you will look across an epistemological divide like this and say, "What they're doing is just crap." If you really look, you will not say that. And so it's giving people the opportunity and the space to really look in that kind of way, that I think develops the kind of respect for disciplinary diversity that's so essential to this. How did we make that happen earlier? I'm not quite sure. But we should try.

1:08:50.9 DG: Oh, go ahead first.

1:08:52.1 José: I will bring back again, the center-based research models that we have, because those things are happening, and we have undergraduates and graduates students, multi-discipline crossing across and functionality in this Gen-4 ARCs, all these things working together. Yeah, I think that's more... It has many units, many disciplines and several institutions.

1:09:28.2 DG: We created this office called Immersed, #practiceyourpurpose. And what Immersed is in the college of engineering is a framework of our experiential learning programs, and I believe that that is the quickest way and the most effective way right now to do this kind of work. Curricular changes take time, but we have about 120, 130 competition teams and outreach activities, and they're going all over the world serving the common good. And we're using that as a framework in which even a first year undergraduate student in engineering is understanding that there is no right or wrong answer, they're trying to understand how to serve community and so on. So, I'm a big fan of saying, "It doesn't happen to have happened only in the classroom, it can actually happen in terms of outreach activities and our competition teams."

1:10:20.6 SP: So we had a number of questions that kind of implicitly were asking the kind of contextual structural question that, there are all these incentives, right? Students have to go out and get jobs, and these jobs might be provided by Microsoft or Palantir or Google, or Department of Defense, etcetera, etcetera. And there's sort of two variations on that, one is, How do we... We're given that employment ecosystem, the economy, how do we... Are there pressures or interest on the part of the employers to foster socially responsible engineering? And if not, what role might universities play in pushing employers to actually incentivize that? 'Cause that's another way beyond accreditation, for example, or research funding.

1:11:18.4 Amy: There's such a big tension there, right? 

1:11:21.6 SP: Yeah, it's not an easy question.

1:11:23.4 Amy: No, I mean, there's many of the students in the audience, online and in here... I'm guessing some of you have a particular motive around studying engineering or computing, right? I certainly did. I grew up poor, I studied computer science to not be poor. And so the idea of withholding that from students doesn't necessarily seem right, and at the same time, that might be the only option that a university can really take is to say, "Do you wanna be in conversation with our students and recruit them? Well, we have some expectations." And we haven't set them yet, we don't often say, especially in computer science, "Here are the conditions for showing up at the career fair. Here's our list of expectations around what your organization does if you don't meet these things, you don't show up, you don't get the resumes, you don't get anything from us." So how do you balance those two things when some of our students might actually deserve some economic justice through access to those jobs, and at the same time, that is a primary leverage point for creating pressure on industry. I don't know how to reconcile those two, but I think that's at the heart of it.

1:12:32.6 DG: I guess I can, very briefly... There is some glimmer of hope from the investment community, ironically. There is this framework called ESG, Environmental, Social and Governance, and companies are scrambling to try to figure out how to show that they are amenable to the positive change associated with ESG. I say that specifically 'cause I sit on boards and we're doing that right now, but it's a very interesting idea about actually universities adopting an ESG framework as well, that's something we have tremendous leverage and could do.

1:13:11.7 Tim: I think even to share with students during the course of that process, a kind of valuation of the companies that come... That's based on this. They do listen to us, they hear what we say when we say something is great or poor. And if the only dimension of that is, What is the starting salary, then that's what will win, so we need to also just keep that in their eyes.

1:13:36.7 SP: I should say, just like I didn't prompt Johanna to sing the praises of STPP. There was a question who was asking about career paths in this area and engineering policy collaborations, and that is something that we also do. We now... We have a program that's been around for 15 years that where... It's small because of course the students self-select into STPP, but by now we have a pretty good sense of, what does the landscape look like outside of the traditional kinds of academia industry jobs? How do you train yourself? What are the pathways into it? And there are... This person also asked about incorporating components of your engineering degree. What I know at other institutions... So Arizona State has something called the PhD plus program. And those are the kinds of things that we've been doing informally, and so whoever wrote this one, still contact me, they're welcome to do that, but I think that those are also mechanisms to kind of do that. But it does take time, because even when I was an undergraduate student, and I was interested in these issues. There still aren't actually clear pathways for people... People still think that they're out in the wilderness and have to make their own way, and I'm happy to say that we have some sense of that now and we can guide...

1:15:06.0 SP: Certainly guide students. So we had a bunch of questions about different aspects of teaching, but I wanted to... Maybe I'll put Johanna a little bit on the spot here. Because there's a question about... I know you've thought a lot about this, which is about introductory courses, so the 101 courses in computing or engineering or other fields, how do you teach those courses? What have you found from your research? I know you spent a lot of time doing, focusing on this very question, how do we change the way we teach those courses to accommodate more diverse ways of thinking and being? 

1:15:47.6 Johanna: Yeah, that's a really good question, and I've only sort of started to uncover the answer to that, there's a number of people doing a bunch of different sorts of things, some of it is giving the problems in the intro level courses context, so instead of sort this random list of numbers that has no real world meaning, it's sort this list of people, or something that just has some level of context in it that allows students to connect the CS concepts to real world things to some extent. Now, if that's the only thing you do that could backfire because then students might be saying, "Oh, I'm never gonna be writing a list that's going to sort people in this kind of a way, I'm gonna go and do computing for economics, and I'm only gonna be sorting lists of numbers for money or whatever it is, right? So students could reject those things.

1:16:38.7 Johanna: So I think it needs to come at it from a couple of different perspectives, the way I think about it is in terms of the practice of computing, so when they're in intro level courses, they're learning a practice, they're learning like, okay, when I am programming, I am thinking about this, and then I'm thinking about this, and then I'm executing this. And so as part of that practice, it needs to not just be a technical practice, there needs to be a reflection built into that, and so what are the points in those intro level classes where we can prompt that reflection, so students always have that reflection as part of the practice. So then when they start building bigger and bigger things, they're used to reflecting on things, and sometimes that reflection is just, "I don't know how to solve this, I have to go down to the hall and get somebody who's an expert in a different domain," but it's the reflection that needs to be built into it from the beginning.

1:17:27.2 SP: That's interesting. What role should teaching play in the training component of graduate training to support equity and serving the common good? So obviously, Tim, you were talking about the pressures on faculty, but the pressures are similar on graduate students in terms of...

1:17:44.6 Tim: For sure.

1:17:46.6 SP: It's sort of seen as, especially I think in engineering, it's seen as problematic if you have to do a teaching fellowship as opposed to a research fellowship or something else. So how might we change PhD training and culture perhaps to think about these kinds of considerations at that level, to focus on questions around equity? 

1:18:12.5 Tim: I think it's a great question, I can think of some interesting examples. It's not rare for graduate students who, after all, bring a lot of experience with education to their work as young educators. They've been working their way through the system, usually successfully, up to that point, so they're really interested in it. They would like to do it well and they work against a system that devalues the work they do as educators. So, in the chemistry department here, for example, they have developed an approach which isn't training discipline-based education researchers in chemistry, but it's training their chemistry students with a bit of that research so it is possible now for a chemistry student here to complete their chemistry thesis with one chapter on an education project, to allow them to work on the education work that they do in a scholarly way without taking off the track their general chemistry degree.

1:19:09.0 Tim: So, that idea that the faculty would welcome that and would allow it to happen and count it as part of their education, gives some space for them to gain the kind of training and to think about the questions and to wrestle with experts who are outside of chemistry often, on questions about equity and inclusion, for example. So, we can build systems that support this kind of thing.

1:19:34.6 Amy: And there's a hunger for them, too, right? 

1:19:35.6 Tim: I think so.

1:19:36.7 Amy: We're launching a new secondary CS teacher education program just in this next week for K-12, middle school, and high school teachers to learn how to teach CS from an equity and justice perspective and we had tons of interest from K-12 teachers and from our pre-service teachers but the majority of the interest was actually from doctoral students on campus, desperate for getting some sort of support around how to support their teaching and their interests in pursuing tenure-track or teaching-track careers and just finding nothing else on campus to do it. So, I think if we build these things, there will at least be interest. The incentives might push against that but I think we'll find people to join.

1:20:16.8 SP: Well, this is also about the long-term, this question about is this just a moment where people are paying attention or a long-term thing? You take advantage of the moment, but then you build long-term capacities.

1:20:29.5 Tim: It's that youthful interest that convinces me it's not just a moment. The next generation of scholars is not us.

1:20:39.0 SP: Yeah.

1:20:39.4 Tim: They're a lot better in a bunch of ways.

1:20:43.6 SP: Well, then this next question might depress you a little bit, Tim.

[laughter]

1:20:49.8 SP: Actually, it's not but it does identify, I think, a challenge. You were all talking about humility in a really interesting way and this question kind of reminds me of the weeder class culture, they called it here. But I think more generally, it's about competition, competitiveness, aggressiveness, certain kinds of characteristics that we see in engineering culture I think in particular, but other areas, certainly of the disciplines that we've talked about. And how do we address that? It seems like that's also something that's at odds with what we're talking about here and then of course, large universities, elite universities, certainly experience. So, how does that affect, how does that weeder class culture affect diversity? That's the question here, but also what do we do about that? Beyond teaching them humility, or maybe it's about how do we operationalize humility in a way that is actually effective in addressing that weeder class culture? 

1:22:00.4 Amy: There's a question here about whether or not the scarcity is real. At the core of a lot of these challenges of scaling engineering education, computer science education, it's a belief that we can't serve everyone and that's what creates these toxic cultures and classrooms where people can't support each other and collaborate with each other and teach each other. And so, I do wonder, sometimes, and there's certainly some research that's exploring this, if that's actually true. What if we completely just reinvented the pedagogy such that the 1000 students on campus who wanna study a particular engineering or CS discipline can support each other, along with faculty help and graduate student help to do that at an arbitrary scale? 

1:22:46.4 Amy: I don't know that we've really done the research to explore how to make that work successfully and at the same level that we do it now, but I don't know that it's impossible. I just don't know that we've had the courage yet to try those models.

1:22:58.2 Tim: We're saddled with it here. We're the leaders and the best and everybody does it every day.

1:23:03.9 DG: Exactly, exactly.

1:23:04.9 Tim: And it's a problematic in one way. So, I think that part of this is, again, getting people to recognize the complexity of the reality they're talking about. So, I do think in a certain way, everyone on campus is a leader, but they also need to recognize the full range and diversity of skills and nature of leadership. You're gonna lead something, you're gonna be the best at something, but guaranteed, you're not the best at everything. In fact, everyone else is better than you at almost everything so it's kind of getting. I think it's true, right? So, thinking in this more high-dimensional way about value and virtue and optimization, again. There's no simple optimization here, there's a kind of multi-variate and multitudinous thing that's going on and we need people to recognize that.

1:23:58.5 Tim: I think the university can be a great place to do it 'cause I know that when I look around and see how great everyone is and how different they are in their greatness, it's like the best demonstration of that that I've ever seen. I think sometimes in a narrower context, maybe in a company or something, there's just a hierarchy and it seems like that's it. But on a university campus, look around.

1:24:21.9 SP: We got it all.

1:24:23.4 Tim: We got it all.

1:24:24.1 DG: So, I was listening... It's related, but I was listening to public radio lamenting the way in which higher ed campuses tend to, if you will, polarize discussions in classes that it's ultra left-leaning, and the feeling is that higher ed is to blame for students not feeling comfortable expressing differences and so on and so forth. And then I'd say, "Wait a minute. You mean they were wide open before we got them? [chuckle] Everything was okay and then somehow... " So frankly, this is a societal problem. We have to be part of the solution or we're part of the problem, but how do we, actually in higher ed, help illustrate the fact that this is happening not just in college, this is happening in high school as well, and before that.

1:25:27.6 SP: Yeah, absolutely. So, one final question, which is about... And I think it relates to some of the things that you were all just talking about, which is that... And Tim said that the next generation, I think is... That we should put a lot at stake and hope in what they have. And I'm just wondering, when you think about building equity-centered education and engineering education, the role that students are or could play in pushing that, because of course, I think students are... At least in my own experience, I've often found that it's really interesting to teach students because they're kind of low on the power totem pole, so they see power. And so then it's a little bit easier to get them to think about those dimensions, but that also suggests perhaps that they might have unique insights and knowledge to help inform how we re-make education. And I'm wondering if you're involved in any of those things or have seen that happen and what the opportunities might be there? 

1:26:36.5 Amy: Yeah. We talked to a lot of students who struggle with wanting change, but not necessarily wanting power. And there's a... I think there's a really big challenge there around how do we cultivate learning contexts that give them the confidence to hold power once, let's say faculty and deans and all of the other people, like the people on this panel, are willing to share it. They're willing to say, "Please come and help co-construct this thing with us." We have to also support them in figuring out what it means to hold power and share it, not to mention educating our own faculty and leadership about how to do it. Our own university, for example, only just two years ago, started saying, "Hey, why don't we bring faculty leaders together and teach them how to be leaders, and hold power, and respond to change?"

1:27:24.0 SP: Absolutely.

1:27:24.4 Amy: So I think we're at the beginning of some of those conversations, like helping faculty and leaders on campuses understand how to do that, so that we can help students understand how to do that so we can share power and make some change together.

1:27:38.6 José: It's not uncommon that colleges of engineering, there may be one here, is an engineering student council, that holds the student societies. And that's one mechanism that could be used to bring together and actually could be even an interesting question for somebody to try to develop and propose some research and mixing with colleagues from other areas, but, me as the professor, over the years, what I did is work with student societies to do some things that others wouldn't do with students, and they worked out well, then the question came about sustainability because somebody has to pick up the baton and continue.

1:28:28.9 SP: Any final thoughts, comments? 

1:28:35.1 Amy: I'm still optimistic. [laughter]

1:28:37.2 SP: Words of wisdom.

1:28:39.5 José: We need to be... We need to keep hope, highs and keep going. Don't...

1:28:42.6 DG: No, we do.

1:28:42.8 José: We just need to use all the mechanisms from NSF, your university, your environment, the faculty, and the colleagues that are gonna work with you. Peer students, peer groups, student societies, and so on because there are a number of opportunities. We would have to deal with the challenges, though. Challenges are not gonna disappear overnight, but it's a matter of doing so, and the investments that the National Science Foundation has put over many years are not small, are not small by any means.

1:29:21.6 Amy: And when you get tired, just pass the baton and wait for the next lap. Make sure you've got community that helps you push forward with it.

1:29:30.2 Tim: This is a classic example of a problem that doesn't have a solution. It's just a problem, we just have to work on it. It's not like you're gonna get to a point where you're gonna say, "Oh."

1:29:43.7 Amy: There it is. That's the solution. We found it.

[laughter]

1:29:45.2 Tim: No, it's gonna just be constantly.

1:29:49.3 SP: Well, I'm certainly inspired by this panel and all of you, so... And the work that you're doing, so thank you so much for joining us.

[applause]