The Magic of Engineering and Computer Science With Dr. Karen Panetta
/LDV Capital invests in people building businesses powered by visual technologies. We thrive on collaborating with deep, technical teams leveraging computer vision, machine learning, and artificial intelligence to analyze visual data. We are the only venture capital firm with this thesis.
Our Women Leading Visual Tech series is here to showcase the leading women whose work in visual tech is reshaping business and society.
Our next guest is Dr. Karen Panetta – an electrical and computer engineer, inventor and Dean of Graduate Education for the School of Engineering at Tufts University. Her research areas include artificial intelligence, machine learning, automated systems, simulation and visual sensing systems. Karen develops signal and imaging processing algorithms, simulation tools and embedded systems for applications for robot vision and biomedical imaging applications.
She has won a number of awards for excellence in research, social impact, teaching and mentoring, ethics, and engineering education. She is the recipient of the Presidential Award for Excellence in Science, Math and Engineering Mentoring from U.S. President Barack Obama.
Dr. Panetta founded the "Nerd Girls" program, which encourages young women to pursue engineering and science. Karen is the editor-in-chief of IEEE WIE Magazine and co-author of the book, "Count Girls In”. Her passion is utilizing her engineering knowledge for the benefit of humanity. Karen has advised world leaders and led humanitarian projects around the globe to change lives, help communities thrive and protect the environment.
Abigail Hunter-Syed, a Former Partner at LDV Capital, chatted with Karen about teaching computers to see like humans, the role of diversity of perspectives, social impact projects, the role of AI in biological image processing and so much more! (Note: After five years with LDV Capital, Abby decided to leave LDV to take a corporate role with fewer responsibilities that will allow her to have more time to focus on her young kids during these crazy times.)
The following is the shortened text version of the interview.
Abby: How would you describe what you do to a two-year-old?
Karen: I had to do this before because I had a two-year-old at one time!
I talk about what I do – engineering and science – as the magic that makes things happen. To other people, that looks like magic.
I’d say, “you can take a bunch of ingredients and when you put it in the oven, it transforms into cupcakes. That's the magic of science – there are ways to put ingredients together and use our imagination to make new things possible that we thought we couldn't do other than by magic”.
Abby: How did you decide to go into electrical computer engineering?
Karen: I'm going to be truly honest: I had no clue what electrical and computer engineering was. The computers were coming out and my dad said to me, "You have a very expensive shopping habit and you are going to need a career that you can afford to your bad habits. I don't want you to be dependent on anybody else. I want you to be financially independent." He chose it for me. He was disappointed that I didn't become a civil engineer.
I loved programming. I thought it was easy and fun, but that wasn't enough. I wanted to know what was inside the box: the magic. I wanted to know how to make that magic myself, so I learned computer architecture. But even after I graduated with a computer engineering degree from Boston University, I said, "computer engineers build computers." That's what we do, so I did become a computer architect. Afterward, I did that for many years and that's where I first got into the simulation, and that's when I started to think, "Wow! There are more things I can do with a computer engineering degree than just build computers.
Abby: How many computers were there on campus at BU at that time?
Karen: Do you know that old stereotype of being an engineer or program in the basement? We actually had to go in the basement, but you had to share. Back then, you had one mainframe the size of a coffin or two and you had to share this computer with other users, so you connected via a terminal. We used to have to sign up and wait in line for 45 minutes to get in that terminal. And then, they'd time you out to say, "You've been on. You got to give somebody else a chance." So it was not just you had to be interested, but you had to wait in line. It was almost like waiting for the newest hot toy to come out! I did that for many years, so when personal computers came out it was like, "What? I can have one in my house?" That was amazing!
Abby: They still are these magic little boxes that perform any task that you can possibly ask of them! When did you first come across computer vision and how did you decide to go down that route?
Karen: I came to Tufts and I was the only woman in electrical and computer engineering. They said, "You're hired to be a mentor for women." Unfortunately, there were no female students.
While I was at Tufts, I was trying to think about what kind of projects could I bring and look at. In some of the research problems I was looking at, everybody kept looking at problems and saying, "A human would say this or a human observer would say that," and they were trying to emulate that on a computer. My first interest was, "Well, if you want to emulate a human, why don't you model the way humans see?”
A simple example: a lot of the standard image processing algorithms used linear arithmetic but your human eye perceives light in a logarithmic manner… To me, that was common sense so I came up with a way to have computer vision determine automatically, without a human in a loop, what a human eye would say, "This is the best quality image." That was groundbreaking at the time, and especially for somebody who was a computer architect coming into this field where other people had been in this for decades. It was like, "Who is she? Where did she come up with this?" It was straightforward and simple, which threw salt in the wound, "Why didn't I think of that?" They didn't have that perspective.
Abby: That lens gave you the perspective that you needed to do something revolutionary. It sounds like it wasn't necessarily an idea that was met openly by your peers.
Karen: Abby, you're right because that was the impetus for me to show why diversity and different perspectives can contribute to making impactful innovation, and that your imagination contributed to that. That huge barrier that I introduced was like, "I came from this different perspective of the digital logic world and here are the efficiencies that I don't see these other fields of image processing even leveraging on."
Dr. Panetta is the recipient of the Presidential Award for Excellence in Science, Math and Engineering Mentoring from U.S. President Barack Obama.
Women are neglected in this equation. At that time, 90% of all minivans were being sold to female customers and it was like, "But the dummy that they did the crash test with were modeled after a guy. You're missing the point. You're missing a market." When I had my son, I saw that the toys that were out there for boys were mechanical engineering-wise, but yet, from when I was a kid, there were the dolls that you feed & clothe. That hasn't changed and that's where I started thinking about gender equity in how we deliver the engineering curriculum.
Abby: You started with color and image enhancement and then you moved on to detection at the edge, then there are some biomedical applications and autonomous vehicles. How do you decide what to work on first or how do you prioritize all of these different applications?
Karen: It sounds complicated, but it's straightforward. I am motivated by social problems. I get into firefighting because I see people can't see through the smoke for search and rescue. I got into biomedical imaging because I see how many people are misdiagnosed.
I come up with engines and then I put them in different vehicles. The engine that I use now is a combination of artificial intelligence, simulation, and image enhancement. I put all these pieces together to build a different vehicle.
I can detect wrong-way traffic and kids on bicycles versus somebody in a motorcycle. I can do deer detection. I started doing elephant conservation – it's a different application, but there’s always the same engine at the core, it’s just tuned correctly for that application.
Abby: You've got this engine that could be used to motivate or drive change in a variety of areas, and then it sounds like you decide what to prioritize or where to use it next based on true need.
Karen: When COVID hit, I was already working on biological image processing and AI. We were one of the first to say, "Let me see if I can do an automatic detection of COVID in X-rays and distinguish it from viral pneumonia." Now, we're looking at, "That's not good enough anymore. We want to be able to tell the severity of it so that we'll know who's going to be hospitalized." It shows you that we can pivot using the same engine.
Abby: Out of all of these projects, what's been your favorite one so far?
Karen: The ones that I've done for conservation and for saving lives like for the firemen, as well as for traffic management are the ones that I see the most immediate impact on. I've done projects on autism for young girls in India but such projects take a lot longer for acceptance and adaptation.
Abby: It must be nice to be able to see a result that leaves the world in a better place than it was when you started your research within a shorter period of time! What drives you behind that? Why do you think it's important to you to use your skills to improve the world that we live in?
Karen: It's funny you’re saying that, because every time I say, "Daddy, I'm working on this new blah blah blah project," he says, "Is this going to make you money? What money are you getting? You're advising the president of Malawi. What are you going to do in Malawi? You're not going there, are you? How are you going to help people in a different country?"
I want to show it's been not about me. It's been about how I can be the best educator and role model for my students. They are my legacy and I can empower them to change the world.
Most women drop out of engineering because they don't see the social impact that they can make. Especially in my field – electrical and computer engineering – they're still the least representative field of cross-engineering. What if you can show how your perspective and skills can make a difference?
I had a young lady who was only in her second year when 9/11 happened. I happened to be working on a high-speed communications device for one of the companies that manufactured cellular communications. They called me and said, "We're going to manufacture your device in the next 48 hours and we need to deploy it to New York City to bring back communications to the first responders." I panicked and my student that was doing the modeling was like, "It's going to be fabricated," and it worked. We brought back communications in New York City.
I asked her, "Did you ever think that you'd be working on something so impactful?" She said, "I never thought anything I learned in your class would be useful." Young people don't see the big picture. In 48 hours, that student saw the big picture.
Abby: When you first got into the field you were the only woman on the faculty. Have you seen any growth in interested women?
Karen: A lot of the barriers are coming down. We put these filters on youth saying, "If you're not the best at math and science... " To a young lady, that means, "I got to get A-pluses." Even when they get that grade, they still think it was luck. "I don't know. I don't feel like I know it." Where if you say that to a young guy, he might say, "I got a C. I'm doing fine," so there are different thresholds.
For the longest time, the projects were not gender equitable. I always give the example as if you go into a classroom and you say, "Today, we're going to design high heels." You said that to boys and girls and you're going to get parents calling you saying, "I don't want my kid design high heels." But if you say, "We're going to design sneakers," now you've made a gender-equitable project because everybody loves sneakers and that's been the predominant barrier.
Young ladies think, "I can't be a cheerleader. I can't be singing. I can't do sports." For the longest time, when I first introduced the Nerd Girls program, I got accused of casting the girls and I was like, "No, these are students," because they thought that, to be an engineer, you have to look like someone who's socially inept and unkempt. They were sportswriters, gymnasts, ballroom dancers. That's the richness and depth that we have to show the world, that this is what you need to make the most impactful, innovative engineers of the future.
Abby: I suspect that most of the male engineers weren't told by their fathers, "because you love fashion so much, you better get a good-paying job." I can only imagine what it was like, not only to be like one of the first women in the area but also to be one of the first women who cared about something more than just the code, that saw a bigger picture and that loved to do it dressed stylishly.
Karen: I showed up before Pretty In Pink or before the Legally Blonde movie came out. I came in this pastel pink suit with matching pink shoes and immediately what you looked like was, "She can't be technically sound. She can't be scholarly." I was shocked by the stigma and the bias that came with just being an individual. Now we understand that it doesn't matter how you dress. You can express yourself any way you want. It's what you bring to the table.
Another myth was that they hired me thinking that all women are nurturing and we're all, by nature, excellent instructors. They were lucky I was, but I had never taught a day in my life and nobody trained me.
Abby: I love thinking of you as the Elle Woods of engineering! You mentioned how you went into this new teaching position without any experience or training. You were thinking about, "How can I rewrite the educational curriculum to be more inclusive?" Why do you think it's important to talk about creativity as part of STEM?
Karen: You can't come up with innovation without imagining things that don't exist. If you put yourself in a box and say, "No, I can't go to the moon. No, I can't be an astronaut. No, I can't come up with a new medicine to conquer these afflictions", you can’t come up with a solution. You have to be able to not be intimidated, even when other people think you're crazy.
I'll give you an example – asteroid mining for resources, making jet fuel out of water in space, or reducing a load of payloads to do mining and set up manufacturing on the moon. People hear this and they think it's science fiction. It took someone courageous and creative to put that idea out there... Acceptance has not come easy for new innovative ideas.
Image credit: Jamie Heller
Abby: You're using creative thought to figure out a way to connect the dots between two things that people had not otherwise hadn't connected. I talked to a friend of mine who's a plastic surgeon and who had been an art student as their minor in college. When I asked her about it, she said, "how in the world am I going to be able to design somebody's face if I can't draw or if I don't have any artistry in me whatsoever." It’s important to recognize how creativity has a major role in STEM. What's your piece of advice to women who are looking to break into the field today?
Karen: I have multiple, but the first one is: get ready for failure. We think our career paths are going to be a beautiful linear line and it's a zigzag, right? It's up and down.
Engineering is about learning from failure because if we feel early and often before it gets out to the intended audience or people that we want to use it, then it's going to be more robust.
Another thing that drives women from engineering is they fail or an experiment fails or something didn't work. And I always say, "If you play sports and you're on a team and you lose the game, do you quit the team and the sport?" The answer is ‘no’, right? You train harder. You're smarter and you try to learn from those around you that have skills you don't. Engineering is the same.
What is going to get you to the next level? It's surrounding yourself with people who empower you. You can always find somebody to shoot holes in your ideas or tell you you're crazy. About every patent I have, somebody has told me I'm crazy. Now, when I propose something and they say I'm crazy, I'm like, "I got it. Here's my next patent."
We often look for affirmation from people that we admire. Even if you admire them, that doesn't mean they're going to empower you. If somebody tells you can't or you're not good enough, or you'll never do it or that's a crazy idea, I usually say, "kick them to the curb." I know that's not an eloquent way to say it, but I do. I kick them to the curb. Move on. Find somebody else who is going to empower you.
Dr. Karen Panetta with Ph.D. student Qianwen Wan circa 2019
You might not find it in your organization. I had to reach out to professional societies like the IEEE to get that camaraderie and mentoring that I couldn't get because there were no other women to mentor me, so I got that outside. You have to be resourceful enough, say, "You know what? I'm missing something. Where can I get this? This is what I need." Or "I don't even know what I need. Who can I talk to?" Reach outside your organization. Don't stop at the people who say, "No, you can't," or, "We don't have that here. We can’t to spend this time with you." Nobody knows you better than you, so don't let anybody hold you back. If I had listened to them, I would not have been anywhere near where I am now today.
Abby: That's such a great point. As women, we like to look for affirmation. We make decisions differently than men do and go out and we ask like, "What do you think of this? What do you think of this? What do you think of this?" as we build up consensus or we build up a thought around it. It sounds like IEEE has been one of those groups for you that has been imperative in helping you make decisions. Tell me a little bit about IEEE and the women's group that you run there.
Karen: When I was a sophomore in college, the same five people from the class were being interviewed for every single job that came out and it was like, "What about the rest of us?" I joined the IEEE and they sent a gentleman from Ohio to come in over a weekend to do a student professional awareness conference. That's where I saw like, "Wow! Look at the way he speaks," and I learned...
This is an example of somebody I want to be able to lecture like. He was so impactful and he believed in us. He didn't even know us and he was like, "You know what? You're in IEEE. You're in engineering. You had enough gumption to dedicate your weekend to your professional development, to be here when you could be out having a good time somewhere else. He says, "You're all going to make it because you're showing the potential. Just the fact that you've vested in yourself."
I kept in touch with him. I joined and became a leader in my Boston section, and then I became the worldwide director of Women in Engineering. Then I came up with the idea of the magazine to tell the stories of creative women making an impact all around the world: women that escaped political prosecution or from cultures where women weren't supposed to be in engineering, or they were confined to the cultural norms of their backgrounds. Telling those stories and sharing them is impactful. It shows that we're not alone. We always feel that we're isolated.
I love Harry Potter for the magic, but there's also a line in there where he feels that the world is against him. Luna Lovegood says to him, "That's because you're not as much as a threat if they think you're alone." That's why ‘women in engineering’ help because it showed, "You know what? We're all experiencing the same story over and over. It's not just me who has experienced this, but millions of women around the world. This is something that we need to talk about rather than just internalizing it, and how do we solve it together?" Empowering and solving these challenges together is what we've been doing because we are a triple threat if we grow in numbers.
Abby: I love that! A fun question that we always like to end the interview on is, if computers didn't exist at all, what do you think your career choice would've been?
Karen: I would've been in music. I'm a professionally trained opera singer. I would've loved to have been on Broadway. I would've been dancing.
Abby: So in college instead of choosing the stage, you chose the basement?
Karen: It's funny because my voice teacher, who was trained with Metropolitan Opera, once asked me, "When it comes a time, will you give up the engineering?" and I was like, "In a heartbeat." Then my dad was like, "No, you're not making any money with that." That sort of killed that, but I'm trying to inspire that love of music with my child.
All of us grow up thinking we are financially constrained to do the right thing to support ourselves and sometimes we sacrifice our creative passions for that. I haven't had to sacrifice a lot of my creative passions, but I didn't get to fully explore them either.
Abby: The world is a much better place because of the time that you have spent engineering. Hopefully, that gives you a bit of solace and that you'll find all of those creative outlets as fun things to do after you've improved the world with all of these amazing applications of engineering. Thank you so much for taking the time to chat with me about the magic of engineering and computer science!
Karen: Thank you so much, Abby. It's been so much fun!
Hope you enjoyed this interview with Dr. Panetta as much as we did! Check out other interviews in our series and meet brilliant women whose work in visual tech is reshaping business and society.
