Dr. Kavita Bala on Bridging the Gap Between Academia and Industry
/LDV Capital invests in people building businesses powered by visual technologies. We thrive on collaborating with deep tech teams leveraging computer vision, machine learning, and artificial intelligence to analyze visual data. We are the only venture capital firm with this thesis. We regularly host Vision events – check out when the next one is scheduled.
Our Women Leading Visual Tech series is here to showcase the leading women whose work in visual tech is reshaping business and society.
Dr. Kavita Bala is an esteemed computer vision researcher, entrepreneur and leader in the computer science space. Her research expertise lies in the area of computer vision and computer graphics using deep learning. Some of her work on cloud rendering is the core production engine in Autodesk's cloud rendering platform.
She served as the Chair of the Department of Computer Science at Cornell University from 2018 through 2020 and has now been appointed dean of the Faculty of Computing and Information Science.
Dean Bala also co-founded GrokStyle, a graphics search company that won our 2016 LDV Vision Summit Entrepreneurial Computer Vision Competition and GrokStyle was acquired by Facebook. In 2019, she was elected as an ACM Fellow "for contributions to rendering and scene understanding” and this year, she received an ACM SIGGRAPH Computer Graphics Achievement Award.
LDV Capital’s Abigail Hunter-Syed spoke with Dr. Bala about her career path, current projects and more. (Note: After five years with LDV Capital, Abby decided to leave LDV to take a corporate role with fewer responsibilities that will allow her to have more time to focus on her young kids during these crazy times.)
The following is the shortened text version of the interview and the unedited video can be found below.
Abby: When you were growing up, what did you answer someone when they asked you what you wanted to be?
Kavita: I had learned Indian classical dance for a long time to the point where I thought that might be a career path for me. I had to decide between an academic path and a performance art path, and I chose the academic one.
I was also interested in law and social studies. I had wide interests when I was younger and I did different activities where I had these points in my life where I had to make important decisions as to which direction I would go. For performance art versus math, at some point, I was doing well academically, and I thought I saw that as a longer-term path, and I went there.
In an Indian school, you have to choose between different areas that you want to focus on relatively early. When I got into 11th grade, I had to choose between biology and a computer science path, which was relatively new at that time. Taking the biology track required dissecting a frog, which I knew I had no interest in. That made me take computer science courses. I fell in love with computer science and that determined my life's trajectory.
Abby: What about your parents? Were they like, "You're crazy. What is computer science? Why would you go down this path?"
Kavita: My brother is an electrical engineer. My parents were not as out of the loop because of him, they knew about this up and coming field of computer science and were open to that.
When I went to my undergraduate, which was the Indian Institute of Technology Bombay, I was the only woman in my class. My parents and my classmates were supportive.
Abby: How many men were there versus you? What was the ratio? 1 to 200? 1 to 300?
Kavita: Hundreds of thousands of students take a college entrance exam, you have to be in the top 150 to get into computer science.
We were this cohort and then we split into different parts of the country. It was a huge bonding experience. Nearly 30 years later, we have a WhatsApp group where we all hang out.
Abby: One of the amazing things about technology is how it allows you to stay connected to all these people throughout your life even though you may be hundreds of thousands of miles away from each other. Do you have a piece of advice that you give to girls all over the world who are getting into a male-dominated industry like computer science?
Kavita: It was tough. It remains tough. Things have been improved, but there's a long way to go before we get to the point.
When I went to MIT for my Ph.D., I had a study group with three women. They were bemoaning how few women there were, and I was like, "Well, there's more than one. This is great.”
At Cornell, our undergraduate population is 36% women. I've been a faculty mentor for the women in computing group at Cornell, and so we've come a long way from where we were. But that's at the undergraduate level. We need to continue to increase the pipeline at the graduate level, the Ph.D., and the faculty level. They're making progress but there's a lot more to do.
I used to not ask questions in class, which was terrible. What I learned over time, is that you still have to be comfortable with who you are and you can engage to the level that you're comfortable with, but maybe push a little beyond your comfort zone, bit by bit till you get there.
This is probably a piece of advice that's good for both men and women: work up to where you feel like you're comfortable and be part of the whole process.
Abby: I was talking to Dr. Timnit Gebru in one of the previous interviews in this series, and one of the things that we discussed was how when you were younger, you would always talk up in class and when you get to the university level, it suddenly becomes precarious or dangerous to speak up. As you get older you start to build back into this being who you were. You start asking questions and become the outspoken person that you'd always been in your head.
Kavita: I went through that same process too. You keep pushing until you get to that point and get comfortable. I'd say things have also changed in the following way: there are a lot more mentorship and allyship support groups, this helps to draw people out who are shy or not comfortable. I hope that we will see more of that resulting in better outcomes where people feel comfortable talking in big and small groups.
With our women in the Cornell computing group, we revisited whether faculty call out students to ask questions in class and if they make sure that both men and women have an opportunity to speak up. Over time this resulted in the women speaking up more, they feel more comfortable with it.
It's one of those things that can be done. If you're aware of the issue, you address it in a classroom setting and it results in better outcomes, and people get used to it.
Abby: It sounds like being involved in such groups has been a big part of your work. How do you manage to balance all of these different roles of mentor, professor, researcher, founder?
Bill Gates listening to Kavita give a presentation in 2015
Kavita: It's been intense. It feels like it's been a continuous roller coaster ride for a five-six year period. I love it, and I do it because I love it. Each one of them brings out a different part of my brain, my skill set, my excitement. I feel like it results in the most fulfilling experience of my life. For me, it’s important to continue the research because it keeps me connected to the academic world.
I came into this job because I enjoy open-ended problems. I have to keep the research alive, but on the other hand, I’m the Dean of Computing and Information Science, and I like entrepreneurship.
I was an academic all the way through till 2015 when we launched GrokStyle. I took my sabbatical, and I also took a leave of absence and I spent two years at GrokStyle working with Sean Bell building up this company, and it was a blast. It was an eye-opening experience for me to the point where I can't imagine not having some sort of participation in entrepreneurial activities going forward. It changed a part of my life, it turned on some switches that were off, and I enjoy it.
Abby: It sounds like it was a natural transition for you to work on something that you love from a research perspective. You said it was 2015 when you guys started GrokStyle and you took a sabbatical to go start working on it. Was that when you started working with Sean Bell or had you started working together before that?
Sean Bell, Co-founder in GrokStyle presents at LDV Vision Summit Competition 2016 © Robert Wright
Kavita: Sean Bell was my co-founder in GrokStyle. He was my Ph.D. student from 2011. We had been working on research together, so we had a great rapport. That's why we got started on the company together.
We had been working on research together the whole time and working on different problems of what is called “scene understanding”. Here’s an example: if you have an image, as a human being you look at an image and you can immediately understand what's in it. You can recognize headphones or a keyboard, and you can tell if something is made of plastic or leather, and so on. You know what you're seeing. This problem with “scene understanding” is a fundamental computer vision problem, and we were working at different parts of it in our research. We were looking at material understanding, and building data sets to build newer networks so that we can do a better job at recognizing scenes.
While we were doing all that research, we were exploring a lot of internet photo collections. People would post images of different things, and as we were exploring them we realized people were asking a lot of questions about each image. "Oh, I wonder what this handbag is in this picture of a celebrity. Or I wonder what the chandelier is in this picture of this house."
That motivated our research, so we created GrokStyle to work on this problem of visual recognition. Given a picture of some furniture, we can tell you not only that it's a chair, but it's the Eames chair and where you can buy it. That was the commercial idea that we explored.
We started the company based on that idea. In 2015. Sean and I founded the company. We came to the LDV Vision Summit, Sean presented our work and we won. This was hot off the press.
It made a big difference in our journey as a company to have that experience and that award. That time, we got two lead investors: Krishna Bharat, who was the founder of Google News, and Amit Singhal who was running Google search at that time. Red Bear Angels from Cornell joined us, among other early investors, and together we got enough seed funding that we could launch the company.
We started in Ithaca then we moved over to San Francisco. We had a team of 10. At some point, we had five women, so it was a balanced team.
Abby: That's exceptional. I remember looking at one of your photos on your website, and it was one of the first things that I had noticed.
Kavita: They are wonderful people. We're hoping to have a reunion sometime (after covid).
Abby: You had a fantastic exit recently to Facebook where they're now envisioning the future of shopping, as they call it, based on your technology. How does that make you feel to have one of the largest tech companies in the world using and deploying your technology?
Kavita: It's a blast. The reason we got into GrokStyle in the first place was we were excited about the technology and wanted to have an impact. When we started talking to Facebook, it made a lot of sense for us to get acquired by Facebook because we thought Facebook was a perfect home for us to push this technology out to have that impact.
Billions of users and images are using GrokStyle’s core technology, called GrokNet, within the context of Facebook. They built it up with a team of engineers. It works across a range of applications, and it's exciting to see it deployed.
Abby: It's not the first time that you've seen some of your research ends up in a major mainstream tech company because Autodesk's cloud renderer is based on some work that you did before, right?
Kavita: That was a different path, but we indeed got our technology adopted. This was work that I did in 2005-2008 with Bruce Walter and other great collaborators at Cornell. We were looking at the problem of rendering, the process of producing images that look real. One of the big problems with that is how do you capture all the physics to produce realistic images. Light bounces around in the real world so how do you efficiently compute that?
Lightcuts was an efficient rendering algorithm, and Autodesk adopted it for their cloud renderer. We published our paper, Autodesk looked at it, they wanted to use this scalable solution to rendering so they adopted it into their pipeline. While it was our technology, we didn't play a visible role in getting it out there and working with the engineers. It was great to talk to them and learn about the new problems they were seeing.
You get inspired by new problems when you work with industry. Both in GrokStyle and in the Lightcuts, we had the benefit of working with industry to help us define some of the problems.
Abby: Do you think that that's one of the biggest differences that you found between academic research and then doing it as a company is those touchpoints with real industry players to understand some of their real needs?
Kavita: That is the biggest part of it. When you're sitting in an academic setting and faced with a problem, you have to make some decisions on how to simplify it. If you don't understand what's important in an industry, you might simplify the problem the wrong way. You might throw away the important piece of it that needs to be solved and solve the uninteresting part of the problem that nobody is ever going to use.
That's where that connection with the industry is critical. It drives in the direction of solving the real problems, and that's why it's so rewarding to be able to work with industry partners.
Abby: I know that you were working on an algorithm to predict fashion trends. Are you still working on that project at the moment? And if you are, do you have any industry collaborators that you're relying upon to help facilitate that?
Kavita: That's an ongoing project in my group. It started with Kevin Matzen and Noah Snavely. Noah is a professor at Cornell Tech. Kevin is now at Facebook. We also have Bharath Hariharan who is a faculty member at Cornell and Utkarsh Mall, who's our Ph.D. student.
When we first started, the project was called Street Style, and now it’s called GeoStyle. The core idea was to take lots of images. In this case, we got images that were public at that time on photo collection sites and tried to understand how people around the world dress. That also piqued the interest of cultural anthropologists we were working with. If you're a cultural anthropologist, the way you solve this problem right now is you do a field study, you go to different parts of the world, you interview 10, 20 people, you try to compile all of that into some knowledge.
When we showed up and said, "Here's an automated tool. You can sit at home and browse the dressing patterns of different parts of the world at different times," they were excited. We got Denise Green from our Fiber Science & Apparel Design department at Cornell to get involved, and we're working with her on understanding the cultural implications of fashion trends.
Abby: Can you see when it's wedding season in India for instance?
Kavita: We have a map of the most salient big events that jumped out during our research. There is Durga Puja, an annual Hindu festival in Calcutta where everybody's got their bright saris and gold jewelry on. We got Fashion Week in Paris. We found the Stanley Cup. Sport and fashion events popped right out of this analysis. And also political rallies - we found a big protest in Spain, where everybody wore yellow shirts and formed a human chain. It's been eye-opening because you learn about the human state across the planet by looking at the images they're posting.
Abby: I'm wondering if it would ultimately be applied to something like war and escalation. If all of a sudden there's a lot more military fatigues showing up in all of these images in certain areas of the world, you can see where something is happening from a wartime perspective.
Kavita: That's a great point. That is one use. The other thing we're trying this technology for is satellite imagery, and we're working with crop scientists to understand how the ecosystem has changed and what are the kinds of trends you're seeing there. All these amazing uses come from treating the vision piece as a tool like a telescope or a microscope – the tool lets me see. Can I now go beyond just seeing to learn something new?
Abby: Are you going to have the opportunity to continue your research? What are you most looking forward to in your new role?
Kavita Bala (Cornell University Professor & Dean of the Faculty of Computing and Information Science) and Serge Belongie (Cornell Tech Professor & Associate Dean, also LDV Capital Expert) tour the new Gates Building at Cornell University in Ithaca. 2013
Kavita: I hope so. As I said, research is a fundamental aspect of both my intellectual joy and engagement in my community.
I told you I was interested in law and social issues early on. Many interesting questions are coming up in the context of the societal implications of things. I'm the dean of Computing and Information Science. We have computer science, information science, data science stats and data science in the same unit. People in these groups are looking at technology and its societal implications. I'm excited to continue to foster that research, promote it and help it succeed.
Abby: It sounds like working with people and helping them facilitate their research as well as doing your own is a big part of why you're most excited about this. Do you see any opportunities for crossover in different sectors that you're looking at?
Kavita: To me, the most exciting types of projects that end up taking off are fundamentally interdisciplinary. Working with the domain expert is critical for the technology to have a massive impact.
Abby: There are so many opportunities to do interdisciplinary research at Cornell!
Kavita: I agree with that. I feel fortunate. It's a broad school, it covers everything, and the culture is to be collaborative. People talk to people from other areas all the time because you're in this town and you bump into people in the playground, in the coffee shops, and you naturally talk to people about broader problems than the ones you're looking at, which is what I love about being part of a university town.
Abby: What industries do you think are going to be reshaped by visual technologies in the coming years?
Kavita: Its impact is going to become more and more realized in robotics for home care and eldercare. We've been talking about personal robots for a long time, but for health and eldercare, that part is going to take off and it's going to be exciting.
Abby: Do you see that as being more like the co-bots side of things where they can be companions for people or do you see them more on the care side of things?
Kavita: I'm excited about the latter. I believe both of them will grow. It's a rising tide that'll raise all boats. I’m excited about them helping in place because that's naturally the way we need to go as a society. That's an incredible growth area.
Abby: What about the agricultural side of things? We recently released our third annual insights report where we deep dive into an industry that's ripe for disruption via visible technologies. This year, we researched the future of food and agriculture.
Kavita: I presumed that agriculture was a given in your list. There will be new kinds of robots. A lot of the existing industrial robots are now getting to the wire. We will be able to deal with messy situations that in the past have always been associated with a human being going in there and manually doing it. That's also right on the threshold, and hopefully, we'll get there.
In this five-year market analysis, we showcase how cameras, sensors, and other visual technologies will impact:
Controlled environment agriculture and vertical farms;
Plant breeding and phenotyping;
Precision agriculture for crop monitoring & input control;
Animal agriculture;
Meat processing automation;
Imaging for food safety improvements;
Future of grocery;
Waste reduction in restaurants and food service;
And more...
Abby: We think that perception and vision have been a game-changer for robotics because you don't need a specially designed warehouse for a robot to be able to come in and work alongside humans in a safe fashion. In agriculture, it doesn't seem to necessarily be moving in the same way at the moment. There still seems to be a lot of stationary robotics where the robots are built for the specific use case. Are you finding the same thing?
Kavita: You should talk to Kirstin Petersen, she is an Assistant Professor in the Electrical Engineering department at Cornell. She is working on building robots that can grab soft fruit and deal with all of the messiness if you want to get a ripe peach off the tree. That is a hard problem and every aspect of it is interesting. That's where a lot of the growth is.
Abby: Do you have any advice for other researchers or professors who are interested in commercializing their research? What can inspire them to finally take that first step out of academia and to build it into a business the way that you and Sean did with GrokStyle?
Kavita: It is a big transition. Pick a problem that has to be solved. That is one of the most rewarding things to do.
Typically, researchers have an incredible solution that they can bring to bear and enjoy every aspect of it but you also need to do everything when you're building a company ground up. You need to do the finances and the business plan. You need to think about technology and the product. You have to become a Jack of all trades. If you don't want to do every part of it, get two people. Co-found a company together where you have complementary skills, it's well worth it.
It's important to learn to enjoy the highs and lows.
Abby: My dad owns his business and one of the things that he said to my sisters and me when we were thinking about different opportunities, "Never start up a business unless you're willing to sweep the floors." That's one of the things that I always play over in my mind.
Kavita: That's right. This idea that I'm an expert in X so I'm going to only do X is not acceptable in a startup. You have to go do the thing that you're not an expert in. Figure it out. That's part of it. I found it exciting. You don't get into a narrow box. It forces you to expand your thinking.
Abby: I think I know the answer to this question already, but if computers didn't exist, what would your career choice have been?
Kavita: Wait! That's interesting. You think you know my answer so you should tell me!
Abby: I was like, "Oh, maybe she'd end up being a dancer."
Kavita: I might have also become a perception psychologist. I’ve always been interested in human perception. There's still the question of how we understand the world. It's open-ended and it will stay that way for a long time, that's what I like about it.
Abby: Thank you for taking the time to chat. It's been a pleasure learning about your journey as an entrepreneur, as an academic, a researcher, professor, Chair of your department, and now dean of Cornell in the Computer Science department. It's wonderful to talk, and we look forward to continuing our conversations in the future.
A video version of this interview is available below:
