Women Leading Visual Tech: Rana el Kaliouby on AI Reading Human Emotions

LDV Capital invests in people building businesses powered by visual technologies. We thrive on collaborating with deep tech teams leveraging computer vision, machine learning, and artificial intelligence to analyze visual data. We are the only venture capital firm with this thesis. We regularly host Vision events – check out when the next one is scheduled.

Our Women Leading Visual Tech series is here to showcase the leading women whose work in visual tech is reshaping business and society.

Rana el Kaliouby is an Egyptian-American scientist, entrepreneur, author, and AI thought leader on a mission to bring emotional intelligence to our digital world. She is the co-founder and CEO of Affectiva, an MIT Media Lab spinoff credited with creating the category of artificial emotional intelligence, or Emotion AI. Emotion AI is built on deep learning, computer vision, speech science, and massive amounts of real-world data.

Raising over $50M from top-tier investors, Dr. el Kaliouby has brought Affectiva’s Emotion AI to 90+ countries and several industries including the automotive industry and media analytics. She has a track record of translating technology innovations into products that address the needs of massive international markets and spearheads the application of Emotion AI to mental health, autism, conversational interfaces, robotics, and education.

Dr. el Kaliouby has been recognized on Fortune’s 40 Under 40 and Forbes America’s Top 50 Women in Tech. Dr. el Kaliouby is a World Economic Forum Young Global Leader. She holds a Ph.D. from the University of Cambridge and a Post Doctorate at MIT. She is also a limited partner of the MIT Media Lab E14 Fund, a Board Member of SIMPEDs at Boston Children’s Hospital, and a Board of Trustees member at the American University in Cairo.

Most recently, Dr. el Kaliouby published her memoir, Girl Decoded: A Scientist's Quest to Reclaim Our Humanity by Bringing Emotional Intelligence to Technology. It follows her journey, growing up in the Middle East and moving to the United States to become an entrepreneur, juxtaposed against her work building Emotion AI.

Rana was interviewed by LDV’s Abigail Hunter-Syed. (Note: After five years with LDV Capital, Abby decided to leave LDV to take a corporate role with fewer responsibilities that will allow her to have more time to focus on her young kids during these crazy times.)

*Rana el Kaliouby at Affectiva office in Boston, MA © Remi Carlioz*

The following is the shortened text version of the interview and the unedited video can be found below. You may want to watch her TED talk first – in just 10 minutes you’ll see a demo of a new powerful technology that reads human facial expressions and matches them to corresponding emotions. Another good idea is to start with a keynote “Emotions are Core to Humanity and Every Aspect of Our Lives” delivered by Nick Langeveld, former President and CEO of Affectiva, at our LDV Vision Summit in 2015.

Abby: Rana, it's great to connect! I enjoyed reading your book and can't wait to find out a little bit more from behind the scenes but here’s a question that we start all of our interviews with: how did you describe to your kids what you do at work when they were little?

Rana: My kids have always been my Guinea pigs. They've been testing the technology from very early on. I say “we're on a mission to build computers that can recognize human emotions” and I have them test our demos. They practice their different facial expressions and then I show them how the technology can detect a smile, a smirk, or a frown. And that inevitably leads to a discussion on, "mom, what are you using it for?".

Abby: I know that your father was a computer programmer. Your mother was one of the first female Egyptian computer programmers, too. You describe her in your book as playing a large role in ensuring that you received a top education. What part do you think that this played in for you deciding to pursue computer science? Do you think that that's going to play a major role in your kids deciding to pursue their future careers as well?

Rana: My parents have always been proponents of my two younger sisters’ and my education.

I think they have an implicit influence but they didn't force it. My daughter is 17 years old. She's at the age where she's deciding and thinking about college. She's decided that she's not a computer scientist. She's more in development and international relations. My son is 11 years old and he's super passionate about programming and robotics, so we'll see!

*Rana at Emotion AI Summit 2019 in Boston, MA © Iulia Nandrea-Miller*

Abby: Do you remember a specific point in your life when you decided that that's where you wanted to dedicate your studies to?

Rana: My dad used to expose us – young kids – to all sorts of technologies. In particular, I used to play Atari with my two sisters. “Space invaders” was my favorite game. I have these memories of all of us huddled around the video console and playing together. It wasn't the game that intrigued or fascinated me. It was bringing us all together. That’s when I realized technology is all about bringing humans together. And that's been a common thread in my research.

In the world we live in today, a lot of our communications are mediated through technology. My hypothesis is if that technology is emotion-blind, our communication is going to be emotion-blind, too. I'm on a mission to reverse that.

Abby: As I read through your book, it seemed like at multiple points throughout your career, whether that was the late-night hours in the university being scrutinized by your neighbors or deciding to start a company at six months pregnant, there were many different junctures where your success as a scientist was challenged because of your gender. Where do you think that you draw the courage to overcome these challenges and stick to your passion for pursuing this human-computer interaction and emotion-based computing?

Rana: I think about my younger self as being a nice Egyptian girl so I'm not the rebellious type of person. I would say I got my courage and my ammo from a deep conviction that I can do something about this, that we need to humanize technology and there's much potential for Emotion AI in mental health, automotive, and many other applications.

Abby: What's the most common question that you get when people hear that you're building emotion into computing?

Rana: The first question that always comes to people's mind is “why” – Why do computers need emotions? My argument is that, if you look at human intelligence, your IQ matters, but your EQ often matters even more. And people who have higher EQs, they're more persuasive, they're more likable, they're more effective in their personal and professional lives. I feel that that's equally true for technology that we interact with on a day-to-day basis. If you think about AI, it's becoming mainstream and it's taking on roles that were traditionally done by humans. These technologies need to understand humans. They need to know how people behave and how people feel to be effective.

Abby: Emotion drives so much of who we are as people that if you're missing the emotional side of things, you miss a lot of the “why”. Here at LDV, we believe that until recently computers have been functionally blind as well but now the ubiquity of cameras on our phones or computers has opened up this world of opportunities for computers to see.

Rana: We are already starting to see interfaces become conversational but the real future lies in actual perception in our technologies. LDV and your portfolio companies are at the forefront of bringing that to fruition. I love that! I've always been a fan of your work and a follower of what you support.

Abby: Thank you!

After almost two decades working on some aspects of Emotion AI, in your opinion, how far are we from the time when cameras might use computer vision to detect human emotions? Will AI ever be able to tell what's driving us?

Rana: I would say we're in the early days. AI is like a toddler in terms of the sophistication of understanding human emotions. There's a lot more to do but we have the recipe for how to do it. It's a ton of diverse data from all over the world combined with state-of-the-art deep learning. We've been able to show that we're able to use that pipeline, and repeat it over and over for different types of emotional and facial expressions.

Abby: I love the analogy that computers are like toddlers at the moment! Being the mom of a toddler, I think one of the things that you do is... you work on those different faces. When you see a frowny face, you explain this means somebody's sad. A happy face means somebody is overjoyed. You try to assign those emotions to it so that she can put those together in her head. It makes complete sense that that's the way that we're going about training computers to understand.

Rana: Exactly! You're probably not teaching your daughter what pride, jealousy or inspiration look like yet. These are advanced.

Abby: How do you even do that?

Rana: We don't know exactly what those look like because they're more complex. I guess we learn them as we grow more mature.

*AFP 2019 Mindshift Keynote in Boston, MA © SY Photography*

Abby: One of the things that you talked about in “Girl Decoded '' is the Baron-Cohen spectrum and that there's the empathizer side of it and that more women fall on the empathizer end of the Baron-Cohen spectrum. Does it make perfect sense that two women, you and Dr. Rosalind Picard, are the driving force behind the Emotion AI?

Rana: I think it does. Simon Baron-Cohen is very clear in his research that it's not exclusive. Of course, some men are more empathizers and some women are more systemizers but I don't think it's a coincidence that Ros Picard, my co-founder, and MIT professor, started this field of research and that a lot of women are drawn to it.

But I will say even at Affectiva – our female-founded tech company – we still struggle to get more females into our pipeline. We have a lot of male engineers and scientists who are passionate about this mission and they've joined the company because they believe in what we do.

Abby: What do you think are the most exciting applications of your technology today? You mentioned automotive and mental health but what's the number one that you're looking forward to for the future?

Rana: Right now we're focused on the automotive industry to increase our road safety. We use cameras to detect if drivers are falling asleep, or if they're distracted, for example, texting while driving or on your phone. We use computer vision to detect not just the driver's state but what other objects the driver is interacting with. I think that can be transformative and save a lot of lives.

*The application of Emotion AI in the automotive industry © Affectiva*

Beyond that, I'm passionate about mental health because we know that there are facial and vocal biomarkers for depression, stress, and anxiety. Given that we're all stuck in front of our computers a lot of time, that's an opportunity to capture people's baseline and flag if somebody's showing early signs of depression or even suicide.

Abby: The moment you open up your computer in the morning and you have your daily scrum with your team, you can automatically sense what's going on with them or if something's happening. What if the computers could do that over the day and measure, when you are most productive, what makes you happy or unhappy, etc.

Rana: I think because of this pandemic, there's going to be a layer of innovation that sets on video conferencing platforms like Zoom and Teams that can quantify and bring objective data to meetings or presentations. For example, imagine if we can quantify pitches. As a founder, I would love to know, when investors perk up or when they're smirking and they're not persuaded. I think there's a lot of valuable data and insights there.

I've been doing a lot of virtual book events and I miss seeing the audience. It's tough when you can't see who you are talking to. I think Emotion AI can be integrated into these video and live streaming platforms in creative ways to create a sense of shared experience. I'm excited to see what comes out of this.

Abby: I would love to see that! Going back to pitches... When you and Dr. Picard founded Affectiva, what do you think was the greatest challenge that you encountered?

Rana: There were a couple of challenges. We were two women coming out of academia, we didn’t have a lot of startup experience. At the time I used to wear a hijab, I was Muslim, and we were pitching an ‘emotion company’. If you look at our early pitches, we never used the word ‘emotion’. We referenced it as the ‘E word’ and we always used a synonym for it. That's why we called the company ‘Affectiva’ – a play on ‘affect’. I think the fact that we were so different than what these predominantly male investors were used to seeing made it hard. I wonder if there were more female founders and female investors in the room there would have been more resonance.

Abby: It's what people have been saying for a while now… that in the beginning you never pitched to a woman or a person of color in the VC world. That's incredible to me just because I have such a wonderful network of women in VC that I work with daily.

Rana: I feel like it's changing. I'm part of an organization called All Raise and we support female founders and female funders. It's a great way to create an ecosystem of female founders and funders, and I'm passionate about it. I'm heavily involved with All Raise Boston.

Abby: I think one of the big things that stood out also is that you guys had a great network coming out of MIT, of advisors and people who put you on the right path to things. Ultimately you end up saying that Dave Berman, who you brought in to be the CEO of Affectiva helps make you into a startup. Would you recommend other researchers who are looking to commercialize their technology for the first time to search for a cofounder that has experience in building a successful business?

Rana: That's a good question. I think so. Dave was amazing because he brought a lot of experience growing a company. He was one of the very early people to join WebEx and by the time he left, he was one of the top three executives at the company. I'm forever grateful for that expertise.

It’s hard to tell what’s the right or wrong answer but the question is whether it makes sense for the founder to be the CEO, even if he or she doesn't have the business experience. In this case, the founder can partner with a COO who has a lot of experience. I think that equation makes more sense but it wasn't the path we took.

Abby: I think coupling somebody who loves business operations and sales with somebody who has a strong technical background, research, drive, and passion for the technology that you're creating, is crucial. We've seen it to be a sweet spot if you can marry those two together.

Rana: I agree. My current COO is also an MIT alum and very experienced in the startup industry in Boston. We have an amazing partnership because I'm visionary and strategic, and he's the nuts and bolts guy. He's detail-oriented and makes sure that all these ideas are translated into actions.

*Sitecore Symposium in Orlando, FL © Billy Cole*

Abby: What advice do you have for women who are researchers to inspire them to take the risk and commercialize their research as you did with Affectiva?

Rana: I think there's something magical about starting a company. For me, it was taking something I'm passionate about and making sure that it's deployed globally at scale, and that it changes the way people go about their daily lives and their daily business. For me, that's always been the drive. That's hard to do within academia, right? If you're passionate about a cause or a technology, and you're an expert in it, then you should by all means go for it. It's an emotional roller coaster of a ride but it's worth it.

Abby: That's fantastic. Getting back to “Girl Decoded”, you started the book with a harrowing story from 2017 of a disabled man who was drowning while teenagers stood by filming it and laughing. And as we've seen recently, there are still incredibly tragic human interactions that are being caught on camera. But now at least they are eliciting some strong emotional responses and protests that will hopefully lead us to meaningful reform. Do these emotional responses give you hope that human-computer interaction is effectively being changed?

Rana: These interactions and these examples give me hope. Emotions are catalysts for action. Emotions of anger, sadness, or grief can act as catalysts for change. I'm excited about that, we should cultivate and encourage that.

I also think we're at a moment where we need to rediscover and celebrate empathy. We're all going through this pandemic together. I think there's a realization that there are systemic injustices in our world and now is the moment where we can all come together and be agents of change. I think technology is going to play a huge part in the solution. It already is.

Abby: I'm with you. I hope that these types of events and emotions are catalyzing all of us to be agents of change in the ways how we can.

One last question for you before I let you go: what is the best piece of advice that you've ever received?

Rana: The best piece of advice I've ever received is about the way to combat inner doubt. When I wanted to become the CEO of Affectiva, a mentor of mine said "Visualize it. Imagine walking into the office as the CEO, and do that over and over again." I had internalized that that was possible and I was able to come up with an action plan that made it real.

Abby: That's brilliant! Thank you for taking the time to chat and to spread a lot of interesting points about Emotion AI, and how human-computer interaction that's going to change over the years to come with the help of the fantastic technology that you've developed and the company that you've got, Affectiva.

Rana: Thank you for having me.

A video version of this interview is available below:

We can’t wait to introduce our next guest! Subscribe to our newsletter to stay connected to a community that is just as passionate as you are about visual technologies.

LDV Blog