CASE STUDY

How Udemy Learns From Their Students

“User research allows us to stay proactive and continue innovating with our users’ feedback in mind.”

Empowering a New Wave of Learning

Learning a new skill has never been easier. Once confined to a classroom or limited to those who have access to a mentor, instruction is now far more accessible thanks to the digital transformation. Leading the way with one of the largest online marketplaces and millions of students is Udemy. Founded in 2010, the platform has grown to offer over 80,000 courses taught by expert instructors globally.

Turning to User Feedback to Augment Traffic Attribution

Because of Udemy’s massive reach and popularity among students, bloggers and online personalities, the company has faced challenges in determining how a person actually lands on their site.

“A lot of people hear about Udemy from a friend, or see an ad, or are exposed to the company in some other way. Then they type ‘Udemy’ into the search bar,” says Claire Menke, Director of UX Research for Udemy. “On the surface, they look like an organic visitor, but in reality, the attribution belongs to another source.”

With so many of these visitors being attributed as “direct load,” Menke realizes that Udemy is missing out on understanding the influence of other forms of marketing on user acquisition. This information is key for any native digital business like Udemy, especially when it comes to allocating advertising budgets. To help determine other factors that influence a user to visit, Menke turns to Qualaroo to survey new students.

“We use Qualaroo to understand exposure and get a true attribution of awareness,” Menke explains. “We begin asking questions weeks prior to a new campaign which gives us a baseline understanding of where people come from. As we turn on different methods of advertising, we can see how those values change.”

Getting Students’ Feedback Through the Qualaroo Identity API

Qualaroo is more than a survey software—it is a robust technology stack backed by an expert customer success team to help you gather user feedback in the best way possible. One of its most powerful offerings is the Qualaroo Identity API, which allows companies to programmatically target users by ID or email address. This tool has helped Udemy zero in on the users they want to target.

“…We’re a global company and being able to target based on location is very important to us. Qualaroo is also effective at capturing people in that moment when you have a question and need to get a response.”

By tapping into your user databases or CRM, you have the power to segment your users in new and powerful ways. In fact, we have seen customers increase their conversion rates by as much as 23% when they segment their audiences in a similar way that Udemy does.

Machine Learning with a Little Human Help

Udemy recognized that students outside of English-speaking countries were enrolling in courses taught in English, even though that was not their native language. To further support these students and help them more easily comprehend, Udemy launched automatic captioning for their video courses in English. That feature based on data insights helped make the Udemy content more accessible to people around the world.

As many people know, however, the machine learning technology behind automated captioning is imperfect. To help improve the development of captioning technology, Udemy turned to its students for feedback.

“We’ve used Qualaroo to ask students to rate the quality of these machine-translated captions and provide suggestions for improvement,” Menke says. “From the responses, we’ve been able to make product optimizations that improve the caption editing capabilities for instructors.”

Turns out that even machine learning can benefit from human feedback.