User Research: Balancing Interviews and Yelp Reviews

Introduction

If you have no idea what user research is, you are definitely not alone. User Research focuses on understanding user behaviors, needs, and motivations through observation techniques, task analysis, and other feedback methodologies. Coupled with user interviews and surveys, social media is one of these many methodologies that companies use to iterate their product and ensure customer satisfaction.

Campus Insights

For the last 8 months, I’ve been working for a startup that specializes in UX Research. Campus Insights helps companies that are targeting college students get feedback on their website or app so that they can build awesome products. We are able to get this feedback for companies by conducting user testing, which usually consist of remote interviews with students from all over the country. During these interviews, we will usually have students download the app we are testing, and then walk through the onboarding process with them, analyzing both their face and their phone screen to deduce confusing parts of the interface flow. We’ll also look for conceptualization errors, making sure the user can understand why they would use the app.

Screen Shot 2016-09-18 at 4.08.07 PM.png

While Campus Insights is a resource for companies to outsource their user testing, many companies actually conduct user research in-house. I worked on the User Research team at Rent the Runway this past summer, and all of their interviews and other methods of user research were conducted by a Senior User Researcher. While RTR has hired a specific person for user testing, many companies do not, and that’s where Campus Insights becomes very valuable for a lot of companies.

Rent the Runway’s Voice of the Customer

Research that is done in-house is a bit different from what user testing with Campus Insights looks like. In the case of Rent the Runway, most of our interviews were held in the New York office, so all of our interviewees were from NYC. The interviewee was questioned by the user researcher as he/she navigated through the RTR app or website, and the interviewer would have cameras on both the interviewee’s face and phone, so that the Product Team could watch the interviews later. Sometimes the Research Team would travel out into the city to customer’s apartments and go through their closets with them, trying to better understand their fashion needs. These off-site interviews were particularly helpful in getting a better idea of who our users were and what they wanted from Rent the Runway.

Screen Shot 2016-09-18 at 4.32.16 PM.png

While interviews are a huge part of User Research within a company, it’s not the only method of getting feedback from users. Rent the Runway utilizes several different channels of feedback, including surveys, satisfaction ratings, and of course, social media. All of these channels provide a different type of feedback for the company, which is all presented in a “Voice of the Customer” biweekly report to the entire company. As an intern, I was delegated to collect feedback from a few of these channels on a weekly basis, with one of my largest tasks being monitoring RTR’s social media presence.

RTR on Social Media

Rent the Runway has a presence on almost every social media site. However, just as we use Facebook differently than we use Yelp, users post different types of feedback based on what social media platform they are using. RTR’s Facebook presence consists of two channels: the actual public Facebook company page, and a private Facebook group composed of RTR’s most loyal customers. The public Facebook page often has postings from customers about their orders, which contains both pictures of happy women in their dresses, and also some unhappy women with wrinkly and dirty dresses. Overall, the Facebook page is a mix of product reviews that are made public to anyone who likes the RTR page. The private group, while still used to gain feedback on the product, contains much different information. Since the group is composed of loyal and frequent users, many of the postings have to do with promotions or new features, rather than direct feedback or complaints about the product. The users in the group are so familiar with the product that most of the feedback ends up being deeper and more concerned with the future features RTR might consider incorporating.

Screen Shot 2016-09-18 at 4.27.23 PM.png

Yelp is also a huge outlet for feedback. Though most people are familiar with RTR’s option to rent via USPS, many users also utilize the brick and mortar stores that are now in New York City, Washington D.C., Las Vegas, and Chicago. These retail locations sound like a great idea in theory, but because of RTR’s value proposition to provide a huge selection of styles and sizes, the Yelp feedback from the brick and mortar stores tends to be very negative. Many Yelp reviewers complain of long lines, too few sizes, and overall lack of organization within the stores. Rent the Runway prides itself on convenience, and thus far the feedback on Yelp communicates that the exact opposite is true of the in-store experience.

Screen Shot 2016-09-18 at 4.07.32 PM.png

User Research & Social Media: Takeaways

Social media as a channel for feedback has brought up some major takeaways for user researchers:
  • A user’s experience is everything. If you’re part of a large company like Rent the Runway, you should expect that disgruntled users are prone to posting on social media about their experience, exposing your company to not only their network, but also anyone who follows the company’s social media page. If you’re part of a company with a terrible onboarding process, you should expect extremely high drop-off rates and very minimal growth. Worse than all of this, if you don’t have members of the team making sure users like your product, you will probably be out of business very soon.
  • Mistakes are out there for the whole world to see, and so is the company’s response. In the case of Rent the Runway, the majority of new renters hear about RTR from friends and family. That is a huge dependance on customer satisfaction – if customers are unhappy with their experience, they won’t refer their friends and family. Studies show that a satisfied customer will tell 2-3 people about his experience with your company. A dissatisfied consumer will share their lament with 8-10 people and some will push that number to twenty. Social media can be a disastrous tool for unhappy customers, but only if the company choses not to respond appropriately.
  • The voice of the customer is the greatest tool for company success. Companies that continually listen and research their customer tend to do very well. If user researchers go to a customer’s house, understand what’s in their closet and how they get dressed in the morning, it’s very likely they will understand the customer’s pain points, and therefore make a great product. User research bridges the product and the customer. Without understanding the customer, there is no way a company can make a viable product.
My experience in user research, both through Rent the Runway and Campus Insights, has made it very clear that feedback on the product is necessary for the success of a company. No matter the size of a company, social media poses as a vital channel for customer feedback that is needed for successful iteration on products.

10 comments

  1. francoismba · ·

    Very interesting post. The CPG company I interned at this summer also used user research to obtain feedback regarding products and to study consumers’ purchasing behaviors. Although companies gain insights from this reasearch, the methods can be obtrussive to the consumer. As a result, particpants of the studies unknowingly provide untruthful responses. Do you think there are better ways in which to collect feedback and study consumers without altering his/her behavior?

  2. francoismba · ·

    Very interesting post. The CPG company I interned at this summer also used user research to obtain feedback regarding products and to study consumers’ purchasing behaviors. Although companies gain insights from this research, the methods can be obtrussive to the consumer. As a result, particpants of the studies unknowingly provide untruthful responses. Do you think there are better ways in which to collect feedback and study consumers without altering his/her behavior?

    1. Really interesting to hear your experience from working in a CPG company. I think that when consumers voluntarily participate in studies and are compensated for their time, it’s very likely that the insights from the interviews will be valuable.

  3. I would think that based on the statistic you mentioned about unhappy customers complaining to 8-10 people over the happy customer sharing their experience to 2-3 people means there would be more negative comments and posts about any given product. Did you find this was true? On another note, how is RTR improving their brick and mortar stores based off the feedback? It seems like a difficult task to mirror a websites capabilities in a physical place.

    1. Great questions! There were definitely more negative comments posted on social media, but I wouldn’t say they drastically outweighed the positive feedback. Many users were excited to share their “Cinderella moment” with their friends on social media and often attached pictures of themselves in the RTR gowns.

      It’s definitely a difficult task to try to improve the in-store experience. A lot of different factors go into the experience, but I think RTR is doing a great job of acting on the feedback they’ve received.

  4. Interesting post. I’m intrigued to see the difference between online and offline customer experience. You’d think the customer would like to see the products in person, but I can see how those are two entirely different business competencies. Will be interesting to see if companies like Amazon experience a similar fate as they attempt to move offline

  5. I think the consumer insights created from in-house research is interesting, especially in RTR’s case. Their value proposition is to provide customers with a wide range of clothing options, yet logistics like shipping and sizing that are necessary to this purchasing process, are what seem to negatively impact the consumer’s experience. It almost seems like when RTR misses, they miss big. But when they get it right, they are really successful and enhance consumer loyalty. I agree with your second to last point, that the company’s response is crucial for addressing any negative experiences and mitigating any negative word of mouth. I also can’t help but wonder how they organize their teams, dividing talent between logistics, PR, and customer satisfaction?

  6. alinacasari · ·

    Awesome post!! I agree that social media is a vital tool for customer feedback. I think with the different channels that RTR has on Facebook by having the distinctions between the public group and the private group with more active members are a great way to collect customer feedback. By having both you are probably getting an entire range of information rather than just the public users who would have more complaints. I never really considered how different groups could have such different feedback. I do think a huge positive is that social media allows customers to openly complain or praise products/services much more easily and freely than in the past.

    I am curious though if companies are receiving more positive or negative feedback depending on the social media platform? I would 100% post an Instagram and tag a brand in a positive way, but never negative. I know I am much more inclined to do nothing or in extreme cases call or email and complain if I have a negative experience- because I wouldn’t want my complaint to be public. However, I would say I’m more likely to post a good review online because in that instance I don’t feel like I risk offending anyone. By having so many social media channels where users can reflect on their experience as a customer I think it opens room for a lot more positive feedback (apart from yelp). I’m curious how much new information companies such as RTR have been able to gather in recent years by their presence on different social media platforms. While negative feedback is helpful, I think positive feedback is just as valuable as companies can learn more about their strengths and what to keep doing.

  7. cmackeenbc · ·

    Cool post–it’s nice to hear from someone who has had firsthand experience exploring the relationship between the company and the online customer. Google’s executive team has recently made some great points about how poor quality products and services cannot survive in the modern world solely because of the availability of these consumer product reviews. I think it sounds like RTR is utilizing this consumer feedback to better themselves, for the most part. However, I do not totally see the connection between the RTR brand and a brick-and-mortar store, though I understand that customers who need a dress ASAP would benefit from this outlet. I think the reputation of variety and high brand quality set by RTR’s online platform likely hinders the customer in-store experience, as their expectations are very high (as you touched upon). I wonder how this would be different had RTR moved in the opposite direction, from brick-and-mortar to web. I also wonder if Amazon will run into similar dissatisfaction as they are supposedly opening a few brick-and-mortar stores soon. Nice job, looking forward to hearing more from you soon!

  8. Tyler O'Neill · ·

    I really enjoyed your post this week! It’s interesting to hear about your experience consulting for other companies as well as working internally with Rent The Runway. It seems like in both cases you are looking at a somewhat homogenous group (ie. BC students at BC, New Yorkers for RTR). Do you compile data from multiple sources in different locations in order to capture a more diverse group of people in your user experience data? If you are targeting a national market, it seems like data from multiple cities would be more desirable in hearing a broader spectrum of customer experiences.

%d bloggers like this: