GO-JEK is a multi-service platform startup based in Indonesia. Founded in 2010, it started life as a ride-hailing call centre, allowing users to hire motorbike instead of a car; a necessity in a city like Jakarta where car drivers spent an average of 63 hours a year in traffic (source). In 2015, it released its mobile application, and soon expanded its service into food delivery, logistic, purchasing service, beauty service, and many more. At the time of writing, GO-JEK transport app alone serves more than 2 millions unique users daily.
I joined GO-JEK in July 2016 and became a part of the transport team in October 2016. On February 2017, I spearheaded and launched GO-JEK transport redesign. You can read the process here.
BACKGROUND
One of my daily routine after the end of GO-JEK redesign is to monitor app reviews from Playstore and Appstore, as well as various social media. It was one way I am able to listen to our users, other than regular user research and reading customer service report.
Of course, I understand that what was written in the app reviews should be read with a grain of salt. Regularly I saw what I deduced to be messages written by drivers instead of customers, as well as confused competitors’ users complained to us about our competitor’s feature or policy. What I use app reviews for is an indication of what is potentially a problem, then check it against our analytics whenever possible.
A few weeks after we fully roll out, and the number of users adoption climbed, I started to notice the rise in complaints on AppStore and Playstore about not being able to cancel after they were assigned a driver. Now, most reviews from these channels are generally vague at best. For transport product, not being able to cancel can mean many things from backend problems to network issues, to usability issues.
I checked with our engineering team to find out if there were issues with our backend or network in the date range that the complain happened. There were none. However, corresponding rise can be seen on the ticket to request for booking cancellation through our Customer Care Unit (CCU). It was interesting to note that the booking cancellation metrics itself, even when combined with the numbers from CCU tickets, seemed to have drop compare to the previous version of our app.
Since there was no clear answer, I work with my researcher to conduct a usability testings to see which of the usability issues causing these complaints. The goal of which is to pinpoint which element confuse users.
WHY CAN’T USERS FIND THE CANCEL BUTTON?
Because of the nature of our product, I was generally more in favour of guerilla testing. It enabled us to see how user would use our product in their environment, and therefore let us see potential distraction that might compete with our users’ attention to our product (e.g., the screen reflection on a hot tropical day made our users unable to read well if we are using dark background).
For this particular case, however, I requested to my researcher to contact users that called our customer service and requested for them to cancel. Through previous experience with this company, I knew that it will be tough to get our result accepted if we use less than 15 people per research. So that’s the number that me and my researcher chose.
I do have a researcher helping me with execution of the testing itself, although, I always made it a principle to also go with him/her whenever possible and observed our users. We also made both video and voice recording of each sessions so that every member of the team can review them whenever needed. We asked users several questions:
What information did they see in this page
What function did they think should be on this page, and
Use case test where we present them with a story and end with them having to cancel their ride
The result was pretty clear: users were able to discern almost all of the function that was on the default state of the screen except one; they don’t know that the card can be expanded.
SOLUTION
On the discussion after the report was presented, I mentioned to my PMs that I planned to bring the card up 25px; just enough to let something peak through that users know that there was something underneath. I did a quick test to 5 users to see if it would work, and it did.
There were, however, hesitations from the PMs (and CTO and CEO) to the idea of fixing this. In their opinion, if the cancel button was easy to discover, users will be much more encouraged to cancel their booking if there was even slight discomfort. I mentioned to them that in GO-JEK v2.0 (the one prior to redesign), we always have cancellation button clearly define in the page. Incidentally, I had also read a case study, where presented with no withdrawal option, users from another fintech company churned much more compare to when a withdrawal option was present. It’s hard not to wonder if the same case could be applied to our situation as well. Luckily, they are curious bunch as well, and gave the green light for this testing to go through #blessed
We agreed to do A/B testings, where 5000 users will be control, and the other 5000 will get the raised notched card. We will then monitor two metrics:
CCU tickets from them regarding cancellation
# of them cancelling an order
Success was defined if the number of CCU tickets regarding cancellation returns to normal, with a caveat that cancellations number did not spike.
I also made sure that the users are all coming from Jakarta; we had noticed that users from different cities behave differently, so it felt best to eliminate variables as much as possible.
After three weeks, the results were in. Users with notched card shown to be less likely to cancel, as well as having less number of CCU tickets. Yay!
The new version deployed on the next release, and just like the test, we saw decreased in numbers of both CCU and cancellation by users.
LEARNINGS
Although I had taken inspirations from applications that were not in the same stream as my project at the time, it was the first time I applied psychological learnings from another case studies to my project. This, in turn, inspires me to not right away dismiss any learnings that other people might have had, even when they are on a different domain.
Anecdotes are always worth checking. While metrics are a great way to measure success, however, it also tends to not tell the entire stories. The numbers on a metric could go up and down for multiple reasons, and to separate them are not easy. So it was always a good idea to keep our ears to the ground, listening for users feedback and reaction, and try to discern what is going on.
Comments