Skillshare - Checkout - Overview
A/B Experiments leading to increased conversion
While I was the Lead Designer on the Growth team, I was responsible for optimizing the checkout experience. This was one of the biggest opportunities to establish trust in our product for our new users. It was also a big business opportunity as better conversion on this page led to increased revenue. Over the course of 1.5 years, we conducted many A/B experiments & 2 major UI improvements, all while adding new payment method options, preparing for internationalization, & staying up to date with changing regulations. This is the story of how we went from a visually crowded page with little room to grow, to an optimized experience that resulted in a significant win for our users & for the company.
The old experience (left) and the new experience (right)
The Team
Designer, 3 Product Managers, 1 Data Analyst, & 3 primary engineers.
What I Did
Created & conducted User Research studies,
Collaborated on strategy & A/B experiment design
Created all design iterations
Led meetings with the C-suite & collaborated with the marketing team.
The Impact
The new design resulted in a
+12% increase in revenue,
+4.77% increase in conversion.
On one of our highest trafficked pages, these increases were significant wins for our users & for the company.
The Problems
We knew from our data analysis that the conversion rate of the checkout flow could be improved. We also heard from our customer support team that users were canceling their payments & requesting refunds due to confusion about what they signed up for. From further analysis, we also saw an opportunity to optimize the experience for mobile web & we knew we had upcoming compliance requirements that were going to be difficult to fit in the old design.
The Solution
Carefully iterate through experimentation to improve the conversion rate, provide clarity about the membership for our users, & adjust the design to be more flexible for future changes.
The Process
We knew the design needed to be optimized for new branding & we wanted to take the opportunity to address the low conversion rate and user confusion feedback. Here is a broad overview of the work we did to get to the new design:
Membership Plan Experiments
The first thing we did was experiment with different membership plan options. We started with 2 plans and ended with a 3-tier plan that benefited the business. We knew the previous design was not built for a 3-tier plan, so I gathered the user feedback from our customer support team and conducted a competitive analysis.
Blue Sky Design proposal
From the research, I created a future-thinking design proposal to show how we could present the 3-tier plan information in a more user-friendly way. As a backup, I also created a proposal to run iterative experiments that could get us to the Blue Sky designs in a controlled way. I presented both proposals to the C-suite. They were not comfortable with the blue sky designs, but they approved the iterative experiments proposal.
Validating with User Research
With the C-suite on board, I got approval to run a user research study to validate the first experiment. I showed the users a new UI design prototype for the plan information. The most significant response was that the membership plan options were confusing & they felt discomfort around choosing a payment method with the old design. They overall preferred the new design. This helped us to feel confident that the new designs would be a success, so we ran an A/B experiment.
The control design (left) and the new design being tested (right)
Spoiler Alert: It didn’t win
This A/B experiment was a valuable failure. The new designs resulted in a -15% decrease in users selecting the 3rd plan option. However, we also saw a big decrease in users asking for refunds (down -37%). This signaled we made the 3rd plan option more clear to our users, but they didn’t select it as much now that they understood it.
The decreases were too significant for us to accept at that time, but the results of this experiment combined with another round of user interviews helped us to reconsider our entire membership plan model a few months later.
More experiments in the safe zone
Knowing our business needed a win, we put the larger design changes on hold in favor of making smaller changes that we had more confidence in. We
added in 2 new payment method options,
consolidated our CC fields to Stripe single card elements,
reduced the information on the page,
moved the total box with the CTA higher up on the page.
Finally, the biggest change we made was to change the membership plan model to a single tier plan.
The next big thing
All that was left now was to update the components to our current design system, and make sure the information was clear. While the smaller experiments were running, I continued to work on new design concepts in the hopes that we could take another big swing one day. Finally, that day came and we got stakeholder buy-in to run another UI design update to checkout. I shared my reasonings for each of the changes with the C-suite and we got approval to run it as an A/B test.
Final Design
The biggest problem to solve in this new design was making the design flexible enough to accommodate all of the 24+ variations and have space for the regulations, payment method changes, & internationalization we had upcoming on our roadmap. I also wanted to target the concerns from our users by making the membership plan information clear & making it easier to select a payment method with confidence. This new design also unblocked future experimentation capabilities like an enhanced discount code experience.
The final designs on desktop & mobile web.
Unified background
The previous design used 2 colors to separate the action from the information on the page. Now that the information was streamlined, I unified the background color to create a sense of focus and unity to the content, signaling that all information on this page was important for the user to process in order to complete the transaction.
New header & components
The previous design didn’t include instruction for what action the user should take first. In this new design I added the header “Select a payment method” to give that instruction & to accommodate the reassuring message tooltip explaining why they need to enter payment information.
New Total Box
The biggest leap in the new design was an updated total box design. This design may look simple, but it was designed to be optimized for the 35+ variations that could occur. It also provided a space for clarity on what the user could expect to pay after their free trial if discounts were applied.
Outcome
The new design was a success. We saw a
+12% increase in revenue,
+4.77% increase in conversion.
On one of our highest trafficked pages, these increases were significant wins for our users & for the company. The new design was more in line with our design system & had less dynamic text which made it easier for engineers to iterate on further. It was also flexible enough to accommodate the changes we had upcoming on the roadmap and fit our internationalization efforts better.
Learnings & Takeaways
This page is deceptively complex. Every time an experiment was run, I designed for and helped QA the 24-30 possible variations of this flow. I created & updated expansive copy docs of all the dynamic text options, heavy Figma files of the visual variations, & developed 4 guiding principles based on the learnings from our experiments. By the end, I was nicknamed “the checkout expert,” and gained valuable experience in documenting complex user flow variations.
