Use Analytics to inform UX design

Your client is an e-commerce platform that sells branded merchandise. They’ve had a good amount of success despite never having looked at their Google Analytics. They’re launching a new campaign on their website soon, and have called you in to help teach them about how their website is performing, and make recommendations. The new campaign will sell a new branded product to raise money for a charity they genuinely believe in.

Project Information

Project type: Web Analysis(OpenClassrooms Project), Wireframing, Hi&Lo Fidelity mock-ups and rapid prototyping
Project year: Feb, 2023
Skills: Create infographics and data visualization, Apply metrics and quantitative data to product development, Use and interpret Google Analytics
Role: UX Designer
Goal: Propose the new campaign which will sell a new branded product to raise money for charity, Improve user experience of the client's site.
Tools; Google Analytics, Google Spredsheet, Optimal Workshop, Pen&Paper, Figma

Part1. Introduction (Client Information)

Google Merchandise Store

Our client was Google Merchandise Store,
which is an e-commerce site selling Google-branded such as Clothes ,  Lifestyle goods and Stationary, Supporting both Universal Analytics and Google Analytics 4 properties

Part1. Research by Google Analytics

1-1. Audience from last 28 days (Dec19,2022 - Jan 15, 2023) Trend from comparison of the privious year

Overview

  • Number of Users: 45,395 (New users are 40,037(88.2%) ) ※vs Previous year +2.25%
  • Language: Most people use English　※vs Previous year en-us +1.79%
  • Country,City: Most people live in English-speaking country, especially U.S
    ※vs Previous year United State +4.10%
  • Browser, types of Device:Most people use Chrome. 61.8% use desktop, 38.4% use Mobile Device
    ※vs Previous year Desktop -0.15%, Chrome +2.73%
  • System Operating, Mobile System Operating: Many people use Windows,Mac, Android, iOS
    ※vs Previous year Windows +5.05%

Language

The site is English-language only, but English is the language of about 90% of users, compared to about 10% of non-native English users (Chinese, Spanish, French, etc.).
Compared to the previous year, the percentage of Chinese language users has increased significantly.

Country

The users mainly browse the site from United States, India, and Canada.
We can also see an large increase in the trend in countries as Canada, Japan, South Korea and China.

Device & Browser

User mainly use Desktop.
Desktops account for about 90% of Conversion transactions.The site is mainly seen by Chrome.The percentage is about 75%. It is followed by Safari, Edge, Firefox, and Android Webview.Compared to the previous year, Edge and Android Webview showed a significant increase.

Operation System

User mainly use Windows, Macintosh, and Android.
Compared to the previous year, the percentages of these OSs have not changed significantly.

1-2. Product Performance Trend from comparison of the previous year

Overview

  • The performance over a one-month period shows that lifestyle goods and stationeryare selling well.
  • Apparel is mainly purchased for seasonal items except for the eco T-shirt
  • “Google Eco Tee Black” and “Google Black Eco Zip Hoodie” is the main product on this site.

last 28 days (Dec 19, 2022 - Jan 15,2023) vs. Previous year

  • The performance over a one-month period shows that lifestyle goods and stationery are selling well.
  • Apparel is mainly purchased for seasonal items
  • Three of the top five items on the list are not from the previous year; new items may sell well over a one-month period.

3 months (Oct 19, 2022 - Jan 15,2023) vs. Previous year

  • In the winter season, apparel products are the highest three month earners.
  • Four of the top five items are year-over-year. In particular, the Google Black Eco Zip Hoodie shows significant growth.
  • Apparel is primarily purchased for seasonal items, with the exception of the Google Eco T-shirt.

Trends in the previous year for the next 3 months(Jan 1- March 31,2021)

T-shirts and hoodies sold well last year.

1-3. Metrics to be captured (Using Google’s HEART framework)  From last 7 days ( Jan 8 - Jan 15)

In this section, we will use Google's HEART framework to define and research metrics that should be captured to improve the user experience.

Overview

  • User Satisfaction, Engagement, Adoption, Retention, and Task Success, with the exception of Engagement, which is below the general site.
  • Usability testing and interviews are needed to further investigate user Happiness( Satisfaction) and Task Success.

Examine which metrics to capture(using HEART framework)

This framework has a vertical axis with categories for measuring UX.The horizontal axis shows the goals, signals, and metrics that should be captured.
We want our clients to check these metrics in order to quantitatively understand the UX of their site.Some of the Hapiness and Task Success items should be checked with Usability Testing and User Interviews.

Hapiness

Goal : Users find the website easy, helpful & fun to useUsers find the product valuable
Signals : Leaving a rating & reviewSharing feedback on surveys
Metrics : Transaction Per User, Net Promoter Score, 5 star review, Feedback rating
  • Transaction Per User(Jan8 - Jan 15)334 Transactions / 15,165 Users = 0.022 Transaction / User
    (General EC site: 0.11)
  • Net Promoter Score (*Based on Survey, NPS = -100%)
  • Perceived ease of use (*Based on Survey, it was bad)
Additional research on usability testing is needed, but the transactions per user is very low and users may not be satisfied with the product or thewebsite

Engagement

Goal : Users enjoy the site and the product & keep engaging with them
Signals : Time spent on the website increases
Metrics : Session (visits) per user, Session time per user
  • Session(visits) Per User
    18,663 session / 15,165 User=1.23 Session / User
    *Although lower than the general website, the 3 month average is the average for the general site.
    The average is around 1.4 sessions/user based on a 2022 report from Littledata survey
  • Session time Per User3:13 / User
    *During recent 3 month: average 3:28
  • Conversion rate 1.79 %
    Although lower than the general website, the 3 month average is the average.
    EC conversion rate in general apparels is 2.5%E C conversion rate in General apparels is 2.5%

Adoption

Lower than general

Goal : Good reviews of the product and UX will lead new users to adopt, New users derive value in the product
Signals : Increased purchasing, Increased sign ups, Increased logins
Metrics : New Users conversion rate, New Users %, DAU (Daily Active User)
  • New Users Conversion rate116 Transactions / 12,772 New visitors=0.91 %(General website : 1-3%)
  • New Users % 77.1%
  • DAU = 1,638 Users
High percentage of new users,  low conversions ⇒ Insufficient Factors (User Interview is needed)
  • Improve the structure of the website
  • Reduce user stress
  • Create pages that are attractive to users
  • Attract the right users for the website's purpose

Retention

Conversion rate is good, but the percentatge of repeat users is low.
‍
Goal :  Keep users coming back to the site to complete their goal
Signals : Growth in Active users, Growth in Repeat ratio
Metrics : Repeat Users Conversion rate, Repeat Users %, DAU,WAU,MAU
  • Repeat Users Conversion rate= 3.67% (General website: 1-3%)
  • Repeat Users%(Returning Visitor)= 22.90 % *Average rate for EC: 30% ~ 40%
  • DAU = 1,638 Users, WAU = 13,523 Users, MAU = 45,395 Users
Insufficient Factors (User Interview is needed)
  • Reward repeat customers
  • Focus on customers service
  • Offer Incentives to repeat cusomers
  • Act on customer feedback
  • Use email to stay top of mind
  • Review customer data
  • Build customer experiences
According to Bain & Company, a 5% increase in retention correlates with at least a 25 % increase in profit

Task Success

The checkout is not quick, smooth, and efficient

Goal : Good reviews of the product and UX will lead new users to adopt, New users derive value in the product
Signals : Increased purchasing, Increased sign ups, Increased logins
Metrics : New Users conversion rate, New Users %, DAU (Daily Active User)
  • Time to complete a task (it is explained in Usability chapter)
  • Error Rate % (Based on Usability testing, 28.5% of Test users experienced user flows they did not expect.)
  • Drop Off %
    Cart 58.9% (but better than average 70% based on Baymard Institute)
    there is some problem such as lack of information the user wants to know, or the page is difficult to read.
    Billing and Shipping 14.8%
    Payment 15.9%
    Review 6.61%
    Purchase Completed (Checkout) 93.0%
    *Anything above 62.6% puts you in the top 20% of eCommerce stores.Anything below 30% puts you in the bottom 20%.)
Insufficient Factors (User Interview is needed)
  • Should look at user flows, stuff like payment options, delivery & return policy, number of form fields,etc.

Part2. Usability Testing, Campaign & Strategy, Propositions, and Next Steps

2-1.Usability Testing & User Interview

I conducted usability tests to understand the lack of HEART framework (Time to complete a task, percent of tasks completed, error rate)

Test Users: 3 Young People who like google(use google products), who wants to stay in touch with new technologies, buy things that have real value, and live in a way that does not impact the environment

Userbility Testing

  • Direct access to Google Merchandise store
  • View apparel page, men’s apparel page and new products page (which have much traffic)
  • View other pages (Ether Brand page or new page, which is not by product name category)
  • View pages which include collaboration or campaign items
  • Add products you want to cart
  • View billing and shipping
  • View payment

User Interview

  • Would you recommend this site or product to the others(Net Promoter Score)
  • Can you tell us why you gave that score?:
  • How easy was the site to use(Perceived ease of use ):
  • Would you definitely buy the products?
  • How do you feel about Google brand?
  • How was cost performance?
  • How was Ease of task?
  • Please list one thing to be improved on this site?
  • Please list one thing to be improved on the products?

Findings of Usability Testing & User Interview

All test users were able to complete the tasks!…However, the website has usability issues. And Users are not satisfied with the site and the products. Metrics of Happiness and Task Success(HEART framework) are Bad.

Positive findings

  • All of them were able to successfully complete the task.
  • Some of the products were favorably received.(small items and badges)
  • Especially the color scheme, Google dino, Google bike, etc. were favorably received

Negative findings

  • Users may not be able to find the site even if they wanted to.They always had to go through Google store. They had to search through the blog.
  • Users are not satisfied with the site or the products. Poor usability does not lead to user satisfaction (size description, price description, unexpected flow, load time).
  • Some tasks were taking too long and many were likely to lead to drop-offs.We found that certain tasks were error-prone and could lead to drop-offs. (Logging in at checkout would bring you back to HOME, and adding items to the cart might not be reflected)
  • It was not easy to find out where campaign products were being sold. If you don't make it easy to identify the campaign products, customers won't even recognize them.
  • There are few pictures, and it doesn't feel like you are shopping because you can't see the details of the items.

Results from User Testing

  • Direct access to Google Merchandise store
    ⇒Searching in Japanese did not show up in the search results.They were all accessed via blogs.
  • View all page freely.
    ⇒No problem
  • View apparel page, men’s or ladies apparel page and new products page (which have much traffic)
    ⇒No problem
  • View pages which include collaboration or campaign items
    ⇒No problem
  • Add products you want to cart
    ⇒Hard to find the size. Difficult to understand the amount in pounds or dollars. They picked one that was sold out. The system did not respond when the number was entered in full-width characters.）
  • View billing and shipping
    ⇒When adding items to the cart, the users were asked to log in, but when logged in, the cart contents was lost.
  • View payment
    ⇒Difficult to understand the amount in pounds or dollars.)

Task Results & Error rate (S:Success, F:Failure, P:Partial Success)

Time Spent on tasks

Result from User Interview

  • Would you recommend this site or product to the others(Net Promoter Score)
    ⇒-100% Bad
  • Can you tell us why you gave that score?:
    ⇒“Hard to find website and products”“I wasn’t attracted to it”“I don’t know anyone who might like Google.”
  • How easy was the site to use(Perceived ease of use )
    ⇒Bad
  • Would you definitely buy the products?
    ⇒There is a potential royal customer.
  • How do you feel about Google brand?
    ⇒Good
  • How was cost performance?
    ⇒Neutral
  • How was Ease of task
    ⇒Difficult to complete task for Japanese
  • Please list one thing to be improved on this site?
    ⇒“More images of the products and easier to see the prices”, “Size description” “ UI movement in response to user operations”
  • Please list one thing to be improved on the products?
    ⇒“More products in google colors””More character lineups like Google Dino”

Examine which metrics to capture(using HEART framework)

Here is a summary of the analysis results in the HEART framework.All results from GA, User tests, and Interviews,are poor, but improving these metrics will lead to improve UX.

2-2.Campaign and Strategy

Product

some of the lifestyle goods sold in a short period of time, and the users interviewed expressed a desire to expand the product lineup.
However,according to the GA, apparel products (Unisex) are the products with stable revenues.Considering branding as a charity product and recognition among other users, we believe that Hoodie, which is more likely to be used as outerwear, should be adopted rather than Tee shirt, which is more likely to be used as innerwear.

Page placement

The campaign will be placed on the home page of the website because of the biggest traffic.

Non-profit organization (World Wildlife Funds)

An environmental conservation organization operating in approximately 100 countries around the world with the following:
  • Prevent global warming
  • Create a sustainable society
  • Protect wildlife
  • Protect forests and oceans

2-3.Proposition

Usability Issues of TOP page

Top page had the following problems in addition to the campaignThese are the problems we found through usability tests and user interviews.
  • Login menu cannot be stored.
  • Difficult to find other than product-specific categories
  • Easy to lose home button. Menu is left justified, home is center.
  • Can not tell status if logged in or not.
  • Images are too large. Not scrollable.
  • The link is not to the product shown. I cannot find product information such as price, size, color, etc.

Proposition - the 1st sketch

  • Add Campaign page to Carouse
  • Add a Limited page to navigation menu
  • Can accommodate a hamburger menu

Proposition - the 2nd sketch

  • Add a Fixed Campaign page which can see goods without moving to other page
  • Add a Limited page to navigation menu
  • Remove Carousel
  • Add Home button
  • Add Account icon
  • Add goods Description, Price

Proposition - the 3rd sketch

I choose this as a next prototype.
  • Add a fixed Campaign banner next to Carousel
  • Add a Limited page to navigation menu
  • Shorten banner height. (Shop Trending Collection can be glanced without scrolling)
  • Add Home button
  • Remove a side menu ( Aggregate its functions to upper right menu)
  • Add Account icon
  • Add Color variation, description, Prices(including local currency), Review

Mockup to match the look and feel

I choosed the 3rd sketch and prototyped it.
  • Add a fixed Campaign banner next to Carousel
  • Add a Limited page to navigation menu
  • Shorten banner height. (Shop Trending Collection can be glanced without scrolling)
  • Add Home button
  • Remove a side menu ( Aggregate its functions to upper right menu)
  • Add Account icon
  • Add Color variation, description, Prices(including local currency), Review

2-3. Measure of success & Next steps

Measures of success

  • The number of people who clicked the “Shop now” botton from the campaign homepage banner.
  • The comparison of the number of "Add to Cart/Basket" of the branded product during the campaign period with the same period last year.
  • The comparison of the number of "Purchase completed" of the branded product during the campaign period with the same period last year.

Next steps

  • Conduct our Proposition
  • To improve the website,
    - SEO measures are needed (if Japanese are to be included in the target audience).
    - Improvement of the flow from user payment to user confusion is needed.
    - It is necessary to build an information architecture that fits the mental model of the users
    - Fix the flow of returning to the top page after logging in back to Home. (Logging in during the payment flow would cause the user to be removed from the flow.)
  • To improve the products, it would be nice if there was a clear description of sizes and automatic price conversion to reduce Drop-off rates.
    Also, information (images and prices) to help people learn more about the products would be nice.
  • Keep an eye for the metrics from Google’s HEART framework.
  • Revise navigation menu contents - conduct card sorting to understand users’ mental model
  • Shorten the load time - optimize image size