Usability Test Your Dream Vacation

Identify areas for improvement by looking for hotel websites that may have room for improvement and conducting usability tests

Project Information

Project type: Usability Testing & User Interview
Project year: Apr, 2022
Skills: Use card sorting to understand mental models, Frame and conduct usability studies
Role: UX Designer
Goal: Conduct Heuristic Evaluation, validate hypotheses, and find further areas of improvement by conducting 101 Usability Testing and Card Sorting
Tools: Zoom, UX tweak, Optimal Workshop, Blackmajic

Part1. About the hotel

Hoshinoya Karuizawa

Hosinoya is a Japanese luxury resort that offers an exceptionally accommodating environment rooted in the stunning nature and sophisticated cultures that define its location. It  provides guests with services and seasonal pleasures that it has refined. I choose this hotel because I feel there is room for improvements in the UI / UX  of the booking site.

Hoshinoya Brands

Hoshinoya is Japan's first Japanese-style luxury hotel based on the concept of "a day off from modernity.”

Hoshinoya Karuizawa

This luxury resort hotel boasts of its natural and free-flowing hot spring water.

5 Star Rating

The hotel you've always dreamed
of staying at once

Part2. Methodogies & Plans

Usability Testing

Usability testing involves putting actual products or prototypes in front of users to see how they interact with it, and if it functions as you'd hoped.

Hoshinoya Brands

Hoshinoya is Japan's first Japanese-style luxury hotel based on the concept of "a day off from modernity.”

Hoshinoya Karuizawa

This luxury resort hotel boasts of its natural and free-flowing hot spring water.

5 Star Rating

The hotel you've always dreamed
of staying at once

Heuristic Evaluation

Focusing on usability, a heuristic analysis is an evaluation method in which one or more experts compare a digital product’s design to a list of predefined design principles (commonly referred to as heuristics) and identify where the product is not following those principles. Before we meet with any users there are things we can do ourselves to think critically about the products, and that will help make tests more effective.

Nielsen Norman Group Jakob's 10 Usability Heuristics

The interface is assessed using Nielsen’s 10 Usability Principles “Heuristic” ( https://www.nngroup.com/articles/ten-usability-heuristics/  )and potential usability problems are found where these principles are violated

Problem Severity Rating

The interface is assessed using Nielsen’s 10 Usability Heuristics Principles ( https://www.nngroup.com/articles/ten-usability-heuristics/  )and potential usability problems are found where these principles are violated

Heuristic Evaluation Results

Poor usability when visiting and operating the site is likely to have a negative impact on the brand. In addition, site organization issues could lead to frequent misinterpretations of the services offered.

Usability Test Plan Dashboard

Usability Testing Goals

  • Identify points of improvements so users can visit this website and book easily a room at this hotel.
  • Identify common usability issues and pain points using Nielsen’s usability Heuristics.

Scenario & Tasks - Reserve a Villa

5 Users did an usability testing through recorded Zoom calls. 
They followed on one of the scripts I sent them, based on the findings we went through

Scenario

You are about to spend the most extraordinary vacation ever away from the city ever with your family and parents. 
You have decided to escape to Karuizawa.
You want to figure out the details of the hotel or hotel Villa(e.g., how many people can stay) and see if there are any discounts available, You would like to proceed with the reservation.

Tasks

  • Choose a Villa & Confirm Services and Facilities(Diner,Location.etc.)
  • Reserve a Villa
  • Cancel Reservation

User Demographic

I asked the following questions to ascertain whether the participants would book a hotel on the website or could be a customer of the hotel.
The hotel have good reviews, location, and rates, and participants value these as criteria for choosing a hotel, and participants are familiar with Internet reservations, making them potential customers.

Part3. An additional Research

Card Sorting Method

We needed to know what the market was like for apps where daily events are posted and for apps that allow users to look up events, so we conducted a competitive analysis of the major players.

Applied to Hoshinoya Karuizawa

using an information architecture tools  “ UX tweak” & “Optimal Workshop”

Closed Card Sorting Results "Popularity Placements Matrix"

The popular placements matrix shows the percentage participants who sorted each card into the corresponding category, and attempts to propose the most popular groups based on each individual card’s highest placement score.

Open Card Sorting Results " Similarity Matrix"

The similarity matrix shows you the percentage of participants who grouped two cards together, and clusters the most closely related pairings along the right edge. The darker the blue and the larger the cluster, the higher the agreement between participants on which cards go together.

Open Card Sorting Results " Dendrogram Best Merge Method"

The dendrograms give you another way to quickly spot popular groups of cards, and to get a general sense of how similar (or different) your participants’ card sorts were to each other. The Best Merge Method (BMM) tells you “X% of participants agree with parts of this grouping”, and so enables you to extract as much as you can from the data.

Card Sorting Results

The current menu wording and content organization could be unclear

The category names for this hotel are too abstract and the sorting of content is too varied from person to person. It would be more user-friendly to have a structure that allows people to group by category while also allowing them to see content within the category.

Card Sorting Recommendation

Drop-down menus make content visible by category

In addition to modifying the wording and structure, using drop-down menus can reduce unintended user transitions

Part4. Findings

Top Positive Findings

All participants indicated that the most attractive feature of the target site is the atmosphere of the photos. They seem to think those photos are appropriate for "Hoshinoya" which is a hot topic in Japan.
... However, users voiced their dissatisfaction. The website has usability issues.

Top Highlights

All users were able to complete the tasks! And there were no rejections of the combination of Social Media and Event App!
... However, users voiced their dissatisfaction. The website has usability issues.

Top Negative Findings & Recommendations 1/3 (Top Page)

Top Negative Findings & Recommendations 2/3 (Dining Page)

Top Negative Findings & Recommendations 3/3 (Dining Page)

General Findings

  • Villa Galleries have nice Photos
  • The booking process did not cause any problems.
  • It describes not only the address but also how to get there.
  • It takes time to go back to the top of the page.
  • No detailed information on the guest room page.
  • There are places where you can press the logo to go back to the Top page and places where you can't.
  • Contents, with the exception of rooms and facilities, have few images.
  • Some content is clickable and unfolds within the same page, while others move pages. There is no sense of unity.
  • Some photos feed automatically, some manually, and some don't feel consistent.
  • The rooms are villa type and the activities around the hotel are mentioned, but there is no floor plan of the facilities, so we cannot see the whole picture of the hotel and how we can spend our time there.
  • The system is designed to search for available rooms by dates, but does not support reservation methods by other purposes. Some people want to make reservations by room.
  • Some of the logos cannot be pressed to return to HOMEPAGE
  • “Reservation” Menu changes when scrolling up and down

Part5. Key Takeaways

Key Takeaways

Hoshinoya Karuizawa can attract potential customers by improving
the usability of its website.
Fitting the user's mental modelleads to good navigation.
Good UI leads topositive impressions and brand reinforcement
it is important not only to check usability tests, but also to learn heuristics and not to make usability worse in advance.

Thanks

Participants | Unsplash for Images | Hoshinoya Karuizawa for my Theme