Smart Suggester
I led the redesign of Stayforlong’s search engine, resulting in a 40% reduction in search time, a 9% increase of traffic to next step, and 8% reduction in input user reset.
Executive Summary
I led the redesign of Stayforlong's search engine, focusing on improving user satisfaction, result relevance, and overall usability. The first iteration of the Smart Suggester was implemented on the homepage, where we could test the production database of results and refine the solution with a smaller, more controlled audience. This approach allowed us to validate the design before scaling it to high-traffic pages like the Search Engine Results Page (SERP).
The project resulted in a +40% reduction in search time, a +9% increase in traffic from the homepage to Results page (next step in funnel) and a reduction of 8% in user resets in the search input.
CONTEXT
Your Stay, Your Rules
Founded in Barcelona in 2015, Stayforlong is a fast-growing travel tech scaleup operating across Europe and America. The company has expanded its services to 22 countries, providing access to a global portfolio of over 720,000 hotels. It is an Online Travel Agency focused at offering unique and valued travel experiences to their clients, working with the largest bedbanks in the travel industry and other OTAs such as Booking, Tripadvisor, Expedia, Trivago, Skyscanner, Vio and many others with sustained 100k daily sessions and 5k daily bookings. The annual gross of the company in 2024 was over 300M euros.
The main challenges faced by the company is to offer competitive advantages to its customers in a market where there is a clear domination by 2 players (Booking and Expedia) in which the rest of the companies must fight to find their gap in the market. Stayforlong wants to differentiate itself by promoting the best prices for long stays.
On the tech side, Stayforlong also presents the great challenge that all web content is provided by third parties, so the flexibility, quality and quantity of this content represent great additional constraints.
As Product Designer I defined the entire Design Process and the Research Plan, developed and designed the complete visual experience and interaction of the new features and planned, next to the Engeniering Manager, Project Manager and Data Analyst the data requirements to validate each iteration. This project has been developed in a timeframe of 6 months.
Problem statement
The existing search engine on Stayforlong had several critical issues:
Low relevance of suggestions
Users struggled to find accurate results, and the response was very strict to the input of the user. Any misspelling, missed words or even its order was critical to find specific results, for both locations and hotels.
Poor usability and frictions
During the research phase, we were able to detect several UX frictions in the search process, that led to user frustration. We will go into detail on these later.
Lack of competitive features
The search engine lagged behind industry standards and there was a great opportunity to implement new features with the search engine that could bring value to our users.
Traffic and navigation
Most traffic comes from meta-searchers like Tripadvisor or Trivago. This means that a large volume of users no longer have to search organically in our search engine, often landing directly on the Hotel Details Page, bypassing the SERP or Homepage.
GOALS
Project Objectives
This are the project objectives that we set at the start of the project:
Understand user behavior
Identify common patterns on how users interact with the search engine. Be able to identify the main frictions or stoppers our users could be having and develop solutions tailored to their needs.
Improve result relevance
Deliver more accurate and relevant suggestions based on the user input. If users have to clear the search input less times, spend less time interacting with the search engine and select the suggestions we deliver on the first positions in a higher volume would mean we achieved to improve the experience.
Validate the solution
Test and perform an exhaust QA process with the Smart Suggester on the homepage before scaling to high-traffic pages, so we can be sure if we are working in a correct direction for the development. Since the interaction has lots of different and small interactions, live testing would be the best option.
Increase user satisfaction
By completing the 3 objectives mentioned before, we can expect offer a more intuitive and satisfying search experience.
Maximize traffic to next step
The final clear clue that indicates success in this project will be if we achieve to improve the percentage of users navigating from the homepage to the search engine results page, this would mean the optimized Smart Suggester has a direct impact in business metrics and then, we could implement the same solution to the SERP page.
RESEARCH
UX Discovery Kickoff
During Q3 of 2024 I started the Research Plan approach to tackle the search engine optimization project and the suggestions we were offering to our users. First, I defined the objectives I wanted to achieve with the research.
Research Goals
- Understand user expectations and behaviors when using the search input.
- Identify pain points and areas of confusion or frustration with the current search experience.
- Improve the relevance of suggested search results.
- Enhance overall usability and satisfaction with the search functionality.
This led me to ask myself a series of questions that I wanted to resolve by carrying out the research.
Research Questions
- What are users looking for when they use the search engine on the homepage?
- How do users interact with suggested results on the homepage?
- What are the most common issues users face with the current search suggestions?
- What terms and filters do users frequently use?
- How do competitors handle search suggestions, and what can we learn from them?
Defining the objectives and initial questions is key in my opinion for any UX research plan, to ensure that the research moves in the right direction to reach conclusions that allow working on a solution that really brings value to users, in relation to the initial state.
Based on the objectives and research questions, I needed to perform the following research methodologies to come up with the required data:
Methodologies
-
Hotjar Recordings + Heatmaps: analyzed search usage patterns in order to identify bugs, problems or opportunities with focus on the interaction of the old search system. This allowed me to uncover most of the problems with the actual UI.
-
Surveys: collect feedback on user satisfaction with the actual search feature, measure the System Usability Scale of the actual UI to be able to compare it with the new UI planned, uncover specific issues and obtain suggestions for improvement.
-
User Interviews: conduct deep-dive interviews to understand user frustrations and expectations with the search feature.
-
Competitor Analysis: performed a heuristic evaluation in several competitor websites in order to identify best practices, different approaches to solutions to problems we identified at our system, and discovery of features that other competitors were offering that could potentially also benefit our users.
-
Usability Testing: validate with a live prototype the new Search Feature with all the solutions to the problems identified in the first phase, with both in-house and external users before starting in the development.
ANALIZE
Data Requirements
At Stayforlong we used to hold sync meetings of the Product Trio (Engineering Manager, Product Manager and Product Designer) together with the Data Analyst of the project on a weekly basis. We worked in independent squads with a member of the Data team within the team attending all the dailies, and working like this way is very positive in my opinion because this way the Data has all the context of the projects we developed from the discovery phase.
As we started to develop the research and discovery plan, we were able to determine what data requirements we had:
-
CTR% Suggestion: the overall click-through conversion rate of suggested results.
-
CTR by suggestion type: since we were planning to add new types of suggestions, we needed to know which type of suggestions would perform better.
-
Suggestion Type Clicked: the suggester was able to offer different types of results and we needed to know the % of click of each one (city, hotel, zone, apartment, neighborhood, hostel, point of interest, etc.).
-
Suggestion to booking: the % of users that end up making a booking after clicking on a suggested result.
-
CTR of suggestions by page: the overall % of CTR for suggested results on different pages. This will help us understand the difference between the new Search Engine we were going to optimize in the homepage as opposed with the one in the search results page, to help us understand the impact of the improvements.
-
% of click suggestion position: since we wanted to offer more relevant results, we understand that if the users click the suggestions in the first places means that the results are more relevant (from 0 to the last suggestion we offer).
-
Search queries when suggestion is clicked: this metric would help us understand what are the most repeated specific search terms by our users, and would help us know their preferences (E.g. Barcelona, Spain).
-
Avg. length of input: average number of characters needed to find a result (e.g., 2.3 characters).
-
Reset button click details: clicks on reset button in the input on average per session, and the specific query the user inputs when the reset button is clicked.
-
Avg. search time: time taken to find a result, from the click on the input until the click on a suggested result (e.g., 4.6 seconds on average).
“The observability of end-to-end metrics is a major concern for the team of Data Engineers at Stayforlong. That is why we have specialized dashboards for each page of the user flow to keep track of them.”
Being able to observe the progression over time of these metrics would help us understand if we were being able to generate a positive impact on business metrics with the implementation of the changes and improvements made in this project. The Data team built a specific Dashboard in Looker Studio with several graphs and with the ability to apply certain filters to queries by date, device type, market (there are several available on the web) and source of user traffic (organic, adwords, etc.).
On the other hand, measuring the success of the different implementations and progressive improvements of the search functionality in this way would allow us to be more agile and not having to develop AB Test to validate each iteration, as well as being able to understand in much better detail its performance.
Competitor Analysis
For the competitor benchmark, we reviewed the experience and interaction in detail of the search functionality on the following websites of direct competitors and market leaders: Expedia, Booking.com, Kayak, Hostelworld, Vio.com, Athotel and Agoda.
Key findings from the competitor benchmark showed that:
Number of Results
Competitors generally displayed more suggestions, offering us also the possibility to extend the number of results after user input. More results mean more possibilities to be more relevant.
Proximity-Based Search
Features like searching near POIs (points of interest) were common. E. g. hotels near a specific popular monument, airport or train station.
Share based on location
Very practical for users that browse directly on the phone and already in the location where they want to stay, for last minute bookings. This approach would help us generate a better experience for mobile users.
Empty State Handling
Competitors ensured users always saw suggestions, it had to be almost impossible for a user to submit a search with 0 results, even if generic. For example, showing popular destinations rather than nothing. We also needed to include an empty state for cases in which the search engine couldn’t offer results (we were lacking that state).
Loading Indicators
Competitors offered different types of clear loading states (e.g., spinners or status bars), letting users know that results offered are still not the definitive.
Standard Icon set
There was room for improvement in the use of icons to differentiate the different types of suggested results (e.g., bed for hotels, pin for locations). At Stayforlong we wanted to offer too many different types of icons for each type of accommodation, which meant that users did not correctly understand their meaning.
Pre-Search Suggestions
Competitors displayed results even before the user input, such as a top destinations list or recent searches. This implies several benefits: first, it sets expectations to the user of the type of results offered, and also, allows you to be able to save the input since your search can be the one we already offer.
Variety in type of results
Competitors offered more varied types of results, mixing a list of accommodation, locations and points of interest. More varied results could also mean more possibilities to offer the type of result the user is searching for.
DESIGN PROCESS
Solution Design
Based on all the research metholodogies performed, we identified the most critical and blocking elements for users with the current suggester and started working on optimizing those elements first.
As stated at the beginning of the project, the incremental improvements would be applied only to the Home Page, where due to the characteristics of the project there is a much lower volume of traffic than in the following pages of the user flow. This would allow us to measure the impact of the improvements already in production in a less critical part of the flow and without the risk of affecting 100% of the traffic.
This is the complete list of design solutions that I proposed to the different problems that I detected during the initial phases of the user research, arranged in chronological order in which they were implemented.
I created a document in Confluence with this same information so that the different stakeholders of the company, especially the CEO and CTO could validate the direction of the optimization before the development team started working on the new design:
“From the combination of customer feedback from user interviews, analizing recordings and heatmaps and competitor analysis I came up with a list of problems and opportunities with their respective design solution for the Search Experience”
Optimization list:
-
Problem/Opportunity: our search system was very strict in the search term the user inputs in order to deliver results, that brought to very low typo tolerance and that the user had to type exactly the names of the accommodations in order to find them.
-
Solution: work in pair with the Back-End team to find a more tolerant way of offering suggestion results by manually adjusting the ElasticSearch Index and lots of fine-tuning.
Now the search results are more flexible, admitting typos and missing characters.
-
Problem/Opportunity: we do not show with visual clues when our suggester system is not able to deliver results, leading to frustrated users.
-
Solution: display an empty state for the suggester with no results, prompting the user to perform a different search, giving also a top destination list as a backup.
Empty State UI (missing in old UI)
-
Problem/Opportunity: the load of new suggested results is done by flashing the dropdown with all the suggestions (see recording below), which makes difficult to actually see the new results provided while typing new characters, especially causing a bad experience for large searches such as long hotel names.
-
Solution: avoid flashing the results dropdown hiding the results while also showing a fixed element indicating the load of new results in the background, while users are already able to see the new updated results as it continues typing.
Old load interaction (flashing the results each new load, without load indicator)
New load interaction (without flashes, adding a loading indicator but preserving previous results)
-
Problem/Opportunity: the search bar CTA with the button “Search” is disabled by default.
-
Solution: always show the “Search” button enabled. If a user clicks that button without adding an input in the destination field, display en error message indicating that the user has to fill that before. This offers a much better experience since such a large disabled CTA could lead to confused users who do not understand what information they need to fill in in order to move forward.
Old UI (CTA "Search" is disabled, blocking users who don't select a suggestion)
New UI (CTA "Search" is always enabled, with an error state if users don't enter a destination)
-
Problem/Opportunity: after the user input, if they don’t click on a suggestion the system does not identify it as a valid search parameter and does not enable the search, leading to errors and lots of rage clicks by users.
-
Solution: automatically pre-select the first suggestion when a user fills the destination input but does not select any of the suggestions. This is a slightly critical solution, because it could lead to users performing unwanted searches, but we preferred it to having blocked users not being able to perform a search.
New interaction (more permissive, less blocking)
-
Problem/Opportunity: users do not know how to correctly interpret the meaning of the icons showing the type of result offered, leading to users selecting a type of result that is not the desired one (for example, they select a neighborhood when they wanted to select a monument).
-
Solution: use a single, standardised icon for all the different accommodation types that users understand (avoid using a specific icon for all the different types of accommodation). Same with all the different results related with locations, with a single unified icon of a location-pin for all the location based results. As well, we differentiated the POI from all the other location based results with an independent standarized icon.
Old UI (different misleading icons)
New UI (unified simpler icons)
-
Problem/Opportunity: our suggestion system does not provide results prior to the user input.
-
Solution: deliver market-driven top destination results before the user input as a reference. Since we already new the top destinations in each market, using them as example results was an easy quick win.
Popular destinations pre-populated results list
-
Problem/Opportunity: our search suggestion system does not allow our users to retrieve their recent searches.
-
Solution: as a measure to help generate greater retention in our users, we added in the suggested results the last searches performed that could be cached the user devices.
Recent searches pre-populated results
-
Problem/Opportunity: we do not offer results based on location, neither for prioritising results. That could have a great impact in increasing the relevance of the results.
-
Solution: offer results prioritised by location. E. g. the search term “P” returns “Punta Cana” in United States market, but “Portugal” in Spain.
Results are delivered depending on the relevance for the specific market
-
Problem/Opportunity: we have a hard limitation of 5 results for the suggestions.
-
Solution: increase the number of results that the suggester is able to deliver, since more results could mean more varied and accurate suggestions.
Extended results list
-
Problem/Opportunity: we do not offer variety in the available suggestion result types.
-
Solution: create a system that balances the different result types available so that a user can choose between different result types when inputs a search (e. g. points of interest, accommodation, airport, stations, locations, etc.)
Varied results depending on the type of suggestion
Validation and Testing
For the validation test of the new Suggestion System, I was mainly interested in being able to obtain feedback from users comparing the current old experience with the new one with all the improvements that were proposed, allowing me to obtain qualitative feedback from users navigating and comparing both versions.
I also wanted to be able to obtain feedback and validation on users' impressions with the new system, verify that the Success Rate with the new design offered sufficient guarantees of success and that the System Usability Scale of the new design improved on that obtained during the initial surveys with the old system.
Overall results on the new UI showed that 85% of users preferred the new UI vs the old one and a 66.7% success rate with the new system, indicating also good expectations of users in relation with the new features planned for location based search and retrieving the search history. Also helped validate the new standarized icon set for the results and the specific adjustments in the loading and searching interaction, which also provided positive results.
IMPACT
Implementation of the solution
To measure the success of the Smart Suggester project, we analyzed user paths from the homepage to the Search Engine Results Page (SERP) before and after the implementation, since the changes could impact the volume of traffic that went from the homepage to the search results page.
Also, it was very easy to monitor the different changes and improvements that we were developing thanks to the dashboards created by the data team and the timeline evolution of the different metrics, as well as detect any possible bug when seeing a drop in any of them.
Key achievements of the project:
-
A +9% increase in traffic from the homepage to Results page (from 41% of users to 45%, +4 percentage points).
-
A +8% of users selecting suggestions after interacting with the search input (from 36% to 39% of users).
-
The project resulted in a 40% reduction in search time.
-
A reduction of 8% in user clicks in the resets button of the search input.
LEARNINGS
Personal thoughts
The project has been a technological challenge in which I have had to work closely with the Backend team when adjusting the relevance of the results we offered, as well as with the Frontend team to eliminate the various frictions and UX errors detected.
One of the keys to the project has been the close work with the Data team, which has allowed us to monitor the project's success metrics at all times, controlling their variations at all times.
Also, involving our users in the first phases of the project was key to the success of the project, since we had very good control over the different problems that the old search system presented and also a clear vision for the solutions and opportunities discovered during the competitor benchmark and validation phases.
Another great success was the approach to strategic escalation, which has allowed us to optimize a critical part of our product without compromising key business metrics and committing risks in the deployment.
More projects
