Skip to main content
CoC ↗

From Text to Context: How We Introduced a Modern Hybrid Search

Track:
PyData: Deep Learning, NLP, CV
Type:
Talk (long session)
Level:
intermediate
Room:
North Hall
Start:
10:45 on 10 July 2024
Duration:
45 minutes

Abstract

Customers only buy the products they are able to find. Improving the search functions on the website is crucial for user-friendliness.

In our talk we present the lessons learnt from improving the search of our global online marketplace, which sells 20 million products per year. We moved from a traditional word-match based approach (BM25) to a modern hybrid solution that combines BM25 with a semantic vector model, an open-source language model that we fine-tuned to our domain.

With numerous references to current literature, we will explain how we designed our new system and solved the multiple challenges we encountered on both the ML and engineering side (data pipeline encoding documents, live service encoding queries, integration with search engine). Our system is based on OpenSearch, the lessons can be applied to other search engines as well.

In particular the presentation will cover:

  • Status and Short-Comings of our old Search
  • Introduction of Hybrid Search
  • Our Machine Learning Solution
  • Architecture and Implementation (with special consideration of latency)
  • Learnings and Next Steps

The speakers

Ansgar Gruene

Ansgar Gruene

Ansgar is a senior data scientist at GetYourGuide in Berlin. His work focuses on ML approaches to improve the users search and discovery experience on the platform. He holds a Ph.D. in Theoretical Computer Science and has several years of experience as backend engineer and data scientist in the travel industry.

Dharin Shah

Dharin Shah

Dharin is a senior engineer working in the Search team at GetYourGuide. He is responsible for all the infrastructure and data processing for search, which is exposed via generic APIs. He has deep interest in performance, distributed systems & databases, and has past experience in contributing to Opensearch. He also enjoys reading technical white papers, as well as reading more about the current AI hot-trends in general.