Following on from my introduction to our tech stack last week, I wanted to share some insights into the work going on behind-the-scenes to improve the core part of our platform: the search engine. Search algorithms have come a long way since the days of the original Yahoo!, Lycos, and AltaVista. The history of search engines is quite fascinating, and charts the way the internet itself has changed. With Google dominating the search engine landscape, you might be wondering why search is even worth working on.
Well, although Google has come to dominate general search, niche search engines can cater to very specific queries that Google can’t. Effectively, we’re building a niche search engine that caters to specific needs of users in the space industry.
With that in mind, we’ve spent a lot of time talking to users, trying to understand what they’re looking for and how to map their expectations to fantastic user experience. This led us to abandoning our original “Google-like” interface (a largely empty page with just a simple search bar in the middle), to a richer environment that allows users to navigate the global space supply chain through a search bar, supplier map, and product categories & tags.
One bit of feedback that we consistently heard from users was that we were serving too many results for simple queries. Our immediate thought was to wonder how too much information is a bad thing? The fact of the matter is that the typical satsearch user has a pretty clear idea of what they’re looking for and they come to our platform to find the shortest path to the answer. Presenting users with hundreds of results only serves to create frustration.
Hence, we’ve started to rethink the way our search engine works. The fundamental question that we’re trying to answer is: how do we serve the right answer to the user in the fewest number of steps? This turns out to be more complicated than it seems. Primarily, the information we serve through satsearch.co today consists of metadata, i.e., high-level data that characterize other data. The “other data” in our case is our corpus of product datasheets. Currently, we serve over 5000 PDF datasheets that contain details about space products offered by suppliers all over the world. Ultimately, our goal is to do away with these PDF documents, and replace them with something much richer, but that’s a topic for another day.
Our user research led us to realizing that a lot of space engineers think of space systems in terms of a traditional product tree, which is often hierarchical and based on functions of different subsystems. An example of this is the ESA generic product tree, which classifies space systems through a hierarchy that runs from system level down to equipment level.
The product metadata we store in our database includes categorization based on a 2-tier hierarchy. Since users typically know which product category they’re interested in, we decided to change our search algorithm to leverage the knowledge of the user. I’m happy to announce that at the start of this week, we shipped our new 2-step product search. The goal of this 2-step product search is to utilize the user’s a priori knowledge to reduce the number of steps that it takes to present him/her with relevant results.
To highlight how this works, let’s consider an example. Previously, if a user searched for “cubesat”, they’d be presented with a long list of products that are in some way associated with cubesats. Now, if a user searches for “cubesat”, they’re taken to an interim results page that presents a list of product categories that contain products associated with cubesats. This allows the user to narrow down their search by selecting the product category that they’re interested in, yielding a shorter, more relevant list of results.
As our database grows, we have a lot more work to do to improve the way our search algorithm works. Here are some of the areas that we’re working on behind the scenes:
- Improved search ranking
- Parametric filtering & sorting
- Improved tagging
I’ll be sharing more insights into how our search evolves in future. If you have thoughts about our new 2-step product search, or if you’re generally interested in how our technology stack works, feel free to join our Slack community, or email me.
Ad astra!