How a 2 Person Team with No Funding is Building an AI Search Engine for Shopping

This is the typical experience of online shopping we faced: Type in a search query on most sites and find ads and products that are probably not the best ones. Every website shows a different set of items - has different sizes, colors, variants, discounts, prices, reviews. To actually compare, we end up opening 5-6 sites/tabs, copying links, checking discounts one by one. This feels less like "search" and more like manual digging.

That's what we're fixing at Minar - a shopping search engine that pulls products, deals, and collections from across the internet into one place - for search and discovery.

With Minar, the search is broad - we fetch data from over 400 sites - which means about 2-3 million products. The search isn't just keyword matching - it's context-aware. We use semantic understanding to interpret the intent behind your words - so we want you to be able to type anything from "red dress" to "something aromatic to wear". It's natural language search that works for broad product queries.

The semantic layer is what makes it feel intelligent, unlike most of traditional e-commerce search. That said, it's not magic - if you ask for something super specific like a "blue dress with polka dots and a red bow from a new Instagram shop," we probably won't get that for you today (it's coming soon though).. But for broad, everyday shopping searches, it works really well.

But the twist - and the content of this blog?
The entire search engine - yes, the part that clusters, ranks, and diversifies millions of products - was built by just two people, on a single VM, with no funding.

The Constraint: Fast and Smart, on One Machine

The challenge wasn't just building search - it was building AI-powered, real-time search that works under tight constraints.

Why did we need real-time search? Because both our real-time processing and our database/storage compute were constrained. We couldn't build a full product catalog by running 2–3 million scraping exercises every few days and then retrieving them properly with such limited compute. So we had to rely on real-time scraping. And since we wanted our search to be fast, we had to build an optimized real-time scraping system. Every query has to trigger a pipeline that:

• Semantically fetches products from our database where we have product category → collection URL pairs saved, and then extracting product data from it, while also exploring other sources like Google based results
• Decides which links are collection pages vs. product pages (since scraping a single product is too heavy and I/O bound for real-time search).
• Consolidates products from all sources.
• Ranks them based on meaning, distances, gaps, thresholds, favourable prices and ratings, user given filters, and site tiers.
• Sorts and diversifies results so you don't see ten links from the same store next to each other.

And all of this had to run on one modest machine, without slowing down.

Debraj, Founder and Shivam, ML Engineer, mid 2025.

Cracking the Clustering Problem

One of the biggest hurdles was deciding exactly which product listing page we needed to scrape out of millions of possibilities. For example, if someone searches for "red dress" we only need to scrape the "red dress" collection from a website or the "dress" collection and then filter out the other-colored dresses from it. Think about it: if someone searches for "wireless headphones", our system has to instantly know which pages across the web actually contain relevant product listings.

We went through multiple approaches here, each with its own trade-offs:

K-Means clustering was our first try. It's lightning-fast, but it expects you to define cluster sizes in advance. For dynamic shopping data where queries, categories and collections keep shifting, that's a problem. It ended up being quick but not very accurate.

HDBSCAN came next. It doesn't need you to specify cluster sizes - it figures them out automatically. So much smarter, yes, but at scale, and especially our scale (millions of data points), it was really slow.

That's when the idea of a hybrid approach clicked. What if we let K-Means do what it does best - quickly narrow down to a few promising clusters - and then let HDBSCAN handle just that smaller subset? The result? A pipeline that could accurately map queries to the right categories, even when the query was vague or quirky, while still keeping response times snappy.

This wasn't just a small tweak - it was days of trial and error, testing different embeddings, running against huge datasets, and constantly balancing speed with accuracy. But it paid off: this hybrid clustering became the backbone of how Minar understands search intent at scale.

Ranking Beyond Keywords

After fetching products, the next thing was ranking them in a way that feels human. Literal keyword matches weren't enough. We wanted the engine to understand the meaning.

We experimented with:

• Bi-encoders for fast similarity scoring.
• Cross-encoders for deeper accuracy (though slower).
• Hybrid methods combining both.
• N-gram based similarity scores with keywords and synonyms

On top of that, we tried semantic similarity using cosine distance, fuzzy matching, and keyword extraction with YAKE. We even ran experiments with Amazon's product datasets, embedding them with all-MiniLM-L6-v2 to see if clustering our products with Amazon's categories improved accuracy. It was promising but inconsistent, so we dropped it.

What finally worked best was a blend of classic and modern methods - a combo of TF-IDF (great at catching literal matches) with semantic embeddings from transformers (great at catching intent).

That gave us relevance scores that pushed past accuracy of 80%, moving closer to our long-term goal of 95%.

The Final Touch: Diversity

Even with high accuracy, we noticed another issue: too many results from the same store. It felt repetitive. So we built a diversification layer. After ranking, results are re-ordered to ensure a healthy mix of brands and sites, making search results more useful and discovery-friendly.

Building Big With Small Resources

What we have today is a functioning AI shopping search engine that fetches, clusters, ranks, and diversifies results in real-time - all on a single VM. Users can search, compare, wishlist, and explore live deals with ease.

And this is just the beginning. With more processing power and funding, Minar can scale into something exponentially faster and smarter. But even now, the fact that a two-person team, with no external funding, has managed to build this system in only 2 months shows what persistence (and a lot of late-night debugging) can achieve.

This is Minar: a fast, smart, and lean shopping search engine built to make online shopping less about endless tabs - and more about finding exactly what you want.