Hi. I’m an AI engineer of an emerging retailer. We’re continuously pushing the boundaries of our user search experience. We’ve got a massive inventory, hence a lot of data to be managed. This got me thinking about the untapped potential of neural search.
I’ve had my hands on OpenAI’s GPT and Deepset’s Haystack lately. Both tools are great in specific scenarios, but integrating them seamlessly at an enterprise scale is challenging, especially when we’re talking about real-time user interactions. The primary challenge remains in managing multimodal data efficiently without sacrificing speed.
To add context, my goal in leveraging something like GPT for e-commerce is to create a more intuitive, conversational, and responsive search function. Imagine a user typing in a vague description or query, and the system providing product suggestions like a seasoned salesperson would. Given the vast product range, the neural search could bridge the gap between user intent and the most relevant product offerings.
If anyone has experience with this I’d like to hear your thoughts, and if you have any other tool recommendations for this pls do share. I’d be grateful for any help
You can do this pretty seamlessly with hybrid versions of opensearch, combining traditional search queries with vector based embeddings. Vectorize your item descriptions and then turn your free form user queries into hybrid queries that you feed to open search. It’s possible to feed complex natural language queries like “find me some cheap crocs or sandals listed in the last 12 days” and ask an LLM to transform that into a JSON format valid opensearch query. Gpt4 can do this flawlessly if you also provide a JSON Schema that matches your open search index.
I did something very similar recently and I also wasted an enormous amount of time screwing around with haystack, llama index, langchain and the like. I highly recommend avoiding these things if you are trying to build anything custom or that needs to scale or that needs to go onsite.
These frameworks all provide a few nice basic app ideas, and then wrap a laundry list of dedicated vectordbs that I predict will disappear within 2 years. Beyond that they have huge Java like OOP implementation styles that make it a huge pain to hack around or prototype without providing the scalability you’d typically like to trade for that bloat. Plus since they try to treat all the backends the same and this ends up giving more powerful multimodal data stores like Postgres and opensearch short Schrift by not supporting their more useful and sophisticated hybrid search features.
I finally discarded all of it and just started over with an api backend implemented with fastapi and opensearch and oss embeddings.
I’m also wary of OpenAI - gpt4 is great but for this use case it’s best to treat it as a prototyping mechanism and then fine tune a llama2 variant to do the query wrangling. Otherwise it’s too slow, to unreliable (service wise) and way too expensive. There’s no advantage to using OpenAI embeddings.
My 2c. Sounds like a super fun project (at least if it goes like mine has).
thanks for sharing! good thing you found success with an API backend implemented with fastapi and opensearch. I’m also impressed by how well you handled the hybrid versions of opensearch but tbh, am curious what challenges did you encounter while integrating the hybrid versions of opensearch?🤔
Is there any reason why you won’t just use a CLIP-based model and why you’re trying to use OpenAI’s GPT?
I’m also in charge of a text-image (text-image, not multimodal in my case) model that my company’s trying to create a search product with. There have been talks about using “ChatGPT” from higher-ups but I just don’t see the reason why we’d have to do this. I figured that a simple NER model or something would work just as well, I mean how many people do online shopping while expecting textual responses from the website.