The Agents Newsletter #10: Creating Nobi’s Agent
Welcome welcome.
In the past few issues, I covered a lot of high-level topics and concepts surrounding the creation of agents. Today’s issue will be a bit different.
I’ve been building LLM-based agents for a couple of years now and I thought it would be informative to walk you through the process I used to create Nobi, an AI shopping assistant. I figured a soup-to-nuts walkthrough could help to make some of these higher-level abstractions more concrete.
Walking through the process I’ve taken so far could help others who are interested in building their own agents from scratch, and it can also help you see that a lot of what goes into building an agent is not just interacting with an AI, rather, it’s a lot of traditional product management and software engineering.
Ok, let’s go.
Start with an objective
Every good product manager knows the problem they’re trying to solve. Having a clear objective in mind for what an application should do makes it far easier to create that application and design it in a way that it adds value for your users. Agents are no different.
When my co-founder and I started creating Nobi, we had a clear idea in our heads of what we wanted (which was the result of a lot of customer interviews, some early sales commitments, and 18 months of experience building a previous agent).
We were looking to replicate the retail sales associate for brick-and-mortar shops into online ecommerce stores. For us, that involves three core skills that our agent needs to be good at: product discovery, recommendations, and product support/Q&A. We saw a world where online shoppers should have a much better experience finding what they’re looking for, which would inevitably lead to brands increasing revenue.
We decided to focus on these key areas and say no to everything else, even when the random customer prospect would bring up something else that might distract us.
Having an area of focus allowed us to identify how our agent would need to behave and what features we’d need to add to ensure it works well for our customers and their customers. And as I’ve mentioned in previous issues, by having a verticalized, focused agent, we’re able to more easily and directly solve our customers problems, with less complexity and a better user experience.
Design
With our objective in place, we set out to visualize how our agent would operate.
Not every agent will need a design. Many of them will operate behind the scenes, without a user ever interacting with a graphical interface for the agent. But, as we learned from our customer interviews, that’s just not true in the world of online commerce.
Shopping is inherently a visual experience. People want to engage with product visuals. They care a lot about colors. Screen real estate is important. Images are crucial, too many words can be negative. All of these issues need to be carefully considered and implemented into a software application that both brands and shoppers are happy with.
A great UX is one that consumers never gets in the way of a customer’s flow. Users should never even realize they’re interacting with a built product. That’s what we wanted for our shopping assistant, and a visual design was the most obvious place for us to start.
Using what we learned from our discovery calls and shopper interviews, we put together a quick set of requirements. We knew that some of our early customers wanted to use Nobi for searching, and that we’d have to be able to plug into their search bars. We also knew that we wanted to show search results prominently, and that we wanted to let shoppers filter results by using natural language, rather than by having to click a lot of buttons and sliders.
So we engaged with one of my favorite designers to create a mockup of our v1, shown below.
Having designs in place allowed us to more clearly understand what the agent would need to do and how it would need to work. We’ve since polished and refined a lot of these designs, and have added more capabilities, but we started with what we had and then moved into prototyping the agent with Figma, and then creating the agent in code.
Engineering
Since I’m a techie, I default to creating working prototypes in code. It’s possible that if I were to use something like Replit or Bolt I might be able to move more quickly in the early days, but I suspect that when taking it from prototype to fully functional I’d incur delays, so I usually just start building.
With the designs in place, I started working on the agent that we’d be providing for our customers.
There are two major pieces that needed to work together for this agent to function properly: the backend “agentic” system that needed to be able to take user requests, and the frontend system that ecom stores inject onto their websites.
Backend
For an agent like Nobi, the backend agent functionality doesn’t need to be super complex. Nobi can function like a single-hop agent, it just needs to figure out a single action to take when a user sends in a request and return a single response.
Since I had spent the prior year and a half building a far more sophisticated agent, I was able to knock out a lot of the agentic parts of the backend quickly — we used pgVector for our vector database, Groq + Llama 3.1 for our LLM, and we added a couple of tools for things like search, product info lookup, etc. A lot of the other work on the backend was for things like ranking search results or more traditional create/read/update/delete functionality. The other part of the backend is a simple scheduled job that updates our indexed database of products and product information from all of our customers on a periodic basis.
And of course, I added a ton of integration tests to make sure everything was working (all of them written by AI itself).
Frontend
The frontend, as it tends to do in most software projects, took up a lot more time. Because Nobi needs to work on any website into which it’s injected, it requires a lot of flexibility around branding, look, feel, and functionality. There was virtually no “agentic” work required from the frontend (vis-a-vis invoking LLMs or contextual memory or similar concepts), and yet it takes up a large fraction of my time.
In my experience, this is common when developing agents, and it’s something that most people will overlook when they first start creating agents. The hard, tedious, time-consuming work will usually not be when interacting with the AI, it’s all the other stuff around it — the traditional crud that goes into good software engineering.
In any case, I quite some time building up the ability for our code to be injected and functional on third party stores, but once I did, it was a simple matter of standing up some cloud servers, our database, and some hosting before this thing went live.
Iteration and customer support
For the most part, going live with Nobi wasn’t too bad compared to many other software projects I’ve been a part of. It was quick and relatively straightforward. But as is the case with all software, there’s a lot that goes into improvements and maintenance.
We work closely with our customers to understand what they need, and will frequently build in new features using their feedback as our guide for what’s needed. Since launching, we’ve built additional new capabilities, some of which affected the agent functionality (like the ability to answer questions about a product when the answer to the question isn’t immediately available).
We’ve also created an A/B test framework to measure the performance of our agent in many aspects, and we provide custom reporting and analysis to our customers about its accuracy, funnel metrics like add to cart and average order value, etc. In addition, I have graphs monitoring system performance around things like response times or how many people close the agent, and there’s a good chunk of work that goes into understanding those metrics and turning that into product improvements.
As you can see, building an agent is very much like building any other piece of software. We we move forward, we’ll continue to build out new features, capabilities, and areas of robustness for the product that we’re building. As much as possible, we’ll follow customer feedback while staying within the bounds of what we’ve identified as the core capabilities we want to be great at. We’ll continue designing and coding and iterating, just like we would with any other product. And when necessary, we’ll add additional capabilities to our agent.
The key is to continue working and iterating on a product that solves a real customer problem in a meaningful way. The way we do that will be determined by what technology is available and is the best fit for the problem of the day.
Thanks for reading this far. Hopefully this was enlightening, but if not, feel free to let me know what questions you have or where I can focus on next time. If this was useful and you’d like to see how we built Locusive, a far more sophisticated agent designed to act as an internal employee for customer success, let me know. And as always, if there’s anything I can help with, feel free to reach out.
Thanks and see you next time.
-Shanif