Making AI agent setup something anyone can finish
Overview
Problem
Setting up an AI phone agent took one long screen of settings that assumed you were an expert. Answering calls and making them took two separate agents.
Solution
I merged the two into one agent and grouped the settings into five plain tabs. I added a reusable knowledge library and a way to test the agent before customers hear it.
- One agent, both directions
- Faster setup for CX & Ops
- Test before going live
Role
Product & UX design: exploration, information architecture, user flows, UI, design-system components. The brief and its constraints were set before I joined; the structure and the implementation were mine.
Agents were limited and confusing
An agent is the thing that actually speaks to a company's customers. It picks up when they call, or it calls them. Setting one up means deciding how it should sound, what it's trying to achieve on the call, and what it's allowed to know. All of that lived on one long screen of settings, and the screen assumed you already understood how any of it worked.
It also made you choose a direction before you started. If you wanted an agent that could answer the phone and also make calls, you built two of them. One inbound agent on one number, then the same phone number and a second agent for outbound. The same business, the same script, maintained twice.
Around that sat a wall of prompts and calls to action with little indication of what you were supposed to do with any of it. Our users are not especially technical, and this asked them to be. The design needed to change.
Exploring further than we could build
The project was already decided when I arrived. The company knew this needed rebuilding and had ruled some things out before I got there. We were told to explore well past those edges anyway in case we stumbled across something great.
And we did. What ruled the alternatives out was not that they were bad ideas. It was developer bandwidth. We were rebuilding every feature in the platform at once and had to ship a working product in under a year. We did it in eight months. Against that, a ground-up interaction model was not something we could responsibly spend the runway on.
So we landed close to what already existed, with enough structural improvement to make it make sense. That was the right call for the timeline, and it was more obviously a relief to stakeholders than to the design team. Here are some of the ideas we had that we couldn’t go with at the time:
Chat-based setup
Describe the agent in conversation instead of filling in a form. Not the right time, but the direction I think this eventually goes.
Hire an agent
Take an agent on the way you'd take on an employee, giving it a role rather than a settings page. Another idea that I think should see the light of day someday.
Playbook
Build the agent and its workflow in one continuous flow. Too complex to ship in the window, but its time to value would greatly improve the onboarding process.
Settings regrouped, with room to grow
Inbound and outbound became one agent. The long screen of settings became tabs, grouped by what each setting is: Agent profile, Personality, Prompts, Tools, Knowledge. Calls to action were rewritten to read like decisions a person could make, not fields to fill.
The tabs also gave new features a place to land. The knowledge library and in-editor testing arrived with this project. Data Capture and Workflow Handoff came later, each as one more tab instead of one more stretch of scrolling, and more complexity and control now added.
Giving the agent something to know
Knowledge used to be a single link. One URL — your website, a Google Doc — and that was the entirety of what an agent could reference.
It became a library. Text stored directly in the platform alongside URLs, so an answer can live in the product rather than in a document someone has to maintain elsewhere. An agent can be assigned several assets at once — pricing on one, FAQs on another — and those assets are reusable across agents, so a company writes its pricing answer once rather than once per agent.
Hearing it before your customers do
Users can now hear what their agent would sound like and how it would act before callers will.
That sounds small and isn't. Configuring an agent is mostly writing. You are choosing words and hoping they produce the behavior you meant. Before, the only way to find out was to make your agent live and call it yourself, which is slow enough that people simply didn't and shipped agents nobody had ever heard speak.
Closing that gap turns setup from a form you submit into something you can actually iterate on. Change the wording, hear the difference, change it again.
What testing told me, and what it didn't fix
I tested with three users and had shorter conversations with CX, operations, and stakeholder representatives inside the company. People understood the new flow. The structure worked.
What surprised me was something the structure could not solve. Users did not want to build an agent. They wanted a very useful agent, they just did not want to spend the brainpower to set it up, train it, and tweak it. That is a different problem from the one I had been solving.
It also explained something I could otherwise have mistaken for a design failure. When customers migrated onto the redesigned app, CX did not hand setup back to them because customers did not want it. The primary users of this flow are our own CX and operations teams, who run it many times over, and for them it got substantially faster. Customers can genuinely do it themselves now, and mostly still choose not to.
But a finished agent still doesn't do anything. It sits there, configured and idle, until someone puts it to work — which is where the next problem starts.
Where I'd take it next
The testing finding points somewhere specific: stop asking people to configure an agent and let them describe the job in plain words, with the agent pre-configuring itself and the structured flow kept as the fallback for fine-tuning. Templated agents are the step in between, and were wanted from the start. The chat-based and hire-an-agent models we ruled out on bandwidth are still the right long-term direction, not dead ideas.
New screens are Figma mockups with made-up data. Older screens are captures of the previous product with customer details removed. The real product keeps changing with business decisions and user needs.