Agentic AI Assistant — Jayesh Maheshwari

Problem

The dashboard covers a lot of ground: segments, experiences, goals, on-site content, ads, and analytics for all of it. Building a segment meant knowing which property to pick, which condition type it takes, and how AND and OR groups nest. Most users knew what they wanted — "people who read pricing twice this week" — not how to express it.

A chatbot that explains the UI doesn't fix that. The assistant had to do the work, on the user's own data, without ever doing something they didn't ask for.

How a turn works

Route

Each message goes to one of nine domain agents — analytics, segments, experiences, goals, content, ads, profiles, marketer, help — each with its own rules. A compact prompt is the fallback when context runs short.

Read

The agent works through a registry of typed tools, each with a JSON schema for its input and output. Reads run freely: analytics, segments, clusters, the site's ideal-customer profile. Reference reads that can't change mid-turn are cached, so the loop budget goes to real work instead of fetching the same docs three times.

Propose

Every tool that writes is flagged as mutating and sits behind a gate. The agent proposes, the user sees exactly what will change, and only an explicit yes executes it. Deletes are gated on every path.

Answer

Tool results are compacted before they go back into context, and the final reply is checked against them for numbers no tool ever returned.

Remember

Durable facts from a conversation — a preference, a constraint, a decision — are extracted, scored and scoped to a user, a site or the whole org, then pulled into later turns by relevance and recency. The current message and live data always win over memory.

What I built

The runtime — routing, the gates, memory — was a team effort. These parts were mine:

Tool handlers for segments, experiences, goals, events and content: turning a model's structured call into a valid segment condition tree or experience payload, and checking every ID it names against what actually exists. A made-up segment ID fails loudly instead of quietly matching nothing.
The Implement flow. AI recommendations on the analytics pages carry an Implement button; the backend seeds the agent with that recommendation, treats the click as the approval, and the agent builds the segment or experience — with a live audience count computed in the background while the segment takes shape.
Page content in the agent's working knowledge, so it creates and edits on-site content with the page in view instead of guessing at it.
A structured AI ledger: every model call across the product tagged with category, operation, entity and source, plus a billing policy and credit bucket, so AI usage can be reported, capped and billed per feature.
An agent-authored flag on everything the agent creates, so analytics can tell agent-made from hand-made.

Decisions

Narrow writes

No general-purpose "run this SQL" or "call this endpoint" power for writes. Each write is a small, named operation scoped to the user's own site — easy to validate, easy to preview, easy to undo. Reads get a read-only SQL escape hatch; writes never do.

The click is the yes

Asking "are you sure?" after someone clicked Implement is friction, not safety. The click approves creates and updates; deletes never ride that path.

Check the model

The model is good at intent and bad at IDs. Anything it names is resolved against real entities before a write, and rejected when it can't be.

Outcome

A user can go from "show me visitors who read pricing twice this week" to a saved segment with a live size in one conversation, and from an analytics recommendation to a running experience in one click. Those recommendations come out of the topic analytics — the next case study.