12 Questions to Ask Before Hiring an AI Automation Agency

Can you show me three production systems you've shipped in the last 18 months?

You're listening for specifics: what it does, what it's built on, how many users or requests it handles. Real builders light up here.

Red flag: demos that are always "in progress" or screenshots with no live system behind them.

Do you have a case study in my industry with a measurable outcome?

A specific client, a specific problem, the system they built, and a number — hours saved, response time cut, leads recovered. Industry familiarity also means less time spent explaining your world to them.

Red flag: only generic claims like "we save clients hundreds of hours" with no named example.

Who actually builds this — and will I talk to them?

Some agencies sell with senior people and deliver with whoever's cheapest. Ask who's writing the code and whether you'll have direct access to them.

What do you build in code versus no-code?

An honest answer separates the two. No-code is fine for simple, low-volume flows; it gets brittle for anything complex or high-stakes. You want someone who knows the difference and chooses deliberately — not someone hiding a pile of no-code behind the word "AI."

How do you handle edge cases and errors?

Every production system makes mistakes. What matters is how they're caught and handled. Listen for monitoring, fallbacks, alerts, and a human-in-the-loop for high-risk actions.

Red flag: "the AI just handles it" with no error strategy.

Which AI models do you use, and why that one?

The best answer is "it depends on the job." Being locked to a single model often means paying a premium or accepting weaker results for tasks it isn't suited to. (This is why we stay model-agnostic.)

Do I own the code, prompts, configs, and data?

The most important question on this list. The code should live in a repository you control, with the prompts and configurations there too. Anything less is a dependency trap.

How will this integrate with the tools I already use?

Your automation should connect cleanly to your CRM, email, and the rest of your stack. Ask specifically how they'll handle any tool you depend on that has a weak or missing API.

Where is my data processed and stored?

Especially important if you handle customer or regulated data. You want clear answers on where data goes, which third parties touch it, and what's retained.

What's your process and timeline, phase by phase?

A reliable agency breaks it into discovery, build, testing, and launch with rough dates. Vague timelines usually mean vague scope.

What does support look like after launch?

Launch isn't the end — models drift, APIs change, edge cases surface. You need a defined support structure, whether that's a retainer or a clear maintenance agreement. (Here's how that's usually priced.)

What's explicitly out of scope — and what happens if you disappear?

Clarity on scope prevents surprise invoices. And because you own the code, a reputable agency will happily explain how you'd hand it to another developer if you ever needed to.