How do I evaluate an AI consultancy without getting burned?
Three filters: (1) do they ship source-traced output with confidence scores? (2) do they refuse to recommend custom builds when off-the-shelf tools fit? (3) do they have a documented refusal pathway when evidence is insufficient? If filter one fails, slow down.
The short answer.
Three filters: (1) do they ship source-traced output with confidence scores? (2) do they refuse to recommend custom builds when off-the-shelf tools fit? (3) do they have a documented refusal pathway when evidence is insufficient? If filter one fails, slow down.
This is a question Aegis hears regularly during discovery. Here is the practical way to frame it.
How Aegis approaches this.
Aegis Boardroom's answer is shaped by three frameworks. Truth Architecture: recommendations are designed to be source-traced. Confidence Contract: recommendations are mapped to the canonical Aegis confidence states (I Know / I Think / I'm Inferring / I Don't Know). Life Integrity Engine: recommendations that may increase irreversible-harm risk are flagged for refusal or human review, not softened.
The fastest path is the AI Readiness Assessment: it returns a confidence-mapped band for your specific situation. From there, the Quick Win Plan or a deeper engagement scopes the right paid Aegis next step.
Frequently asked questions.
What is the first thing to check when vetting an AI consultancy?
Whether they ship source-traced output with confidence scores. If you can't see where a recommendation came from and how sure they are, slow down. That is filter one, and the one that matters most.
Is it a red flag if a consultant pushes a custom build?
Often yes. A good filter is whether they will refuse to recommend a custom build when an off-the-shelf tool fits. Reaching for custom by default usually serves the consultant, not you.
How can I tell if a consultancy will admit when it does not know?
Ask whether they have a documented refusal pathway for when the evidence is insufficient. A firm that always has an answer is a firm that will eventually give you a wrong one with confidence.