I had been evaluating AI chatbots using Nielsen's 10 Usability Heuristics. Not because I thought they were the right tool, but because they were the available tool. After my third audit on a data-centric GenAI product, I stopped pretending the heuristics were working.

Nielsen's principles were codified in 1994 for visual, form-based interfaces. They are built around the assumption that the product's interaction surface is a screen: that users navigate through menus, read labels, fill in forms. None of that maps to a conversational interface. When a user types a query to an AI assistant, there are no menus, no navigation paths, no visible affordances. The design surface is entirely in how the system responds to language.

So when I tried to evaluate whether a data chatbot was working, whether it understood what users were actually asking, whether it handled ambiguity gracefully, whether the data it returned was trustworthy, I was reaching for concepts the heuristics don't have words for. "Visibility of system status" does not help you evaluate hallucination risk. "Consistency and standards" does not address context amnesia. "Error prevention" does not tell you anything about what happens when a user asks an LLM a question that is outside its data scope.

The heuristics were not wrong. They just weren't written for this. I needed something that was.

The secondary pressure came from the kinds of products I was working with. Data chatbots and GenAI assistants have a failure mode that visual products don't: they can be fluent, polished, and factually wrong simultaneously. A well-designed button doesn't fabricate data. A well-designed LLM response might. Evaluating these products without a framework specifically designed for that class of risk felt increasingly irresponsible.

That was the origin of AXIS. Not a desire to build a framework, frameworks are means, not ends, but a specific professional need that existing tools couldn't fill.

There are three categories of existing evaluation frameworks, and all three fail for the same underlying reason: they were designed for interfaces where the user navigates toward information through deliberate visual steps. Conversational interfaces reverse that: the user asks for information, and the system has to navigate toward an answer through language.

The gap is not that Nielsen's heuristics are bad. The gap is that the product category has changed more than the evaluation toolkit has. The enterprise AI assistant of 2026 shares almost nothing with the form-based web application of 1994, except that we are still evaluating it with the same checklist.

There are newer frameworks, conversational design guidelines from Google, Amazon, and Microsoft, but these tend to be prescriptive rather than evaluative. They tell you how to write good conversation flows, not how to determine whether what you shipped actually works. AXIS is evaluative by design: it does not tell you what to build, only whether what you built is doing its job.

AXIS is organised into five domains, each corresponding to a distinct failure category in conversational AI products. The domains are not arbitrary. They were derived from recurring failure patterns observed across audits of data chatbots, GenAI assistants, and task automation interfaces.

Each axis contains three to four evaluation criteria, each scored on a tri-state scale: Optimal (1.0), Developing (0.5), or Critical (0.0). The tri-state is deliberate. Binary scoring creates false precision, while a five-point scale creates false granularity. In heuristic evaluation, the useful question is almost always "is this working, almost working, or not working?" The tri-state answers exactly that.

The axis names are deliberately non-generic. "The Listening Problem," "Ground Truth," "The Graceful Fall": these names carry meaning about what each axis is actually evaluating, and they are calibrated to be memorable to evaluators and legible to product teams who weren't trained in UX. A framework that only researchers can use is not operational.

The first full AXIS audit was run against a prototype of a ServiceNow Request Assistant, an AI-powered chatbot designed to help enterprise employees raise IT and HR service requests without navigating the ServiceNow interface directly. The assistant could theoretically handle request creation, status checks, category routing, and escalation, but the prototype had not been formally evaluated.

The audit used AXIS across all five domains, running structured test scenarios across each axis's criteria. The findings were not a surprise in type. Most of them were what you'd expect from a first prototype, but the framework surfaced them in a way that the standard "just use it and see" testing approach had not.

The audit identified findings across eleven of the fourteen axes. Key findings were concentrated in three areas.

The AXIS audit produced a prioritised finding report structured around criticality tiers: Critical findings blocking launch, Developing findings requiring iteration before GA, and Optimal areas to preserve. This structure is deliberate: it converts evaluation output directly into a sprint backlog, without requiring the product team to interpret severity themselves.

The three critical findings were addressed before the prototype was presented to stakeholders. The scope overstatement was corrected through revised onboarding copy and a visible capability manifest in the opening message. Data synchronisation was flagged to the engineering team as a real-time API integration requirement, not a UX fix. The context reset issue was resolved by implementing session state persistence.

The developing findings were incorporated into the next sprint. The over-explanation issue was addressed at the system prompt level with explicit length constraints and format directives. Domain vocabulary gaps were partially resolved through a terminology mapping exercise: a list of common internal request names was added to the training context.

More important than the fixes themselves: the AXIS framework gave the product team a shared language. Conversations that had previously been vague ("the responses feel off" / "users seem confused at the start") became specific ("we have a Cold Start problem, the first-turn success rate is 78%, threshold is 85%"). Evaluators and engineers were reasoning about the same criteria, in the same terms.

That shared language is the non-obvious value of a structured evaluation framework. The findings are useful once. The vocabulary is useful forever.

AXIS is not an attempt to replace Nielsen's 10 Heuristics. Those heuristics remain useful for evaluating visual, form-based interfaces, which most enterprise products still contain, alongside their conversational layers. The frameworks are complementary, not competitive.

AXIS is specifically for the conversational surface: the layer of the product where users speak in natural language and the system responds. That layer has grown from a novelty to the primary interface for a significant and rapidly expanding category of enterprise software, and it deserves evaluation criteria that were written for it.

It is also not finished. Every framework is a hypothesis, and AXIS is no different. The axes are grounded in observed failure patterns, Gricean principles of cooperative communication, speech act theory, and domain-specific research on data veracity and AI transparency, but they will be refined by what future audits surface. The ServiceNow audit was the first test. It will not be the last.

AXIS is designed to be used across four contexts: heuristic evaluation (structured expert review), usability testing facilitation (as a scoring rubric during moderated sessions), design QA during build (as a checklist at milestone reviews), and designer onboarding (as a conceptual foundation for practitioners new to conversational product design). Each context uses the same axes and criteria: the application changes, the framework does not.

The full interactive framework is available on the next tab. Every axis, all criteria, the tri-state scoring system, and the evaluation method tags are documented there in full.