The refusals that matter more than the answers
Judging a business assistant by what it can answer is the wrong test. The useful test is what it refuses, and whether those refusals are enforced in code or written politely into a prompt.
Why do refusals matter more than capabilities?
Because a confident wrong answer costs far more than no answer. Every business has a small set of questions where being wrong is expensive: a binding price, a safety question, a clinical or legal opinion, a promise about timing.
Capabilities are easy to demonstrate and easy to copy. A written, enforced refusal list is neither, which is why almost nobody publishes one.
What is the difference between an enforced refusal and a prompt instruction?
A prompt instruction asks the model not to do something and can be ignored, especially under pressure from a caller who pushes. An enforced refusal is asserted by a test suite before deployment and re-checked after every model update.
We ran into this ourselves on a smaller matter. Our own chat assistant is told never to use an em dash, and it used one in its first live reply. If a written instruction cannot reliably control punctuation, it should not be the only thing standing between a caller and a clinical opinion.
What should I ask a vendor about refusals?
Ask for the list, ask whether it is enforced or requested, and ask what happens after a model update. If none of those have answers, the assistant is being sold on what it can do and nobody has thought about what it should not.