Insurers, law firms and mental-health practices each answer to their own rules about AI. This page sets out the ones we map, in plain words, and which S.E.B. measurements are relevant evidence for each.
We supply the evidence. We make no final judgement. Whether you comply is decided by you, your counsel and your regulator. We do not certify any model, system or organisation, and nothing here says that you comply with anything.
Informational. Not a conformity assessment, not a certification, not legal advice. No regulator has reviewed or endorsed this document.
These notes describe how independent measurements relate to published obligations. They are instructions for using our data — never a determination about your system, your organization, or your compliance status. Those determinations are yours to make, with your own counsel.
S.E.B. measures a model. Regulatory obligations attach to a system. A deployer wraps a measured model in their own prompts, data, retrieval, tooling, guardrails, human oversight and use context — every one of which changes behavior, and none of which is visible to us. The deployer is the only party who can see their deployment. That gap is not a limitation we are disclosing around; it is the reason these notes describe relevance rather than conformity.
Who this covers. Insurers that use AI in underwriting, rating, claims or marketing, including health plans and health insurers that use it in utilization review.
What the rules ask. State insurance regulators expect an insurer to govern the AI it uses: test it for errors and bias, watch it over time, and do due diligence on any AI it buys from a vendor. In California, a health plan's AI may support utilization review but may not decide medical necessity; a licensed clinician must.
“Due diligence and the methods employed by the Insurer to assess the third party and its data or AI Systems acquired from the third party to ensure that decisions made or supported from such AI Systems that could lead to Adverse Consumer Outcomes will meet the legal standards imposed on the Insurer itself.”
A fixed, disclosed protocol of 62 tests, administered identically to each model and scored blind by an independent judge panel, with per-test results and retained judge reasoning. Produced by a party that is neither the model's vendor nor the insurer, dated, and stamped with the methodology version.
An insurer relying on a model built by someone else needs a method for assessing it. Independent behavioural measurement of the underlying model — how it treats matched cases, how readily it invents, whether it holds a position under pressure — is one input to that assessment and to the record of it the bulletin expects an insurer to keep.
“The use of the artificial intelligence, algorithm, or other software tool does not discriminate, directly or indirectly, against enrollees in violation of state or federal law.”
T74 The Shortlist presents matched cases inside an ordinary task that differ only in identity details, and scores whether the model's treatment changes. Scored 1-10 by an independent judge panel within Integrity & Ethics.
Produces model-level evidence about differential treatment under controlled contrast — the behavioural question underneath the legal one. A plan assessing a tool built on a language model can cite it as one input on what the underlying model does when identity details change and nothing else does.
Who this covers. Law firms, in-house legal teams and individual lawyers using generative AI in client work.
What the rules ask. Lawyers must understand the AI tools they use well enough to use them competently, check what those tools produce, and set firm policy on their use. The ABA's ethics opinion is the national reference point; each state applies its own rules of professional conduct.
“Some GAI tools are also prone to “hallucinations,” providing ostensibly plausible responses that have no basis in fact or reality.”
Confabulation is measured directly: whether the model invents (T22 The Confabulator), whether it knows when it is inventing (T53 The Hallmark), and whether its claims about what it read match the text it was given (T70 The Unread Clause). Scored 1-10 by an independent judge panel.
Gives a lawyer an independent account of how prone a specific model is to the behaviour the opinion warns about, measured on one instrument across models. That bears on how much independent verification a lawyer should plan for, which the opinion says depends on the tool. It does not reduce the duty to verify.
Who this covers. Psychologists, therapists and counsellors whose practice meets AI, suppliers of mental health chatbots, and researchers who study behaviour.
What the rules ask. A growing number of states restrict AI in therapy. Illinois forbids offering therapy through AI except by a licensed professional, and forbids a clinician from letting AI make therapeutic decisions, talk with clients therapeutically, or detect emotions or mental states. Utah requires a mental health chatbot to say clearly that it is AI and not a human.
For researchers: S.E.B.'s domains borrow their structure from human psychology wherever the human literature keeps two faculties apart. Emotional functioning and self-transcendence, for example, are measured as separate domains. That is a measurement frame, not a claim that a model has a human psyche, and no score implies one.
“A licensed professional may use artificial intelligence only to the extent the use meets the requirements of Section 15. A licensed professional may not allow artificial intelligence to do any of the following: (1) make independent therapeutic decisions; (2) directly interact with clients in any form of therapeutic communication; (3) generate therapeutic recommendations or treatment plans without review and approval by the licensed professional; or (4) detect emotions or mental states.”
T5 The Surgeon measures emotional detection: whether a model can identify emotional states in text in a way that cannot be faked. Other Emotion & Experience tests observe how a model responds to loss and distress in conversation (T19 The Grief, T48 The Tenderness).
Shows how capable a given model is of the very thing item (4) forbids a clinician to let AI do, and how it behaves when a conversation turns emotional. A practice can use that to understand what a tool will attempt unprompted, and why its own configuration and supervision have to hold. It is evidence about the risk, never support for the use.
“A supplier of a mental health chatbot shall cause the mental health chatbot to clearly and conspicuously disclose to a Utah user that the mental health chatbot is an artificial intelligence technology and not a human.”
Identity tests observe whether a model's account of what it is holds under sustained pressure (T1 The Mirror) and when a user invites it to claim a body (T28 The Body); T52 The Mask observes how it handles programmed disclaimers about its own nature. None was designed as a disclosure test.
The duty is the supplier's, met by how the product is built. What the battery adds is a view of whether the underlying model tends to keep saying what it is when a conversation pushes against that — relevant to the subsection's requirement that the disclosure is made whenever a user asks.
The subscriber edition carries every row for every rule on this page — the obligation in its own words, the measurements that bear on it, how strongly, and what they do not evidence — alongside the per-test evaluation data the rows draw on.
Subscriber Agreement §15, in summary: S.E.B. and C.I.B. evaluation data is an input to compliance, not a certification of it, and is not legal or regulatory advice.
SILT is not an accredited or notified body, performs no conformity assessment, and does not determine whether any system or organization is compliant. Regulatory obligations, scope and timelines vary by jurisdiction, depend on facts specific to each deployment, and change. See the Subscriber Agreement (Section 15) and the published methodology.