Ongoing AI Monitoring: Monitoring a model after it goes live

NIST AI RMF GOVERN 1.5 covers ongoing AI monitoring. In most assessments the control itself is present and the record proving it is not, which is why this outcome produces findings at organizations that are, in practice, doing the work.

What the framework says

Ongoing monitoring and periodic review of the risk management process and its outcomes are planned and organizational roles and responsibilities clearly defined, including determining the frequency of periodic review.

That is the text of NIST AI RMF GOVERN 1.5. Read it closely: it describes an outcome to achieve, not a product to buy, which is why two practices of the same size can satisfy it in different ways and both be right.

What this looks like in practice

In an assessment, NIST AI RMF GOVERN 1.5 is not one question. It resolves into several, and each one is really asking for different evidence. These are among the questions asked about it:

• Is there a planned schedule for ongoing monitoring and periodic review of the AI risk management process, with named owners and a defined review frequency?
• Has the organization decided what it measures about the AI program's performance and effectiveness, the methods, how often and who analyzes the results, and does it act on what the measures show?

Ownership usually sits with the AI governance lead, though the evidence is often produced by someone else, which is where the trail breaks. A control that works but has no owner tends to stop working the month the person who quietly maintained it changes roles.

Who this applies to

NIST AI RMF is voluntary rather than binding, and that changes how it should be used. Adopting it does not create a legal obligation, and declining it is not a violation. It earns its place when a customer contract names it, when it gives structure to AI governance work you already owe under a law or a contract, or when a recognized practice is worth having on record. Treat it as a way of organizing that work, not as a second regime.

Building it, in order

The sequence below is the order that produces evidence as a by-product rather than as a separate documentation exercise:

1. Set the frequency of periodic review for the AI risk management process and its outcomes.
2. Assign named roles responsible for conducting and approving each review.
3. Record the findings and resulting changes of each completed review.
4. Choose indicators for the program's performance and effectiveness (for example goal progress, overdue assessments, incidents, audit findings).
5. Define the method, frequency and owner for collecting and analyzing each indicator.
6. Report the analysis to the program owner and senior leadership.

Where else this maps

The questions behind NIST AI RMF GOVERN 1.5 also map to ISO/IEC 42001 9.1, which is voluntary guidance and standards rather than law; following it does not by itself establish compliance with any law. That matters for scoping: the same evidence can be written once and referenced from each, rather than rewritten per framework.

Evidence an assessor asks for

For this outcome the evidence types that satisfy it are plan and report. The distinction matters more than it looks: a policy states what you intend to do, a procedure states how, and a record proves it happened on a date. Auditors ask for all three, and a practice that has written the first two often has nothing for the third.

Date every document and keep the superseded versions. The framework sets no retention period, so this is our recommendation rather than a requirement: keep each version for as long as the system it describes is in use and for a period after it is retired, because a question about a past decision is answered from the version that was in force when it was made.

Where this usually goes wrong

The usual gap is a control that exists in practice and nowhere in writing. Someone does the work, reliably, and it has never been written down, so the organization cannot demonstrate it and cannot notice when it stops. The fix is not more control, it is a dated record and a named owner.

What to do about it

• Produce the evidence, not the intention. For NIST AI RMF GOVERN 1.5 that means a written plan naming who does what, and evidence it was tested and a dated report with the period it covers and who reviewed it. If the only evidence is that someone knows how this works, you have a practice and not a record.
• Put a name on it. Your AI governance lead should be recorded as the owner. A control with no owner stops working the month the person who quietly maintained it changes roles, and nobody notices.
• Re-read it when something changes. A new system, a new vendor or a departure is what makes a correct control stale. Tie the review to those events as well as to the calendar, and date the result.
• Do not treat this as a legal obligation. It is a voluntary framework. Adopt it where a customer contract or a regulator's guidance names it, or where it gives structure to AI governance work you owe for other reasons, and say in your own documentation which of those reasons applies.

The practical test for NIST AI RMF GOVERN 1.5 is whether someone unfamiliar with your organization could read your documentation and tell what you do, who does it, and when it last happened. If they can, the documentation will hold up; whether the outcome is met is a judgment about what it shows, and your counsel owns that conclusion. If they need you in the room to explain it, it will not.

Related outcomes

• Risk Management Processes for AI: Sizing AI risk management to the use case
• AI Roles and Responsibilities: Who owns an AI decision in a clinical setting
• Trustworthy AI Characteristics: Defining what trustworthy means for clinical AI

References

• GOVERN 1.5, in NIST AI Risk Management Framework 1.0 (NIST AI 100-1). This is a voluntary framework, not a legal requirement. The quoted statement is NIST's own, from the Core tables of NIST AI 100-1.
• Crosswalk references in this post are the mappings in our question bank, and are not verified against the published text of those standards.
• This post describes what a framework outcome asks for and what evidence documents it. It is not legal advice, and your counsel owns the conclusion. State law may impose additional obligations of its own.