AI Risk Assessment: Use NIST’s 4 Functions to Choose What to Test

An AI risk assessment is a documented process for identifying the harms an AI system could cause, analyzing their likelihood and impact, and deciding which controls make the remaining risk acceptable. Run one for high-consequence use cases, after major model, data, or operating changes, and wherever regulation or policy requires it. We use the NIST AI RMF and NIST’s ARIA evaluation approach as our primary references.

TL;DR:

• For high consequence systems, combine automated model tests with red teaming and real user sessions; benchmarks alone miss misuse and contextual harms.
• Schedule quarterly reviews for higher tiers and annual reviews for lower tiers; reassess after material changes to data, models, prompts, integrations, users, or operating conditions.
• Record the framework edition, an accountable decision maker, uncertainty, and supporting evidence; numeric scores can imply false precision when likelihood or severity is poorly understood.
• Integrate assessments into existing security, privacy, procurement, and project risk processes; regulatory duties depend on your role, system, use case, and jurisdiction.

Table of Contents

• Why do AI risk assessments matter for your projects?
• How do the NIST AI RMF functions work in practice?
• What is the step-by-step AI risk assessment checklist?
• How do model testing, red teaming, and user testing compare?
• Which templates and artifacts should you produce?
• How does AI risk assessment fit your existing governance?
• When should you reassess an AI system?
• What does AI risk assessment look like for a freelance PM?
• What mistakes should you avoid, and what is the 6-item checklist?
• How can we help you run AI risk assessments?
• FAQ
• Sources

Why do AI risk assessments matter for your projects?

AI risk is socio-technical. The same model can be harmless in one workflow and damaging in another, because the harm depends on who uses it, who is affected, and what decisions it feeds. NIST makes this point directly: trustworthy AI is contextual, and characteristics such as safety, privacy, and fairness may trade off against each other depending on purpose and stakeholders.

When you build your list of harms, cover these categories:

• Safety failures that cause physical, financial, or reputational damage
• Privacy exposure of client or personal data
• Unfair or biased outputs that affect specific groups
• Information integrity problems, such as fabricated facts presented confidently
• Security weaknesses, including prompt injection and data leakage
• Operational failures when a tool changes, breaks, or disappears

Proportionality keeps this manageable. Spend your deepest effort on systems whose failure would hurt people or the business, and give low-stakes tools a lighter pass. Do not build an isolated AI-only compliance silo that duplicates what your security and project risk processes already do.

How do the NIST AI RMF functions work in practice?

The NIST AI Risk Management Framework is voluntary and use-case agnostic. It organizes the work into four functions. Govern applies organization-wide, while Map, Measure, and Manage apply to a particular system and lifecycle stage. In practice, teams turn them into these actions:

1. Govern: Set policies, risk appetite, and clear roles. Decide who can approve an AI use case and who can stop one.
2. Map: Define scope, stakeholders, affected groups, system boundaries, data sources, and third-party dependencies.
3. Measure: Choose evaluation goals, metrics, test scenarios, and the evidence methods you will use to check trustworthiness characteristics.
4. Manage: Select mitigations, record acceptance decisions, set up monitoring, and define escalation paths.

The framework keeps moving. NIST’s materials show ongoing revisions and new profiles, so an assessment written today may reference guidance that has since changed. Always write down which edition and profile you used, and tailor the activities to your context rather than copying them wholesale. For generative systems, NIST’s Generative AI Profile maps suggested actions to RMF subcategories and treats some risks as unique to, or worsened by, generative AI.

Pro Tip: Add a “framework version” line to the top of every assessment document so a reviewer can tell in seconds which guidance you followed.

What is the step-by-step AI risk assessment checklist?

This sequence takes you from scoping to a documented decision. It follows the practical flow NIST describes: define purpose, identify harms, measure, decide, and document.

1. Define the system. Write down its intended purpose, users, affected people, operating context, and boundaries.
2. Inventory components. List data sources, models, third-party services, and integrations.
3. Identify foreseeable harms. Map each harm to the groups it could affect.
4. Analyze likelihood and impact. Set proportional risk tiers and acceptance criteria before you score anything.
5. Choose controls and measurement methods. For high-consequence systems, add ARIA elements: model testing, red teaming, and user testing.
6. Decide on residual risk. Assign an owner, and record the mitigations and monitoring that justify the decision.
7. Publish and schedule. Share the risk register, then set a review cadence and escalation paths.

Keep a few habits throughout:

• Name a human who is accountable for the acceptance decision
• Record uncertainty explicitly instead of hiding it behind a score
• Involve someone who represents affected users, not only technical staff
• Store evidence where an auditor or client can find it later

How do model testing, red teaming, and user testing compare?

NIST’s ARIA evaluation planning manual describes combining three methods, and for high-consequence systems, benchmarks alone are not enough. The full manual also recommends defining a single target concept per evaluation.

• Model testing: Automated prompts and benchmarks give repeatable coverage across many scenarios. They are fast and cheap, but they only test what you thought to include.
• Red teaming: Adversarial human testers hunt for failure modes you did not anticipate, such as jailbreaks or misuse paths. It is slower and depends on tester skill.
• User testing: Observing real interactions catches contextual harms and usability-driven failures that neither of the other methods will surface.

A defensible plan states its goals, scenarios, metrics, human testers, acceptance thresholds, and evidence-retention approach.

Pro Tip: Run model testing first to clear obvious defects, then spend your scarce human tester time on red teaming and user sessions.

Which templates and artifacts should you produce?

Four artifacts cover most needs, and each should be easy to audit.

• Risk register: system identifier, scope, harm description, affected groups, likelihood, impact, controls, residual risk, owner, and review date
• Scoring and tiering guide: descriptive tiers such as prohibited, high, limited, and minimal. Risk-analysis research warns about false precision: a numeric score can look rigorous even when probability or severity is poorly understood, so note your uncertainty and keep a human decision-maker accountable.
• Evaluation plan: scope, target concepts, test data, metrics, human testers, red-team objectives, acceptance criteria, and evidence retention
• Monitoring log: observed incidents, metric drift, and the trigger events that force a reassessment

University guides can serve as starting points, including the University of California AI Council guide for procurement and development reviews.

How does AI risk assessment fit your existing governance?

Treat the assessment as one input to the risk processes you already run. Assign an accountable decision-maker and a system owner for each AI use case, then feed outputs into your existing risk registers instead of keeping a parallel one.

• Align AI controls with your security and privacy programs, and reuse established control sets where they apply
• Embed proportional AI checks into procurement, onboarding, and project gates
• Use ISO/IEC 23894:2023, which offers internationally applicable guidance for integrating AI risk management into existing organizational activities and can be customized to context

Regulation adds another layer. The EU AI Act uses a risk-based structure that separates prohibited practices, high-risk requirements, and transparency duties, and whether it applies to you depends on your role, system, and use case. For a practical walkthrough, see this EU AI Act compliance checklist. Confirm applicability for your jurisdiction rather than assuming it.

When should you reassess an AI system?

An AI risk assessment is not a one-time checklist. NIST’s Generative AI Profile points to reassessing after material changes to data, model, prompts, integrations, users, or operating environment. Typical triggers are a model update, a shift in data sources, a new integration, a new user group, or a new operating context.

Between triggers, set a periodic review: quarterly for higher tiers, annually for lower ones. Define the metrics you will watch, such as error reports and escalations. Record the framework edition and profile with each review so the history stays auditable.

What does AI risk assessment look like for a freelance PM?

Consider a freelance PM who uses an AI assistant to summarize client meetings and draft status reports. The assessment starts with scope: the tool sees client names, budgets, and sometimes confidential strategy. The foreseeable harms are leaked client data, a fabricated action item that someone acts on, and a summary that quietly drops a dissenting voice. Impact is moderate, so the PM tiers it as limited, with controls proportional to that: no confidential attachments, human review of every summary before it goes out, and a note in the client contract that AI assists drafting.

A second example is a consultant recommending an AI scheduling tool to a client team. Here the risk shifts to procurement, so the consultant inventories the vendor’s data handling, runs a small pilot with real users, and documents the residual risk and owner. In both cases, a concise register and a regular review date can help keep the assessment manageable.

Your prompts matter too. Our guide to prompt templates and a shared prompt library shows how to version them so prompt changes become reassessment triggers. If you also use AI during discovery, our piece on AI for requirements gathering covers templates worth reviewing.

What mistakes should you avoid, and what is the 6-item checklist?

Most failures are process failures: treating the assessment as a one-time form, trusting opaque numeric scores, and isolating AI governance from the rest of the business. I would add one more: skipping the people affected by the system, because they find the problems your test suite cannot. Good assessments are small, owned, and revisited. This checklist follows NIST and ARIA guidance.

• Scope the system clearly
• Include affected users
• Pick proportional controls
• Combine testing methods for evidence
• Assign owners
• Set a monitoring and review cadence

— Ashanti

How can we help you run AI risk assessments?

Assessments get easier when you have templates, peers, and someone to pressure-test your approach. At The Freelance PM Club, we offer coaching, a library of templates and tools, and a community of independent project managers and consultants who are working through the same questions. Our freelance PM trends guide explains why governing AI use is becoming part of the job. Explore our coaching and membership options to get hands-on help building your first risk register.

FAQ

What are the main types of AI risk?

There is no single official list of four. NIST’s framework centers on trustworthiness characteristics: validity and reliability, safety, security and resilience, accountability and transparency, explainability, privacy, and fairness. In practice, teams group harms into categories such as safety, privacy, fairness, information integrity, security, and operational failure.

How do you do an AI risk assessment?

Define the system’s purpose, users, and boundaries, inventory its data and components, identify foreseeable harms, and rate likelihood and impact against acceptance criteria. Then choose controls, decide whether residual risk is acceptable, assign owners, and document everything with a review date.

Can AI write a risk assessment?

AI can draft a risk register, suggest harms, and structure documents, but it cannot own the decision. Research on risk analysis cautions that scores can create false precision, so an accountable human must review the output and approve acceptance.

Is there a 30% rule for AI?

We found no authoritative framework that defines a 30% rule for AI, and neither NIST nor ISO guidance uses one. Treat any such rule of thumb as a local policy choice, and base your decisions on documented tiers and acceptance criteria instead.

Sources

• AI Risk Management Framework | NIST
• EUR-Lex - Official Journal of the European Union
• Artificial intelligence for risk analysis—A risk characterization perspective on advances, opportunities, and limitations - PMC

Recommended

• AI for Requirements Gathering: 5 Steps & Templates for BAs & Devs
• 6 Prompt Templates for PMs and How to Build a Shared Prompt Library