All articles
federal7 min read

How FedRAMP authorization changes for AI enabled services

The significant change request is being replaced by a three tier notification model, mandatory for Rev5 services in January 2027. Here is what actually moves in a package when inference enters the boundary: where the line falls, which controls stop inheriting, and what it does to your timeline.

By Andriy Markiv, Goatechs LLC

Share LinkedIn X Facebook

 

The mechanism most teams plan around when they add a large language model to an authorized system, the significant change request, is on its way out. FedRAMP now sorts changes into three tiers, and for two of them you no longer wait on government approval before you start. For Rev5 cloud services the new notification rules become mandatory on January 1, 2027, which turns this from background reading into a dated planning problem.

That is the headline. The rest of this is about how FedRAMP authorization changes for AI enabled services once inference is actually inside the boundary, because the documentation burden moves in directions a commercial compliance program does not prepare you for.

30 business days
advance notice before a transformative change
10 business days
after the fact for an adaptive change
Jan 2027
when notification becomes mandatory for Rev5 services

Why a model does not behave like your other components

Everything else inside your boundary is specified by the code you wrote. A model is not. Its behavior is a property of weights you did not author, it is nondeterministic by design, and if you consume it as an API the provider can change it underneath you on a Tuesday.

That single property is what breaks the assumptions FedRAMP was built on. The framework assumes a boundary you can draw and behavior you can reproduce. Inference gives you neither for free.

Where the boundary line actually falls

The first question is simple to ask and decides most of the rest of the work.

Who runs the inference?
A commercial model API
Outside your boundary
You document it as an external dependency: what goes out, what comes back, and how you enforce data handling at the edge.
Your own infrastructure
Inside your boundary
Weights, inference runtime, serving endpoint and data pipeline all become components you secure, monitor and version.
Either answer is defensible. Only one of them is cheap to document, and it is not the one most teams expect.

Self hosting looks like the heavier option and in control terms it often is not, because you own a known quantity instead of describing a moving one. A commercial API removes the infrastructure work and replaces it with an evidence problem.

Worth checking before you assume the API route is unauthorized territory: some managed model services run inside a cloud boundary that already holds an authorization, and some do not. That is a question about which boundary the service sits in, not about the model, and the answer changes what you have to write.

Retrieval widens the question. A retrieval augmented system has more moving parts than the model, and each one lands on a side of the line.

A RAG system, split by where each piece lives
Inside the boundary
  • Retrieval and ranking logic
  • Prompt assembly and templates
  • Output validation and filtering
  • Logging of every inference call
  • A self hosted vector store and its backups
Outside, and still your problem to describe
  • Model weights held by a provider
  • The provider's inference fleet
  • A managed vector store in someone else's environment
  • Any embedding API you call out to
A vector store hosted outside your environment is the one that catches people, because it is where the sensitive text actually sits.

If the vector store is hosted outside your environment, the question an assessor will ask is not whether it is secure. It is whether controlled unclassified information or personal data is reaching a system that has no authorization to hold it.

What you inherit, and what stays yours

Control inheritance is the thing that makes FedRAMP survivable. Run on an authorized platform and you stop having to describe data center locks and hypervisor segmentation. Add a model and the inheritance map has to be redrawn, because the platform never inherits the part that matters.

Who owns which control
Commercial model APIManaged service on an authorized cloudSelf hosted open weights
Physical and networkinheritedinheritedinherited
Host hardening and patchingproviderinheritedyours
Model weights and versionproviderprovideryours
Inference runtimeprovidersharedyours
Input sanitizationyoursyoursyours
Output validationyoursyoursyours
Inference loggingyoursyoursyours
Detecting a model changeyoursyoursyours
Every row below the first is yours in at least one column. The infrastructure is the only thing that inherits cleanly.

Read the bottom four rows again. Nothing inherits them, under any hosting choice. Whatever you pick, you are writing those controls yourself, which means the hosting decision buys you less than the architecture diagram suggests.

The three tiers that replaced the change request

This is the part that has genuinely moved. Instead of one process gated on advance approval, changes now fall into three categories with different obligations.

How a change gets handled now
Notice to giveAgency reviewTypical AI example
Routine recurringNoneNot requiredReindexing a corpus you already authorized
AdaptiveWithin 10 business days, afterRequiredSwapping an embedding model inside an authorized pattern
Transformative30 business days, beforeRequiredIntroducing generative inference to a service that had none
Impact categorization changes are the exception and still run the legacy process with advance approval.

The practical consequence is that the categorization argument now happens earlier and matters more than the paperwork that follows it. Getting a change called adaptive rather than transformative is worth real weeks.

Two caveats before you build a plan on this. Changes to a system's impact level still go through the legacy route with advance approval, and the wider FedRAMP 20x work that this sits inside is still moving, including a proposed replacement for the boundary guidance itself. Treat the tiers as firm and the surrounding machinery as a draft.

One thing that has not changed: a model alias that silently points at a new version is a change you did not notice. Pinning versions is a compliance control, not a preference.

Where continuous monitoring quietly fails

Continuous monitoring is where the work gets underestimated, and the reason is structural rather than lazy. Existing monitoring watches infrastructure and security events. A model failure is usually neither.

Take a mid sized claims processor that indexes adjuster notes for retrieval. Nothing in a normal monitoring stack is pointed at the thing most likely to go wrong.

The same two weeks, read two ways
What the monitoring dashboard showed
Every inference call returned 200, latency flat
What was actually happening
A share of responses carried verbatim text from indexed notes
What the monitoring dashboard showed
No security events, no failed scans
What was actually happening
Retrieval was pulling documents the caller had no right to see
What the monitoring dashboard showed
No user complaints in the queue
What was actually happening
Nobody downstream had been told what a leak would look like

Every signal on the left is green. None of them is measuring the output, which is the only place the failure was visible.

The fix is unglamorous and has to be designed in, because you cannot retrofit a baseline you never captured.

The monitoring loop an AI workload actually needs
1Sample outputs
On a fixed schedule, not when someone remembers
2Check against policy
Automated filters first, human review on the sample
3Compare to a baseline
A stored test suite you can rerun against the live model
4Investigate drift
A behavior change is a finding even when nothing broke
5Notify and document
Which tier the change falls into decides your clock

What this does to your timeline

Published timelines vary enough that any single number is marketing, and most of the figures in circulation come from firms with a stake in the answer. The ranges below are planning windows rather than measurements, and they are wide on purpose.

Planning windows, new work in scope
Adding inference, controls already built
2 to 4
Assumes logging, output validation and a model evaluation process already exist.
Adding inference, controls already built: 2 to 4 months
Adding inference, starting the controls now
6 to 12
The gap is almost entirely documentation and evidence that did not exist yet.
Adding inference, starting the controls now: 6 to 12 months
A new Moderate authorization with AI in scope
12 to 18
A new Moderate authorization with AI in scope: 12 to 18 months
061218
months
Commonly cited industry ranges, not Goatechs measurements. FedRAMP's own reporting has shown authorizations running beyond a year.

The distance between the first two bars is the whole argument of this article. It is not a harder review, it is the same review arriving before you were ready for it.

One cost that catches budgets late: an assessor may want adversarial testing of the model, which means prompt injection, jailbreaking and data exfiltration through outputs. That is a different skill set from network penetration testing, and the firms that do it well are fewer.

Frequently asked questions

Can I use a commercial model API inside a FedRAMP authorized system?

Often yes, but the answer depends on which cloud boundary the model service runs in rather than on the model itself. Some managed model services sit inside a cloud environment that already holds an authorization, which changes the conversation entirely. Where that is not the case, the API is an external dependency: you document the data flows, show that controlled unclassified information and personal data are not crossing the line, and carry the input sanitization, output validation and logging yourself.

What happens if the provider updates the model without telling me?

That is a continuous monitoring failure waiting to be found, and the defense is a stored baseline. Run a test suite against the service on a schedule, compare outputs to what you recorded at assessment time, and pin to explicit model versions rather than aliases that can move. If you detect a meaningful behavior change, decide which change tier it falls into, because that decides whether you owe notice in ten business days or thirty before the fact.

Does training data have to be inside my authorization boundary?

If you train or fine tune, yes: the training data is yours, and you document its provenance, how it is secured, and how you established that it holds nothing that should not end up reflected in outputs. If you use a pretrained model, the training data is not in your boundary, but you are still expected to describe what you know about it and what testing you did for leakage and policy compliance. Not knowing is an answer, as long as you say so and show the compensating controls.

How do you document a control for nondeterministic behavior?

You write it probabilistically, because a test that asserts an exact output will fail intermittently and teach your team to ignore it. That means sampling outputs and measuring the rate at which they meet policy, filters that catch violations regardless of what the model produced, and a defined fallback when validation fails, such as a safe default response or escalation to a person. The goal is to show you have mapped the failure modes and can detect and respond to them, not to claim you have prevented them.

What to do before you open the conversation

None of this is hard in the way a novel algorithm is hard. It is hard in the way that showing your work is hard, and almost all of it can be built before an authorizing official is in the room.

Build these first
  • Name the model as a component in the system security plan, with its version pinned
  • Draw the data flows in and out, including every retrieval hop
  • Capture a behavioral baseline you can rerun later
  • Ship input sanitization and output validation as code, not as policy text
  • Log every inference call in a form a reviewer can sample
  • Write the rollback for a model version that turns out to be worse
  • Decide, in advance, which changes you will treat as transformative

Teams that arrive with those seven things get a conversation about risk. Teams that arrive with a plan and a promise get a conversation about schedule.

If you are working out how FedRAMP authorization changes for AI enabled services in your own architecture, and what that does to your timeline and your budget, get in touch.

fedrampfederal procurementai authorization