How FedRAMP authorization changes for AI enabled services
The significant change request is being replaced by a three tier notification model, mandatory for Rev5 services in January 2027. Here is what actually moves in a package when inference enters the boundary: where the line falls, which controls stop inheriting, and what it does to your timeline.
By Andriy Markiv, Goatechs LLC
The mechanism most teams plan around when they add a large language model to an authorized system, the significant change request, is on its way out. FedRAMP now sorts changes into three tiers, and for two of them you no longer wait on government approval before you start. For Rev5 cloud services the new notification rules become mandatory on January 1, 2027, which turns this from background reading into a dated planning problem.
That is the headline. The rest of this is about how FedRAMP authorization changes for AI enabled services once inference is actually inside the boundary, because the documentation burden moves in directions a commercial compliance program does not prepare you for.
Why a model does not behave like your other components
Everything else inside your boundary is specified by the code you wrote. A model is not. Its behavior is a property of weights you did not author, it is nondeterministic by design, and if you consume it as an API the provider can change it underneath you on a Tuesday.
That single property is what breaks the assumptions FedRAMP was built on. The framework assumes a boundary you can draw and behavior you can reproduce. Inference gives you neither for free.
Where the boundary line actually falls
The first question is simple to ask and decides most of the rest of the work.
Self hosting looks like the heavier option and in control terms it often is not, because you own a known quantity instead of describing a moving one. A commercial API removes the infrastructure work and replaces it with an evidence problem.
Worth checking before you assume the API route is unauthorized territory: some managed model services run inside a cloud boundary that already holds an authorization, and some do not. That is a question about which boundary the service sits in, not about the model, and the answer changes what you have to write.
Retrieval widens the question. A retrieval augmented system has more moving parts than the model, and each one lands on a side of the line.
- Retrieval and ranking logic
- Prompt assembly and templates
- Output validation and filtering
- Logging of every inference call
- A self hosted vector store and its backups
- Model weights held by a provider
- The provider's inference fleet
- A managed vector store in someone else's environment
- Any embedding API you call out to
If the vector store is hosted outside your environment, the question an assessor will ask is not whether it is secure. It is whether controlled unclassified information or personal data is reaching a system that has no authorization to hold it.
What you inherit, and what stays yours
Control inheritance is the thing that makes FedRAMP survivable. Run on an authorized platform and you stop having to describe data center locks and hypervisor segmentation. Add a model and the inheritance map has to be redrawn, because the platform never inherits the part that matters.
| Commercial model API | Managed service on an authorized cloud | Self hosted open weights | |
|---|---|---|---|
| Physical and network | inherited | inherited | inherited |
| Host hardening and patching | provider | inherited | yours |
| Model weights and version | provider | provider | yours |
| Inference runtime | provider | shared | yours |
| Input sanitization | yours | yours | yours |
| Output validation | yours | yours | yours |
| Inference logging | yours | yours | yours |
| Detecting a model change | yours | yours | yours |
Read the bottom four rows again. Nothing inherits them, under any hosting choice. Whatever you pick, you are writing those controls yourself, which means the hosting decision buys you less than the architecture diagram suggests.
The three tiers that replaced the change request
This is the part that has genuinely moved. Instead of one process gated on advance approval, changes now fall into three categories with different obligations.
| Notice to give | Agency review | Typical AI example | |
|---|---|---|---|
| Routine recurring | None | Not required | Reindexing a corpus you already authorized |
| Adaptive | Within 10 business days, after | Required | Swapping an embedding model inside an authorized pattern |
| Transformative | 30 business days, before | Required | Introducing generative inference to a service that had none |
The practical consequence is that the categorization argument now happens earlier and matters more than the paperwork that follows it. Getting a change called adaptive rather than transformative is worth real weeks.
Two caveats before you build a plan on this. Changes to a system's impact level still go through the legacy route with advance approval, and the wider FedRAMP 20x work that this sits inside is still moving, including a proposed replacement for the boundary guidance itself. Treat the tiers as firm and the surrounding machinery as a draft.
One thing that has not changed: a model alias that silently points at a new version is a change you did not notice. Pinning versions is a compliance control, not a preference.
Where continuous monitoring quietly fails
Continuous monitoring is where the work gets underestimated, and the reason is structural rather than lazy. Existing monitoring watches infrastructure and security events. A model failure is usually neither.
Take a mid sized claims processor that indexes adjuster notes for retrieval. Nothing in a normal monitoring stack is pointed at the thing most likely to go wrong.
Every signal on the left is green. None of them is measuring the output, which is the only place the failure was visible.
The fix is unglamorous and has to be designed in, because you cannot retrofit a baseline you never captured.
What this does to your timeline
Published timelines vary enough that any single number is marketing, and most of the figures in circulation come from firms with a stake in the answer. The ranges below are planning windows rather than measurements, and they are wide on purpose.
The distance between the first two bars is the whole argument of this article. It is not a harder review, it is the same review arriving before you were ready for it.
One cost that catches budgets late: an assessor may want adversarial testing of the model, which means prompt injection, jailbreaking and data exfiltration through outputs. That is a different skill set from network penetration testing, and the firms that do it well are fewer.
Frequently asked questions
Can I use a commercial model API inside a FedRAMP authorized system?
Often yes, but the answer depends on which cloud boundary the model service runs in rather than on the model itself. Some managed model services sit inside a cloud environment that already holds an authorization, which changes the conversation entirely. Where that is not the case, the API is an external dependency: you document the data flows, show that controlled unclassified information and personal data are not crossing the line, and carry the input sanitization, output validation and logging yourself.
What happens if the provider updates the model without telling me?
That is a continuous monitoring failure waiting to be found, and the defense is a stored baseline. Run a test suite against the service on a schedule, compare outputs to what you recorded at assessment time, and pin to explicit model versions rather than aliases that can move. If you detect a meaningful behavior change, decide which change tier it falls into, because that decides whether you owe notice in ten business days or thirty before the fact.
Does training data have to be inside my authorization boundary?
If you train or fine tune, yes: the training data is yours, and you document its provenance, how it is secured, and how you established that it holds nothing that should not end up reflected in outputs. If you use a pretrained model, the training data is not in your boundary, but you are still expected to describe what you know about it and what testing you did for leakage and policy compliance. Not knowing is an answer, as long as you say so and show the compensating controls.
How do you document a control for nondeterministic behavior?
You write it probabilistically, because a test that asserts an exact output will fail intermittently and teach your team to ignore it. That means sampling outputs and measuring the rate at which they meet policy, filters that catch violations regardless of what the model produced, and a defined fallback when validation fails, such as a safe default response or escalation to a person. The goal is to show you have mapped the failure modes and can detect and respond to them, not to claim you have prevented them.
What to do before you open the conversation
None of this is hard in the way a novel algorithm is hard. It is hard in the way that showing your work is hard, and almost all of it can be built before an authorizing official is in the room.
- Name the model as a component in the system security plan, with its version pinned
- Draw the data flows in and out, including every retrieval hop
- Capture a behavioral baseline you can rerun later
- Ship input sanitization and output validation as code, not as policy text
- Log every inference call in a form a reviewer can sample
- Write the rollback for a model version that turns out to be worse
- Decide, in advance, which changes you will treat as transformative
Teams that arrive with those seven things get a conversation about risk. Teams that arrive with a plan and a promise get a conversation about schedule.
If you are working out how FedRAMP authorization changes for AI enabled services in your own architecture, and what that does to your timeline and your budget, get in touch.