What a CSA Audit Actually Asks For
·
The most common preparation failure is not a missing control. It is an agency that has the control, has the policy describing the control, and cannot demonstrate the control ran.
Under the legacy policy-area structure that was frequently enough. Under the 800-53 structure it is not, and the difference shows up in the specific artifacts an assessor asks for.
The shape of the change
Compare how two controls are written.
The pre-modernization framing asked whether the agency had a policy covering an area, and whether personnel had been trained. Both are answerable with a document and a roster.
The modernized framing asks whether the control operates. CA-7 Continuous Monitoring requires an established frequency for monitoring and for assessment of control effectiveness, and ongoing control assessments in accordance with the strategy. “Ongoing” is not satisfiable by a document. It is satisfiable by records showing the thing happened, repeatedly, on the stated frequency.
That is an audit of operation, not an audit of intent.
Four concrete contrasts
Training
What agencies bring: a completion report showing 34 of 35 personnel completed security awareness training this year.
What the control asks: AT-4 Training Records requires the agency to document and monitor information security and privacy training activities, including security and privacy awareness training and specific role-based security and privacy training, and to retain individual training records for a minimum of three years.
Two gaps open up. Role-based training is a separate obligation from general awareness training, and a single completion report usually evidences only the latter. And three years of retention means the assessor can ask for the prior two cycles, not just the current one.
The stronger position: training records that show role assignment, the role-matched curriculum, completion date, and a retention trail covering the triennial window.
Access control
What agencies bring: a written access control policy and a current user list.
What the control asks: AC-2 Account Management requires the agency to define and document the types of accounts allowed and specifically prohibited, assign account managers, and require conditions for group and role membership.
The prohibited-account clause catches agencies out. A policy that describes which accounts are permitted, without stating which are prohibited, does not meet the control as written. Shared accounts, generic service accounts, and vendor-held administrative accounts are the usual subjects.
Named account managers are a second gap. The control asks who is accountable for each account type, by name or role, not that accounts are managed in general.
The stronger position: the documented account taxonomy including prohibitions, named account managers, and evidence of the periodic review actually occurring on its stated frequency.
Vendor and external service oversight
What agencies bring: a signed CJIS Security Addendum from each vendor.
What the control asks: SA-9 External System Services requires the agency to define and document organizational oversight and user roles and responsibilities for external system services, and to employ processes to monitor control compliance by external service providers on an ongoing basis. It requires, at a minimum, triennial audit of all external service providers with access to the information system.
The Addendum is the contractual instrument. It is not the oversight process, and it is not the audit. An agency with five signed Addenda and no vendor audit record has evidence of a contract, not evidence of a control.
This is the obligation small agencies most often do not know they hold, because it runs in the opposite direction from the one they expect. The CSA audits the agency. The agency audits its vendors.
The stronger position: a vendor register listing who holds CJI access, the oversight owner for each, and dated audit records within the triennial window.
Continuous monitoring
What agencies bring: confirmation that logging is enabled.
What the control asks: CA-7 requires a system-level continuous monitoring strategy including established metrics, an established frequency for monitoring and for assessing control effectiveness, and ongoing assessments and monitoring in accordance with that strategy. It names 20 specific metrics.
Logging being enabled is a prerequisite, not the control. The control is the strategy, the frequency, and the record that the monitoring and assessment occurred at that frequency.
The stronger position: a written strategy naming the metrics and frequencies, plus the monitoring output for enough consecutive periods to demonstrate the frequency was honoured.
The pattern underneath all four
Every one of these has the same shape. The agency holds the artifact that proves the control was established. The control asks for evidence it operates.
Establishment evidence is a policy, a signed agreement, a configuration setting, a completion report. It is generated once.
Operation evidence is a record produced each time the control runs. It is generated repeatedly, and it carries dates.
The second kind cannot be assembled the week before an audit. That is the whole difficulty. An agency that discovers the gap at audit has no remedy, because the missing artifact is a record of something that should have happened in the past.
What an assessor is actually testing
Three questions, in order, on any control.
Does the control exist as designed. This is where the policy document belongs and it is the easiest to satisfy.
Did the control operate during the period. This requires dated records covering the review period, not a snapshot.
Would a failure have been detected. This is where CA-7's monitoring obligation and CA-5's plan of action and milestones connect. A control that operates but whose failures go unnoticed is a control with no assurance behind it.
Most agencies prepare thoroughly for the first question and not at all for the third.
Practical preparation
Pick your five highest-priority controls. For each, write down the specific artifact you would hand an assessor to demonstrate it operated during the last twelve months. Not the policy. The record.
Where you cannot name the artifact, you have found a gap, and you have found it with time to start generating the record rather than at the audit.
That exercise takes an afternoon and is more useful than most maturity assessments.
Written by Dr. Alexis P. Weekes
EDBA-IS · J.M. · CISA · CISM · CRISC · CDPSE
Founder, CJIS360 LLC
Sources: CJIS Security Policy v6.1, 06/25/2026 — AC-2 Account Management; AT-4 Training Records; CA-5 Plan of Action and Milestones; CA-7 Continuous Monitoring; SA-9 External System Services.