5.2Evaluating operations and maintenance controls

A system is operated for a long time after go-live, with daily job execution, program changes, incident response, and backups continuing. The auditor evaluates whether controls are designed into these operations and maintenance processes, records (trails) are kept, and they actually operate. In particular, the focus is on the effectiveness of controls that tend to become hollow in the field—whether changes that skip approval or records under the name of emergency response are rampant, and whether backups are not only taken but verified to be restorable—confirmed using evidence trails.

5.2.1Key controls in operations and maintenance

• Change management controls program/configuration changes to production through the sequence request → impact analysis → approval → testing → application → recording. The auditor reviews change requests and approval trails as evidence and evaluates whether changes without approval (uncontrolled changes) have reached production. For emergency changes too, it confirms that an approval/recording procedure—even after the fact—is defined and actually followed.
• Configuration management is a control that accurately grasps and maintains configuration information for hardware, software, and settings. The auditor confirms whether configuration information matches reality (reconciling the register against the actual assets), and if there is a discrepancy, suspects a missing detective control or hollowed-out operation. Job management concerns whether batch-job schedules, dependencies, and rerun procedures on abnormal termination are defined, and execution results (logs) are monitored.
• Incident management controls detection, recording, response to, and prevention of recurrence of incidents. The auditor evaluates whether incident records are kept without omission and whether escalation by severity and permanent countermeasures are implemented. For backups, what matters is not only periodic data capture but also verification that recovery from the captured backup is actually possible (restore testing) and a design such as offsite storage.

Suppose a systems auditor is evaluating the operating status of change-management controls in the operations department for a core system. On paper, the rule states that "all changes to production programs must file a change request and pass impact analysis and manager approval before application," and the design appears sound. But when the auditor reconciles the past six months of production-deployment records against the change requests, it emerges that many "emergency changes" cited for incident response were applied to production with neither a change request nor any after-the-fact approval record kept. The auditor diagnoses this as an operating deficiency: the rule is designed, but it is not actually operated as prescribed. However, since requiring the same full pre-approval in an emergency as in normal times would delay recovery and is unrealistic, the auditor does not reject the design but recommends making the control effective specifically for emergency changes. Concretely, they propose formalizing a procedure where an emergency change may be applied first at the field's discretion, but a change request must be filed within a set time after application and undergo after-the-fact manager approval and impact review, and monitoring whether this after-the-fact approval is ever missed by periodically reconciling the change log against the approval records (a detective control). In addition, regarding backup controls, the auditor does not take the operator's statement that "we back up daily" at face value, but confirms as evidence whether records of actual restore tests exist, and if capture logs exist but there is no evidence of recovery verification, flags it as "an operating deficiency where recoverability is not assured." In this way, the auditor's judgment in operations and maintenance control evaluation is not to judge effectiveness from the existence of rules alone, but to confirm operating effectiveness using trails as evidence, and to make effective the controls that tend to become hollow, such as emergency changes and recovery verification.

5.2.2Section summary

• The auditor evaluates change management by reconciling change requests against approval trails, diagnosing the operating deficiency where approval/recording is skipped for emergency changes
• For backups, the auditor confirms via trails that recoverability is verified by a restore test, not just captured
• Configuration management is confirmed by reconciling the register against actual assets; a divergence is flagged as a missing detective control or hollowed-out operation