5.1Evaluating planning and development controls

A systems auditor is not the party building the system, but the party that independently evaluates whether appropriate controls are designed into each phase of development and actually operating. In the planning and development process, each stage—project management, requirements definition, testing, migration, and production release—carries risk, and each is designed with preventive controls (which prevent errors or risks from arising at all) and detective controls (which detect errors that have arisen). This section covers how the auditor evaluates both the design (is the design sound?) and operating effectiveness (is it carried out as designed?) of these controls, and diagnoses a deficiency by distinguishing whether it is a design deficiency or an operating deficiency.

5.1.1Controls in each development phase and audit focus

• Controls over requirements definition concern whether requirements are agreed with the user department, approval records are kept, and changes are managed. The auditor checks for approval trails on the requirements specification and whether requirement changes have expanded without control (scope creep).
• Controls over testing concern whether test plans, test cases, and result records are in place and cover the requirements. The auditor reviews test evidence (who verified what and when) and evaluates whether unresolved defects remain when release proceeds.
• Controls over migration and production release concern pre-release approval, rollback procedures, and verification of migration from legacy data. Typical examples are a preventive control ("require a manager's approval before applying to production") and a detective control ("detect anomalies by reconciling record counts and amounts after migration").

5.1.2Segregation of duties between development and operations (core preventive control)

• Segregation of duties (SoD) is a preventive control that prevents unilateral fraud or error by dividing duties that should check one another among different people. In development, the core is separating the "developer (who creates and changes programs)" from the "operator (who applies and runs them in production)."
• If this separation is missing, a developer can apply their own changes directly to production without verification, creating the risk of deliberately injected malicious code, or unverified code running in production without review or approval. The auditor first checks whether developers hold direct write access to production, and if so, flags this as a design deficiency (the control's design itself is inadequate).

Suppose a systems auditor conducts a site visit and interviews to evaluate the development setup of a core system. While confirming the production-deployment procedure, the auditor discovers that a developer holds authority to release programs they themselves created or changed directly into production, without any third-party review or approval by an operator. The auditor diagnoses this not as an "operating deficiency" (where a control exists but was not followed on some occasion), but as a design deficiency: the control design itself lacks any check. The reason is that this setup structurally allows unverified and unapproved code to run in production at one person's discretion, and no mechanism was designed in from the outset to prevent injection of malicious code or a serious bug slipping through. As an improvement recommendation, the auditor proposes establishing the preventive control of segregation of duties between development and operations—specifically, not granting developers direct write access to production, and having an independent operator perform production deployment only for reviewed and approved release artifacts. In addition, complementing the preventive control, the auditor recommends a detective control: capturing a change history of production (who deployed what and when) and periodically reconciling and monitoring it for deployments that violate segregation of duties. In this way, rather than merely stating that "a control is missing," the auditor's judgment in control evaluation is to distinguish which risk the control should prevent, whether the deficiency is one of design or of operation, and to recommend improvement using the corresponding control type (preventive or detective).

5.1.3Section summary

• The auditor evaluates controls in each planning/development phase on both design and operating effectiveness, distinguishing a deficiency as a design deficiency or an operating deficiency
• Segregation of duties between development and operations is the core preventive control; a setup where developers can release directly to production is flagged as a design deficiency
• Improvement recommendations combine the preventive control (approval, SoD) and detective control (reconciliation/monitoring of change logs) that address the risk