What's changed: Initial version
3.2Problem management
Covers how the purpose of problem management is to prevent recurrence through root-cause identification and a permanent fix (in contrast with incident management, whose purpose is rapid restoration), the flow of raising a problem from recurring faults or major incidents and performing root-cause analysis (RCA), the use of a known error database (KEDB) that accumulates causes and workarounds, and choosing between reactive (waiting for a fault) and proactive (getting ahead via trend analysis) problem management.
If the previous section's incident management is the role that "stops the bleeding and restores service first," then problem management is the role that "finds out why the bleeding occurred and ensures it never happens again." Merely getting by with a workaround each time the same fault recurs never removes the root. After clarifying the difference in purpose between incident and problem management, this section covers the judgments of at what point to promote an incident to a problem, how to conduct root-cause analysis, and how to feed the accumulated knowledge (the KEDB) back into the next incident response.
3.2.1The difference in purpose from incident management
| Aspect | Incident management | Problem management |
|---|---|---|
| Purpose | Rapid restoration of service (minimizing impact) | Prevent recurrence via root-cause identification and a permanent fix |
| Time horizon | Immediate, within the SLA outage ceiling | Deliberate cause analysis (immediacy is not required) |
| Handling of the cause | Restoration via workaround is acceptable even if the cause is unknown | Identify and eliminate the root cause (the known error) |
| Main output | A restored service and an incident record | A permanent fix and known errors/workarounds registered in the KEDB |
3.2.2Root-cause analysis and the KEDB
- A problem is the root cause (or potential cause) of one or more incidents. Recurring incidents or a major incident of unknown cause are raised as a problem, and root-cause analysis (RCA) (such as the 5 Whys or timeline analysis) is used to pin down the true cause.
- A known error is a problem whose root cause is identified and for which a workaround has been established. Accumulating these forms the known error database (KEDB). When a similar incident recurs, referring to the KEDB lets you skip cause investigation and immediately apply the known workaround, greatly shortening restoration time.
- A permanent fix often involves a configuration change and is therefore carried out in coordination with change management (in the design/transition area). Once the permanent fix is complete and the cause is eliminated, the problem can be closed.
3.2.3Reactive and proactive
- Reactive problem management is an after-the-fact activity that analyzes the cause of an incident after it occurs and prevents recurrence. It is triggered by a fault happening.
- Proactive problem management is a preventive activity that discovers latent problems that have not yet surfaced from incident trend analysis and log/monitoring data, and acts before a fault occurs. It takes action from signs such as "the same warning is increasing" or "disk usage is trending upward."
Most-tested: "the purpose of problem management is preventing recurrence via root-cause identification and a permanent fix (rapid restoration is incident management)," "a known error is a problem whose cause and workaround are identified, and referring to the KEDB shortens restoration on recurrence," and "proactive problem management prevents latent problems in advance through trend analysis." The biggest pitfall is confusing the purposes of incident and problem management—always distinguish restoration from recurrence prevention as a pair.
Suppose a service manager notices that over the past month an incident—"a particular web service becomes unresponsive at night and recovers on restart"—has recurred five times. Each incident response has rapidly restored service with the workaround of a restart to stay within the SLA outage time, and that in itself is the correct call. But since the same fault keeps recurring, merely getting by with restarts is not a fundamental solution. The service manager therefore raises this cluster of recurring incidents as a single problem and moves it to problem management. Pursuing root-cause analysis (RCA) reveals that the nightly backup job contends with the application's memory-release processing, causing a memory leak that leads to unresponsiveness. Once the cause (the memory leak) and a workaround (restart, or shifting the nightly backup window) are established, this is registered in the KEDB as a known error. Thereafter, if a similar incident recurs, the service desk can refer to the KEDB, skip cause investigation, immediately apply the workaround, and shorten restoration time. The eventual permanent fix (correcting the application's memory-release processing) involves a configuration change and is carried out in coordination with change management; once the cause is eliminated, the problem is closed. The service manager also proactively inspects monitoring data for similar upward memory-usage trends in other services, catching latent problems before they turn into faults. "Prioritize restoration, yet catch recurring signs to eliminate the root cause as a problem, and leave the knowledge in the KEDB"—this is the judgment of a service manager who correctly links incident and problem management.
Trap: "Even if the same fault keeps recurring, problem management is unnecessary as long as a workaround rapidly restores service each time" is wrong—restoration (incident management) is correct for meeting the SLA, but as long as it recurs, recurrence will not stop unless problem management identifies the root cause and applies a permanent fix. Also wrong: "the purpose of problem management is to rapidly restore individual incidents"—that is the purpose of incident management; the purpose of problem management is preventing recurrence by eliminating the root cause.
3.2.4Section summary
- The purpose of problem management is preventing recurrence via root-cause identification and a permanent fix, forming a clear pair with incident management, whose purpose is rapid restoration
- Raise recurring incidents as a problem, identify the true cause with RCA, and register the cause and workaround in the KEDB to shorten restoration on recurrence
- Distinguish reactive (eliminating causes after a fault) from proactive (getting ahead of latent problems via trend analysis)
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. The same service outage has recurred five times in a month, each time rapidly restored by a server restart, and the SLA has been met. What is the most appropriate next action for the service manager?
Q2. A service manager analyzes past incident trends and an upward disk-usage trend from monitoring data, finds a latent bottleneck that has not yet caused a fault, and addresses it in advance. What best describes this activity?
Q3. As a result of problem management, the root cause of a service outage is identified and a reliable workaround is established. What is the most appropriate operational benefit of registering this knowledge in the KEDB (known error database)?