Basic SAID Functions

Basic Concepts

• SAID node: detects, diagnoses, and rectifies faults on a device's modules in the SAID. SAID nodes are classified into the following types:

  • Module-level SAID node: defends against, detects, diagnoses, and rectifies faults on a module.
  • SAID-level SAID node: detects, diagnoses, and rectifies faults on multiple modules.

• SAID node state machine: state triggered when a SAID node detects, diagnoses, and rectifies faults. A SAID node involves seven states: initial, detecting, diagnosing, invalid-diagnose, recovering, judging, and service exception states.

• SAID tracing: The SAID collects and stores information generated when a SAID node detects, diagnoses, and rectifies faults. The information can be used to locate the root cause of a fault.

SAID

Fault locating in the SAID involves the fault detection, diagnosis, and recovery phases. The SAID has multiple SAID nodes. Each time valid diagnosis is triggered (that is, the recovery process has been triggered), the SAID records the diagnosis process information for fault tracing. The SAID's main processes are described as follows:

1. Defense startup phase: After the system runs, it instructs modules to deploy fault defense (for example, periodic logic re-loading and entry synchronization), starting the entire device's fault defense.

2. Detection phase: A SAID node detects faults and finds prerequisites for problem occurrence. Fault detection is classified as periodic detection (for example, periodic traffic decrease detection) or triggered detection (for example, IS-IS Down detection).

3. Diagnosis phase: Once a SAID node detects a fault, the SAID node diagnoses the fault and collects various fault entries to locate fault causes (only causes based on which recovery measures can be taken need to be located).

4. Recovery phase: After recording information, the SAID node starts to rectify the fault by level. After the recovery action is completed at each level, the SAID node determines whether services recover (by determining whether the fault symptom disappears). If the fault persists, the SAID node continues to perform the recovery action at the next level until the fault is rectified. The recovery action is gradually performed from a lightweight level to a heavyweight level.

5. Tracing phase: If the SAID determines the fault and its cause, this fault diagnosis is a valid diagnosis. The SAID then records the diagnosis process. After entering the recovery phase, the SAID records the recovery process for subsequent analysis.

SAID Node State Machine

The fault detection, diagnosis, and recovery processes of a SAID node are implemented through state machines.

Figure 1 Process of SAID node state transition

All state transition scenarios are as follows:

1. When detecting a trigger event in the initial state, the SAID node enters the detecting state.

2. If the detection is not completed in the detecting state, the SAID node keeps working in this state.

3. If a detection timeout occurs or no fault is detected in the detecting state, the SAID node enters the initial state.

4. When detecting a fault in the detecting state, the SAID node enters the diagnosing state.

5. If the diagnosis action is not completed in the diagnosing state, the SAID node keeps working in this state.

6. If an environmental change occurs in the diagnosing state or another SAID node enters the recovering state, the SAID node enters the invalid-diagnose state.

7. If the diagnosis action is not completed in the invalid-diagnose state, the SAID node keeps working in this state.

8. If no device exception is detected after the diagnosis action is completed in the diagnosing state, the SAID node enters the initial state.

9. If a device exception is detected after the diagnosis action is completed in the diagnosing state, the SAID node enters the recovering state.

10. If the recovery action is not completed in the recovering state, the SAID node keeps working in this state.

11. If the recovery action is completed in the recovering state, the SAID node enters the judging state.

12. If the judgment action is not completed in the judging state, the SAID node keeps working in this state.

13. If the service does not recover in the judging state and a secondary recovery action exists, the SAID node enters the recovering state.

14. If the service does not recover in the judging state and no secondary recovery action exists, the SAID node enters the service exception state.

15. In the service exception state, the SAID node periodically checks whether the service recovers.

16. If the service recovers in the judging state, the SAID node enters the initial state.