Training Courses

Failure Scenarios – Critical Thinking

Download course PDF
SOP Training for Failure Scenarios SOP Training for Failure Scenarios

SOP training course length is customizable based on your needs.

Our SOP training focuses on facility standard operating procedures and prepares for failure scenarios. The successful operation of critical facilities depends on a well trained staff that can take appropriate action when a problem occurs in a data center and while keeping supervisory staff well informed. Knowledge of facility policies and procedure is a key factor in being able to respond appropriately to emergencies in a safe and timely manner.

The purpose of this SOP training course is to train operating staff on facility operating procedures and escalation policies by engaging is a series of staged activities based on possible real world situations. The class starts with a review of facility procedure manual along with emphasis on reporting and escalation requirements. Following the review section the attendees will be broken in to small groups and each group given a failure scenario. Each team would evaluate their failure and would report back to the class on actions they would take to mitigate the failure. Each failure scenario would be complicated by additional events that would force escalation and an evaluation on how the failure affects operation of the facility. The evaluation process would require each team to make use of facility documentation such as drawings and procedures to support their proposed actions.

This SOP training is appropriate for staff in critical facilities that are responsible responding to equipment or system alarms and their supervisors who receive the escalation reports. The failures scenarios used in the class could be based on concerns that the site management has identified.

Sample failure scenario

Building Automation System (BAS) alarms and reports water leak in the data center via the leak detection system.

  • The leak location is near a CRAC unit and identified as being from a chill water valve.
  • Chill water distribution system isolation valve jams/fails to isolate the leak.
  • The on call supervisor cannot be reached.

The team would be expected to provide appropriate actions for the initial problem and any complications and points at which escalation to a higher level would take place. At each step the team would also need to evaluate and report on the potential impacts to the operation of the critical processes in the facility.

Class Materials

  • Site operations manuals, Facility drawings

In Plant

  • In plant portions are possible if facility operations allow