A detailed alarm management report as a tool for the treatment of bad actors at a gas logistics plant

2.1 Designing alarms for safety

Historically, several industrial disasters have been related to inadequate alarm system performance. In the oil and gas industry, the Longford gas explosions, the Milford Haven Refinery, and the Buncefield Oil Storage Depot are the most representative events of poor alarm system performance. The deterioration of alarm performance is related to announcement failures, alarms announced outside the ideal time range, an avalanche of alarms that prevents the plant operator from diagnosing the situation, or alarm avalanches that cause some critical alarms to not be announced [1, 4].

To prevent these issues, several efforts have been made to improve alarm systems, synthesized as alarm management [1]. From a wider perspective, the alarm rates and interface are elements in which diverse organizational and human factors play an important role in industrial safety. Therefore, operating experience feedback (OEF) is essential in the alarm management process, including work system design and modifications aimed at safety improvement [16, 20]. Recent studies based on empirical data have reinforced the importance of tacit knowledge in the construction of safety and how these aspects interact with explicit rules [15].

A qualitative study on safety in the oil industry highlighted the importance of operational procedures – work prescriptions- as the most frequent organizational factor influencing safety. The data were collected through interviews with workers from diverse backgrounds and positions within the organizations involved in fatal accidents [17]. As OEF can help reduce the gap between prescriptions and real work constraints, it allows the practical knowledge of plant operators to support the enhancement of alarm systems [6].

Interdisciplinary by definition, the AMC raison d’être is to conciliate multiple points of view and logic, to propose suited changes in the system. Sometimes referred to as debates or dialogs, the idea of discussing work is an essential resource for interventions of this kind. To avoid any consideration based on common sense, we will use the term proposed by Rocha [8] – structured debate – since it implies a situated and planned activity. But it may take less structured forms, as identified by Leuridan [18].

We agree with Leuridan’s [18] considerations about the actual state of the art not allowing further explanations on the inner workings and different forms of debates, as it may contribute to transforming work. Yet, the literature allows us to describe this process and characterize conditions that may foster the expected results.

The application of structured debate by organized groups can improve health, safety, and efficiency by considering real work conditions and constraints [12, 21]. Rocha et al. [5] stress the principle that long-term benefits can be reaped by the implementation of solutions provided by those on the lower levels of the organization’s hierarchy, called as principle of subsidiarity.

Structured debates should be centered on real work activities, taking into consideration the activity ergonomics approach [16, 21]. It is suggested that reference to real work situations must be frequent to support the debates, but should not aim for an exhaustive detailing of the activity. The debates should be part of a long-term strategy offering consistent results and evaluations contributing to the reliability of the method.

Structured debates require a designated work debate space and their institutionalization is based on the assessment of the situation through observations and interviews with workers. In this way, goals can be adjusted and workers can be mobilized. Additional support is also required, such as technical and human resources [8, 9].

The configuration of these spaces has been described by Detchessahar et al. [23] in two dimensions: the material substrate, which includes the environment and devices available for the debate; and the conventional substrate, which includes the knowledge of the participants and the adopted norms and behavior of the debate. Instead of making an exhaustive list of requisites, Detchessahar [23] suggests a reflection on how these elements may hinder or foster the debates, allowing the process to be optimized. Rocha et al. [9] argues for the necessity of incorporating structured debates into existing managerial meetings. This implies the versatility of this procedure and introduces the possibility of implementing structured debates without the need for additional meetings.

The use of structured debates brings a wide range of potential improvement opportunities in addition to the initial goals set. When plant operators and other professionals participate in debates, the opportunity to make further reflections about real work practices allows for the development of expertise and builds a collective work experience. The sharing of multiple perspectives allows workers to respond better to real-time situations, improving performance on related issues. It is also possible for participants to develop a more proactive attitude, allowing the anticipation of probable future issues [8, 20].

In an organizational culture approach, Leuridan [18] posits that debates may contribute to the reliability and resilience of a plant through feedback on discussed situations. In addition, structured debates may shed light on important but not well-characterized activities, which the author classifies as “interstitial” activities [24]. These activities are not focused on traditional approaches to risk assessment and reliability, but play an important role in these properties.

2.2 Method

This study conducted exploratory qualitative research through the development of a case study. The first part of this Section presents the context in which the practical need arose and the approach to the solution over time – the development of an Alarm Management Report (AMR) to determine AMC decisions through structured debates. The second part presents the AMC meeting routine, composition, and goals, and the procedures of data collection. The last part of the protocol, using qualitative content analysis, extracts and analyzes the AMC’s actions to evaluate the use of the AMR and its influence on the proposed changes [25].

2.3 Context of the case study

The case addressed in this study was in a gas logistics plant. The main objective of the plant is to store and move gaseous hydrocarbons, such as liquified petroleum gas (LPG), from and to tanks, pipelines, and ships. This requires the operation and storage conditions to be monitored. In particular, the operation of transferring gases from pressurized storage (at room temperature) to refrigerated storage (between –25°C to –43°C) requires control of flow and temperature to keep the tank inside operational limits, which requires close monitoring.

Since August 2019, AMCs have been installed in several units of said oil and gas logistics company by the director’s determination. The company’s standard for alarm management was inspired by ANSI-ISA 18.2, which uses the same process structure of the mentioned standard along with adaptations and improvements. At the gas logistics plant considered, AMC meetings were held monthly.

Regarding the conditions for a structured debate [8, 9], the monthly meeting already had characteristics of a work debate space, such as institutionalization, well-defined goals, and availability of technical and human resources [8, 9], including workers involved in the plant operation and maintenance. The structured debates aimed to investigate the characteristics of bad actors and applied techniques to reduce or remove unwanted announcements from the system.

Prompts are virtual points of control that are linked to sensors on the plant and may trigger when a certain value or signal is reached. If the prompt was set as an alarm, it was announced to the plant operator when triggered. Therefore, the AMC was required to decide whether an alarm needed to exist, and if so, on which values should it be set. The focus on handling signals and adjusting the absolute and variation limits is important, but may be insufficient to resolve issues of decision-making and interpretation in real work situations, as noted by the AMC.

With the impossibility of resorting to monthly ergonomic work analysis [22] or another method for analyzing real work situations, an alarm management report was developed. It discussed the list of alarms with plant operators to build a work context or scenario and was used to guide the debates of the AMC with elements of real work.

2.4 Alarm management report building

In the first two AMC meetings, the debates were guided by operational procedures, alarm adjustment techniques recommended in the company’s internal procedures, and a list of the 20 most announced alarms in the month. The preparation of the AMRs was rooted in the perception that the information discussed in the AMC meetings was insufficient to support the changes in the alarm system. In some cases, the AMC could not understand what caused an alarm to be triggered and, at first, the abnormality of the process was unknown and the conditions did not clearly fit the workers’ experiences of operation and maintenance. As a result, some alarms were considered “under investigation,” impairing the ability to make immediate meaningful changes.

In pursuit of using real work information to better understand the operations and promote changes, AMRs were developed to be supplementary in AMC meetings. The reports were made by a control room operator, the main author of this study. The AMR structure was developed during the meetings, considering the perceived needs of the AMC meetings and the theoretical input.

The final version of the AMR consists of the descriptions of the real work situations in which each alarm considered as a bad actor was triggered, including the operations, worker strategies, and the status of the equipment associated with the alarm. Four elements were identified as components of narrative building: operational information, operational information validated with peers, equipment information, and briefings. Table 1 illustrates the additions of these elements corresponding to the AMC meetings in which they were implemented.

The first version of the AMR was used in the third AMC meeting. It was built using operational information – a list of all operations that occurred in the month (operations summary), and a description of unexpected or unwanted conditions associated with systems where bad actors were found. This information was based on change-of-shift reports and personal observations discussed inside the shift group.

Starting at the sixth AMC meeting, bad actors identified as operations and shifts displaying unwanted behavior, were analyzed. Then, descriptions of the operation relevant to the sensor/alarm and eventual deviations on the process were discussed with the on-duty shift workers. As the details of operational conditions or particular issues were discussed with coworkers of at least two teams to get multiple perceptions and an agreement about the plant state, this procedure was categorized as operational information validated with peers. Equipment information included maintenance notes, reports of odd behavior, or malfunctioning of equipment involved in the investigated alarms. A quick briefing of the major findings was added to begin structured debate within the AMC. All elements are illustrated in Fig. 2.

OPEN IN VIEWER

Fig. 2

AMR elements.

The description of the operational conditions was carried out via a discussion with the plant operators directly using a systematic approach. It mainly consisted of an interview conducted during change-of-shift meetings and aimed to unveil the causes for unwanted alarm behavior. The placement of an additional routine at the change-of-shift meeting does not show any issues so far. The sources for each element are summarized in Table 2.

In a larger picture, one could consider that two instances of debates were related to this process: the debates with users, which produced the report, and the debates within the committee using the report. Our study focuses the debates held inside the AMC.

These elements were used to build real scenarios in which every alarm was triggered, and these situations were discussed in the AMC meetings. For illustration purposes, a scenario is presented 1 related to an alarm of low pressure on a vapor separator:

Naturally, some scenarios were simpler and did not require much elaboration as a sensor was damaged or a setpoint was mistakenly adjusted.

The participants of this case study were the members of the AMC for the gas logistics plant composed of the following professionals: two maintenance representatives, an operational coordinator, and a control room operator.

Nine AMC meetings were held between September 2019 and August 2020. The first four meetings lasted four hours, and the subsequent five had an average duration of 2.5 hours. The aim of these meetings was to assess, based on a list of the 20 most triggered alarms in the plant, whether the alarm announcements were relevant, and propose changes in the system aiming for better operating conditions.

These changes may occur in alarm systems, plants, or work procedures. The nature of the changes are as follows:

At the alarm system/control system: removing or creating an alarm, changing its priority, or enabling or disabling the setup of variable limits.

Accordingly, some changes may unfold in multiple orders. For example, a new sensor would require installation on the plant, making its information available at the system and considering its use in work procedures.

It is worth noting that not all 20 alarms were discussed in all of the nine AMC meetings. Minutes were recorded for every meeting, administrative matters were registered, orders and tasks were monitored. Changes regarding alarms, which were gathered from the maintenance notes and operational duties, were also recorded. The data collection strategies included the participative observations of nine AMC meetings and the analysis of the minutes of those meetings.

2.6 Data analysis procedures

Data derived from field notes of participative observations and AMC meeting minutes were analyzed using qualitative content analysis procedures. Participative observation data were used to describe the motivations and development of the AMR as well as the three debated situations. The AMC meeting minutes were analyzed to identify the type of information the AMC used to justify the proposed changes. In this way, it was possible to understand the utility of the AMR in structured debates and analyze its influence on the proposed changes [22].

3.1 Analyzing alarms through structured debates

Minutes covering nine meetings and 11 months of operation were analyzed (September 2019 to August 2020). The meeting minutes were organized as a table in numbered rows, with columns corresponding to deliberations, responsible persons, and deadlines. The rows regarding the treatment of bad actors contain the following items:

Tag: short name of the prompt; alarm description – description of the prompt, as shown for p lant operators; new description – in case of description adjustment; justification – reasons to support the change; and recommended action – changes required to improve alarm efficiency.

In qualitative content analysis, building a coding frame allows for a systematic description of the data [22]. To inquire how real structured debates based on the AMRs influenced the treatment of bad actors, we analyzed the cells corresponding to each justification and recommended action. Thus, the change proposed by the AMC was classified by the evidence that supported the decision. Our analysis was based on one dimension, “decision support,” indicating the nature of the information that was used in the structured debate to support the proposed change. The following four categories were identified 3 :

The signal treatment is based on the behavior of the readings of the sensor, and its changes consist of the adjustment of variable signals and tolerance values, mainly taking into consideration the sensor properties and project limit requirements.

A short description of each category is shown in Table 3, with total occurrences depicted through numbers and percentages. The results show that treatment using the signal treatment and operational procedures – prescribed methods – made up 12 % of the occurrences. Most of the solutions relied on workers’ practical knowledge. Although the AMR was not available in all AMC meetings, most resolutions for bad actors were based on structured debates about real work situations (63 %).

The percentage of alarms treated per justification is illustrated in Fig. 3. Meeting 3 did not have a treated alarm, since it was identified as an issue in the database, and the registered alarms were not representative. The use of the AMR began at meeting 3.

OPEN IN VIEWER

Fig. 3

Percentage of alarms treated by justification per meeting.

The number of AMC meetings and the number of treated alarms did not allow us to make a proper statistical treatment of the data. However, with some caution, it is possible to adopt the use of an AMR in the structured debate routine. It is noticed that the decisions supported by operational procedure were not used after meeting 4. The frequency of changes supported by experience decreased but was still sufficient to justify some changes. Signal treatment support was only applied in very specific cases, mostly addressing sensor sensibility.

In some instances, an increased scope was noted in the changes proposed by the AMC. Three examples of structured debate scenarios using the AMR were described to illustrate these extended changes, followed by considerations on how the scenario would be solved without the information of the AMR. Scenario #1 describes how the information provided by the AMR was used to challenge sensor reliability and promote an alternative method to monitor an operation. In Scenario #2, the AMR allowed the AMC to recognize the strategy adopted by the plant operators under exceptional situations associated with an alarm. Scenario #3 shows how the AMR helped the AMC formalize an unprescribed condition considering the variables of the complete system. In addition, a new alarm adjustment interface is proposed for the level and pressure alarms.

Scenario #1: Mass transfer with a high rate of false alarms – a discussion on estimates and sensor reliability based on operational information.

This scenario shows how an estimate of flow is unreliable using level indicators and should be performed using other variables. In the operation of receiving gas from ships, many variables, such as pressure, flow, and temperature are monitored. A huge number of alarms announced for level rates was identified as bad actors. As verified by the AMR, this operation did not have any odd episodes, such as leakage or overpressure. In this specific system, there was no flow meter, and the level rate was used to estimate the transfer flow. The level sensor was a floating-gauge device calibrated for a still surface. It was argued that although flow monitoring is important, the transfer disturbs the liquid’s surface sufficiently for the gauge to oscillate, generating false high and low flow indications (false alarms). It was concluded that this alarm should be removed because it is unreliable. The variable can still be observed for estimating the rate and comparing it with the ship information, and monitoring should be based on multiple pressure sensors available. Thus, it is possible to estimate flow variations instantly by pressure oscillation.

Conclusions: Without real work information to ensure that the operational conditions were optimum, it would not have been possible to challenge the reliability of the level sensor for estimating the rate or propose another method to monitor the operation.

Scenario #2: Restrictions change alarm usage and operational strategy.

In this scenario, an unusual operating condition and its effect on alarm rates are portrayed, highlighting an operational variable related to performance.

The terminal has two kinds of gas tanks with two different storage conditions: refrigerated storage (temperatures around –40°C and maximum pressure 0.1 kgf/cm²) and pressurized storage (temperatures over 10°C and pressure above 3 kgf/cm²). The operation of transferring gas from low-temperature tanking to high-pressure tanking is performed using a heat exchanger to warm the gas using seawater.

The pipelines involved in the operation of pressurized gas are built to withstand temperatures ranging from 10°C to 50°C. Usually, the temperature of the gas leaving the heat exchanger stays between 15°C and 18°C. Because the plant is situated in a tropical area, there is no risk in lowering the temperature of the pipeline. The low-temperature alarm was set at 13°C. When the temperature is below 13°C, the plant operator should increase the flow of seawater or reduce the flow of gas. This is a stable process, and the alarm has rarely been announced. Surprisingly, this alarm was featured as a bad actor in one AMC meeting.

The structured debates used to build the AMR helped us understand this issue. In a certain transfer operation, originating from refrigerated storage at the shore to a ship with pressurized storage, the vessel had problems dealing with high pressure. This caused the vessel to request lower transfer rates and even to halt the operation. To make the transfer as efficient as possible, the team decided to transfer the gas with the lowest possible value within safe limits to minimize the pressure on the ship tanks. This alarm was announced several times in a transfer because the operation required multiple corrective interventions, and controlling the temperature was difficult because the backpressure of the ship made it difficult to maintain a steady flow. Despite appearing to be a bad actor, an assessment of the situation allowed us to conclude that all of the alarms were relevant. This strategy allowed the best possible transfer rate inside the safety limits, contributing to a quicker operation. However, this context was considered sufficiently rare to justify any change in the system.

Conclusions: The AMR information revealed a strategy applied to overcome a situational constraint, making it possible for the AMC to be sure about the relevance of the alarm. It helped avoid unnecessary actions such as sensor calibration or changing the limits of the alarm.

Scenario #3: System working outside prescribed conditions

This scenario describes a system operating under normal operational conditions and its effects on all associated alarms. It used real work information from the scenario described in Subsection 3.2 and an alarm log from the system.

When transferring from pressurized storage to refrigerated storage, the gas must be cooled from –30 °C to –40°C. The cooling operation is performed using a compressor unit, which uses a cooling fluid in a closed gas circuit at low temperatures, removing the heat from the gas in two heat exchangers in series. This operation is particularly complicated because the rate and temperature of the two gas streams must be matched to maintain the exchanger liquid/vapor levels and pressure. High coolant levels on the exchangers could cause liquid to reach the compressors, which is a very dangerous situation. The system automatically shuts down the process when the levels reach a threshold, which avoids an accident but causes a loss of cooling fluid and time. The use of a cooling fluid without ideal characteristics and a mechanical problem in the compressors that reduced the efficiency of the system caused an abnormal fluctuation in the pressure and levels of the heat exchangers. As a result, level and pressure alarms appeared as bad actors. As these two conditions – different cooling fluid and performance issues of the compressors – could occur again, the AMC decided to propose a formal change in the procedure, recommending new level goals, pressures, and alarm margins. An interface feature was installed, allowing the margins to be changed more easily depending on the coolant composition and behavior.

Conclusions: The AMR allowed the AMC to analyze a less than ideal situation and set operational limits for several sensors involved. Additionally, based on the difficulties reported by the workers, a new interface was designed to make it easier to adjust the alarm limits for the level and pressure alarms, and reduce unwanted triggers.

These scenarios illustrate that, when supported by real work information, proposed changes can be relevant and applied widely throughout the system. Prior to AMR usage, the changes were limited to the alarm setting itself.

Regarding the differences between discussions with plant operators and structured debates within the AMC, we noticed a slight difference. The structured debates with plant operators were more descriptive, as they tried to explain their reasoning and expose the strategies used in different real work situations. Naturally, as the AMC has more disciplines involved, the structured debates considered the plant operator needs and working strategies as starting points and used the collective AMC knowledge to propose feasible changes for overcoming difficulties.

Transfer operations typically have continuous process characteristics with frequent starts and finishes and a high variability in the number of operations and operating conditions from one month to the next. Therefore, the evolution of alarm rates cannot be used as an indicator of success for the approach.

The evolution of the types of justification for alarms is also an unreliable metric, considering that the series is short, and that the nature of the alarm may or may not require structured debate on real work situations. For example, when an alarm is triggered many times owing to sensor failure, the treatment should repair and calibrate the hardware. Therefore, no consideration of this activity is required.