Testing the Control or Testing the Change?

On-site this week, I was involved in testing a new control. I cannot share much of the detail, but what we learned from the test felt worth sharing.

Before we started, I had completed the non-standard risk assessment and put together a simple plan for the test. It began, naturally enough, with a coffee, followed by a morning brief with the team, an inspection of the equipment and setting up a controlled test area.

I wasn't interested in simply proving that the new control worked. I also wanted to see what else changed when we introduced it, particularly whether we had actually improved Separation. That became much more important once we started testing it in practice.

Once the new control was in use, one of the expected benefits became clear. It created greater physical Separation between the person and the hazardous energy during an important part of the task.

If we had stopped there, it would have been easy to call the test a success, pat each other on the back and celebrate over another coffee and perhaps some cake.

But practical testing showed us something else. Introducing the control had changed other parts of the task. There were different access requirements, additional handling, different body positions, extra time in parts of the process and new interactions with the equipment.

None of those things automatically meant the control was wrong. What they showed us was that introducing the control had changed the system around it.

That is why the second day became less about proving the control and more about challenging what we thought we had learned on the first.

One of the most useful findings was that the benefit was not identical for everyone. I found that fascinating because we often talk about a control or barrier as though it will work the same way for everyone and in every situation.

For some people, the new method clearly reduced an existing exposure. For others, the way they already performed the task meant introducing the new control could create an exposure they did not already have.

It was the same task and the same control, but people were starting from different conditions. That was probably the most important learning from the test.

It would have been very easy to look at the control in isolation and conclude that because it increased Separation at one point in the task, it had made the task safer. The practical test showed why that conclusion would have been too simple.

We had seen that the control could provide the benefit we expected. What became more interesting was understanding what else had changed because we had introduced it.

That changed the way I looked at the test. Instead of only asking, “Does the control work?”, I found myself asking, “What has changed because the control now exists?”

That takes you back to the whole task. I started looking at whether new energy had been introduced, whether the pathway to exposure had changed and whether people were now doing something they did not have to do before. We also needed to consider changes in body position, access and handling, along with the barriers protecting people and what would keep those barriers effective over time.

It also means looking at the people doing the work. If people are starting from different conditions, the same control may not change their exposure in the same way. Something that creates greater Separation for one person should not automatically be assumed to do the same for everybody else.

That does not make the control ineffective. It means we need to understand the change properly, and I think that is the part I will take away from the test.

When we introduce a new control, we are not simply adding something to an existing task and leaving everything else untouched. We are changing the system around it. People may move differently, new interactions can appear, and existing pathways can change, or new ones can be created.

Some of those changes may strengthen Separation while others may quietly weaken it. That is why I came away from the test thinking there are two questions worth asking when we introduce a new control: does the control do what we expected it to do, and what changed because we introduced it?