Standardizing Pain Behavior Protocols - Should we do it?

1

Many scientists believe that science is experiencing a ‘reproducibility crisis’.

The causes for this are many,I’m sure, but one apparent factor is the lack of standardized protocols for many key methods. Moreover, even when methods are reported, the details are sparse, and seemingly unimportant variables are unaccounted for.

In pain research, and neuroscience broadly, behavioral assays make up a huge component. We rely on them to assess the effects of our interventions and manipulations. Given their prominence, it’d be reasonable to have a set of standardized protocols that we as pain researchers established and followed. But in fact, we don’t have anything like that. Instead, each lab, and even each person in the lab, uses their own variations of behavior protocols. Even with simple assays, like the Von Frey assay, or Hargreaves, there are many variables at play, and often not just the ones we focus on. The ones we don’t discuss frequently, like time of day, location of testing, sex of the tester, etc., might make a big difference, and we often make decisions on the fly about these other variables.

So, I’m wondering if it would not be a good idea to establish a set of standardized protocols, as a community, and adhere to them. I know this wouldn’t be a magic solution, but it could help combat the inability to reproduce other people’s results. There is precedence for this in other fields, so I think we could do it to.

What do you think? Would this be worthwhile? And do you think it would even help?

@jmogil - What do you think?

-Alex

2

This conversation has been had in other fields before, especially behaviour genetics. The problem is this: how would we decide who’s standard to follow?

3

Oops. Horrible grammar mistake above. I meant: whose standard to follow?

4

Maybe not one from a particular person/lab, but one that is crafted collectively? I imagine some people have strong opinions for particular features of a test, but I’d bet for the most part, people aren’t committed to theirs because it was rationally designed, but rather inherited from other people in the lab who settled on something.

So concretely, perhaps a way to do this would be to have people post their protocols here, and then discuss about which features to keep/modify?

De facto, we do this in a way. For von Frey, for example, everybody reference the “Dixon Method.” So Dixon, once upon a time, set a standard, and we’ve been somewhat following ever since, except for all those unspecified variables I mentioned, which would be good to spell out.

5

I think this could be a first step, but eventually there would need to be a meeting, I would think…

6

I agree. That would eventually need to happen. I think it’s worthwhile to look at the benefit that such standardization has brought to behavioral neuroscience in other areas. As you said, this has been tried before, so it’d be a useful guide to see if those efforts have actually had a positive effect.

I think though, just more transparency and discussion could help. I’m going to begin by posting a series of videos and protocols soon here on the forum. I’m willing to open myself to critique (and learning!) if it will help us clarify the optimal ways of doing things. Or at least be aware that people do things in different ways, and to use that as a lens to appraise results.