Clinical significance in real world settings

Abstract

[Extract] A while ago, for those of you who read these pieces regularly, you might remember that I spent a bit of time talking about inferential statistics and the question of significance. One of the areas that I touched on, relatively briefly, was the concept of clinical significance. I've been thinking a bit more about this concept lately and thought it might be worthwhile spending more time talking about this.

The idea of Clinical Significance came about because there was an increasing criticism of the types of research methodologies applied to real world clinical interventions. Until quite recently the assessment of the relevance of data in psychotherapy outcome studies has tended to rely on methodologies which have emphasised internal validity (e.g. Randomised Controlled Trials). This approach essentially establishes efficacy — whether or not there is some 'active ingredient' that may be causally related to change. But, there has been a growing recognition that the question that is of most relevance, particularly to clinicians, is one of effectiveness. That is, whether the use of an intervention under more realistic (i.e. 'less controlled') conditions leads to change in 'real' clients. There are a range of arguments against using naturalistic design methodologies from a science perspective, but many clinicians intuitively accept the argument that unless you are using real clients under real conditions it is very hard to argue that an intervention has been properly tested.