Author Topic: Repeated Measure Effect Sizes  (Read 25340 times)

0 Members and 1 Guest are viewing this topic.

Offline Dev

  • Administrator
  • Sr. Member
  • *****
  • Posts: 60
  • Karma: 2
  • Forum Administrator
    • ClinTools
Repeated Measure Effect Sizes
« Reply #1 on: 17 May, 2007, 05:43:28 PM »
There seems to be some confusion regarding within versus between effect sizes. Here's a typical email (and the answer is below):

Hi, I have some data on pre and post treatment measures that I entered into your effect size calculator, and I am very pleased with the way the program works; however I noticed that it seemed to treat data from two independent groups the same way that it would treat data from the same group at two different time points. I asked for feedback from my statistics instructor and got the following reply:

"About the effect size...  I probably didn't catch that you were looking at a paired difference (pre-post change).  The problem with the effect size, as it's normally calculated, is that it assumes you are interested in the mean difference between two
*independent* groups.  The basic formula (for Cohen's d, not Hedges g) is
(difference in means) / pooled SD --- where the pooled SD is essentially the average SD The formula for Hedges g is similar but it accounts for an imbalance between the sample sizes of the two independent groups.

With a paired difference, there's the correlation between observations that's not accounted for in the pooled SD calculation.  So the formula you really want is (mean difference) / (SD of difference) --- note that I'm distinguishing between the difference in means from the mean difference"

-Are there any references you can point me to that would help resolve this disparity of opinion one way or the other? 


Yes, your stats instructor is quite right. The only thing I would add is that Hedges' g also corrects for N irrespective of an imbalance in sample sizes (although it corrects more for that too). The main / front page of the Effect Size Generator (ESG) works with non-repeated measures. For repeated measure estimates of Cohen's d then one should do one of two things:

1. Use the within groups estimation that is provided on the 'extra statistics' tab of the Effect Size Generator when just looking at the change over time for one group.

2. To compute an repeated measures interaction effect size then: use the average of the change scores for each group in the time 1 and time 2 box on the main ESG page. For SD, use the standard deviation of the change scores for each group.

You need the raw data for option 2 above and need to know the correlation between time 1 and time 2 for the group to use option 1 above.

When you can't do either, just use the usual method and assume you're slightly underestimating for looking at within groups change for just one group. For a repeated measure interaction effect size where you don't have the raw data see the "Compute a Repeated measure interaction effect size" command button on the 'extra statistics' tab and also consult the help file.

Hope this helps.

I am logged on, therefore I am.