2 questions related to effect size calculations - please help!

[Dev]

Indeed Ya-Ya, all are estimates to guage how we're going. I've recently had a paper accepted in JCCP where we use the between groups effect size for a repeated measures design. The purists would baulk, but the difference is at the second, third or, and most likely, fourth decimal point - pointless.

Best,
Dev

[ya-ya]

Here's the context: From Libon et al. (2007):   SOC/EXEC > AD made more errors on the Trails B  (z= 2.13, p   0.016). Patients with SOC/EXEC also exhibited a trend toward more errors on the Trail Making Testâ€"Part B compared to patients with SemD (z = 1.5, p   0.066). 

I want to find convert the standard normal deviate z = 2.13 to an effect size (Cohen's d)

The standard normal deviate z = 2.13 is equivalent to .9834 or negative value (z = -2.13) = 1.0 - .9834 =  .0166, meaning that 1.66% of the standard normal distribution is below this value, and 98.34 % is above.


So Dev, I used Rosenthal's formula that you cited: r = Z/squareroot N

i.e., Z divided by the square root of N (Rosenthal, 1991)

Square root of n (54) = 7.348

r = 2.13/7.348 = .2899

Then I used EFG to convert r to d = .60582


Additionally, I interpolated the t-statistic associated with p = .9834, but I think I used the wrong df (I used n-1 instead of n-2), and came up with ? (lost the piece of paper, which is why I didn't post this sooner!). Anyway, with this method, I came up with d=.66831 (interpolation by hand, but with df = 53)

I did this because I could not find an interpolation procedure in Clin Tools or when I looked quickly on the web. Of course, later I found many, and I was curious if I did the interpolations correctly since its been like 15 years.

So using the t-distribution calculator here: http://www.stat.tamu.edu/~west/applets/tdemo.html

with p = .9834 and df=52, t = 2.188 for df=52 and would be 2.187 for df=53.

Converted value of t=2.188 =  Cohen’s d of .664 (with unequal sample sizes, 38 and 16)

So, any way you slice it, they are both medium effect sizes.

Please feel free to let me know if I did anything wrong here!

Thanks again for your help!

Ya-Ya

[ya-ya]

yes, Dev, I do believe that would work!!!! I saw that EFG computes r from z, but for some reason, I wasn't sure if I could then use an r with all of my d's. of course, that makes sense!! actually both ways make sense, you're brilliant --

I'll let you know how it turns out - thanks!!!

[Dev]

Ah. If you're looking at the standard normal deviate (Z) associated with a certain p level then one can estimate r by:

Z/squareroot N

i.e., Z divided by the square root of N (Rosenthal, 1991)

Then one can compute d from r. There's probably a direct translation but I can't remember any at the moment. Of course, you could find the t statistic (with df from the data you have) related to the p-level you have for Z (ESG will help here on the last tab - tests of significance) and then translate the t to d (corrected for unequal N or not) with ClinTools.

Does that help?

I don't think I quite understand the delayed / immediate recall question but I'm sure your supervisor does.

Best,

Dev

[ya-ya]

Hey Dev, thanks for getting back to me. This is actually a z-statistic, along with a p-value. I'm not sure what the relationship is to a standardized z-score??  Maybe I didn't state the question correctly! So does that change anything regarding making it possible to convert to an effect size?  I know if you have an F-statistic with a p-value, there is a method to convert it (with your software, for example) to an effect size without having the means and SD's -- was wondering if there is anything similar for a z-statistic?? I've been searching online and reading all my old stats books but can't come up with anything.

2nd question: I do have the mean and standard deviation of the immediate and delayed recall, is that what you are asking? However, I am not looking at the difference between these two variables, I'm using them to calculate percent of material recalled at delay - mean score on delayed recall divided by mean score on immediate recall x 100.  Previously, I was looking at the difference between these two variables,  but I ran in too many floor effects so I changed the variable to "percent retention." So, there is not a way to estimate SD in this case other than using what is found in the literature, right?

Thanks again!!!!!!

ya-ya

[Dev]

Hi Ya-ya,

1st problem: It's not possible to translate a z-score into an effect size. A z-score is a point on the normal distribution relating to probability rather than size of difference. In a meta-analysis a z-score for each study is computed as a way of pointing out where it lies amongst the distribution of other studies included in the meta-analysis. But with regards to just translating a z-score to an effect size this doesn't make too much sense.

2nd problem: without the raw data, it becomes much more difficult as your supervisor says. using published data from other studies is probably the best way to go. Of course, if you do have the raw scores then the percent makes no difference (it just multiplies everything by 100). Just enter the mean immediate recall and sd and N in the effect size generator and then do the same for the delayed. the derived d or g will be the effect size of the difference between immediate and delayed recall.

Not sure if this helps..

Good luck,

Dev

[ya-ya]

Okay, well I had my meeting with my Advisor and we still couldn't figure out a good method of converting the z-statistic  (seems to be a difference score), with p-value to Cohen's d. Anyone else have any brilliant ideas?? I guess this is the one conversion not included in the Effect Size Generator software!!  I think there should be a way to move backwards and figure out what size difference would be equal to a z-score of 1.2 and convert it somehow to Cohen's d that way??

Regarding the second question, she told me that there really is no way to estimate the SD for percent retention from the means and SDs of immediate and delayed recall, when I don't have any raw data. She suggested that my best estimate for the SDs would be to use those found in the literature from the few studies that do use this variable with my study population.  She agreed that I'm not supposed to be doing a meta-analysis and that I have gone WAY overboard with this!!! But I'm still liking the program...

[ya-ya]

Same questions with a bit more detail:

First question:  Is there a way to convert a z-score to Cohen's d? Or is the z-score essentially the effect size in this case? I would like to include a particular study in my power analysis, but I don't have the means and SD's-- only the z-statistic and the p-value. I'm also not certain that the z-statistic (difference between two groups on a particular measure) is the same thing as a z-score?

2nd question: What is the best method of determining the SD in the case where I create a new variable that is a percent conversion of two existing variables. Example: delayed recall/immediate recall x 100 = percent retention.

In this case I only have the means and SD's for the delayed recall and immediate recall variables, but I want to calculate an effect size for percent retention using the information available. The sample sizes for the two groups are not always equal.

Ideally, I think the numbers I want would be:


mean of immediate recall +/- SD
------------------------------------------- X 100
mean of delayed recall +/- SD


but I can't figure out how to arrive at these numbers to calculate the effect sizes!

Thanks again for any assistance!

ya-ya

[ya-ya]

Wow, I'm shocked you replied this quickly - maybe there is hope yet that I can get this done by tomorrow (thursday). It seems I'm going to be pulling an all-nighter as I'm in the Seattle area and its 4am! I'm out of classes (for a long time!) and working on my dissertation and it seems I am turning my power analysis into a major meta-analysis (not sure how that happened!!) Anyway, I'm loving this program. Looking forward to hearing from you...  Thanks Dev!

[Dev]

Hang on Ya-ya,

Back after a meeting .....

Dev

[ya-ya]

First question is probably simple, but I not sure....  Is there a way to convert a z-score to Cohen's d? Or is the z-score essentially the effect size in this case?  Trying to enter the data in a meta-analysis but I don't have the means and SD's only the z-score and the p-value. 

2nd question: what is the best method of determining the SD in the case where I create a new variable that is percent conversion of two existing variables. Example: delayed recall/immediate recall x 100 = percent retention.

This is a commonly reported variable in its own right, but in this case I only have the means and SD's for the delayed recall and immediate recall variable.

Thanks in advance for any assistance, the sooner the better as I need to finish this by morning!

ya-ya