statistic hw assignment

PSYC 354

Homework 6

Effect Size Assignment

Answer each question about effect size. Remember to show your work. Submit your assignment to Blackboard as a Word document. (35 pts possible)

1. A company decides to add a new program that prepares randomly selected sales personnel to increase their number of sales per month. The mean number of sales per month for the overall population of sales people at this national company is 25 with a standard deviation of 4. The mean number of sales per month for those who participated in the new program was 29. Compute the effect size of the new sales program. (5 pts)

2. On a certain anxiety questionnaire, the population is known to have a mean of 12 and a standard deviation of 2.3. A higher score represents higher levels of anxiety. Participants in a new relaxation program complete the questionnaire after completing the program, and have a mean score of 10.2. What is the effect size of the relaxation program? (5 pts)

3. A private tutoring company has made the claim that its tutoring package will improve GRE Verbal test scores by at least 25 points. A graduate student in psychology hears this claim and decides to test their theory. After gathering 10 undergraduate students who are willing to participate in the program, he wants to predict the effect size for an increase of 25 points, assuming a GRE Verbal test mean in the population of 462 and a standard deviation of 119. What is the predicted effect size for a projected increase of 25 points? (5 pts)

4. A school psychologist is planning to study the effect of a new anger management program on the aggressive behavior of students at her high school. She knows that the mean number of aggressive behaviors at her school is 14 per week, with a standard deviation of 3.2, but she would like to cut this number in half to 7 events per week as a first benchmark goal of the anger management program. What is the predicted effect size for this projected decrease in aggressive behaviors? (5 pts)

5. A residential treatment facility tests a new group therapy for patients with self-destructive behaviors. The therapists hope to decrease scores on a measure of self-destructive behaviors that has a mean in the overall residential treatment population of 35 and a standard deviation of 4.7. The mean score for the patients after the new group therapy is 27. What is the effect size of the new group therapy? (5 pts)

6. Use Cohen’s conventions for effect sizes to label the following as small, medium, or large effects: (1 pt each for a-e)

a. .15

b. -.81

c. -.59

d. .32

e. .67

7. Using a diagnostic interview, a counselor notes a significant decrease in symptoms in patients who have been treated for depression. He computes the effect size based on his data, and finds that d = -.67. Based on Cohen’s conventions, what size is this effect? (5 pts)

Welcome to the presentation Effect Size and Statistical Power for Psychology 354 at Liberty University. This presentation is meant to be part of a series, so please be sure that you have watched all preceding presentations in order before viewing this one.

Over the past several weeks, you have read and heard a lot about the concept of statistical significance. When talking about the results of hypothesis tests, statistical significance signifies that there is a difference between means that is probably not due to chance. However, this concept gives us no information about the magnitude of this difference. Researchers increasingly use a measure called effect size to address this, because effect size measures the size of the difference between the means.

Here is the formula for effect size. There are different ways of computing effect size, but for this course we concentrate on Cohen’s d, which is defined as the difference in population means divided by the population standard deviation. Based on an extensive review of psychology studies, the psychologist and statistician Jacob Cohen provided guidelines for the size of this effect: a small effect = .2 (or -.2), a medium effect = plus or minus .5, and a large effect = plus or minus .8.

Let’s look at an example. Suppose I want to study the effectiveness of a new diet pill on weight loss, as is so often done in the media. I see a commercial that claims that women on a certain diet pill for a two month period lost significantly more weight than women in a control group (who did not take the diet pill). Though the commercial states that the results are statistically significant, it does not mention the size of the difference between the means of the two groups, or, in other words, the effect size. I look into this diet pill study further, and I am able to find the following information: the women in the study were between 30 and 40 years old. The mean amount of weight lost for the women in the control group is 8.9 pounds with a standard deviation of 3 pounds. Because this is the control group, it represents the distribution of weight loss for this age group in the general population, with no treatment of any kind. The mean amount of weight lost in the group of women who took the diet pill is 9.4 pounds.

I take this information and plug it into the formula to compute the effect size of the study, as shown here. As you can see, the mean of the group of interest is mu one, which is 9.4, the mean of the population in general is mu 2, which is 8.9, and the standard deviation is 3. When I do the math, I calculate an effect size of .167. According to the standards discussed earlier, this is a small effect size. In fact, it’s on the smaller side of small effect sizes. I conclude that, even though the diet pill produced significant weight loss according to the study in the commercial, the difference between the means is really not that large.

Sometimes, psychologists and other scientists want to look at the effect of some type of procedure or treatment that has been extensively studied. There are different methods for reviewing results from past studies, but one quantitative way of looking at these results is called meta-analysis. Meta-analysis is a technique that computes effect sizes across many different studies to give an overall picture of the effectiveness of a certain experimental technique. For example, if you wanted to conduct a meta-analysis of cognitive behavioral therapy for conduct disorder, you would review all of the studies pertaining to this subject, compute effect sizes based on the measures in each study, and draw conclusions based on your results. The process is a bit more complex than this in practice, and some of the effect size formulas can differ, but this is the basic procedure that is used by anyone who conducts a meta-analysis. This procedure is used more and more often in contemporary psychology.

Now to move on to our next topic: statistical power. This is the probability that the study will support the research hypothesis if the research hypothesis is true, and lead to a rejection of the null hypothesis. In other words, power is the probability that your results will be statistically significant when they are supposed to be.

So what factors affect the power of a study? The biggest factors are sample size and effect size, which are marked here because of their importance. The other factors are alpha level, the direction of the test, and the type of hypothesis test used. As you can see, a higher sample size, effect size, or alpha level increases the power of a study. A one-tailed test has a higher chance of detecting a significant result if the research hypothesis is true, so it has higher power than a two-tailed test. Finally, different types of hypothesis tests increase power in different situations, but this is not something covered in this course.

Let’s say you’re designing a study and you want to know how to increase its power. One thing you can do is increase the predicted effect size by either increasing the predicted difference between means or decreasing the population standard deviation. Both of these procedures create a larger predicted effect size, which increases the power of the study. Second, you can increase your sample size. This is one of the easiest and most commonly used ways to increase power—a larger N will increase your chances of detecting a significant result. Third, you could decide to use a less extreme alpha level, like p = .2. What this basically does is make your rejection region larger, so that you have a greater chance of getting a result that will lead you to reject the null hypothesis. This is not recommended, as it increases your chances of making a Type I error and rejecting the null hypothesis when the null hypothesis is actually true. Fourth, using a one-tailed test increases power by placing a larger rejection region in just one tail of the distribution. Finally, you can sometimes choose to use a more sensitive hypothesis testing procedure.

Many studies end in results that are not statistically significant. In fact, one of the criticisms of scientific journals in general is that they do not publish studies that may be designed well but did not conclude with significant results. Why is this important? Well, the main reason is because there are many factors that contribute to results not being significant. You have learned about some of these during this course. What is of interest to many researchers is the power level in studies that contain non-significant results. If a study has a high level of power, this means that it has a high chance of detecting a significant difference if one exists. Therefore, a non-significant result from a study with high power is considered a pretty conclusive indicator that the research hypothesis can’t be supported. Knowing this helps scientists make decisions about which research hypotheses to focus on in future studies. A research hypothesis that is not supported over several high-power studies may be less worth a researchers’ time and money than some other alternative hypothesis.

However, if a study has a low level of power, this means that the chances of detecting a significant result are much lower. A non-significant result from a study with low power is inconclusive at best—it does not tell us that the research hypothesis from that study should be discarded. In fact, a researcher might test the same hypothesis in a study with higher power and get significant results. This is one reason why knowing the power of a study is very important.

Page 4 of 4

PSYC 354

Excel Homework 7

(70 pts Possible)

Research Question:A clinical psychologist is treating 25 patients with clinical depression. She wants to find out whether these patients score differently than the general population on an emotional response scale with a population mean μ = 9.5. She is only interested in whether there is a difference, but not in the direction of the difference at this point. Open the “Data Set 7” attachment under the “Excel Homework 7” header located in the Assignment Instructions folder and perform a single-sample t-test to evaluate this hypothesis.

1. State the null and alternative (research) hypothesis in symbolic form. Remember that your hypothesis should include evaluators such as =, <, > or a combination of these.(7 pts)

2. Remember that we are only doing 2-tailed t-tests at this point.

3. Fill in the cells for the degrees of freedom andμ, which are already known. These will be used in formulas as shown in this week’s presentation. (7 pts)

4. In the appropriate cell in the table, compute the standard deviation of the sample (s) using the steps shown in this module/week’s presentation. (7 pts)

5. Fill in the sample mean (M), using the AVERAGE function for the raw scores on the left. (7 pts)

6. We are going to test our hypothesis at the .01 level of significance. Enter the alpha value in the appropriate cell. (7 pts)

7. Find the critical t value for our test, based on the alpha level, using the TINV function as shown in this module/week’s presentation. Remember that we are running a two-tailed test, so no change to the alpha level is needed. (7 pts)

8. Compute the sample t score using the steps gone over in this week’s presentation. (7 pts)

9. Fill in the critical p-value based on your alpha level. (7 pts)

10. Compute your sample’s p-value (based on your sample’s t score) by using the TDIST function as shown in this module/week’s presentation. (7 pts)

11. Answer the 2 questions underneath the second table directly in Data Set 7, after each question as indicated. (Questions = 3.5 points each for total of 7 pts.)

Save your work as “yourname_HW7.xls” and submit it by 11:59 p.m. (ET) on Friday of Module/Week 8.

2

>Sheet1

)

5

5

8

8

4

6

2

8
4

6

2

5
4

4

7

5
4
6

Score on Emotional Response Scale

(range 0-1

5

Null Hypothesis (H0)

Research Hypothesis (H1)

8

One- or Two-tailed?

11

7

Degrees of Freedom (df = N-1)

4

Population Mean (μ)

Sample Standard Deviation (s)

6

Sample Mean (M)

Alpha

10

Critical t value (cut-off score)

Sample t

Critical p-value

3

Sample p-value

1. Are your results statistically significant? How did you make this decision?

2. What is your decision about your hypotheses based on these results? Include detailed statements about each hypothesis.

Sheet2

Sheet3