USA: +1(669)289-8683 
Sign in
Annotated Bibliography
Read the instructions for the final paper to determine the kinds of articles you will need to use as references. Also read the Example Research Proposal provided in the course materials to help you visualize your final paper. Search the Ashford University Library’s databases and the Research Methods research guide to find appropriate peer-reviewed sources for your proposal. Read the selected articles and reread relevant sections of the textbook and the full text of the study you have been working with throughout the course. Using the Sample Annotated Bibliography information and example from the Writing Center as a guide, create an annotated bibliography of the sources you will use in your final research proposal.
The references in the annotated bibliography must be listed in alphabetical order, formatted in APA style, and published within the past 10 years. All selected sources other than the textbook must be available in full text in the Ashford University Library. After each reference, insert two paragraphs. The first paragraph should summarize the main points of the source, in your own words. Do not use any quotations or verbatim wording from the source for this assignment. In the second paragraph, explain how you will use the source to support your research proposal.
For this assignment, a title page in APA format is required but it is not necessary to include a separate reference page, because the paper itself is the reference page with additional information inserted. If you do include a separate page of references, be aware that it will not be counted towards the required page count. Your paper must be a minimum of four pages (excluding title page) and formatted according to APA style as outlined in the Ashford Writing Center.
Carefully review the Grading Rubric for the criteria that will be used to evaluate your assignment.
Stuck on a problem? Don’t skip that assignment – click the button to chat with a live tutor. It is free and here to help you now.
Go to top of page
Waypoint Assignment Submission
The assignments in this course will be submitted to Waypoint. Please refer to the instructions below to submit your assignment.
Click on the Assignment Submission button below. The Waypoint “Student Dashboard” will open in a new browser window.
Browse for your assignment.
Click Upload.
Confirm that your assignment was successfully submitted by viewing the appropriate week’s assignment tab in Waypoint.
For more detailed instructions, refer to the Waypoint Tutorial.
1
RESEARCH PROPOSAL
2
RESEARCH PROPOSAL 2
Running Head: RESEARCH PROPOSAL
Example Research Proposal
Pamela Murphy
PSY 326 Research Methods
Instructor’s Name
Date Submitted
NOTE: The details in this example research proposal are based on a published study which I coauthored with Charles B. Hodges and my doctoral dissertation, both in 2009. Portions of the text are excerpted from the published article (Hodges & Murphy, 2009) and the dissertation (Murphy, 2009).
Example Research Proposal
Introduction
The concept of self-efficacy was introduced nearly 40 years ago. “Perceived self-efficacy refers to beliefs in one’s capabilities to organize and execute the courses of action required to produce given attainments” (Bandura, 1977, p. 3). Self-efficacy has been identified as an important construct for academic achievement in traditional learning environments for at least two decades. Zimmerman and Schunk (2003) go so far as to say that “the predictive power of self-efficacy beliefs on students’ academic functioning has been extensively verified” (p. 446). Its importance has been noted consistently through all levels of the educational process, with various student populations, and in varied domains of learning.
While learner self-efficacy has a well-established literature base in the context of traditional learning environments, self-efficacy research related to learners in online and other non-traditional learning environments is relatively new. Hodges (2008a) has called for researchers to explore self-efficacy in online learning environments. Additionally, in terms of students’ self-efficacy beliefs toward academic achievement, “there have been few efforts to investigate the sources underlying these self-beliefs” (Usher, 2009, p. 275). The purpose of the proposed study is to investigate the relative strength of the four traditionally proposed sources of self-efficacy beliefs of students enrolled in a technology-intensive asynchronous college math college.
Literature Review
Self-efficacy beliefs have been found to be significant contributors to motivation and performance in academic achievement (Multon, Brown, & Lent, 1991), group functioning
(Gully, Incalcaterra, Joshi, & Beaubien, 2002; Stajkovic & Luthans, 1998), health (Holden, 1991), and sports performance (Moritz, Feltz, Fahrbach, & Mack, 2000). Research revealing the connection between self-efficacy and mathematics, the context of the proposed study, includes many cultures and levels of education (Malpass, O’Neil, & Hocevar, 1999; Pietsch, Walker, &
Chapman, 2003; Randhawa, Beamer, & Lundberg, 1993; Stevens, Olivarez, Lan, & Tallent-
Runnels, 2004) and continues to the present (Usher, 2009).
Sources of Self-Efficacy
Albert Bandura’s (1977) introduction of self-efficacy theory included the proposition that self-efficacy is derived from four principal sources: mastery experiences, vicarious experience, social persuasion, and physiological/affective states. These four areas are generally accepted in the literature as core elements in the development of self-efficacy beliefs, but an ordering of the importance of each of these sources is unsettled.
Mastery Experiences. Mastery experiences refer to previous, successful experiences a learner has had performing a task. Successes build positive self-efficacy beliefs and failures undermine self-efficacy. If failures are experienced before a firm positive belief in one’s selfefficacy is formed, the creation of positive self-efficacy beliefs is more difficult.
Vicarious Experience. Vicarious experience refers to one’s observation of a role model performing a task. Knowledge of how others have performed a similar task helps one determine whether or not a performance should be judged a success or failure. Surpassing the performances of others increases self-efficacy and falling below others’ performances lowers self-efficacy.
Note the importance of the selection of individuals for comparison. Self-efficacy beliefs will vary depending on the abilities of those chosen for comparison, thus, models for comparison should be selected carefully (Wood, 1989).
Social Persuasion. Social persuasion is commonly used due to the ease with which it can be dispensed. The believability of the persuader(s) is important in the use of social persuasion. The receiver must view the persuader as competent to provide meaningful and accurate feedback. Bandura (1997) cautions that verbal persuasion consists of more than flippant, offhand comments of encouragement. Unrealistic comments from the persuader may mislead the receiver, which may decrease self-efficacy and diminish the belief in the persuader as one competent to evaluate the performance. “Skilled efficacy builders encourage people to measure their successes in terms of self-improvement rather than in terms of triumphs over others” (Bandura, 1997, p. 106).
Physiological/Affective States. Stress, emotion, mood, pain, and fatigue are all interpreted when making judgments regarding self-efficacy. For example, someone may have prepared well for an exam, but upon learning of some unfortunate news, stress may reduce concentration, thus impacting performance on the exam. In general, success is expected when one is not in a state of aversive arousal (Bandura, 1997).
Usher and Pajares (2006) summarize the inconsistent findings regarding the relative strength of each self-efficacy source well. They follow with the proposition that “exploring the predictive value of the sources of students’ academic self-efficacy beliefs and determining whether this prediction varies as a function of group membership such as gender, academic ability, and race/ethnicity is a matter of import” (p. 130).
Methods
Design
The proposed study is quantitative in nature and will use a survey research design
(Newman, 2011). Survey research falls into the non-experimental category of research designs. The survey questions use mostly ordinal scales and will result in numeric scores summarizing the extent of use of each source of self-efficacy beliefs as well as a score representing the level of self-efficacy held by each student in relation to the ability to learn math in an asynchronous learning environment.
Participants
Approximately 300 students in an asynchronous college algebra course offered at a large, state supported university in the mid-Atlantic region of the United States will be invited to participate in a survey. This is a convenience sample, and participation is voluntary, so the final sample size may be considerably smaller than the number of students invited. The course is delivered using an emporium format (Twigg, 2003) which is technology intensive. The students enrolled in the course tend to be engaged in academic majors that are not math-intensive. They may have a high degree of math anxiety or at least some negative feelings toward their math abilities. In addition, the emporium model may be an unfamiliar concept for them.
Procedure/Measures
This course is offered through the Math Emporium and has no traditional class meetings.
After a brief, face-to-face, orientation meeting, students complete the course asynchronously.
There are weekly deadlines for quizzes, and proctored tests are administered periodically. Students prepare for the quizzes and tests by taking advantage of various technology resources available to them online. Lesson pages serve as an online textbook for the course, short streaming video lectures are available on most topics, and an unlimited number of practice quizzes are available. For students who desire it, face-to-face interactions with assistants in the computer lab are available several hours each week. No appointment is needed for the face-toface assistance.
At the conclusion of the course, data will be collected using a web-based survey tool. Students who provide informed consent to participate will be given an ID number and survey access information. They may access the survey either in the Math Emporium or offsite through the internet. Specific instruments to be used are the Self-Efficacy for Learning Mathematics Asynchronously (SELMA) survey (Hodges, 2008b), a demographics survey, and the Sources of Mathematics Self-Efficacy (SMSE) scale (Lent, Lopez, & Bieschke, 1991).
The SELMA survey is a 25-question survey constructed for use in college algebra and trigonometry courses offered in an emporium model. A validation study showed an internal consistency Cronbach’s alpha value of 0.87 (Hodges, 2008b) which is greater than the 0.80 minimum level recommended by Gable and Wolf (1993) for instruments in the affective domain. The SMSE scale consists of four 10-question subscales designed to measure each of the four sources of self-efficacy: mastery, vicarious experiences, social persuasion, and affective/physiological state. In a validation study of the SMSE, Lent et al. (1991) reported internal consistencies of 0.86 for mastery, 0.56 for vicarious, 0.74 for persuasion, and 0.90 for affective/physiological arousal.
Data Analysis
To investigate the relative strength of the four traditional sources of self-efficacy beliefs of students in an asynchronous math course, analysis of variance (ANOVA) and multiple regression will be used. Scores from each of the four subscales of the SMSE will be used as predictors of the SELMA score. Bivariate correlations will also be examined. Significant correlations among the predictor variables may present a problem of multicollinearity. If necessary, additional statistical tests such as ridge regression (Joe & Mendoza, 1989; Kidwell &
Brown, 1982) will be applied to solve this problem.
Ethical Issues
Participation in the survey will be strictly voluntary, and will not be tied to evaluation of the student’s performance in the course in any way. As a non-experimental survey study, no deception will be used. Signed informed consent will be obtained from those who wish to participate. Those who agree to participate may withdraw from the study at any time without any type of penalty.
Confidentiality of participants will be protected by the assignment of ID numbers to be used on the survey documents instead of names or any other type of identifying information. A single copy of the list matching the ID numbers with participants’ names will be kept in a secure, locked location for a period of three years after the completion of the study. After three years, the list will be destroyed in accordance with the instructions of the Institutional Review Board
(IRB).
As a token of appreciation, all participants will be entered into a drawing for an Amazon gift card. The proposed amount of the gift card, subject to IRB approval, is $25. University facilities, including the computer lab known as the Math Emporium, its computers and a survey software program, will be used if this study is approved. This project will not receive any external funding from commercial or other sources, and no conflicts of interest are reported by the researchers.
Conclusion
Self-efficacy and its relationship to academic achievement in asynchronous online learning environments are only recently beginning to be researched (Hodges, 2008a). Given the growing prominence of asynchronous online learning, it is essential that we understand what role constructs such as self-efficacy play in these learning environments. The proposed study will address this need by using a survey research design. The surveys will provide data on the four sources of self-efficacy which will serve as predictors of students’ self-efficacy for learning mathematics in an asynchronous online setting. A multiple regression model using the four predictors with the SELMA survey score as the dependent variable will indicate how much each source contributes to self-efficacy.
The results of this study are expected to be important to instructional designers and educational practitioners who either currently use or are considering using an emporium model, as they will give indications of which elements of the asynchronous course design should be emphasized to best promote students’ self-efficacy relating to the subject matter. An expedited review of this proposal by the IRB is requested for approval to begin this research as soon as possible.
References
Allison, P. D. (1999). Multiple regression: A primer. Thousand Oaks, CA: Pine Forge Press.
Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change.
Psychological Review, 84(2), 191-215.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman and
Company.
Gable, R. K., & Wolf, M. B. (1993). Instrument development in the affective domain: Measuring attitudes and values in corporate and school settings, 2nd ed. Boston: Kluwer Academic Publishers.
Gully, S. M., Incalcaterra, K A., Joshi, A., & Beaubien, J. M. (2002). A meta-analysis of teamefficacy, potency, and performance: Interdependence and level of analysis as moderators of observed relationships. Journal of Applied Psychology, 87, 819-832.
Hodges, C. B. (2008a). Self-efficacy in the context of online learning environments: A review of the literature and directions for research. Performance Improvement Quarterly, 20(3-4), 7-25.
Hodges, C. B. (2008b). Self-efficacy, motivational email, and achievement in an asynchronous math course. Journal of Computers in Mathematics and Science Teaching, 27(3), 265-
285.
Hodges, C. B., & Murphy, P. F. (2009). Sources of self-efficacy beliefs of students in a technology-intensive asynchronous college algebra course. Internet and Higher
Education, 12(2), 93-97. doi:10.1016/j.iheduc.2009.06.005.
Holden, G. (1991). The relationship of self-efficacy appraisals to subsequent health-related outcomes: A meta-analysis. Social Work in Health Care, 16, 53-93.
Howell, D. C. (2002). Statistical methods for psychology, 5th ed. Pacific Grove, CA: Duxbury.
Joe, G. W., & Mendoza, J. L. (1989). The internal correlation: Its applications in statistics and psychometrics. Journal of Educational Statistics, 14(3), 211-226.
Kidwell, J. S., & Brown, L. H. (1982). Ridge regression as a technique for analyzing models with multicollinearity. Journal of Marriage and the Family, 44(2), 287-299.
Lent, R. W., Lopez, F. G., & Bieschke, K. J. (1991). Mathematics self-efficacy: Sources and relation to science-based career choice. Journal of Counseling Psychology, 38(4), 424430.
Malpass, J. R., O’Neil, H. F., & Hocevar, D. (1999). Self-regulation, goal orientation, selfefficacy, worry, and high-stakes math achievement for mathematically gifted high school students. Roeper Review, 21, 281-295.
Moritz, S. E., Feltz, D. L., Fahrbach, K. R., & Mack, D. E. (2000). The relation of self-efficacy measures to sport performance: A meta-analytic review. Research Quarterly for Exercise and Sport, 71, 280-294.
Multon, K. D., Brown, S. D., & Lent, R. W. (1991). Relation of self-efficacy beliefs to academic outcomes: A meta-analytic investigation. Journal of Counseling Psychology, 38(1), 30-
38.
Murphy, P. F. (2009). Relationships of parenting practices, independent learning, achievement, and family structure (Doctoral dissertation). Virginia Tech, Blacksburg, VA. Retrieved from
http://scholar.lib.vt.edu/theses/available/etd
–
04022009
–
174950/
Newman, M. (2011). Research methods in psychology. San Diego, CA: Bridgepoint Education.
Pietsch, J., Walker, R., & Chapman, E. (2003). The relationship among self-concept, selfefficacy, and performance in mathematics during secondary school. Journal of
Educational Psychology, 95, 589-603.
Randhawa, B. S., Beamer, J. E., & Lundberg, I. (1993). Role of mathematics self-efficacy in the structural model of mathematics achievement. Journal of Educational Psychology, 85(1),
41-48.
Stajkovic, A. D., & Luthans, F. (1998). Self-efficacy and work-related performance: A metaanalysis. Psychological Bulletin, 124, 240-261.
Stevens, T., Olivarez, A. J., Lan, W. Y., & Tallent-Runnels, M. K. (2004). Role of mathematics self-efficacy and motivation in mathematics performance across ethnicity. Journal of Educational Research, 97(4), 208-221.
Twigg, C. A. (2003). Improving learning and reducing costs: New models for online learning.
EDUCAUSE Review (September/October), 28-38.
Usher, E. L. (2009). Sources of middle school students’ self-efficacy in mathematics: A qualitative investigation. American Educational Research Journal, 46(1), 275-314.
Usher, E. L., & Pajares, F. (2006). Sources of academic and self-regulatory efficacy beliefs of entering middle school students. Contemporary Educational Psychology, 31(2), 125-141.
Wood, J. V. (1989). Theory and research concerning social comparison of personal attributes.
Psychological Bulletin, 106(2), 231-248.
Zimmerman, B. J., & Schunk, D. H. (2003). Albert Bandura: The scholar and his contributions to educational psychology. In B. J. Zimmerman, & D. H. Schunk (Eds.), Educational psychology: A century of contributions (pp. 431-457). Mahwah, NJ: L. Erlbaum
Associates.
CHAPTER 4
NONE
X
PERIMENTAL
QUANTITATIVE RESEARCH
GABRIELLA BELLI
KE
Y
IDEAS
n The distinction between experimental and nonexperimental research rests on the manipulation of treatments and on random assignment.
n Any quantitative study without manipulation of treatments or random assignment is a nonexperimental study.
n Nonexperimental research is used when variables of interest cannot be manipulated because they are naturally existing attributes or when random assignment of individuals to a given treatment condition would be unethical.
n Numbers are used to represent different amounts of quantitative variables and different classifications of categorical variables.
n Nonexperimental studies may be classified along two dimensions: one based on the purpose of the study and the other on the time frame of the data collection. n Evidence of a relationship is not convincing evidence of causality.
n Alternative explanations for results in nonexperimental research should be explored and ruled out.
NOTE: My thanks to Professor Bill Frakes, from the Computer Science Department at Virginia Tech, and to students, including many from my Research Methods class in Fall 2007, for reviewing a prior draft of this chapter. Their insightful comments and suggestions helped improve this version. I take responsibility for any remaining elements of confusion that may remain.
60
OVERVIEW OF NONEXPERIMENTAL RESEARCH
QUANTITATIVE RESEARCH is empirical, using numeric and quantifiable data. Conclusions are based on experimentation and on objective and systematic observations. Quantitative research may be divided into two general categories: experimental and nonexperimental. The essential elements of experimental research, which was discussed in detail in the previous chapter, are presented here first as a contrast to nonexperimental research. A primary goal for experimental research is to provide strong evidence for cause-and-effect relationships. This is done by demonstrating that manipulations of at least one variable, called the treatment or independent variable (IV), produce different outcomes in another variable, called the dependent variable (DV). An experimental study involves at least one IV that is manipulated or controlled by the researcher, random assignment to different treatment conditions, and the measurement of some DV after treatments are applied. Any resulting differences in the DV across the treatment groups can then be attributed to the differences in the treatment conditions that were applied.
In contrast to experimental research, nonexperimental research involves variables that are not manipulated by the researcher and instead are studied as they exist. One reason for using nonexperimental research is that many variables of interest in social science cannot be manipulated because they are attribute variables, such as gender, socioeconomic status, learning style, or any other personal characteristic or trait. For example, a researcher cannot randomly place individuals into different groups based on gender or learning style because these are naturally existing attributes.
Another reason to use nonexperimental research is that, in some cases, it would be unethical to randomly assign individuals to different treatment conditions. A classic example of this is that one could not study the effects of smoking by randomly assigning individuals to either a smoking or a nonsmoking group for a given number of years. The only ethical way to investigate the potential effects of smoking would be to identify a group of smokers and a group of nonsmokers and compare them for differences in their current state of health. The researcher, however, would also need to take other variables into account, such as how long people had smoked, their gender, age, and general health level. To do so would be important because the researcher cannot take for granted that the groups are comparable in aspects other than smoking behavior. This is in contrast to experimental groups, which, due to the process of random assignment, start out equal in all respects except for the treatment condition in which they are placed. In nonexperimental research, groups based on different traits or on self-selection, such as being or not being a smoker, may differ for any number of reasons other than the variable under investigation. Therefore, in nonexperimental studies, one cannot be as certain as in experimental studies that outcome differences are due to the independent variable under investigation. The researcher needs to consider possible alternative explanations, to jointly analyze several variables, and to present conclusions without making definitive causal statements.
Variables and Their Measurement 61
In this chapter, you will learn how to characterize nonexperimental studies that do not rely on either manipulation of variables or random assignment of subjects to groups. Different types of nonexperimental studies will be explained, and you will learn how to characterize them using a two-dimensional classification system. By the end of the chapter, you will understand the basic elements of nonexperimental studies, as well as the rationale for their use. Nonexperimental research examples, including published studies, will be incorporated into the discussion to facilitate understanding. At the end of the chapter, text and Web resources are provided to help you locate supplemental materials and additional information.
VARIABLES AND THEIR MEASUREMENT
To facilitate reading the remainder of the chapter, a brief review of variables and some of their different aspects is presented. A variable is any characteristic or attribute that can differ across people or things; it can take on different values. Some variables are inherent traits, such as gender or height. Others may vary due to experimenter manipulation, such as treatment groups of drug versus placebo, or due to self-selection, such as attending a two- or a four-year college. In quantitative research, variables are measured in some way and those numerical values are then used in statistical analyses. The nature of variables is important because, to some extent, it dictates the way research questions are asked and which analysis is used.
One basic distinction is that variables can be either categorical or quantitative. Categorical variables are those that differ across two or more distinct categories. The researcher assigns arbitrary numbers to the categories, but the numbers have no interpretable numerical meaning. For example, for categories of the variable “employment status,” we could assign the value “1” to employed full-time, “2” to employed part-time, and “3” to not employed. Additional examples of categorical variables that are individual traits are gender, ethnicity, and learning style; some that are self-selected are marital status, political party affiliation, and field of study.
Quantitative variables can be measured across a scale, their numeric values have meaning, and they can be subjected to arithmetic operations. The following are all examples of quantitative variables: age, height, weight, grade point average (GPA), job satisfaction, and motivation. There is an important distinction between the first three and the last three variables in this list. For such variables as age, height, and weight, zero is a meaningful value that indicates the absence of the characteristic being measured, as in something that is brand new or has no weight. The numbers have interpretable meaning. We know what five years or five feet means because there is no arbitrariness about these values or how to interpret them.
In contrast, zero is an arbitrary value for variables such as GPA, satisfaction, or motivation. A zero motivation score does not mean one has no motivation, but merely that one attained the lowest possible score for the particular instrument being used. GPA in most schools in the United States is given on a continuum from 0.0 to 4.0 but, for example, at the Massachusetts Institute of Technology (MIT), it goes from 0.0 to 5.0 (see GPA calculation and unit conversion in MIT Web page at http://web.mit.edu/registrar/gpacalc.html). The International Baccalaureate grades range from 1 to 7, based on a rubric developed from the standardized curriculum.
62
For another example, consider measurements for temperature. The freezing point of water is represented as zero on a Celsius thermometer, but as 32 on a Fahrenheit thermometer. In neither case does a zero represent the absence of temperature. In each case, we understand what the numbers mean because specific interpretations have been assigned to them.
Interpretation of different grading schemes or thermometers is possible because of commonly understood unit descriptors. This is not so for such variables as job satisfaction or motivation, where scores are arbitrary and depend on the measurement instrument being used and how it has been designed. Typically, such scores are the sum or the average of responses to a set of items. The items may be statements, constructed so that all are related to the variable to be measured, and responses are often, but not always, on a Likert scale from 1 (strongly agree) to 5 (strongly disagree). The terms scale and index are often used to describe such sets of related items that, together, produce a score about some characteristic or phenomenon. For example, the Multidimensional Job Satisfaction Scale (Shouksmith, Pajo, & Jepsen, 1990) contains eleven different subscales, each a multi-item scale measure of a different dimension of job satisfaction. Another instrument, the Job Satisfaction Survey (Spector, 1985), consists of nine four-item subscales to assess employee attitudes about the job. As you can see from this example, different researchers developed different measures of the same construct, job satisfaction.
Exact interpretation of a scale score’s value, or measure, for variables such as motivation or satisfaction is not important. What is important is to know that the higher the score, the more one has of the characteristic being measured and vice versa. One could, for example, examine whether males or females had higher levels of job satisfaction or if people with higher levels of job satisfaction also tended to have higher levels of motivation. To be confident of results, it is also important to know that the measures being used are reliable and have been validated.
Reliability relates to the consistency or dependability of a measure. Basically, if it is reliable, you can be confident that all the items that make up the measure are consistent with each other and that, if you were to use the measure again with the same individuals, they would be rated similarly to the first time. Validity relates to whether it is measuring what we intend it to measure, and represents the overarching quality of the measure. The purpose of using the measure is an important consideration in evaluating validity because it could be valid for one use but not for another. These concepts are complex and beyond the scope of this chapter (see Trochim, 2005 for a very understandable description of validity and reliability of measures). As a consumer of research, you should at least be aware of them and look for how research authors deal with these concepts. Do they describe their measures in detail and provide some indication of reliability and validity?
Defining Variables
Although some variables are inherently categorical or quantitative, others may be defined in either way. Imagine, for example, that you are interested in measuring the education level of a group of individuals. You could do this categorically, by defining
Variables and Their Measurement 63
education as “highest degree earned” and using five values representing none, high school, college, masters, or doctorate as different levels of education. Or, you could do this quantitatively by defining education as “number of years of schooling,” where the resulting values would be meaningfully interpreted. This distinction is important if one is interested in studying the relationship between educational level and salary, a quantitative variable, because it relates to how the data might be analyzed and how research questions would be phrased. Using the categorical definition, you could compare the median salary value across the five categories of “highest degree earned.” The median represents the midpoint when all the salaries are listed from lowest to highest. One could then determine if there were any appreciable differences in salary across the five groups and whether more education (represented by having a higher degree) corresponded to higher salary.
Using the quantitative definition, you could graph the two variables in a scatter plot or compute a correlation coefficient (a measure of strength and direction of relationship for two variables) for the number of years of schooling and salary. The first would provide a visual representation of their relationship and the second a numerical one. Figure 4.1 shows how resulting data might be depicted in the two cases described. The table shows the number of people in each group and their median salary. The scatter plot shows all the data points. The correlation for this data set is 0.66. Correlation
64
values range from −1 to +1, with zero indicating no relationship and 1 indicating either a negative or a positive perfect relationship depending on the sign. We could say these data showed a moderate positive relationship. Fewer years of schooling tend to correspond to lower salaries and more schooling to higher salaries.
Phrasing Questions
In the first case demonstrated in Figure 4.1, you would be comparing groups with different levels of education on some measure (salary), and in the second case, you would be relating two sets of numeric scores (years and salary). The research questions of interest in the two cases would be: (1) how do groups, based on highest degree earned, differ from each other with respect to salary? and (2) how does number of years of schooling relate to salary? Phrased generically, the key questions in the two situations are: How do groups differ from each other on some measure? How are the variables related to each other? The distinction between these two cases depends only on the fact that education was conceptualized as either categorical or quantitative and not on the nature of the relationship involved.
REFLECTION QUESTIONS
By now, you should be able to
:
1. Describe the difference between experimental and nonexperimental studies
2. Give an example of an independent and a dependent variable within the context of a research question
3. Give an example of a categorical and a measured, quantitative variable
CLASSIFYING NONEXPERIMENTAL RESEARCH
In the literature on experimental studies, there is agreement on the distinction between true- and quasi-experiments. Although both involve treatment manipulation, trueexperiments use random assignment of subjects to groups and random assignment of groups to treatments. Quasi-experiments use preexisting intact groups, which are randomly assigned to treatment conditions.
For nonexperimental designs, there appears to be no consistent agreement on typology. In 1991, Elazar Pedhazur and Liora Schmelkin stated that “there is no consensus regarding the term used to refer to designs” which were presented in their chapter on nonexperimental designs (p. 305). Two commonly used terms for nonexperimental studies are “correlational research” and “survey research.” However, the term correlation relates more to an analysis strategy than to a research design and the term survey describes a method of gathering data that can be used in different types of research.
Ten years later, Burke Johnson (2001) came to the same conclusion. Based on a review of twenty-three leading methods textbooks in education and related fields (thirteen explicitly from education and the rest from anthropology, psychology, political science, and sociology), he found little consistency in how nonexperimental studies were classified. He discovered over two dozen different labels being used, sometimes
Lapan c04.tex V1 – 09/02/2008 2:46pm Page 59
Lapan c04.tex V1 – 09/02/2008 2:46pm Page 60
Nonexperimental Quantitative Research
Lapan c04.tex V1 – 09/02/2008 2:46pm Page 59
65
with slight variations in the wording. The most frequently used labels in these texts were survey (twelve times), correlational (ten times), descriptive (eight times), and causal-comparative (five times). The result of my informal review of six additional research methods texts was consistent with Johnson’s findings.
In an attempt to remedy this confusion, Johnson (2001) proposed a categorization scheme consisting of two basic dimensions, each with three categories. The first dimension represents a characterization of the basic goal or main purpose for conducting the nonexperimental quantitative study. The second dimension allows the research to be classified according to the time frame in which data were collected. These two dimensions will be presented here and discussed separately in the next two sections. In your reading of published articles or research methods textbooks, you will probably encounter other terms for nonexperimental research. You may want to read Johnson (2001) to familiarize yourself with these terms and with the problems that arise because of their use.
Classification Based on Purpose (Dimension 1)
The categories of the first dimension for classifying nonexperimental studies, which are based on the main purpose of the study, are:
1. Descriptive nonexperimental research, in which the primary focus for the research is to describe some phenomenon or to document its characteristics. Such studies are needed in order to document the status quo or do a needs assessment in a given area of interest.
2. Predictive nonexperimental research, in which the primary focus for the research is to predict some variable of interest (typically called the criterion) using information from other variables (called predictors). The development of the proper set of predictors for a given variable is often the focus of such studies.
3. Explanatory nonexperimental research, in which the primary focus for the research is to explain how some phenomenon works or why it operates. The objective is often to test a theory about the phenomenon. Hypotheses derived from a given theoretical orientation are tested in attempts to validate the theory.
The three categories could be seen as answers to the question: Was the main purpose of the research to describe a phenomenon, to study how to predict some future event, or to understand how something operates or what drives it?
To help explain these three categories, consider the use of exit interviews. Such interviews are often conducted by organizations with employees who leave or by school systems with departing teachers and graduating seniors. An exit interview study can be descriptive if the purpose is to collect data in order to get a comprehensive picture of reasons for employees leaving their organization or school. These descriptions might be used to determine if people leave for reasons related to the organization or for personal reasons. On the other hand, the study would be predictive if exit data were collected and then related to hiring data for the same individuals for the purpose of using the results to screen potential employees and hiring people who might be less likely to leave. Finally, the study would be explanatory if the data were analyzed with the
66
purpose of testing hypotheses about how personal characteristics might be related to employee or student feelings about their organization or school.
A good example of a published descriptive study is the 39th Annual Phi Delta Kappa/Gallup Poll of the Public’s Attitudes toward the Public Schools (Rose & Gallup, 2007). Begun as an effort to inform educators, the annual survey now provides information that has policy implications. Although the accumulated database can be used to track changes in attitudes about Pre-K–12 schooling over a long period of time, the design for each yearly survey is purely descriptive in terms of its purpose. Results are a descriptive representation of how the general public feels about different aspects of public schools.
A study by Leslie Halpern and Thomas Dodson (2006) to develop a set of indicators that could identify women likely to report injuries related to intimate partner violence is an example of a predictive study. They tried to develop markers that could be used in hospital settings to make predictions about likelihood of intimate partner violence. They identified two variables as potential predictors: injury location and responses to a standard screening questionnaire. They included them, along with demographic variables, in developing a prediction model.
An explanatory study was done to examine the relationships among the variables of attachment, work satisfaction, marital satisfaction, parental satisfaction, and life satisfaction (Perrone, Webb, & Jackson, 2007). This research was informed by attachment theory, which describes “parental attachment as a stable connection that provides a feeling of safety and security for the child” (p. 238). The researchers used five published instruments and present a very good description of reliability and validity for each one.
Classification Based on Time (Dimension 2)
The categories of the second dimension for classifying nonexperimental research, which refer to time, are:
1. Cross-sectional research, in which data are collected at one point in time, often in order to make comparisons across different types of respondents or participants.
2. Prospective or longitudinal research, in which data are collected on multiple occasions starting with the present and going into the future for comparisons across time. Data are sometimes collected on different groups over time in order to determine subsequent differences on some other variable.
3. Retrospective research, in which the researcher looks back in time using existing or available data to explain or explore an existing occurrence. This backwards examination may be an attempt to find potential explanations for current group differences.
These categories could be seen as answers to the question: Were the data collected at a single time point, across some time span into the future, or were already existing data explored? You could think of them as representing the past (retrospective), present (cross-sectional), and future (prospective) with respect to timing of data collection. As an example, suppose you were interested in assessing differences in college students’ attitudes toward potential careers. In a cross-sectional study, you might take a
67
random sample of first-year college students (freshmen) and fourth-year college students (seniors) and compare their attitudes. Your purpose might be to show that more mature students (seniors) view career options differently from less mature students (freshmen).
Now consider assessing career attitudes in a prospective study. There are actually three options: trend, cohort, or panel study. To distinguish among these three approaches, think of a four-year prospective study starting in 2008 with college freshmen. The population of interest is all college freshmen in the United States. In 2008, a random sample of college freshman is taken for all three approaches. Table 4.1 describes the samples in the subsequent three years for each approach. In the trend study, the same general population (college freshmen) is tracked. In the cohort study, the same specific population (college freshmen in 2008) is tracked. In the panel study, the same individuals are tracked. One of the advantages of a panel study is that you can look for changes and not simply report on trends. A disadvantage is that you have to start with a fairly large sample due to attrition over time, particularly for a lengthy study.
An example of a retrospective study could be an examination of the educational background and experience of very successful teachers and less successful teachers. The idea is to look backward in time and examine what differences existed that might provide an explanation for the present differences in success. To the extent that such a study needed to depend on people’s memories of relevant background information, it would be less accurate than if prior data were available for examination.
For a published example, consider one question addressed by Michael Heise (2004), which was whether key actors in a criminal court case view case complexity in the same way. The results of his cross-sectional comparison of three key actor groups (juries, attorneys, and judges) suggest that they do possess slightly different views on whether crimes are complex.
Examples of both prospective and retrospective research are based on the Nurses’ Health Study, a large scale longitudinal study started in 1976 with a mailed survey of 121,700 female registered nurses between thirty and fifty-five years of age who lived in eleven states. Descriptive information about risk factors for major chronic diseases and related issues were gathered every two years. Although most of the information gathered
TABLE 4.1. Description of Samples After Initial 2008 Sampling of College Freshmen
2009 2010 2011
Trend New sample—college New sample—college New sample—college freshmen freshmen freshmen
Cohort New sample—college New sample—college New sample—college sophomores juniors seniors
Panel Same sample from Same sample from 2008, Same sample from 2008, who are now who are now juniors 2008, who are now
sophomores seniors
68
was identical, new questions were added periodically. The Nurses’ Health Study Web page (www.channing.harvard.edu/nhs) contains a complete list of publications based on these data.
One such study was conducted by Francine Laden et al. (2000). They examined the responses from the 87,497 women who answered newly included questions about lifetime use of electric blankets and heated waterbeds. Using data from the larger study, Laden and her colleagues focused their attention on the relationship between electric blanket use and breast cancer from both a prospective and retrospective view. This was done because electric blanket use is a source of electric and magnetic fields (EMFs) exposure, and EMF exposure had been hypothesized to increase the risk of breast cancer. The relevant year is 1992, when information about use of electric blankets and waterbeds was first documented. For the prospective part of their study, they considered women who had not been diagnosed with cancer as of 1992 and analyzed the occurrence of breast cancer from 1992 to 1996 for groups according to electric blanket or waterbed usage. For the retrospective part, they used records from 1976 to 1992, considering only women who were cancer free in 1976. In the prospective part of the study “exposure to electric blankets and waterbed use was assessed prior to the occurrence of breast cancer,” while in the retrospective analysis “exposure was ascertained after diagnosis” (Laden et al., 2000, p. 42).
Retrospective studies may be based on past records, as in the previous example, or on retrospective questions, that is, on questions about past behaviors or experiences. Merely using already existing data, however, does not make it retrospective. The key distinction is the study’s purpose. Are you looking backwards to discover some potential cause or explanation for a current situation, or are you using data from one point in time to predict data from a later time? Notice that Laden and her colleagues (2000) used preexisting data for both retrospective and prospective studies. For the prospective part, women who had not been diagnosed with cancer in 1992 were divided into groups based on whether they did or did not use electric blankets, and the groups were then compared with respect to breast cancer incidents by 1996. For the retrospective part, they divided the women into two groups based on whether they had or had not been diagnosed with cancer as of 1992 and then compared them in terms of reported prior use of electric blankets.
Combining Classification Dimensions
When used together, Johnson’s two dimensions (2001) combine to form a 3 × 3 design for a total of nine distinct categories that may be used to describe nonexperimental research. Examples of all nine may be found in the National Education Longitudinal Study of 1988 (NELS:88), which was a large-scale data collection effort. A nationally representative sample of eighth graders were first surveyed in 1988, with subsequent follow-up surveys every two years until 1994, and then once again in 2000. The National Center for Education Statistics’ Web page (http://nces.ed.gov/surveys/nels88) describes this study, and also provides an annotated bibliography of research done using the various data sets. Depending on which data were selected for each study and the study
69
TABLE 4.2. Articles Classified According to Both Research Objective and
Time of Dimensions
Retrospective Cross-Sectional Prospective
Descriptive Type1 Type2 Type3
Behavioral responses Criminal case The stability of of substance-exposed complexity: An undergraduate newborns: a empirical perspective students’ cognitive test retrospective study (Heise, 2004) anxiety levels (Cassady,
(Higley & Morin, 2004) 2001)
Predictive Type4 Type5 Type6
Electric blanket use A predictive model to Electric blanket use and and breast cancer in identify women with breast cancer in the the nurses’ health injuries related to nurses’ health study study (Laden et al., intimate partner (Laden et al., 2000)
2000) violence (Halpern &
Dodson, 2006)
Explanatory Type7 Type8 Type9
A further look at youth Relationships between Thirty-year stability and intellectual giftedness parental attachment, predictive validity of and its correlates: work and family roles, vocational interests. values, interests, and life satisfaction (Rottinghaus, Coon, performance, and (Perrone, Webb, & Gaffey, & Zytowski, behavior (Roznowski, Jackson, 2007) 2007)
Reith, & Hong, 2000)
purpose, different NELS:88 studies might be classified using all nine of the purpose by time frame classifications. To help clarify this cross-classification scheme, Table 4.2 gives the titles of articles representing each type, which are then described.
Type 1—Descriptive retrospective. Using retrospective chart review, Anne Marie Higley and Karen Morin (2004) described the behavior of infants whose mothers had a drug history. Their findings supported the use of an assessment tool to guide parents in providing a supportive care environment to help infants recover.
Type 2—Descriptive cross-sectional. This study was discussed earlier as an example of a cross-sectional study. It is descriptive because the goal was to document the extent to which juries, attorneys, and judges held similar or different views about a case. The results have implications for legal reform efforts.
Type 3—Descriptive prospective. This was an investigation of the stability of test anxiety measures over time and testing formats, with data collected at three time points in an academic semester, therefore making it prospective. The purpose for the description was to determine if test anxiety was a stable condition or if it is necessary to
70
include a test anxiety measure with every test in a longitudinal study. Results indicated that it is not necessary to measure anxiety with every test; it is only necessary to measure anxiety in one test-taking situation.
Type 4—Predictive retrospective and Type 6—Predictive prospective. The two parts of this study were described earlier as examples of retrospective and prospective studies. Both parts were predictive in nature, using a backward and a forward perspective to determine the extent to which electric blanket and waterbed use could be used to predict breast cancer. Although results did not exclude small risks, neither analysis supported an association between breast cancer risk and use of electric blankets and waterbeds.
Type 5—Predictive cross-sectional. In this study, discussed as an example of a predictive study, a one-time data collection was used. The authors’ aim was to develop and validate a predictive model. They subdivided their sample, using one group to develop their model and the second group to validate, or test it. Their work produced a predictive and validated model of three components: risk of self-report of intimate partner violence related injury, age, and race. The researchers then hypothesized that these three variables could be used to develop a protocol to assist in the early diagnosis of intimate partner violence in an emergency department and outpatient clinical setting.
Type 7—Explanatory retrospective. This study was explanatory because a goal was to further previous work on giftedness and knowledge and understanding of several related variables. The data came from the High School and Beyond database, a longitudinal study with baseline information on 14,825 students who were high school sophomores in 1980. The data for this study included the base year and the third follow-up survey, four years later, after graduation. The data set “allowed for more comparisons than could reasonably be included in a single study. Variables were chosen that would either serve to replicate previous findings or expand psychological and behavioral profiles of gifted male and female students into more detail” (Roznowski, Reith, & Hong, 2000, p. 96). A retrospective conclusion was that educational attainment differences of gifted males and females had their origins in the early high school years.
Type 8—Explanatory cross-sectional. Already discussed as an example of an explanatory study, this study was based on data from the fifteenth annual survey of a longitudinal study that started in 1988 with 1,724 participants. About 1,200 participants were lost in the first three years. Only 108 participants were left for this study, which shows the dramatic attrition that can happen in a longitudinal study. Although the data were from a longitudinal study, these authors only used the fifteenth year’s data, thereby making it cross-sectional.
Type 9—Explanatory prospective. The authors suggested that “Assessing the predictive validity of an interest inventory is essentially answering the question, ‘Do early interest scores match one’s future occupation?’” (Rottinghaus et al., 2007, p. 7). To answer this question, they did a thirty-year follow-up of 107 former high school juniors and seniors whose interests were assessed in 1975. The first author had collected the initial data. Their results extend research on vocational interests, indicating that interests were fairly stable even after such a long time span.
Lapan c04.tex V1 – 09/02/2008 2:46pm Page 65
Classifying Nonexperimental Research
Lapan c04.tex V1 – 09/02/2008 2:46pm Page 60
Nonexperimental Quantitative Research
Lapan c04.tex V1 – 09/02/2008 2:46pm Page 65
Classifying Nonexperimental Research
Causal Explanations and Nonexperimental Studies 71
REFLECTION QUESTIONS
1. How do descriptive, predictive, and explanatory studies differ?
2. How do retrospective, cross-sectional, and prospective studies differ?
3. Find several recent articles in your field of study where a nonexperimental design was used. Classify their main purpose as being descriptive, predictive, or explanatory and classify the time dimension as retrospective, cross-sectional, or prospective.
CAUSAL EXPLANATIONS AND NONEXPERIMENTAL STUDIES
Using Johnson’s classification system (2001), many nonexperimental studies are either descriptive or predictive. For those, the notion of causation is not relevant. However, a goal for many explanatory nonexperimental research studies is to explore potentially causal relationships. A causal relationship is one in which a given action is likely to produce a particular result.
The terms independent and dependent refer to the different roles variables play in experimental studies. If a causal relationship exists, then the outcome (the measured DV) depends on, or is a direct result of, the nature of the assigned independent treatment condition. Strictly speaking, these terms are not applicable in nonexperimental research, although they are often used. The more appropriate terms in nonexperimental studies are criterion and predictor variables, criterion being the presumed outcome of one or more predictor variables. When the intent is to use nonexperimental research to study potential cause-and-effect relationships where experimentation is not possible, the concept of IV and DV may still be of interest, but conclusions about causation that can be made from nonexperimental studies are weaker than those that can be made from true-experimental studies. Additionally, great care needs to be taken to assure that nothing essential has been overlooked.
As explained earlier, the distinction is often made between nonexperimental studies that involve both categorical and quantitative variables and those that involve only quantitative variables. Considering only two variables for the sake of simplicity, an example of the first type of study is a comparison of gender differences in mathematics achievement in high school. Gender, with male and female as the two categories, is considered the independent variable and some mathematics achievement score is the measured dependent variable. Examples where both variables are quantitative might be an examination of the relationship between test scores and time spent studying, or between scores on some measure of motivation and scores on an achievement test. Examples like these, of very simple cases involving only two variables, are neither very interesting nor very informative. Additional variables could be included in order to examine more complex relationships.
No matter which type of design or which type of variable is used, evidence of a relationship would not be convincing evidence of causality. Recall the example described earlier about investigating the relationship between education level and salary and the
72
two ways that education level could be measured. Regardless of whether education level was construed as categorical (highest degree earned) or as quantitative (number of years of schooling), it should not be concluded that one’s educational level caused or produced a different level of salary. If dramatic differences across the five groups with different degrees were found such that those with higher education had higher median salaries, all that can be concluded is that there was a relationship between educational level and salary. This same conclusion would be possible if results indicated a strong positive correlation between years of schooling and salary: that people with fewer years of school tended to have low salaries and people with more years of school tended to have high salaries (see Figure 4.1 for graphical representation of a positive relationship). The scatter plot for a negative relationship would go from the upper left corner to the lower right corner, indicating that low scores on one variable tended to go with high scores on the other variable.
The differences in the wording of the research questions in the previous two cases reflect the nature of the variables used (categorical or quantitative). They would require different analysis strategies, either to test if the median values did differ more than you might expect by chance, or to determine the strength and direction of the relationship. Differences in wording or analysis do not, however, reflect any difference in the nature of the relationship between the variables. Explanatory nonexperimental research articles often have conclusions phrased in causal language. Therefore, the next section is a review of the essential elements needed to establish cause-and-effect relationships and a discussion of their applicability to nonexperimental studies.
Requirements for Causality
There are three conditions necessary in order to be able to argue that some variable X (the presumed independent) causes another variable Y (the presumed dependent).
1. The two variables X and Y must be related. If they are not related, it is impossible for one to cause the other. For nonexperimental research, that means that it must be demonstrated that differences in X are associated with differences in Y.
2. Changes in X must happen before observed changes in Y. This is always the case when X is a manipulated treatment variable in an experiment. But establishing that a cause happened before an effect needs to be documented in some way or logically explained in nonexperimental studies. This is impossible to do when the data are cross-sectional and collected simultaneously.
3. There is no possible alternative explanation for the relationship between X and Y. That is, there is no plausible third variable that might explain the observed relationship between X and Y, possibly having caused both of them.
In nonexperimental studies, the first requirement can be established easily with correlational analyses. The second could also be established if longitudinal data are used so that predictor variables are measured before the criterion. The third requirement is more difficult to demonstrate. To do so requires a thorough knowledge of the literature and the underlying theory or theories governing the topic being investigated, logical arguments, plus testing and ruling out of alternative possibilities.
Causal Explanations and Nonexperimental Studies 73
The fact that two variables are related does not inform us of which one influences the other. There are at least three reasons why two variables could be related and it is not possible to know from the correlation which one is the correct reality. Three potential explanations are: (1) that X causes or influences Y, (2) that Y causes or influences X, or (3) that Z, a third variable, causes both X and Y. Consider the following headline: “Migraines plague the poor more than the rich.” It could be argued that the stresses of living in poverty and other poverty-related conditions could trigger migraine headaches. It could also be argued that migraines cause one to miss work and eventually lose employment, thereby inducing poverty for a subset of individuals prone to migraines. Which is the correct interpretation? It is impossible to tell.
Although there is no formal way to prove causation in nonexperimental research, it may be possible to suggest it. This is done through careful consideration, by referring to the three conditions for cause, by presenting logical arguments, and by testing likely alternatives in order to make a case for the likely conclusion of a causal relationship. One must be careful, however, not to phrase conclusions as proof of causation.
Ruling Out Alternative Hypotheses
To demonstrate the process for ruling out alternative hypotheses, we will use a medical example. Consider the process a doctor goes through in diagnosing a new patient’s illness. First, the doctor considers the symptoms. The list of symptoms is used to select potential problems with similar symptoms and to rule out problems with different symptoms. Tests are ordered to confirm the most likely diagnosis and remedies are tried. If the test results are negative or the remedies do not work, then the original diagnosis is discarded, and other possible diagnoses are considered and tested. How does this process relate to research? The first step is matching observations (the reported symptoms) to theory (known symptoms for an illness). The second step is to test a hunch or tentative hypothesis (initial diagnosis) and rule out alternative hypotheses (other potential diagnoses). The process continues until a reasonable conclusion is reached. The analogy breaks down because, ideally, the correct diagnosis is made and the patient is cured, although results are never as conclusive in nonexperimental studies.
Given a theory that is driving the research, how does one rule out potential alternative hypotheses? One way is to consider all likely confounding or lurking variables. In an experimental study, two variables are confounded when their effects on a dependent variable cannot be distinguished. The following example, although purely correlational, should clarify the concept of confounding or lurking variables.
One would expect that grades and standardized tests, such as
SAT
scores, would be related more to each other than they would to socioeconomic status (SES). In many studies, however, SES and SAT appear to have a much stronger relationship than do grades and SAT. Rebecca Zwick and Jennifer Green (2007) explored reasons for such results with data from a random sample of 98,391 students from 7,330 high schools. They performed two different analyses. In the first analysis, they found the correlation for grades and SAT for the entire sample and, in the second analysis, they did so for each school individually and then averaged the school-level results to get one overall measure of relationship. The second analysis produced a much stronger relationship
74
between grades and SAT scores than did the first analysis. This is because the first analysis ignored the fact that there are school-level differences in SES as well as other variables.
Figure 4.2 should help you visualize this discussion. In part A, the two smaller ovals represent a scatter plot of scores for two schools, where both grades and SAT scores tend to be higher in
School 2
than in
School 1
. The lines bisecting these two ovals provide a linear representation of the relationship between the variables within each school and are called regression lines. Both ovals are rather narrow in width, being fairly close to their regression lines, and thereby give a visual representation of a relatively strong positive relationship between grades and SAT within each school. The larger oval represents the relationship between grades and SAT scores as it would appear across or between schools, that is, if school membership were ignored in the analysis. It is much more spread out around its regression line (the dotted line), erroneously indicating a much weaker relationship between grades and SAT. The two smaller ovals correspond to Zwick and Green’s second analysis (2007) and the larger oval to their first analysis. Ignoring the differences between the schools confounds the relationship between grades and SAT being investigated.
Part B of Figure 4.2 shows a worst-case scenario of ignoring a lurking variable. Suppose the relationship between two variables, X and Y, is negative for each of two groups. This is shown by the two smaller ovals, where lower scores on X tend to go with higher scores on Y and vice versa within each group. Ignoring groups,
FIGURE 4.2.
Representation of Effects of Confounding Variables
SAT
Grades
School 2
School 1
Y
X
Group 2
Group 1
A. Fairly strong positive relationship
between grades and SAT within each
school. Weak relationship when
school membership is ignored.
B. Fairly strong negative relationship
between X and Y within each group.
A seemingly positive relationship
when group membership is ignored.
Analysis and Interpretation in Nonexperimental Studies 75
however, would produce a positive relationship, which would be a completely wrong conclusion.
REFLECTION QUESTIONS
By now, you should be able to
1. List and explain three essential requirements to argue cause
2. Explain why even a strong correlation does not imply causation
3. Describe why ruling out alternative hypotheses is important.
4. Find one or two nonexperimental studies in your field of study where hypotheses were tested or where a theory was explored. What extraneous variables or potential alternative hypotheses were discussed? Can you think of others that were not discussed? How might inclusion of those variables have changed results?
ANALYSIS AND INTERPRETATION IN NONEXPERIMENTAL STUDIES
Data analyses in nonexperimental studies depend on both the goal for the study and the nature of the variables in the data set. Almost any analysis may be possible and a useful presentation is not reasonable here. There are ample books and sources for details about statistical methods and their use. A few examples are given at the end of the chapter; also see the discussion on understanding quantitative data in Chapter Six.
You need to be aware of the basic distinction between descriptive and inferential statistics. Descriptive statistics involve summarizing and describing quantitative information in meaningful ways. For example, a mean, or arithmetic average, is a statistic used to describe a central value for a set of numbers. Inferential statistics are used to make conclusions beyond the data collected and to test hypotheses. Statistical tests are used to make conclusions about populations based on results from random samples or to determine the probability that results are not due to random chance.
Interpretation of results in nonexperimental studies should be consistent with the nature of the work, which is based on nonmanipulated variables. Therefore, conclusions about cause and effect are not appropriate in any nonexperimental study. As you read empirical articles, you should be attuned to how conclusions are discussed and be wary of causal language. Robinson, Levin, Thomas, Pituch, and Vaughn (2007) reviewed 274 empirical articles in five teaching-and-learning research journals in 1994 and 2004. They recorded causal and noncausal language use in abstracts and discussion sections. Their two main conclusions were: (1) experimental articles in teaching-and-learning declined in the ten-year span, and (2) on average, the use of causal conclusions made in nonexperimental and qualitative studies increased. They conclude by saying that “as journal readers, we have an obligation to search an article for information about how the data were collected so we are not unduly influenced by unwarranted conclusions” (Robinson et al., 2007, p. 412). Ideally, after studying this chapter you will be able to search through articles for information about how the study was conducted and use that to consider conclusions.
Lapan c04.tex V1 – 09/02/2008 2:46pm Page 59
Lapan c04.tex V1 – 09/02/2008 2:46pm Page 60
Nonexperimental Quantitative Research
Lapan c04.tex V1 – 09/02/2008 2:46pm Page 59
76
SUMMARY
The goal for this chapter was to present adequate information about nonexperimental designs so that a practitioner could read the literature and have a basic understanding of methods used. Nonexperimental research is described in many ways and covers any quantitative study that does not have manipulated variables or random assignment. A topic of research interest can be modified to serve alternative purposes, and data can be collected over different time frames. The two-dimensional classification system presented here should help you categorize articles. Reading any of the articles listed in Table 4.2 that are of interest to you could be useful in understanding why it was classified according to the two dimensions given. A good place to start, with a relatively straightforward example, would be the Cassady (2001) article, which is an example of Type 3, a descriptive prospective study. A good exercise would be to find other nonexperimental studies and classify them according to the two dimensions of purpose and time of data collection.
A key to understanding published research is to identify the goal of the research, evaluate what was done in relation to that goal, and consider aspects and variables that may have been overlooked. Most important, consider the language used in published works and be skeptical if overzealous researchers present their nonexperimental results in causal terms. Regardless of what type of research is presented, be a wary consumer.
KEY TERMS
attribute variables categorical variables confounding or lurking variables correlation coefficient criterion cross-sectional research dependent variable descriptive nonexperimental research descriptive statistics experimental research explanatory nonexperimental research independent variable index
inferential statistics median nonexperimental research predictive nonexperimental research predictors prospective or longitudinal research quantitative variables quasi-experiments random assignment random sample regression line
reliability retrospective research scale scatter plot true-experiments validity variable
Analysis and Interpretation in Nonexperimental Studies 77
FURTHER READINGS AND RESOURCES
Suggested Readings
Allison, P. D. (1999). Multiple regression: A primer. Thousand Oaks, CA: Pine Forge Press.
This basic text, discussing an analysis technique often used in nonexperimental studies, is written in an understandable manner, using examples from social science research literature to develop the concepts.
Johnson, R. B., & Christensen, L. See lecture in Chapter Eleven: Nonexperimental quantitative research, based on Educational Research: Quantitative, Qualitative, and Mixed Applications. Retrieved March 13, 2008, from www.southalabama.edu/coe/bset/johnson/2lectures.htm.
Discusses steps in nonexperimental research, ways to control extraneous variables in nonexperimental research, and Johnson’s classification scheme for nonexperimental research, and provides a graphic description of controlling for a third variable.
Locke, L. F., Silverman, S. J., & Waneen, W. S. (2004). Reading and understanding research (2nd ed). Thousand Oaks: Sage.
Although this book deals with research in general, it is an easily understandable resource with good examples to help you read and understand published research articles. Aimed at consumers of research, the approach is nontechnical and user-friendly.
Lowry, R. (1999–2008). Concepts and applications of inferential statistics. Retrieved October 10, 2007, from http://faculty.vassar.edu/lowry/webtext.html.
Chapter Three of this free, full-length statistics textbook provides an introduction to linear correlation and regression using examples and diagrams. This is useful for understanding the basic analyses used with nonexperimental data.
Meltzoff, J. (1997). Critical thinking about research: Psychology and related fields. Washington, DC: American Psychological Association.
This text should help develop critical thinking skills via research by critiquing exercises of different types of research studies. It combines fundamental content with practice articles.
Trochim, W. M. The research methods knowledge base (2nd ed.). Retrieved October 20, 2006, from www.socialresearchmethods.net/kb.
Of particular use is the Language of Research part of the Foundation section, where types of relationships are clearly described, using simple examples and graphs.
THESELF MANAGEMENT OF LONGER TERM DEPRESSION 1
Topic: Research Article Identification
Student’s name: Pamela Jackson
Professor’s name: Mirella Brooks
Course title: PSY 325
Institution: Ashford University
Date: 12/18/17
The Self-management of longer term depression: learning from the patient, a qualitative study by Eleni Chambers, Sarah Cook, Anna Thake, Alexis Foster, Sue Shaw, Rebecca Hutten, Glenys Parry and Tom Ricketts
This is a qualitative survey carried out by several authors with the intention of comprehending the manner in which individuals with longer haul depression deal with the sickness, the manner in which services can best back up self-administration and if the standards along with conceptions of the recuperation strategy would be beneficial. The study is making an effort to address the questions on how individuals encountered depression along with what assisted them to self administer (Chambers, Cook, Thake, Foster, Shaw, Hutten, Parry & Ricketts, 2015). This study on the self-management of longer term depression can be associated with biological psychology as a research area in psychology. This is due to the fact that the study entails looking in depth the lives of individuals with continuing depression and how they help themselves to live with it. This is to say that to some extent the study looks into physiological psychology along with health psychology. These two aspects are connected to biological psychology thus making it possible to associate the two.
Ethical issues are unavoidable in the course of carrying out a research, it is therefore advisable that psychologists look into the different ethical issues and considerations that might come up prior to, in the course of and after a research. This study addressed several ethical issues, they comprise of; informed consent; the study team enlightened participants about the survey and what they were aiming to address and giving them the chance to decide whether or not to take part in the study. The second principle was do not harm; this principle was dealt with by the survey team making use of abilities to reduce any probable discomfort or stress in the course of the interview procedure. This is to show that they had the participants’ wellbeing in mind. There was also respect for anonymity and privacy; this ethical issue is highly linked to the privileged of do not harm, esteem for dignity along with honesty. The researchers made this issue achievable by making certain that participants were not forced into giving out their personal details. On top of this, the study team gave room such that the interviews along with audio recording could be halted on the participant’s request. This is a clear indication that they did not want to go against or violate any person’s rights in the course of this survey (Chambers, Cook, Thake, Foster, Shaw, Hutten, Parry & Ricketts, 2015).
I discern that vulnerable groups of people are an ethical issue that applies but was not discussed. Individuals who are known to have been suffering from disconsolation for long periods of time are clearly are known to be vulnerable. This means that they are somehow to some extent not in a position to safeguard their personal privileges along with wellbeing. It also means that they are not capable of giving an informed consent as they can be easily manipulated. The research team ought to have addressed this ethical issue due to the fact that it was dealing with and studying a vulnerable set of individuals.
Another ethical issue that I discern should have been mentioned in the study is; the abilities of the researcher. The surveyors ought to have introduced themselves before the participants and stated their different skills as researchers. This would have acted as a way of assuring the participants that they were dealing with professional researchers and that they were safe in their hands.
Reference
Chambers, E., Cook, S., Thake, A., Foster, A., Shaw, S., Hutten, R., Parry, G., & Ricketts, T. (2015). The self-management of longer-term depression: Learning from the patient, A qualitative study. BMC Psychiatry, 15, 172. doi:10.1186/s12888-015-0550-6
