Week four asked students to dig a bit deeper into the issues involved in quantitative data analysis: the assumptions, limitations and how the data types used may be interpreted.
There is currently a debate about how much beliefs about the nature of learning should be included in the concept of epistemic cognition. Hofer and Pintrich (1997) argue that “[a] belief about what knowledge is and how it can be described is not the same as a belief about how quickly one might go about learning” (p. 109) in reaction to Schommer’s (1990) inclusion of the speed of learning and incremental/entity beliefs of intelligence in her 63-item questionnaire on epistemological beliefs. Elby (2009) takes an empiricist position contending that much research is still required before “the community should decide, now, to etch ‘views about the nature of learning”’ into the definition of personal epistemology” (p. 138). One study that investigates one aspect of how learning beliefs impact on personal epistemologies is Bråten and Strømsø (2004) who utilised a Norwegian translation of Schommer’s (1998) Epistemological Questionnaire (SEQ) in their survey of Norwegian teacher trainers’ epistemic cognitive beliefs and how those beliefs influenced performance-approach and performance-avoidance goals.
Underpinning the research basis of Bråten and Strømsø’s (2004) design is the concept that a construct, an unobservable explanatory variable, can be detected through its conceptual overlaps with other measurable phenomenon in a process called operationalisation (Cohen, Manion, & Morrison, 2011). Bråten and Strømsø need to make assumptions regarding the acceptability of statements such as “The only thing that is certain is uncertainty itself” (Schommer, 1998, p. 561, a Norwegian translation of which used in the current study) being able to account for part of an individual’s belief about how certain knowledge itself is. A naïve psychological respondent may approach this statement either as a general opinion about the vagaries of life or as an account of how they view knowledge. Bråten and Strømsø assume the latter, although this position is not immune from criticism (Hofer & Pintrich, 1997). This short exposition cannot pursue potential deficits in the operationalisation process and will proceed with a simple description of how Bråten and Strømsø present their analysis.
Bråten and Strømsø’s (2004) two hypotheses were presented in a verbal form rather than as a statement of null and alternative hypotheses (Cohen et al., 2011). For example, they stated;
“we predicted that naïve epistemological beliefs about knowledge and knowledge acquisition would be negatively related to mastery goals but positively related to performance-avoidance”
which is a restatement of the null hypothesis of;
H0: naïve epistemological beliefs about knowledge and knowledge acquisition have no relation to mastery goals nor performance-avoidance
Alternatively, this null hypothesis could also be two-tailed rather than the single-tailed version given by the researchers if that were deemed relevant.
Bråten and Strømsø (2004) described a prior study that they had conducted in which they had conducted a factor analysis of the SEQ with business administration and education students and found support of Schommer’s (1990) four dimensions of personal epistemology. Each factors’ independence was judged to be high (“high loadings … and low overlap” (p. 377), indicating a satisfactory separation with low collinearity The paper used other tools to provide operational definitions of intelligence and motivational and attitudinal aspects (i.e. Dweck’s Theories of Intelligence Scale, Midgley and colleagues’ measures of personal goal orientation). A critical component underpinning this study was the reliability of these measures, and, accordingly, Bråten and Strømsø spent much time describing the reliability measures of each instrument that they used.
A point of caution is necessary at this point. Bråten and Strømsø (2004) rely on zero-order correlations (i.e. Pearson’s r) which do not show relationships between two variables. Rather, the statistic is a measure of how much the variables are associated. In other words, when one aspect is present, Pearson’s r indicates the amount that another variable is likely to be present. The connection, therefore, between any variable presented in numeric form must be made verbally and cannot be imputed through the strength of the numerical relationship. This is done by Bråten and Strømsø in their statement of the problem. Furthermore, the degree of correlation is not a percentage. Cohen, Manion and Morrison’s example of a 0.65 correlation results in only a 42.24% overlap (Cohen et al., 2011, p. 636). The highest zero-order correlation reported by Bråten and Strømsø is .78 but most of the others are much lower. Any claims made about the hypotheses must necessarily be weak. This weak linkage is further evidenced in the strength of the standardised regression coefficients reported for the regression analyses. Although the p-values were high, the actual beta-statistics were uniformly low. This indicates the ability to be confident in the existence of correlations between variables but that the actual correlations may be quite weak. For example, the r2 statistic returned to explain the amount of variance in performance-avoidance orientation is .31, a figure on the edge of being a “modest fit” (Cohen et al., 2011). However, Bråten and Strømsø seem confident in the veracity of their conclusions, presenting their conclusions without reference to the inherent weaknesses.
Bråten, I., & Strømsø, H. I. (2004). Epistemological beliefs and implicit theories of intelligence as predictors of achievement goals. Contemporary Educational Psychology, 29, 371–388. http://doi.org/10.1016/j.cedpsych.2003.10.001
Cohen, L., Manion, L., & Morrison, K. (2011). Research Methods in Education. Professional Development in Education (7th ed.). Abingdon: Routledge. http://doi.org/10.1080/19415257.2011.643130
Elby, A. (2009). Defining Personal Epistemology: A Response to Hofer & Pintrich (1997) and Sandoval (2005). Journal of the Learning Sciences, 18(1), 138–149. http://doi.org/10.1080/10508400802581684
Hofer, B. K., & Pintrich, P. R. (1997). The Development of Epistemological Theories: Beliefs About Knowledge and Knowing and Their Relation to Learning. Review of Educational Research, 67(1), 88–140. http://doi.org/10.3102/00346543067001088
Schommer, M. (1990). Effects of beliefs about the nature of knowledge on comprehension. Journal of Educational Psychology, 82(3), 498–504. http://doi.org/10.1037/0022-06220.127.116.118
Schommer, M. (1998). The influence of age and education on epistemological beliefs. British Journal of Educational Psychology, 68(4), 551–562. http://doi.org/10.1111/j.2044-8279.1998.tb01311.x