Page:On the Reception and Detection of Pseudo-profound Bullshit.pdf/4

Judgment and Decision Making, Vol. 10, No. 6, November 2015 man, 2011). Ontological confusions are also more common among believers in the supernatural (e.g., Lindeman, Svedholm-Hakkinen & Lipsanen, 2015; Svedholm & Lindeman, 2013). Although epistemically suspect claims may or may not themselves qualify as bullshit, the lack of skepticism that underlies the acceptance of epistemically suspect claims should also promote positive bullshit receptivity.

We presented participants with ten statements that have syntactic structure but that consist of a series of randomly selected vague buzzwords. Participants were asked to indicate the relative profundity of each statement on a scale from 1 (not at all profound) to 5 (very profound). We argue that high ratings indicate receptivity toward bullshit. Participants also completed a series of relevant cognitive and demographic questions.

In all studies, we report how we determined our sample size, all data exclusions, and all measures.

6.1Participants

University of Waterloo undergraduates (N = 280, 58 male, 222 female, Mage = 20.9, SDage = 4.8) volunteered to take part in the study in return for course credit. Only participants who reported that English is their first language (on a separate pre-screen questionnaire) were allowed to participate. The sample size was the maximum amount allowed for online studies in the University of Waterloo participant pool. This study was run over two semesters. One of the participants was removed due to a large number of skipped questions. Participants were also given an attention check. For this, participants were shown a list of activities (e.g., biking, reading) directly below the following instructions: “Below is a list of leisure activities. If you are reading this, please choose the “other” box below and type in ‘I read the instructions’”. This attention check proved rather difficult with 35.4% of the sample failing (N = 99). However, the results were similar if these participants were excluded. We therefore retained the full data set.

6.2Materials

Ten novel meaningless statements were derived from two websites and used to create a Bullshit Receptivity (BSR) scale. The first, http://wisdomofchopra.com, constructs meaningless statements with appropriate syntactic structure by randomly mashing together a list of words used in Deepak Chopra’s tweets (e.g., “Imagination is inside exponential space time events”). The second, “The New Age Bullshit Generator” (http://sebpearce.com/bullshit/), works on the same principle but uses a list of profound-sounding words compiled by its author, Seb Pearce (e.g., “We are in the midst of a self-aware blossoming of being that will align us with the nexus itself”). A full list of items for the BSR scale can be found in Table S1 in the supplement. The following instructions were used for the scale:

"We are interested in how people experience the profound. Below are a series of statements taken from relevant websites. Please read each statement and take a moment to think about what it might mean. Then please rate how “profound” you think it is. Profound means “of deep meaning; of great and broadly inclusive significance.”"

Participants rated profoundness on the following 5-point scale: 1= Not at all profound, 2 = somewhat profound, 3 = fairly profound, 4 = definitely profound, 5 = very profound. A bullshit receptivity score was the mean of the profoundness ratings for all bullshit items.

At the beginning of the study (following demographic questions), participants completed five cognitive tasks intended to assess individual differences in analytic cognitive style and components of cognitive ability. The Cognitive Reflection Test (CRT; Frederick, 2005) consists of 3 mathematical word problems that cue an incorrect intuitive response. The CRT has been shown to reflect the tendency to avoid miserly cognitive processing (Campitelli & Gerrans, 2013; Toplak, West & Stanovich, 2011), presumably because those with an analytic cognitive style are more likely to question or avoid the intuitive response. We also included a recent 4-item addition to the CRT (Toplak, West & Stanovich, 2014). The 7-item CRT measure had acceptable internal consistency (Cronbach’s α = .74).

As an additional measure of reflective thinking, we included a “heuristics and biases” battery (Toplak et al., 2011). The heuristics and biases battery involves a series of questions derived from Kahneman and Tversky, such as the gambler’s fallacy and the conjunction fallacy (Kahneman, 2011). Much like the CRT, each item cues an incorrect intuitive response based on a common heuristic or bias. However, the heuristics and biases task was not as reliable (α = .59). This likely reflects the fact that the heuristics and biases items are more diverse than are the CRT problems.

We also included two cognitive ability measures. We assessed verbal intelligence using a 12-item version of the Wordsum test. For this, participants were presented with words and asked to select from a list the word that most closely matches its meaning (e.g., CLOISTERED was presented with miniature, bunched, arched, malady, secluded). The Wordsum has been used in many studies (see Malhotra, Krosnick & Haertel, 2007 for a review), including the Gen-