Page:On the Reception and Detection of Pseudo-profound Bullshit.pdf/6

Judgment and Decision Making, Vol. 10, No. 6, November 2015 tion of vague buzzwords organized in a sentence with syntactic structure. This tendency was significantly related to cognitive variables of conceptual interest in expected ways. In Study 2 we set out to replicate this pattern of results using real-world examples of bullshit. For this, we created an additional scale using particularly vague “tweets” from Deepak Chopra’s “Twitter” account (see Table S2). We also expanded our measures of analytic cognitive style by including self-report measures of analytic and intuitive thinking disposition. Finally, we expanded our cognitive ability measures by increasing the number of items on the numeracy test and including a common measure of fluid intelligence.

9.1Participants

A total of 198 participants (98 male, 100 female, Mage = 36, SDage = 11.4) were recruited from Amazon’s Mechanical Turk in return for pay. Only American residents were permitted to sign up for the study. All participants reported speaking fluent English. Given the novelty of the phenomenon, we chose 200 participants as an arbitrary target sample size, as we determined this would provide adequate power and stability of the correlations. These data were not analyzed until the full sample was completed.

Eleven participants were removed because they responded affirmatively when asked if they responded randomly at any time during the study. In addition, 23 participants failed at least one of three attention check questions. The instruction check questions included the one used in Study 1 as well as the following question inserted into questionnaires at the middle and end of the survey: “I have been to every country in the world” (all participants who selected any option but “strongly disagree” were removed). However, as in Study 1, the results were similar when these participants were excluded and we therefore retained the full sample.

9.2Materials

In addition to the 10 meaningless statements used in Study 1, we obtained 10 novel items from http://wisdomofchopra. com and http://sebpearce.com/bullshit/. As noted, we also obtained 10 items from Deepak Chopra’s Twitter feed (http://twitter.com/deepakchopra; e.g. “Nature is a selfregulating ecosystem of awareness”). These items can be found in Table S2. We excluded hash tags and expanded any shortened words and abbreviations, but the tweets were not otherwise altered. We emphasize that we deliberately selected tweets that seemed vague and, therefore, the selected statements should not be taken as representative of Chopra’s tweet history or body of work. Also, to reiterate, we focus on Chopra here merely because others have claimed that some of the things that he has written seem like “woo-woo nonsense” (e.g., Shermer, 2010) and because of the connection between these claims and the bullshit generator websites that we used. None of this is intended to imply that every statement in Chopra’s tweet history is bullshit. Participants were given the same instructions as Study 1 and, therefore, we did not indicate the author of the statements.

Participants completed one cognitive task and one selfreport questionnaire intended to assess individual differences in analytic cognitive style. Participants were given the heuristics and biases battery (as in Study 1; α = .75) along with Pacini and Epstein’s (1999) Rational-Experiential Inventory. The latter includes the 20-item Need for Cognition (NFC) scale and the 20-item Faith in Intuition scale (FI). Both scales had excellent reliability: α = .93 (NFC) and .94 (FI). Participants were given questions such as “reasoning things out carefully is not one of my strong points” (NFC, reverse scored) and “I like to rely on my intuitive impressions” (FI). They were asked to respond based on a 5 point scale from 1-Definitely not true of myself to 5-Definitely true of myself.

To assess cognitive ability, we retained the Wordsum (α = .63), and the numeracy test from Study 1. However, given the low reliability for the 3-item numeracy test in Study 1, we used an additional 6 items (Lipkus et al., 2001), which lead to better reliability for the full 9-item scale (α = .63). We also added a short form of Raven’s Advanced Progressive Matrices (APM) that consists of 12 problems. The APM are a widely used measure of fluid intelligence and the short form has been validated in multiple studies (Arthur & Day, 1994; Chiesi, Ciancaleoni, Galli, Morsanyi & Primi, 2012). It had acceptable internal consistency in our sample (α = .69).

We used the same ontological confusion (α = .75) and religious belief measure (α = .96) as in Study 1. Finally, we administered the Paranormal Belief Scale (Tobacyk, 2004; Pennycook et al., 2012) with the religious belief items excluded. The scale consisted of 22 items sampled from 6 categories of supernatural belief (example items in parentheses): Psi (“Mind reading is possible”), Witchcraft (“Witches do exist”), Omens of luck (“Black cats can bring bad luck”), Spiritualism (“It is possible to communicate with the dead”), Extraordinary life forms (“The Loch Ness monster of Scotland exists”) and Precognition (“Astrology is a way to accurately predict the future”). The full scale had excellent internal consistency (α = .96).

Participants also completed wealth distribution and political ideology measures. These measures were included as part of separate investigations and will not be analyzed or discussed further.