Page:On the Reception and Detection of Pseudo-profound Bullshit.pdf/10

Judgment and Decision Making, Vol. 10, No. 6, November 2015

Twenty-three participants were removed because they responded affirmatively when asked if they responded randomly at any time during the study. Twelve participants failed an attention check question but were retained as removing them had no effect on the pattern of results.

15.2Materials

We used the BSR (10 items) from Study 1. We used the same motivational quotation scale from Study 3 (see Table S6 for full list). Participants also completed the heuristics and biases battery (α = .67) from Studies 1-3 and the paranormal belief scale (including religious belief items; α = .96) from Study 2. We measured conspiracy ideation using a 15-item general conspiracy beliefs scale (Brotherton, French & Pickering, 2013). The scale included items such as “A small, secret group of people is responsible for making all major world decisions, such as going to war” (α = .95). Responses were made on the following 5-point scale: 1) Definitely not true, 2) Probably not true, 3) Not sure/cannot decide, 4) Probably true, 5) Definitely true. For the complementary and alternative medicine scale, we asked participants to rate the degree to which they believe in the efficacy of 10 common types of alternative medicines (CAM; Complementary and Alternative Medicine, e.g., homeopathy) on the following 5-point scale (Lindeman, 2011): 0) Don’t know/cannot say [removed from analysis], 1) Do not believe at all, 2) Slightly believe, 3) Moderately believe, 4) Believe fully. An overall CAM score was created by summing the responses (α = .94).

Participants also completed a ten item personality scale (Gosling, Rentfrow & Swann, 2003) that indexes individual differences in the Big Five personality traits (extraversion, agreeableness, conscientiousness, emotional stability, and openness). These data will not be considered further.

15.3Procedure

The bullshit and motivational statements were presented first in a unique random order for each participant. Participants then completed the remainder of the tasks in the following order: Heuristics and biases battery, personality scale, paranormal belief scale, conspiracy ideation scale, and CAM scale.

Of the 217 participants, 98 (42.2%) indicated that they know who Deepak Chopra is (“uncertain”: N = 33, 14.2%; “no”: N = 101, 43.5%). This knowledge was not associated with lower profundity ratings for bullshit statements (“yes” M = 2.2; “no/maybe” M = 2.35), t(230) = 1.34, SE = .10, p = .182. Nonetheless, in keeping with Studies 2 and 3, we report our correlational analyses with the full and restricted sample.

Focusing on the full sample, the 10-item BSR scale had good internal consistency (α = .89) and the 10-item motivational quotation scale was also reliable (α = .80). The mean profoundness rating was higher for the motivational quotations (M = 3.13, SD = .67) than the BSR items (M = 2.29, SD = .82), participant-level: t(231) = 15.93, SE = .05, p < .001, item-level: t(18) = 9.45, SE = .09, p < .001, although the motivational quotations were far from ceiling.

BSR was negatively correlated with heuristics and biases performance and positively correlated with paranormal belief, conspiracist ideation, and belief in the efficacy of complementary and alternative medicine. However, the mean profoundness ratings for the BSR and motivational quotations was strongly correlated (r = .43) and, in contrast to Study 3, the motivational quotation scale was correlated with heuristics and biases performance (p = .035). The mean profoundness rating for motivational quotations was also positively correlated with conspiracist ideation, complementary and alternative medicine, and (marginally) paranormal belief (p = .088). Thus, as in Study 3, we computed a “bullshit sensitivity” variable by subtracting the mean profundity ratings for the motivational quotations from the bullshit items. Unlike in Study 3, however, heuristics and biases performance was not significantly correlated with bullshit sensitivity in the full sample (r = .10, p = .121). There was also no correlation between bullshit sensitivity and conspiracist ideation (r = –.03, p = .652) or complementary and alternative medicine (r = –.08, p = .218). In contrast, paranormal belief remained negatively correlated with bullshit sensitivity (r = –.21, p = .002).

Unlike in Studies 2 and 3, the pattern of results was different when the analysis was restricted to those with no knowledge of Deepak Chopra. Namely, when the analysis was restricted, bullshit sensitivity was significantly positively correlated with heuristics and biases performance (r = .19, p = .032). Moreover, conspiracist ideation was marginally negatively associated with bullshit sensitivity (r = –.16, p = .070). Paranormal belief remained negatively correlated (r = –.23, p = .009) and complementary and alternative remained uncorrelated (r = –.06, p = .497) with bullshit sensitivity. These results support the idea that the difference between profundity ratings for genuine motivational quotations and pseudo-profound bullshit can be used as a measure of bullshit sensitivity. However, they also indicate that caution is required – at least when the 10-item scales are used – as familiarity with Deepak Chopra may limit the usefulness of the scale. Chopra has a distinct style and it is possible that prior knowledge may have confounded our bullshit measure. For example, it may have helped some people detect the bullshit. Conversely, among those who have a favorable opinion of Chopra, this may have artificially inflated profoundness ratings for the bullshit.