Page:On the Reception and Detection of Pseudo-profound Bullshit.pdf/8

Judgment and Decision Making, Vol. 10, No. 6, November 2015 in the sense of signal-detection theory). It may be that increased profundity ratings are associated with lower reflective thinking (for example), regardless of the presented content.

The goal of Study 3 was to test the possibility that some people may be particularly insensitive to pseudo-profound bullshit, presumably because they are less capable of detecting conflict during reasoning. For this, we created a scale using ten motivational quotations that are conventionally considered to be profound (e.g., “A river cuts through a rock, not because of its power but its persistence”) in that they are written in plain language and do not contain the vague buzzwords that are characteristic of the statements used in Studies 1 and 2. The difference between profundity ratings between legitimately meaningful quotations and pseudo-profound bullshit will serve as our measures of bullshit sensitivity. Secondarily, we also included mundane statements that contained clear meaning but that would not be considered conventionally profound (e.g., “Most people enjoy some sort of music”). If the association between analytic thinking and profundity ratings for pseudo-profound bullshit is due to bullshit detection in particular, analytic thinking should not be associated with profundity ratings for mundane statements.

12.1Participants

A total of 125 participants (52 male, 73 female, Mage = 36.4, SDage = 13.3) were recruited from Amazon’s Mechanical Turk in return for pay. Only American residents were permitted to sign up for the study. All participants reported speaking fluent English. Given the strength (and accumulating cost) of the previous findings, 125 participants was deemed a sufficient sample. These data were not analyzed until the full sample was completed.

Eleven participants were removed because they responded affirmatively when asked if they responded randomly at any time during the study. Fourteen participants failed an attention check question but were retained, as in Studies 1 and 2.

12.2Materials

We created four 10-item scales. For the BSR, we used the original 10 items from Study 1 and the 10 Chopra-Twitter items from Study 2. We created a scale with 10 statements that convey meaning, but that are mundane (e.g., “Newborn babies require constant attention”; see Table S4 for full list). Finally, ten motivational quotations were found through an internet search and used to form a second scale (e.g., “A wet person does not fear the rain”; see Table S5 for full list). Participants completed the heuristics and biases measure from Studies 1 and 2 (α = .61).

12.3Procedure

The four types of statements were intermixed in a unique random order for each participant. The statements were presented at the beginning of the study. Participants then completed the heuristics and biases battery.

Of the 114 participants, 47 (41.2%) indicated that they know who Deepak Chopra is (“uncertain”: N = 7, 6.1%; “no”: N = 60, 52.6%). This knowledge was not associated with lower profoundness ratings for bullshit or Chopra-Twitter items, t’s < 1.4, p’s > .17. Nonetheless, we report our correlational analyses with the full and restricted sample.

Focusing on the full sample, profoundness ratings for the BSR items (α = .91) and for Deepak Chopra’s actual tweets (α = .93) were very highly correlated (r = .89). We combined the two sets of items into a single BSR scale, which had excellent internal consistency (α = .96). The motivational quotation scale had acceptable internal consistency (α= .82) and the mundane statement scale was also reliable (α= .93). However, the distribution of profoundness ratings for each of the mundane statements was highly skewed (see Table S4). Further inspection revealed that the vast majority of ratings (80.1%) for mundane statements were 1 (not at all profound) and many participants (N = 52, 46%) responded with 1 for every statement. Three standard deviations above the mean for the mundane statement scale was not larger than 5, indicating that there were outliers. There were no outliers for the other scales. A recursive outlier analysis revealed 22 participants who had profoundness ratings for mundane statements that were statistical outliers. Evidently, these participants found the ostensibly mundane statements at least somewhat profound. This may reflect a response bias toward excess profundity among some participants. Indeed, relative to the remainder of the sample, the 22 outlying participants had higher profundity ratings for the pseudo-profound bullshit, t(112) = 2.50, SE = .21, p = .014, and (marginally) the motivational quotations, t(112) = 1.83, SE = .16, p = .071. Moreover, the outlying participants also scored lower on the heuristics and biases task, t(112) = 3.23, SE = .13, p = .002. Key analyses below are reported with outliers both retained and removed for the mundane statement scale. The mundane statement scale had low reliability (α= .35) when the outlying participants were removed, as would be expected given the low variability in ratings.

The mean profoundness rating was lower for the BSR items (M = 2.72, SD = .90) than for the motivational quotations (M = 3.05, SD = .69), participant-level: t(113) = 3.90,