Page:The World Within Wikipedia： An Ecology of Mind.pdf/13

Information 2012, 3

4. Study 2: Semantic Feature Production Norms

Although the WordSimilarity-353 data set has been widely used, it does have one notable weakness as a measure of human semantic behavior, namely its size. Given its limited size, it is possible that the results from Study 1 might not generalize. In order to assess the ability of the W3C3 model to generalize to new data sets, we applied the constituent models and W3C3 model from Study 1 to a large set of semantic feature production norms. For the sake of exposition we will refer to these norms using the first initial of the last name of the authors presenting the norms, MCSM.

The MCSM norms consist of 541 nouns, or concepts. Participants were asked to list features for a concept such as the physical, functional, or encyclopedic aspects of the concept. The generated features were regularized, e.g., usually has a tail and tail would be coded as has tail. After regularization, 2,526 features remained. Thus the data can be represented as a 541 × 2,526 matrix whose cell values υij are the number of production occurrences for a feature j given a concept i. From this matrix, a 541 × 541 matrix was created in which the value at each cell is the cosine of two 2,526 dimension row vectors associated with their respective concepts. The 541 × 541 matrix has 95,437 non-zero pairs representing similarities between concepts. Although the collection methodology for the feature norms is an associative production task, this 541 × 541 matrix represents the feature overlap between concepts, which is more comparative in nature.

Table 8 presents the Pearson correlations between the similarities from the 541 × 541 similarity matrix and both the predictions from the constituent models and the W3C3 model. The overall pattern of correlations in Table 8 has a striking similarity to those of Table 3. First, the correlations of ESA and WLM are almost identical in both cases. Secondly, the correlation for COALS is significantly greater than that of both ESA and WLM. And finally, the correlation of the W3C3 model is significantly greater than all of the constituent models. That the pattern of correlations is the same in both cases, especially when the MCSM set is so large, suggests that the properties of the models observed in Study 1 are generalizing in a systematic way.

In order the assess the relative contributions of each constituent model to MCSM performance, a linear regression was conducted. The regression used COALS, ESA, and WLM raw scores to predict the MCSM similarity scores. The results of the linear regression are presented in Table 9. Tolerance analyses were conducted to test for multicollinearity of COALS, ESA, and WLM by regressing each on the other two. The obtained tolerances, all between 0.55 and 0.68, suggest that the three models are not collinear. In contrast to the WordSimilarity-353 regression presented in Table 6, the constituent models