Page:The World Within Wikipedia： An Ecology of Mind.pdf/6

Information 2012, 3 the columns of context words to the most frequent words only. Although the usefulness of these variations has been disputed in other frameworks, we nevertheless follow the original process for replication purposes. A third variant of COALS transforms the co-occurrence matrix using singular value decomposition (SVD). SVD is the key step in LSA and is used in COALS in a similar way: To eliminate noise in the matrix. In this variant, called COALS-SVD, the matrix A is first reduced to its most common 15,000 rows and 14,000 columns, forming the submatrix B. Phi-normalization as described above is applied to B, and finally B is transformed using SVD into three matrices

$$B=U \Sigma V^T$$

where U and V are orthonormal matrices and Σ = diag(σ1, . . . , σn ) and σ1 ≥. . . ≥ σn ≥ 0. The σi are the singular values of the matrix B. The desired matrix for word-word comparisons is U, whose row vectors are the SVD-denoised versions of B’s row vectors. Observe that right multiplying B by VΣ−1 yields U

$$BV\Sigma^{-1}=U^A$$

By this identity, the full vocabulary of the original A matrix may be projected into the SVD solution for B, as long as the column dimensions of A and B match (e.g., 14,000). To do this simply right multiply A by VΣ−1

$$AV\Sigma^{-1}=U^A$$

UA’s row vectors are SVD-denoised versions of A’s row vectors, defined by the SVD solution of B.

2.2. Explicit Semantic Analysis

Explicit Semantic Analysis (ESA) uses the article structure of Wikipedia without considering the link structure,. The intuition behind ESA is that while traditional corpora are arranged in paragraphs, which might contain a mixture of latent topics, the topics in Wikipedia are explicit: Each article is a topic, or correspondingly a concept. ESA defines term vectors in terms of their occurrence in Wikipedia articles. Because the meaning representation of each word is defined by its co-occurrence with article concepts, ESA can be considered as a word-concept level model. ESA vectors are based on frequency counts weighted by a variation of term frequency-inverse document frequency (tf-idf) [35]:

$$tf_{ij}=\begin{cases} 1+log(\upsilon_{ij}) & \mbox{if }\upsilon>0 \\ 0 & \mbox{otherwise} \end{cases}$$

where υij is the number of occurrences of a term i in an article j. Correspondingly:

$$idf_i=log \frac{|A|}{|a_j : t_i \mathcal{2} a_j|}$$

where |A| is the total number of Wikipedia articles and the denominator is the number of articles in Wikipedia that contain a given term. An ESA vector for term i is defined by: