Latent Semantic Indexing and Vector Space Model




Consider a model of retrieval that combines the vector space model (VSM) with the latent semantic indexing (LSI). For the VSM you can use your results from lab-6 with the DLARS collection. Use also your results from lab-7 and construct the term x document matrix, say . Apply SVD analysis (Singular Value Decomposition) on matrix X and calculate a k-rank approximation of matrix X. Project the documents and the queries in the k-dimensional space and calculate the similarities between documents and queries in the new space.
Let are the similarity scores of VSM and LSI respectively. Calculate the final score between a document and a query from the relation:

Calculate the final score of the documents for all the queries and evaluate your results with trec_eval. Investigate the performance of the combined system for several values of k and .