Lucene's default similarity function

Lucene's scoring Function is defined by the function

1. tf(t in d) denotes the term's frequency, defined as the number of times the term t appears in the currently scored document d. Documents that have more occurrences of a given term receive a higher score.The default computation for tf(t in d) in DefaultSimilarity is:
   


2. idf(t) stands for Inverse Document Frequency. This value correlates to the inverse of docFreq (the number of documents in which the term t appears). This means rarer terms give higher contribution to the total score. idf(t) appears for t in both the query and the document, hence it is squared in the equation. The default computation for idf(t) in DefaultSimilarity is:
   


3. coord(q,d) is a score factor based on how many of the query terms are found in the specified document. Typically, a document that contains more of the query's terms will receive a higher score than another document with fewer query terms. This is a search time factor computed in coord(q,d) function by the Similarity in effect at search time.
   
   where, overlap is the number of query terms matched in the document and maxOverlap the total number of terms in the query.


4. queryNorm(q) is a normalizing factor used to make scores between queries comparable. This factor does not affect document ranking (since all ranked documents are multiplied by the same factor), but rather just attempts to make scores from different queries (or even different indexes) comparable. This is a search time factor computed by the Similarity in effect at search time. The default computation in DefaultSimilarity produces a Euclidean norm:
   
   where, the sum of squared weights (of the query terms) is computed by the query Weight object as:
   sumOfSquaredWeights = q.getBoost()^ 2 * Sum t in q ( idf(t) * t.getBoost() ) ^2
   Lucene uses by default the value 1.0 for the factors q.getBoost() and t.getBoost().


5. t.getBoost() is a search time boost of term t in the query q. Notice that there is really no direct API for accessing a boost of one term in a multi term query, but rather multi terms are represented in a query as multi TermQuery objects, and so the boost of a term in the query is accessible by calling the sub-query getBoost() .
   The boost is 1.0 by default !


6. norm(t,d) encapsulates a few (indexing time) boost and length factors:
   
   • Document boost - set by calling doc.setBoost() before adding the document to the index. This value is 1.0 by default!
   • Field boost - set by calling field.setBoost() before adding the field to a document. This value is 1.0 by default!
   • lengthNorm(field) - computed when the document is added to the index in accordance with the number of tokens of this field in the document, so that shorter fields contribute more to the score. LengthNorm is computed by the Similarity class in effect at indexing. This function is defined as lengthNorm(String fieldName,int numTokens) , where fieldName is the name of the field and numTokens is the total number of tokens contained in fields named fieldName of doc. The returning value is a normalization factor for hits on this field of this document.