Which of the following retrieval metrics is a harmonic mean of recall and precision?

The Correct answer is F1 Score

Key Points

• The F-score (also known as F-measure or F1 score) is the harmonic mean of precision and recall, calculated by the formula:
  • 2 × (Precision × Recall) / (Precision + Recall).
• A criticism of the F-score is that it may not fully reflect a system's performance, as a high F-score can result from an imbalance between precision and recall.
• Systems with high accuracy often struggle to improve one measure without negatively affecting the other.
• For applications where information retrieval is more critical than accuracy—accepting false positives to ensure all true positives are found—the F2 measure is used, with more emphasis on recall.
• Conversely, the F0.5 measure gives more weight to precision.

Additional Information

• Cumulative Gain:
  • This metric measures the total gain achieved by retrieving a set of relevant documents.
  • The gain is typically represented as the number of relevant documents retrieved.
  • It does not consider the precision and recall balance and does not calculate the harmonic mean of these two metrics.
• Discounted Cumulative Gain (DCG):
  • DCG is an improvement over cumulative gain that accounts for the position of relevant documents in the ranking.
  • It penalizes relevant documents that appear lower in the ranking by applying a discount factor, typically based on the rank of the document.
  • While it gives more weight to documents ranked higher, it does not directly involve a harmonic mean of precision and recall.
• Normalized Discounted Cumulative Gain (NDCG):
  • NDCG is a normalization of DCG to make it comparable across different queries or systems.
  • It normalizes DCG by dividing the DCG by the ideal DCG (IDCG), which represents the best possible ranking for that query.
  • Like DCG, it does not involve a harmonic mean of precision and recall.