Back

Using query intent to boost retrieval results

4 minute read3 minute read

Understanding the intent behind a user’s query is crucial for delivering accurate and relevant search results. By determining whether a user is searching for specific documents or seeking answers to questions, we can tailor our retrieval systems to provide better results.

How to determine query intent

The first step in improving retrieval results is to determine the user’s intent. Queries can generally be classified into two categories:

  • Search Intent: The user knows what they’re looking for and wants to retrieve specific documents or information.
  • Answer Intent: The user is seeking an answer to a question and may not know where to find it.

By accurately identifying the intent, we can adjust our retrieval strategies accordingly.

Custom query intents

Depending on your application, you might have more specific or custom query intents. For example, you may add intents like Navigation, Command, or Clarification. Defining custom intents allows you to handle queries more precisely.

Implementing intent detection

To determine the query intent, we can use a language model or rule-based system. Below is a Python example using a language model to classify intent.

from enum import Enum
from typing import List
from pydantic import BaseModel

class Intent(str, Enum):
    Search = "SEARCH"
    Answer = "ANSWER"

class Query(BaseModel):
    intent: Intent
    keywords: List[str]
    term: str

def determine_query(input_text: str) -> Query:
    query = client.messages.create(
        messages=[{
            "role": "user",
            "content": f"Determine the query intent of <query>{input_text}</query>"
        }],
        response_model=Query
    )

    return query

Here’s how it can be used

query = determine_query("q4 planning notes")

is_search = query.intent == Intent.Search

semantic_weight = 0.3 if is_search else 0.7
fts_weight = 1 - semantic_weight

results = retrieve_documents(
    query=query.term,
    semantic_weight=semantic_weight,
    keyword_weight=fts_weight
)

In this example:

  • We define an Intent enum with possible intents.
  • The determine_query function uses a LLM to classify the intent. We could improve this by training a custom model on a bunch of examples. This is definitely doable after you’ve deployed this system.
  • We adjust the weights for semantic and keyword-based retrieval methods based on the detected intent.

Taking it further

You can extend this implementation by adding more sophisticated logic or handling additional intents.

Including confidence scores

You might want to include a confidence score for the intent prediction.

class Query(BaseModel):
    intent: Intent
    confidence: float
    keywords: List[str]
    term: str

query = determine_query(user_input)

if query.confidence < 0.8:
    semantic_weight = 0.5
    keyword_weight = 0.5
elif is_search:
    ...

Handling search intent

When the user’s intent is to search for specific information, we should prioritize exact matches and consider potential typos or misspellings. Boosting keyword-driven techniques like BM25 can significantly improve results in this case.

Example: “Q4 planning notes”

In this query, the user is likely looking for a specific document or set of documents related to Q4 planning notes. To handle this:

  • Boost BM25 and Keyword Search: Since the user probably knows what they’re looking for, prioritize documents with exact keyword matches.
  • Handle Typos and Misspellings: Implement fuzzy search techniques using tools like pg_trgm in PostgreSQL. This might not be relevant if your’re doing a semantic search.

Handling answer intent

When the user is seeking an answer, semantic understanding becomes more important. In this case, boosting semantic search techniques can help retrieve documents that contain the answer, even if they don’t directly match the query terms.

Example: “What did we decide during the Q4 planning meeting last week?”

To address this query:

  • Boost Semantic Search: Use embeddings and similarity measures to find documents that are contextually relevant.
  • Leverage Contextual Information: Incorporate metadata like dates to narrow down results to the relevant time frame.

Avoid relying on specific query patterns and punctuation

I’ve seen query intent being determined by ?. This is not a good idea. It’s a perfect example of when LLMs provide a much better user experience.

Additional examples

QueryIntentApproach
”Open the Q4 planning spreadsheet”SearchBoost exact match and consider user permissions to ensure they can access the document.
”How did our sales compare between Q3 and Q4 last year?”AnswerUse semantic search to find documents discussing sales comparisons between quarters.
”List all customers who purchased product X in the last month.”AnswerUse semantic search along with metadata filters to retrieve accurate results.

Conclusion

Effectively determining query intent can truly enhance your retrieval systems, leading to more accurate results and happier users. As always, try different methods and and evaluete to see what performs best for you. And whenever you have enough data, use that to actually optimize for what your users are searching for.

Intent behind query decides result quality. Specific documents, or answer to question? Know which, tailor retrieval, better results.

How to determine query intent

First step: determine intent. Two broad categories:

  • Search Intent: User knows target, wants specific documents or information.
  • Answer Intent: User wants answer to question, may not know where it lives.

Intent identified, retrieval strategy adjusted.

Custom query intents

Your application may need more: Navigation, Command, Clarification. Custom intents handle queries more precisely.

Implementing intent detection

Language model or rule-based system. Python example with language model:

from enum import Enum
from typing import List
from pydantic import BaseModel

class Intent(str, Enum):
    Search = "SEARCH"
    Answer = "ANSWER"

class Query(BaseModel):
    intent: Intent
    keywords: List[str]
    term: str

def determine_query(input_text: str) -> Query:
    query = client.messages.create(
        messages=[{
            "role": "user",
            "content": f"Determine the query intent of <query>{input_text}</query>"
        }],
        response_model=Query
    )

    return query

Usage

query = determine_query("q4 planning notes")

is_search = query.intent == Intent.Search

semantic_weight = 0.3 if is_search else 0.7
fts_weight = 1 - semantic_weight

results = retrieve_documents(
    query=query.term,
    semantic_weight=semantic_weight,
    keyword_weight=fts_weight
)

Here:

  • Intent enum lists intents.
  • determine_query classifies with LLM. Later: custom model trained on many examples. Doable once deployed.
  • Semantic and keyword weights follow detected intent.

Taking it further

Extend with more logic or more intents.

Including confidence scores

Confidence score on prediction.

class Query(BaseModel):
    intent: Intent
    confidence: float
    keywords: List[str]
    term: str

query = determine_query(user_input)

if query.confidence < 0.8:
    semantic_weight = 0.5
    keyword_weight = 0.5
elif is_search:
    ...

Handling search intent

Search intent: prioritize exact matches, expect typos and misspellings. Boost keyword techniques like BM25. Big gains.

Example: “Q4 planning notes”

User wants specific document(s) on Q4 planning notes. Handle:

  • Boost BM25 and Keyword Search: User knows target, prioritize exact keyword matches.
  • Handle Typos and Misspellings: Fuzzy search, e.g. pg_trgm in PostgreSQL. Less relevant for semantic search.

Handling answer intent

Answer intent: semantic understanding matters more. Boost semantic search. Finds documents with answer, even without matching query terms.

Example: “What did we decide during the Q4 planning meeting last week?”

Handle:

  • Boost Semantic Search: Embeddings and similarity find contextually relevant documents.
  • Leverage Contextual Information: Metadata like dates narrows time frame.

Avoid relying on specific query patterns and punctuation

Seen intent determined by ?. Bad idea. Perfect example of LLMs giving much better user experience.

Additional examples

QueryIntentApproach
”Open the Q4 planning spreadsheet”SearchBoost exact match, check user permissions for document access.
”How did our sales compare between Q3 and Q4 last year?”AnswerSemantic search for documents on sales comparisons between quarters.
”List all customers who purchased product X in the last month.”AnswerSemantic search plus metadata filters.

Conclusion

Intent detection improves retrieval: more accurate results, happier users. Try methods, evaluate, keep what performs. Enough data? Optimize for what users search for.