[Bug]: Uncaught Exception in "default_parse_choice_select_answer_fn" when using default LLMRerank pieces

[crawftv]

Bug Description

The default rerank LLM was delivering answers that could not be parsed by "default_parse_choice_select_answer_fn".
The LLMRerank answer can add extra text to an answer that breaks the parsing.
The "DEFAULT_CHOICE_SELECT_PROMPT_TMPL" sets the expectation of an answer as "Doc: 9, Relevance: 7\n" but an llm can add extra text like in my case: ' Doc: 2, Relevance: 8 (The document mentions taking a "tasty turn around Barcelona\'s Santa Caterina market" and listening to an episode about Barcelona.)'

I used a regular expression to be more spercific about what the function is doind and solve this problem for myself.

Version

0.10.4

Steps to Reproduce

rerank_llm = Ollama(model="mistral", temperature=0.1)
query_engine = index.as_query_engine(similarity_top_k=10, llm=rerank_llm,
                                     node_postprocessors=[
                                         LLMRerank(
                                             llm=rerank_llm,
                                             choice_batch_size=5,
                                             top_n=2,
                                         )
                                     ],
                                     response_mode="tree_summarize",
                                     )

my answer from the llm_rerank from my pipeline and data:

answer = ' Doc: 2, Relevance: 8 (The document mentions taking a "tasty turn around Barcelona\'s Santa Caterina market" and listening to an episode about Barcelona.)\nDoc: 4, Relevance: 6 (The document mentions Ferramenta in Barcelona and recommends cocktails and pasta dishes there.)' 
default_parse_choice_select_answer_fn(answer,5)

Relevant Logs/Tracbacks

/.venv/bin/python /high_life/high_life_agent.py
Traceback (most recent call last):
  File "/high_life/high_life_agent.py", line 40, in <module>
    response = query_engine.query(query_str)
  File "/.venv/lib/python3.10/site-packages/llama_index/core/base/base_query_engine.py", line 40, in query
    return self._query(str_or_query_bundle)
  File "/.venv/lib/python3.10/site-packages/llama_index/core/query_engine/retriever_query_engine.py", line 186, in _query
    nodes = self.retrieve(query_bundle)
  File "/.venv/lib/python3.10/site-packages/llama_index/core/query_engine/retriever_query_engine.py", line 143, in retrieve
    return self._apply_node_postprocessors(nodes, query_bundle=query_bundle)
  File "/.venv/lib/python3.10/site-packages/llama_index/core/query_engine/retriever_query_engine.py", line 136, in _apply_node_postprocessors
    nodes = node_postprocessor.postprocess_nodes(
  File "/.venv/lib/python3.10/site-packages/llama_index/core/postprocessor/types.py", line 55, in postprocess_nodes
    return self._postprocess_nodes(nodes, query_bundle)
  File "/.venv/lib/python3.10/site-packages/llama_index/core/postprocessor/llm_rerank.py", line 99, in _postprocess_nodes
    raw_choices, relevances = self._parse_choice_select_answer_fn(
  File "/.venv/lib/python3.10/site-packages/llama_index/core/indices/utils.py", line 113, in default_parse_choice_select_answer_fn
    answer_relevances.append(float(line_tokens[1].split(":")[1].strip()))
ValueError: could not convert string to float: '8 (The document mentions taking a "tasty turn around Barcelona\'s Santa Caterina market" and listening to an episode about Barcelona.)'

[dosubot]

🤖

It's great to see that you've identified the issue and even come up with a solution using regular expressions to handle the extra text added by LLMRerank. Your proposed solution sounds like a step in the right direction. If you're willing to contribute your solution to the repository, creating a pull request would be a fantastic way to share your improvement with the community. Keep up the good work!

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.