In Fact, Retrieval Augmented Generation Is Also Not Enough to Meet User Requirements in Healthcare.
KGKannan Govindarajan · 1 min read
Because Sometimes You Want Exhaustive Answers
In the previous blog, we discussed how LLMs by themselves are insufficient for satisfying user
requirements in healthcare. We suggested, at the minimum, we needed frameworks like
RAG to provide a natural language interface for retrieval-oriented tasks in
healthcare. We also discussed ways to mitigate hallucinations in retrieval-oriented
applications in healthcare where there is ground truth like data in the EHR system.
However, dealing with hallucinations only aims to ensure that the assertions in the
text generated by the LLM are true. We have two additional requirements from
healthcare users. These are: exhaustive enumeration, and semantics of not finding
anything. Let us look at each in turn and see why these requirements cannot be met
reliably even with RAG like frameworks.
Exhaustive Enumeration
Our users often want information like what allergies the patient has, or the entire list
of medications. In many such situations, leaving out any piece of information can be
problematic. Imagine giving a medication that interacts poorly with the one left out of
the enumeration or triggering an allergic reaction because we were not aware of an
allergy. It is in these situations where LLMs, even RAG-like approaches struggle. To
illustrate this problem, below are a couple of screenshots of sample interactions with
ChatGPT. I was trying to see what happens when I type in two natural language queries
that are semantically equivalent to most humans and what the response is. I have also
truncated the output to the first bullet point, because it was sufficient to make my
point.
You own your healthcare data.
You should be able to use it as you like, with AI you trust. Let’s start the discussion.