8/17/24, 7:13 PM Anshika AI Intern Assignment.

ipynb - Colab

keyboard_arrow_down Assignment: Understanding and Mitigating Hallucinations in Large Language Models

Section 1: Theoretical Knowledge

1.1. Understanding Hallucinations in LLMs

Question: What is a hallucination in the context of Large Language Models? Provide an example.

Answer: Hallucinations are defined as a phenomenon where LLMs (Large Language Models) generate contents which are non - existent,
factually incorrect or disconnected from the input prompt, creating outputs that are nonsensical or altogether inaccurate.

As a user if I am prompting a GenAI toll, I expect it to generate output which appropriately addresses the prompt (i.e., a correct answer to a
question). However, sometimes AI algorithms produce outputs that are not based on training data, are incorrectly decoded by the transformer
or do not follow any identifiable pattern. In other words, it “hallucinates” the response.

Some examples of AI hallucinations are:

1. Google’s Bard chatbot incorrectly claiming that the first person to walk on the moon was Charles Lindbergh in 1951 during the Lunar
Pioneer mission.
2. Claude giving list of research papers on codemixing from EMNLP which doesn't even exists.
3. Microsoft’s chat AI, Sydney, admitting to falling in love with users and spying on Bing employees.

Hallucinations in LLMs can be of two type: factuality hallucinations or faithfulness hallucinations.

Factuality hallucination occurs when the LLMs generate contents which are factualy incorrect. For instance, a model might claim that Charles
Lindbergh was the first to walk on the moon, which is a factual error. This type of hallucination arises due to the model's limited contextual
understanding and the inherent noise or errors in the training data.

Faithfulness hallucination occurs when the model generated unfaithful content or is inconsistent with the provided source content. When the
model output deviates from the user's input or the context provided. For example, in the context of summarization, if an article states that the
FDA approved the first Ebola vaccine in 2019, a faithfulness hallucination would include a summary claiming that the FDA rejected it or that
China started testing a COVID-19 vaccine, neither of which is mentioned in the original article​.

1.2. Causes of Hallucinations in LLMs (10 Points)

Question: Discuss some of the primary causes of hallucinations in LLMs

Answer: Multiple factors in the creation and use of Large Language Models (LLMs) contribute to their tendency to generate inaccurate or
fabricated information.

One of the significant factor is the training of the model. The nature of the training data. LLMs, such as GPT, Falcon, and LlaMa, undergo
extensive unsupervised training with large and diverse datasets from multiple origins. Verifying this data's fairness, unbiasedness, and factual
correctness is challenging. As these models learn to generate text, they may also pick up and replicate factual inaccuracies in the training data.
This leads to scenarios where the models cannot distinguish between truth and fiction and may generate outputs that deviate from facts or
logical reasoning​.

Hallucinations can also arise due to flaws in model architecture. An architectural flaw or a misaligned training objective can lead the model to
produce outputs that do not align with the intended use or expected performance.

Prompt Engineering is another possible reason for hallucinations in llms. The LLM might generate an incorrect or unrelated answer if a prompt
lacks adequate context or is worded ambiguously.

Several factors during the inference stage can also lead to hallucination. Such as defective decoding strategies and randomness in how the
model selects its words can lead to mistakes. Insufficient context attention or the information bottleneck in decoding can also lead to outputs
being factually incorrect or not related to the given context.

Sometimes, LLMs are optimized for certain outcomes, such as longer outputs, which can lead to verbose and irrelevant responses. This over-
optimization can cause models to stray from providing concise, accurate information to producing more content that may include
hallucinations.

1.3. Implications of Hallucinations

Question: Why are hallucinations problematic in real-world applications of LLMs? Provide two specific examples.

LLM halucinations can have serious implications on real world applications.

For example, a healthcare AI model can generate incorrect treatment measures for a fever, or it can incorrectly identify a benign skin lesion as
malignant, leading to unnecessary medical interventions. AI hallucination problems can also contribute to the spread of misinformation.

https://colab.research.google.com/drive/1w23ewLYzXoX93S3DaYXPR5tguPMpUx6a#scrollTo=jloB1DLgjn44&printMode=true 1/4
8/17/24, 7:13 PM Anshika AI Intern Assignment.ipynb - Colab
Another example where llm hallucinated and resulted in a disastrous outcome was when a New York attorney used ChatGPT for legal research,
leading to the inclusion of fabricated citations and quotes in a federal case. Steven Schwartz admitted he used ChatGPT to help research the
brief in a client's personal injury case against Colombian airline Avianca and unknowingly included the false citations.

It highlights the direct consequence of relying on AI generated content without verification. It not only undermines the user's trust in the tool but
can also lead to serious professional and legal repercussions, as was the case with the attorneys in Mata v. Avianca, who faced sanctions for
their reliance on AI-generated, non-existent case law​.

One significant source of hallucination in machine learning algorithms is input bias. If an AI model is trained on a dataset comprising biased or
unrepresentative data, it may hallucinate patterns or features that reflect these biases. LLM hallucinations have significant security concern,
especially in sensitive areas like cybersecurity and autonomous vehicle technologies. AI researchers are constantly developing guardrails to
protect AI tools against adversarial attacks.But in the meantime, vigilance in the training and fact-checking phases is essential.

Section 2: Practical Skills

2.1. Identifying Hallucinations

Task: Given the following generated text from an LLM, identify potential hallucinations and suggest corrections.

Inadequate information. No sample texts were provided.

2.2. Implementing a Simple Fact-Checking Mechanism

Task: Write a Python script that uses a basic fact-checking mechanism to validate whether a given statement is true or false. Assume you have
a small database of correct facts.

#Solution:
import re

# Small database of facts

fact_database = {
"The Earth orbits the Sun": True,
"Water boils at 100 degrees Celsius at sea level": True,
"The Great Wall of China is visible from space": False,
"Python is a programming language": True,
"The capital of France is London": False
}

def preprocess_statement(statement):
"""Preprocess the statement for better matching by converting them to lowercase and removing leading/trailing whitespace."""
return statement.strip().lower()

def fact_check(statement):
"""Check if the statement is true, false, or unknown."""
processed_statement = preprocess_statement(statement)

for fact, truth_value in fact_database.items():

if processed_statement == preprocess_statement(fact):
return truth_value

# If no exact match, try partial matching

for fact, truth_value in fact_database.items():
if processed_statement in preprocess_statement(fact):
return truth_value

return "Unknown"

def main():
while True:
user_statement = input("Enter a statement to fact-check (or 'quit' to exit): ")

if user_statement.lower() == 'quit':
break

result = fact_check(user_statement)

if result == True:
print("The statement is true.")
elif result == False:
print("The statement is false.")
else:
print("The truth of this statement is unknown or not in our database.")

if __name__ == "__main__":
main()

https://colab.research.google.com/drive/1w23ewLYzXoX93S3DaYXPR5tguPMpUx6a#scrollTo=jloB1DLgjn44&printMode=true 2/4
8/17/24, 7:13 PM Anshika AI Intern Assignment.ipynb - Colab

Enter a statement to fact-check (or 'quit' to exit): The sun revolves around earth
The truth of this statement is unknown or not in our database.
Enter a statement to fact-check (or 'quit' to exit): great wall of china is visible from space
The statement is false.
Enter a statement to fact-check (or 'quit' to exit): quit

2.3. Basic Mitigation Strategy

Task: Describe how you would implement a retrieval-based approach to reduce hallucinations in LLMs. Provide a high-level pseudocode.

Solution: RAG (Retrieval Augmented Generation) is a powerful technique that combines prompt engineering with context retrieval from external
data sources to improve the performance and relevance of LLMs. It gives additional context to the llms reducing the hallucination. This system
fetches relevant information from a trusted database before the model generates a response.

Here's a high-level description of the approach, followed by pseudocode:

1. When a query is received, use it to search the external knowledge base.

2. Retrieve the most relevant documents or information.
3. Provide this retrieved information as additional context to the LLM.
4. Have the LLM generate a response based on both the original query and the retrieved information.
5. Implement a fact-checking step to verify the generated response against the retrieved information.

Here's a high-level pseudocode for this approach:

'''
function reduce_hallucinations(user_query):
# Step 1: Retrieve relevant information
relevant_docs = knowledge_retrieval(user_query)

# Step 2: Prepare context

context = prepare_context(user_query, relevant_docs)

# Step 3: Generate response using LLM

response = generate_llm_response(context)

# Step 4: Fact-checking
response = fact_check(response, relevant_docs)

return response

function knowledge_retrieval(query):
# Search the knowledge base and return relevant documents
return search_knowledge_base(query)

function prepare_context(query, docs):

# Combine the original query with retrieved information
return format_prompt(query, docs)

function generate_llm_response(context):
# Use the LLM to generate a response based on the context
return llm.generate(context)

function fact_check(response, docs):

# Compare the generated response with the retrieved documents
# Modify the response if inconsistencies are found
return verified_response
'''

'\nfunction reduce_hallucinations(user_query):\n # Step 1: Retrieve relevant info

rmation\n relevant_docs = knowledge_retrieval(user_query)\n \n # Step 2: Pr
epare context\n context = prepare_context(user_query, relevant_docs)\n \n #
Step 3: Generate response using LLM\n response = generate_llm_response(context)\n
\n # Step 4: Fact-checking\n response = fact_check(response, relevant_docs)\n
\n return response\n\nfunction knowledge retrieval(query):\n # Search the know

https://colab.research.google.com/drive/1w23ewLYzXoX93S3DaYXPR5tguPMpUx6a#scrollTo=jloB1DLgjn44&printMode=true 3/4
8/17/24, 7:13 PM Anshika AI Intern Assignment.ipynb - Colab

https://colab.research.google.com/drive/1w23ewLYzXoX93S3DaYXPR5tguPMpUx6a#scrollTo=jloB1DLgjn44&printMode=true 4/4