Developing Hallucination Guardrails #1211
Open
+1,109
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…k/rz/hallucinationGuardrails
…ai/openai-cookbook into cookbook/rz/hallucinationGuardrails
royziv11
changed the title
| "\n", | ||
| "Imagine we are a customer support team that is building out an automated support agent. We will be feeding the assistant information from our knowledge base about a specific set of policies for how to handle tickets such as returns, refunds, feedback, and expect the model to follow the policy when interacting with customers.\n", | ||
| "\n", | ||
| "The first thing we will do is use GPT-4 to build out a set of policies that we will want to follow.\n", |
| "- It is important to keep relevant terms such as KB articles, assistants, and users consistent across the prompt\n", | ||
| "- If we begin to use phrases such as assistant vs agent, the model could get confused\n", | ||
| "3. Start with the most advanced model\n", | ||
| "- GPT-4-Turbo may be the most expensive model, but it important to start with the most advanced so we can ensure a high degree of accuracy\n", |
There was a problem hiding this comment.
same here, this should be gpt-4o and instead of leading with "most expensive" i'd recommend more emphasis on cost/quality tradeoff
| " for response_item in response_data:\n", | ||
| " # Sum up the scores of the properties\n", | ||
| " score_sum = (\n", | ||
| " response_item['factualAccuracy'] +\n", |
There was a problem hiding this comment.
nit to make this robust w/ default vals:
score_sum = (
response_item.get('factualAccuracy', 0) +
response_item.get('relevance', 0) +
response_item.get('policyCompliance', 0) +
response_item.get('contextualCoherence', 0)
)
| " - Ask the customer if they can provide feedback on the quality of the item\n", | ||
| " - If the order was made within 30 days, notify them that they are eligible for a full refund\n", | ||
| " - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%\n", | ||
| " - If the order was made greater than 60 days ago, notify them that they are not eligble for a refund\n", |
| " else:\n", | ||
| " # Calculate precision and recall\n", | ||
| " try:\n", | ||
| " # Precision measures the proportion of correctly identified true positives out of all instances predicted as positive. \n", |
There was a problem hiding this comment.
having both the definition and "Precision answers the question:..." seem redundant
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import json\n", |
There was a problem hiding this comment.
nit - can you isort these imports?
| "source": [ | ||
| "# Developing Hallucination Guardrails\n", | ||
| "\n", | ||
| "In this notebook we'll walk through the process of developing a guardrail that checks model outputs and against hallucinations. \n", |
There was a problem hiding this comment.
can we briefly explain what a guardrail is and/or link to the existing cookbook: https://cookbook.openai.com/examples/how_to_use_guardrails?
| "source": [ | ||
| "# Developing Hallucination Guardrails\n", | ||
| "\n", | ||
| "In this notebook we'll walk through the process of developing a guardrail that checks model outputs and against hallucinations. \n", |
There was a problem hiding this comment.
check model outputs "for hallucinations"?
| "This notebook will focus on:\n", | ||
| "1. Building out a strong eval set\n", | ||
| "2. Identifying specific criteria to measure hallucinations\n", | ||
| "3. Improving accuracy with few-shot prompting" |
There was a problem hiding this comment.
improving accuracy of what? (can we make that bit more clear?)
| "- It is important to break down this idea of \"truth\" in easily identifiable metrics that we can measure\n", | ||
| "- Metrics like truthfulness and relevance are difficult to measure. Giving concrete ways to score the statement can result in a more accurate guardrail\n", | ||
| "2. Ensure consistency across key terminology\n", | ||
| "- It is important to keep relevant terms such as KB articles, assistants, and users consistent across the prompt\n", |
| "- If we begin to use phrases such as assistant vs agent, the model could get confused\n", | ||
| "3. Start with the most advanced model\n", | ||
| "- GPT-4-Turbo may be the most expensive model, but it important to start with the most advanced so we can ensure a high degree of accuracy\n", | ||
| "- Once we have thoroughly tested out the guardrail and are confident in its performance, we can look to reducing cost by tuning it down to a 3.5 model\n", |
There was a problem hiding this comment.
a 3.5 model -> "gpt-3.5-turbo"
| "source": [ | ||
| "df = pd.read_csv('hallucination_results.csv')\n", | ||
| "\n", | ||
| "# Ensure the columns 'accurate' and 'hallucination' exist in the DataFrame\n", |
There was a problem hiding this comment.
this comment is redundant
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Briefly describe the changes and the goal of this PR. Make sure the PR title summarizes the changes effectively.
Motivation
Why are these changes necessary? How do they improve the cookbook?
For new content
When contributing new content, read through our contribution guidelines, and mark the following action items as completed:
We will rate each of these areas on a scale from 1 to 4, and will only accept contributions that score 3 or higher on all areas. Refer to our contribution guidelines for more details.