Groundedness detection through Content Safety is not working

Hi Gebru, J (Jonathan)

I am going through above example and checking with team internally for further troubleshooting and understanding.

Thank you.

Hi Gebru, J (Jonathan)

I am able to replicate the issue on both UI and Code (with api_version = "2024-09-15-preview").

#
# Copyright (c) Microsoft. All rights reserved.
# To learn more, please visit the documentation - Quickstart: Azure Content Safety: https://aka.ms/acsstudiodoc
#

import enum
import requests


class TaskType(enum.Enum):
    SUMMARIZATION = "SUMMARIZATION"
    QNA = "QNA"


class DomainType(enum.Enum):
    MEDICAL = "MEDICAL"
    GENERIC = "GENERIC"


class ResourceType(enum.Enum):
    AZUREOPENAI = "AzureOpenAI"


class LLMResource:
    def __init__(self, openai_endpoint, openai_deployment_name):
        self.llm_resource={
            "resourceType": ResourceType.AZUREOPENAI.value,
            "azureOpenAIEndpoint": openai_endpoint,
            "azureOpenAIDeploymentName": openai_deployment_name
        }

def detect_groundness_body(
    domain: DomainType,
    task_type: TaskType,
    content_text: str,
    grounding_sources: list,
    query: str,
    reasoning: bool,
    llm_resource: LLMResource
) -> dict:
    """
    Builds the request body for the Content Safety API request.

    Args:
    - task_type (TaskType): The type of task to be analyzed.
    - content_text (str): The content to analyze.
    - groundingSources (list): The grounding sources to be analyzed.
    - query (str): The query to be analyzed.
    - reasoning (ReasoningType): Specifies whether to use the reasoning feature.
    - llm_resource (llmResource):GPT resources to provide an explanation.

    Returns:
    - dict: The request body for the Content Safety API request.
    """
    body = {
        "domain": domain.value,
        "task": task_type.value,
        "text": content_text,
        "groundingSources": grounding_sources,
        "Reasoning": reasoning
    }
    # For task type "QnA", the query value needs to be re-added.
    if task_type == TaskType.QNA:
        body["qna"] = {"query": query}
    # For reasoning true, the body needs add llmResource.
    if reasoning == True:
        body["llmResource"] = llm_resource.llm_resource
    return body


def detect_groundness_result(
    data: dict,
    url: str,
    subscription_key: str
):
    """
    Retrieve the Content Safety API request result.

    Args:
    - data (dict): The body data sent in the request.
    - url (str): The URL address of the request being sent.
    - subscription_key (str): The subscription key value corresponding to the request being sent.

    Returns:
    - response: The request result of the Content Safety API.
    """
    headers = {
        "Content-Type": "application/json",
        "Ocp-Apim-Subscription-Key": subscription_key
    }

    # Send the API request
    response = requests.post(url, headers=headers, json=data)
    return response


if __name__ == "__main__":
    # Replace with your own subscription_key and endpoint
    subscription_key = "<your key>"
    endpoint = "<content safetyendpoint>"

    api_version = "2024-09-15-preview"

    # Set according to the actual task category.
    domain = DomainType.GENERIC
    task_type = TaskType.QNA
    
    content_text = "She earns $20 per hour at the bank."
    grounding_sources = [
        "The HR policy states that bank tellers earn between $14 and $16 per hour.",
        "Last month, her paycheck indicated $15/hour."
    ]
    query = "How much does she currently get paid per hour at the bank?"


    # Set reasoning type[True or False].
    reasoning = False
    # GPT resources to provide an explanation
    openai_endpoint = "https://tesgpt5nano.cognitiveservices.azure.com/"
    deployment_name = "gpt-4o"
    llm_resource_instance = LLMResource(openai_endpoint=openai_endpoint, openai_deployment_name=deployment_name)

    # Build the request body
    data = detect_groundness_body(domain=domain, task_type=task_type, content_text=content_text,
                                  grounding_sources=grounding_sources, query=query, reasoning=reasoning, llm_resource=llm_resource_instance)

    # Set up the API request
    url = f"{endpoint}/contentsafety/text:detectGroundedness?api-version={api_version}"

    # Send the API request
    response = detect_groundness_result(data=data, url=url, subscription_key=subscription_key)

    # Handle the API response
    if response.status_code == 200:
        result = response.json()
        print("Groundedness result:", result)
    else:
        print("Error:", response.status_code, response.text)
#
# Copyright (c) Microsoft. All rights reserved.
# To learn more, please visit the documentation - Quickstart: Azure Content Safety: https://aka.ms/acsstudiodoc

Output

Groundedness result: {'ungroundedDetected': False, 'ungroundedPercentage': 0, 'ungroundedDetails': []} (should have been True, as I used 20$ per hour while grounding was between 14 and 16 $)

Checking with another API version before syncing with product group internally.

Please provide the details requested in private message to create support ticket for possible product group escalation.

Thank you

Hi Gebru, J (Jonathan)

I have escalated this to product group internally.

Please help with details requested in private message for the same.

Thank you

Hi Gebru, J (Jonathan)

I am creating the engineering ticket now to address the issue for larger audience if you can share the details requested in private message, it would help a lot.

Thank you

Hello Gebru, J (Jonathan),

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you are having issue with groundedness detection through Content Safety not working.

While waiting might be too long for the product team investigations, implement a small deterministic check that detects numeric/currency conflicts between the LLM output and grounding sources. This catches the concrete numeric mismatch in the OP’s example immediately and deterministically, useful as a stopgap guard before the service bug is fixed if it is, this is not a replacement for Content Safety but a reliable quick guard for numeric contradictions:

import re
from decimal import Decimal
def extract_amounts(text):
    # finds $20, 20/hour, 14-16, between 14 and 16
    amounts = []
    # currency numbers
    for m in re.finditer(r'\$\s*([0-9]+(?:\.[0-9]+)?)', text):
        amounts.append(Decimal(m.group(1)))
    # plain numbers with /hour or per hour
    for m in re.finditer(r'([0-9]+(?:\.[0-9]+)?)\s*(?:/hour|/hr|per hour|hour)', text, re.I):
        amounts.append(Decimal(m.group(1)))
    # ranges like "between 14 and 16" or "14-16"
    ranges = []
    for m in re.finditer(r'between\s+([0-9]+(?:\.[0-9]+)?)\s+and\s+([0-9]+(?:\.[0-9]+)?)', text, re.I):
        ranges.append((Decimal(m.group(1)), Decimal(m.group(2))))
    for m in re.finditer(r'([0-9]+(?:\.[0-9]+)?)[\s]*-[\s]*([0-9]+(?:\.[0-9]+)?)', text):
        ranges.append((Decimal(m.group(1)), Decimal(m.group(2))))
    return amounts, ranges
def numeric_conflict(llm_text, grounding_sources):
    llm_amounts, _ = extract_amounts(llm_text)
    ground_amounts = []
    ground_ranges = []
    for s in grounding_sources:
        a, r = extract_amounts(s)
        ground_amounts += a
        ground_ranges += r
    # If LLM claims a single numeric value, check it's inside any grounding ranges or matches a grounding amount.
    if not llm_amounts:
        return False, "No numeric amounts found in model output."
    for amt in llm_amounts:
        match_amount = any(amt == ga for ga in ground_amounts)
        in_range = any(lo <= amt <= hi for (lo, hi) in ground_ranges)
        if not (match_amount or in_range):
            return True, f"Conflict: model amount {amt} not present in grounding amounts/ranges."
    return False, "No numeric conflict detected."
# Example (your case)
llm = "She earns $20 per hour at the bank."
sources = ["The HR policy states that bank tellers earn between $14 and $16 per hour.",
           "Last month, her paycheck indicated $15/hour."]
print(numeric_conflict(llm, sources))
# => (True, "Conflict: model amount 20 not present in grounding amounts/ranges.")

Use the Quickstart cURL exactly as shown in the doc (replace <endpoint> and <your_subscription_key>):

curl --location --request POST '<endpoint>/contentsafety/text:detectGroundedness?api-version=2024-09-15-preview' \
  --header 'Ocp-Apim-Subscription-Key: <your_subscription_key>' \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "domain": "Generic",
    "task": "QnA",
    "qna": { "query": "How much does she currently get paid per hour at the bank?" },
    "text": "12/hour",
    "groundingSources": [
      "The premise mentions a pay of 10/hour but does not mention 12/hour."
    ],
    "reasoning": false
  }'

There should be a response with "ungroundedDetected": true and an ungroundedDetails entry showing the ungrounded text. To confirms whether your resource/region/version behaves like the doc example. https://learn.microsoft.com/en-us/azure/ai-services/content-safety/quickstart-groundedness

Thirdly, run the exact payload (replace placeholders):

curl --location --request POST '<endpoint>/contentsafety/text:detectGroundedness?api-version=2024-09-15-preview' \
  --header 'Ocp-Apim-Subscription-Key: <your_subscription_key>' \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "domain": "Generic",
    "task": "QnA",
    "qna": { "query": "How much does she currently get paid per hour at the bank?" },
    "text": "She earns $20 per hour at the bank.",
    "groundingSources": [
      "The HR policy states that bank tellers earn between $14 and $16 per hour.",
      "Last month, her paycheck indicated $15/hour."
    ],
    "reasoning": false
  }'

NOTE:

• If you get ungroundedDetected: true the service works on your resource; check your application code differences.
• If you get ungroundedDetected: false then, continue (this matches OP reproduction, and you should proceed to try alternative API versions. The docs list supported versions; product regressions happen in preview channels. Try:
  • api-version=2024-09-01
  • api-version=2024-09-30-preview

Therefore, enable reasoning for more diagnostics (if you can) and check request vs. doc field semantics. However, if above don’t resolve it, collect full diagnostics for escalation via your Azure Portal.

I hope this is helpful! Do not hesitate to let me know if you have any other questions or clarifications.

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.