Enable Log Scrapping or Data Masking in Azure APIM for Request and Response Bodies

Hi Sumit Gaur.

Thank you for reaching out to Microsoft Q&A

APIM API, diagnostics/Application Insights does not support Built-in body field masking. The native Data Masking settings only apply to headers and query parameters, and you can optionally limit how many body bytes are logged (up to 8192), but there’s no JSON field level redaction for request/response bodies in the diagnostics pipeline.

APIM does offer powerful policy primitives (e.g., find-and-replace, set-body, set-variable) that you can use to scrub/redact payloads before you log them to Application Insights via trace policies or to Event Hub, Storage, etc.

As an alternative you can refer the following steps below:

1. Use APIM policies to create a redacted clone and log that clone
2. Identify fields to redact (e.g., cardNumber, cvv, password).
3. Before backend call (inbound), make a sanitized copy of the request body:
4. Read the raw body into a variable.
5. Apply find-and-replace for known keys/regexes (e.g., replace digits in cardNumber with ****).
6. Store the sanitized copy in a context variable.
7. Log the sanitized copy only (to Application Insights via trace, or to Event Hub via log-to-eventhub).
8. Do not enable body logging in diagnostics or set BodyDiagnosticSettings.bytes to 0/disable request/response body logging, to prevent unredacted bodies from slipping into diagnostics.

Reference example:

<policies>
  <inbound>
    <base />
    <!-- Read inbound body -->
    <set-variable name="rawBody" value="@(context.Request.Body.As<string>(preserveContent: true))" />
    <!-- Simple redactions (illustrative) -->
    <find-and-replace in="rawBody" find="(\"cardNumber\"\s*:\s*\")\d+(\")" replace="$1****$2" />
    <find-and-replace in="rawBody" find="(\"cvv\"\s*:\s*\")\d+(\")" replace="$1***$2" />
    <find-and-replace in="rawBody" find="(\"password\"\s*:\s*\")[^\"]+(\")" replace="$1********$2" />
    <!-- Log sanitized request to Event Hub (or use trace for AI) -->
    <log-to-eventhub logger-id="eh-logger">
      @{
        var redacted = (string)context.Variables["rawBody"];
        return redacted;
      }
    </log-to-eventhub>
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound>
    <base />
    <!-- Repeat similar redaction for response if needed -->
    <set-variable name="rawResp" value="@(context.Response.Body.As<string>(preserveContent: true))" />
    <find-and-replace in="rawResp" find="(\"cardNumber\"\s*:\s*\")\d+(\")" replace="$1****$2" />
    <find-and-replace in="rawResp" find="(\"cvv\"\s*:\s*\")\d+(\")" replace="$1***$2" />
    <!-- Send sanitized response log -->
    <log-to-eventhub logger-id="eh-logger">
      @(context.Variables.GetValueOrDefault<string>("rawResp"))
    </log-to-eventhub>
  </outbound>
  <on-error>
    <base />
  </on-error>
</

If your organization is PCI Compliant, you can use Azure Key vault, Or Azure DevOps Variables to pass the Credit card data through them and then reference the Azure Key Vault secret or Azure DevOps variable in your API. This will hide and protect your data without a need to mask it at the logging.

To read secret from Key vault refer: https://stackoverflow.com/questions/77499670/read-a-secret-key-from-azure-key-vault-in-apim

To Integrate Azure DevOps variables and key vault secrets in APIM: https://medium.com/versent-tech-blog/apiops-for-azure-api-management-bcd324a0117d