FastAPI remove duplicate logs in App Insights

how do i check multiple SDKs?

the code you see there is what I used. I only installed azure-monitor-opentelemetry.

I've tried logger.propagate = False

everything about how app is run (exact command) is above. This wasn't dummy code, this was the exact code and it's not an excerpt of a big project, that was all the code.

Hello , Thanks for the update. The behavior you’re seeing is consistent with two log pipelines being active in Code 1 when FastAPI is involved:

• Your explicit configure_azure_monitor(logger_name="dice-game-logger") attaches an OpenTelemetry LoggingHandler to that logger.
• The Azure Monitor OpenTelemetry distro also enables logging instrumentation (alongside FastAPI/ASGI instrumentation). Under FastAPI, that instrumentation can pick up the same Python logging record again, yielding two exports of the same message to Application Insights. Code 2 doesn’t activate the ASGI path, so you only see one copy

Below are ways to make changes in your Code that emit exactly one log per message.

Let the distro capture logs, don’t attach your own handler. Remove the logger_name and any handler/propagation tweaks. The distro’s logging instrumentation will capture your records once.

# --- Code 1 (deduped): use the distro's logging instrumentation only ---
import logging
import random
import asyncio
from fastapi import FastAPI
from azure.monitor.opentelemetry import configure_azure_monitor

CONNECTION_STRING = "XXX"

# 1) Configure once, no logger_name (no manual handler)
configure_azure_monitor(connection_string=CONNECTION_STRING)

# 2) Use a normal named logger; don't add handlers or set propagate=False
logger = logging.getLogger("dice-game-logger")
logger.setLevel(logging.INFO)

app = FastAPI()

@app.get("/roll")
async def roll_dice():
    dice1_value = random.randint(1, 6)
    dice2_value = random.randint(1, 6)
    value = dice1_value + dice2_value
    
logger.info(f"Rolled a {value}", extra={"dice.value": value})
    await async_log()
  
if value == 12:
        logger.error("You rolled 2 sixes!!!")
    return {"dice_value": value}

async def async_log():
    await asyncio.sleep(1)
    logger.warning("Slowing down between rolls")

The distro bundles instrumentation that collects logs automatically. Attaching a second handler to the same logger (via logger_name=) creates a second export path which is what produced the duplicates.

Other way is keep your handler, but disable the extra logging instrumentation:

If you prefer explicit control over the logger (and want to keep logger_name=), remove the auto logging instrumentation after you call configure_azure_monitor():

# --- Code 1 (deduped): keep your handler and turn off OTel logging instrumentation ---
import logging
import random
import asyncio
from fastapi import FastAPI
from azure.monitor.opentelemetry import configure_azure_monitor

# Disable OTel logging instrumentation after configure()
from opentelemetry.instrumentation.logging import LoggingInstrumentor

CONNECTION_STRING = "XXX"

configure_azure_monitor(
    logger_name="dice-game-logger",
    connection_string=CONNECTION_STRING
)

# Turn off the global LoggingInstrumentor to avoid double export
LoggingInstrumentor().uninstrument()
logger = logging.getLogger("dice-game-logger")
logger.setLevel(logging.INFO)

# Keep propagate=True so records don't bubble to other handlers unintentionally
logger.propagate = True
app = FastAPI()

print("Handlers on dice-game-logger:", logger.handlers) 
print("Handlers on root logger:", logging.getLogger().handlers)

@app.get("/roll")
async def roll_dice():
    """Roll a dice, emit trace/logs/metrics"""
    dice1_value = random.randint(1, 6)
    dice2_value = random.randint(1, 6)
 
    value = dice1_value + dice2_value
 
    # Log
    logger.info(f"Rolled a {value}", extra={"dice.value": value})
 
    # Demonstrate async work under the same trace
    await async_log()
 
    # Put out error if someone rolls 2 sixes
    if (value == 12):
        logger.error("You rolled 2 sixes!!!")
 
    # Return response
    return {"dice_value": value}
 
async def async_log():
    await asyncio.sleep(1)
    logger.warning("Slowing down between rolls")

The distro’s logging instrumentation can capture the same Python record that your LoggingHandler already exports. Disabling that instrumentation leaves only your handler active. See the “Avoid duplicate telemetry” guidance.

If you still observe duplication after Option B, double‑check that you’re not running with multiple workers or a reloader that re‑initializes exporters (each worker would send separately). Uvicorn’s logging can also add handlers that lead to double handling at the root if you configure additional handlers yourself

Reference: https://learn.microsoft.com/en-us/troubleshoot/azure/azure-monitor/app-insights/telemetry/opentelemetry-troubleshooting-python

To check what’s installed in this virtual environment,

# Confirm interpreter
python -V
# List telemetry-related packages (installed SDKs)
pip freeze | grep -Ei \

Hope this helps! Please let us know if this helps. Thanks

I tried option one (of removing logger_name from configure_azure_monitor())

And duplicates still showing

and comparing the logs, they are identical:

For option B (disabling opentelemetry instrumentation).

I had to install the module first as it wasn't present (pip install opentelemetry-instrumentation-logging

and same result: