Azure AI Foundry Agent Service: Is There Any Way to Force Sequential Execution of Multiple Tools (Azure AI Search -> Bing Grounding -> Response)?

Hi Mahesh Babaji Sondkar

Thank you for sharing your observation. Shall review with product group internally. Please share the details requested in private message. 

Hello Mahesh Babaji Sondkar,

Welcome to Microsoft Q&A and thank you for reaching out.

I understand that you’re trying to ensure that both Azure AI Search and Bing Grounding tools execute sequentially for every query in your Azure AI Agent setup. Let’s go through the current capabilities, limitations, and possible workarounds.

At this time, the Azure AI Agent Service does not support enforcing sequential execution of multiple tools (for example, Azure AI Search → Bing Grounding → Response) within a single run. The agent’s orchestration is LLM-driven, meaning the model autonomously decides if, when, and which tools to call based on the context, system prompt, and tool definitions.

• tool_choice="required" ensures at least one tool is called but does not guarantee which one or enforce an order.
• tool_choice={"type": "tool_name"} can be used to force a single specific tool to execute for that run.
• Currently, there’s no option such as tool_choice="all" or tool_choice=["tool1","tool2"] to enforce multiple sequential tool calls. This functionality is being considered for future updates.

Guidance on Tool Invocation

1. Tool Choice Parameter

The tool_choice parameter remains the most deterministic control available for tool invocation. However, it cannot enforce strict multi-tool or sequential execution within a single turn. When tool_choice="required" is used, the model guarantees one tool will be called, but the agent still retains control over the sequence and may skip others based on contextual reasoning.

If you want both tools to run deterministically, you can invoke them in separate runs, like this:

# Example: Forcing Azure AI Search first
run = project_client.agents.runs.create_and_process(
    thread_id=thread.id,
    agent_id=agent.id,
    tool_choice={"type": "azure_ai_search"}  # Call Azure AI Search first
)

# After completion, trigger Bing Grounding
run = project_client.agents.runs.create_and_process(
    thread_id=thread.id,
    agent_id=agent.id,
    tool_choice={"type": "bing_grounding"}  # Call Bing Grounding next
)

This approach allows you to control the sequence externally, ensuring both tools run before generating a final model response.

2. Instructions Parameter

Make sure your agent’s Instructions (system prompt) explicitly describe your expected order of tool execution. Although this won’t enforce behavior programmatically, it can help guide the LLM’s reasoning process. For example:

“First, use Azure AI Search to retrieve internal knowledge base data. Then, use Bing Grounding to collect relevant external web information based on the Azure AI Search results before generating a final response.”

You can experiment with prompt phrasing to encourage consistent multi-tool invocation.

Recommended Workaround – External Orchestration

If your use case requires deterministic, sequential execution (i.e., both tools must always run in order), the most reliable solution is to orchestrate the sequence externally in your application logic:

Invoke Azure AI Search and collect internal results.

Invoke Bing Grounding (or external web search) using the search results as context.

Pass both sets of results to the model for synthesis and final response generation.

This approach guarantees consistent tool execution order and provides full control over auditing and debugging.

• Also Ensure you’re using the latest version of the azure-ai-projects SDK for up-to-date orchestration capabilities.
• Review execution logs to check whether tool invocations are skipped or encountering runtime issues.
• Continue monitoring the Azure AI Foundry documentation and Azure Updates for future enhancements that may support multi-tool orchestration control.

Answers to Your Specific Questions

Force all attached tools to execute:

Not directly possible in a single run. You can, however, call them sequentially in separate runs.

Guarantee multiple tools (e.g., Azure AI Search and Bing Grounding) are called:

Not deterministically the model’s internal logic governs selection.

Does tool selection override system prompt instructions?

Yes, tool_choice takes precedence over prompt-level instructions.

Is a tool_choice="all" or similar feature planned?

Not currently available, but it’s under consideration for future releases.

Force the agent to call a single tool consistently:

Yes, using tool_choice={"type": "tool_name"} ensures that tool is invoked for that run, though behavior may vary in subsequent turns based on query context.

Please refer this
Troubleshoot problems with agent tools integration

Use system instruction to help model invoke the right tool

Grounding with Bing Search

Azure AI Search tool

I hope this helps, do let me know if you have any further queries.

If this answers your query, could you please take a moment to retake the survey by accepting this response? Your feedback is greatly appreciated.

Thank you!

Thanks for your detailed response. It appears that there’s no way to force the agent to execute both tools (Azure AI Search → Bing Grounding) deterministically in a single run. I’ll call them sequentially in separate runs and will update you on how it goes.

I also wanted to share some experiments I’ve done with the tool_choice parameter and the observations I’ve made:

• tool_choice="auto" – Default behavior where the model decides which tools to use. No tool calls were made for the queries I tested.
• tool_choice="required" – Only the Bing Grounding tool was called for the queries tested.
• tool_choice={"type": "bing_grounding"} – Only the Bing Grounding tool was called for the queries tested.
• tool_choice={"type": "azure_ai_search"} – Surprisingly, Both Azure AI Search and Bing Grounding tools were called for the queries tested. Is this expected behavior, or could it be a bug? Also, do you know if this behavior can be consistently reproduced across different queries or environments?

Hi Mahesh Babaji Sondkar,

Thanks for sharing your additional observations this is really helpful, and it also aligns with how tool orchestration currently behaves in the Azure AI Agent Service.

Let me clarify what you’re seeing and why.

Why tool_choice={"type": "azure_ai_search"} Sometimes Triggers Both Tools

Yes it is possible (and expected in some cases) that specifying a single tool in tool_choice={"type": "azure_ai_search"} results in multiple tools being called, even though only one tool was explicitly requested.

This happens because:

The tool_choice parameter tells the model which tool it must call at least once.

But it does not prohibit the model from calling additional tools afterward if it believes they are helpful for answering the query.

This behavior is intentional, because Azure AI Agents still allow the model to reason about tool usage beyond the enforced tool.

tool_choice={"type": "azure_ai_search"} guarantees Azure AI Search will be invoked, But it does not restrict the model from calling Bing Grounding afterward

This is the same core reason the reverse does not happen when bing_grounding is enforced, the model is not necessarily motivated to call Azure AI Search unless your query strongly signals it.

Is this behavior reproducible?

You may see variations across:

The query wording

Temperature / model randomness

The specific dataset in Azure AI Search

How your tools are described in the schema

Any additional reasoning the LLM performs during tool orchestration

Across multiple customers, we have observed that:

Forcing Azure AI Search first often results in both tools being used

Forcing Bing Grounding typically results in Bing-only execution

tool_choice="auto" frequently leads to no tools being used when the LLM thinks it can answer from prior knowledge

So yes, your results are consistent with what we see internally. However, because the underlying orchestration is LLM-driven, exact reproducibility across environments cannot be guaranteed.

Is it a bug?

No based on the current architecture, this is expected behavior, not a bug.

The orchestration engine is designed to:

Enforce one required tool (if specified)

Allow additional tool calls at the model’s discretion

Drive reasoning based on context rather than strict deterministic workflows

This means the model retains autonomy unless you fully externalize the workflow in your application code.

Recommended approach for deterministic execution

Since you want strict ordering (Azure AI Search → Bing Grounding → Response), the safest approach remains:

Call Azure AI Search in a separate run

Pass its results into the next run

Force Bing Grounding in the second run

Generate the final response in a third run (if needed)

This fully removes LLM variability and guarantees predictable execution.

I hope this helps, do let me know if you have any further queries.

If this answers your query, could you please take a moment to retake the survey by accepting this response? Your feedback is greatly appreciated.

Thank you!

Hi Mahesh Babaji Sondkar,

Did you get any chance to review the above response. Do let me know if you have any further queries.

Could you please take a moment to retake the survey on the above response? Your feedback is greatly appreciated.

Thank you!