Call to NotificationHubClient.SendTemplatenotificationAsync() returns ServiceUnavailable response

Question

Call to NotificationHubClient.SendTemplatenotificationAsync() returns ServiceUnavailable response

OpenDental Developer 0

Our calls to NotificationHubclient.SendTemplatenotificationAsync() had been working up until yesterday. With no changes to the service making calls to NotificationHubClient.SendTemplatenotificationAsync(), We started receiving the below errors.

Our registrations are no where near the device limit, nor is our number of requests near our threshold. As of now there are no outages reported for our region, West US.

Unknown error has occurred.. TrackingId:5aea6da7-0259-45e4-af24-8466a7e4f482,TimeStamp:9/9/2025 4:50:50 PM +00:00 Microsoft.Azure.NotificationHubs.Messaging.MessagingException The remote server returned an error: (503) ServiceUnavailable. Reason: Service Unavailable..TrackingId:e4d569f9-1458-47fd-9ede-d6648964f5aa,TimeStamp:2025-09-09T15:51:10.3491014Z Microsoft.Azure.NotificationHubs.Messaging.ServerBusyException

RAMAMURTHY MAKARAPU 1,130 Reputation points Microsoft External Staff Moderator

2025-09-12T03:10:26.36+00:00

Hi @OpenDental Developer ,

Could you please review my comments, follow the steps, and let me know whether your issue has been resolved?

1 answer

Your answer

RAMAMURTHY MAKARAPU 1,130 Reputation points Microsoft External Staff Moderator

2025-09-12T03:10:26.36+00:00

Hi @OpenDental Developer ,

Could you please review my comments, follow the steps, and let me know whether your issue has been resolved?

Answer 1

RAMAMURTHY MAKARAPU 1,130 Microsoft External Staff Moderator

Hello @OpenDental Developer ,

Thank you for submitting your question on Microsoft Q&A.

This typically points to a transient backend issue on Azure Notification Hubs rather than a misconfiguration on your side. Even though your region (West US) shows no outage in the Service Health dashboard, Notification Hubs sometimes throttle or return 503 Service Unavailable when:

Backend load spikes (temporary service-side issue).
High request concurrency from your app (even if you’re under limits, sudden bursts can trigger throttling).
Template notification bugs (FCMv1 in particular has had instability with template sends).

Recommended steps to resolve the issue

Implement Retry Logic

Wrap SendTemplateNotificationAsync() with exponential backoff retries.
Microsoft recommends retrying on ServerBusyException or MessagingException. Example backoff: 2s → 4s → 8s → up to ~1 minute.

Check Azure Service Health (per-resource)

In the Azure Portal → Notification Hub → Service Health → Resource Health
This often shows localized transient issues even if the region-wide dashboard is green.

Validate Registration Payloads

Ensure no malformed tags or template parameters.
Known bug: very complex tag expressions or exceeding the limit (20 OR-only, 6 mixed) can cause 5xx errors.

Scale Out Usage

If traffic is spiky, consider distributing sends (batching messages, using multiple hubs if needed).

In most cases, these 503 errors are short-lived service-side throttles and resolve after retry.

OpenDental Developer 0 Reputation points

2025-09-23T22:49:13.02+00:00
Implement Retry Logic

Traffic has had no notable spikes. I also am not sure that this is a reasonable solution as NO notifications are making their way through, manually retrying or otherwise.

Check Azure Service Health (per-resource)

Reports no issues over the last four weeks.

Validate Registration Payloads

Registration payloads are not malformed, and remain unchanged since before breaking.

Our tag expressions do not exceed the 6 mixed rule (we only have 5 conditions)

Scale Out Usage

Traffic doesn't seem spikey by my definition.
RAMAMURTHY MAKARAPU 1,130 Reputation points Microsoft External Staff Moderator

2025-09-30T03:33:43.2566667+00:00
Hello @OpenDental Developer ,

Thank you for your update and for confirming that traffic spikes, payload issues, and tag complexity are not the cause of the 503 ServiceUnavailable errors with NotificationHubClient.SendTemplateNotificationAsync(). Since Azure Service Health reports no problems, the issue could be related to FCMv1 template sends, backend throttling, or SDK handling.

Here’s a step-by-step plan to troubleshoot and resolve the issue:

Enable Detailed Diagnostic Logging

Action: Turn on diagnostic logging for your Notification Hub in the Azure Portal. Go to Monitoring → Diagnostic settings → Add diagnostic setting → Select AllLogs → Send to a storage account.

Note the Tracking IDs from your error reports.

Reason: Logs can help identify hidden problems like authentication failures or transient throttling not shown in Service Health.

Verify FCMv1 Configuration

Action: In Google Cloud Console, check that the Firebase Cloud Messaging API is enabled. In Azure Portal, re-validate your FCMv1 service account JSON and Project ID. Try sending a non-template FCM message to confirm connectivity.

Reason: Incorrect FCMv1 credentials can cause 503 errors even with valid payloads.

Update the SDK

Action: Upgrade to Microsoft.Azure.NotificationHubs v4.2.0 or newer using Update-Package in your project.

Reason: Older SDK versions may not handle FCMv1 template sends properly, resulting in 503 errors.

Test Template Sends with Diagnostic Feedback

Action: Use test sends in your code or the portal to validate templates and capture NotificationOutcome details for failures.

Reason: This can quickly reveal template-related errors and backend issues.

Reference:

Monitor Azure Notification Hubs | Microsoft Learn

Azure Notification Hubs and the Google Firebase Cloud Messaging (FCM) Migration Using SDKs | Micros…

Microsoft.Azure.NotificationHubs Namespace - Azure for .NET Developers | Microsoft Learn

Diagnose dropped notifications in Azure Notification Hubs | Microsoft Learn

Kindly let us know if the above comment helps or you need further assistance on this issue.

Please "upvote" if the information helped you. This will help us and others in the community as well.
RAMAMURTHY MAKARAPU 1,130 Reputation points Microsoft External Staff Moderator

2025-10-02T02:28:46.25+00:00

Hello @OpenDental Developer ,

Let me know if you’re still facing this issue or if it’s been resolved. From our detailed investigation, we found that there was an incident (ICM) running around the time you encountered the problem so It might the reason for your issue.

On September 9, 2025, Azure Notification Hubs in the West US region had a major service disruption. During this time, notification sends dropped sharply, and many users were getting frequent 503 Service Unavailable errors when trying to deliver messages. Essentially, the service wasn’t able to process requests, which caused delays and failures in sending notifications.

The incident lasted for about 6.5 hours and impacted several customer workloads, leading to drops, latency, and degraded user experiences. The root cause was traced back to some backend compute resources in the West US region that had gone unhealthy. It's been resolved on 12 September could you please check still you're facing this issue?

Kindly let us know if the above comment helps or you need further assistance on this issue.

Please "upvote" if the information helped you. This will help us and others in the community as well.

Share via

Call to NotificationHubClient.SendTemplatenotificationAsync() returns ServiceUnavailable response

1 answer

Your answer