Hi Sumit Gaur,
This situation involves Azure-based file polling and processing possibly through Logic Apps or API Management where file corruption is happening despite having concurrency turned off and limiting the maximum number of files per trigger run
Use V2 Connectors
If you're using Azure Logic Apps, switching to V2 connectors (if available) can help improve performance and reliability. V2 connectors are optimized for better scalability and error handling. Ensure you’re using the latest version of the connector for file polling.
Verify File Integrity
Corrupted files can occur due to incomplete writes or network issues during transfer. To mitigate this:
- Check File Locks: Ensure the files being polled are not being written to or modified during the trigger run.
Use Checksums: Implement checksum validation (e.g., MD5 or SHA256) to verify file integrity before processing.
Retry Logic - Add retry policies in your workflow to handle transient issues.
Enable Logging and Diagnostics
Enable detailed logging in your workflow to identify the root cause of corruption
Azure Monitor - Use Azure Monitor or Application Insights to track file processing and errors.
Custom Logs - Log file names, sizes, and timestamps to detect patterns in corrupted files.
I hope the provided answer is helpful,
Please "Up Vote" if the information helped you. This will help us and others in the community as well.
Thank you.Hi Sumit Gaur,
This situation involves Azure-based file polling and processing possibly through Logic Apps or API Management where file corruption is happening despite having concurrency turned off and limiting the maximum number of files per trigger run
Use V2 Connectors
If you're using Azure Logic Apps, switching to V2 connectors (if available) can help improve performance and reliability. V2 connectors are optimized for better scalability and error handling. Ensure you’re using the latest version of the connector for file polling.
Verify File Integrity
Corrupted files can occur due to incomplete writes or network issues during transfer. To mitigate this:
Check File Locks: Ensure the files being polled are not being written to or modified during the trigger run.
Use Checksums: Implement checksum validation (e.g., MD5 or SHA256) to verify file integrity before processing.
Retry Logic - Add retry policies in your workflow to handle transient issues.
Enable Logging and Diagnostics
Enable detailed logging in your workflow to identify the root cause of corruption
Azure Monitor - Use Azure Monitor or Application Insights to track file processing and errors.
Custom Logs - Log file names, sizes, and timestamps to detect patterns in corrupted files.
I hope the provided answer is helpful,
Please "Up Vote" if the information helped you. This will help us and others in the community as well.
Thank you.