Azure Custom Translator - Training documents not processed

Hello kristianandersen,

Thanks for rasing this question in Q&A forum. The "UserFilesUploaded" state in Azure Custom Translator indicates that your documents have been uploaded but are queued for processing, which can stall due to format validation failures, size limits, or service throttling intermittent processing suggests the service is retrying after transient errors, but some documents fail initial validation. Here's how to troubleshoot and ensure consistent processing:

Quick Checks and Fixes

1. Document Format and Encoding:
   • File Types: Ensure you're using supported formats: .txt, .tmx, .xliff, .xlsx, .docx, .pdf, .html, .csv, .tsv (UTF-8 or UTF-16 encoding).
   • Encoding: Open files in Notepad++ > Encoding > Confirm UTF-8 (BOM-free). BOM causes parsing failures—save as "UTF-8 without BOM".
   • Structure: For .txt, each line must be a parallel sentence pair separated by tab: source<TAB>target. No empty lines or headers.
   • Size: Each document must be < 1 GB and < 1 million segments (lines). Split larger files.
2. Document Set Validation:
   • Portal: Custom Translator > Your project > Document sets > Select stuck set > Validate (runs pre-check). Look for errors (e.g., "Invalid segment count", "Encoding mismatch").
   • Fix errors, re-upload, and Process again.
3. Processing Stuck? Force Retry:
   • Portal: Document set > Cancel processing > Re-upload (or select existing) > Process.
   • API: Use Cancel operation then Process again:

     
          # Cancel
     
          Invoke-RestMethod -Uri "https://customtranslator.cognitive.microsofttranslator.com/translator/customtranslator/v1.0/document-sets/{docSetId}/cancel" -Method Post -Headers @{ "Ocp-Apim-Subscription-Key" = "your-key" }
     
          
     
          # Process
     
          Invoke-RestMethod -Uri "https://customtranslator.cognitive.microsofttranslator.com/translator/customtranslator/v1.0/document-sets/{docSetId}/process" -Method Post -Headers @{ "Ocp-Apim-Subscription-Key" = "your-key" }
     
     

4. Service Health and Throttling:
   • Check Azure status: status.azure.com > AI Translator—if degraded, wait for fix.
   • Throttling: Custom Translator has 10 concurrent processing jobs per subscription. If you have many sets, queue them sequentially (wait for one to complete before starting next).
   • API Version: Use latest (v3.0) in endpoints: https://api.cognitive.microsofttranslator.com/customtranslator/v3.0/document-sets.
5. Best Practices for Reliable Processing:
   • Batch Size: Upload < 5 document sets at once; process one by one.
   • Naming: Avoid special characters in filenames (use a-z, 0-9, underscores).
   • Parallel Segments: Ensure source and target line counts match exactly—mismatches cause validation hang.
   • Language Codes: Use correct BCP-47 codes (e.g., en-us, de-de)—invalid codes stall processing.

If Still Stuck

• Support Ticket: Portal > Custom Translator > Help > New support request > Technical > "Document sets stuck in UserFilesUploaded" > Attach document set ID and sample file (first 10 lines).
• Free Tier Limit: Free tier has lower priority; upgrade to S1 SKU ($10 per million characters) for faster, more reliable processing.

Process validation first this resolves 70% of stalls. If helpful, please accept the answer.

Best Regards,

Jerald Felix