Train your orchestration workflow model

Training is the process where the model learns from your labeled utterances. After training is completed, you can view model performance.

To train a model, start a training job. Only successfully completed jobs create a model. Training jobs expire after seven days, after this time you can no longer be able to retrieve the job details. If your training job completed successfully and a model is created, it isn't affected if the job expires. You can only have one training job running at a time, and you can't start other jobs in the same project.

The training times can be anywhere from a few seconds when dealing with simple projects, up to a couple of hours when you reach the maximum limit of utterances.

Model evaluation is triggered automatically after training is completed successfully. The evaluation process starts by using the trained model to run predictions on the utterances in the testing set, and compares the predicted results with the provided labels (which establishes a baseline of truth). The results are returned so you can review the model's performance.

Prerequisites

• A successfully created project with a configured Azure blob storage account

See the project development lifecycle.

Data splitting

Before you start the training process, labeled utterances in your project are divided into a training set and a testing set. Each one of them serves a different function. The training set is used in training the model. It's the set from which the model learns the labeled utterances. The testing set is a blind set that isn't introduced to the model during training but only during evaluation.

After the model is trained successfully, the model can be used to make predictions from the utterances in the testing set. These predictions are used to calculate evaluation metrics.

We recommend that all your intents are adequately represented in both the training and testing set.

Orchestration workflow supports two methods for data splitting:

• Automatically splitting the testing set from training data: The system splits your tagged data between the training and testing sets, according to the percentages you choose. The recommended percentage split is 80% for training and 20% for testing.

• Use a manual split of training and testing data: This method enables users to define which utterances should belong to which set. This step is only enabled if you added utterances to your testing set during labeling.

Train model

Start training job

Create a POST request using the following URL, headers, and JSON body to submit a training job.

Request URL

Use the following URL when creating your API request. Replace the placeholder values with your own values.

{ENDPOINT}/language/authoring/analyze-conversations/projects/{PROJECT-NAME}/:train?api-version={API-VERSION}

Headers

Use the following header to authenticate your request.

Request body

Use the following object in your request. The model will be named MyModel once training is complete.

{
  "modelLabel": "{MODEL-NAME}",
  "trainingMode": "standard",
  "trainingConfigVersion": "{CONFIG-VERSION}",
  "evaluationOptions": {
    "kind": "percentage",
    "testingSplitPercentage": 20,
    "trainingSplitPercentage": 80
  }
}

Once you send your API request, you receive a 202 response indicating success. In the response headers, extract the operation-location value formatted like this:

{ENDPOINT}/language/authoring/analyze-conversations/projects/{PROJECT-NAME}/train/jobs/{JOB-ID}?api-version={API-VERSION}

You can use this URL to get the training job status.

Get training job status

Training could take sometime depending on the size of your training data and complexity of your schema. You can use the following request to keep polling the status of the training job until it successfully completes.

Use the following GET request to get the status of your model's training progress. Replace the placeholder values with your own values.

Request URL

{ENDPOINT}/language/authoring/analyze-conversations/projects/{PROJECT-NAME}/train/jobs/{JOB-ID}?api-version={API-VERSION}

Headers

Use the following header to authenticate your request.

Response Body

Once you send the request, you get the following response. Keep polling this endpoint until the status parameter changes to "succeeded".

{
  "result": {
    "modelLabel": "{MODEL-LABEL}",
    "trainingConfigVersion": "{TRAINING-CONFIG-VERSION}",
    "estimatedEndDateTime": "2022-04-18T15:47:58.8190649Z",
    "trainingStatus": {
      "percentComplete": 3,
      "startDateTime": "2022-04-18T15:45:06.8190649Z",
      "status": "running"
    },
    "evaluationStatus": {
      "percentComplete": 0,
      "status": "notStarted"
    }
  },
  "jobId": "xxxxxx-xxxxx-xxxxxx-xxxxxx",
  "createdDateTime": "2022-04-18T15:44:44Z",
  "lastUpdatedDateTime": "2022-04-18T15:45:48Z",
  "expirationDateTime": "2022-04-25T15:44:44Z",
  "status": "running"
}

Cancel training job

Create a POST request using the following URL, headers, and JSON body to cancel a training job.

Request URL

Use the following URL when creating your API request. Replace the placeholder values with your own values.

{ENDPOINT}/language/authoring/analyze-conversations/projects/{PROJECT-NAME}/train/jobs/{JOB-ID}/:cancel?api-version={API-VERSION}

Headers

Use the following header to authenticate your request.

Once you send your API request, you receive a 202 response indicating success, which means your training job has been canceled. A successful call results with an Operation-Location header used to check the status of the job.

Next steps

• Model evaluation metrics concepts
• How to deploy a model