Skip to main content

MT customization analysis

  • Updated

With MT Studio you can evaluate the results of MT customization in a smart and targeted fashion. To grasp the difference between two translations, you don’t need to manually review your whole test set—just a small subset of segments. Our tool focuses your attention on those segments.

 

Evaluation overview

1vs2.pngEvaluation chart

MT Studio builds a chart where every point represents a text segment. The point’s position indicates how close this segment’s two translations are to the reference translation.

If a point is above the diagonal, translation 2 is closer to reference than translation 1.

If a point is below the diagonal, translation 2 is farther from reference than translation 1.

If a point is right on the diagonal, this segment’s translations are equally close to reference.

Green points represent the improved segments for review. For these segments, translation 2 is significantly closer to reference than translation 1.

Red points represent the degraded segments for review. For these segments, translation 2 is significantly farther from reference than translation 1.

Review the improved and degraded segments to identify both translations' strengths and weaknesses.

 

Statistics

Screen_Shot_2021-01-26_at_20.35.39.png

Segments in test set

Shows the number of segments in your test data.

 

Spotlight

Shows the number and the percentage of segments to review.

 

Corpus scores

Shows the corpus scores for the translations.

 

Review segments

To analyze the translations in more detail and leave comments, select Review segments.

 

Download the report 

To download the evaluation results without further analysis, select Download report. The report includes:

  • The evaluation chart

  • A chart with corpus scores

  • A spreadsheet with corpus scores

  • A spreadsheet with translations and their scores

Segments in the spreadsheet are sorted by score change. Degraded segments are at the top and improved segments are at the bottom.

 

Detailed analysis

Screen_Shot_2021-01-26_at_20.41.07.png

MT Studio shows you segments to review one by one, first degraded segments, then improved segments.

View each segment and evaluate both translations. If you see an issue in a translation, select Add issue type and choose an issue from the list. You can add more than one issue to a translation. You can also write comments to the segment, in the Comment field below both translations. The issues that you add and your comments will be saved in the report.

Select Mark segment, if you want to return to the segment later.

 

Filters

Select Degraded to see only the degraded segments for review.

Select Improved to see only the improved segments for review.

Select Reviewed to see the segments that you have already seen. The percentage of segments that you have seen is in the brackets.

Select With issues to see the segments to which you have added issues.

Select Marked to see the segments that you have marked.

 

Download the report

Whenever you’re ready, select Download report to download:

  • The evaluation chart

  • A chart with corpus scores

  • A spreadsheet with corpus scores

  • A spreadsheet with translations, their scores, the issues that you have added, and your comments and marks

MT Studio saves your project, so you can come back to it anytime.

 

View evaluation summary

When you have reviewed all segments, select View evaluation summary to see the number and types of issues in each translation.

 

 

 

Related articles