MQM Explained: How to Measure Translation Quality Objectively

What is MQM?

MQM stands for Multidimensional Quality Metrics. It is an open framework for translation quality evaluation that originated in European research projects in the early 2010s. Today it is one of the most widely used approaches for structured translation quality assessment, both in the localization industry and in research on machine translation.

MQM does not tell you whether a translation is “good” in the abstract. Instead, it gives reviewers a shared vocabulary for what went wrong and how much it matters. That makes quality something you can measure, compare and improve.

Why opinions are not enough

Without a framework, quality reviews tend to drift into personal preference. One reviewer rewrites half a page, another approves it untouched, and nobody can say whether quality is improving over time. MQM solves this by separating real errors from preferences and by weighting errors by their impact on the reader.

The main error categories

MQM organizes errors into top-level dimensions, each with more specific subtypes. The ones you will use most often are:

You do not have to use every category. A good evaluation setup selects the categories that matter for the content and the client.

Severity levels

• Minor: noticeable, but does not affect meaning or usability. The reader may not even notice.
• Major: changes the meaning, confuses the reader or noticeably damages the experience.
• Critical: could cause serious legal, safety, financial or reputational harm, or makes the product unusable.

Many teams also log neutral issues, such as preferential changes, without any penalty. This keeps useful feedback without distorting the score.

Turning errors into a score

Each severity is assigned a penalty weight. A common setup is 1 point for Minor, 5 for Major and 25 for Critical, but weights should be agreed with the client for each program.

One simple way to score a sample:

1. Add up the penalty points for all errors.
2. Normalize by the number of words evaluated, for example as penalty points per 1,000 words.
3. Compare the result with a pass threshold agreed before the evaluation.

Example: a reviewer evaluates 1,000 words and logs 4 Minor errors and 1 Major error. That is 4 × 1 + 1 × 5 = 9 penalty points per 1,000 words. If the agreed threshold is 10, the sample passes. If a single Critical error had been found, the sample would fail no matter how clean the rest was.

How to set up an MQM evaluation

1. Define the scope. Which content, how many words and how the sample is selected.
2. Choose categories and write severity guidelines with examples from your own content.
3. Agree on weights and a pass threshold before anyone starts reviewing.
4. Calibrate reviewers by having them annotate the same small sample and discuss the differences.
5. Annotate in a spreadsheet or an LQA tool that supports MQM.
6. Report the score, the error breakdown by category and representative examples, so the result is actionable.

Using MQM to evaluate AI and machine translation

MQM is one of the most practical tools for deciding how to use AI in a localization program. You can:

• Compare engines or prompts on the same sample and pick the best performer for each content type.
• Decide whether a content type needs light or full post-editing.
• Track whether glossary and prompt improvements actually reduce errors over time.

AI can also pre-annotate likely errors to speed up the review. The final judgement on category and severity should stay with an experienced linguist.

How Comoloc uses MQM

We run LQA and MQM evaluations for human, MT and AI translation, and deliver scorecards with a clear error breakdown and recommendations. If you want to know where your current translations stand, an MQM evaluation of a representative sample is a good place to start.