Thank you.
On Wednesday, July 24, 2024 at 09:13:01 a.m. EDT, Fred Blain via Mt-list
<[email protected]> wrote:
Dear all,
the QE shared task 2024 is ON!
You can now submit and test your quality estimation system(s) on a set of
different languages and tasks: to predict translation quality at sentence
level, to detect error spans, or even to correct translations!
For information on how to access the test data and the submission platforms,
visit the shared task's webpage:
https://www2.statmt.org/wmt24/qe-task.html
Deadline to participate is July 31 (AoE).
Looking forward to receiving your predictions!
--
Best wishes,
on behalf of the organisers.
From:Mt-list <[email protected]> on behalf of Fred Blain via
Mt-list <[email protected]>
Date: Friday, 21 June 2024 at 3:59 PM
To: [email protected] <[email protected]>
Subject: Shared Task on Quality Estimation at WMT'24
Dear all,
we are happy to invite you to participate in the Shared Task on Quality
Estimation at WMT'24.
The details of the task can be found at:
https://www2.statmt.org/wmt24/qe-task.html
New this year:
- We introduce a new language pair (zero-shot): English-Spanish
- Continuing from the previous edition, we will also analyse the robustness
of submitted QE systems to a set of different phenomena which will span from
hallucinations and biases to localized errors, which can significantly impact
real-world applications.
- We also introduce a new task, seeking not only to detect but also to
correct errors: Quality-aware Automatic Post-Editing! We invite participants to
submit systems capable of automatically generating QE predictions for
machine-translated text and the corresponding output corrections.
2024 QE Tasks:
Task 1 -- Sentence-level quality estimation
This task follows the same format as last year but with fresh test sets and a
new language pair: English-Spanish. We will test the following language pairs:
- English to German (MQM)
- English to Spanish (MQM)
- English to Hindi (MQM & DA)
- English to Gujarati (DA)
- English to Telugu (DA)
- English to Tamil (DA)
More details: https://www2.statmt.org/wmt24/qe-subtask1.html
Task 2 -- Fine-grained error span detection
Sequence labelling task: predict the error spans in each translation and the
associated error severity: Major or Minor.
We will test the following language pairs:
- English to German (MQM)
- English to Spanish (MQM)
- English to Hindi (MQM)
More details: https://www2.statmt.org/wmt24/qe-subtask2.html
Task 3 -- Quality-aware Automatic Post-editing
We expect submissions of post edits correcting detected error spans of the
original translation. Although the task is focused on quality-informed APE, we
also allow participants to submit APE output without QE predictions to
understand the impact of their QE system. Submissions w/o QE predictions will
also be considered official.
We will test the following language pairs:
- English to Hindi
- English to Tamil
More details: https://www2.statmt.org/wmt24/qe-subtask3.html
Important dates:
- Test sets will be released on July 15th.
- Participants can submit their systems by July 23rd on codalab.
- System paper submissions are due by 20th August [aligned with WMT
deadlines].
Note: Like last year, we aligned with the General MT and Metrics shared tasks
to facilitate cross-submission on the common language pairs: English-German,
English-Spanish, and English-Hindi (MQM).
We look forward to your submissions and feel free to contact us if you have any
more questions!
Best wishes,
on behalf of the organisers.
