Dear all,
the “Test suites” sub-task will be included for the sixth time in the
General MT Shared Task of the Conference on Machine Translation (WMT23).
*OVERVIEW*
Test suites are custom extensions to the test sets of the General MT
Shared Task, constructed so that they can focus on particular aspects of
the MT output. They cοnsist of a source-side test-set and a customized
evaluation service. As opposed to the standard evaluation process which
produces generic quality scores, test suites often produce separate
fine-grained results for each phenomenon.
Given the massive improvement of MT and the emergence of LLMs recently,
test suites can be useful at revealing serious flaws of otherwise highly
scoring systems.
*IMPORTANT DATES*
Test suite source texts must reach us:
19th June
Translated test suites shipped back to test suites authors:
27th July
Test suite description and analysis paper:
TBC - September
Potential participants are kindly requested to fill in this form
https://forms.gle/s4JEJt9WSqAzP74o6
*MORE INFORMATION*
The sub-task is co-ordinated by Ondrej Bojar and Eleftherios Avramidis.
Further information can be found in the dedicated page of the WMT website
http://www2.statmt.org/wmt23/testsuite-subtask.html
(apologies for multiple postings)
--
Dr. Eleftherios Avramidis
DFKI GmbH, Alt-Moabit 91c, 10559 Berlin
Tel. +49-30 238 95-1806
Fax. +49-30 238 95-1810
-------------------------------------------------------------
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH
Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany
Amtsgericht Kaiserslautern, HRB 2313
-------------------------------------------------------------
--
You received this message because you are subscribed to the Google
Groups "Workshop on Statistical Machine Translation" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/wmt-tasks/bdda1e00-c5b8-8a86-fe87-c87ebfbe5637%40dfki.de
<https://groups.google.com/d/msgid/wmt-tasks/bdda1e00-c5b8-8a86-fe87-c87ebfbe5637%40dfki.de?utm_medium=email&utm_source=footer>.
