****************************************************** ********* EVALITA 2026: Call for tasks ********* ******* NEW DEADLINES and TIMELINE ****** ******************************************************
EVALITA 2026 is an initiative of AILC (Associazione Italiana di Linguistica Computazionale, AILC https://www.ai-lc.it/). As in the previous editions (https://www.evalita.it/), EVALITA 2026 will be organized along a few selected tasks, which provide participants with opportunities to discuss and explore both emerging and traditional areas of Natural Language Processing and Speech. The participation is encouraged for teams working both in academic institutions and industrial organizations. TASK PROPOSAL SUBMISSION Task proposals should be no longer than 4 pages and should include: - task title and acronym; - names and affiliation of the organizers (minimum 2 organizers); - brief task description, including motivations and state of the art; - explanation of the international relevance of the task; - description and examples of the data, including information about their availability, development stage, and issues concerning privacy and data sensitivity. The examples are mandatory because they are intended to give potential participants an idea of what the task data will look like, how it’ll be formatted, etc. - expected number of participants and attendees; - names and contact information of the organizers. We also accept the re-annotation/expansion of datasets from previous years and previous challenges with new annotation levels, and texts from publicly available corpora. However, test annotations must be new and unpublished, as participants must not have access to the test data annotations until the end of EVALITA campaign. For new tasks, organizers must specify in the proposal why it would attract a reasonable number of participants, and why it is needed. For re-runs, organizers must describe the element of novelty from previous challenges. In submitting your proposal, please bear in mind that we strongly encourage: - tasks that pose non-trivial challenges and stimulate the creation of innovative systems (i.e., that integrate linguistic insights or external knowledge sources), rather than being easily addressed by off-the-shelf LLM prompting techniques; - tasks focused on multimodality, e.g., considering both textual and visual or any other modality; - tasks characterized by different levels of complexity, e.g., with a straightforward main subtask and one or more sophisticated additional subtasks; - to consider providing competitive baselines (e.g., small-scale LLMs in zero-shot setups), which participants are expected to improve upon, in order to encourage the design of advanced solutions; - application-oriented tasks, that is, tasks that have a clearly defined end-user application showcasing; - multilingual tasks, i.e. with data both in Italian and in other languages; - industrial tasks, i.e. tasks with real data provided by companies. The organizers of the accepted tasks should take care of planning, according to the scheduled deadlines (see below): - the development and distribution of datasets needed for the contest, i.e. data for training and development, and data for testing; the scorer to be used to evaluate the submitted systems should be included in the release of development data; - the development of task guidelines, where all the instructions for the participation are made clear, together with a detailed description of data and evaluation metrics applied for the evaluation of the participant's results; - the collection of participants' results; - the evaluation of participants' results according to standard metrics and baseline(s); - the solicitation of participation and submissions; - the reviewing process of the papers describing the participants' approach and results (according to the template to be made available by the EVALITA 2026 chairs); - the production of a paper describing the task (according to the template to be made available by the EVALITA 2026 chairs). *** Email your proposal in PDF format to [email protected] with "EVALITA 2026 TASK Proposal" as the subject line by the submission deadline: July 28th 2025. *** Please feel free to contact the EVALITA 2026 chairs at [email protected] in case of any questions or suggestions. Deadlines of the task proposal: - July 21th 2025 July 28th 2025: submission of task proposals - July 31th 2025 August 7th 2025: notification of task proposal acceptance Timelines of EVALITA 2026: - 22nd September 2025: development data available to participants - 3 - 17th November 2025: evaluation windows - 28th November 2025: assessments returned to participants - 15th December 2025: final reports (from participants) due to task organizers - 22nd December 2025: final reports (from task organizers) due to EVALITA chairs - 19th January 2025: review deadline - 2nd February 2026: camera-ready version deadline - 26 - 27th February 2026: final workshop in Bari EVALITA 2026 CHAIRS Francesco Cutugno (Università di Napoli) Alessio Miaschi (Istituto di Lingustica Computazionale “A. Zampolli” - CNR) Alessio Palmero Aprosio (Università di Trento) Giulia Rambelli (Università di Bologna) Lucia Siciliani (Università di Bari) Marco Antonio Stranisci (Università di Torino) FURTHER INFORMATION Website: https://www.evalita.it/campaigns/evalita-2026/call-for-tasks/ Mail: [email protected] Marco, UNITO <https://www.unito.it/persone/mstranis> and aequa-tech <https://aequa-tech.com/>
_______________________________________________ Corpora mailing list -- [email protected] https://list.elra.info/mailman3/postorius/lists/corpora.list.elra.info/ To unsubscribe send an email to [email protected]
