(Apologies for multiple postings)

*CALL FOR PAPERS*
*****************************************************************************************
*IPMU 2022 -*
Information Processing and Management of Uncertainty in Knowledge-Based
Systems /
July 11-15, 2022 – Milan, Italy
https://ipmu2022.disco.unimib.it/
*Special Session on "Data Perspectivism in Ground Truthing and Artificial
Intelligence"*
S3 - https://ipmu2022.disco.unimib.it/special-sessions/
**************************************************
*Description, scope and aims*

Many Artificial Intelligence applications are based on supervised machine
learning (ML), which ultimately grounds on manually annotated data. The
annotation process (often called ground-truthing) is often performed in
terms of a majority vote and this has been proved to be often problematic,
as highlighted by recent studies on the evaluation of ML models. Recently,
a different paradigm for ground-truthing has started to emerge, called data
perspectivism [1], which moves away from traditional majority aggregated
datasets, towards the adoption of methods that integrate different opinions
and perspectives within the knowledge representation, training, and
evaluation steps of ML processes, by adopting a non-aggregation policy.
This alternative paradigm obviously implies a radical change in how we
develop and evaluate ML systems: such ML systems have to take into account
multiple, uncertain, and potentially mutually conflicting views [2]. This
obviously brings both opportunities and difficulties: novel models or
training techniques may need to be designed, and the validation phase may
become more complex. Nonetheless, initial works have shown that data
perspectivism can lead to better performances [3,4], and could also have
important implications in terms of human-in-the-loop and interpretable AI,
as well as in regard to the ethical issues or concerns related to the use
of AI systems [5]. Data perspectivism is a framework to treat uncertainty
(the main theme of IPMU) at the level of knowledge modeling and its
integration in the development and evaluation of systems.

The scope of this special session is to attract contributions related to
the management of subjective, uncertain, multi-perspective, or otherwise
non-aggregated data in ground-truthing, machine learning, and more
generally artificial intelligence systems.

*Invited contributions: full research papers and research in progress
papers.*

*Topics of interest:*


   - Subjective, uncertain, or conflicting information in annotation and
   crowdsourcing processes;
   - Limits and problems with standard data annotation and aggregation
   processes;
   - Theoretical studies on the problem of learning from multi-rater and
   non-aggregated data;
   - Participation mechanisms/incentives/gamification for rater engagement
   and crowdsourcing;
   - Ethical and legal concerns related to annotation and aggregation
   processes in ground-truthing;
   - Creation and documentation of multi-rater and non-aggregated datasets
   and benchmarks;
   - Development of ML algorithms for multi-rater and non-aggregated data;
   - Development of techniques to detect and manage multiple forms of
   uncertainty in multi-rater and non-aggregated data;
   - Techniques for the evaluation of ML systems based on multi-rater and
   non-aggregated data;
   - Applications of data perspectivism and non-aggregated data to
   interpretable, human-in-the-loop AI and algorithmic fairness;
   - Experimental and application studies of ML/AI systems on multi-rater
   and non-aggregated data, in possibly different application domains (e.g.
   NLP, medicine,legal studies, etc.)

***************************************************
*Important dates*

Paper Submission deadline: Friday, 14 January 2022 Friday, *February 18,
2022 *(STRICT)
Notification of acceptance: April 1st, 2022
Camera ready due: April, 22nd, 2022
IPMU Conference: July 11th -15th, 2022

*************************************************

*Author Guidelines:*

Please refer to the IPMU 2022 page where guidelines and templates are
available, in the main conference Web site (
https://ipmu2022.disco.unimib.it/submission/).
All submissions accepted for presentation at IPMU 2022 will be published in
the Communications in Computer and Information Science (CCIS) series, by
Springer.

***************************************************

*IPMU 2022 S3 Special Session Co-Chairs:*

Andrea Campagner (University of Milano-Bicocca, Italy),
Teresa Scantamburlo (Ca’ Foscari University of Venice, Italy),
Valerio Basile (University of Turin, Italy),
Federico Cabitza (University of Milano-Bicocca, Italy)

***************************************************


*Related readings*
[1] Basile, V., Cabitza, F., Campagner, A., Fell, M. (2021)
Toward a Perspectivist Turn in Ground Truthing for Predictive Computing
arXiv preprint, arXiv:2109.04270
https://arxiv.org/pdf/2109.04270.pdf
[2] Zhang, J., Wu, X., Sheng, V.S. (2016)
Learning from crowdsourced labeled data: A survey.
Artificial Intelligence Review
[3] Fornaciari, T., Uma, A., Paun, S., Plank, B., Hovy, D., Poesio, M.
(2021)
Beyond Black & White: Leveraging Annotator Disagreement via Soft-Label
Multi-Task
Learning
Conference of the North American Chapter of the Association for
Computational Linguistics:
Human Language Technologies (NAACL 2021)
[4] Campagner, A., Ciucci, D., Svensson, C.M., Figge, M. T., Cabitza, F.
(2021)
Ground truthing from multi-rater labeling with three-way decision and
possibility theory
Information Sciences
[5] Basile, V., Fell, M., Fornaciari, T., Hovy, D., Paun, S., Plank, B.,
Poesio, M., Uma, A.
(2021)
We Need to Consider Disagreement in Evaluation
1st Workshop on Benchmarking: Past, Present and Future at ACL 2021

-- 
Teresa Scantamburlo, PhD
Department of Environmental Science, Informatics and Statistics (DAIS)
url : www.dais.unive.it/~scantamburlo
e-mail: teresa.scantambu...@unive.it
_______________________________________________
nexa mailing list
nexa@server-nexa.polito.it
https://server-nexa.polito.it/cgi-bin/mailman/listinfo/nexa

Reply via email to