At this point in our technological development, the convergence of humans and machines—through machine learning and increasingly intelligent (and autonomous) devices—promises a transformational impact on daily life. Already machines can outperform humans at certain tasks, but complete autonomy remains elusive, and predictions will continue to benefit from human wisdom for the foreseeable future. Beyond accurate predictions, decisions require insight into the preferences or utility functions of the people that they impact. Decision-support tools must go beyond observational data to actively elicit both information and preferences from stakeholders, to reward contributors appropriately, and to combine the inputs in a way that abides fairness constraints and practical limits on computational power.
The DIMACS Special Focus on Mechanisms and Algorithms to Augment Human Decision Making aims to improve decision-support systems by leveraging both human and machine intelligence through study of tools to augment decision making in individuals and organizations. Available tools include:
- mechanisms to elicit complex probabilities and preferences from people, rewarding them appropriately
- algorithms to combine human judgments and data-driven predictions
- algorithms to aggregate potentially conflicting preferences under social-choice objectives.
The special focus builds on recent advances in computational social choice, crowdsourced democracy, and crowdsourced forecasting, including prediction markets and scoring rules. Topics include eliciting probability distributions and statistics of distributions, decentralized elicitation using markets or blockchain, complexity of elicitation, participatory budgeting, fair division, incentivizing exploration, and digital democracy. Within each topic, we seek to characterize what is impossible, intractable, and tractable to compute either exactly or approximately.
Approaches to eliciting information have been studied by various disciplines under a variety of names, notably scoring rules in statistics, prediction markets in economics, polls and surveys in marketing, and peer prediction, crowdsourcing, combinatorial prediction markets, and incentivized exploration in computer and social sciences. The special focus aims to harness these multiple perspectives by bringing together theoretical computer scientists studying algorithmic game theory, machine learning theory, and NP-hard counting problems; AI computer scientists studying human computation, Bayesian inference, and satisfiability; statisticians studying scoring rules and belief aggregation; economists studying prediction markets, financial markets, and wagering mechanisms; marketing scientists studying surveys and polls; blockchain pioneers implementing decentralized prediction markets and other experimental market constructs; social and behavioral scientists studying human behavior modeling; and human-computer interaction researches designing interfaces to facilitate elicitation.
Together researchers will explore how to use human-machine integration to modernize organizations, markets, and governments by improving how they collect and combine information to make decisions.
Planned Workshops: Special focus workshops will explore questions such as: which statistics of distributions can we compute by minimizing a loss (or maximizing a score) over elicited data with a limited number of responses; how can we characterize or design loss functions to expose desired statistics; and can we develop a more rigorous theory for how to combine potentially conflicting preferences reasonably and fairly. Workshops will investigate how organizations can reduce barriers by rewarding people in ways that enhance inputs to machine intelligence, improving the predictions and decisions that organizations make. Predictions are only one side of the decision-making coin. Good decisions also require accurate representations of individual preferences—which must be elicited—and algorithms to turn elicited votes into organization-level decisions that optimize an objective subject to fundamental axioms and efficient computation. Special focus events will investigate predictions and preferences, as well as the decision-making systems that bring it all together. Planned workshops include:
- Workshop on Forecasting: From Forecasts to Decisions
- Workshop on Algorithmic Social Choice
- Workshop on Eliciting Beyond Labels from the Crowd
- Workshop on Preference Aggregation
- Workshop on Learning from Partially Reliable Data (or Learning from Real Data)
Organizing Committee:
- David Pennock, DIMACS
- Fred Roberts, DIMACS
- DIMACS Focal Point Person: Tamra Carpenter, DIMACS
Send an email to the organizers: dimacs_mechanisms_committee (at) email.rutgers.edu
Opportunities to Participate:
-
Calendar of Events: A variety of workshops and other events are part of the Special Focus.
-
Graduate Student Support: Students interested in attending Special Focus workshops are encouraged to apply to the workshop organizers or to the Special Focus organizers.
-
Materials and Publications: We anticipate that activities of the Special Focus will be documented through slides and video of workshop presentations as well as research publications.
Join the Mailing List:
If you would like to receive updates and announcements about future activities you can subscribe to the mailing list for the Special Focus on Mechanisms and Algorithms to Augment Human Decision Making. Alternatively, you can contact the DIMACS Publicity Coordinator and ask to be placed on the SF_Mechanisms mailing list.
Sponsorship:
The DIMACS Special Focus on Mechanisms and Algorithms to Augment Human Decision Making is supported by DIMACS and its partners, and by the National Science Foundation under grant number CCF-1941871.