Large-scale coordination problems such as driving on highways or negotiating effectively with another agent are ubiquitous. Our ability to develop solutions (sometimes interpreted as norms or equilibria in strategic games) to these decentralized coordination problems have been critical to the development of large-scale societies. Intelligent and autonomous agents need to understand and positively influence these norms.
Under this theme, one of our focuses is on mixed-autonomy traffic networks for the goal of analyzing societal implications of autonomy. We are investigating the effects of autonomous cars on societal objectives such as traffic congestion to see how they can increase traffic flow and reduce delays. More recently, we have studied other mechanisms for influencing to enable reaching socially optimum equilibria beyond the space of driving. We are also interested in mixed-initiative settings, and developing AI agents that can negotiate and coordinate with each other in the presence or absence of a communication channel.
Influencing Human Driving Policies
Autonomus cars influence human drivers so that they can align on the roads for higher efficiency.
In addition to the platooning capabilities of autonomous vehicles, they also have the ability to influence other drivers' behaviors. We develop interaction-level policies for autonomous cars to increase the efficiency of the traffic networks by influencing human drivers and maximizing the gain obtained from platooning. Our work is one of the first to connect micro-level vehicle interactions with macro-level traffic models [CDC 2018, TCNS 2021].
Influencing Routing Policies
From a higher-level perspective, it is possible to reduce traffic congestion by carefully optimizing for the autonomy level of the roads. To achieve that, we first analyze different behaviors (Nash Equilibria, Best Nash Equilibria, Robust Best Nash Equilibria) that emerge in mixed-autonomy traffic networks. We introduce the notion of altruistic autonomy that models how a fleet of autonomous vehicles can act altruistically (as opposed to selfishly), by accepting a larger delay. We analyze Best Altruistic Nash Equilibria that can reduce congestion on parallel traffic networks [WAFR 2018, TCNS 2021].
We further develop reinforcement learning algorithms that optimize autonomous cars' routing choices to make sure all cars will experience the minimum possible latency.
We generalize our works in both absence and presence of altruistic drivers who can (be made) take longer routes for social good, as well as in the presence of perturbations such as accidents [TR-C].
Emergent Prosociality in Multi-Agent Games
In addition to the altruistic agents, we also investigate alternative ways to incentivize multi-agent systems to move from a socially suboptimal Nash equilibrium to a more socially desirable one. We study and analyze gifting, a decentralized peer-rewarding mechanism, in matrix games such as Stag Hunt. We show reinforcement learning agents equipped with gifting actions can reach the prosocial equilibrium more often even when they are completely selfish [IJCAI 2021].
Designing Negotiation Agents
Many real-world interactions are mixed-incentive, where agents have partially aligned goals. As AI agents become embedded in society, it is critical they learn to coordinate with their partners to achieve equitable outcomes. Of the skills necessary to do this, negotiation is paramount. Effective negotiators therefore need to optimize for their own self-interest while also being able to compromise where it makes sense for them to do so. We build negotiation agents that improve their capacity to achieve negotiation outcomes that advance their self-interest and are also Pareto-optimal. We accomplish this through targeted data acquisition, using active learning to acquire new data and expand the pool of negotiation examples we train on [ICML 2021].
Incomplete List of Related Publications:
Erdem Bıyık*, Daniel A. Lazar*, Ramtin Pedarsani, Dorsa Sadigh. Incentivizing Efficient Equilibria in Traffic Networks with Mixed Autonomy. IEEE Transactions on Control of Network Systems (TCNS), 2021. [PDF]
Daniel A. Lazar*, Erdem Bıyık*, Dorsa Sadigh, Ramtin Pedarsani. Learning How to Dynamically Route Autonomous Vehicles on Shared Roads. Transportation Research Part C: Emerging Technologies (TR_C), 2021.[PDF]
Woodrow Z. Wang*, Mark Beliaev*, Erdem Bıyık*, Daniel A. Lazar, Ramtin Pedarsani, Dorsa Sadigh. Emergent Prosociality in Multi-Agent Games Through Gifting. 30th International Joint Conference on Artificial Intelligence (IJCAI) 2021. [PDF]
Minae Kwon, Siddharth Karamcheti, Mariano-Florentino Cuéllar, Dorsa Sadigh. Targeted Data Acquisition for Evolving Negotiation Agents. International Conference on Machine Learning (ICML), 2021. [PDF]
Erdem Bıyık, Daniel A. Lazar, Ramtin Pedarsani, Dorsa Sadigh. The Green Choice: Learning and Influencing Human Decisions on Shared Roads. Proceedings of the 58th IEEE Conference on Decision and Control (CDC), December 2019. [PDF]
