Many technological advancements in the last decade have changed the ways in which we can interact with intelligent systems or robots (from natural language processing, to interactive learning or collaborative physical interaction). It has often been envisioned that technical systems will move from being tools to becoming cooperation partners. Yet, the field is still struggling with many challenges in less ambitious interaction scenarios, like coping with the individual differences of users, task- or context-dependencies of behavior, or longterm acceptance. I will argue that one missing key ingredient for truly cooperative interactions is a level of artificial social intelligence that allows for understanding and predicting the dynamic, continuous adaptation processes taking place within and between interaction partners. I will present work towards socially intelligent agents that can engage in fast inter-agent coordination during dialogue, actively shape user’s mental models through explanations, and leverage human-like model-switching in theory of mind tasks.