This book constitutes the thoroughly refereed post-proceedings of the Second International Workshop on Machine Learning for Multimodal Interaction, MLMI 2005, held in Edinburgh, UK in July 2005. The 38 revised full papers presented together with 2 invited papers were carefully selected during two rounds of reviewing and revision. The papers are organized in topical sections on multimodal processing, HCI and applications, discourse and dialogue, emotion, visual processing, speech and audio processing, and NIST meeting recognition evaluation.