Séminaire au DIC: «Perception tokens for visual reasoning» par Ranjay Krishna
Événement
Séminaire ayant lieu dans le cadre du Doctorat en informatique cognitive, en collaboration avec le centre de recherche CRIA TITRE : Perception tokens for visual reasoning Ranjay KRISHNA Jeudi le 19 novembre 2026 à 10h30 Local PK-5115 (Il est possible d'y assister en virtuel en vous inscrivant ici) RÉSUMÉ Today’s vision-language models can describe images fluently, yet they still struggle with the spatial and perceptual reasoning required for robotics. I will argue that visual reasoning—reasoning through points, sketches, trajectories, depth maps, and other grounded representations—will become more important than language reasoning for physical agents. It traces progress from benchmarks exposing fundamental perceptual failures to methods that teach models to reason directly in visual space. These ideas culminate in MolmoAct and MolmoAct2, open vision-language-action models that convert visual plans into robot actions. The resulting systems are interpretable, steerable, deployable zero-shot on affordable hardware, and competitive with proprietary models trained on substantially more data—advancing the goal of accessible, general-purpose robots for real-world environments. BIOGRAPHIE Ranjay KRISHNA is Assistant Professor in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, where he co-directs the RAIVN lab. His research bridges computer vision, natural language processing, robotics, and human-computer interaction, focusing on teaching machines to perceive the visual world and interact with people by drawing on behavioral and social-science frameworks. He previously directed the multimodal and embodied AI team at the Allen Institute for AI. RÉFÉRENCES Bigverdi, M., Luo, Z., Hsieh, C.-Y., Shen, E., Chen, D., Shapiro, L. G., & Krishna, R. (2025). Perception Tokens Enhance Visual Reasoning in Multimodal Language Models. Conference on Computer Vision and Pattern Recognition (CVPR). Liu, B., Dong, Y., Wang, Y., Ma, Z., Tang, Y., Tang, L., Rao, Y., Ma, W.-C., & Krishna, R. (2025). Coarse Correspondences Boost Spatial-Temporal Reasoning in Multimodal Language Model. Conference on Computer Vision and Pattern Recognition (CVPR), 3783–3792.
Lieux
- UQAM - Pavillon Président-Kennedy (PK) — 201, avenue du Président-Kennedy
Dates et horaires
- 2026-11-19 à 10:30
Tarifs
Non renseigné
Informations et réservations
La fiche interactive s’ouvre automatiquement lorsque l’application est chargée.