Models
Datasets
Spaces
Docs
Enterprise
Pricing
Log In
Sign Up

Collections

Discover the best community collections!

Collections including paper arxiv:2409.08264

a collection of algorithmic agents for user interfaces/interactions, program synthesis, and robotics

End-to-End Goal-Driven Web Navigation

Paper • 1602.02261 • Published Feb 6, 2016
Learning Language Games through Interaction

Paper • 1606.02447 • Published Jun 8, 2016
Naturalizing a Programming Language via Interactive Learning

Paper • 1704.06956 • Published Apr 23, 2017
Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration

Paper • 1802.08802 • Published Feb 24, 2018 • 1

Awesome Computer Use Agents

https://github.com/ranpox/awesome-computer-use

Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction

Paper • 2412.04454 • Published Dec 5, 2024 • 71
Tree Search for Language Model Agents

Paper • 2407.01476 • Published Jul 1, 2024 • 1
SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents

Paper • 2401.10935 • Published Jan 17, 2024 • 5
OmniParser for Pure Vision Based GUI Agent

Paper • 2408.00203 • Published Aug 1, 2024 • 24

Automated Design of Agentic Systems

Paper • 2408.08435 • Published Aug 15, 2024 • 40
Self-Refine: Iterative Refinement with Self-Feedback

Paper • 2303.17651 • Published Mar 30, 2023 • 2
Automating Thought of Search: A Journey Towards Soundness and Completeness

Paper • 2408.11326 • Published Aug 21, 2024 • 3
Building Math Agents with Multi-Turn Iterative Preference Learning

Paper • 2409.02392 • Published Sep 4, 2024 • 16

Embodiment enables interaction of model with environment. Key is to anticipate what change could've come with its current action.

Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation

Paper • 2408.11812 • Published Aug 21, 2024 • 6
WebArena: A Realistic Web Environment for Building Autonomous Agents

Paper • 2307.13854 • Published Jul 25, 2023 • 26
Agent Workflow Memory

Paper • 2409.07429 • Published Sep 11, 2024 • 32
Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale

Paper • 2409.08264 • Published Sep 12, 2024 • 48

Research on GUI Models

Qwen2.5-VL Technical Report

Paper • 2502.13923 • Published Feb 19, 2025 • 213
Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs

Paper • 2404.05719 • Published Apr 8, 2024 • 83
ShowUI: One Vision-Language-Action Model for GUI Visual Agent

Paper • 2411.17465 • Published Nov 26, 2024 • 89
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

Paper • 2409.12191 • Published Sep 18, 2024 • 78

visualwebbench/VisualWebBench

Viewer • Updated Apr 11, 2024 • 1.54k • 515 • 18
X-LANCE/WebSRC_v1.0

Updated Sep 18, 2025 • 82 • 6
rootsautomation/RICO-ScreenQA

Viewer • Updated Apr 16, 2024 • 86k • 117 • 11
rootsautomation/ScreenSpot

Viewer • Updated Apr 10, 2024 • 1.27k • 906 • 44

Papers related to web agents, screen agents, autonomous agents that can mimic human users in computer and/or web use

ScreenAI: A Vision-Language Model for UI and Infographics Understanding

Paper • 2402.04615 • Published Feb 7, 2024 • 44
WebArena: A Realistic Web Environment for Building Autonomous Agents

Paper • 2307.13854 • Published Jul 25, 2023 • 26
Mind2Web: Towards a Generalist Agent for the Web

Paper • 2306.06070 • Published Jun 9, 2023 • 20
Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation

Paper • 2410.13232 • Published Oct 17, 2024 • 44

Law of Vision Representation in MLLMs

Paper • 2408.16357 • Published Aug 29, 2024 • 95
CogVLM2: Visual Language Models for Image and Video Understanding

Paper • 2408.16500 • Published Aug 29, 2024 • 57
Learning to Move Like Professional Counter-Strike Players

Paper • 2408.13934 • Published Aug 25, 2024 • 23
Building and better understanding vision-language models: insights and future directions

Paper • 2408.12637 • Published Aug 22, 2024 • 133

Agentic-ly agentic

Automated Design of Agentic Systems

Paper • 2408.08435 • Published Aug 15, 2024 • 40
On the limits of agency in agent-based models

Paper • 2409.10568 • Published Sep 14, 2024 • 14
On the Diagram of Thought

Paper • 2409.10038 • Published Sep 16, 2024 • 13
DSBench: How Far Are Data Science Agents to Becoming Data Science Experts?

Paper • 2409.07703 • Published Sep 12, 2024 • 66

Can large language models explore in-context?

Paper • 2403.15371 • Published Mar 22, 2024 • 33
Advancing LLM Reasoning Generalists with Preference Trees

Paper • 2404.02078 • Published Apr 2, 2024 • 46
Long-context LLMs Struggle with Long In-context Learning

Paper • 2404.02060 • Published Apr 2, 2024 • 37
Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences

Paper • 2404.03715 • Published Apr 4, 2024 • 62

a collection of algorithmic agents for user interfaces/interactions, program synthesis, and robotics

End-to-End Goal-Driven Web Navigation

Paper • 1602.02261 • Published Feb 6, 2016
Learning Language Games through Interaction

Paper • 1606.02447 • Published Jun 8, 2016
Naturalizing a Programming Language via Interactive Learning

Paper • 1704.06956 • Published Apr 23, 2017
Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration

Paper • 1802.08802 • Published Feb 24, 2018 • 1

visualwebbench/VisualWebBench

Viewer • Updated Apr 11, 2024 • 1.54k • 515 • 18
X-LANCE/WebSRC_v1.0

Updated Sep 18, 2025 • 82 • 6
rootsautomation/RICO-ScreenQA

Viewer • Updated Apr 16, 2024 • 86k • 117 • 11
rootsautomation/ScreenSpot

Viewer • Updated Apr 10, 2024 • 1.27k • 906 • 44

Awesome Computer Use Agents

https://github.com/ranpox/awesome-computer-use

Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction

Paper • 2412.04454 • Published Dec 5, 2024 • 71
Tree Search for Language Model Agents

Paper • 2407.01476 • Published Jul 1, 2024 • 1
SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents

Paper • 2401.10935 • Published Jan 17, 2024 • 5
OmniParser for Pure Vision Based GUI Agent

Paper • 2408.00203 • Published Aug 1, 2024 • 24

Papers related to web agents, screen agents, autonomous agents that can mimic human users in computer and/or web use

ScreenAI: A Vision-Language Model for UI and Infographics Understanding

Paper • 2402.04615 • Published Feb 7, 2024 • 44
WebArena: A Realistic Web Environment for Building Autonomous Agents

Paper • 2307.13854 • Published Jul 25, 2023 • 26
Mind2Web: Towards a Generalist Agent for the Web

Paper • 2306.06070 • Published Jun 9, 2023 • 20
Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation

Paper • 2410.13232 • Published Oct 17, 2024 • 44

Automated Design of Agentic Systems

Paper • 2408.08435 • Published Aug 15, 2024 • 40
Self-Refine: Iterative Refinement with Self-Feedback

Paper • 2303.17651 • Published Mar 30, 2023 • 2
Automating Thought of Search: A Journey Towards Soundness and Completeness

Paper • 2408.11326 • Published Aug 21, 2024 • 3
Building Math Agents with Multi-Turn Iterative Preference Learning

Paper • 2409.02392 • Published Sep 4, 2024 • 16

Law of Vision Representation in MLLMs

Paper • 2408.16357 • Published Aug 29, 2024 • 95
CogVLM2: Visual Language Models for Image and Video Understanding

Paper • 2408.16500 • Published Aug 29, 2024 • 57
Learning to Move Like Professional Counter-Strike Players

Paper • 2408.13934 • Published Aug 25, 2024 • 23
Building and better understanding vision-language models: insights and future directions

Paper • 2408.12637 • Published Aug 22, 2024 • 133

Embodiment enables interaction of model with environment. Key is to anticipate what change could've come with its current action.

Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation

Paper • 2408.11812 • Published Aug 21, 2024 • 6
WebArena: A Realistic Web Environment for Building Autonomous Agents

Paper • 2307.13854 • Published Jul 25, 2023 • 26
Agent Workflow Memory

Paper • 2409.07429 • Published Sep 11, 2024 • 32
Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale

Paper • 2409.08264 • Published Sep 12, 2024 • 48

Agentic-ly agentic

Automated Design of Agentic Systems

Paper • 2408.08435 • Published Aug 15, 2024 • 40
On the limits of agency in agent-based models

Paper • 2409.10568 • Published Sep 14, 2024 • 14
On the Diagram of Thought

Paper • 2409.10038 • Published Sep 16, 2024 • 13
DSBench: How Far Are Data Science Agents to Becoming Data Science Experts?

Paper • 2409.07703 • Published Sep 12, 2024 • 66

Research on GUI Models

Qwen2.5-VL Technical Report

Paper • 2502.13923 • Published Feb 19, 2025 • 213
Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs

Paper • 2404.05719 • Published Apr 8, 2024 • 83
ShowUI: One Vision-Language-Action Model for GUI Visual Agent

Paper • 2411.17465 • Published Nov 26, 2024 • 89
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

Paper • 2409.12191 • Published Sep 18, 2024 • 78

Can large language models explore in-context?

Paper • 2403.15371 • Published Mar 22, 2024 • 33
Advancing LLM Reasoning Generalists with Preference Trees

Paper • 2404.02078 • Published Apr 2, 2024 • 46
Long-context LLMs Struggle with Long In-context Learning

Paper • 2404.02060 • Published Apr 2, 2024 • 37
Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences

Paper • 2404.03715 • Published Apr 4, 2024 • 62

Previous
1
2
Next

Company

TOS Privacy About Careers

Website

Models Datasets Spaces Pricing Docs