Skip to content

CogAgent

An open-source GUI agent built on a vision language model

Visit official site

What CogAgent is

The model is described as an open-source end-to-end GUI agent based on a vision language model, interacting through screenshots and natural language in Chinese and English.

What you can do with it

  • Interact with a GUI through screenshots and natural language
  • Carry out GUI perception and reasoning prediction
  • Automate operations on desktop apps

Getting started

  • Install Python 3.10.16 or newer and the listed dependencies
  • Run the model with the right command line arguments
  • Supply an input image and receive action instructions

What to watch out for

  • Output follows a strict string format without JSON
  • Continuous dialogue is not supported
  • Bounding box results mark the action region

Pros & cons

✓ What we like

  • Screenshots and natural language drive the interaction
  • Multi-stage training with policy improvement is described

! What to watch out for

  • Output is a strict string and JSON output is not supported
  • Continuous dialogue is unsupported, though execution history is

FAQ

What output format is used?

The material states output is a strict string format and JSON output is not supported.

Which base model is named?

CogAgent-9B-20241220 is described as based on GLM-4V-9B.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review