GLM-PC

What GLM-PC is

GLM-PC is an open agent for graphical interfaces: given a picture of a screen and an instruction in plain words, it returns the clicks and keystrokes that would carry the instruction out.

The point of it is operating an interface, not holding a conversation, and the nine-billion-parameter build is the one shipped inside a commercial desktop assistant.

A note on where this comes from: the product page draws itself in the browser and would not yield its text, so everything here rests on the project repository, which is the authority on the model but not on any product built with it.

What you can do with it

Feed it a picture of the screen and a task described in words, and it answers with an action and the place on screen to perform it, with the set of possible actions written out for clicking, typing, and scrolling.

You have to say which platform is involved, and Windows, macOS, and Android each have their own path, so one instruction is dealt with differently depending on where it runs.

A picture is compulsory; words on their own will not do, and there is no ongoing conversation, although earlier steps can be handed back in so a longer job can carry on. Scores hold up on interface-grounding and one-step tests and fall a little short of the strongest rival models on desktop jobs with several steps. The code carries an open licence, and the weights come under a different one that needs reading before commercial use.

Who it is for

It is presented as something for researchers and developers who work on interface agents, and for organisations putting desktop automation over a vision model.

Teams building automation and applied AI groups are named too, and it is explicit that this is not a product for end users.

Anybody expecting a product to install and use should look elsewhere: this is a model and a repository, and the audience is people building on top of it.

What to watch out for

Anything the account running it is allowed to do, this can do. Looking at a screen implies no sandbox of its own, so a model acting on a misread instruction is able to remove a file, dispatch a message, or complete a purchase, and any restraint has to be built around it.

Those behind it say outright that safe behaviour is not something they can promise, and that the demo is meant for academic reference with responsibility disclaimed. That is a candid stance, and it deserves being read literally instead of being skimmed as standard wording.

Running it on your own machine asks a lot: tens of gigabytes of memory at the usual precision, and the most compressed variant is advised against because results fall off sharply. The compressed builds also want particular hardware, which narrows who can host it themselves.

Nothing newer than late 2024 has been released, and the project is framed as finished research rather than something kept up to date. Desktop jobs with several steps are where it falls behind the strongest rival models, and several steps is exactly what automation consists of.