Windows

[gitgo1994]

Hi, I loved your demo and I downloaded the project, edited the code to send requests to different model than ACE (since I didn't have access yet)

Only to notice then that the current implementation of the Computer class is designed exclusively for macOS. It relies on the screencapture command, which does not exist on Windows. I was wondering if I would edit it to Capture screenshots using Windows-compatible libraries like pyautogui or mss and Simulate mouse and keyboard input using pyautogui or pywinauto would yield the same results for the speed? If answer is maybe, any specific models that can do the same job as ACE or is the model itself not a big factor in terms of speed?

[gitgo1994]

update: it worked, its able to open the browser, opened google or any site as I instruct it instantly, and followed simple instructions, but mouse clicks is not accurate at all, // not reliable, any idea what could be the problem? I have a large screen. 3340x1440p.

[imomin]

Here is an open source project, you can give it a try.
https://github.com/niuzaisheng/ScreenAgent

[thelege2nd]

Here is an open source project, you can give it a try. https://github.com/niuzaisheng/ScreenAgent

Thanks for sharing, is it as fast as generalagents demo?

[imomin]

@thelege2nd, GenralAgents demo is impressive, however, I have to test it myself to see how fast it is :).
I doubt ScreenAgent as fast as shown in the demo since it uses screenshots with vision model. Where as, "Ace leverages a new behavioral training paradigm. Unlike language and vision models which are trained on text and images, Ace is trained on behavior—the process that generates text, images, and other work outputs. Training on behavior generalizes better, as corroborated by the use of step-by-step reasoning in training frontier language models."...