Task and UI test automation with Computer Vision/OCR. Ui.Vision combines browser automation and desktop automation.

Latest Version

Version
9.3.8
Update
December 8, 2024
Size
8.08MiB
Developer
ui.vision
Category
Users
100,000+
Rating
3.92166 ★
Manifest Version
3
Web Store Status
Languages Support
Deutsch
English
中文 (简体)
See more
Permissions
Bookmarks
ClipboardRead
ClipboardWrite
Cookies
Debugger
Downloads
Downloads.ui
Notifications
Storage
Tabs
ActiveTab
Proxy
NativeMessaging
ContextMenus
WebRequest
WebRequestAuthProvider
SidePanel
Scripting
See more

Ui.Vision Extension

New Dec 5, 2024 Update: Anthropic Claude Computer Use Integration

Open-Source Ui.Vision has consistently been at the forefront of visual web automation. With Claude’s integration, we’re taking the next step forward. The aiComputerUse command allows you to automate complex tasks with a single line of code that would traditionally require hundreds of lines of classic Ui.Vision commands (such as XClick, OCRExtractScreenshot, If/then statements, and more). For example, you can teach Ui.Vision to play TicTacToe with just one short \"Play this game...\" prompt.

--
Ui.Vision is an open-source automation RPA software that combines classic browser automation with modern computer vision and OCR:

(1) **Visual Browser Automation**

Ui.Vision's visual UI testing commands assist web designers and developers in checking and ensuring the accuracy of website layouts and canvas elements. It can identify and read images and text within canvas elements, images, and videos.

(2) **Visual Desktop Automation for Windows, Mac, and Linux**

Beyond web browser automation, Ui.Vision uses image and text recognition (OCR) to automate browser extensions and desktop environments as well. It can interpret images and text on the desktop, executing actions like clicking, moving, dragging and dropping the mouse, and simulating keyboard inputs.

This desktop automation requires installing the free Ui.Vision XModules, available for Windows, Mac, and Linux. These modules provide Ui.Vision with the necessary capabilities for desktop interaction.

(3) **Selenium IDE compatible commands**

Ui.Vision includes Selenium-style commands for web automation, testing, form filling, and web scraping. Learning Ui.Vision also means learning Selenium IDE, and vice versa.

However, Ui.Vision differs in philosophy from the classic Selenium IDE. Ui.Vision offers features not found in the classic Selenium IDE, including computer vision for UI testing, image comparison, file download automation, OCR screen scraping, PDF testing, and capturing full web page and desktop screenshots.

**Command Line API**

Ui.Vision provides a detailed command line API for integration with other applications, often used with Jenkins, CI/CD tools, or the Windows task scheduler. It can be automated and controlled using any programming or scripting language, such as Python or PowerShell.

**Open-Source (AGPL license)**

The Ui.Vision extension source code is available on Github. This makes Ui.Vision a good open-source Selenium IDE alternative and iMacros alternative.

**100% Local Software**

Free and Open-Source. No cloud and no subscription. No recurring payments. The software does not send any data back to us or any other place. Everything, including image recognition and OCR processing, is done locally on your machine.

The only exception to the \"all data is processed locally\" rule is if you select an optional online OCR engine or the AI Computer Use commands. All cloud-features are disabled by default. Only when you explicitly enable them in the settings does Ui.Vision send screenshots to cloud services. The default OCR options are Javascript OCR or XModule OCR, which both run 100% locally on the machine.

**Happy Automating!**

For questions and suggestions, please visit the Ui.Vision community forum at https://forum.ui.vision.
Read more

Source Manifest.json

{"update_url": "https://clients2.google.com/service/update2/crx", "manifest_version": 3, "name": "__MSG_name__", "description": "__MSG_description__", "short_name": "__MSG_short_name__", "default_locale": "en", "version": "9.3.8", "icons": { "128": "logo128.png" }, "action": { "default_icon": { "38": "logo38.png" } }, "background": { "service_worker": "background.js" }, "web_accessible_resources": [ { "resources": [ "content_script.js", "inject.js", "logo.png" ], "matches": [ "" ], "extension_ids": [] } ], "content_scripts": [ { "matches": [ "" ], "all_frames": true, "match_about_blank": true, "js": [ "content_script.js" ], "run_at": "document_start" } ], "side_panel": { "default_path": "sidepanel.html" }, "options_page": "options.html", "offline_enabled": true, "permissions": [ "bookmarks", "clipboardRead", "clipboardWrite", "cookies", "debugger", "downloads", "downloads.ui", "notifications", "storage", "tabs", "activeTab", "proxy", "nativeMessaging", "contextMenus", "webRequest", "webRequestAuthProvider", "sidePanel", "scripting" ], "host_permissions": [ "" ], "content_security_policy": { "extension_pages": "script-src 'self' 'wasm-unsafe-eval';" }}
Read more

Similar Extensions

icon

Selenium IDE Extension · Developer Tools 100,000+ 286 ★ 3.96MiB

icon

Chromium browser automation Extension · Developer Tools 100,000+ 209 ★ 136KiB

icon

Automation Extension · Developer Tools 100,000+ 10 ★ 33.59KiB

icon

Automa Extension · Developer Tools 100,000+ 205 ★ 2.12MiB

icon

Katalon Recorder (Selenium tests generator) Extension · Productivity 100,000+ 263 ★ 6.23MiB

icon

Selenium Object Finder Extension · Developer Tools 100,000+ 20 ★ 181KiB

icon

SiteMacro Extension · Productivity 100,000+ 14 ★ 43.98KiB

icon

Browserflow - Web Scraping & Web Automation Extension · Productivity 100,000+ 222 ★ 1.06MiB

icon

SnapTest Extension · Developer Tools 100,000+ 23 ★ 9.27MiB

icon

Axiom Browser Automation & Web Scraping Extension · Developer Tools 100,000+ 185 ★ 11.25MiB

icon

Robotcorder Extension · Developer Tools 100,000+ 18 ★ 210KiB

icon

Selenium Page Object Generator Extension · Developer Tools 100,000+ 64 ★ 335KiB

icon

Record/Replay Extension · Developer Tools 100,000+ 9 ★ 2.44MiB

icon

Wildfire Extension · Productivity 100,000+ 93 ★ 23.27MiB

icon

Applitools for Selenium IDE Extension · Developer Tools 100,000+ 11 ★ 2.45MiB

icon

Agenty - Advanced Web Scraper Extension · Developer Tools 100,000+ 175 ★ 604KiB

icon

Simplescraper — a fast and free web scraper Extension · Productivity 100,000+ 327 ★ 1.03MiB

icon

Ghost Inspector - Web Test Recorder Extension · Developer Tools 100,000+ 30 ★ 648KiB

icon

Automatio - Web Automation & Scraping Extension · Productivity 100,000+ 219 ★ 1.77MiB

icon

Automation Anywhere Extension · Developer Tools 100,000+ 18 ★ 209KiB

icon

Element Locator Extension · Developer Tools 100,000+ 22 ★ 264KiB

icon

xPath Finder Extension · Developer Tools 100,000+ 34 ★ 15.28KiB

icon

NoCoding Data Scraper - Easy Web Scraping Extension · Productivity 100,000+ 176 ★ 7.0MiB

icon

Ranorex Automation Extension · Developer Tools 100,000+ 58 ★ 37.23KiB

User reviews

icon

Joshua McFarlane (ID: 000000bf41cd8e3b) Voted 3 ★ - Updated 12/04/2023 Good (although an extremely steep learning curve) when it works. But lately all I get is 'DOM not found' over and over again.

icon

ali raza (ID: 0000002d1b19dc0f) Voted 5 ★ - Updated 10/05/2023 if anyone want to create automation in UI vision extention i can help you.

icon

Marco Cetina (ID: 00000089f1dc39be) Voted 5 ★ - Updated 08/03/2023 It really makes our life way easier!!! I love it!!!!

icon

COW KING (THE COW KING) (ID: 0000007f1ee8a960) Voted 3 ★ - Updated 07/08/2023 seems the developer has put some effort in it, but it's still unintuitive and difficult to use, lacks simple features. I need something simple enough to just help me click a specific button on a webpage when a specific url pattern is opened (such as automatically thumbup every video I watched on youtube). sadly it's not able to do that

icon

Lulu (ID: 0000009e4194deb6) Voted 5 ★ - Updated 07/05/2023 Loving it so far! It is so easy to use. It is saving me a half day of work today. At first I couldn't get it to run. I then added and created another macro and saved it. Then I was able to run it! I didn't watch the tutorial so I'm sure this was discussed. No coding experience here!