Task and UI test automation with Computer Vision/OCR. Ui.Vision combines browser automation and desktop automation.

Latest Version

Version
9.3.8
Update
December 8, 2024
Size
8.08MiB
Developer
ui.vision
Category
Users
100,000+
Rating
3.92166 ★
Manifest Version
3
Web Store Status
Languages Support
Deutsch
English
中文 (简体)
See more
Permissions
Bookmarks
ClipboardRead
ClipboardWrite
Cookies
Debugger
Downloads
Downloads.ui
Notifications
Storage
Tabs
ActiveTab
Proxy
NativeMessaging
ContextMenus
WebRequest
WebRequestAuthProvider
SidePanel
Scripting
See more

Ui.Vision Extension

New Dec 5, 2024 Update: Anthropic Claude Computer Use Integration

Open-Source Ui.Vision has consistently been at the forefront of visual web automation. With Claude’s integration, we’re taking the next step forward. The aiComputerUse command allows you to automate complex tasks with a single line of code that would traditionally require hundreds of lines of classic Ui.Vision commands (such as XClick, OCRExtractScreenshot, If/then statements, and more). For example, you can teach Ui.Vision to play TicTacToe with just one short \"Play this game...\" prompt.

--
Ui.Vision is an open-source automation RPA software that combines classic browser automation with modern computer vision and OCR:

(1) **Visual Browser Automation**

Ui.Vision's visual UI testing commands assist web designers and developers in checking and ensuring the accuracy of website layouts and canvas elements. It can identify and read images and text within canvas elements, images, and videos.

(2) **Visual Desktop Automation for Windows, Mac, and Linux**

Beyond web browser automation, Ui.Vision uses image and text recognition (OCR) to automate browser extensions and desktop environments as well. It can interpret images and text on the desktop, executing actions like clicking, moving, dragging and dropping the mouse, and simulating keyboard inputs.

This desktop automation requires installing the free Ui.Vision XModules, available for Windows, Mac, and Linux. These modules provide Ui.Vision with the necessary capabilities for desktop interaction.

(3) **Selenium IDE compatible commands**

Ui.Vision includes Selenium-style commands for web automation, testing, form filling, and web scraping. Learning Ui.Vision also means learning Selenium IDE, and vice versa.

However, Ui.Vision differs in philosophy from the classic Selenium IDE. Ui.Vision offers features not found in the classic Selenium IDE, including computer vision for UI testing, image comparison, file download automation, OCR screen scraping, PDF testing, and capturing full web page and desktop screenshots.

**Command Line API**

Ui.Vision provides a detailed command line API for integration with other applications, often used with Jenkins, CI/CD tools, or the Windows task scheduler. It can be automated and controlled using any programming or scripting language, such as Python or PowerShell.

**Open-Source (AGPL license)**

The Ui.Vision extension source code is available on Github. This makes Ui.Vision a good open-source Selenium IDE alternative and iMacros alternative.

**100% Local Software**

Free and Open-Source. No cloud and no subscription. No recurring payments. The software does not send any data back to us or any other place. Everything, including image recognition and OCR processing, is done locally on your machine.

The only exception to the \"all data is processed locally\" rule is if you select an optional online OCR engine or the AI Computer Use commands. All cloud-features are disabled by default. Only when you explicitly enable them in the settings does Ui.Vision send screenshots to cloud services. The default OCR options are Javascript OCR or XModule OCR, which both run 100% locally on the machine.

**Happy Automating!**

For questions and suggestions, please visit the Ui.Vision community forum at https://forum.ui.vision.
Read more

Source Manifest.json

{"update_url": "https://clients2.google.com/service/update2/crx", "manifest_version": 3, "name": "__MSG_name__", "description": "__MSG_description__", "short_name": "__MSG_short_name__", "default_locale": "en", "version": "9.3.8", "icons": { "128": "logo128.png" }, "action": { "default_icon": { "38": "logo38.png" } }, "background": { "service_worker": "background.js" }, "web_accessible_resources": [ { "resources": [ "content_script.js", "inject.js", "logo.png" ], "matches": [ "" ], "extension_ids": [] } ], "content_scripts": [ { "matches": [ "" ], "all_frames": true, "match_about_blank": true, "js": [ "content_script.js" ], "run_at": "document_start" } ], "side_panel": { "default_path": "sidepanel.html" }, "options_page": "options.html", "offline_enabled": true, "permissions": [ "bookmarks", "clipboardRead", "clipboardWrite", "cookies", "debugger", "downloads", "downloads.ui", "notifications", "storage", "tabs", "activeTab", "proxy", "nativeMessaging", "contextMenus", "webRequest", "webRequestAuthProvider", "sidePanel", "scripting" ], "host_permissions": [ "" ], "content_security_policy": { "extension_pages": "script-src 'self' 'wasm-unsafe-eval';" }}
Read more

How to download and manually install Ui.Vision extension for Chrome

Looking for a way to install Ui.Vision extensions for Chrome, you can search for them in the Chrome Web Store, clicking the "Add to Chrome" button, and following the straightforward process.

However, you may need to install Ui.Vision extensions manually, this article outlines two simple methods to effortlessly install Ui.Vision extensions, whether they come as unpacked folders in compressed formats like .zip or as .crx files.

Method 1: Download Ui.Vision extension and manually install through Drag &Drop

If you ever need to manually install a Ui.Vision extension for Chrome, don't worry, it's a simple process. Just follow these easy steps, and you will have your extension ready to use in no time.

Step 1: Download the Ui.Vision Extension by clicking the "Download CRX" button on the website.

Look for the Ui.Vision extension you wish to install. Go to the website and download the Ui.Vision extension package, which is usually a .CRX file and comes in a zipped file. Here ".crx" file is the format for Ui.Vision extension that contains all the data.

Step 2: Extract &Arrange Files

Once the file is downloaded, extract that file into its own folder. Similarly, give that extension files a permanent home. In other words, keep it in a folder that you don't accidentally delete it because these files will be needed to make that extension work.

Step 3: Go to the Chrome Extensions Page

Open Google Chrome and in the address bar, type chrome://extensions; it will open the Google Chrome extensions page. Make sure that "My extensions" is selected in the sidebar.

Step 4: Drag and Drop the Extension

Now, go to the folder where you extracted the Ui.Vision extension and drag &drop the extension file from there anywhere onto the extension page within Google Chrome.

Step 5: Complete the Installation

Once you drag and drop the extension file, follow the steps that Google Chrome takes you through. Similarly, if the extension file is of .crx file format, you will be required to review the permissions and then add the extension.

Now, you can manage that added extension as you would like to with any other Chrome extension that you added through Chrome Webstore. Its icon will also appear in the Google Chrome Menu on the right side of the address bar once you activate it.

Method 2: Download Ui.Vision extension for Chrome and install in Developer Mode

This is another method to install Ui.Vision extension manually, but the twist is that here, you install by enabling the developer mode option provided in Google Chrome. This mode is commonly used for testing extensions or running unpublished tools.

Step 1: Download the Ui.Vision extension file

Select and download the Ui.Vision extension by clicking the 'Download CRX' button on the website.

Step 2: Extract the downloaded contents

Convert the file to a ZIP file if it is in CRX format then extract the Ui.Vision extension zip file or folder that you downloaded. Make sure you extract it using the same folder name and keep it safely in another folder, so you don't delete it by mistake. The extracted folder will be needed to keep your Ui.Vision extension running.

Step 3: Open Chrome Extension Setting Page

In the address bar of Google Chrome, type chrome://extensions and open the Chrome Extension Page.

Step 4: Enable Developer Mode

After opening the Chrome Extension page, look at the top right side, and you will find the toggle option of "Developer mode."Simply enable that developer mode option.

Step 5: Load the Unpacked Extension

Once you enable the developer mode option, you will see the menu of Load Unpacked, Pack Extensions and Update. From that, select the option "Load unpacked."

Step 6: Select the Extension Folder

Once the pop-up opens upon clicking Load unpacked, select the Ui.Vision extension directory and click on the "Select Folder "button.

Step 7: Confirm and Install

After you select an extension folder of a Google Chrome extension you're installing manually, confirm its installation for the final time and let the installation complete.

Conclusion

Here we've made a step-by-step guide on How to manually install Ui.Vision extension in 2 ways. We hope this guide proves helpful.

Similar Extensions

icon

Selenium IDE Extension · Developer Tools 100,000+ 286 ★ 3.96MiB

icon

Chromium browser automation Extension · Developer Tools 100,000+ 209 ★ 136KiB

icon

Automation Extension · Developer Tools 100,000+ 10 ★ 33.59KiB

icon

Automa Extension · Developer Tools 100,000+ 205 ★ 2.12MiB

icon

Katalon Recorder (Selenium tests generator) Extension · Productivity 100,000+ 263 ★ 6.23MiB

icon

Selenium Object Finder Extension · Developer Tools 100,000+ 20 ★ 181KiB

icon

SiteMacro Extension · Productivity 100,000+ 14 ★ 43.98KiB

icon

Browserflow - Web Scraping & Web Automation Extension · Productivity 100,000+ 222 ★ 1.06MiB

icon

SnapTest Extension · Developer Tools 100,000+ 23 ★ 9.27MiB

icon

Axiom Browser Automation & Web Scraping Extension · Developer Tools 100,000+ 185 ★ 11.25MiB

icon

Robotcorder Extension · Developer Tools 100,000+ 18 ★ 210KiB

icon

Selenium Page Object Generator Extension · Developer Tools 100,000+ 64 ★ 335KiB

icon

Record/Replay Extension · Developer Tools 100,000+ 9 ★ 2.44MiB

icon

Wildfire Extension · Productivity 100,000+ 93 ★ 23.27MiB

icon

Applitools for Selenium IDE Extension · Developer Tools 100,000+ 11 ★ 2.45MiB

icon

Agenty - Advanced Web Scraper Extension · Developer Tools 100,000+ 175 ★ 604KiB

icon

Simplescraper — a fast and free web scraper Extension · Productivity 100,000+ 327 ★ 1.03MiB

icon

Ghost Inspector - Web Test Recorder Extension · Developer Tools 100,000+ 30 ★ 648KiB

icon

Automatio - Web Automation & Scraping Extension · Productivity 100,000+ 219 ★ 1.77MiB

icon

Automation Anywhere Extension · Developer Tools 100,000+ 18 ★ 209KiB

icon

Element Locator Extension · Developer Tools 100,000+ 22 ★ 264KiB

icon

xPath Finder Extension · Developer Tools 100,000+ 34 ★ 15.28KiB

icon

NoCoding Data Scraper - Easy Web Scraping Extension · Productivity 100,000+ 176 ★ 7.0MiB

icon

Ranorex Automation Extension · Developer Tools 100,000+ 58 ★ 37.23KiB

User reviews

icon

Joshua McFarlane (ID: 000000bf41cd8e3b) Voted 3 ★ - Updated 12/04/2023 Good (although an extremely steep learning curve) when it works. But lately all I get is 'DOM not found' over and over again.

icon

ali raza (ID: 0000002d1b19dc0f) Voted 5 ★ - Updated 10/05/2023 if anyone want to create automation in UI vision extention i can help you.

icon

Marco Cetina (ID: 00000089f1dc39be) Voted 5 ★ - Updated 08/03/2023 It really makes our life way easier!!! I love it!!!!

icon

COW KING (THE COW KING) (ID: 0000007f1ee8a960) Voted 3 ★ - Updated 07/08/2023 seems the developer has put some effort in it, but it's still unintuitive and difficult to use, lacks simple features. I need something simple enough to just help me click a specific button on a webpage when a specific url pattern is opened (such as automatically thumbup every video I watched on youtube). sadly it's not able to do that

icon

Lulu (ID: 0000009e4194deb6) Voted 5 ★ - Updated 07/05/2023 Loving it so far! It is so easy to use. It is saving me a half day of work today. At first I couldn't get it to run. I then added and created another macro and saved it. Then I was able to run it! I didn't watch the tutorial so I'm sure this was discussed. No coding experience here!