Web pages are crawled by being loaded into browser using multiple tabs parallelly

Latest Version

Version
0.0.3
Update
March 8, 2019
Size
121KiB
Developer
Category
Users
46
Rating
1 ★
Manifest Version
2
Web Store Status
Languages Support
English (United States)
See more
Permissions
Tabs
ActiveTab
WebNavigation
See more

Wild Spider Extension

WATCH OUT: more tabs you use, more computer resources (CPU, memory) will be used, and each page costs a bit disk to save the content (in IndexedDb, accessible from extensions -> Inspect views: background page).

The \"spider\" works in this way:
1) The current url is used as the starting point, and it's loaded again in a new tab.
2) After this page is loaded, fetch all the links on the page.
3) Get all the links on the page, including relative urls.
4) Open the extracted link parallelly in all the tabs used (by default 3, set in eventPage).
5) repeat 2-4

All source code at: https://github.com/nobodxbodon/ChromeCrawlerWildSpider
Read more

Source Manifest.json

{"update_url": "https://clients2.google.com/service/update2/crx", "name": "Wild Spider", "short_name": "demo web crawler that's still in experimenting", "description": "web pages are crawled by being loaded into browser using multiple tabs parallelly", "version": "0.0.3", "browser_action": { "default_icon": "icon.png" }, "permissions": [ "tabs", "activeTab", "webNavigation" ], "background": { "scripts": ["Dexie.js", "eventPage.js"], "persistent": false }, "content_scripts": [ { "matches": ["*://*/*"], "js": ["htmlparser2.js", "content.js"] } ], "manifest_version": 2}
Read more

How to download and manually install Wild Spider extension for Chrome

Looking for a way to install Wild Spider extensions for Chrome, you can search for them in the Chrome Web Store, clicking the "Add to Chrome" button, and following the straightforward process.

However, you may need to install Wild Spider extensions manually, this article outlines two simple methods to effortlessly install Wild Spider extensions, whether they come as unpacked folders in compressed formats like .zip or as .crx files.

Method 1: Download Wild Spider extension and manually install through Drag &Drop

If you ever need to manually install a Wild Spider extension for Chrome, don't worry, it's a simple process. Just follow these easy steps, and you will have your extension ready to use in no time.

Step 1: Download the Wild Spider Extension by clicking the "Download CRX" button on the website.

Look for the Wild Spider extension you wish to install. Go to the website and download the Wild Spider extension package, which is usually a .CRX file and comes in a zipped file. Here ".crx" file is the format for Wild Spider extension that contains all the data.

Step 2: Extract &Arrange Files

Once the file is downloaded, extract that file into its own folder. Similarly, give that extension files a permanent home. In other words, keep it in a folder that you don't accidentally delete it because these files will be needed to make that extension work.

Step 3: Go to the Chrome Extensions Page

Open Google Chrome and in the address bar, type chrome://extensions; it will open the Google Chrome extensions page. Make sure that "My extensions" is selected in the sidebar.

Step 4: Drag and Drop the Extension

Now, go to the folder where you extracted the Wild Spider extension and drag &drop the extension file from there anywhere onto the extension page within Google Chrome.

Step 5: Complete the Installation

Once you drag and drop the extension file, follow the steps that Google Chrome takes you through. Similarly, if the extension file is of .crx file format, you will be required to review the permissions and then add the extension.

Now, you can manage that added extension as you would like to with any other Chrome extension that you added through Chrome Webstore. Its icon will also appear in the Google Chrome Menu on the right side of the address bar once you activate it.

Method 2: Download Wild Spider extension for Chrome and install in Developer Mode

This is another method to install Wild Spider extension manually, but the twist is that here, you install by enabling the developer mode option provided in Google Chrome. This mode is commonly used for testing extensions or running unpublished tools.

Step 1: Download the Wild Spider extension file

Select and download the Wild Spider extension by clicking the 'Download CRX' button on the website.

Step 2: Extract the downloaded contents

Convert the file to a ZIP file if it is in CRX format then extract the Wild Spider extension zip file or folder that you downloaded. Make sure you extract it using the same folder name and keep it safely in another folder, so you don't delete it by mistake. The extracted folder will be needed to keep your Wild Spider extension running.

Step 3: Open Chrome Extension Setting Page

In the address bar of Google Chrome, type chrome://extensions and open the Chrome Extension Page.

Step 4: Enable Developer Mode

After opening the Chrome Extension page, look at the top right side, and you will find the toggle option of "Developer mode."Simply enable that developer mode option.

Step 5: Load the Unpacked Extension

Once you enable the developer mode option, you will see the menu of Load Unpacked, Pack Extensions and Update. From that, select the option "Load unpacked."

Step 6: Select the Extension Folder

Once the pop-up opens upon clicking Load unpacked, select the Wild Spider extension directory and click on the "Select Folder "button.

Step 7: Confirm and Install

After you select an extension folder of a Google Chrome extension you're installing manually, confirm its installation for the final time and let the installation complete.

Conclusion

Here we've made a step-by-step guide on How to manually install Wild Spider extension in 2 ways. We hope this guide proves helpful.

Similar Extensions

User reviews

icon

Matt C (ID: 0000000b621d3a6c) Voted 1 ★ - Updated 03/19/2018 doesnt work