71 packages returned for Tags:"crawler"

This is an agile HTML parser that builds a read/write DOM and supports plain XPATH or XSLT (you actually don't HAVE to understand XPATH nor XSLT to use it, don't worry...). It is a .NET code library that allows you to parse "out of the web" HTML files. The parser is very tolerant with "real world"... More information

Deprecated as there's new maintainer for original HAP project. Please check the new repo at https://github.com/zzzprojects/html-agility-pack. This is a port of HtmlAgilityPack library created by Simon Mourrier and Jeff Klawiter for .NET Core platform. This NuGet package supports can be used with... More information

Abot is an open source C# web crawler built for speed and flexibility. It takes care of the low level plumbing (multithreading, http requests, scheduling, link parsing, etc..). You just register for events to process the page data. You can also plugin your own implementations of core interfaces to... More information

A powerful C# web crawler that makes advanced crawling features easy to use. AbotX builds upon the open source Abot C# Web Crawler by providing a powerful set of wrappers and extensions.

dcsoup is a .NET library for working with real-world HTML. It provides a very convenient API for extracting and manipulating data, using the best of DOM, CSS, and jquery-like methods. This library is basically a port of jsoup, a Java HTML parser library. see also: http://jsoup.org/ API reference is... More information

ASP.NET Core Detection Crawler resolver components

Web scraper / crawler / spider. Supports robots protocol and user agent.

This is an agile HTML parser that builds a read/write DOM and supports plain XPATH or XSLT (you actually don't HAVE to understand XPATH nor XSLT to use it, don't worry...). It is a .NET code library that allows you to parse "out of the web" HTML files. The parser is very tolerant with "real world"... More information

A .NET Standard web crawling library similar to WebMagic and Scrapy. It is a lightweight ,efficient and fast high-level web crawling & scraping framework for .NET

.NET Core port of sjdirect/abot. Abot is an open source C# web crawler built for speed and flexibility. It takes care of the low level plumbing (multithreading, http requests, scheduling, link parsing, etc..). You just register for events to process the page data. You can also plugin your own... More information

A web scraping framework for .Net

A .NET Standard web crawling library similar to WebMagic and Scrapy. It is a lightweight ,efficient and fast high-level web crawling & scraping framework for .NET

Crawler-Lib Concurrency Testing allows to write unit tests with multiple threads to test the concurrency behavior of components. It has synchronization mechanisms to control the workflow of the threads and to record the execution steps. It is also possible to use it for client/server tests.... More information

This is a port of HtmlAgilityPack library created by Simon Mourrier and Jeff Klawiter for .NET Core platform. This NuGet package supports can be used with Universal Windows Platform, ASP.NET 5 (using .NET Core) and full .NET Framework 4.6. Original description: This is an agile HTML parser that... More information

Simple web crawler that return IObservable using Reactive Extension(Rx) and async await.

AppStoresScraper is library for downloading app metadata from Steam, Google, Apple and Windows app stores

A web scraping framework for .Net

Library allows you to extract different data from text or web sources, analyze text content, extract different information and write your own data mining & extracting applications and services.

Crawler for main social networks

简单、易用、高效 一个有态度的开源.Net Http请求框架!可以用制作爬虫，api请求等等。让你感受一个简易到极致的HTTP编程. 让编程更简易,代码更简洁。用法请查看：https://github.com/stulzq/HttpCode.Core