RafaelEstevam.Simple.Spider.SqliteStorage
0.8.1
There is a newer version of this package available.
See the version list below for details.
See the version list below for details.
dotnet add package RafaelEstevam.Simple.Spider.SqliteStorage --version 0.8.1
NuGet\Install-Package RafaelEstevam.Simple.Spider.SqliteStorage -Version 0.8.1
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="RafaelEstevam.Simple.Spider.SqliteStorage" Version="0.8.1" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add RafaelEstevam.Simple.Spider.SqliteStorage --version 0.8.1
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
#r "nuget: RafaelEstevam.Simple.Spider.SqliteStorage, 0.8.1"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install RafaelEstevam.Simple.Spider.SqliteStorage as a Cake Addin
#addin nuget:?package=RafaelEstevam.Simple.Spider.SqliteStorage&version=0.8.1
// Install RafaelEstevam.Simple.Spider.SqliteStorage as a Cake Tool
#tool nuget:?package=RafaelEstevam.Simple.Spider.SqliteStorage&version=0.8.1
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
Simple.Spider
A simple and modular web spider written in C# .Net
- Core 3.1
- Net 5.0
This Package was moved from [Net.RafaelEstevam.Spider.Simple.Lib]
Content
Some advantages
- Very simple to use and operate, ideal for lots of small projects or personal ones
- Easy html filter with HObject (a HtmlNode wrap with use similar to JObject)
- Internal conversion from html to XElement, no need to external tools on use
- Automatic Json parser to JObject
- Automatic Json deserialize <T>
- Modular Parser engine (you can add your own parsers!)
- JSON and XML already included
- Modular Caching engine (you can add your own!)
- Stand alone Cache engine included, no need to external softwares
- Modular Downloader engine (you can add your own!)
- WebClient with cookies or HttpClient download engine included
Easy import with NuGet
Getting started
- Start a new console project and add Nuget Reference
- PM> Install-Package Net.RafaelEstevam.Spider.Simple.Lib
- Create a class for your spider (or leave in program)
- create a new instance of SimpleSpider
- Give it a name, cache and log will be saved with that name
- Give it a domain (your spider will not fleet from it)
- Add a event
FetchCompleted
to - Optionally give a first page with
AddPage
. If omitted, it will use the home page of the domain - Call
Execute()
void run()
{
var spider = new SimpleSpider("QuotesToScrape", new Uri("http://quotes.toscrape.com/"));
// Set the completed event to implement your stuff
spider.FetchCompleted += fetchCompleted_items;
// execute
spider.Execute();
}
void fetchCompleted_items(object Sender, FetchCompleteEventArgs args)
{
// walk around ...
// TIP: inspect args to see stuff
var hObj = args.GetHObject();
string[] quotes = hObj["span > .text"];
}
TIP: Use the Simple.Tests project to see examples and poke around
Samples
See all samples at Simple.Tests
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net5.0 is compatible. net5.0-windows was computed. net6.0 was computed. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
.NET Core | netcoreapp3.1 is compatible. |
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.
-
.NETCoreApp 3.1
- RafaelEstevam.Simple.Spider (>= 0.7.425)
- System.Data.SQLite.Core (>= 1.0.113.6)
-
net5.0
- RafaelEstevam.Simple.Spider (>= 0.7.425)
- System.Data.SQLite.Core (>= 1.0.113.6)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
Working release
Commit 74898f3