Skip To Content
Toggle navigation
Packages
Upload
Statistics
Documentation
Downloads
Blog
Sign in
Frameworks
.NET
net7.0
net6.0
net5.0
.NET Core
netcoreapp3.1
netcoreapp3.0
netcoreapp2.2
netcoreapp2.1
netcoreapp2.0
netcoreapp1.1
netcoreapp1.0
.NET Standard
netstandard2.1
netstandard2.0
netstandard1.6
netstandard1.5
netstandard1.4
netstandard1.3
netstandard1.2
netstandard1.1
netstandard1.0
.NET Framework
net481
net48
net472
net471
net47
net462
net461
net46
net452
net451
net45
net40
net35
net30
net20
Package type
All types
Dependency
.NET tool
Template
Options
Include prerelease
21 packages returned for Tags:"robots.
txt"
Sort by
Relevance
Downloads
Recently updated
AspNetCore.
SEOHelper
by:
esty_c
171,334 total downloads
last updated
5/11/2020
Latest version:
1.0.1
Robots.txt
sitemap.xml
SEO
Search Engine management tools. This library helps developers to create routing for robots.txt and sitemap.xml. Also provides facility for creating sitemap.xml file and load existing sitemap.xml file.
RobotsTxt
by:
tsllbn
50,250 total downloads
last updated
2/19/2014
Latest version:
2014.2.19
Robots.Txt
Parser
Parsing
Library
A robots.txt parser for .NET Supports ; - Allow directives. - Crawl-delay directives. - Sitemap declarations. - * and $ wildcards. See https://bitbucket.org/cagdas/robotstxt for usage examples.
Our.
Umbraco.
RobotsTxtEditor
by:
leekelleher
lottepitcher
MikeMasey
29,382 total downloads
last updated
10/27/2019
Latest version:
8.0.0
umbraco
robots.txt
editor
An Umbraco package to edit your robots.txt file
SimpleSitemap
by:
pure krome
67,789 total downloads
last updated
9/24/2020
Latest version:
3.2.0
simple
sitemap
sitemaps
robots
robots.txt
unicorn
magicalunicorn
magical-unicorn
worlddomination
worldomination
SimpleSitemap is a lite library that helps you create web sitemaps for collections or lists of items. These sitemaps follow the Sitemap Protocol. Both sitemapindex and urlset links are generated, based upon...
More information
GenHTTP.
Modules.
Robots
by:
Kaliumhexacyanoferrat
15,086 total downloads
last updated
7/3/2023
Latest version:
7.0.0
HTTP
Webserver
C#
Module
Robots
Bots
Robots.txt
Allows to serve a robots instruction file.
RobotsTxt.
Checker
by:
Dobryak
8,282 total downloads
last updated
2/18/2020
Latest version:
0.0.13
utility
robots.txt
check
Small utility, used to check robots.txt on web app and make sure that it is valid. RobotsTxt check is based on https://www.nuget.org/packages/RobotsTxt, but this project is no longer maintained, so I copied it...
More information
RobotRules
by:
vodevil
6,263 total downloads
last updated
6/10/2014
Latest version:
1.5.2.4
Robots.txt
robot
crawler
web
A Simple and Powerful Library to Deal with Web Robots Control Strategy.
SiteMaps.
NET
by:
LiveOrDevTrying
11,585 total downloads
last updated
9/25/2020
Latest version:
2.1.3
sitemap
sitemap.xml
sitemaps
robots
robot
robots.txt
SEO
search
engine
optimization
A simple middleware built on reflection designed to support your Search Engine Optimized (SEO) app by dynamically creating a Sitemap.xml and Robots.txt. This package is designed to be very simple for simple use...
More information
HalloJoe.
RobotsTxt
by:
Hallojoe
1,883 total downloads
last updated
4/3/2018
Latest version:
0.1.0
robots.txt
robotstxt
A tool for accessing and querying the contents of a robots.txt file.
HalloJoe.
XmlSitemap
by:
Hallojoe
1,596 total downloads
last updated
4/3/2018
Latest version:
0.1.0
XmlSitemap
robots.txt
A tool for loading and reading XmlSitemaps.
TamTam.
ConditionalRobots
by:
tamtam
2,649 total downloads
last updated
4/4/2015
Latest version:
1.0.1.5
robots.txt
allow
disallow
- It adds a robots.txt and a robots_closed.txt to the root of the website - It adds a rewrite rule to the web.config that rewrites the robots.txt to robots_closed.txt for all urls ending with tamtam.nl
RobotsSharpParser
by:
nicholasb.za
18,021 total downloads
last updated
12/22/2021
Latest version:
2.0.0
robots
parse
robots
web
crawling
robots.txt
web
scraping
spider
sitemap
More tags
This client library enables working with Robots.txt. Key Features: - Parse robots.txt into Typed object. - Lookup Allowed/Disallowed/Crawldelay based on User-Agent. - Traverse...
More information
Louw.
SitemapParser
by:
louislouw
6,502 total downloads
last updated
1/6/2017
Latest version:
0.0.2
sitemaps.org
sitemap
index
sitemap
sitemap
parser
sitemap
reader
robots.txt
robots.txt
More tags
Louw.SitemapParser
SunamoRobotsTxt
by:
sunamo
2,033 total downloads
last updated
3/29/2023
Latest version:
1.0.9
robots.txt
parser
generator
Robots.txt parser and generator which I'm using on www.sunamo.cz and other sites.
ParkSquare.
AspNetCore.
Sitemap
by:
parksq
10,105 total downloads
last updated
8/15/2023
Latest version:
6.0.2
sitemap
robots
dotnetcore
asp.net
xml
search
generator
mvc
seo
googlebot
More tags
Automatic generation of sitemap.xml and robots.txt based on ASP.NET routes. Extensible engine allows custom URL providers to get URLs from any source. Includes exclude from sitemap feature and video sitemap...
More information
ChordsTech.
MultiSiteRobots
by:
ChordsTech
934 total downloads
last updated
5/4/2020
Latest version:
1.0.0
C#
ASP.Net
Robots.txt
A simple lightweight .Net solution for websites which need easy management of multiple robots.txt
SeoTags
by:
mjebrahimi
4,803 total downloads
last updated
1/21/2022
Latest version:
2.0.1
SEO
OpenGraph
JSON-LD
SiteMap
Robots.txt
AspNetCore
MVC
RazorPages
HtmlHelper
SeoTags create all SEO tags you need such as meta, link, twitter card (twitter:), open graph (og:), and JSON-LD schema (structred data).
Toimik.
RobotsProtocol
by:
nurhafiz
3,698 total downloads
last updated
12/22/2022
Latest version:
0.4.0
robots.txt
robots-txt
robotstxt
robots-exclusion-standard
robots-exclusion-protocol
robots-meta-tag
x-robots-tag
robots
Parsers for Robots Exclusion Standard (aka robots.txt), Robots Meta Tag, and X-Robots-Tag. Visit Project Site for documentation.
Robots.
Txt.
Parser
by:
drmathias
437 total downloads
last updated
9/2/2023
Latest version:
1.0.0-rc8
Robots.Txt
Sitemap
Scraper
Crawler
Spider
Parses robots.txt files and sitemaps. Supports the proposed RFC9309 Robots Exclusion Protocol standard.
Nick.
RobotsParser
by:
nicholasb.za
2,213 total downloads
last updated
1/1/2023
Latest version:
2.0.6
robots
parse
robots
web
crawling
robots.txt
web
scraping
spider
sitemap
More tags
https://github.com/nicholasbergesen/RobotsParser/blob/master/README.md
Previous
Next