NipahTokenizer 3.1.7
See the version list below for details.
dotnet add package NipahTokenizer --version 3.1.7
NuGet\Install-Package NipahTokenizer -Version 3.1.7
<PackageReference Include="NipahTokenizer" Version="3.1.7" />
paket add NipahTokenizer --version 3.1.7
#r "nuget: NipahTokenizer, 3.1.7"
// Install NipahTokenizer as a Cake Addin #addin nuget:?package=NipahTokenizer&version=3.1.7 // Install NipahTokenizer as a Cake Tool #tool nuget:?package=NipahTokenizer&version=3.1.7
Nipah Tokenizer
Installing
dotnet add package NipahTokenizer
Using
First setup the tokenizer options as needed, you can use the default as follows:
var tokenizerOptions = TokenizerOptions.Default;
Then just create a new instance of tokenizer and call "Tokenize" passing tokenizer options as arg:
var tokenizer = new Tokenizer();
var tokens = tokenizer.Tokenize("any-text-here", tokenizerOptions);
You can now just iterate over tokens as needed.
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net6.0 is compatible. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
-
net6.0
- No dependencies.
NuGet packages (1)
Showing the top 1 NuGet packages that depend on NipahTokenizer:
Package | Downloads |
---|---|
Nipah.Markdown
A library to make simple parses of markdown |
GitHub repositories
This package is not used by any popular GitHub repositories.
Tokens now will parse by default integer literals as '64-bit' and floating point number literals also as '64-bit', this should make using them to cast down easier in parsers