Extract tokens from a string of text for use with NLP tools or statistical analysis.
Install-Package McBits.Tokenization -Version 2.0.0
dotnet add package McBits.Tokenization --version 2.0.0
<PackageReference Include="McBits.Tokenization" Version="2.0.0" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add McBits.Tokenization --version 2.0.0
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
- No dependencies.
This package is not used by any popular GitHub repositories.