SmartAICompendium.TextGeneration 1.1.0

This package allow easy and typed text generation with a local model and a embedded python. Compatible with windows, linux and mac.

Generation Parameters

**valid parameters will depend of the model

Usage examples

Generation with huggingface model

The model can be a huggingface repoId ("{user}/{model}") or local folder containing the model. The model will be downloaded if not already present (see SmartAICompendium.Common)

var model = new TextInference("aihub-app/zyte-1B");

var text = model.Generate("Instruct: Give the list of french speaking countries\nOutput:",
maxNewTokens: 256, badWords:null, topK: 40, topP: 0.95f, temperature: 0.8f, repetitionPenalty: 1.1f, lastNTokens: 64, int seed:-1, reset: true, batchSize: 8);

Console.WriteLine(text);

Chat Generation with local reference

var model = new TextInference("C:\\TinyLlama-1.1B-Chat-v1.0");

var message = model.GenerateChat(new ChatMessage[]{
        new(){
            Role = "system",
            Content = "You are an adventurer who always responds in the style of a pirate."
        },
        new(){
            Role = "user",
            Content = "Tell me more about your last adventure."
        }
    },
    maxNewTokens: 256, temperature: 0.7f, topK: 50, topP: 0.95f);

Console.WriteLine(message.Content);

Loading packages before use

if not done, the packages will be installed at the class initialization.

TextInference.Update();

Support My work