syntaxist 1.3.0

Syntaxist

A software library for analysing and rewriting texts and translations. The power behind Fairslator.

Note: Syntaxist is copyrighted software. The author and copyright holder is me, Michal Měchura. Contact me for licensing and permissions: michmech@lexiconista.com.

Setting up

1. Install this NuGet package in your project. Everything Syntaxist provides will be available to you under the namespace Lexiconista.Syntaxist.

2. Take the lexicon files I have sent to you, unzip them, put them in a folder anywhere on your computer, and tell Syntaxist the name of the folder they are in:

using Syntaxist = Lexiconista.Syntaxist;
namespace MyProject {
    internal class Program {
        static void Main(string[] args) {
            
            Syntaxist.Settings.LexiconsFolder=@"C:\MyLexicons";

        }
    }
}

Note: These lexicon files are necessary for Syntaxist to work. Syntaxist performs no useful function without them. The rest of this document assumes the lexicons are in place.

Analyzing texts

Syntaxist can take a string of text in some language (currently: only German) and tell you interesting facts about people mentioned in it.

First person: if the first person is mentioned in the text (typically through pronouns such as I and we), Syntaxist can tell you whether it is singular or plural, which gender it is (if any), and whether it can be rewritten into the opposite gender:

Syntaxist.Analyzer analyzer = new Syntaxist.Analyzer("de");
Syntaxist.Analysis analysis = analyzer.Analyze("Ich bin ein guter Student."); //"I am a good student."
Console.WriteLine(analysis.p1.num);        //"s" = singular (or "p" = plural)
Console.WriteLine(analysis.p1.gen);        //"m" = male (or "f" = female)
Console.WriteLine(analysis.p1.genFreedom); //"mfb" = can change between "m" male, "f" female and "b" gender-neutral

Second person: if the second person is mentioned in the text (typically through pronouns such as you), Syntaxist can tell you whether it is singular or plural, which gender it is (if any), which register it is (informal or formal), and whether it can be rewritten into the opposite gender and/or the opposite register:

Syntaxist.Analyzer analyzer = new Syntaxist.Analyzer("de");
Syntaxist.Analysis analysis = analyzer.Analyze("Bist du ein guter Student?"); //"Are you a good student?"
Console.WriteLine(analysis.p2.num); //"s" = singular
Console.WriteLine(analysis.p2.gen); //"m" = male
Console.WriteLine(analysis.p2.genFreedom); //"mfb"
Console.WriteLine(analysis.p2.reg); //"t" = informal (or "v" = formal)
Console.WriteLine(analysis.p2.regFreedom); //"tv" = can change between "t" informal and "v" formal

Third persons: if any other persons are mentioned in the text, typically through pronouns such as he, she, they or through nouns like student that refer to people, Syntaxist can tell you whether each of them is singular or plural, which gender it is, and whether it can be rewritten into the opposite gender:

Syntaxist.Analyzer analyzer = new Syntaxist.Analyzer("de");
Syntaxist.Analysis analysis = analyzer.Analyze("Die Studenten mögen die Lehrerin."); //The students like the teacher

Console.WriteLine(analysis.p3[0].nicknameWord); //"Studenten" = a keyword taken from text
Console.WriteLine(analysis.p3[0].num); //"p" = plural
Console.WriteLine(analysis.p3[0].gen); //"m" = male (or rather, the male default)
Console.WriteLine(analysis.p3[0].genFreedom); //"mfb"

Console.WriteLine(analysis.p3[1].nicknameWord); //"Lehrerin"
Console.WriteLine(analysis.p3[1].num); //"s" = singular
Console.WriteLine(analysis.p3[1].gen); //"f" = female
Console.WriteLine(analysis.p3[1].genFreedom); //"mfb"

Analyzing translations

If you know that the text you are analyzing is a translation from some other language (currently: only English), then you can pass the source text in as an optional argument. In some cases this will help Syntaxist understand the text more accurately. For example, in the following German sentence Syntaxist will understand that the "Sie" is second-person formal ("you are") and not third-person plural ("they are"): the English sentence helps to disambiguate that.

Syntaxist.Analyzer analyzer = new Syntaxist.Analyzer("de");
Syntaxist.Analysis analysis = analyzer.Analyze("Sie sind hier.", "en", "You are here.");

If you pass in a translation source, third-person nickname words will be keywords taken from the source text, not from the translation:

Syntaxist.Analyzer analyzer = new Syntaxist.Analyzer("de");
Syntaxist.Analysis analysis = analyzer.Analyze("Die Studenten mögen die Lehrerin.", "en", "The students like the teacher");
Console.WriteLine(analysis.p3[0].nicknameWord); //"students"
Console.WriteLine(analysis.p3[1].nicknameWord); //"teacher"

Analyzing texts with inline markup

If the text you are analyzing contains inline markup (such as XML, HTML or Markdown), you can optionally pass in a regular expression to recognize the markup. Syntaxist will ignore the markup when analyzing the text:

Syntaxist.Analysis analysis = analyzer.Analyze("Das ist der <b>neue</b> Arzt.", "<[^>]+>");

If you are analyzing a translation along with its source text, they can both contain inline markup as long as it's markup of the same kind (recognized by the same regular expression):

Syntaxist.Analysis analysis = analyzer.Analyze("Das ist der <b>neue</b> Arzt.", "en", "That's the <b>new</b> doctor.", "<[^>]+>");

Rewriting texts

Syntaxist can take a string of text in some language (currently: only German) and rewrite it by changing the genders of people mentioned in the text and their forms of address.

Rewriting with and without analysis

There are two ways you can do rewriting in Syntaxist. One way is to analyze the text first, then change a few settings in the analysis object, and then ask Syntaxist to reinflect the text in accordance with your changed settings:

//analyze a text:
Syntaxist.Analyzer analyzer = new Syntaxist.Analyzer("de");
Syntaxist.Analysis analysis = analyzer.Analyze("Sie sind ein guter Student."); //"You are a good student."

//change properties of the second person:
analysis.p2.genChange="f"; //change gender to female
analysis.p2.regChange="t"; //gender register to informal
analysis.p2.numChange="s"; //change number to singular
            
//rewrite with the new properties:
Syntaxist.Reinflector reinflector = new Syntaxist.Reinflector("de");
string s = reinflector.Reinflect(analysis);
Console.WriteLine(s); //"Du bist eine gute Studentin."

Another way is to skip the intial analysis and tell Syntaxist straight away how you want to rewrite certain things in the text:

Syntaxist.Reinflector reinflector = new Syntaxist.Reinflector("de");
string s = reinflector.Reinflect(
    "Sie sind ein guter Student.", //"You are a good student."
    "",    //no changes to the first person
    "fts", //change second person to "f" female, "t" informal, "s" singular
    ""     //no changes to third persons
);
Console.WriteLine(s); //"Du bist eine gute Studentin."

When used without performing analysis first, the Reinfect method performs its own analysis internally every time it is called.

Reinflection parameters

The Reinflect method, when used without performing analysis first, takes four arguments:

• text: the text you want to rewrite
• p1: instructions for how first-person references should be rewritten
• p2: instructions for how second-person references should be rewritten
• p3: instructions for how (all and any) third-person references should be rewritten

The strings passed as values of p1, p2 and p3 can be any combination of the following characters, in any order:

• s or p, to change the number to singular or plural. When neither s nor p is present in the string then it means "do not change the number".

• m or f, to change the gender to male or female, or b to combine male and female words into a gender-neutral neoform such as German Lehrer:in (male-or-female teacher) and Student:innen (male-or-female students). When neither m nor f nor b is present in the string then it means "do not change the gender".

• t or v to change the register to informal (like German du or ihr) or formal (like German Sie). This only affects the second person p2. When neither t nor v is present in the string then it means "do not change the register". Note: The codes t for informal and v for informal are based on the T-V distinction.

Note that number (s or p) and register (t or v) often work together to determine which second-person pronouns end up in the rewritten string. In German, ts means du (informal singular you), tp means ihr (informal plural you) and v means Sie (formal you, same for singular and plural numbers).

Rewriting translations

The Reinflect method, when used without performing analysis first, can optionally take two more arguments containing the source from which the text has been translated:

string s = reinflector.Reinflect(
    "Sie sind ein guter Student.",
    "", "fts", "",
    "en", "You are a good student."
);

In some cases this will help Syntaxist understand the text more accurately when analyzing it.

Rewriting texts with inline markup

The Reinflect method, when used without performing analysis first, can optionally take one more argument containing a regular expression to recognize inline markup in the text (and in its translation source, if given). The inline markup will be preserved in the rewritten string:

Syntaxist.Reinflector reinflector = new Syntaxist.Reinflector("de");
string s = reinflector.Reinflect(
    "Sie sind ein <b>sehr</b> guter Student.",
    "", "fts", "",
    "en", "You are a <b>very</b> good student.",
    "<[^>]+>"
);
Console.WriteLine(s); //"Du bist eine <b>sehr</b> gute Studentin."

Reinflection examples (German)

Here are a few more examples to demonstrate Syntaxist's rewriting capabilities on German sentences.

Reinflecting the first person (in singular)

reinflector.Reinflect(
    "Ich bin Arzt in diesem Spital.", //"I am a doctor in this hospital."
    "f", "", ""
); //"Ich bin Ärztin in diesem Spital."

reinflector.Reinflect(
    "Ich bin Arzt in diesem Spital.", //"I am a doctor in this hospital."
    "b", "", ""
); //"Ich bin Ärzt:in in diesem Spital."

Reinflecting the first person (in plural)

reinflector.Reinflect(
    "Wir sind Studenten aus Österreich.", //"We are students from Austria."
    "f", "", ""
); //"Wir sind Studentinnen aus Österreich."

reinflector.Reinflect(
    "Wir sind Studenten aus Österreich.", //"We are students from Austria."
    "b", "", ""
); //"Wir sind Student:innen aus Österreich."

Reinflecting the second person (number and register)

reinflector.Reinflect(
    "Suchen Sie Ihre Kinder?", //"Are you looking for your children?"
    "", "ts", ""
); //"Suchst du deine Kinder?"

reinflector.Reinflect(
    "Suchen Sie Ihre Kinder?", //"Are you looking for your children?"
    "", "tp", ""
); //"Sucht ihr eure Kinder?"

Reinflecting the second person (imperative)

reinflector.Reinflect(
    "Melde dich jetzt an!", //"Register now!"
    "", "v", ""
); //"Melden Sie sich jetzt an!"

reinflector.Reinflect(
    "Melde dich jetzt an!", //"Register now!"
    "", "tp", ""
); //"Meldet euch jetzt an!"

Reinflecting the second person (gender and/or register)

reinflector.Reinflect(
    "Sind Sie der neue Bundeskanzler?", //"Are you the new federal chancellor?"
    "", "f", ""
); //"Sind Sie die neue Bundeskanzlerin?"

reinflector.Reinflect(
    "Sind Sie der neue Bundeskanzler?", //"Are you the new federal chancellor?"
    "", "t", ""
); //"Bist du der neue Bundeskanzler?"

reinflector.Reinflect(
    "Sind Sie der neue Bundeskanzler?", //"Are you the new federal chancellor?"
    "", "tf", ""
); //"Bist du die neue Bundeskanzlerin?"

Reinflecting third persons (in plural)

reinflector.Reinflect(
    "Alle Studenten müssen sich registrieren.", //"All students must register."
    "", "", "f"
); //"Alle Studentinnen müssen sich registrieren."

reinflector.Reinflect(
    "Alle Studenten müssen sich registrieren.", //"All students must register."
    "", "", "b"
); //"Alle Student:innen müssen sich registrieren."

Reinflecting third persons (in singular)

reinflector.Reinflect(
    "Das habe ich dem Manager gesagt.", //"That's what I told the manager."
    "", "", "f"
); //"Das habe ich der Managerin gesagt."

reinflector.Reinflect(
    "Das habe ich dem Manager gesagt.", //"That's what I told the manager."
    "", "", "b"
); //"Das habe ich dem:der Manager:in gesagt."

Reinflecting several persons at once

reinflector.Reinflect(
    "Wie gefällt dir deine neue Lehrerin?", //"How do you like your new teacher?"
    "", "tp", "m"
); //"Wie gefällt euch euer neuer Lehrer?"

reinflector.Reinflect(
    "Kennen Sie schon unseren neuen Direktor?", //"Have you met our new director yet?"
    "", "ts", "f"
); //"Kennst du schon unsere neue Direktorin?"