TDG.IO 0.2.1

TestDataGenerator

An utility capable of producing simple datasets with test data with accordance to user-provided patterns and settings. Datasets can be afterwards exported to JSON and CSV files as well as to database table.

Source code release:

Windows app release:

NuGet package available here:

Build and test:

Code coverage:

Quick start guide

1. Basic usage scenario
2. Column and ColumnSettings
3. Value generators
4. ColumnSet
5. DataSet
6. Exporting dataset

Basic usage scenario <a name="basic-usage-scenario"></a>

1. Define data columns
2. Setup columns
3. Combine columns into columnset
4. Create dataset
5. Generate data

These steps can be illustrated in a sample console application code below:

using System;
using System.Globalization;
using System.Linq;
using TestDataGenerator.Abstractions.Interfaces.Rows;
using TestDataGenerator.Core;
using TestDataGenerator.Core.Columns;
using TestDataGenerator.Core.ValueGenerators;
using TestDataGenerator.Core.ValueGenerators.ValueRangeGenerator;

namespace TestDataGeneratorSample
{
    class Program
    {
        static void Main(string[] args)
        {
            // Set up first column
            Column integerColumn = new Column(new ColumnSettings()
            {
                ColumnName = "Column1",
                Culture = CultureInfo.InvariantCulture,
                TargetValueType = typeof(int),
                ValueSupplier = new ValueRangeValueGenerator(new ValueRange(0, 100))
            });

            // Set up second column
            string[] stringValuesToSelectFrom = new string[] {
                "Tom", "Matthew", "Earl", "Tony", "Jane", "Pam"
            };
            Column stringColumn = new Column(new ColumnSettings()
            {
                ColumnName = "Column2",
                Culture = CultureInfo.InvariantCulture,
                TargetValueType = typeof(string),
                ValueSupplier = new SelectFromListValueGenerator(stringValuesToSelectFrom.Cast<object>())
            });

            // Combine columns into column set
            ColumnSet colSet = new ColumnSet(new Column[] { integerColumn, stringColumn });

            // Create dataset
            TestDataSet dset = new TestDataSet("MyFirstDataset", colSet);

            // Check if dataset is ready to generate some data for you (optional)
            bool isDatasetReady = dset.CanGenerateDataSet(out string err);

            // If everything is ok, generate 20 rows of data
            if (isDatasetReady)
            {
                IRow[] resultRows = dset.GenerateNewDataSet(20);

                // Print data to console window
                Console.WriteLine(string.Join("\t", dset.GetColumns().Select(col => col.GetColumnName())));
                foreach (var row in resultRows)
                    Console.WriteLine(string.Join("\t", row));
            }
        }
    }
}

One of possible outputs of the program above:

Column1 Column2
18      Tom
13      Tony
31      Tony
25      Jane
60      Tony
19      Tom
62      Tony
48      Tony
81      Tony
87      Tony
90      Tony
18      Tony
82      Tom
21      Earl
33      Tony
98      Pam
17      Tom
92      Jane
28      Tony
100     Tony

Column and ColumnSettings <a name="column-and-column-settings"></a>

Column is basic entity which represents a single... well, column in a dataset. Despite the fact that column is supposed to be part of dataset, it still can be used on its own to generate a one-dimensional array of objects using provided ColumnSettings:

// setting up the column
Column exampleColumn = new Column(new ColumnSettings() { 
    ColumnName = "Column1", // name
    TargetValueType = typeof(string), // desired output values type
    Culture = CultureInfo.InvariantCulture, // input/output formats of values
    ValueSupplier = new SelectFromListValueGenerator(
        new object[] { 1.15, new DateTime(1990, 1, 1), true, "somestring" } // array of items to select from
        )
});

// generating 15 values
const int valuesToGenerate = 15;
object[] generatedValues = new object[valuesToGenerate];
for (int i = 0; i < valuesToGenerate; i++)
    generatedValues[i] = exampleColumn.GenerateValue(); // generate single value
            
// printing result to console
Console.Write(string.Join("\r\n", generatedValues));

Example of program output:

True
True
01.01.1990 0:00:00
1,15
somestring
somestring
01.01.1990 0:00:00
somestring
True
01.01.1990 0:00:00
True
1,15
1,15
1,15
True

Currently, ColumnSettings has following parameters:

• ColumnName. Can be arbitrary one.
• TargetValueType. Desired output value type. At moment, following types are supported: <strong>int, double, string, bool, DateTime</strong>.
• Culture. Input and output format for values.
• ValueSupplier. One of value generators, please refer to Value generators section

Value generators <a name="value-generators"></a>

Value generators are classes which implement IValueGenerator interface. This makes them capable of producing random values using specific logic describe in GenerateValue() method. For now, three variations of value generators are implemented:

• ValuePatternParser. This value generator returns values based on Regex-like input strings. ValuePatternParser exposes bool PatternIsValid property which defines, whether provided string pattern correct or not. Here are some usage examples:

// will generate integer values from 1 to 99
Column intColumn = new Column(new ColumnSettings() { 
    ColumnName = "IntCol",
    Culture = CultureInfo.InvariantCulture,
    TargetValueType = typeof(int),
    ValueSupplier = new ValuePatternParser("[1-9][0-9]{0-1}")
});

// [1-9] tells generator to get one digit from 1 to 9
// [0-9]{0-1} tells to pick either no or one value from range 0 to 9

// will generate double values within 0.00-0.99 range
Column doubleColumn = new Column(new ColumnSettings() { 
    ColumnName = "DoubleCol",
    Culture = CultureInfo.InvariantCulture,
    TargetValueType = typeof(double),
    ValueSupplier = new ValuePatternParser("[0][.][0-9]{2}")
});

// [0] tells always to put 0 digit first
// [.] always sets '.' in second position
// [0-9]{2} makes generator to select exactly two digits from 0-9 range

// generates string like vehicle registration number: 'XX 11 1111'
Column strColumn = new Column(new ColumnSettings() { 
    ColumnName = "StrCol",
    Culture = CultureInfo.InvariantCulture,
    TargetValueType = typeof(string),
    ValueSupplier = new ValuePatternParser("[A-Z]{2}[ ][0-9]{2}[ ][0-9]{3}")
});

// Possible output:
// HQ 27 827
// JE 16 039
// RL 16 204
// YI 05 050
// ZM 05 454
// EP 25 150
// WD 90 999

• SelectFromListValueGenerator. Select random value from supplied object[] array. Usage example:

// randomly select from supplied assortment of values and convert thos values to string type
Column fromListSelectColumn = new Column(new ColumnSettings() { 
    ColumnName = "TestColumn",
    Culture = CultureInfo.InvariantCulture,
    TargetValueType = typeof(string),
    ValueSupplier = new SelectFromListValueGenerator(
        new object[] { 1.123, 41, 53.1, true, "str1", "str2" }
        )
});

• ValueRangeValueGenerator. Select any value between provided range of <strong>int, double or DateTime</strong>. Code example:

// selects integer from 0 to 1000
Column fromListSelectColumn = new Column(new ColumnSettings() { 
    ColumnName = "TestColumn",
    Culture = CultureInfo.InvariantCulture,
    TargetValueType = typeof(int),
    ValueSupplier = new ValueRangeValueGenerator(
        new ValueRange(0, 1000)
        )
});

• IncrementalValueGenerator. Set starting value, maximum value and value step to get items with regular step increase (or decrease). Available for <strong>int, double and DateTime</strong>. Beware not to exceed upper bound since this thing is not yet controlled by API. Example:

// get values from 0 to 10 with regular step of 0.25
Column exampleColumn = new Column(new ColumnSettings()
{
    ColumnName = "Column1",
    TargetValueType = typeof(double),
    Culture = CultureInfo.InvariantCulture,
    ValueSupplier = new IncrementalValueGenerator(new ValueIncrement(0.00, 10.00, 0.25))
    // beware that maximum amount of generatable values here is (10.00 - 0.00) / 0.25 = 40 + 1
});

ColumnSet <a name="columnset"></a>

ColumnSet class unites several columns into one entity. This makes possible to generate rows of values using GetNewRow() method:

// setup columns
Column col1 = new Column(new ColumnSettings() { 
    ColumnName = "col1",
    Culture = CultureInfo.InvariantCulture,
    TargetValueType = typeof(string),
    ValueSupplier = new ValuePatternParser("[A-z]{5}")
});
Column col2 = new Column(new ColumnSettings()
{
    ColumnName = "col2",
    Culture = CultureInfo.InvariantCulture,
    TargetValueType = typeof(int),
    ValueSupplier = new ValuePatternParser("[0-9]{5}")
});

// create columnset
Column[] columns = new Column[] { col1, col2 };
ColumnSet colSet = new ColumnSet(columns);

// generate row
object[] rowOfValues = colSet.GetNewRow();

// print out
Console.WriteLine(string.Join("; ", rowOfValues)); // TwsYB; 53674

ColumnSet can be serialized to JSON (and deserialized as well) in case of need for persistent storage of designed columns and their value producers:

// create columnset
Column[] columns = new Column[] { col1, col2 };
ColumnSet colSet = new ColumnSet(columns);

// serialize (and save to file, if needed)
string jsonSerializedColumnSet = colSet.SerializeAsJson();

// restore JSON back to ColumnSet instance
ColumnSet deserializedFromJson = ColumnSet.DeserializeFromJson(jsonSerializedColumnSet);

DataSet <a name="dataset"></a>

TestDataSet is next abstraction layer which consumes instance of IColumnSet interface. Dataset exposes GenerateNewDataSet() method which allows to generate certain number of rows of values and store them inside dataset instance. Usage example:

// setup columns
Column col1 = new Column(new ColumnSettings() { 
    ColumnName = "col1",
    Culture = CultureInfo.InvariantCulture,
    TargetValueType = typeof(string),
    ValueSupplier = new ValuePatternParser("[A-z]{5}")
});
Column col2 = new Column(new ColumnSettings()
{
    ColumnName = "col2",
    Culture = CultureInfo.InvariantCulture,
    TargetValueType = typeof(int),
    ValueSupplier = new ValuePatternParser("[0-9]{5}")
});

// create columnset
Column[] columns = new Column[] { col1, col2 };
ColumnSet colSet = new ColumnSet(columns);

// create dataset
TestDataSet dset = new TestDataSet("datasetName", colSet);
dset.GenerateNewDataSet(20); // generate 20 rows

// print dataset row by row
foreach (IRow row in dset.GetExistingDataSet())
    Console.WriteLine(string.Join(", ", row));

Exporting dataset <a name="exporting-dataset"></a>

There are three possible deisgned ways to export data to some sort of persistent storage. Writer classes are available in TestDataGenerator.IO namespace.

Export to JSON

JSON writer is not configureable at the moment. All you need is to specify path to output file and dataset with data:

// create dataset
TestDataSet dset = new TestDataSet("datasetName", colSet);

// create json writer
JsonDataSetWriter jsonWriter = new JsonDataSetWriter();
// write dataset to file
jsonWriter.WriteToFileAsync("dataset.json", dset);

Export to CSV

CSV writer can be configured with CultureInfo instance. Also newline sequence can be specified using exposed NewLineSequence property. Usage:

// create dataset
TestDataSet dset = new TestDataSet("datasetName", colSet);

// create csv writer with provided culture
CsvDataSetWriter csvWriter = new CsvDataSetWriter(CultureInfo.InvariantCulture);
// export to csv
csvWriter.WriteToFileAsync("dataset.csv", dset);

Export to database

Dataset can be exported to MS SQL Server or PostgreSQL database using DbDataSetWriter class with predefined implementations of IDbExportExecutor interface. Thus, DbDataSetWriter consumes instance of IDbExportExecutor and ITestDataSet to write data to database:

// setup columns
Column col1 = new Column(new ColumnSettings()
{
    ColumnName = "col1",
    Culture = CultureInfo.InvariantCulture,
    TargetValueType = typeof(string),
    ValueSupplier = new ValuePatternParser("[A-z]{5}")
});
Column col2 = new Column(new ColumnSettings()
{
    ColumnName = "col2",
    Culture = CultureInfo.InvariantCulture,
    TargetValueType = typeof(int),
    ValueSupplier = new ValuePatternParser("[0-9]{5}")
});

// create columnset
Column[] columns = new Column[] { col1, col2 };
ColumnSet colSet = new ColumnSet(columns);

// create dataset
TestDataSet dset = new TestDataSet("datasetName", colSet);
dset.GenerateNewDataSet(20); // generate 20 rows

// create db writer instance
DbDataSetWriter writer = new DbDataSetWriter();

// create SQL Server export executor
SqlServerExportExecutor sqlServerExecutor = new SqlServerExportExecutor(
        "Server=localhost;Database=TestDb;Integrated Security=true;", "TestTable"
    );

// or PostgreSQL export executor
PostgresExportExecutor postgresExecutor = new PostgresExportExecutor(
        "User ID=TstUser;Password=1234567890;Host=localhost;Port=5432;Database=TestDb", "test_table"
    );

// export using either executor
writer.WriteToDatabase(sqlServerExecutor, dset);
writer.WriteToDatabase(postgresExecutor, dset);