BTDB 32.15.1

There is a newer version of this package available.
See the version list below for details.
dotnet add package BTDB --version 32.15.1                
NuGet\Install-Package BTDB -Version 32.15.1                
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="BTDB" Version="32.15.1" />                
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add BTDB --version 32.15.1                
#r "nuget: BTDB, 32.15.1"                
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install BTDB as a Cake Addin
#addin nuget:?package=BTDB&version=32.15.1

// Install BTDB as a Cake Tool
#tool nuget:?package=BTDB&version=32.15.1                

BTDB

Currently this project these parts:

  • Key Value Database
  • Wrapped Dynamic IL generation with debugging + extensions
  • IOC Container (SourceGenerator based)
  • Object Database with Relations
  • Snappy Compression
  • Event Storage
  • Bon (Binary object notation)
  • Incremental SourceGenerator for IOC Container factory generation

All code written in C# 11 and licensed under very permissive MIT license. Targeting .Net 8.0, main code has just 1 dependency (Microsoft.Extensions.Primitives). Code is tested using xUnit Framework. Used in production on Windows and Linux, on MacOS works as well. Please is you find it useful or have questions, write me e-mail boris.letocha@gmail.com so I know that it is used. It is available in Nuget http://www.nuget.org/packages/BTDB. Source code drops are Github releases.


Key Value Database

Features

  • This is Key Value store written in C# with 2 implementation old on managed heap and new on native heap (has also prefix compression).
  • It is easily embeddable.
  • One storage is just one directory.
  • It has ACID properties with MVCC.
  • At one time there could be multiple read only transactions and one read/write transaction.
  • Export/Import to stream - could be used for compaction, snapshotting
  • Automatic compaction
  • Customizable compression
  • Relatively Fast DB Open due to key index file - though it still needs to load all keys to memory
  • Inspired by Bitcask [https://github.com/basho/bitcask/blob/develop/doc/bitcask-intro.pdf]

Design limits

  • All keys data needs to fit in RAM
  • Maximum Key length is limited by 31bits (2GB).
  • Maximum value length is limited by 31bits (2GB).

Sample code

using (var fileCollection = new InMemoryFileCollection())
using (IKeyValueDB db = new KeyValueDB(fileCollection))
{
    using (var tr = db.StartTransaction())
    {
        tr.CreateOrUpdateKeyValue(new byte[] { 1 }, new byte[100000]);
        tr.Commit();
    }
}

Roadmap

  • Everything is there just use it

Wrapped Dynamic IL generation with debugging + extensions

This help you to write fluent code which generates IL code in runtime. It is used in Object Database part.

Sample code

var method = ILBuilder.Instance.NewMethod<Func<Nested>>("SampleCall");
var il = method.Generator;
var local = il.DeclareLocal(typeof(Nested), "n");
il
    .Newobj(() => new Nested())
    .Dup()
    .Stloc(local)
    .Ldstr("Test")
    .Call(() => ((Nested)null).Fun(""))
    .Ldloc(local)
    .Ret();
var action = method.Create();

Roadmap

  • Add support for all IL instructions as needed
  • Deprecate this by replacing its usages by Source Generator

Object Database

Features

  • Builds on top of Key Value Database and Reflection.Emit extensions.
  • It stores Plain .Net Objects and only their public properties with getters and setters.
  • All ACID and MVCC properties are preserved of course.
  • Automatic upgrading of model on read with dynamically generated optimal IL code.
  • Automatic versioning of model changes.
  • Enumeration of all objects
  • Each object type could store its "singleton" - very useful for root objects
  • Relations - Table with primary key and multiple secondary keys
  • By default objects are stored inline in parent object, use IIndirect for objects with Oid which will load lazily

Documentation: [https://github.com/Bobris/BTDB/blob/master/Doc/ODBDictionary.md]

Relations doc: [https://github.com/Bobris/BTDB/blob/master/Doc/Relations.md]

Sample code

public class Person
{
    public string Name { get; set; }
    public uint Age { get; set; }
}

using (var tr = _db.StartTransaction())
{
    tr.Store(new Person { Name = "Bobris", Age = 35 });
    tr.Commit();
}
using (var tr = _db.StartTransaction())
{
    var p = tr.Enumerate<Person>().First();
    Assert.AreEqual("Bobris", p.Name);
    Assert.AreEqual(35, p.Age);
}

Roadmap

  • Support more types of properties
  • Free text search (far future if ever)

Event storage

Features

  • Optimal serialization with metadata
  • Deserialization also to dynamic
  • Storage is transactional
  • As storage could be used Azure Page Blobs
  • EventStorage2 is specialized to be used with Kafka, metadata are stored in separate topic

Bon

Bon Binary object notation is allows creating and reading JavaScript/C# values with extensions like Dictionary/Map into binary notation. It is much faster to parse, write, skip, search by keys than JSON, size will be also smaller in most cases, in some cases much more smaller.


Snappy Compression

Features

  • Ported and inspired mainly by Go version of Snappy Compression [http://code.google.com/p/snappy/]
  • Fully compatible with original
  • Fully managed and safe implementation
  • Compression is aborted when target buffer size is not big enough

Roadmap

  • Some speed optimizations around Spans would help
Product Compatible and additional computed target framework versions.
.NET net8.0 is compatible.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (1)

Showing the top 1 NuGet packages that depend on BTDB:

Package Downloads
Bbcore.Lib

Package Description

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
32.16.1 0 10/11/2024
32.16.0 2,506 10/4/2024
32.15.1 725 8/29/2024
32.15.0 92 8/29/2024
32.14.1 823 8/17/2024
32.14.0 143 8/4/2024
32.13.1 27,814 7/31/2024
32.13.0 2,079 7/26/2024
32.12.1 237 7/21/2024
32.12.0 156 7/18/2024
32.11.0 3,968 7/6/2024
32.10.0 4,347 6/20/2024
32.9.0 173 6/15/2024
32.8.0 10,009 6/14/2024
32.7.1 17,479 5/15/2024
32.7.0 118 5/14/2024
32.6.2 3,658 5/6/2024
32.6.1 113 5/6/2024
32.6.0 195 5/1/2024
32.5.0 53,684 2/29/2024
32.4.29 455 2/26/2024
32.4.28 108 2/26/2024
32.4.26 15,079 2/21/2024
32.4.25 2,040 2/20/2024
32.4.24 491 2/19/2024
32.4.23 3,537 2/14/2024
32.4.22 124 2/13/2024
32.4.21 3,858 2/12/2024
32.4.20 264 2/7/2024
32.4.19 172 2/7/2024
32.4.18 254 2/1/2024
32.4.17 114 1/31/2024
32.4.16 110 1/30/2024
32.4.15 238 1/29/2024
32.4.14 116 1/29/2024
32.4.13 117 1/27/2024
32.4.12 117 1/26/2024
32.4.11 113 1/26/2024
32.4.10 104 1/26/2024
32.4.9 111 1/26/2024
32.4.8 107 1/26/2024
32.4.7 98 1/25/2024
32.4.6 109 1/25/2024
32.4.5 125 1/25/2024
32.4.4 105 1/24/2024
32.4.3 114 1/23/2024
32.4.2 106 1/23/2024
32.4.1 109 1/22/2024
32.4.0 122 1/21/2024
32.3.2 128 1/19/2024
32.3.1 114 1/18/2024
32.3.0 121 1/18/2024
32.2.0 126 1/17/2024
32.1.0 132 1/14/2024
32.0.3 144 1/8/2024
32.0.2 574 1/2/2024
32.0.1 134 1/2/2024
32.0.0 146 1/1/2024
31.15.0 9,785 1/22/2024
31.14.0 24,034 11/23/2023
31.13.0 4,378 11/21/2023
31.12.7 1,629 11/16/2023
31.12.6 248 11/16/2023
31.12.5 215 11/13/2023 31.12.5 is deprecated because it has critical bugs.
31.12.4 638 11/12/2023
31.12.3 8,906 10/30/2023
31.12.2 4,672 10/16/2023
31.12.1 5,108 10/5/2023
31.12.0 241 9/17/2023
31.11.3 30,139 8/10/2023
31.11.2 189 8/9/2023
31.11.1 170 8/8/2023
31.11.0 4,951 8/8/2023
31.10.0 1,049 8/3/2023
31.9.0 204 8/1/2023
31.8.2 2,749 8/1/2023
31.8.1 169 8/1/2023
31.8.0 25,027 6/8/2023
31.7.0 10,293 5/23/2023
31.6.0 209 5/21/2023
31.5.2 1,667 5/16/2023
31.5.1 17,633 4/13/2023
31.5.0 8,876 3/28/2023
31.4.3 3,690 3/24/2023
31.4.2 3,804 3/21/2023
31.4.1 281 3/21/2023
31.4.0 310 3/21/2023
31.3.2 17,034 2/20/2023
31.3.1 645 2/17/2023
31.3.0 312 2/15/2023
31.2.1 325 2/15/2023
31.2.0 1,995 2/15/2023
31.1.1 27,143 2/4/2023
31.1.0 359 2/3/2023
31.0.0 363 2/3/2023
30.5.0 13,909 1/10/2023
30.4.0 558 1/9/2023
30.3.0 6,774 1/3/2023
30.2.2 10,356 1/2/2023
30.2.1 408 1/2/2023
30.2.0 426 1/1/2023
30.1.0 369 12/21/2022
30.0.0 23,059 11/14/2022
29.12.1 482 11/3/2022
29.12.0 904 11/2/2022
29.11.0 519 9/7/2022
29.10.1 526 8/1/2022
29.10.0 507 8/1/2022
29.9.1 524 7/22/2022
29.9.0 541 7/17/2022
29.8.0 548 6/13/2022
29.7.0 539 6/8/2022
29.6.0 535 6/5/2022
29.5.0 554 5/18/2022
29.4.0 549 5/17/2022
29.3.0 543 5/13/2022
29.2.0 572 5/3/2022
29.1.1 8,760 3/29/2022
29.1.0 558 3/28/2022
29.0.0 588 2/28/2022
28.3.2 612 1/21/2022
28.3.1 558 1/21/2022
28.3.0 417 1/4/2022
28.2.0 426 12/19/2021
28.1.2 443 12/7/2021
28.1.1 377 12/6/2021
28.1.0 398 11/29/2021
28.0.0 419 11/17/2021
27.0.3 474 11/17/2021
27.0.2 1,679 11/7/2021
27.0.0 431 11/2/2021
26.2.1 715 5/8/2021
24.0.2 21,210 6/15/2020
24.0.1 807 6/13/2020
24.0.0 671 6/13/2020
23.1.0 835 5/23/2020
23.0.0 664 5/18/2020
22.2.2 673 5/14/2020
22.2.1 648 5/14/2020
22.2.0 627 5/13/2020
22.1.0 645 5/12/2020
22.0.0 661 5/10/2020
21.0.0 695 5/6/2020
20.3.0 735 5/2/2020
20.0.0 782 4/25/2020
19.5.0 1,233 3/10/2020
19.4.0 1,681 11/30/2019
19.3.0 1,101 11/6/2019
19.2.0 783 11/3/2019
19.1.0 969 10/13/2019
19.0.0 810 10/6/2019
18.2.2 1,088 9/19/2019
18.2.1 857 9/16/2019
18.2.0 884 9/8/2019
18.1.0 907 9/2/2019
18.0.0 792 9/1/2019
17.10.0 825 8/28/2019
17.9.0 1,986 7/24/2019
17.8.0 1,715 7/13/2019
17.7.0 1,451 6/9/2019
17.6.0 1,364 5/10/2019
17.5.2 1,388 5/9/2019
17.5.1 1,405 5/9/2019
17.5.0 1,430 4/16/2019
17.4.2 1,546 2/12/2019
17.4.1 1,473 2/6/2019
17.4.0 1,469 2/6/2019
17.3.0 1,466 1/25/2019
17.2.0 1,522 1/8/2019
17.1.0 1,447 12/24/2018
17.0.1 1,514 12/20/2018
17.0.0 1,522 12/16/2018
16.2.1 1,566 12/11/2018
14.12.2 1,685 10/15/2018
14.7.0 1,866 8/16/2018
14.6.0 1,653 8/2/2018
14.5.2 1,773 7/9/2018
14.5.1 1,735 6/20/2018
14.5.0 1,739 6/13/2018
14.4.0 1,776 6/10/2018
14.3.0 1,821 6/10/2018
14.2.1 1,766 6/7/2018
14.2.0 1,674 6/6/2018
14.1.0 1,655 6/6/2018
14.0.0 1,723 6/5/2018
13.1.0 1,755 6/3/2018
13.0.0 1,785 5/7/2018
12.7.0 1,861 4/15/2018
12.6.1 1,829 3/23/2018
12.6.0 1,837 3/18/2018
12.5.1 1,799 2/26/2018
12.5.0 1,836 2/12/2018
12.4.0 1,861 1/31/2018
12.3.0 1,794 1/29/2018
12.2.0 1,817 1/9/2018
12.1.0 1,873 12/10/2017
12.0.0 1,760 12/3/2017
11.10.1 1,938 11/20/2017
11.10.0 1,953 11/3/2017
11.9.1 1,936 11/2/2017
11.9.0 1,906 10/31/2017
11.8.1 1,902 10/30/2017
11.8.0 1,910 10/28/2017
11.7.0 1,939 10/26/2017
11.6.5 1,954 10/24/2017
11.6.4 1,957 10/23/2017
11.6.3 1,977 10/20/2017
11.6.2 1,926 10/19/2017
11.6.1 1,932 10/18/2017
11.6.0 1,933 10/15/2017
11.5.0 1,923 10/8/2017
11.4.0 2,174 9/6/2017
11.3.0 1,940 9/4/2017
11.2.0 1,748 8/20/2017
11.1.0 1,704 8/9/2017
11.0.4 1,763 7/30/2017
11.0.3 1,783 7/24/2017
11.0.2 1,883 7/21/2017
11.0.1 1,740 7/21/2017
11.0.0 1,810 7/19/2017
10.6.0 1,796 6/27/2017
10.5.2 1,859 6/23/2017
10.5.1 1,826 6/22/2017
10.5.0 1,787 6/17/2017
10.4.3 1,777 5/31/2017
10.4.2 1,746 5/25/2017
10.4.1 1,812 5/22/2017
10.4.0 1,747 5/19/2017
10.3.0 1,762 5/11/2017
10.2.0 1,769 4/21/2017
10.1.2 1,744 4/14/2017
10.1.1 1,742 4/12/2017
10.1.0 1,773 4/7/2017
10.0.2 1,729 3/28/2017
10.0.1 1,754 3/26/2017
10.0.0 1,782 3/20/2017
9.4.0 1,747 3/12/2017
9.3.1 1,771 3/9/2017
9.3.0 1,780 3/6/2017
9.2.1 1,757 2/28/2017
9.2.0 1,834 2/21/2017
9.1.1 1,773 2/15/2017
9.1.0 1,885 2/8/2017
9.0.4 1,871 1/23/2017
9.0.3 1,825 12/20/2016
9.0.2 1,756 12/18/2016
9.0.0 1,823 12/12/2016
8.0.1 1,824 12/10/2016
8.0.0 1,836 11/28/2016
7.0.0 1,822 11/23/2016
6.0.0 2,145 11/17/2016
5.7.0 2,388 11/3/2016
5.6.4 2,345 11/2/2016
5.6.3 2,047 10/30/2016
5.6.2 2,058 10/26/2016
5.6.1 2,015 10/26/2016
5.6.0 2,036 10/24/2016
5.5.0 2,062 10/22/2016
5.4.0 2,096 10/21/2016
5.3.2 2,094 10/16/2016
5.3.1 2,063 9/14/2016
5.3.0 2,121 9/13/2016
5.2.0 2,058 9/11/2016
5.1.0 2,051 9/9/2016
5.0.0 2,004 9/8/2016
4.0.1 2,023 9/8/2016
4.0.0 2,026 9/7/2016
3.6.0 2,035 9/4/2016
3.5.0 2,067 9/1/2016
3.4.0 2,100 8/8/2016
3.3.0 2,123 7/18/2016
3.2.3 2,106 6/28/2016
3.2.2 2,527 6/15/2016
3.2.1 2,088 6/7/2016
3.2.0 2,072 6/5/2016
3.1.0 2,087 5/30/2016
3.0.0 2,157 5/19/2016
2.12.0 2,202 4/10/2016
2.11.0 2,201 4/9/2016
2.10.1 1,909 3/16/2016
2.10.0 2,184 2/17/2016
2.9.1 2,184 12/13/2015
2.9.0 2,148 12/8/2015
2.8.5 2,224 11/27/2015
2.8.4 2,175 11/27/2015
2.8.2 2,165 11/4/2015
2.8.1 2,113 11/4/2015
2.8.0 2,165 11/1/2015
2.7.0 2,343 6/27/2015
2.6.3 2,209 6/16/2015
2.6.2 2,335 6/14/2015
2.6.1 2,233 5/15/2015
2.6.0 2,294 4/13/2015
2.5.3 2,316 1/26/2015
2.5.2 2,541 12/13/2014
2.5.1 2,445 11/18/2014
2.5.0 2,320 10/13/2014
2.4.5 2,261 10/6/2014
2.4.4 2,443 6/3/2014
2.4.3 2,285 6/2/2014
2.4.2 2,310 5/30/2014
2.4.0 2,647 3/14/2014
2.3.2 2,214 2/3/2014
2.3.1 2,076 12/15/2013
2.3.0 2,050 12/14/2013
2.2.2 2,159 12/12/2013
2.2.1 2,120 12/11/2013
2.2.0 2,171 12/8/2013
2.1.0 2,159 12/8/2013
2.0.2 2,216 12/2/2013
2.0.1 2,204 11/26/2013
2.0.0 2,154 11/14/2013
1.2.1 2,130 11/11/2013
1.1.0 2,189 9/5/2013
1.0.0 2,166 9/1/2013