DeltaLake 0.1.0-rc.8

This is a prerelease version of DeltaLake.
dotnet add package DeltaLake --version 0.1.0-rc.8                
NuGet\Install-Package DeltaLake -Version 0.1.0-rc.8                
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="DeltaLake" Version="0.1.0-rc.8" />                
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add DeltaLake --version 0.1.0-rc.8                
#r "nuget: DeltaLake, 0.1.0-rc.8"                
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install DeltaLake as a Cake Addin
#addin nuget:?package=DeltaLake&version=0.1.0-rc.8&prerelease

// Install DeltaLake as a Cake Tool
#tool nuget:?package=DeltaLake&version=0.1.0-rc.8&prerelease                

<p align="center"> <img src="https://raw.githubusercontent.com/johnsusi/delta-net/main/docs/logo.svg" alt="delta-net logo" height="200"> </p> <p align="center"> A dotnet library for Delta Lake. </p>

Introduction

This project uses Delta Lake, an open-source storage layer that brings ACID (Atomicity, Consistency, Isolation, Durability) transactions to big data workloads.

Delta Lake provides the ability to perform batch and streaming workloads on a single platform with high reliability and performance. It offers schema enforcement and evolution, ensuring data integrity. It also provides a full historical audit trail of all the changes made to the data.

Getting Started

Install the package

dotnet add package DeltaLake

Usage

Reading a table

using DeltaLake;

var table = new DeltaTable.Builder()
    .WithFileSystem("file:///path/to/table")
    .Build();

await foreach (var batch in table.GetRecordBatches())
{
    Console.WriteLine(batch);
}

Reading a typed table

using Apache.Arrow;
using Apache.Arrow.Types;
using DeltaLake;

record FooTable(int Id, string? Value) : ITable<FooTable>
{
    public static Schema Schema { get; } = new([
        new("id", Int32Type.Default, false, []),
        new("value", StringType.Default, true, [])
    ], []);

    public static IEnumerable<FooTable> Enumerate(RecordBatch batch)
    {
        for (var i = 0; i < batch.Length; i++)
        {
            var idArray = batch.Column(0) as IReadOnlyList<int?> ?? throw new Exception("Expected non-null array");
            var valueArray = batch.Column(1) as IReadOnlyList<string?> ?? throw new Exception("Expected non-null array");
            yield return new FooTable(idArray[i] ?? throw new Exception("Cannot be null"), valueArray[i]);
        }
    }

var table = new DeltaTable<FooTable>.Builder()
    .WithFileSystem("file:///path/to/table")
    .Build();

await foreach (var row in table.ReadAll())
{
    Console.WriteLine($"Id: {row.Id}, Value: {row.Value}");
}



Create a table

using DeltaLake;

var table = new DeltaTable.Builder()
    .WithFileSystem("file:///path/to/table")
    .WithSchema(schema)
    .EnsureCreated()
    .Build();

Update a table

using DeltaLake;

var table = ...;

using var data = new RecordBatch(table.Schema, [
    new Int32Array
        .Builder()
        .Append(1)
        .Append(2)
        .Append(3)
        .Build(),
    new StringArray
        .Builder()
        .Append("one")
        .AppendNull()
        .Append("two")
        .Build(),
], 3);

table = new DeltaTable.Builder()
    .FromTable(table)
    .Add(data)
    .Build();

Product Compatible and additional computed target framework versions.
.NET net8.0 is compatible.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (1)

Showing the top 1 NuGet packages that depend on DeltaLake:

Package Downloads
DeltaLake.Azure

A native .NET library for Delta Lake in Azure Data Lake

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
0.1.0-rc.8 490 7/3/2024
0.1.0-rc.7 297 4/25/2024
0.1.0-rc.6 93 4/24/2024
0.1.0-rc.5 155 4/21/2024
0.1.0-rc.4 59 4/21/2024
0.1.0-rc.3 64 4/21/2024
0.1.0-rc.2 96 4/11/2024
0.1.0-rc.1 72 4/11/2024
0.1.0-alpha.2 74 3/23/2024