DeltaLake 0.1.0-rc.8
dotnet add package DeltaLake --version 0.1.0-rc.8
NuGet\Install-Package DeltaLake -Version 0.1.0-rc.8
<PackageReference Include="DeltaLake" Version="0.1.0-rc.8" />
paket add DeltaLake --version 0.1.0-rc.8
#r "nuget: DeltaLake, 0.1.0-rc.8"
// Install DeltaLake as a Cake Addin #addin nuget:?package=DeltaLake&version=0.1.0-rc.8&prerelease // Install DeltaLake as a Cake Tool #tool nuget:?package=DeltaLake&version=0.1.0-rc.8&prerelease
<p align="center"> <img src="https://raw.githubusercontent.com/johnsusi/delta-net/main/docs/logo.svg" alt="delta-net logo" height="200"> </p> <p align="center"> A dotnet library for Delta Lake. </p>
Introduction
This project uses Delta Lake, an open-source storage layer that brings ACID (Atomicity, Consistency, Isolation, Durability) transactions to big data workloads.
Delta Lake provides the ability to perform batch and streaming workloads on a single platform with high reliability and performance. It offers schema enforcement and evolution, ensuring data integrity. It also provides a full historical audit trail of all the changes made to the data.
Getting Started
Install the package
dotnet add package DeltaLake
Usage
Reading a table
using DeltaLake;
var table = new DeltaTable.Builder()
.WithFileSystem("file:///path/to/table")
.Build();
await foreach (var batch in table.GetRecordBatches())
{
Console.WriteLine(batch);
}
Reading a typed table
using Apache.Arrow;
using Apache.Arrow.Types;
using DeltaLake;
record FooTable(int Id, string? Value) : ITable<FooTable>
{
public static Schema Schema { get; } = new([
new("id", Int32Type.Default, false, []),
new("value", StringType.Default, true, [])
], []);
public static IEnumerable<FooTable> Enumerate(RecordBatch batch)
{
for (var i = 0; i < batch.Length; i++)
{
var idArray = batch.Column(0) as IReadOnlyList<int?> ?? throw new Exception("Expected non-null array");
var valueArray = batch.Column(1) as IReadOnlyList<string?> ?? throw new Exception("Expected non-null array");
yield return new FooTable(idArray[i] ?? throw new Exception("Cannot be null"), valueArray[i]);
}
}
var table = new DeltaTable<FooTable>.Builder()
.WithFileSystem("file:///path/to/table")
.Build();
await foreach (var row in table.ReadAll())
{
Console.WriteLine($"Id: {row.Id}, Value: {row.Value}");
}
Create a table
using DeltaLake;
var table = new DeltaTable.Builder()
.WithFileSystem("file:///path/to/table")
.WithSchema(schema)
.EnsureCreated()
.Build();
Update a table
using DeltaLake;
var table = ...;
using var data = new RecordBatch(table.Schema, [
new Int32Array
.Builder()
.Append(1)
.Append(2)
.Append(3)
.Build(),
new StringArray
.Builder()
.Append("one")
.AppendNull()
.Append("two")
.Build(),
], 3);
table = new DeltaTable.Builder()
.FromTable(table)
.Add(data)
.Build();
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net8.0 is compatible. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
-
net8.0
- Apache.Arrow (>= 15.0.2)
- ParquetSharp (>= 15.0.2-beta2)
NuGet packages (1)
Showing the top 1 NuGet packages that depend on DeltaLake:
Package | Downloads |
---|---|
DeltaLake.Azure
A native .NET library for Delta Lake in Azure Data Lake |
GitHub repositories
This package is not used by any popular GitHub repositories.
Version | Downloads | Last updated |
---|---|---|
0.1.0-rc.8 | 490 | 7/3/2024 |
0.1.0-rc.7 | 297 | 4/25/2024 |
0.1.0-rc.6 | 93 | 4/24/2024 |
0.1.0-rc.5 | 155 | 4/21/2024 |
0.1.0-rc.4 | 59 | 4/21/2024 |
0.1.0-rc.3 | 64 | 4/21/2024 |
0.1.0-rc.2 | 96 | 4/11/2024 |
0.1.0-rc.1 | 72 | 4/11/2024 |
0.1.0-alpha.2 | 74 | 3/23/2024 |