Microsoft.Azure.DurableTask.Netherite.AzureFunctions
2.0.0
Prefix Reserved
See the version list below for details.
dotnet add package Microsoft.Azure.DurableTask.Netherite.AzureFunctions --version 2.0.0
NuGet\Install-Package Microsoft.Azure.DurableTask.Netherite.AzureFunctions -Version 2.0.0
<PackageReference Include="Microsoft.Azure.DurableTask.Netherite.AzureFunctions" Version="2.0.0" />
paket add Microsoft.Azure.DurableTask.Netherite.AzureFunctions --version 2.0.0
#r "nuget: Microsoft.Azure.DurableTask.Netherite.AzureFunctions, 2.0.0"
// Install Microsoft.Azure.DurableTask.Netherite.AzureFunctions as a Cake Addin #addin nuget:?package=Microsoft.Azure.DurableTask.Netherite.AzureFunctions&version=2.0.0 // Install Microsoft.Azure.DurableTask.Netherite.AzureFunctions as a Cake Tool #tool nuget:?package=Microsoft.Azure.DurableTask.Netherite.AzureFunctions&version=2.0.0
<img align="right" src="src/DurableTask.Netherite/icon.png"/>
Netherite
Netherite is a distributed workflow execution engine for Durable Functions (DF) and the Durable Task Framework (DTFx).
It is of potential interest to anyone developing applications on those platforms who has an appetite for performance, scalability, and reliability.
As Netherite is intended to be a drop-in backend replacement, it does not modify the application API. Existing DF and DTFx applications can switch to this backend with little effort. However, we do not support migrating existing task hub contents between different backends.
Getting Started
To get started, you can either try out the sample, or take an existing DF app and switch it to the Netherite backend. You can also read our documentation.
The hello sample.
For a comprehensive quick start on using Netherite with Durable Functions, take a look at hello sample walkthrough, and the associated video content. We included several scripts that make it easy to build, run, and deploy this application, both locally and in the cloud. Also, this sample is a great starting point for creating your own projects.
Configure an existing Durable Functions app for Netherite.
If you have a .NET Durable Functions application already, and want to configure it to use Netherite as the backend, do the following:
- Add the NuGet package
Microsoft.Azure.DurableTask.Netherite.AzureFunctions
to your functions project (if using .NET) or your extensions project (if using TypeScript or Python). - Add
"type" : "Netherite"
to thestorageProvider
section of your host.json. See recommended host.json settings. - Configure your function app to run on 64 bit, if not already the case. You can do this in the Azure portal, or using the Azure CLI. Netherite does not run on 32 bit.
- Create an EventHubs namespace. You can do this in the Azure portal, or using the Azure CLI.
- Configure
EventHubsConnection
with the connection string for the Event Hubs namespace. You can do this using an environment variable, or with a function app configuration settings.
For more information, see the .NET sample, the Python sample, or the TypeScript sample.
Configure an existing Durable Task Application for Netherite.
If you have an application that uses the Durable Task Framework already, and want to configure it to use Netherite as the backend, do the following:
- Create an EventHubs namespace. You can do this in the Azure portal, or using the Azure CLI.
- Add the NuGet package
Microsoft.Azure.DurableTask.Netherite
to your project. - Update the server startup code to construct a
NetheriteOrchestrationService
object with the required settings, and then pass it as an argument to the constructors ofTaskHubClient
andTaskHubWorker
.
For more information, see the DTFx sample.
Why a new engine?
The default Azure Storage engine stores messages in Azure Storage queues and instance states in Azure Storage tables. It executes large numbers of small storage accesses. For example, executing a single orchestration with three activities may require a total of 4 dequeue operations, 3 enqueue operations, 4 table reads, and 4 table writes. Thus, the overall throughput quickly becomes limited by how many I/O operations Azure Storage allows per second.
To achieve better performance, Netherite represents queues and partition states differently, to improve batching:
- Partitions communicate via ordered streams, using EventHubs.
- The state of a partition is stored using a combination of an immutable log and checkpoints, in Azure PageBlobs.
To learn more about the Netherite architecture, our VLDB 2022 paper is the best reference. There is also an earlier preprint paper on arXiv.
For some other considerations about how to choose the engine, see the documentation.
Status
The current version of Netherite is 2.0.0. Netherite supports almost all of the DT and DF APIs.
Some notable differences to the default Azure Table storage provider include:
- Instance queries and purge requests are not issued directly against Azure Storage, but are processed by the function app. Thus, the performance (latency and throughput) of queries heavily depends on the current scale status and load of the function app. In particular, queries do not work if the function app is stopped, and may experience cold start symptoms on consumption plans.
- Scale out of activities (not just orchestrations) is limited by the partition count configuration setting, which defaults to 12. If you need to scale out beyond 12 workers, you should increase it prior to starting you application (it cannot be changed after the task hub has been created).
- The rewind feature is not available on Netherite.
Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repositories using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
Security
Microsoft takes the security of our software products and services seriously, which includes Microsoft, Azure, DotNet, AspNet, Xamarin, and our GitHub organizations.
If you believe you have found a security vulnerability in any Microsoft-owned repository that meets Microsoft's Microsoft's definition of a security vulnerability, please report it to us at the Microsoft Security Response Center (MSRC) at https://msrc.microsoft.com/create-report. Do not report security vulnerabilities through GitHub issues.
Trademarks
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net5.0 was computed. net5.0-windows was computed. net6.0 was computed. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
.NET Core | netcoreapp2.0 was computed. netcoreapp2.1 was computed. netcoreapp2.2 was computed. netcoreapp3.0 was computed. netcoreapp3.1 is compatible. |
.NET Standard | netstandard2.0 is compatible. netstandard2.1 is compatible. |
.NET Framework | net461 was computed. net462 was computed. net463 was computed. net47 was computed. net471 was computed. net472 was computed. net48 was computed. net481 was computed. |
MonoAndroid | monoandroid was computed. |
MonoMac | monomac was computed. |
MonoTouch | monotouch was computed. |
Tizen | tizen40 was computed. tizen60 was computed. |
Xamarin.iOS | xamarinios was computed. |
Xamarin.Mac | xamarinmac was computed. |
Xamarin.TVOS | xamarintvos was computed. |
Xamarin.WatchOS | xamarinwatchos was computed. |
-
.NETCoreApp 3.1
- Microsoft.Azure.DurableTask.Core (>= 2.17.1)
- Microsoft.Azure.DurableTask.Netherite (>= 2.0.0)
- Microsoft.Azure.WebJobs.Extensions.DurableTask (>= 2.13.6)
-
.NETStandard 2.0
- Microsoft.Azure.DurableTask.Core (>= 2.17.1)
- Microsoft.Azure.DurableTask.Netherite (>= 2.0.0)
- Microsoft.Azure.WebJobs.Extensions.DurableTask (>= 2.13.6)
-
.NETStandard 2.1
- Microsoft.Azure.DurableTask.Core (>= 2.17.1)
- Microsoft.Azure.DurableTask.Netherite (>= 2.0.0)
- Microsoft.Azure.WebJobs.Extensions.DurableTask (>= 2.13.6)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories (2)
Showing the top 2 popular GitHub repositories that depend on Microsoft.Azure.DurableTask.Netherite.AzureFunctions:
Repository | Stars |
---|---|
Azure/azure-functions-durable-extension
Durable Task Framework extension for Azure Functions
|
|
microsoft/durabletask-netherite
A new engine for Durable Functions. https://microsoft.github.io/durabletask-netherite
|
Version | Downloads | Last updated |
---|---|---|
2.1.0 | 965 | 10/31/2024 |
2.0.0 | 2,469 | 10/3/2024 |
1.5.4 | 6,116 | 7/31/2024 |
1.5.3 | 6,450 | 6/6/2024 |
1.5.1 | 1,227 | 4/30/2024 |
1.5.0 | 3,173 | 4/23/2024 |
1.4.3 | 422 | 4/17/2024 |
1.4.2 | 5,579 | 2/16/2024 |
1.4.1 | 17,730 | 10/24/2023 |
1.4.0 | 9,375 | 7/8/2023 |
1.3.5 | 8,976 | 5/4/2023 |
1.3.4 | 802 | 4/12/2023 |
1.3.3 | 4,780 | 3/17/2023 |
1.3.2 | 2,683 | 3/9/2023 |
1.3.1 | 2,520 | 1/24/2023 |
1.3.0 | 4,685 | 1/3/2023 |
1.2.0 | 1,476 | 11/17/2022 |
1.1.1 | 2,601 | 8/5/2022 |
1.1.0 | 651 | 8/2/2022 |
1.0.3 | 2,228 | 5/10/2022 |
1.0.2 | 907 | 4/22/2022 |
1.0.1 | 886 | 4/18/2022 |
1.0.0 | 771 | 4/14/2022 |
1.0.0-rc1 | 369 | 4/8/2022 |
0.7.0-beta | 423 | 1/12/2022 |
0.6.1-alpha | 322 | 12/9/2021 |
0.6.0-alpha | 468 | 11/17/2021 |
0.5.0-alpha | 644 | 8/13/2021 |
0.4.0-alpha | 394 | 6/24/2021 |
0.3.0-alpha | 587 | 4/1/2021 |
0.2.0-alpha | 386 | 2/5/2021 |
0.1.0-alpha | 492 | 12/16/2020 |