Microsoft.Azure.Data.SchemaRegistry.ApacheAvro
1.0.1
Prefix Reserved
dotnet add package Microsoft.Azure.Data.SchemaRegistry.ApacheAvro --version 1.0.1
NuGet\Install-Package Microsoft.Azure.Data.SchemaRegistry.ApacheAvro -Version 1.0.1
<PackageReference Include="Microsoft.Azure.Data.SchemaRegistry.ApacheAvro" Version="1.0.1" />
paket add Microsoft.Azure.Data.SchemaRegistry.ApacheAvro --version 1.0.1
#r "nuget: Microsoft.Azure.Data.SchemaRegistry.ApacheAvro, 1.0.1"
// Install Microsoft.Azure.Data.SchemaRegistry.ApacheAvro as a Cake Addin #addin nuget:?package=Microsoft.Azure.Data.SchemaRegistry.ApacheAvro&version=1.0.1 // Install Microsoft.Azure.Data.SchemaRegistry.ApacheAvro as a Cake Tool #tool nuget:?package=Microsoft.Azure.Data.SchemaRegistry.ApacheAvro&version=1.0.1
Azure Schema Registry Apache Avro client library for .NET
Azure Schema Registry is a schema repository service hosted by Azure Event Hubs, providing schema storage, versioning, and management. This package provides an Avro serializer capable of serializing and deserializing payloads containing Schema Registry schema identifiers and Avro-serialized data.
Getting started
Install the package
Install the Azure Schema Registry Apache Avro library for .NET with NuGet:
dotnet add package Microsoft.Azure.Data.SchemaRegistry.ApacheAvro
Prerequisites
If you need to create an Event Hubs namespace, you can use the Azure Portal or Azure PowerShell.
You can use Azure PowerShell to create the Event Hubs namespace with the following command:
New-AzEventHubNamespace -ResourceGroupName myResourceGroup -NamespaceName namespace_name -Location eastus
Authenticate the client
In order to interact with the Azure Schema Registry service, you'll need to create an instance of the Schema Registry Client class. To create this client, you'll need Azure resource credentials and the Event Hubs namespace hostname.
Get credentials
To acquire authenticated credentials and start interacting with Azure resources, please see the quickstart guide here.
Get Event Hubs namespace hostname
The simplest way is to use the Azure portal and navigate to your Event Hubs namespace. From the Overview tab, you'll see Host name
. Copy the value from this field.
Create SchemaRegistryClient
Once you have the Azure resource credentials and the Event Hubs namespace hostname, you can create the SchemaRegistryClient. You'll also need the Azure.Identity package to create the credential.
// Create a new SchemaRegistry client using the default credential from Azure.Identity using environment variables previously set,
// including AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, and AZURE_TENANT_ID.
// For more information on Azure.Identity usage, see: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/identity/Azure.Identity/README.md
var schemaRegistryClient = new SchemaRegistryClient(fullyQualifiedNamespace: fullyQualifiedNamespace, credential: new DefaultAzureCredential());
Key concepts
Serializer
This library provides a serializer, SchemaRegistryAvroSerializer, that interacts with EventData
events. The SchemaRegistryAvroSerializer utilizes a SchemaRegistryClient to enrich the EventData
events with the schema ID for the schema used to serialize the data.
This serializer requires the Apache Avro library. The payload types accepted by this serializer include GenericRecord and ISpecificRecord.
Examples
The following shows examples of what is available through the SchemaRegistryAvroSerializer
. There are both sync and async methods available for these operations. These examples use a generated Apache Avro class Employee.cs created using this schema:
{
"type" : "record",
"namespace" : "TestSchema",
"name" : "Employee",
"fields" : [
{ "name" : "Name" , "type" : "string" },
{ "name" : "Age", "type" : "int" }
]
}
Details on generating a class using the Apache Avro library can be found in the Avro C# Documentation.
Serialize and deserialize data using the Event Hub EventData model
In order to serialize an EventData
instance with Avro information, you can do the following:
var serializer = new SchemaRegistryAvroSerializer(client, groupName, new SchemaRegistryAvroSerializerOptions { AutoRegisterSchemas = true });
var employee = new Employee { Age = 42, Name = "Caketown" };
EventData eventData = (EventData) await serializer.SerializeAsync(employee, messageType: typeof(EventData));
// the schema Id will be included as a parameter of the content type
Console.WriteLine(eventData.ContentType);
// the serialized Avro data will be stored in the EventBody
Console.WriteLine(eventData.EventBody);
// construct a publisher and publish the events to our event hub
var fullyQualifiedNamespace = "<< FULLY-QUALIFIED EVENT HUBS NAMESPACE (like something.servicebus.windows.net) >>";
var eventHubName = "<< NAME OF THE EVENT HUB >>";
var credential = new DefaultAzureCredential();
// It is recommended that you cache the Event Hubs clients for the lifetime of your
// application, closing or disposing when application ends. This example disposes
// after the immediate scope for simplicity.
await using var producer = new EventHubProducerClient(fullyQualifiedNamespace, eventHubName, credential);
await producer.SendAsync(new EventData[] { eventData });
To deserialize an EventData
event that you are consuming:
// construct a consumer and consume the event from our event hub
// It is recommended that you cache the Event Hubs clients for the lifetime of your
// application, closing or disposing when application ends. This example disposes
// after the immediate scope for simplicity.
await using var consumer = new EventHubConsumerClient(EventHubConsumerClient.DefaultConsumerGroupName, fullyQualifiedNamespace, eventHubName, credential);
await foreach (PartitionEvent receivedEvent in consumer.ReadEventsAsync())
{
Employee deserialized = (Employee) await serializer.DeserializeAsync(eventData, typeof(Employee));
Console.WriteLine(deserialized.Age);
Console.WriteLine(deserialized.Name);
break;
}
You can also use generic methods to serialize and deserialize the data. This may be more convenient if you are not building a library on top of the Avro serializer, as you won't have to worry about the virality of generics:
var serializer = new SchemaRegistryAvroSerializer(client, groupName, new SchemaRegistryAvroSerializerOptions { AutoRegisterSchemas = true });
var employee = new Employee { Age = 42, Name = "Caketown" };
EventData eventData = await serializer.SerializeAsync<EventData, Employee>(employee);
// the schema Id will be included as a parameter of the content type
Console.WriteLine(eventData.ContentType);
// the serialized Avro data will be stored in the EventBody
Console.WriteLine(eventData.EventBody);
Similarly, to deserialize:
Employee deserialized = await serializer.DeserializeAsync<Employee>(eventData);
Console.WriteLine(deserialized.Age);
Console.WriteLine(deserialized.Name);
Serialize and deserialize data using MessageContent
directly
It is also possible to serialize and deserialize using MessageContent
. Use this option if you are not integrating with any of the messaging libraries that work with MessageContent
.
var serializer = new SchemaRegistryAvroSerializer(client, groupName, new SchemaRegistryAvroSerializerOptions { AutoRegisterSchemas = true });
MessageContent content = await serializer.SerializeAsync<MessageContent, Employee>(employee);
Employee deserializedEmployee = await serializer.DeserializeAsync<Employee>(content);
Troubleshooting
If you encounter errors when communicating with the Schema Registry service, these errors will be thrown as a RequestFailedException. The serializer will only communicate with the service the first time it encounters a schema (when serializing) or a schema ID (when deserializing). Any errors related to invalid Content-Types will be thrown as a FormatException
. Errors related to invalid schemas will be thrown as an Exception
, and the InnerException
property will contain the underlying exception that was thrown from the Apache Avro library. This type of error would typically be caught during testing and should not be handled in code. Any errors related to incompatible schemas will be thrown as an Exception
with the InnerException
property set to the underlying exception from the Apache Avro library.
Next steps
See Azure Schema Registry for additional information.
Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net5.0 was computed. net5.0-windows was computed. net6.0 was computed. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
.NET Core | netcoreapp2.0 was computed. netcoreapp2.1 was computed. netcoreapp2.2 was computed. netcoreapp3.0 was computed. netcoreapp3.1 was computed. |
.NET Standard | netstandard2.0 is compatible. netstandard2.1 was computed. |
.NET Framework | net461 was computed. net462 was computed. net463 was computed. net47 was computed. net471 was computed. net472 was computed. net48 was computed. net481 was computed. |
MonoAndroid | monoandroid was computed. |
MonoMac | monomac was computed. |
MonoTouch | monotouch was computed. |
Tizen | tizen40 was computed. tizen60 was computed. |
Xamarin.iOS | xamarinios was computed. |
Xamarin.Mac | xamarinmac was computed. |
Xamarin.TVOS | xamarintvos was computed. |
Xamarin.WatchOS | xamarinwatchos was computed. |
-
.NETStandard 2.0
- Apache.Avro (>= 1.11.0)
- Azure.Core (>= 1.38.0)
- Azure.Data.SchemaRegistry (>= 1.2.0)
- Newtonsoft.Json (>= 13.0.3)
NuGet packages (3)
Showing the top 3 NuGet packages that depend on Microsoft.Azure.Data.SchemaRegistry.ApacheAvro:
Package | Downloads |
---|---|
Microsoft.Azure.Kafka.SchemaRegistry.Avro
Azure Schema Registry is a hosted schema repository service provided by Azure Event Hubs, designed to simplify schema management and data governance. |
|
Furly.Azure.EventHubs
Furly Azure Event Hubs extension |
|
EventProducer
A SDK for producing events for Cheetah |
GitHub repositories
This package is not used by any popular GitHub repositories.
Version | Downloads | Last updated |
---|---|---|
1.0.1 | 62,729 | 3/29/2024 |
1.0.0 | 274,801 | 5/12/2022 |
1.0.0-beta.8 | 3,287 | 4/6/2022 |
1.0.0-beta.7 | 32,005 | 3/11/2022 |
1.0.0-beta.6 | 443 | 2/10/2022 |
1.0.0-beta.5 | 4,313 | 1/14/2022 |
1.0.0-beta.4 | 18,823 | 11/12/2021 |
1.0.0-beta.3 | 8,837 | 10/6/2021 |
1.0.0-beta.2 | 2,514 | 8/17/2021 |
1.0.0-beta.1 | 93,042 | 9/9/2020 |