AvroConvert 2.3.0

Apache Avro serializer for .NET.

There is a newer version of this package available.
See the version list below for details.
Install-Package AvroConvert -Version 2.3.0
dotnet add package AvroConvert --version 2.3.0
<PackageReference Include="AvroConvert" Version="2.3.0" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add AvroConvert --version 2.3.0
The NuGet Team does not provide support for this client. Please contact its maintainers for support.

Nuget
Github

AvroConvert

Avro format combines readability of JSON format and compression of binary data serialization.
<br></br>

The main purpose of the project was to enhance communication between microservices. Replacing JSON with Avro brought two main benefits:

  • Reduced amount of data send via HTTP by about 30%
  • Increased communication security - the data was not visible in plain JSON text

Documentation

General information: http://avro.apache.org/

Wiki: https://cwiki.apache.org/confluence/display/AVRO/Index

Why choose Avro?

Benchmark with comparison to Newtonsoft.Json:

| Converter | Compression time [ms] | Compressed size [kB] |
|-------------------------|-----------------------|----------------------|
| Json | 140 | 9945 |
| Avro (null encoding) | 235 | 2435 |
| Avro (Deflate encoding) | 144 | 206 |
| Avro (Snappy encoding) | 127 | 420 |
| Avro (Gzip encoding) | 143 | 206 |

You can find the benchmark under tests/AvroConvertTests/Benchmark directory.

Conclusion: <br>
Using Avro for communication between your services reduces up to almost 50 times (!!!) network traffic. This makes huge time and bandwidth savings.

Code samples

Serialization
 byte[] avroObject = AvroConvert.Serialize(object yourObject);

Using encoding

 byte[] avroObject = AvroConvert.Serialize(object yourObject, CodecType.Snappy);

Supported encoding types:

  • Null (default)
  • Deflate
  • Snappy
  • GZip
Deserialization
//Using generic method
CustomClass deserializedObject = AvroConvert.Deserialize<CustomClass>(byte[] avroObject);

//Using dynamic method
CustomClass deserializedObject = AvroConvert.Deserialize(byte[] avroObject, typeof(CustomClass));

Deserialization when a property value is null, but schema contains information about default value

//Model used for serialization
public class DefaultValueClass
{
    [DefaultValue("Let's go")]
    public string justSomeProperty { get; set; }

    [DefaultValue(2137)]
    public long? andLongProperty { get; set; }
}

//Deserializing object with null data
 DefaultValueClass deserializedObject = AvroConvert.Deserialize<DefaultValueClass>(byte[] avroObject);

//Produces following object:
>  deserializedObject.justSomeProperty
>  "Let's go"

>  deserializedObject.andLongProperty
>  2137
Generating Avro schema for C# classes

Using simple class


//Model
public class SimpleTestClass
{
	public string justSomeProperty { get; set; }

	public long andLongProperty { get; set; }
}


//Action
string schemaInJsonFormat = AvroConvert.GenerateSchema(typeof(SimpleTestClass));


//Produces following schema:
"{"type":"record","name":"AvroConvert.SimpleTestClass","fields":[{"name":"justSomeProperty","type":["null","string"]},{"name":"andLongProperty","type":"long"}]}"

Using class decorated with attributes

//Model
[DataContract(Name = "User", Namespace = "user")]
public class AttributeClass
{
	[DataMember(Name = "name")]
	public string StringProperty { get; set; }

	[DataMember(Name = "favorite_number")]
	[NullableSchema]
	public int? NullableIntProperty { get; set; }

	[DataMember(Name = "favorite_color")]
	public string AndAnotherString { get; set; }
}


//Action
string schemaInJsonFormat = AvroConvert.GenerateSchema(typeof(AttributeClass));


//Produces following schema:
"{"type":"record","name":"user.User","fields":[{"name":"name","type":["null","string"]},{"name":"favorite_number","type":["null","int"]},{"name":"favorite_color","type":["null","string"]}]}"
Reading Avro schema from Avro encoded object
string schemaInJsonFormat = AvroConvert.GetSchema(byte[] avroObject)

License

AvroConvert is licensed under Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0) - see License for details. For commercial purposes purchase AvroConvert on website - Xabe

Contribution

We want to improve AvroConvert as much as possible. If you have any idea, found next possible feature, optimization opportunity or better way for integration, leave a comment or pull request.

Nuget
Github

AvroConvert

Avro format combines readability of JSON format and compression of binary data serialization.
<br></br>

The main purpose of the project was to enhance communication between microservices. Replacing JSON with Avro brought two main benefits:

  • Reduced amount of data send via HTTP by about 30%
  • Increased communication security - the data was not visible in plain JSON text

Documentation

General information: http://avro.apache.org/

Wiki: https://cwiki.apache.org/confluence/display/AVRO/Index

Why choose Avro?

Benchmark with comparison to Newtonsoft.Json:

| Converter | Compression time [ms] | Compressed size [kB] |
|-------------------------|-----------------------|----------------------|
| Json | 140 | 9945 |
| Avro (null encoding) | 235 | 2435 |
| Avro (Deflate encoding) | 144 | 206 |
| Avro (Snappy encoding) | 127 | 420 |
| Avro (Gzip encoding) | 143 | 206 |

You can find the benchmark under tests/AvroConvertTests/Benchmark directory.

Conclusion: <br>
Using Avro for communication between your services reduces up to almost 50 times (!!!) network traffic. This makes huge time and bandwidth savings.

Code samples

Serialization
 byte[] avroObject = AvroConvert.Serialize(object yourObject);

Using encoding

 byte[] avroObject = AvroConvert.Serialize(object yourObject, CodecType.Snappy);

Supported encoding types:

  • Null (default)
  • Deflate
  • Snappy
  • GZip
Deserialization
//Using generic method
CustomClass deserializedObject = AvroConvert.Deserialize<CustomClass>(byte[] avroObject);

//Using dynamic method
CustomClass deserializedObject = AvroConvert.Deserialize(byte[] avroObject, typeof(CustomClass));

Deserialization when a property value is null, but schema contains information about default value

//Model used for serialization
public class DefaultValueClass
{
    [DefaultValue("Let's go")]
    public string justSomeProperty { get; set; }

    [DefaultValue(2137)]
    public long? andLongProperty { get; set; }
}

//Deserializing object with null data
 DefaultValueClass deserializedObject = AvroConvert.Deserialize<DefaultValueClass>(byte[] avroObject);

//Produces following object:
>  deserializedObject.justSomeProperty
>  "Let's go"

>  deserializedObject.andLongProperty
>  2137
Generating Avro schema for C# classes

Using simple class


//Model
public class SimpleTestClass
{
	public string justSomeProperty { get; set; }

	public long andLongProperty { get; set; }
}


//Action
string schemaInJsonFormat = AvroConvert.GenerateSchema(typeof(SimpleTestClass));


//Produces following schema:
"{"type":"record","name":"AvroConvert.SimpleTestClass","fields":[{"name":"justSomeProperty","type":["null","string"]},{"name":"andLongProperty","type":"long"}]}"

Using class decorated with attributes

//Model
[DataContract(Name = "User", Namespace = "user")]
public class AttributeClass
{
	[DataMember(Name = "name")]
	public string StringProperty { get; set; }

	[DataMember(Name = "favorite_number")]
	[NullableSchema]
	public int? NullableIntProperty { get; set; }

	[DataMember(Name = "favorite_color")]
	public string AndAnotherString { get; set; }
}


//Action
string schemaInJsonFormat = AvroConvert.GenerateSchema(typeof(AttributeClass));


//Produces following schema:
"{"type":"record","name":"user.User","fields":[{"name":"name","type":["null","string"]},{"name":"favorite_number","type":["null","int"]},{"name":"favorite_color","type":["null","string"]}]}"
Reading Avro schema from Avro encoded object
string schemaInJsonFormat = AvroConvert.GetSchema(byte[] avroObject)

License

AvroConvert is licensed under Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0) - see License for details. For commercial purposes purchase AvroConvert on website - Xabe

Contribution

We want to improve AvroConvert as much as possible. If you have any idea, found next possible feature, optimization opportunity or better way for integration, leave a comment or pull request.

NuGet packages (3)

Showing the top 3 NuGet packages that depend on AvroConvert:

Package Downloads
SolTechnology.Avro.Http
Library containing functionalities, which enable communication between microservices using Avro data format
SolTechnology.Avro.Kafka
Library containing extensions for Confluent Kafka platform
SlimMessageBus.Host.Serialization.AvroConvert
Extension to SlimMessageBus that provides Avro serialization based on the AvroConvert library

GitHub repositories

This package is not used by any popular GitHub repositories.

Version History

Version Downloads Last updated
2.7.0 2,001 12/18/2020
2.6.3 1,536 10/25/2020
2.6.2 87 10/23/2020
2.6.1 1,745 7/25/2020
2.6.0 160 7/22/2020
2.5.1 940 6/11/2020
2.5.0 470 5/15/2020
2.4.1 959 4/14/2020
2.4.0 447 3/25/2020
2.3.0 363 3/19/2020
2.2.2 579 2/7/2020
2.2.1 174 2/7/2020
2.2.0 224 1/27/2020
2.1.0 154 1/16/2020
2.0.0 153 1/4/2020
1.8.1 438 12/9/2019
1.8.0 880 8/30/2019
1.7.0 224 8/22/2019
1.6.0 244 7/22/2019
1.5.1 269 7/14/2019
1.5.0 267 7/11/2019
1.4.0 219 6/17/2019
1.3.0 261 6/4/2019
1.2.0 260 6/3/2019
1.1.0 249 5/28/2019
1.0.1 258 5/21/2019
1.0.0 418 4/24/2019
0.8.0 373 5/1/2019
0.7.0 455 4/22/2019
0.6.3 389 4/19/2019
0.6.2 403 4/19/2019
0.6.1 387 4/19/2019
0.6.0 393 4/19/2019
0.5.0 419 4/18/2019
0.3.0 501 4/11/2019
0.2.1 393 4/11/2019
0.2.0 395 4/11/2019
0.1.0 412 4/10/2019
Show less