EasyCompressor.Snappy
2.0.2
This package will no longer be maintained. Use EasyCompressor.Snappier, instead.
dotnet add package EasyCompressor.Snappy --version 2.0.2
NuGet\Install-Package EasyCompressor.Snappy -Version 2.0.2
<PackageReference Include="EasyCompressor.Snappy" Version="2.0.2" />
paket add EasyCompressor.Snappy --version 2.0.2
#r "nuget: EasyCompressor.Snappy, 2.0.2"
// Install EasyCompressor.Snappy as a Cake Addin #addin nuget:?package=EasyCompressor.Snappy&version=2.0.2 // Install EasyCompressor.Snappy as a Cake Tool #tool nuget:?package=EasyCompressor.Snappy&version=2.0.2
EasyCompressor
<img src="/src/EasyCompressor.png" width="100" height="100" align="left"/> An Easy-to-Use and Optimized compression library for .NET that unified several compression algorithms including LZ4, Snappy, Zstd, LZMA, Brotli, GZip, ZLib, and Deflate.
Along with a great Performance Benchmark between different compression algorithms.
This library aids in Improving Performance by Reducing Memory Usage and Bandwidth Usage. (see How)
Usage
- Compress your BLOB data for Archiving and Saving the Storage (on average from 30% to 90%)
- Compress your Caching objects for Saving the Memory Usage (it also has a nice integration with EasyCaching)
- Reduce the Bandwidth Usage of your network by reducing the volume of data sent or received. (see How)
- Improve the Performance of your I/O Operations like Service-to-Service Communication. (see How)
Features
- Offers a range of compression algorithms, including LZ4, Snappy, Zstd, LZMA, Brotli, GZip, ZLib, and Deflate.
- Support for async/await operations with support of CancellationToken.
- Stream operations are fully supported.
- Optimized and implemented with a focus on performance.
Note
The default comparison levels are carefully configured based on extensive benchmarking to ensure the highest level of efficiency and speed at a reasonable compression ratio.
Improving Data Transfer Speed by Sending/Receiving Less
Compression/Decompression has overhead but it reduces the size of your data, which can potentially result in faster transfer times, even when accounting for the additional time required for compression and decompression.
When a file is compressed, it becomes smaller in size, which means it requires less bandwidth to transfer. If the compression ratio is significant, the reduced file size can compensate for the extra time needed for compression and decompression.
For example, let's say you have an uncompressed file that takes 10 seconds to transfer. If you compress this file using a fast compressor like LZ4, Snappy, or Zstd, the compression time might be around 1 second. However, the compressed file size is significantly smaller, let's say it's only 20% of the original size. Now, when you transfer the compressed file, it will only take 2 seconds (20% of the original transfer time). In this scenario, the total time (compression time + transfer time) would be 3 seconds (1 second for compression + 2 seconds for transfer), which is less than the original 10 seconds it would have taken to transfer the uncompressed file.
It's important to note that the actual time savings will depend on various factors, such as the compression ratio achieved, the speed of the compression and decompression algorithms, the network bandwidth, and other system-specific considerations. However, with fast compressors like LZ4, Snappy, or Zstd and significant compression ratios, it is possible to achieve overall time savings when transferring compressed files compared to transferring uncompressed files.
Nuget Packages
Package | Description |
---|---|
EasyCompressor | Including Algorithms :<br/>- Brotli (Highest compression ratio - the Smallest size) (Only available in .NETCoreApp2.1, .NETStandard2.1 and above)<br/>- GZip<br/>- Deflate<br/>- ZLib (Only available in .NET6.0 and above) |
EasyCompressor.LZ4⭐️ | Algorithm: LZ4<br/>Extremely Fast (Recommended - see Benchmarks) |
EasyCompressor.Snappier⭐️ | Algorithm: Snappy<br/>Extremely Fast (Recommended - see Benchmarks) |
EasyCompressor.ZstdSharp⭐️ | Algorithm: Zstd (Zstandard)<br/>Extremely Fast (Recommended - see Benchmarks) |
EasyCompressor.LZMA | Algorithm: LZMA<br/>High compression ratio (small size) but very Slow (Not recommended - see Benchmarks) |
EasyCompressor.Zstd (deprecated) | Instead, use EasyCompressor.ZstdSharp. |
EasyCompressor.Snappy (deprecated) | Instead, use EasyCompressor.Snappier |
EasyCompressor.BrotliNET (deprecated) | Instead, use BrotliCompressor in EasyCompressor itself (base package)<br/>(Use only if your project targets .NETFramework462 and above or .NETCoreApp2.0) |
EasyCaching.Extensions.EasyCompressor⭐️ | A winning combination by integrating with EasyCaching to compress your cache data. (Recommended)<br/>See How to use |
**Note 😗*
All of these packages are cross-platform except EasyCompressor.Zstd
and EasyCompressor.Snappy
which are not cross-platform because their underlying library are just a wrapper around the native dlls only for windows.
Get Started
1. Install Package
PM> Install-Package EasyCompressor.LZ4
PM> # Install-Package EasyCompressor (for Brotli, GZip, Deflate, ZLib)
PM> # Install-Package EasyCompressor.Snappier
PM> # Install-Package EasyCompressor.ZstdSharp
PM> # Install-Package EasyCompressor.LZMA
PM> # Install-Package EasyCompressor.Zstd (deprecated)
PM> # Install-Package EasyCompressor.Snappy (deprecated)
PM> # Install-Package EasyCompressor.BrotliNET (deprecated)
2. Using New Instance or the Shared Instance
public class YourClass
{
private readonly ICompressor _compressor;
public YourClass()
{
//--------------------------------------- New Instance ---------------------------------------
_compressor = new LZ4Compressor(); //package : EasyCompressor.LZ4
//_compressor = new ZstdSharpCompressor(); //package : EasyCompressor.Snappier
//_compressor = new BrotliCompressor(); //package : EasyCompressor
//_compressor = new GZipCompressor(); //package : EasyCompressor
//_compressor = new DeflateCompressor(); //package : EasyCompressor
//_compressor = new ZLibCompressor(); //package : EasyCompressor
//_compressor = new LZMACompressor(); //package : EasyCompressor.LZMA
//_compressor = new ZstdCompressor(); //package : EasyCompressor.Zstd (deprecated)
//_compressor = new SnappyCompressor(); //package : EasyCompressor.Snappy (deprecated)
//_compressor = new BrotliNETCompressor(); //package : EasyCompressor.BrotliNET (deprecated)
//--------------------------------------- Shared Instance ---------------------------------------
_compressor = LZ4Compressor.Shared; //package : EasyCompressor.LZ4
//_compressor = ZstdSharpCompressor.Shared; //package : EasyCompressor.Snappier
//_compressor = BrotliCompressor.Shared; //package : EasyCompressor
//_compressor = GZipCompressor.Shared; //package : EasyCompressor
//_compressor = DeflateCompressor.Shared; //package : EasyCompressor
//_compressor = ZLibCompressor.Shared; //package : EasyCompressor
//_compressor = LZMACompressor.Shared; //package : EasyCompressor.LZMA
//_compressor = ZstdCompressor.Shared; //package : EasyCompressor.Zstd (deprecated)
//_compressor = SnappyCompressor.Shared; //package : EasyCompressor.Snappy (deprecated)
//_compressor = BrotliNETCompressor.Shared; //package : EasyCompressor.BrotliNET (deprecated)
}
static static void ProcessData(byte[] bytes)
{
// Compress your original byte[] and return compressed byte[]
var compressedBytes = _compressor.Compress(bytes);
// Decompress compressed byte[] and return uncompressed byte[]
var uncompressedBytes = _compressor.Decompress(compressedBytes);
}
public static void ProcessStream(Stream input, stream output)
{
// Read input stream and Compress into output stream
_compressor.Compress(input, output);
// Read input stream and Decompress into output stream
_compressor.Decompress(input, output);
}
}
3. Using Dependency Injection
Add Services
public void ConfigureServices(IServiceCollection services)
{
//...
services.AddLZ4Compressor(); //package : EasyCompressor.LZ4
//services.AddSnappierCompressor(); //package : EasyCompressor.Snappier
//services.AddZstdSharpCompressor(); //package : EasyCompressor.ZstdSharp
//services.AddBrotliCompressor(); //package : EasyCompressor
//services.AddGZipCompressor(); //package : EasyCompressor
//services.AddDeflateCompressor(); //package : EasyCompressor
//services.AddZLibCompressor(); //package : EasyCompressor
//services.AddLZMACompressor(); //package : EasyCompressor.LZMA
//services.AddZstdCompressor(); //package : EasyCompressor.Zstd (deprecated)
//services.AddSnappyCompressor(); //package : EasyCompressor.Snappy (deprecated)
//services.AddBrotliNETCompressor(); //package : EasyCompressor.BrotliNET (deprecated)
}
Inject/Resolve it and use it
using EasyCompressor;
public class YourClass
{
private readonly ICompressor _compressor;
public YourClass(ICompressor compressor) //Inject using dependency injection
{
_compressor = compressor;
//Or resolve it using IServiceProvider
//_compressor = serviceProvider.GetService<ICompressor>()
}
public void ProcessData(byte[] bytes)
{
// Compress your original byte[] and return compressed byte[]
var compressedBytes = _compressor.Compress(bytes);
// Decompress compressed byte[] and return uncompressed byte[]
var uncompressedBytes = _compressor.Decompress(compressedBytes);
}
public void ProcessStream(Stream input, stream output)
{
// Read input stream and Compress into output stream
_compressor.Compress(input, output);
// Read input stream and Decompress into output stream
_compressor.Decompress(input, output);
}
}
4. Using Named Instances
Register Named compressors
public void ConfigureServices(IServiceCollection services)
{
//...
services.AddLZ4Compressor("lz4"); //package : EasyCompressor.LZ4
services.AddSnappierCompressor("snappier"); //package : EasyCompressor.Snappier
services.AddZstdSharpCompressor("zstdsharp"); //package : EasyCompressor.ZstdSharp
//services.AddBrotliCompressor("brotli"); //package : EasyCompressor
//services.AddGZipCompressor("gzip"); //package : EasyCompressor
//services.AddDeflateCompressor("deflate"); //package : EasyCompressor
//services.AddZLibCompressor("zlib"); //package : EasyCompressor
//services.AddLZMACompressor("lzma"); //package : EasyCompressor.LZMA
//services.AddZstdCompressor("zstd"); //package : EasyCompressor.Zstd (deprecated)
//services.AddSnappyCompressor("snappy"); //package : EasyCompressor.Snappy (deprecated)
//services.AddBrotliNETCompressor("brotlinet"); //package : EasyCompressor.BrotliNET (deprecated)
}
Resolve it using ICompressorProvider
using EasyCompressor;
public class YourClass
{
private readonly ICompressor _lz4Compressor;
private readonly ICompressor _snappierCompressor;
private readonly ICompressor _zstdsharpCompressor;
public YourClass(ICompressorProvider compressorProvider)
{
_lz4Compressor = compressorProvider.GetCompressor("lz4");
_snappierCompressor = compressorProvider.GetCompressor("snappier");
_zstdsharpCompressor = compressor.GetCompressor("zstdsharp");
}
}
Benchmarks
👉 To view the Full benchmark, visit this HTML Page. 👈
Other Benchmarks
Compressing/Decompressing Binary Data (byte[])
- Comparison in terms of Speed (Mean/Execution Time) (visit it's HTML or Image)
- Comparison in terms of Memory Usage (Allocation Size) (visit it's HTML or Image)
Compressing/Decompressing Stream Data
- Comparison in terms of Speed (Mean/Execution Time) (visit it's HTML or Image)
- Comparison in terms of Memory Usage (Allocation Size) (visit it's HTML or Image)
Compressing/Decompressing Stream Data (Async)
- Comparison in terms of Speed (Mean/Execution Time) (visit it's HTML or Image)
- Comparison in terms of Memory Usage (Allocation Size) (visit it's HTML or Image)
Key Results and Conclusion
Best Compressors based on Overall Performance (Speed and Memory Allocation) in each case
Operation | Binary | Stream | StreamAsync |
---|---|---|---|
Compress | SnappierCompressor<br/>LZ4Compressor<br/>ZstdSharpCompressor | SnappierCompressor<br/>LZ4Compressor<br/>BrotliCompressor | LZ4Compressor<br/>BrotliCompressor<br/>--- |
Decompress | LZ4Compressor<br/>SnappierCompressor<br/>ZstdSharpCompressor | SnappierCompressor<br/>LZ4Compressor<br/>ZstdSharpCompressor | ZstdSharpCompressor<br/>LZ4Compressor<br/>--- |
Best Compressors based on Highest compression (Smallest size)
- BrotliCompressor (smaller in medium/small data with moderate speed and memory usage)
- LZMACompressor (smaller in large data but very slow and memory inefficient)
- ZstdSharpCompressor (fastest meanwhile with acceptable/good enough level of compression)
BenchmarkDotNetVisualizer🌈
All the benchmarks are visualized using BenchmarkDotNetVisualizer.
What's Changed from v1.4.0 to v2.0.2
Many improvements were made including performance optimizations and bug fixes.
A great Performance Benchmark between different compression algorithms added.
Two new compressors added: EasyCompressor.Snappier and EasyCompressor.ZstdSharp
Shared instances added to each compressor for Ease-of-Use
LZ4Compressor.Shared.Compress(bytes);
EasyCompressor.Zstd bug fixed and the lastest version of zstd.dll (v1.5.5) where included to the package.
These packages deprecated: EasyCompressor.Zstd, EasyCompressor.Snappy and EasyCompressor.BrotliNET because their development has been stopped and newer and better packages have replaced them.
The default compression levels have changed and are carefully configured based on extensive benchmarking to ensure the highest level of efficiency and speed at a reasonable compression ratio.
Three new LZ4 binary compression modes added:
LZ4BinaryCompressionMode.Optimal (new Default mode) Default compression mode. (NOT compatible with other modes neither LegacyCompatible nor StreamCompatible modes) But it's fast and most efficient in memory allocation. (Best Performance overall - Fast_GCEfficient) It applies only to binary Compress/Decompress and does not affect Stream/Stream[Async] methods.
LZ4BinaryCompressionMode.LegacyCompatible (legacy Default mode) Legacy compatibility with old/legacy versions. (NOT compatible with other modes neither StreamCompatible nor Optimal modes) It's the fastest mode (a bit faster than Optimal) but less efficient in memory allocation. (Fast_GCInefficient) It prepends 4 bytes to the beginning of the array to define the original array length. It applies only to binary Compress/Decompress and does not affect Stream/Stream[Async] methods.
LZ4BinaryCompressionMode.StreamCompatible StreamCompatible which is compatible with Stream's output. (NOT compatible with other modes neither LegacyCompatible nor Optimal modes) It's slower than other modes but moderate in memory allocation. (Slow_GCModerated)
Full Changelog: https://github.com/mjebrahimi/EasyCompressor/compare/1.4.0...2.0.2
Todo
Open an issue or discussion and tell me which integration or feature you like the most.
- Write integrations with Binary Serializer libraries such as MessagePack, MsgPack.Cli, MemoryPack, Google.Protobuf, protobuf-net. (and maybe Bond.CSharp and Hyperion)
- Write integrations with Redis client libraries such as StackExchange.Redis, ServiceStack.Redis, CSRedisCore, FreeRedis.
- Write integrations with Caching libraries such as Microsoft.Extensions.Caching.*, LazyCache, and EnyimMemcachedCore. (and maybe CachingFramework.Redis, CacheManager.Core)
- Write integrations with Message Broker libraries such as RabbitMQ.Client, Azure.Messaging.ServiceBus, Confluent.Kafka, EasyNetQ, NetMQ, NATS.Client, Apache.NMS. (and maybe RawRabbit, kafka-sharp, Microsoft.Azure.ServiceBus).
- Write integrations with Service Bus libraries such as MassTransit, NServiceBus, Rebus, and DotNetCore.CAP.
- Write integrations with other libraries such as Polly.Caching.Distributed
Contributing
Create an issue or discussion if you found a BUG or have a Suggestion or Question.
Or if you want to develop this project:
- Fork it
- Create your feature branch:
git checkout -b my-new-feature
- Commit your changes:
git commit -am 'Add some feature'
- Push to the branch:
git push origin my-new-feature
- Submit a pull request
Give a Star! ⭐️
If you find this repository useful and like it, why not give it a star? if not, never mind! 😃
License
Copyright © 2020 Mohammad Javad Ebrahimi under the MIT License.
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net5.0 was computed. net5.0-windows was computed. net6.0 is compatible. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 is compatible. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 is compatible. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
.NET Core | netcoreapp2.0 was computed. netcoreapp2.1 was computed. netcoreapp2.2 was computed. netcoreapp3.0 was computed. netcoreapp3.1 was computed. |
.NET Standard | netstandard2.0 is compatible. netstandard2.1 is compatible. |
.NET Framework | net461 was computed. net462 is compatible. net463 was computed. net47 was computed. net471 was computed. net472 was computed. net48 was computed. net481 was computed. |
MonoAndroid | monoandroid was computed. |
MonoMac | monomac was computed. |
MonoTouch | monotouch was computed. |
Tizen | tizen40 was computed. tizen60 was computed. |
Xamarin.iOS | xamarinios was computed. |
Xamarin.Mac | xamarinmac was computed. |
Xamarin.TVOS | xamarintvos was computed. |
Xamarin.WatchOS | xamarinwatchos was computed. |
-
.NETFramework 4.6.2
- EasyCompressor (>= 2.0.2)
- Snappy.Standard (>= 0.2.0)
-
.NETStandard 2.0
- EasyCompressor (>= 2.0.2)
- Snappy.Standard (>= 0.2.0)
-
.NETStandard 2.1
- EasyCompressor (>= 2.0.2)
- Snappy.Standard (>= 0.2.0)
-
net6.0
- EasyCompressor (>= 2.0.2)
- Snappy.Standard (>= 0.2.0)
-
net7.0
- EasyCompressor (>= 2.0.2)
- Snappy.Standard (>= 0.2.0)
-
net8.0
- EasyCompressor (>= 2.0.2)
- Snappy.Standard (>= 0.2.0)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.