ExternalSort 1.0.0

.NET 8.0 This package targets .NET 8.0. The package is compatible with this framework or higher.
There is a newer version of this package available.
See the version list below for details. 
dotnet add package ExternalSort --version 1.0.0                

                    
                

            
NuGet\Install-Package ExternalSort -Version 1.0.0
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package. 
<PackageReference Include="ExternalSort" Version="1.0.0" />
For projects that support PackageReference, copy this XML node into the project file to reference the package. 
paket add ExternalSort --version 1.0.0                

                    
                

            
#r "nuget: ExternalSort, 1.0.0"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package. 
// Install ExternalSort as a Cake Addin
#addin nuget:?package=ExternalSort&version=1.0.0

// Install ExternalSort as a Cake Tool
#tool nuget:?package=ExternalSort&version=1.0.0

External Sort

Contains various utilities to sort and join large amounts of data that is larger than the amount of available RAM.

Documentation

OrderByExternal()

Sort IAsyncEnumerable<T> data.

  IAsyncEnumerable<User> myUsersToSort = ...
  var sortedUsers = myUsersToSort
    .OrderByExternal(u => u.Email);

GroupJoinExternal()

GroupJoin (left outer join) on IAsyncEnumerable<T> data.

  IAsyncEnumerable<User> myUsers = ...
  IAsyncEnumerable<UserComments> myUserComments = ...
  
  var joinedUsersAndComments = myUsersToSort
    .GroupJoinExternal(myUserComments, u => u.UserId, uc => uc.UserId, (user, comments) => new
    {
        User = user,
        Comments = comments.ToList()
    });

ExceptByExternal()

ExceptBy (NOT IN) on IAsyncEnumerable<T> data.

  IAsyncEnumerable<User> myUsers = ...
  IAsyncEnumerable<int> myStaffUserIds = ...
  
  var nonStaffUsers = myUsers
    .ExceptByExternal(myStaffUserIds, u => u.UserId);

Internally, ExternalSort uses Parquet temp files. See https://github.com/aloneguid/parquet-dotnet for class serialisation options.

Thanks to https://josef.codes/sorting-really-large-files-with-c-sharp/ for the inspiration.

Product Compatible and additional computed target framework versions.
.NET net8.0 is compatible.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
2.0.0-beta02 1,798 7/4/2024
2.0.0-beta01 91 7/4/2024
1.2.0 97 5/15/2024
1.2.0-beta001 818 4/15/2024
1.1.0 131 4/14/2024
1.1.0-beta003 172 4/11/2024
1.1.0-beta002 87 4/11/2024
1.1.0-beta001 109 4/10/2024
1.0.0 110 4/9/2024
1.0.0-beta005 145 4/8/2024
1.0.0-beta004 80 4/8/2024
1.0.0-beta003 118 4/8/2024
1.0.0-beta002 100 4/7/2024
1.0.0-beta001 95 4/4/2024