June 22, 2021

Implementing a Batch in Linq

When enumeration database records in Linq, it is important to batch the work otherwise LINQ can generate SQL statements that are too big (regardless of whether Entity Frameworks are being used or LINQ to SQL). Another problem is that it can take a long time to process all the records, not only can that require a lot of memory, after a while you will start to wonder if the code has crashed or not. It is better to see a batch finishing every now and then and thus know that it is actually progressing.
// Source here : https://stackoverflow.com/questions/13731796/create-batches-in-linq
// Example here : https://dotnetfiddle.net/HpRgd5
public static class BatchLinq
{
    /// <summary>
    /// Full lazy implementation of a batch
    /// Known issue with this approach is that each batch must be 
    /// enumerated and enumerated fully before moving to the next batch.
    /// </summary>
    /// <typeparam name="T">The type being enumerated</typeparam>
    /// <param name="source">The source enumerable</param>
    /// <param name="size">Size of the batch</param>
    /// <returns>An enumeration of batches with size elements 
    /// in each batch</returns>
    public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> source, int size)
    {
        if (size <= 0)
        {
            throw new ArgumentOutOfRangeException("size", "Must be greater than zero.");
        }
        using (var enumerator = source.GetEnumerator())
        {
            while (enumerator.MoveNext())
            {
                var innerBatch = new InnerBatch();
                var enumerable = innerBatch.Batch(enumerator, size);
                yield return enumerable;
                if (!innerBatch.IsFinished)
                    enumerable.Count();
            }
        }
    }

    private class InnerBatch
    {
        public bool IsFinished { get; private set; } = false;

        public IEnumerable<T> Batch<T>(IEnumerator<T> source, int size)
        {
            int i = 0;
            do 
                yield return source.Current;
            while (++i < size && source.MoveNext());
            IsFinished = true;
        }
    }

    // This implementation splits the enumeration up into batches of lists
    // This is safe to use if the batches are small but it is not a fully lazy 
    // evaluation of the batch like the above method. However if you do not 
    // enumerate each batch fully then this version must be used
    public static IEnumerable<IList<T>> BatchByList<T>(this IEnumerable<T> source, int size)
    {
        List<T> batch = new List<T>(size);

        foreach (var item in source)
        {
            batch.Add(item);

            if (batch.Count >= size)
            {
                yield return batch;
                batch = new List<T>(size);
            }
        }

        if (batch.Count > 0)
        {
            yield return batch;
        }
    }
}

Using AutoFac for Dependency Injection

Create the Autofac builder:
    // Step 1. Create the builder with which components/services are registered.
    var builder = new ContainerBuilder();

    // Step 2. 
    RegisterTypesWithAutoFac(builder); // Register your types here (see below)

    // Step 3.
    // Build the container to finalize registrations
    // and prepare for object resolution.
    var container = builder.Build();

    // Step 4.
    // Now you can resolve services using Autofac.
    using(var scope = container.BeginLifetimeScope())
    {
      var reader = scope.Resolve<IConfigReader>();
    }
However in MVC 5 replace Step 4 with this:
    var mvcResolver = new AutofacDependencyResolver(container);
    DependencyResolver.SetResolver(mvcResolver);

Registering Types
Here are a few key examples of registering types.
Case 1: Instance on Request:

builder.RegisterType<SystemDistanceCalculator>().
    As<ISystemDistanceCalculator>().
    InstancePerRequest();
Case 2: A singleton instance
builder.RegisterType<SystemIdsCachesManager>().
    As<ISystemIdsCachesManager>().
    SingleInstance();
Case 3: Initializing Parameters Example
builder.RegisterType<PermitSystemsFactory>().
    As<IPermitSystemsFactory>().
    WithParameter("repositoryPath", EliteDangerousPaths.PermitSystemsJsonlPath).
    SingleInstance();
Case 4: Using a lambda to perform some of the object initialization
builder.Register(c =>
{
    var edSystemsDb = new EdSystemsSqlDb();
    var sqlServerConfig = c.Resolve<ISqlServerConfiguration>();
    edSystemsDb.Initialise(sqlServerConfig.ConnectionString);
    return edSystemsDb;
}).
    As<IEdSystemsDb>().
    SingleInstance();