A class for parallelizing any For Loop in C#

From:


foreach (Entity item in collection)
{
     DoWork(item);
}


To:


LoopParallelizer.Execute<Entity>(TaskCreationOptions.PreferFairness, 
                                           collection,
                                           new Action<Entity>(DoWork));


This will basically split any for loop into a configurable number of threads and run these threads in parallel and wait till all of them are over. The bigger the collection, the faster will be the performance gain by doing this.


public static class LoopParallelizer
    {
 
        public static void Execute<T>(TaskCreationOptions options, ICollection collection, Action<T> work) where T : class
        {
            int threadCount = 4;
            int parts = (collection.Count / threadCount);
 
            List<List<Action>> actions = new List<List<Action>>();
 
            for (int i = 0; i < threadCount; i++)
            {
                actions.Add(new List<Action>());
            }
 
            int threadCounter = 0;
            int counter = 0;
 
            foreach (T item in collection)
            {
                actions[threadCounter].Add(delegate { work(item); });
                counter++;
 
                if (counter == parts && threadCounter < (actions.Count - 1))
                {
                    counter = 0;
                    threadCounter++;
                }
            }
 
            List<Task> tasks = new List<Task>();
 
            for (int i = 0; i < actions.Count; i++)
            {
                //Split the work to prevent concurrency issues.
                //actions[i] does not work properly in delegate as well.
                List<Action> actual = new List<Action>();
 
                for (int j = 0; j < actions[i].Count; j++)
                {
                    actual.AddRange(actions[i]);
                }
 
                tasks.Add(Task.Factory.StartNew(
                                  delegate 
                                  {
                                      for (int j = 0; j < actual.Count; j++)
                                      {
                                          actual[j]();
                                      }
                                  }, 
                                  options)
                 );
            }
 
                for (int i = 0; i < tasks.Count; i++)
                {
                    try
                    {
                        tasks[i].Wait();                        
                    }
                    catch (AggregateException ae)
                    {
                        throw (ae.Flatten());
                    }
                }
        }     }

Note: This is an experiment, you can use the Parallel.For instead. Only problem both solutions is that as it uses thread pool thread where setting its priority is not recommended, it can use 100% CPU if it goes through the for loop very fast - and no other processes can use system resources at the time because it runs at normal priority. See my next post on what to do then.

Comments

Popular posts from this blog

Tutorial: Using Google Cloud Storage from C# and .NET

Late 2008 Macbook only giving 1.5 gb/s speed with 6 gb/s Intel SSD?

Enable Visual Studio to use more than 2GB of memory