Bizarre shenanigans of .NET task performance

At some moment in my career I was asked to write a piece of software that performed a relatively long operation on small pieces of data that were aplenty. The whole process was about to take several days.

I approached it from the most straightforward direction: wrote a program in the contemporary C#.NET 4.7.1 that started as many tasks as it had CPU cores. Each task created its own progress bar on a form. They reported their progress through those progress bars. The process scaled linearly.

Now, I am being asked to do the same but with a different but similar data processing algorithm, and things go bananas: the same Visual Studio, the same C#, the same .NET, different main processing class/method, and the performance does not scale at all, period. It divides instead, i.e. it for example gives 10,000 per second single-threaded, 4500 double-threaded, 2000 quad-threaded, etc, per core. Even more bizarre is the observation that 2 separate instances of that program divide their performance even though they do not talk to one another.

Having spent several days trying different approaches, including scheduling the affinity of the tasks' threads, I am at my wits' end. I even tried to wrap a C library that does the same, but it also divides its performance when in SMP mode. The only common thing is the underlying algorithm.

What is peculiar is that the algorithm is very wide-spread. It runs literally everywhere, around the clock, non-stop. So I, a developer with several decades of experience in writing advanced multi-threaded software, is baffled. That is not an easy achievement for .NET or whatnot. Congratulations!

Posted by: LinuxLies at 05:53 AM | No Comments | Add Comment
Post contains 283 words, total size 2 kb.