Could be both. Who knows. For high performance computing Linux is the de facto standard because it has better performances than windows, and Linux distros are usually better, stabler OSes overall when one needs raw performances. In this case, who knows, someone should investigate further
Tbf, on Linux performances for hpc are better even on standard desktop distros, no need to clean it up. It manages resources and jobs much better. For performances that is the critical part, once the code is optimized.
I think it has more to do with Linux being easier to tweak, not some inherent performance difference. You can tweak the scheduler, page sizes, and all manner of other things to get a bit more performance if you know what your workload looks like. So it being open source and ubiquitous is a bigger contributor imo than anything inherent to the design of the kernel.
Regular users aren’t going to go through that level of tweaking, so the difference should be a lot smaller and will benefit more from general code-level optimizations than system tweaks. General purpose, high performance computing works just fine on Windows, it’s just easier to tweak Linux for production compute use cases.
No no, it is better. Take a real hpc library, install debian and test it yourself. No tweak needed. Linux as kernel and the overall OS manages resources much better. Linux is a better kernel than windows kernel.
I’ve been doing hpc for over 15 years now. People install standard distros on their workstations and clusters. No tweak needed
How big are we talking? I looked and couldn’t find benchmarks, but then again I’m not familiar enough with HPC to know what benchmarks to look for.
I’ve been Linux exclusive for something like 15 years now (before Steam even came to Linux), so I’m not exactly familiar with Windows performance on the stuff I use. I casually look at larger projects and benchmarks they run (for example, I remember async on Linux vs Windows was a significant issue in the early days of node.js).
I do dabble a bit in hpc, but only on Windows and macOS. I’ve done signal processing and some high thread count number crunching, but I haven’t needed to run benchmarks, just get things running well enough (as in, minutes vs hours, not 10-20% difference).
When I talk about hpc, I don’t talk about a script in Matlab. I talk about the work you do on supercomputers, real computing intensive jobs that takes weeks or months on hundreds or thousands of processors. I guess you don’t find benchmarks simply because no one uses windows, same reason you probably don’t find a fiat panda in the Nürburgring rankings.
K. Everything I’ve done is basically translating a Matlab script to Python using numpy or tensorflow or something. So we’d go from Matlab taking hours to Python taking minutes. The biggest project was a Monte Carlo simulation of signals produced from explosions (looking for seismographic impact) that takes something like 45 min per run when run on our cloud infra.
So something a little more interesting than plotting an FFT if overlaid signals, but still on the simpler end of the spectrum.
We’re nowhere near the point where tuning the OS is interesting, we just use Linux because it’s convenient. I wonder how much of the HPC crowd has a similar perspective, at least until you get to the higher end where tuning the OS becomes important.
You don’t really need to tune the operating system, you just need a good one. The hpc crowd has a pretty unique perspective. You won’t find anyone doing any real hpc on windows, not even Microsoft
Could be both. Who knows. For high performance computing Linux is the de facto standard because it has better performances than windows, and Linux distros are usually better, stabler OSes overall when one needs raw performances. In this case, who knows, someone should investigate further
deleted by creator
Hey now! Starships aren’t nearly as complicated!
deleted by creator
Tbf, on Linux performances for hpc are better even on standard desktop distros, no need to clean it up. It manages resources and jobs much better. For performances that is the critical part, once the code is optimized.
I think it has more to do with Linux being easier to tweak, not some inherent performance difference. You can tweak the scheduler, page sizes, and all manner of other things to get a bit more performance if you know what your workload looks like. So it being open source and ubiquitous is a bigger contributor imo than anything inherent to the design of the kernel.
Regular users aren’t going to go through that level of tweaking, so the difference should be a lot smaller and will benefit more from general code-level optimizations than system tweaks. General purpose, high performance computing works just fine on Windows, it’s just easier to tweak Linux for production compute use cases.
No no, it is better. Take a real hpc library, install debian and test it yourself. No tweak needed. Linux as kernel and the overall OS manages resources much better. Linux is a better kernel than windows kernel.
I’ve been doing hpc for over 15 years now. People install standard distros on their workstations and clusters. No tweak needed
How big are we talking? I looked and couldn’t find benchmarks, but then again I’m not familiar enough with HPC to know what benchmarks to look for.
I’ve been Linux exclusive for something like 15 years now (before Steam even came to Linux), so I’m not exactly familiar with Windows performance on the stuff I use. I casually look at larger projects and benchmarks they run (for example, I remember async on Linux vs Windows was a significant issue in the early days of node.js).
I do dabble a bit in hpc, but only on Windows and macOS. I’ve done signal processing and some high thread count number crunching, but I haven’t needed to run benchmarks, just get things running well enough (as in, minutes vs hours, not 10-20% difference).
When I talk about hpc, I don’t talk about a script in Matlab. I talk about the work you do on supercomputers, real computing intensive jobs that takes weeks or months on hundreds or thousands of processors. I guess you don’t find benchmarks simply because no one uses windows, same reason you probably don’t find a fiat panda in the Nürburgring rankings.
K. Everything I’ve done is basically translating a Matlab script to Python using numpy or tensorflow or something. So we’d go from Matlab taking hours to Python taking minutes. The biggest project was a Monte Carlo simulation of signals produced from explosions (looking for seismographic impact) that takes something like 45 min per run when run on our cloud infra.
So something a little more interesting than plotting an FFT if overlaid signals, but still on the simpler end of the spectrum.
We’re nowhere near the point where tuning the OS is interesting, we just use Linux because it’s convenient. I wonder how much of the HPC crowd has a similar perspective, at least until you get to the higher end where tuning the OS becomes important.
You don’t really need to tune the operating system, you just need a good one. The hpc crowd has a pretty unique perspective. You won’t find anyone doing any real hpc on windows, not even Microsoft