This is something I've been wondering for a while now, and maybe (most likely ) somebody here knows the answer to this:

I'm using Windows 10 on a fairly recent quad core i5 CPU. This system is plenty fast. Which is why I've always wondered: how come doing two (or more) copy jobs at once slows the copy speeds down so much? I can understand it somewhat if you're copying to the same drive of the same computer, because that would mean the heads of the HD has to switch position all the time, but I've noticed this is also true for copying to devices external of the computer.

Eg. : I'm currently copying some folders containing mp3 files to my unRAID server on my LAN. At the same time, I would like to copy the same files to an external USB3 disk. Doing this at the same time slows copy speeds down to about half, meaning I make zero time gains by doing it this way. Ok, the HD's heads also have to change position because of this, but this is reading, not writing. And only about 20% of the CPU power is used, so that's not the problem as well.

Could somebody explain this to me?

If mechanical disks, then reading vs writing is irrelevant.

I would expect though that the first copy process should provide an element of caching assuming the files are read in the same order by the second copy process.

Probably just a "Microsoft thing". On Linux, I just see maximum throughput on the network at the same time as on the SATA and/or USB3 buses. No issues.

You may be right. I recall there was a tweak on Windows 2000 that radically increased disk performance, but that tweak was eliminated on XP. It annoyed me for years...

That's what I already suspected.


Makes you wonder why they would do that. I mean, who could be against improved disk speed?

People who are for reliability...?

The former does not in any way preclude the latter.

Essentially this. If there's hard drives involved anywhere in the process, it's going to slow it down when any one of them is trying to do more then one operation. The one hard drive in the system is having to thrash a bit to keep sending data to the hard drive in the USB 3 enclosure while also sending it to the spinning hard drives in the unRAID system. At work recently I showed how one SAS SSD sitting in a server was as performant as a 24 drive RAID array of 10k spinning disks for the use case we had.

Every OS and filesystem and RAID controller and so on have various optimizations they try to apply to working with hard drives, and there can be some variances there in how well they do. Those optimizations helped hard drives achieve some form of multitasking. And they vary based on what level of the storage stack. Storage controllers added command queuing and reordering at various points to help minimize head movement. However this stack was rarely that deep, with SATA offering 32, and SAS offering 64 command depth. Most filesystems try to optimize metadata based on spinning disks and various different workloads.

Simply dismissing the issue as a "Microsoft thing" is just the same style of FUD and trolling that MS was engaged in decades ago. Things change. All the way to the point that Windows 10 is a desktop Linux environment these days.

Right now the major change going on in storage continues to be around SSDs. Seeing multiple tiers of SSD tech has been enlightening on where bottlenecks were in our systems due to designing them around spinning magnetic disks. Take for example the command queue depth mentioned earlier. That still applies to SSDs attached to SATA or SAS buses (or USB buses doing USB->SATA translation). There's a somewhat newer bus for SSDs called NVMe that started from scratch instead of basing some layers off prior hard drive tech. It offers 64k of command depth in it's queue, and has 64k queues instead of one. Playing with NVMe is when you stop measuring drive performance in MB/s and start measuring in GB/s. Latency stops being measured in ms, and instead in µs.

I've been testing NVMe vs SAS SSDs vs HDD RAIDs lately, and the order of magnitude in difference between them is amazing. Filesystems are also starting to come out designed purely around SSDs, such as APFS already running on millions of iOS devices. Storage is finally accelerating in speed as the same pace as other tech, and in time will eventually be cost competitive to replace almost all use of spinning hard drives. One interesting advance will be NVDIMM, it's already showing up in commodity x64 based servers as an option.

Thanks for this explanation Tom, very enlightening!
I had no idea there still was such a big difference between SATA SSD's and NVMe SSD's. I know NVMe is coupled directly onto the PCIexpress bus, so it's a lot faster than SATA still, but I had no idea there was more to it that that.

Thanks!

Thanks, indeed! (This is one of the many reasons I love this community.)

-jk