Rather by trial-and-error I found out that when using an USB extension cord for placing the USB receiver (such as a Logitech unifying receiver) nearer to mouse and keyboard will help a lot. In fact this moves the USB receiver away from the external hard disk or memory stick that seem to be the cause for RF interference.
Later I learned (see below) that connecting this USB extension cord for the USB receiver to a USB 2.0 port on the back of the computer improves the situation even more. As you plug in USB 3.0 external hard disks or memory sticks into the front, that mitigates the RF noise caused by these devices. But only if the USB 3.0 ports next to it are properly shielded.
Intel on USB 3.0 RF Noise
Recently I've stumbled across this interesting Logitech forum post: Unifying Receiver stops working when a USB 3.0 device is plugged in. This post is highly enlighting as it links to an Intel report on USB 3.0 Radio Frequency Interference Impact on 2.4GHz Wireless Devices. This report is 22 pages, yet an interesting and easy read even for the not so technically inclined.
In a nutshell: unfortunately, many badly designed USB 3.0 devices, but also motherboard USB 3.0 ports, emit a lot of broadband radio frequency noise. And this RF noise covers also the 2.4GHz band: this is where we have wireless LAN (WiFi), Bluetooth, and many wireless mouse and keyboard systems.
Add Insult to the Injury
Just as a bad joke on this whole enchilada: whenever inserting a Logitech unified receiver into the USB 3.0 port of my Microsoft Surface Pro 3 completely kills the wireless LAN connectivity of the Surface Pro 3. Sweet irony.
I found out because I had to update the firmware of the Logitech unifying receiver (well, Mouse Jack). Of course, such a firmware update requires Windows. But even for the update you need to search Logitech's forum to find out how to apply the firmware update. I see that Logitech needs to improve its software quality by a billion notches...