2

I am looking for an FTP client that works on Windows 7 and can open a folder with millions of files quickly (e.g. 50 millions). It should be GUI and list all the files. Both FileZilla and WinSCP takes forever to display the folder's content.

  • 1
    If you care about performance, you should specify the time and your hardware specs (especially the network connection speed) to get comparable results. FTP over a 56k modem will definitely take forever. It's much faster if you use 10000BaseT fiber. Scale with hardware :-) – Thomas Weller Dec 7 '15 at 20:06
  • 2
    50 Million File names at - if DOS 8.3 ASCII = 11 bytes = 55 MB of data to transfer for the file names alone - if there is 1MB link anywhere on the path that is a 50 second transfer for that alone. Also with 50M GUI objects you are likely to run out of GUI IDs on windows systems. – Steve Barnes Dec 7 '15 at 20:47
  • 2
    @SteveBarnes: Your calculation should result in 550 MB, not 55 MB. And FTP also transfers modification time, owner information and user/group info, ~96 bytes per file. It does that with MSLD, which already switches to an optimized transfer mode. It's 4.8 GB (!) to transfer. On a 1 Gbps link, this alone takes 35 seconds. Keeping the 4.8 GB of data in RAM will require a 64 bit application. – Thomas Weller Dec 7 '15 at 21:20
  • 1
    Besides that, 4.8 GB on Ethernet (packet size 1514 bytes) will require 3.1 M packets to be sent. At 1ms ping time, 2ms round trip time, this accounts for another 1:45 hours! – Thomas Weller Dec 7 '15 at 21:23
  • 1
    If it's a local Intranet, you could try to enable Jumbo packets which have a size of 4096 bytes. This might bring it down to 0:40 hours. You have to do it in Windows (Registry) and all switches/routers need to support it. – Thomas Weller Dec 7 '15 at 21:27
3

I already replied to your similar question here:

https://softwarerecs.stackexchange.com/a/27511/2336

Specifically in this question (FTP client), also network data transfer speed comes in as significant factor – see comments of Thomas Weller under your question. But you can still attempt to put files on remote side into single archive (TAR, ZIP) and after transferring the archive to your local machine, you can unpack it and go with one of solutions sketched in the answer linked above.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.