FileShareFreak

THE SOURCE FOR BITTORRENT & P2P TIPS, TRICKS AND INFO

Increase FTP Speeds With Segmented Downloading

One of the biggest complaints from seedbox users is not about the BitTorrent upload/download speeds within the server itself; rather, the FTP speeds in which the files are transferred back to the home PC. Often these sluggish FTP speeds can be attributed directly to the server provider; OVH is one of the usual suspects here. We’ll show you how to get the most out of your bandwidth using FTP clients that support segmented downloading.

Multiple Concurrent Transfers — This allows for multiple transfers to occur during the same session (ie - you can download more than one file at the same time) but this doesn’t necessarily imply that segmentation occurs. For example, most FTP clients support multiple simultaneous concurrent transfers (such as FileZilla). However, Filezilla doesn’t support segmented downloading. Therefore if you can only get 100KB/s per connection, and you’re only retrieving one file, your speed will be limited to 100KB/s.

Multi-threading vs Segmented — A common misconception is that multithreading = segmented downloading. This is not the case. GoFTP is a client that claims to support multi-threading, but in truth doesn’t offer segmented. Segmentation allows an FTP client to split a single large file into multiple parts and use multiple transfers to download those parts simultaneously (ie - separate segmented parallel FTP connections). These parts are then recombined into a single file upon completion. During the active download you’ll notice multiple (temporary) parts of the same file:

For scene releases it doesn’t really matter if you use segmentation or not, since the files are generally quite small anyways. But for extracted files such as x264 (.mkv) movies or single *.avi files, segmentation is extremely useful to maximize your server and/or home ISP’s bandwidth.

FTP Clients That Support Segmentation:

Below are some common examples of how to configure FTP clients that support segmented downloads. There are probably other clients as well, but these were the most obvious & popular choices.

CuteFTP 8 Pro (Windows)

By default, CuteFTP Pro is already configured to allow for segmented downloads. Here’s how you can optimize CuteFTP to get even better speeds.

1.

Assuming you’ve already created a profile for your FTP server/connection in CuteFTP, right-click on the profile and select Properties.

2.

Click on the Options tab, and select “Use site specific option” - next, change the site max concurrent transfers to a value of 8.

3.

In the main CuteFTP window, go into Tools > Global Options… and click on the **Connection

** tab. Change the default values (12 & 4) to something a little higher (we used 50 & 20).

4.

While still in the Global Options, click the **Transfer

** tab. Change the value to “8” for the number of threads as shown below. Click OK to save the new settings.

5.

Now connect to your server - and once connected, right-click a file or folder you wish to download and select ”Download Advanced > Multi-part download > MAX (8 parts)

In the Queue Window at the bottom, click the ” + ” to expand the single file - you’ll notice there are 8 separate parts downloading simultaneously. Once all parts are complete, the temp files will automatically be deleted leaving you with just the single completed file.

TIP: If you find that these settings are too high and you’re unable to do little else with your Internet connection, then adjust or reduce the settings accordingly.

SmartFTP 4 Pro (Windows)

In SmartFTP (Pro), there’s a feature that allows for segmented downloading, although you’ll need to do this per each transfer (there are no options to enable it for global transfers in the program settings). For this, we recommend you use CuteFTP since users are able to highlight multiple files/folders and select multi-part downloads (for all of them at once), whereas you can’t in SmartFTP. But we’ll explain how it works in SmartFTP:

Once connected to your FTP, add a file to the Transfer Queue, but do not start the job. Where it says Workers 1 - change this to a value of 10. Next, right-click on the file and select Properties.

Click on the last tab ”Advanced” and put a check in the box ”Enable” for Multi Part Transfers. As well, change the number of workers to 10. Click OK to finish.

Start the transfer. If, by accident, the transfer was already running when you changed the properties, you’ll need to stop and restart it.

Captain FTP (Mac)

Captain FTP is specifically designed and built for Mac OS X. It combines a powerful FTP transfer capability with user friendly functionality, and also contains an accelerated transfer feature - the transfer of large files is split into segments to vastly accelerate the transfer speeds.

LFTP (Linux)

I haven’t tested this myself, but *Nux users everywhere swear by it. LFTP is capable of handling multi-threaded, multiple transfers and segmented. Users are able to specify segmented size, and the pget command is used to download a file using several connections. See their homepage for a complete list of features.

  1. SHARPIE Says:

    Good tips, thanx!

  2. LOL Says:

    Unfortunately a lot of users dont know this, and it causes a real hassle for server providers. Thanks for the guide Sharky and I can also recommend ftprush (scene standard) since it uses multi-kernel-threaded rather than segmented.

  3. tid Says:

    Not sure why but my OVH box doesn’t seem to have this problem. FTP downloads generally run at 10-12Mb/s

  4. sharky Says:

    I tried FTPRush and wanted to add it into this, but I couldn’t find any options to enable that feature.

  5. Leandro Says:

    I use used DownThemAll with firefox, using the ftp url (ftp://user:pass@host/filepath), which was insane.. Let me try with CaptainFTP, thanks!

  6. hlx98007 Says:

    flashget/idm/nettransfer etc can do this already.

  7. NO U Says:

    Thats what rars are for lamers

  8. sK0Tt Says:

    lol @ 7

    good tutorial. keep em coming.

  9. Black3d Says:

    For FTP rush, you’ll find it by rightclick on the file or folder and choose advanced transfer -> multi-stream download, here as a picture: http://i39.tinypic.com/vmptvd.jpg and there you go with a very powerfull multistream download to max out your bandwith :)
    even though I dont have any complaints about my ovh seedbox, I have fullspeed (around 17000kb/s) on only one file, one transfer, no multistream, traffic goes to germany, so not that long way…

  10. mcpain Says:

    get premium accounts here:

    http://razordox.darkbb.com/forum.htm

    invitation code to join: mcpain

    we share premium accounts and scene releases

  11. RAWR Says:

    Thanks Skarky! Anyone know if this is possible with Transmit on a Mac? Either that, or I guess I’m switching to Captain http://FTP. :)

  12. ovh Says:

    I don’t think this solves anything, from my ovh box it still downloads at ~250kB/s at peak times. Bandwidth isn’t magically created just by opening another connection

  13. Jack Black Says:

    @OVH

    Yes, it can be “magically” created, over long distances, a single tcp connection can be boggled down by the numerous relay stations it needs to go through to get to you. For most people who don’t live in Europe, this means a packet traveling out of your seedbox, to the bandwidth provider, across the pond, to the main relay station, to your local station, to your ISP and finally to your box.

    By opening multiple connections, you’re essentially adding more runners to the marathon.

  14. anon Says:

    Thats a retarded stance, use a real FTPD instead of these shite windows programs. vsftpd is far far greater at pushing data, or use drftpd and use PRET for multi-transfers. Clients trying to do things like this are just hacks.

  15. mcpain Says:

    get premium account here:

    http://razordox.darkbb.com/forum.htm

    invitation code to join: mcpain

    we share premium accounts everyday and scene releases as well

    we also share high speed direct download link to games and movies

  16. WarezBot Says:

    FTP is really old protocol..but we are still love and use it..
    I recently shifted to rsync these days..

  17. ETONIAN Says:

    MCPAIN,

    Please stop posting links to shitty sites you dumb peasant-brained cunt.

    Thank you

  18. ovh Says:

    @JACK BLACK
    I am in Europe as the majority of the ovh users probably are. When OVH feels like it I download at most ~250kB/s total with 10 connections. I am 12 hops away from my seedbox. The strange this is right now it’s downloading almost at full speed

  19. zebus Says:

    Bummer, for some reason CuteFTP has horrible speeds compared to my solution I’m using now of Filezilla + creating split rar achieves. The way you explain SmartFTP just makes it seem like a hassle, FTPRush mentioned in comments evidently doesn’t work on Win7, won’t launch, and I don’t have linux or mac so the other two are out..

    Anyone know anything else :P, guess I could try VM Linux + LFPT

  20. zebus Says:

    Actually screw that.. LFTP is command line Oo

  21. qwerty Says:

    cuteftp FTW! excellent post sharky. keep up the good work.

  22. H1flux Says:

    Nice post sharky!
    A tip, if you using box on can setup folder (like VPS, or offers of Feral), You can setup a download folder, then downloading using web browser and using a download manager, link internet download manager, dap, or jdownloader. You get high speed.
    ;)

  23. me Says:

    I’m using filezilla as well and I max out my home connection using port 21.. :)

    Happy seedbox owner here :)

  24. PurgatoryPriest Says:

    I have tested ALL the major & not so major FTP clients (11 clients in all) including ones with and without Multi-part downloading. After approx. 3 weeks this is my conclusion as posted somewhere else. For the single download, FileZilla cannot be beat. It constantly runs for weeks without crashing. SmartFTP is my second choice for Multi-part downloading followed by Free Download Manager’s Site Explorer/FTP feature.

    But BitKinex is hands down the winner IMO. Low memory, super stable and fast! I own a copy of the latest CuteFTP Pro and not matter what I do, after 4 or 5GB in file transfer, CuteFTP crashes or freezes. They really need to update that engine. I found that SmartFTP offers me too much and is a hassle to go through its many features but would be my choice had it not been for BitKinex. I also hate to have to set the Workers/Parts for every transfer in SmartFTP to achieve Multi-part. The developer says this is by design. Stupid design if you ask me. I’ll try CuteFTP again using your settings as illustrated and see what happens. Even if it continues to work properly, I’ll still stick with BitKinex… the interface is a bit clumsy but I quickly got the hang of it. Below is my review and thoughts. Thanks for the article.

    These tests were conducted by me to find out which is the most efficient, fast, accurate & error free way (via FTP) to transfer your media to your local is more important now (to me at least) because of my temporary reduced home speeds. These tests also benefit anyone living in the Western part of Earth that use a seedbox or anyone world over with slow Internet speeds at home/office or wherever that find downloading from their seedbox (and elsewhere) via FTP is painfully slow compared to how fast you can torrent to your seedbox.

    XXX suggested about a month and a half ago a FTP client called BitKinex which I had never heard of. I did some informal testing using the multipart download feature but at the time for one reason or another, I did not follow through thoroughly and pit BitKinex up against the personal fav. client FileZilla. FileZilla does not have multipart download but does support multiple simultaneous transfers.That is more than one different file at a time. I tried this but over the hours, the download speed for each file decrease by approx. 60 to 80KB/s. For me it was no different than downloading two large files via HTTP, each file would get approx. 50% of the bandwidth give or take a few KB. I ended up downloading one file at a time. Once I moved to my current location, and had to switch to a slower Internet speeds until the upgrades in my area are done, I noticed my FTP download speeds were at half of what they were before. That added several hours in downloading time for each file. This sparked me to go on another quest for a solution. That solution as it stands now is BitKinex. It is one hell of a program, easy to use but a bit complex if you do not know the lingo. Do not let that stop you from giving it a whirl and posting your results back here to share with others. It has a very good Help content and once you get the hang of the features, it is child’s play. Compared to CuteFTP’s engine (ftpte) & its multipart download feature, BitKinex wipes the floor everyday with it. Rinse & repeat.

    Some folks have asked what the pros are and cons to multipart downloading of very large files. This FileZilla link about it started 5 years ago but is still relevant & active. Here is the link: http://trac.filezilla-project.org/ticket/2309
    In my opinion, I see no huge loss of Internet speeds when I am downloading a 14GB file using BitKinex with multipart downloads. YouTube stutters a little and online game play is a bit slower but nothing that is painful. You could always pause the download and resume at a later time without having to start over from scratch. *BitKinex throttles the connection if it detects browsing activity.

    My results are as follows:

    Short Test with a 560MB File - Results:

    BitKinex - 560MB file x6 Connections (multipart download) =642KB/s
    Beginning Time: 3:36:55 AM
    Ending Time: 3:52:53 AM
    Difference Between Times
    0 hours
    15 minutes
    58 seconds

    -
    FileZilla 560MB file test:
    Time
    Beginning Time: 3:10:02 AM
    Ending Time: 3:32:51 AM
    Difference Between Times
    0 hours
    22 minutes
    49 seconds
    FileZilla - 1 Connection (not a multipart download client)=451KB/s

    ————————————————————————-
    Long Test with a 14.3GB File - Results:

    1. BitKinex - x6 Connections @ 642KB/s = 6 hours - 52 minutes (2.20GB Per Hour) File is Divided into 6 parts at 107KB/s each. -
    2. FileZilla - x1 Connections (not a multipart download client)@ 445KB/s = 10 hours - 32 minutes
    -That is a 3 hour and Forty Minute difference! That is a huge time saver for such a large file IMO.-

    Mental notes:
    *Seems that my original testing/assessments were wrong. I am constantly downloading in MultiPart at 642KB/s instead of the 350 to 430KB/s with FileZilla. MultiPart with CuteFTP Pro sucked and I was wrong in comparing MultiPart using only one or two clients to test. It seems perfectly clear now that BitKinex rules in this territory.*

    Extra thoughts from my brain while testing:

    | Above 7 connections seems to not make a difference in overall constant speeds)- Server limitation?/My Internet speed limitation=yes. Research & post results. For now, 5 or 6 connections recommended. Oh! What is this? Out of the 7 connections I had set, one was reserved for browsing my remote server directories. I’ll have to bump that up to 15 or so and see what happens.

    CONS: Downloads multiparts to a TEMP folder on your C/Main Drive by default. - Can this be changed? - YES _ Point to any location you want. I have mine pointed to the same drive I download to with FileZilla. That way the finished and re-assembled data does not have to be moved from drive to drive. However if you have the space, use another drive for the incoming parts, this way when they are re-assembling to the actual storage drive, the local transfer is a bit faster (Read/Write from the same drive slows down the process a bit). !Note!: - Directory for temporary transfer files - “This option specifies the location where temporary transfer files are created. If a path specification is invalid, it will be impossible to perform any transfers.” - No Time Remaining or estimation info? (*Completed so far in percentage is there though). I use this free (no install/Runs as Standalone) program for estimations in download time: http://fwd4.me/F52.

    Pros: Low memory footprint - 7 to 9MB (WORKING) to 70MB at its highest peak (When browsing your server/local computer). CPU: Light at 3 to 7 percent. Some spikes when poking around the programs interface as you are downloading but nothing to gripe about. |

    I’m I done yet? I think so. One more note (I think) - Unfortunately BitKinex’s development has now stopped and the latest version is being distributed under the freeware license. I see the use of the client for the next several years unless something comes along and betters it. Works on Windows 7 x64 too!

    And that’s a wrap! ( fingers fall off…again. where’s the glue? - PP

  25. zebus Says:

    Wow, now that was a reply if I ever saw one, thanks for all the info PurgatoryPriest, I’ll have to look into bitkinex

  26. Usenet Says:

    Bah something fishy is going on. I have Astraweb and access their servers from across the pond. It might not be FTP but I max out my internet connection easily. Sounds like to me they are doing some bandwidth throttling of some type for some reason.

  27. LOL Says:

    FTPRush cannot be beaten imo.

    I use it with win 7 and it works fine.

    I have about 6 tabs open connected to 12 servers and it runs fine with no hassle.

  28. PurgatoryPriest Says:

    “LOL Says: FTPRush cannot be beaten imo.”

    If it was not for BitKinex being a bit faster, I would agree with you. Not matter what setting(s), tweaks and adjustments I do on both the FTP server and to FTPRush, BitKinex is still approx. 100KB/s faster. That is around 3/4mins per Gigabyte. I’m still going to look for the issue as to why BitKinex is a bit faster than FTPRush. I prefer the interface offered by FTPRush over the clumsy BitKinex windows. However that is not going to convert me if I cannot get FTPRush’s speed to match or come closer to BitKinex. Testing shortly on a 14GB file with FTPRush and then again (same file, same environment) with BitKinex. I’ll report back with the results later in the week. - PP

  29. Jack Black Says:

    For those of you who use Linux

    lftp can use both parallel and segmented downloading!

    read up on the man page for lftp about parallel and pget options, parallel will download multiple files simultaneously, and pget will segment 1 file into several pieces and piece them together after downloading them simultaneously.

    Just discovered this

  30. pTomic Says:

    PurgatoryPriest, thank you for your review.

  31. PurgatoryPriest Says:

    Ok, here is my final report for now.

    Having tweaked and set a few options on both FileZilla FTP Server and Wing FTP Server, I’m able to match the speed of BitKinex using FTPRush. Funny enough, I really only tweaked a few FileZilla options. Out of the box it was ready to rock. I gained approx. 5KB/s after the tweaks to FZS. Wing FTP Server was handling certain commands a bit crazy. I matched/set what was in FileZilla and presto! Wing FTP Server is a bit overkill compared to FZS. I don’t know if I will keep it running as the main FTP server.

    But there’s more! FTPRush did not finish the download on the first day of testing. Same file that it successfully downloaded the next day using same FTP server (FileZilla). Hopefully that was a fluke. Stability wise, not as solid as BitKinex. Interface, perfect. Easy to use Options+Features that I like in FTPRush; File Comparison, Skip/Excludes, SSL to LIST/Connect but CLEAR CHANNEL COMMAND (no SSL) when wanting to transfer files are just a few. Better than BitKinex for these features and interface handling.

    Long Test with a 14.1GB File* - Results:

    1. BitKinex - x2 Connections @ 642KB/s total = 6 hours - 23 minutes

    2. FTPRush - x2 Connections @ 644KB/s total = 6 hours - 22 minutes

    *Above tests downloaded via FileZilla FTP Server software.

    Wing FTP Server software was +/- 2KB/s or 2 to 3 mins. for the file download speed & progress.

    ~Speeds measured in Kilobytes

    zebus, pTomic, glad to be of service and hopefully this will help you out and save some time.

    Until we meet again, happy segmenting! - PP

  32. some1 Says:

    Can anyone share a list of good ftp server to download stuff from. Did try http://filesharefreak.com/2009/03/31/active-list-of-ftp-dump-sites-for-warez-and-music/
    but every link I tried was dead, possible an updated list

  33. Bob Says:

    Big thanks to PurgatoryPriest for the work and time he spent.

    I’ve tried CuteFTP Pro, FileZilla, SmartFTP, FTPRush, and Bitkinex. Of those I greatly prefer Bitkinex, it’s just that good. My only complaint is their web site sucks and I fear since it’s free development will stop.

    The Multi-Stream option seems to have disappeared from FTPRush since it went freeware.

  34. TorrentFlux hosting Says:

    Thnx for the tutorial
    Regards
    http://www.rapidleechhosting.net/

  35. MasT3R Says:

    Loved the LFTP for Linux. Ok, it isn’t really nice to use CLI, but this program use Unix-like aliases so it’s easy to learn for someone who use Linux for a long time. And the speeds are great, even without using pget mode.

Comments