Ever been really angry at your BitTorrent downloads being uber slow? Chances are nobody ever told you about what upload actually does in your connection.

The TCP/IP Request

Quite simply, the root cause of most BitTorrent problems is uploading. Yes, although people will usually look at problems in the download area, which is logical because it’s where the problem is showing up, the real problem is actually upload.

To understand why, all you have to do is understand how downloads actually work. When you download a file off the Internet, anything, whether it be a movie, an email attachment, whatever, it is transferred in packets. Those packets form the basis of the TCP/IP model, the TCP/IP being the name of the family of protocols that make your networking work.

Packets are essentially small pieces of your file. The reason packets exist is highly technical, but put simply, the Internet’s resiliency is in large part thanks to packets, because if one fails, you simply have to resend it, and it’s no big deal because individual packets are small.

In fact, the verification of the integrity of your file is done thanks to packets. If a packet is loss, it’s simply resent, and so your transfer won’t fail because of a simple connection problem. Conversely, transferring files from two hard disks is much less resilient, as one mistake or problem will make the transfer fail and render the copied file unusable. This packet-based nature of the Internet allow more sophisticated browsers like Firefox and Opera to pause and restart downloads halfway when they fail, and Firefox can even restart a download halfway after having closed or forcefully shutdown. Note all three of Internet Explorer, Safari and Chrome possess no such ability. If you’re downloading lots of files through your browser, I recommend using Firefox for better assurance that your downloads will get through.

When you download a file, your computer has to constantly send messages back to the server from where it’s downloading the file in order to receive the proper packets. For example, if a packet fails to be received, the browser will simply ask for that packet again. If the connection is lost, a smart browser like Firefox can also re-establish the connection without having to re-download the whole file.

This explains the mostly unknown fact that when you download something, you’re also uploading data.

Why are your BitTorrent downloads so slow?

Except for cases where your ISP (Internet Service Provider) might be actively throttling BitTorrent downloads, where there is little to do in most cases (encryption might not actually save you because the throttling is behavior-based, e.g. sandvine hardware installed at Bell Canada), the root cause of your BitTorrent transfers being slow is because their upload speed is not limited.

Unlike an HTTP connection, when you download a file via BitTorrent, you also upload back parts of that file to a bunch of people that are also downloading the file. This sharing phenomenon is what allows torrents to live without a central server, but it’s also the root cause of speed woes. That’s because when you download a file, you constantly have to ask the server, or in this case the other files owners over the P2P network, for pieces of the file.

This doesn’t take a lot of data in normal situations like an HTTP-based download (stuff you download in your browser). But in BitTorrent software the measures of use can be grossly exaggerated because you will be downloading the file from regularly over 40 people simultaneously instead of just one. In traditional downloads, the average required upload speed is something around 1/50th and sometimes less of your connection, which means a 50 Mb/s download and 1 Mb/s upload connection, like what the top ISPs offer in Canada, is just enough to consistently download a single file at full speed.

Of course, torrents fare on a much lower level, and it’s very hard with such connections to attain full speed with torrents, unless one of your P2P friend has a crazy upload speed. The other problem on top of the fact you have to upload messages to over 40 clients acting as servers at the same time is the fact that BitTorrent transfers often share the file while downloading it.

This means that on top of the 1/50th of your connection’s download speed, and on top of the affected ratio by torrents, you also have to add a diminished total upload bandwidth. Some sophisticated clients like μTorrent have the ability to automatically limit your torrent upload speed, although the feature does not cover the messages you send to a server of course, and neither is it enabled by default for other good reasons.

When you let a torrent download with full upload speed, once you have downloaded enough of it to share it, it will usually grind to a super slow download speed and only upload, because the practical totality of your upload bandwidth will be dedicated to sharing with others. You’ll still download, but what’s happening is that the torrent software will be fighting for bandwidth in order to share and for bandwidth in order to message download servers for more packets. The torrent sharing uploads usually take more requests, hence they tend to preside over upload messaging and end up slowing your download speed considerably.

The solution is to limit the upload speed of your torrent. The difference should be practically instantaneous. If it is not, your torrent file might not be popular enough to have enough peers to achieve high download speeds. More sophisticated BitTorrent applications like μTorrent can also limit the sharing speed by individual torrents and globally.

Note that the same happens when you upload a file. You then also need to download the messages sent to you by peers requesting more packets. This behavior can be observed with a bandwidth monitor utility such as iStat Menus for Mac OS X. Note that bandwidth utilities are required to observe this despite torrent software offering speed statistics, as most of them do not take messaging bandwidth as part of their metric.

Why does the rest of my network grind to a halt when I download torrents?

The problem is twofold.

First off, it is exactly the same problem as previously mentioned. If your torrent operations take the entirety of your network’s upload capacity, regular operations, including yours, will grind to a halt. Routers have no problem balancing HTTP requests between users, but the sheer level of simultaneous connections required by BitTorrent transfers far outpaces any home router’s ability to properly handle balancing.

The other reason behind the upload problem is that in general, routers simply give the bandwidth in proportion to the number of requests. That is, if you’re downloading two items and your relative is downloading one at the same time, you will rightfully have two thirds of the network speed, because you occupy two of the three total simultaneous active connections to a server in your house. Torrents on the other hand can easily garner, under regular BitTorrent software settings, around 200 simultaneous connections to servers, or so-called peers, literally making your router unresponsive to single-connection requests like loading a web page.

This can be substantially minimized if you limit your torrent upload speed, or even better limit the number of simultaneous torrents downloading together, but at the cost of downloading speed for your torrents.

However, despite efforts such as limiting upload speed, you might find your network to still be slow, or worse, you might find your network to be totally unresponsive after downloading a torrent file, despite having shut down the BitTorrent client. Lots of people blame their ISP for the slow speed, and then get asked by the technical support representative if they download torrents, and for very good technical reasons.

The technical reason is precisely the second problem with BitTorrent and the rest of your network; Routing Tables, and it’s on a higher technical level. It’s probably the number one ignored fact about networks that grind to a halt everywhere.

What’s a Routing Table?

When you access the Internet, you communicate with servers with the help of IP addresses. An IP address, or Internet Protocol address, is what identifies you and other computers like servers on the Internet.

A routing table is special memory in your router dedicated to storing a multitude of information regarding the route to a particular network destination. This so-called destination is an IP address with which you communicate. The inner working of a routing table also tell which computer is asking for what. So in order for multiple computers to use the same network, each computer’s connections must be stored in the routing table.

In a traditional home network, this doesn’t usually cause problems, as computers make very few simultaneous connections. Hence, the routing table is never truly filled up, and the router has the time in-between connections to flush the data contained in the routing table, making place for new connections.

And then you have the BitTorrent protocol, which by itself makes significantly more simultaneous connections than regular network activity, easily filling up the routing table of a home router. Once filled up, the router has to flush various entries in order to make space for more connections. However, torrents constantly ask for more connections and can some time fill up a routing table to the point the router cannot flush any connection because they are all legitimately active, leaving no space at all for any other traffic on the network.

This explains why your network might “crash” and become unable to connect to even Google, while torrents still work. It also explains why chat software like Skype that make use of P2P instead of central relay servers continue to work in such events, because they simply have to maintain that one active connection spot on the routing table.

Hurray, network mystery solved! When this happens, one solution is to wait for a short while. Competent routers should be able to flush the routing table, but some older machines, notably the very common blue and black Linksys boxes, take considerably time to flush any entry in their routing table, or seemingly don’t flush them at all. The other solution which fixes even incompetent router is to completely turn off the machine, and then turn it back on, which should in theory completely flush the entire routing table, putting back your network to a healthy state.

If you’re experiencing this problem daily without even downloading torrents, you just might be too many people on the same router. This problem is getting more and more common in small businesses that connect to a router via WiFi. In traditional systems, home routers would only have 3 or 4 Ethernet ports, obligating businesses to upgrade routers if they wanted to put more people on them. WiFi routers have no such limitations but are nonetheless constrained by their routing table. Most home routers, WiFi or not, were only designed to handle up to four clients, and even less with torrents, so it’s not surprising WiFi woes are common amongst small businesses that can’t afford expensive network equipment or a personal IT department. Recent, more expensive routers, and imposing strict policies on the use of torrents, can help solve that problem.

However, if your network is going to serve anything more than a dozen people, contacting a sales person in a business routing company like Cisco might be a good idea.