An Update on Blocklists

Quite a few of you have taken issue, in the comments, about my disparaging posts on blocklists, that they don't work and never have. The arguments have boiled down to

1. If they block just one IP, it's worth it.
2. Of course they work, I've used one and never got a letter
3. How much are the anti-p2p companies paying you for this?

There are a number of problems with all these arguments. Let's deal with them in order.

1) "If they block just one, it's worth it"

OK, this has a basic assumption made about it. First, the IP that's logging has to be a) identified, and the b) put into the lists. Therein lies the problem. There is no easy way to identify a peer on a torrent that is logging, and secondly, they don't even have to be on the torrents, so it's impossible to block them.

Let's start with the second. It's trivial to get the IP and port of a user from the tracker - it's the basis of the bittorrent protocol, after all. no matter if you're running a blocklist that blocks every since anti-P2P IP address, you're still going to get a letter, because they DON'T NEED TO CONNECT TO YOU. The information on the tracker is enough to send a letter, is enough evidence for an accusation. We know that this has been used in the past, because it's been observed in practice, by the University of Washington.

How about if they do verify the users on a torrent, and participate in a swarm. Well, if you run a blocklist there, you're probably out of luck too. Logging the activity in a swarm takes no special activity at all, the standard actions of a client behaving normally is ample evidence. Why modify a client to act strangely, when they don't need to. My longtime friend, and fellow researcher recently made a video showing just how trivially easy it is to turn a client into a logger suitable for monitoring swarms, and generating log files for evidence. It can be done using any client, from the first generation of mainline, to the latest Vuze, µTorrent, or Transmission. If the client doesn't support the logging inside it, many firewalls can do the logging, or at a pinch a network traffic monitoring tool like wireshark can.

Remember the amount of money being paid here, typically a few thousand a week, per 'work', and each client can handle a few hundred. It's not difficult to hire a few people to work from home (maybe using adverts like this) and have them send off a logfile daily. $50 for an internet connection, and $50/day and that'll get you a nice part time job for a student, or 'silver surfer' looking to suppliment their money stream. 

So, your blocklists are populated with IPs for corporate ranges, yet they can, and do, use residential connection ranges (it's a lot cheaper, easier and completely nullifies blocklists).

EXAMPLE

Lets have a little example of the problem, shall we? lets say there's 1000 peers on the torrent, and one IP is doing data collection for lawsuits. The blocklists by bluetack block (let's lowball) 20% meaning 200 peers are blocked, and 800 are available. In order for your blocklist to protect you, the IP has to a) be in that 200, and b) not be just using tracker scrapes.

i) If he is using tracker scrapes, you've blocked 20% for no gain, meaning you've fewer peers to connect to (usually the faster connections are in the blocklist - funny that) and fewer to seed to afterwards, which can mean you're on a torrent longer, increasing your window of vulnerability.

ii) If he's using peer traffic logging, and in the 20%, you're safe. 

iii) If he's using peer-traffic logging, and NOT on your list, then he's in the 80% you're able to connect to.

Now, let's make the assumption you're connecting to 50 peers to download the torrent (not an unreasonable number, for a properly set up client)

In case 1, he's likely to get you (depending on the tracker) irrespective of a blocklist. It depends on the tracker's scrape settings, however the reduced ability to connect can mean you're on the torrent longer, increasing the risk if they're only doing partial scrapes

In case 2, the odds have gone from 50:1000  (5%) to 0:800 (0%) if you use a blocklist.

In case 3, the odds have gone from 50:1000  (5%) to 50:800 (6.25%)

As you can see, using a blocklist can increase your risks, if the peer logging isn't on the list, but how likely is that? Very likely. There is a false belief that antiP2P companies can not participate in a swarm. They can, because part of the contract generally involves giving them a limited license to distribute. It's not entrapment (it's nothing like entrapment) as often claimed, nor is it authorising you, since you have no license to distribute, that they do makes no difference. The second common myth is that they must use business connections. I'd love to know what idiot came up with that, but it's also false. They can use any connection they want. They also have access to the lists (they can download them just as you can). So, let's employ a little role-play. Say you're the head of an antip2p company, and the IP address you've been using has appeared on the blocklists, what do you do, can you think of any way around them? Personally, I'd just change my IP. 

That's the most basic problem with the blocklists - it's easy to find out the contents of them. If the contents couldn't be found out (impossible) they might work, but since all the company has to do is load it onto a computer, and try pinging it's servers, it's ultimately futile.

Anyone with half a braincell, and a few thousand dollars can avoid being on blocklists at will. AntiP2P companies have plenty of money, and despite what people think, they've certainly got braincells (they're charging thousands for very simple services - that takes brains).

2) Of course they work, I've used one and never got a letter

OK, this might be true, but the majority of people out there never have either. The abundance of letters is ENORMOUSLY overstated by both the anti-P2P groups, and those behind blocklists (and some claim they're one and the same) The chances of getting a letter are similar to that of being struck by lightning. There's also a little fact those behind the blocklists don't want you to know. Even running a blocklist, people do get letters. 

The problem is, there's no evidence that blocklists have ever protected anyone. Loggers are not on every torrent (in fact, they're on very few) so "no letter" can (and generally does) mean "no logger" rather than 'protected from logger'.

There is a simple analogy for this argument. Take an umbrella, stand outside, and open it up. Done that? Have you been struck by lightning? No? So the umbrella protected you. Sound foolish? Replace umbrella with blocklists, going outside with torrenting, and being struck by lightning with getting a letter. It's the exact same argument.

Most of the time you're outside, it's not a thunderstorm, just like most of the time, there are no loggers on your torrent. To claim the absence of loggers, as a victory in protection for the blocklist is dishonest at the very least (and if you have to lie about the effectiveness like this, that means you have a bad product)

3) How much are the anti-p2p companies paying you for this?

Nada, zip, bupkis. Why would they pay me? I'm not telling anyone anything that's favourable to them. In fact, I'm highlighting a scam that works to their benefit. If anything, they should be paying the blocklist companies (or maybe they already are!). It's the same reason the cops aren't a great fan of people knowing they're allowed to lie, even if you directly ask them 'are you a cop', because the false sense of security of people who think they're protected (but are really wide open) makes them ripe for targeting.

Here's something to ponder. I've been doing reporting on this subject for 5-6 years now. In that time, no-ones been able to point to any of my articles, here or on torrentfreak (or piracyisacrime before that) and call it a lie. I've never shied away from debates, and I'm perfectly willing to debate facts, figures, and talk to people. Why is it that those that support (and run) the blocklists are unable to do the same. They have no rational arguments, data, theories or anything else.

Here's a new theory.

What if the people behind blocklists, such as Bluetack, are actually AntiP2P companies?

It's not that much of a stretch. WE've seen that in practice, their blocklists were significantly worse than even filling the lists at random would account for, when compared to the ONLY verified source of IPs used for AntiP2P attacks. They aren't much in the way of transparency for themselves, yet demand it of everyone else. No-one know's the real identity of the people that work for BlueTack, except for the people that work at BlueTack. They're also hypocritical when things don't go their way. As I pointed out last time when ESET's NOD32 marked programs using their lists as potentially unwanted, they threw a fit, and got people to spam ESET. It is a simple action to have Nod32 'ignore' such programs. It started because they listed Nod32 update servers in their blocklist. When this was pointed out, they said 'well people can just chose to ignore that rule and select allow' - the EXACT same argument and action they were so against in Nod32. AV programs have a proven track record at preventing a proven threat. The blocklist by BlueTack - no evidence it's ever worked.

If you run a blocklist, consider the source of it. Do you really know who is running it, where the funds for it are coming from, and how they populate their lists. They demand to keep such information secret 'for security purposes' yet it's been proven time and time again that these security measures aren't infalable. In fact, they're openly scornful of the 'security by obscurity' approach, as they have pointed out at lenght, such as back in 2006 when they were demanding utorrent be open source, after their first (completely inaccurate) attack on it.

Conclusion

Blocklists are easy to circumvent, and have never been proven to work. Every single study that has looked at the effectiveness of them has ranged from  'doesn't make a difference' to 'doesn't work'. They don't like to mention those. They do like to mention one though, called The P2P war: "Someone is monitoring your activities!" by Anirban Banerjee, Michalis Faloutsos, and Laxmi Bhuyan. You can read a copy here. There's just one problem with this study, summed up by this sentence in the introduction.

Note that it is not our intention here to examine how accurate and comprehensive these lists are, though this would be interesting and challenging future work. 

The paper actually boils down, not to evaluating the effectiveness of blocklists, or blocklist software, but "how likely is it you'll connect to a peer on the lists" and "if you have a list comprising X peers, will this program block X peers" The 'surprise' answer is 'very' and 'yes'. Really, the first is a matter of basic maths, the second is just simple list-matching. It's a paper worthy of a 14 year old. I'll do a detailed analysis of it another time, but let's handle it quickly.

Let's say, again, the lists comprise 20% of the net. Let's also say you're connecting to 50 peers. The odds of not connecting to anyone on the list is thus 80%, and to be clear, you have to do that 50 times.  0.8^50 = 1.42x10^-5 (0.0000142, or 0.00142%). The real figure's more like 30% blocked, so it's a lot lot rarer, but even if they only blocked 2%, the rarity of not having ONE IP hit it is 36%

Regardless, there is not a single bit of evidence ANYWHERE that blocklists work to keep you 'safe'. Of course, if they have some evidence, I'm more than willing to see it and accept it. I've only been asking to see it for the last 5 years after all.

The Exception

There is one way, and only one way, that blocklists are effective. When dealing with swarm disruption techniques, blocklists can be effective. Certain antiP2P companies (such as MediaDefender) specialised in fake torrents, and disrupting swarms. Clients would get on torrents and purposefully send bad data to peers, to waste their bandwidth and annoy the user. Or fake torrents would get uploaded to sites, which never finished, or were corrupted when finished, discouraging people. Good moderation dealt with the later, and improvements in clients have dealt with the former, meaning it's not longer a major issue. Many clients now have automatic banning, and adapt to lots of bad peers close together. The advantage over a blocklist is easy - Instead of identifying the IP, submitting it, and then getting it in a list the next time it's updated, your client deals with it there and then.

You also don't get IP's put on lists for ever (it seems) because of a small problem. Bad data can come from packet corruption caused by firewalls, by hard drives, cpus and ram starting to fail, or a poor quality connection. Blocklists would list the IP as 'antiP2P' and it'll never be released.

Should you run a blocklist? Personally, no, I don't, and never will (except when analysing their behaviour). They're ultimately pointless from a technical perspective (they just can't do what they claim), as well as sinister and shady from the personal perspective. How anyone can trust the quality of information from such an unknown source is amazing, especially one with a track record of refusing to answer any sort of question, and being as closed and secrative as the people they purportedly protect you from.

Of course, should Bluetack, or anyone else associated with blocklists like to debate this, I'm perfectly happy to do so, in a nice public neutral debate. I say 3rd party, because every time I've come to you with questions, you've acted like Bill O'Reilly in the famous clip of his debate with Jeremy Glick (also, I believe, the experience of many others)

All I can say is, if they work, where's the evidence? Where's the proof that the lists work, that the lists are in any way accurate. Most importantly, Why should anyone trust you for their security, since no-one knows who you are? Follow your own calls, open source your methods, your information, because right now (and for the last 5+ years), you're acting guilty as hell, with something to hide.

If you have any facts, bring them, I'm happy to learn. If all you have are insults, and logical fallacies (like the ones above), then do yourself a favour - keep them to yourself and avoid looking stupid! If you just want to throw insults, you're clearly not afraid of looking stupid, since it's a lifestyle choice for you.

I'll leave the last word to the University of Washington study, who got many hundreds of DMCA letters while running a blocklist.

We have further demonstrated that IP blacklists, a standard method for avoiding systematic monitoring, are often ineffective given current identification techniques and provide only limited coverage of likely monitoring agents.