ISP Peer 2 Peer throttling

Snarky's picture
Submitted by Snarky on Sat, 08/06/2011 - 11:06
Forums: 
Soulseek General Discussion

I guess I am not alone in finding my Soulseek transfers are being interfered with by my ISP, now to such a degree that I do not even have to exceed the "Fair Use" amount, ANY Soulseek upload and a lot of downloads are being mercilessly crippled, my normal upload speeds of 200-300kbytes are being limited to 1-2kBytes, this is a time specific effect, weekdays 1700-0000 and weekends 1200-0000.

I must assume that they are analysing the protocol in order to do this, which leads me to ask HOW?
and can anything be done to disguise this protocol, or encapsulate it somehow. I appreciate that any Soulseek server traffic will be easily monitored as the server has a unique IP range, but surely when one
is equipped with a direct port and that port is punched through the NAT router OK, transfers can and do take place on an IP to IP basis.

I am wondering if there could be a dual protocol that could be switched to once a direct link is established
between downloaders/up-loaders.

This is my first post to this forum, I only joined today, in fact I only got told about Qt today such is the lack of information going around about it.

I know that SSL (TLS) would be the answer, but the cost of everyone having certificates would put true SSL out of reach of most, any ideas how we could overcome this increasing problem? maybe a Soulseek PROXY server with protocol translation, I'd donate to that for sure if it were possible, this is going to be a major problem specially with World time zones, there are going to be serious restrictions and very little common time when one or other user is NOT limited.

Snarky.

Submitted by Nir on Sat, 08/06/2011 - 18:16.

Hi Snarky,

Might I ask which ISP is doing this? We're aware that some providers out there are crippling Soulseek connections and we're worried that this may be happening on a rather wide scale, as we occasionally experience sudden drops in our user numbers that are hard to explain otherwise. Obfuscating Soulseek peer connections, if not outright encrypting them, is technically easy to do. For example, transmit a random one byte value at the beginning of each connection that is used as a XOR mask for every following byte. Something like that might be enough to fool a basic packet inspection scheme on the ISP's side. The bigger challenge is implementing it in SoulseekQt and maintaining compatibility with the original client. It's going to be a very long time before SoulseekQt can be considered a proper replacement to the current client, and even then I don't really want to force an upgrade. You don't want any part of the peer connection initialization protocol to be predictable, so extra identification information to allow SoulseekQt clients to find each other at the outset is not ideal. One thing I think would work very well would be to have SoulseekQt maintain a second listening port which would be communicated by the server to all other clients (but only read by other SoulseekQt clients). Connections accepted on the secondary port will be obfuscated from the get go. Still, it would be a while before you would really benefit from this as so few people are using SoulseekQt right now, but I think that's definitely something I want to do in the long run.

Snarky's picture

Submitted by Snarky on Sun, 08/07/2011 - 02:50.

Hiya Nir,
Firstly, thanks for all you are doing, the new client is V A S T L Y different from the old one, and I do wonder if such a fundamental change will put people off from moving over to it, time will tell of course, specially if this throttling does become more in evidence, Worldwide.
For my sins, I subscribe to VIRGIN MEDIA (Spit...!!) took over NTL in the UK, and although they have upgraded me virtually for free, I would prefer to have 2megbits UP on a permanent basis, not 5 megabits
for some time and then probably 100kbits for the rest. I might add here that other services like their reccomended speedtest always show top rate +- the normal busy time variations (but they would wouldn't they) Likewise SSL which I am forced to use now with TEAMVIEWER with some of my friends, JUST to use the file transfer feature unfettered, is untouched, apart from the normal fluctuations.

I think I like the ideas of a second port, that would be easy to implement through the router here, but of course not everyone is prepared or even able to mess with their port forwarding tables, specially those that are unfortunately enough to find themselves with an ISP that uses a giant NAT router and issues the user with a 10.x.x.x IP .

a Random ONE byte value, hmmm only 255, perhaps a random word would be better, and easier to XOR into the result, why not even a 16 bit hash of the users name, with a private key exchanged on a direct connection end to end. You are the man, you know more about the magic of TCP/IP than I do.

If you need a guinea pig, I'm always willing to offer to test any ideas out, if you see me on line, drop me an IM, you've got my email, that although only a throwaway for registering for things, is monitored, you'd be surprised at what turns up on there !!

Snarky.

Submitted by Nir on Mon, 08/08/2011 - 16:51.

Ah, that's good to know this is happening in the UK. We have a lot of british users, so it makes sense that something like that might visibly impact our user numbers.

There are definitely advantages to using more than one byte as a XOR mask, such as less of a correspondence between repeating characters in the original sequence and the masked result, and it might be more efficient. Of course no matter how many bytes you use as a mask, the first byte will always have a value of 0-255 and everything after that will be seemingly random :) In any case, we can always add more ports later on in case whatever method we use initially becomes ineffective. Just as a side note though, the problem with using a username hash is that it's not clear who the connection is coming from right away, so you can't ensure an unpredictable initialization sequence from the very first byte.

Anyway, this is something I'm very interested in doing, but there are definitely things I need to get to first, stuff like making the wish list work, dealing with any crippling bugs that may surface in the near future... but once I do, we can definitely do some preliminary testing with me setting the client to share and you trying to download from me during the restricted hours, and see if it's making a difference.

Thanks much, Nir

Submitted by secretsquizza on Tue, 08/09/2011 - 11:36.

Hi,

Just been reading this post - very interesting. I must say I'm quite excited (sad i know) about some security for the end user being added into the client. In my experience (I've been with o2 and virgin as providers in the UK) and o2 were able to see straight through protocol obfuscation ( I used it with utorrent and it didn't not stop the throttling). I have heard that an ISP can determine p2p traffic by analysing the patterns of the traffic, so do not require to read the protocol as they can tell its p2p by the pattern of the data flowing (no idea if this is true).

Bandwidth throttling is a very real issue in the UK and its almost impossible to find a provider that doesn't do this to some degree. o2 were terrible (total of 5k/s up and down for ALL p2p traffic from 8pm -> 12pm). Virgin seemed the best of a bad bunch as they didn't (at the time of sign up) restrict specific types of traffic only throttled after a "fair use allowance" was exceeeded. That said, i do notice that i get very slow transfers to some clients (less than a few k/s) but ALL other Virgin clients get blistering speeds at all times.... its a bit strange that.

Was interested to read about the mention of SSL and I don't think it should be written off purely on cost of certificates, as these could be created and signed for free if the slsk server was its own rootCA (i'm sure there is free certificate authority software available, although i can't remember names off the top of my head). It could then distribute certificates to all the clients that they can be certain was created by the server at registration time (if the client comes with an embedded public key from the server and a hash digest result to ensure the client hasn't been tampered with). It also would not pose an issue with old / new interoperability as old clients (that have not got a certificate) can talk on an unencrypted channel and those (new to new clients) that both have a certificate can open an SSL channel that can be cross-certified by the slsk server. I think the main cost here would be development time rather than financial expense.

Regards,

Squizz.

Snarky's picture

Submitted by Snarky on Fri, 08/12/2011 - 00:16.

Nir,
Agreed, certainly fix the main issues before adding the extras, less code to de-bug, less side roads.
Squizz, thanks for your insight, specially about O2 I was not aware they were doing it too, though it is easy to imagine they would be. I do have access to an O2 connection though this is only on a friends BTFON router, this by nature is somewhat speed limited on the free port, so never really noticed the slowing down in certain time ranges, and only use it when there is a problem on the fibre-optic connection, yes it DOES happen at times.
I also have a 3G modem on the laptop, this although generally slower (circa50k bytes) is not throttled at any times I've tried it, usually when the Virgin starts throttling, I switch over (IP-switch) to the 3G to allow friends to get a reasonable speed.
Recently I have noticed that the throttling times have changed, yesterday (11 Aug) the throughput continued right through the evening at a reasonable rate, this may have been because the downloaders were themselves restricted, I'll watch how things develop in the forthcoming weeks, I suspect Virgin may have taken notice of all the complaints, and are applying (as they should) P2P restrictions ONLY on those that exceed fair limits and not as they were, carte blanche. I cannot see me coming into the excess user category, I only allow one album or equivilent per person/ per day and usually only ever have two open slots + two on friends list, so maybe 400megs per day, hardly excessive compared to those that download whole HD movies !!

Snarky.

Submitted by secretsquizza on Fri, 08/12/2011 - 10:00.

I think o2 were only crippling the p2p traffic for their non-ADSL2 customers (eg those through the old BT backbone). I managed to cancel with them and go to virgin when they started and they didn't even argue - guess they just wanted shot of me - LOL.

Interestingly the problems i've experienced with slow transfers against non-Virgin customers has entirely dissappeared on the new client. So I'm sticking with this one! It's bloody marvellous.