For the last week or so I have been using my ISP’s DNS servers. Nothing unusual in that of course, except normally I prefer not to, I find that an internal DNS server with a reliable upstream server that I trust makes more sense and it’s generally a little bit quicker.
In any case, Virgin recently announced a new “Advanced Network Error Search” a system whereby erroneously entered URL’s, rather than returning the relevant DNS response code to such a search (whether it be unassigned or otherwise…) returns a nice neat record, not related to the search, but rather a redirect to virgins own, Yahoo powered search pages.
Virgin’s own description of the service is fairly transparent:
We all make mistakes when we type in website addresses. Perhaps we miss a few letters, or the website doesn't exist any longer. If an address you enter doesn’t locate a site, this handy feature will convert the incorrect address into a web search, so instead of an error message you will get a list of our closest matches, plus some additional related links."
Obviously for certain users this makes a certain amount of sense, but given that not everyone wants to be faces with a list of search results when entering a URL, or indeed may have applications that depend on the correct response to function properly, Virgin quite sensibly offer an opt out. More than that it is apparently a proper opt out, those that opt out get the proper DNS responses (rather than a cookie based mess returning a false page for the browser but a valid DNS response…).
So I opted out. That is I ticked the box that said “No – I would not like to use the advanced network error search”. Fairly clear, or so I thought…
Opt Out is shown as being active.
All has been well since then until today, I should be clear that nothing has changed (honest) my cable modem not rebooted since the initial opt out and nothing else should matter. Indeed on virgins own opt out page, the opt out is still showing as active.
However as of today, I have started to see virgins search pages when, on occasion, I have entered a malformed URL. Initially it was inconsistent, sometimes I would, sometimes I wouldn’t, but as of 1900 (GMT) every such request has plonked me in front of a virgin search page. It should be said that the results aren’t exactly wonderful either.
The result when mistyping reddit.com (reddit.cm)
Just to be clear, my DNS servers haven’t changed. They are 194.168.8.100 and 194.168.4.100. I have now rebooting my router, curious to see if I would receive a different set of servers, but alas no, they remain the same (and as I said my opt out is still showing…). I even tried to opt out again, but alas, as I’ve already opted out, I can’t do that again…
Once opted out, you can't opt out again. Which makes sense.
So lets see what is happening.
Process.
I thought I would take a quick look to see if anything unusual was happening, making a valid and then invalid request whilst sniffing my traffic with wireshark didn’t indicate anything nefarious (or at least nothing I hadn’t expected) but it does let me describe the process that is occurring:
First, as a user types in address into search bar, in my case reddit.cm, a DNS request is made to virgins DNS server (in this case 194.168.8.100). Essentially what happens is that my machine goes and asks a question of the DNS server I have specified. It asks just one question, can the DNS server give it the Host address for reddit.cm. Given my opt out I should get …. However I don’t. I get a non-authoritative response from Virgins DNS server informing me that a valid host address for reddit.cm is 81.200.64.50. Obviously it isn’t.
I can check that using nslookup:
username computer:pts/1 (~)
Wed,16 Sep @ 19:07 $ nslookup reddit.cm 100%(1:55:44)
Server: 194.168.8.100
Address: 194.168.8.100#53
Non-authoritative answer:
Name: reddit.cm
Address: 81.200.64.50
This works for anything that looks like a valid domain (just in case anyone has doubts about the existence of reddit.cm…:
username computer:pts/1 (~)
Wed,16 Sep @ 19:09 $ nslookup www.this.is.not.a.real.network.address.really.it.isnt.but.i.will.still.get.a.valid.ip.from.virgin.dns.servers.even.with.my.opt.out.com
Server: 194.168.8.100
Address: 194.168.8.100#53
Non-authoritative answer:
Name: www.this.is.not.a.real.network.address.really.it.isnt.but.i.will.still.get.a.valid.ip.from.virgin.dns.servers.even.with.my.opt.out.com
Address: 81.200.64.50
In fact if we do a quick reverse whois of the address returned we find, as expected that it is in fact the IP address belonging to advancedsearch.virginmedia.com.
username computer:pts/1 (~)
Wed,16 Sep @ 19:07 $ nslookup 81.200.64.50 100%(1:55:44)
Server: 194.168.8.100
Address: 194.168.8.100#53
Non-authoritative answer:
50.64.200.81.in-addr.arpa name = advancedsearch.virginmedia.com.
Of course what should be happening assuming the opt out worked… is the following (which is what my non-public DNS server supplies) :
username computer:pts/1 (~)
Wed,16 Sep @ 19:30 $ nslookup reddit.cm 172.16.200.2 99%(0:01:47)
Server: 199.7.83.42
Address: 199.7.83.42#53
Non-authoritative answer:
*** Can’t find reddit.cm: No answer
So back to the process Virgin is using to get us to their search pages and return relevant results:
Now that my machine thinks it knows where reddit.cm is, it can proceede, first establishing a TCP connection, then sending a HTTP GET request. The request goes something like this:
!/GET / HTTP/1.1
Host: reddit.cm
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.1.3pre)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
The virgin server that recieves it knows that I am expecting to go to reddit.cm, it is in the GET request, it can now pass me a lovely page with Yahoo search results relating to the term (no doubt making virgin a little cash per request). And it returns its search result. Here is one for debian.og.
A search for debian.og gives the same search page albeit with slightly better results.
Tech support
One thing I will say is that Virgin have decent first line tech support, it may not be able to solve your issues, but the guys and girls working for Virgin appear fairly informed and polite. When I called I the technician I spoke to was decent enough to state that he had no idea about the issue and that he hadn’t seen it before and that he was in fact very interested to find out what was going wrong. That made two of us.
So a quick escalation to Virgin’s second level of support was called for. Sadly that was as far as we got. After the initial comment that Virgin don’t support Linux, we decided that I knew how to restart my machine, flush my DNS cache and check to see if my DNS servers were correctly configured and that I wasn’t an idiot and fairly well clued in, so my comments and thoughts on the matter could be used, it was determined that no one had any idea what was going on.
Basically, Virgin would look into it, I should call them back in a couple of days if the problem persisted. I informed the first line tech that I would do so and in the meantime I would switch DNS servers, so my problem at least was solved, even if the larger issue wasn’t.
End Result,
Of course Virgins “Advanced Network Error Search” isn’t unique, many ISPs offer such a service and many won’t let you opt out (we will see where this goes with Virgin of course…). Indeed virgins search pages are considerably less objectionable than others (if we ignore the general breakage…). ‘DNS made easy’ has a similar system with a rather less slick end result (see http://205.234.170.218/).
username computer:pts/1 (~)
Wed,16 Sep @ 19:30 $ nslookup reddit.cm 205.234.170.215 99%(0:01:47)
Server: 205.234.170.215
Address: 205.234.170.215#53
Non-authoritative answer:
Name: reddit.cm
Address: 205.234.170.218
Even openDNS a popular alternative to ISP DNS servers amongst the more technical implements the same system, presumably to get the revenue required to provide a free DNS system in the first place. So this kind of quasi DNS-hijack is common but not yet universal, the more memorable DNS servers (such as 4.2.2.1…) and others I can remember (like those at blueyonder, or whatever it is now) still do things properly, of course so your mileage will vary.
Thanks and excuse the ramble. I will update this in a few days when I have more, hopefully either a fix or a response from Virgin.
