Zone Transfers on The Alexa Top 1 Million

At work as part of every assessment we do a some reconnaissance which includes attempting a DNS Zone Transfer (axfr) and conducting a subdomain brute force on the target domain/s. The subdomain brute force is only as good as your wordlist, the Zone Transfer is a matter of luck.

Alexa release a list of the top 1 million sites which is updated on a daily basis. To create a better subdomain wordlist to conduct subdomain brute forcing I attempted a DNS Zone Transfer against the first 2000 sites in the Alexa Top 1 Million list. With every successful Zone Transfer the DNS A records were stored in a CSV file.

This was all done using Carlos Perez’s dnsrecon DNS enumeration tool. Dnsrecon was ever so slightly modified to only save A records, apart from that I just used a bash script to iterate over the Top 1 Million list and ran dnsrecon’s axfr option for each site with CSV output enabled.

The Results

A nice side effect to creating the subdomain wordlist is knowing how many DNS Name Servers have Zone Transfers enabled and which sites. Out of the top 2000 sites, 98 had at least one Name Server with Zone Transfer enabled (4.9%). This included sites we all know and/or use such as Pingdom, Mega Upload, Spotify, Gravatar, American Express and 93 other sites in the top 2000. Some of these sites may have Zone Transfers enabled on purpose, the majority probably don’t know it is enabled. The full list of domains with Zone Transfers enabled and their Alexa Ranking can be found here – http://ethicalhack3r.co.uk/files/misc/axfr_domains.txt

Top 10 Alexa domains with Zone Transfers enabled:

Rank,Domain
7,wikipedia.org
87,about.com
119,livedoor.com
120,weather.com
147,kickass.to
156,wikimedia.org
173,liveinternet.ru
194,goo.ne.jp
216,ehow.com
233,hardsextube.com

In total there were 55,450 A records gathered from the 98 sites. After sorting the list of subdomains by the number of sites each subdomain was found on, removing any duplicates (some sites listed more than one of the same subdomain) and removing subdomains that were only found on one site, the final subdomain list consists of 859 lines. The final list including the number of instances each subdomain was seen across the 98 sites can be found here – http://ethicalhack3r.co.uk/files/misc/subdomain_count.txt

The top 10 subdomains were:

54 mail
47 www
35 ns2
34 ns1
28 blog
26 localhost
25 m
23 ftp
19 mobile
16 ns3

The ns2 subdomain is apparently more popular than the ns1 subdomain which is unexpected. The localhost subdomain seemed to always point to the localhost (127.0.0.1). The mail subdomain was the most popular subdomain overall.

And finally, the subdomain wordlist itself sorted by popularity can be found here – http://ethicalhack3r.co.uk/files/fuzzing/subdomains.txt (859 lines). I would recommend combining this list with the list you’re already using for the best results.

And this is the code used to sort the dnsrecon CSV output files:

#!/usr/bin/env ruby

require 'public_suffix'
require 'uri'

results = `cat axfr_results/*.csv`
output = Hash.new(0)
already_seen = []

results.split("\n").each do |line|
  domain    = line.split(',')[1]
  if ! already_seen.include?(domain)
    already_seen << domain
    subdomain = PublicSuffix.parse( domain ).trd if PublicSuffix.valid?( domain )
    output[subdomain] += 1
  end
end

Hash[output.sort_by{|k, v| v}.reverse].each_pair do |key, value|
 next if key == '@' || key == '*'
 puts "#{value} #{key}" if value != 1
end

The next step if anyone has the time and resources is to conduct the test against the full top 1 million list. The top 2000 took about 12 hours or so.

7 thoughts on “Zone Transfers on The Alexa Top 1 Million

  1. Bill E. Ghote

    I tested zone transfers of gTLD’s back in May, 2013, and found 40 top-level domains allowed it, including .ARPA.

  2. Matthijs

    @Bill: ah yes, in the 2011 test, 48 tld’s where AXFRable: al, an, ao, arpa, aw, bb, bi, bj, bm, ci, cr, cv, ec, ga, gd, ge, gl, gn, gp, gq, gt, gy, int, jm, jo, kg, lk, lr, mc, mo, ne, pf, pg, pk, pro, pw, sc, sk, sl, sn, sv, sz, tc, tj, tn, to, vg, vi.

  3. แทรมโพลีน

    You should of course start off slowly and build up your
    intensity, but there is really nothing new here to learn. Everyone wants to get high
    quality parts for trampoline. Usually, they
    are not able to carry out their daily tasks by themselves without anybody helping them.

  4. http://www.gnkg.pl

    Furthermore, my experience comes with identifying the competitive advantage
    for any few different companies and crafting
    that advantage into a online strategy to develop home based business.
    The simple truth is, people join MLM opportunities as a result of who introduced them.
    If you’re still at a loss, you can contact the client care team either by email, live chat, or phone during standard west coast
    business hours.

    Here is my web-site: http://www.gnkg.pl

Comments are closed.