Tag Archives: GFW

Securing DNS Traffic in China

Overview

DNS poisoning is one of the most common cause of nuisance when accessing websites
that are outside this 1.4 billion-people Oriental country. So far, the best way to protect yourself from this trouble is to route all your DNS traffic through an encrypted channel, and the method I am going to introduce is DNSCrypt. There is not yet a standard for encrypted DNS, DNSCrypt is a project done by OpenDNS. According my experience, DNSCrypt is very reliable and robust, the cryptography of the protocol is called DNSCurve, which is a public-key crypto that employes an extremely strong elliptic-curve cryptography called Curve25519.

If you have read my previous writing, you should know my setup is a Raspberry Pi, and so the rest of this article is based on that, running Raspbian. Dnsmasq will be used as the first DNS caching proxy to serve incoming DNS queries from machines on the network. If the queried domain name is a China one, the request will be served by a China DNS. This is necessary because for some domains, answers from DNS servers in China and global ones could be different. If the requested domain does not belong to any known China domains, the request will be forwarded to dnscrypt-proxy, which will ask a DNSCrypt server for an answer.

After DNSCrypt is used, your DNS traffic will look like this:

                                             +----------------------+
                                             |   China DNS server   |
                                      +----> |                      | 
                                      |      | e.g. 114.114.114.114 |
                                    China    +----------------------+
                                   domains
                                      |
                                +-----|---------------------------------+
                                | +-------+   Other   +----------------+|
       +------+                 | |dnsmasq|---------->| dnscrypt-proxy ||
       | Host |-- DNS query --> | +-------+  domains  +----------------+|
       +------+                 |                              |        |
                                |            Raspberry Pi      |        |
                                +------------------------------|--------+
                                                               V
                                                       +---------------+
                                                       |DNSCrypt server|
                                                       +---------------+

Setting up DNSCrypt

As illustrated in the above diagram, dnscrypt-proxy is the piece of software that handles DNSCrypt, but it is not available in Raspbian’s Wheezy and Jessie releases, only in testing (currently Stretch). You can either compile it yourself, or grab the debian package I built and install it. You can find the package here. It is based on the Raspbian package in testing repo, with some modification to debian packaging files, since the one in testing depends on systemd, which had not yet been adopted when Wheezy was released.

If you really want to build the package yourself, first install the libsodium packages. The package are also not available in Wheezy repo but the ones from testing, libsodium13_1.0.3-1_armhf.deb and libsodium-dev_1.0.3-1_armhf.deb, can be installed without any problem. Download and install them, then follow these steps to build your dnscrypt-proxy package:

$ sudo apt-get install autotools-dev debhelper pkg-config
$ git clone https://github.com/anthonywong/dnscrypt-proxy-raspbian-wheezy.git
$ cd dnscrypt-proxy-raspbian-wheezy.git
$ fakeroot debian/rules binary

After dnscrypt-proxy is installed, you have to update the port it uses. Change DNSCRYPT_PROXY_LOCAL_ADDRESS in /etc/default/dnscrypt-proxy to another port other than 53 (as it will be used by dnsmasq later), like this:

DNSCRYPT_PROXY_LOCAL_ADDRESS=127.0.0.1:5353

You can also change the remote DNSCrypt server, but since the default (cisco) works well for me, I left it unchanged.

Now test it to make sure it works as expected:

$ dig @localhost -p 5353 www.facebook.com

; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> @localhost -p 5353 www.facebook.com
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9735
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.facebook.com.      IN  A

;; ANSWER SECTION:
www.facebook.com.   2725    IN  CNAME   star-mini.c10r.facebook.com.
star-mini.c10r.facebook.com. 47 IN  A   31.13.77.36

;; Query time: 140 msec
;; SERVER: 127.0.0.1#5353(127.0.0.1)
;; WHEN: Mon Jan 18 00:39:36 2016
;; MSG SIZE  rcvd: 90

$ dig @localhost -p 5353 www.sina.com.cn

; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> @localhost -p 5353 www.sina.com.cn
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22371
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.sina.com.cn.       IN  A

;; ANSWER SECTION:
www.sina.com.cn.    2733    IN  CNAME   jupiter.sina.com.cn.
jupiter.sina.com.cn.    3580    IN  CNAME   region.sina.csglb.txcdn.cn.
region.sina.csglb.txcdn.cn. 1261 IN CNAME   n2wous.panthercdn.com.
n2wous.panthercdn.com.  20  IN  A   103.4.200.227
n2wous.panthercdn.com.  20  IN  A   103.4.200.235

;; Query time: 125 msec
;; SERVER: 127.0.0.1#5353(127.0.0.1)
;; WHEN: Mon Jan 18 00:38:25 2016
;; MSG SIZE  rcvd: 171

Setting up dnsmasq

Dnsmasq is very common and is available in Raspbian, installing it is easy:

$ sudo apt-get install dnsmasq

Now we have to do some configuration in /etc/dnsmasq.conf. These are my recommended settings. Please note that the interface option is the network interface that dnsmasq will serve, and in my case that is wlan0. You have to change it to the one that applies to your case.

domain-needed
bogus-priv
no-resolv
interface=wlan0

Now comes the interesting part. We are going to tell dnsmasq to use a China DNS server (114.114.114.114 in my example) for China domains and DNSCrypt server for all others. This is done by using the server option in /etc/dnsmasq.conf. Here is an example:

# Add other name servers here, with domain specs if they are for
# non-public domains.
server=/baidu.com/114.114.114.114
server=/sina.com/114.114.114.114
server=/ifeng.com/114.114.114.114
server=/phoenixtv.com/114.114.114.114
...
server=/deppon.com/114.114.114.114
server=127.0.0.1#5353

This is pretty straightforward. The last line tells dnsmasq to use your dnscrypt proxy if the domain you query does not match any China domains. In my config file there are 12238 lines for China domains so I'm not going to post them all here, you can get the snippet of my dnsmasq.conf here, and put it into your own dnsmasq.conf. The problem is to maintain the list for all China hosts. I am now using the list from the fqrouter project, it has been serving me well, since most common domains are already there. What's worrying is due to the abandon of the project by it's author, the list is now unmaintained. If you know a more updated list, please let me know!

Obfuscated SSH tunnel

VPN providers were cracked down, open source anti-censorship tools were eliminated. This is what’s happening in China and has become even more severe than ever. Shadowsocks alone is no longer reliable due to more powerful deep packet inspection implemented at the GFW.

I am now replacing shadowsocks on my gateway with obfuscated SSH tunnel, based on Tor‘s obfsproxy. To the impatient ones, I will first give a concise summary of the necessary steps of my set up. You can follow it without drilling down the details. I will explain in more details later. But please note that you have to follow the other instructions in this blog post to complete the whole set up.

Quick Set up

On your server

Assume your server runs Debian 8 (jessie) or Ubuntu, and its IP is 1.2.3.4, run these commands:

$ sudo apt-get install obfsproxy
$ mkdir /tmp/obfsproxy-temp
$ obfsproxy --data-dir=/tmp/obfsproxy-temp scramblesuit --password=QWLY42YHB75J3B57XOKYNNFJPOQ7APHL --dest=127.0.0.1:22 server 0.0.0.0:7700
On Raspberry Pi gateway
$ sudo apt-get install python-dev
$ sudo pip install obfsproxy

Edit ~/.ssh/config:

Host 1.2.3.4
  ProxyCommand nc -X 5 -x 127.0.0.1:3333 %h 7700

Run these commands:

$ obfsproxy --log-file=obfsproxy.log --log-min-severity=info scramblesuit --password=QWLY42YHB75J3B57XOKYNNFJPOQ7APHL socks 127.0.0.1:3333
$ ssh user@1.2.3.4 -N -D 1080

Deploy Shadowsocks on Raspberry Pi with ChinaDNS and Redsocks

Assume you already have a Raspberry Pi configured as a WIFI router like mine shown below, but you live in China and have to deal with the fact that many websites can’t be accessed due to GFW. Don’t be despair and with some hacking you can get your Internet freedom back.

IMG_20150714_134341

The mechanism is to use shadowsocks on your router which directs any traffic to a shadowsocks server in the free world. It’s simple to get it up and running on a local machine, but on a router you need to use redsocks to redirect traffic to the shadowsocks client running on your Raspberry Pi. DNS traffic has to be routed by redsocks as well otherwise your DNS replies will be contaminated. To remain as fast as normal when accessing China websites you also need to skip routing traffic to Redsocks for anything within the China IP ranges. Even if you don’t care about performance, this is still necessary in some circumstances like geoip restriction such as tv.sohu.com does not deliver contents if you live outside of China.

Finally, we want to improve the performance even further by using ChinaDNS. To avoid DNS poisoning, we can always resolve DNS over our secured shadowsocks connection, but this is not optimal if a China website have CDNs outside China. ChinaDNS queries local DNS servers to resolve Chinese domains and queries foreign DNS servers to resolve foreign domains, and from my testing it is useful to avoid DNS poisoning with the “DNS compression pointer mutation” option. (Update: I have switched from ChinaDNS to dnsmasq+dnscrypt, please read Securing DNS Traffic in China to see how it works.)

Shadowsocks

I assume that you have shadowsocks server running on a public server, so I will skip that part and only talk about the client side.

Installing shadowsocks is very simple, note that it will be installed under /usr/local/.

pi@raspberrypi $ sudo apt-get install python-pip
pi@raspberrypi $ sudo pip install shadowsocks
Downloading/unpacking shadowsocks
  Running setup.py egg_info for package shadowsocks
    
Installing collected packages: shadowsocks
  Running setup.py install for shadowsocks
    
    Installing sslocal script to /usr/local/bin
    Installing ssserver script to /usr/local/bin
Successfully installed shadowsocks
Cleaning up...

Start up shadowsocks while listening on local port 1080:

pi@raspberrypi $ sslocal -s  -p  -k  -b 127.0.0.1 -l 1080

Redsocks

Install redsocks, simply apt-get from the archive:

pi@raspberrypi $ sudo apt-get install redsocks

Then you need to change the START option in /etc/default/redsocks from NO to YES, so that redsocks will start automatically at boot time and also can be started by sudo /etc/init.d/redsocks start:

pi@raspberrypi $ sudo vi /etc/default/redsocks
START=yes

Then update /etc/redsocks.conf. Most of the default settings work fine, just need to change local_ip in the redsocks section to your address of the network interface that accepts traffic from your local network. The default is 127.0.0.1, but that does not work well if you want to re-route traffic from other machines on your network, so change it to something like:

redsocks {
        /* `local_ip' defaults to 127.0.0.1 for security reasons,
         * use 0.0.0.0 if you want to listen on every interface.
         * `local_*' are used as port to redirect to.
         */
        local_ip = 192.168.0.1;
        local_port = 12345;
        ...
}

But we want traffic from other hosts in your network to be redirected by redsocks to your local shadowsocks client, which in turn sent to the the remote shadowsocks server. We need to pay special attention to DNS traffic, as DNS poisoning is prevalent in China. We need to take special care to redirect DNS traffic through redsocks/shadowsocks.

We also want all China traffic NOT to go through shadowsocks for performance. This can be easily done by looking at the destination IP, if it is in the China IP range we skip going through the REDSOCKS china. First we need to get all network segments allocated to China and save it to a file called chnroute.txt:

pi@raspberrypi $ curl 'http://ftp.apnic.net/apnic/stats/apnic/delegated-apnic-latest' | grep ipv4 | grep CN | awk -F\| '{ printf("%s/%d\n", $4, 32-log($5)/log(2)) }' > chnroute.txt

These all can be accomplished by iptables. You need to run the following iptables commands, or put them in a local script and run it with sudo.

# Specify your shadowsocks server
SS_SERVER_IP=11.22.33.44

iptables -t nat -N REDSOCKS  # Create a new chain called REDSOCKS

# Do not redirect to shadowsocks server
iptables -t nat -A REDSOCKS -d $SS_SERVER_IP -j RETURN

# Do not redirect local traffic
iptables -t nat -A REDSOCKS -d 0.0.0.0/8 -j RETURN
iptables -t nat -A REDSOCKS -d 10.0.0.0/8 -j RETURN
iptables -t nat -A REDSOCKS -d 127.0.0.0/8 -j RETURN
iptables -t nat -A REDSOCKS -d 169.254.0.0/16 -j RETURN
iptables -t nat -A REDSOCKS -d 172.16.0.0/12 -j RETURN
iptables -t nat -A REDSOCKS -d 192.168.0.0/16 -j RETURN
iptables -t nat -A REDSOCKS -d 224.0.0.0/4 -j RETURN
iptables -t nat -A REDSOCKS -d 240.0.0.0/4 -j RETURN

# China traffic does not go through REDSOCKS
while read subnet; do
  sudo iptables -t nat -A REDSOCKS -d $subnet -j RETURN
done < /tmp/chnroute.txt

# Redirect all TCP traffic to redsocks, which listens on port 12345
iptables -t nat -A REDSOCKS -p tcp -j REDIRECT --to-ports 12345

# These traffic go to REDSOCKS chain first
iptables -t nat -A PREROUTING -p tcp -j REDSOCKS

Run iptables -t nat -L -n to make sure the rules have been added correctly. Now start up redsocks by sudo /etc/init.d/redsocks start and let's test it out by doing some web browsing on another computer in your local network. If that works fine, congratulations and you have set up everything correctly! If not, look at shadowsocks output and also turn on redsock's log_debug and check if there is anything useful in /var/log/daemon.log.

ChinaDNS

ChinaDNS is not absolutely necessary, but as explained at the beginning it is desirable. There is no pre-built package so we need to compile it. It is simple to do:

pi@raspberrypi $ sudo apt-get install git automake
pi@raspberrypi $ git clone https://github.com/clowwindy/ChinaDNS.git
Cloning into 'ChinaDNS'...
remote: Counting objects: 815, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 815 (delta 0), reused 0 (delta 0), pack-reused 810
Receiving objects: 100% (815/815), 213.52 KiB | 111 KiB/s, done.
Resolving deltas: 100% (426/426), done.

pi@raspberrypi $ ./autogen.sh
pi@raspberrypi $ ls
aclocal.m4  autom4te.cache  chnroute.txt  configure     COPYING  install-sh  Makefile.am  missing  packaging  src
autogen.sh  CHANGES         config.h.in   configure.ac  depcomp  iplist.txt  Makefile.in  openwrt  README.md  tests

pi@raspberrypi $ ./configure 
...
pi@raspberrypi $ make
...

After it is successfully compiled, test it out:

pi@raspberrypi $ sudo src/chinadns
pi@raspberrypi $ host video.sina.com.cn 
video.sina.com.cn has address 58.63.237.200
pi@raspberrypi $ host www.youtube.com
www.youtube.com is an alias for youtube-ui.l.google.com.
youtube-ui.l.google.com is an alias for youtube-ui-china.l.google.com.
youtube-ui-china.l.google.com has address 46.82.174.68
youtube-ui-china.l.google.com has IPv6 address 2404:6800:4005:80b::200e
pi@raspberrypi $ sudo src/chinadns -m -c chnroute.txt
pi@raspberrypi $ host video.sina.com.cn 
video.sina.com.cn has address 123.125.22.225
pi@raspberrypi $ host www.youtube.com
www.youtube.com is an alias for youtube-ui.l.google.com.
youtube-ui.l.google.com is an alias for youtube-ui-china.l.google.com.
youtube-ui-china.l.google.com has address 74.125.203.138
youtube-ui-china.l.google.com has address 74.125.203.100
youtube-ui-china.l.google.com has address 74.125.203.113
youtube-ui-china.l.google.com has address 74.125.203.139
youtube-ui-china.l.google.com has address 74.125.203.101
youtube-ui-china.l.google.com has address 74.125.203.102
youtube-ui-china.l.google.com has IPv6 address 2404:6800:4005:808::200e

If it goes well, run src/chinadns -m -c chnroute.txt when your router boots.

That's it! I hope these are useful to you.

Update: I have switched from ChinaDNS to dnsmasq+dnscrypt, please read Securing DNS Traffic in China to see how to set it up.

解決 WordPress 中 Google Web Font 被 GFW 屏蔽問題

自從本 blog 的 wordpress 升級並更換 theme 後,偶爾發現 page load 得很慢,用 chrome 自帶的 developer tool 查了一下才發現原來 theme 用了 google web font,因爲 google service 被牆了所以在大陸瀏覽就像卡住了一樣。由於我絕大部份時間都是科學上網,所以一開始沒有發現問題。

Google font is blocked in China

既然 google web font 不能用,最簡單就是乾脆不用,把相關代碼拿掉就行,不過這樣做的後果是網站會極其難看。上策還是把 google web font 放到 wordpress server,再將 CSS 配置好。Google 一下發現有個很棒的 shell script 能幫個大忙:https://neverpanic.de/blog/2014/03/19/downloading-google-web-fonts-for-local-hosting/,它能幫你把需要的字體下載下來並產生一個立馬能用的 CSS 文檔。

WordPress 最少有兩個地方會加載 google web font,第一是 wordpress 本身用到的 Open Sans,另外是個別 theme 可能會用到的字體。視乎你的 theme 有沒有用 google font,有可能兩個地方都要處理。

目前我用的 theme 基於 twentyfourteen(我把原來左邊的 navigation 拿掉,header 加了 facebook, twitter 等幾個 icon),best practice 是在 wordpress 的 themes 目錄開一個叫 twentyfourteen-child 的新目錄,裏面只放用來 override parent theme 的東西。

解決問題的具體步驟如下:

  1. 下載剛才介紹過的網站的 shell script,我把它命名爲 downloadgooglefont.sh
  2. 先下載 wordpress 會下載的所有 Open Sans 字體(包括不同粗幼和斜體),在 downloadgooglefont.sh 裏要指定它們:
    families+=('Open Sans:300')
    families+=('Open Sans:400')
    families+=('Open Sans:600')
    families+=('Open Sans:300italic')
    families+=('Open Sans:400italic')
    families+=('Open Sans:600italic')
  3. 針對你使用的 theme,需要把 theme 用到的 google 字體寫進去。你要查一下你的 theme 會用到什麼字體,例如 twentyfourteen 會用到的字體在
    themes/twentyfourteen/functions.php

    的 215 行有提到:

                            
         'family' => urlencode( 'Lato:300,400,700,900,300italic,400italic,700italic' ),
    

    總共有7個字體,所以 script 需要加上:

    families+=('Lato:300')
    families+=('Lato:400')
    families+=('Lato:700')
    families+=('Lato:900')
    families+=('Lato:300italic')
    families+=('Lato:400italic')
    families+=('Lato:700italic')
    
  4. 執行該 script:
    $ bash ./downloadgooglefont.sh
    Downloading Open Sans:300... eot woff ttf svg 
    Downloading Open Sans:400... eot woff ttf svg 
    Downloading Open Sans:600... eot woff ttf svg 
    Downloading Open Sans:300italic... eot woff ttf svg 
    Downloading Open Sans:400italic... eot woff ttf svg 
    Downloading Open Sans:600italic... eot woff ttf svg 
    Downloading Lato:300... eot woff ttf svg 
    Downloading Lato:400... eot woff ttf svg 
    Downloading Lato:700... eot woff ttf svg 
    Downloading Lato:900... eot woff ttf svg 
    Downloading Lato:300italic... eot woff ttf svg 
    Downloading Lato:400italic... eot woff ttf svg 
    Downloading Lato:700italic... eot woff ttf svg

    這些就是下載完的所有文檔,包括適用於 chrome/chromium, firefox, safari 會用到的字體,還有一個 CSS file:

    Lato_300italic.svg
    Lato_300italic.ttf
    Lato_300italic.woff
    Lato_300.svg
    Lato_300.woff
    Lato_400italic.svg
    Lato_400italic.ttf
    Lato_400italic.woff
    Lato_400.svg
    Lato_400.ttf
    Lato_400.woff
    Lato_700italic.svg
    Lato_700italic.ttf
    Lato_700italic.woff
    Lato_700.svg
    Lato_700.ttf
    Lato_700.woff
    Lato_900.svg
    Lato_900.ttf
    Lato_900.woff
    LNM3t2QINm
    Open_Sans_300italic.svg
    Open_Sans_300italic.ttf
    Open_Sans_300italic.woff
    Open_Sans_300.svg
    Open_Sans_300.ttf
    Open_Sans_300.woff
    Open_Sans_400italic.svg
    Open_Sans_400italic.ttf
    Open_Sans_400italic.woff
    Open_Sans_400.svg
    Open_Sans_400.ttf
    Open_Sans_400.woff
    Open_Sans_600italic.svg
    Open_Sans_600italic.ttf
    Open_Sans_600italic.woff
    Open_Sans_600.svg
    Open_Sans_600.ttf
    Open_Sans_600.woff
    font.css
  5. 把上面所有的字體上傳到 wordpress server 的 wp-content/themes/twentyfourteen-child/ 目錄。
  6. 把 font.css 複製到 wp-content/themes/twentyfourteen-chile/style.css。
  7. 由於 twentyfourteen 在 functions.php 通過 wp_enqueue_style 加上 fonts.googleapis.com 的 link 來加載 Lato 字體,要想辦法把它拿掉:
            // Add Lato font, used in the main stylesheet.
            wp_enqueue_style( 'twentyfourteen-lato', twentyfourteen_font_url(), array(), null );
    

    不過我不喜歡直接修改 parent theme,所以我選擇在 twentyfourteen-child 把它 dequeue 掉。注意 add_action 的 priority 必須低於 10,dequeue 才會成功。theme_enqueue_styles 的前兩行是用來禁止 Open Sans 字體從 google server 加載,第3行才是禁止 Lato。
    _twentyfourteen-child/functions.php: