Category Archives: Open Source

Boost WIFI Speed by Forcing 40 MHz Channels in hostapd on RPi

802.11n can double the channel bandwidth of 802.11g from 20 MHz to 40 MHz, but this operation mode is not recommended in areas that the spectrums are congested and likely interfere with existing WIFI and bluetooth devices. As a result, hostapd will not enable 40 MHz when it finds other channels are being used, like what is seen from the hostapd log below:

nl80211: New scan results available
nl80211: Received scan results (16 BSSes)
40 MHz affected channel range: [2397,2447] MHz
Neighboring BSS: 1c:fa:68:8e:a6:e0 freq=2412 pri=1 sec=5
Neighboring BSS: e0:05:c5:4c:e2:b6 freq=2427 pri=0 sec=0
Neighboring BSS: b0:48:7a:6a:9d:32 freq=2437 pri=6 sec=10
40 MHz pri/sec mismatch with BSS b0:48:7a:6a:9d:32 <2437,2457> (chan=6+) vs. <2412,2432>
20/40 MHz operation not permitted on channel pri=1 sec=5 based on overlapping BSSes

However, this is unrealistic in modern cities. You should be grateful the primary channel you choose has not been used already, let alone the additional one for 40 MHz. What can we do? We have to force hostapd to turn on 40 MHz anyway. But since hostapd does not have any configuration options for that, we have to recompile hostapd from source. Actually, this has been done in other distros. For example, OpenWRT and Arch has patched their hostapd with the noscan patch, which adds an option to force 40MHz mode regardless of the environment.

Before turning on 40 MHz, the iwconfig on my laptop is like this, notice the bit rate is only 54 Mb/s:

wlan0     IEEE 802.11abgn  ESSID:"bibiworld"  
          Mode:Managed  Frequency:2.437 GHz  Access Point: C8:3A:35:C9:17:4A   
          Bit Rate=54 Mb/s   Tx-Power=15 dBm   
          Retry short limit:7   RTS thr:off   Fragment thr:off
          Power Management:off
          Link Quality=70/70  Signal level=-32 dBm  
          Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
          Tx excessive retries:30  Invalid misc:683   Missed beacon:0

Now let us start. First you need to add a deb-src source to apt so that we can download source packages. Note that I am still using Raspbian Wheezy. If you use Jessie, change the deb-src line according to your original deb line.

pi@raspberrypi $ echo "deb-src http://mirrordirector.raspbian.org/raspbian/ wheezy main contrib non-free rpi" \
 | sudo tee /etc/apt/sources.list
pi@raspberrypi $ sudo apt-get update
pi@raspberrypi $ apt-get source hostapd            # download source of hostapd
pi@raspberrypi $ apt-get build-dep hostapd    # download all build dependencies
pi@raspberrypi $ cd wpa-1.0

Save the following patch into a file:

--- wpa-1.0.orig/src/ap/hw_features.c      2012-05-10 05:56:09.000000000 +0800
+++ wpa-1.0/src/ap/hw_features.c        2015-07-18 17:22:37.788215009 +0800
@@ -430,6 +430,7 @@
                oper40 = ieee80211n_check_40mhz_2g4(iface, scan_res);
        wpa_scan_results_free(scan_res);
 
+#if 0
        if (!oper40) {
                wpa_printf(MSG_INFO, "20/40 MHz operation not permitted on "
                           "channel pri=%d sec=%d based on overlapping BSSes",
@@ -439,6 +440,12 @@
                iface->conf->secondary_channel = 0;
                iface->conf->ht_capab &= ~HT_CAP_INFO_SUPP_CHANNEL_WIDTH_SET;
        }
+#endif
+       wpa_printf(MSG_INFO, "Force 20/40 MHz operation on "
+                  "channel pri=%d sec=%d even there are overlapping BSSes",
+                          iface->conf->channel,
+                          iface->conf->channel +
+                          iface->conf->secondary_channel * 4);
 
        res = ieee80211n_allowed_ht40_channel_pair(iface);
        hostapd_setup_interface_complete(iface, !res);
pi@raspberrypi $ patch -p1 < patch.diff

Because we are going to create a new hostapd package, it'd be a good practice to increment the version number by adding a new section at the top of debian/changelog, like this:

wpa (1.0-3+deb7u3) wheezy; urgency=low

  * Force 20/40 MHz.

 -- Anthony Wong   Sat, 18 Jul 2015 16:30:43 +0800

wpa (1.0-3+deb7u2) wheezy-security; urgency=high
...

Now we are ready to compile the packages:

pi@raspberrypi $ fakeroot debian/rules binary

If it succeeds, install the new package with dpkg -i.

Restart hostapd, it should now force 40 MHz mode:

nl80211: New scan results available
nl80211: Received scan results (23 BSSes)
40 MHz affected channel range: [2397,2447] MHz   
Neighboring BSS: e0:05:c5:4c:e2:b6 freq=2427 pri=0 sec=0
Neighboring BSS: 00:d0:41:c4:25:31 freq=2437 pri=6 sec=2
40 MHz pri/sec mismatch with BSS 00:d0:41:c4:25:31 <2437,2417> (chan=6-) vs. <2412,2432>
Force 20/40 MHz operation on channel pri=1 sec=5 even there are overlapping BSSes
HT40: control channel: 1  secondary channel: 5
Completing interface initialization

And my iwconfig output now shows Bit Rate is 150 Mb/s, yay!

wlan0     IEEE 802.11abgn  ESSID:"bibiworld"  
          Mode:Managed  Frequency:2.412 GHz  Access Point: C8:3A:35:C9:17:4A   
          Bit Rate=150 Mb/s   Tx-Power=15 dBm   
          Retry short limit:7   RTS thr:off   Fragment thr:off
          Power Management:off
          Link Quality=70/70  Signal level=-28 dBm  
          Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
          Tx excessive retries:31  Invalid misc:330   Missed beacon:0

Deploy Shadowsocks on Raspberry Pi with ChinaDNS and Redsocks

Assume you already have a Raspberry Pi configured as a WIFI router like mine shown below, but you live in China and have to deal with the fact that many websites can’t be accessed due to GFW. Don’t be despair and with some hacking you can get your Internet freedom back.

IMG_20150714_134341

The mechanism is to use shadowsocks on your router which directs any traffic to a shadowsocks server in the free world. It’s simple to get it up and running on a local machine, but on a router you need to use redsocks to redirect traffic to the shadowsocks client running on your Raspberry Pi. DNS traffic has to be routed by redsocks as well otherwise your DNS replies will be contaminated. To remain as fast as normal when accessing China websites you also need to skip routing traffic to Redsocks for anything within the China IP ranges. Even if you don’t care about performance, this is still necessary in some circumstances like geoip restriction such as tv.sohu.com does not deliver contents if you live outside of China.

Finally, we want to improve the performance even further by using ChinaDNS. To avoid DNS poisoning, we can always resolve DNS over our secured shadowsocks connection, but this is not optimal if a China website have CDNs outside China. ChinaDNS queries local DNS servers to resolve Chinese domains and queries foreign DNS servers to resolve foreign domains, and from my testing it is useful to avoid DNS poisoning with the “DNS compression pointer mutation” option. (Update: I have switched from ChinaDNS to dnsmasq+dnscrypt, please read Securing DNS Traffic in China to see how it works.)

Shadowsocks

I assume that you have shadowsocks server running on a public server, so I will skip that part and only talk about the client side.

Installing shadowsocks is very simple, note that it will be installed under /usr/local/.

pi@raspberrypi $ sudo apt-get install python-pip
pi@raspberrypi $ sudo pip install shadowsocks
Downloading/unpacking shadowsocks
  Running setup.py egg_info for package shadowsocks
    
Installing collected packages: shadowsocks
  Running setup.py install for shadowsocks
    
    Installing sslocal script to /usr/local/bin
    Installing ssserver script to /usr/local/bin
Successfully installed shadowsocks
Cleaning up...

Start up shadowsocks while listening on local port 1080:

pi@raspberrypi $ sslocal -s  -p  -k  -b 127.0.0.1 -l 1080

Redsocks

Install redsocks, simply apt-get from the archive:

pi@raspberrypi $ sudo apt-get install redsocks

Then you need to change the START option in /etc/default/redsocks from NO to YES, so that redsocks will start automatically at boot time and also can be started by sudo /etc/init.d/redsocks start:

pi@raspberrypi $ sudo vi /etc/default/redsocks
START=yes

Then update /etc/redsocks.conf. Most of the default settings work fine, just need to change local_ip in the redsocks section to your address of the network interface that accepts traffic from your local network. The default is 127.0.0.1, but that does not work well if you want to re-route traffic from other machines on your network, so change it to something like:

redsocks {
        /* `local_ip' defaults to 127.0.0.1 for security reasons,
         * use 0.0.0.0 if you want to listen on every interface.
         * `local_*' are used as port to redirect to.
         */
        local_ip = 192.168.0.1;
        local_port = 12345;
        ...
}

But we want traffic from other hosts in your network to be redirected by redsocks to your local shadowsocks client, which in turn sent to the the remote shadowsocks server. We need to pay special attention to DNS traffic, as DNS poisoning is prevalent in China. We need to take special care to redirect DNS traffic through redsocks/shadowsocks.

We also want all China traffic NOT to go through shadowsocks for performance. This can be easily done by looking at the destination IP, if it is in the China IP range we skip going through the REDSOCKS china. First we need to get all network segments allocated to China and save it to a file called chnroute.txt:

pi@raspberrypi $ curl 'http://ftp.apnic.net/apnic/stats/apnic/delegated-apnic-latest' | grep ipv4 | grep CN | awk -F\| '{ printf("%s/%d\n", $4, 32-log($5)/log(2)) }' > chnroute.txt

These all can be accomplished by iptables. You need to run the following iptables commands, or put them in a local script and run it with sudo.

# Specify your shadowsocks server
SS_SERVER_IP=11.22.33.44

iptables -t nat -N REDSOCKS  # Create a new chain called REDSOCKS

# Do not redirect to shadowsocks server
iptables -t nat -A REDSOCKS -d $SS_SERVER_IP -j RETURN

# Do not redirect local traffic
iptables -t nat -A REDSOCKS -d 0.0.0.0/8 -j RETURN
iptables -t nat -A REDSOCKS -d 10.0.0.0/8 -j RETURN
iptables -t nat -A REDSOCKS -d 127.0.0.0/8 -j RETURN
iptables -t nat -A REDSOCKS -d 169.254.0.0/16 -j RETURN
iptables -t nat -A REDSOCKS -d 172.16.0.0/12 -j RETURN
iptables -t nat -A REDSOCKS -d 192.168.0.0/16 -j RETURN
iptables -t nat -A REDSOCKS -d 224.0.0.0/4 -j RETURN
iptables -t nat -A REDSOCKS -d 240.0.0.0/4 -j RETURN

# China traffic does not go through REDSOCKS
while read subnet; do
  sudo iptables -t nat -A REDSOCKS -d $subnet -j RETURN
done < /tmp/chnroute.txt

# Redirect all TCP traffic to redsocks, which listens on port 12345
iptables -t nat -A REDSOCKS -p tcp -j REDIRECT --to-ports 12345

# These traffic go to REDSOCKS chain first
iptables -t nat -A PREROUTING -p tcp -j REDSOCKS

Run iptables -t nat -L -n to make sure the rules have been added correctly. Now start up redsocks by sudo /etc/init.d/redsocks start and let's test it out by doing some web browsing on another computer in your local network. If that works fine, congratulations and you have set up everything correctly! If not, look at shadowsocks output and also turn on redsock's log_debug and check if there is anything useful in /var/log/daemon.log.

ChinaDNS

ChinaDNS is not absolutely necessary, but as explained at the beginning it is desirable. There is no pre-built package so we need to compile it. It is simple to do:

pi@raspberrypi $ sudo apt-get install git automake
pi@raspberrypi $ git clone https://github.com/clowwindy/ChinaDNS.git
Cloning into 'ChinaDNS'...
remote: Counting objects: 815, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 815 (delta 0), reused 0 (delta 0), pack-reused 810
Receiving objects: 100% (815/815), 213.52 KiB | 111 KiB/s, done.
Resolving deltas: 100% (426/426), done.

pi@raspberrypi $ ./autogen.sh
pi@raspberrypi $ ls
aclocal.m4  autom4te.cache  chnroute.txt  configure     COPYING  install-sh  Makefile.am  missing  packaging  src
autogen.sh  CHANGES         config.h.in   configure.ac  depcomp  iplist.txt  Makefile.in  openwrt  README.md  tests

pi@raspberrypi $ ./configure 
...
pi@raspberrypi $ make
...

After it is successfully compiled, test it out:

pi@raspberrypi $ sudo src/chinadns
pi@raspberrypi $ host video.sina.com.cn 
video.sina.com.cn has address 58.63.237.200
pi@raspberrypi $ host www.youtube.com
www.youtube.com is an alias for youtube-ui.l.google.com.
youtube-ui.l.google.com is an alias for youtube-ui-china.l.google.com.
youtube-ui-china.l.google.com has address 46.82.174.68
youtube-ui-china.l.google.com has IPv6 address 2404:6800:4005:80b::200e
pi@raspberrypi $ sudo src/chinadns -m -c chnroute.txt
pi@raspberrypi $ host video.sina.com.cn 
video.sina.com.cn has address 123.125.22.225
pi@raspberrypi $ host www.youtube.com
www.youtube.com is an alias for youtube-ui.l.google.com.
youtube-ui.l.google.com is an alias for youtube-ui-china.l.google.com.
youtube-ui-china.l.google.com has address 74.125.203.138
youtube-ui-china.l.google.com has address 74.125.203.100
youtube-ui-china.l.google.com has address 74.125.203.113
youtube-ui-china.l.google.com has address 74.125.203.139
youtube-ui-china.l.google.com has address 74.125.203.101
youtube-ui-china.l.google.com has address 74.125.203.102
youtube-ui-china.l.google.com has IPv6 address 2404:6800:4005:808::200e

If it goes well, run src/chinadns -m -c chnroute.txt when your router boots.

That's it! I hope these are useful to you.

Update: I have switched from ChinaDNS to dnsmasq+dnscrypt, please read Securing DNS Traffic in China to see how to set it up.

思源/Noto pan-CJK 字體 & Ubuntu

以開源協議授權釋出的字體大多數以拉丁字母等歐洲語言爲主,但最近 Adobe 和 Google 共同推出的思源/Noto pan-CJK 開源字體不只涵蓋了大部份中日韓所需要的字型,而且繁中、簡中、日文和韓文都用了不同的 OTF 檔案,因此對同一個漢字在不同地區的不同寫法就能個別處理,算是解決了 Unicode 一直爲人詬病的 Han unification 問題。

香港人最關心的應該是該字體是否覆蓋 HKSCS,我檢查過部份 hkscs-2008-big5-iso.txt 的 unicode 碼,在 NotoSansHant 裏都能找到,而且目測在 CJK BCD 區裏都有覆蓋,所以應該夠用(在下面的圖找找吧)。

用 Fontforge 查看 NotoSansHant-Regular.otf
用 Fontforge 查看 NotoSansHant-Regular.otf

從下面幾張 screenshot 可以看到香港字顯示效果理想:


想將 Ubuntu 的桌面 UI 和程式的預設字體改成思源/Noto,可以參考 Ingram Chen 的 blog。小弟改良了一下 Ingram 的 fontconfig 設定檔,使系統在不同 locale 下能優先選擇適當的字體,比如 zh_TW 下繁體的 Noto Sans T Chinese 是第一選擇,在 zh_CN 下則爲簡體的 Noto Sans S Chinese,之後其他的 CJK 字體作爲 fallback。

設定檔已放在 20-noto-cjk.conf,下載後執行:

[shell]
mkdir ~/.config/fontconfig/conf.d
mv 20-noto-cjk.conf ~/.config/fontconfig/conf.d
[/shell]

另外如果要配置英文字體,可以下載 10-latin.conf,修改一下檔案中的字體部份,同樣放進 ~/.config/fontconfig/conf.d 即可。

GPG key transition: 7BD22F74 → D28DA8DC

I should have transitioned my old GPG key to a stronger one for long time, it’s finally done today, with the help of here and here. You can find my signed letter at http://ubuntuone.com/6O2OCf1rg9ulw1eWi13zc2. I am also copying the letter below for your convenience.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256,SHA1

Due to rapid advancement of computing, my old 1024-bit DSA GPG key,
which was created 14 years ago in 1999, has long been deemed insecure.
Therefore, I am transitioning to a much stronger 8192-bit RSA key, by
using a slightly modified gnupg in [1], as the default gnupg does
not allow creation of keys greater than 4096-bit. Hopefully this new
key can survive much longer.

The old key will continue to be valid for some time, but I prefer all
future correspondence to come to the new one. I would also like this
new key to be re-integrated into the web of trust.  This message is
signed by both keys to certify the transition.

If you have signed my old key, I would appreciate signatures on my new
key as well, provided that your signing policy permits that without
reauthenticating me.

The old key, which I am transitioning away from, is:

pub   1024D/7BD22F74 1999-09-12
      Key fingerprint = CD09 4F7B BBEE 93CD 7966  6299 34B3 A9A0 7BD2 2F74
uid          Anthony Y. P. Wong (Personal) <ypwong@ypwong.org>

And the new key is:

pub   8192R/D28DA8DC 2013-06-01
      Key fingerprint = 8DF0 9030 F103 F760 C18C  BA06 605A A53D D28D A8DC
uid          Anthony Y. P. Wong (黃彥邦) <ypwong@ypwong.org>

To fetch the new key from a public key server using GnuPG, run:

  gpg --keyserver pgp.mit.edu --recv-key D28DA8DC

If you have already validated my old key, you can then validate that the
new key is signed by my old key:

  gpg --check-sigs D28DA8DC

If you are satisfied that you've got the right key, and the UIDs match
what you expect, I'd appreciate if you would sign my new key.

A simple and safe way to do that is by using caff (shipped in
Debian/Ubuntu as part of the "signing-party" package) as follows:

  caff D28DA8DC

Alternatively, you can sign the key by using gpg and send it to me (if
you have a functional MTA configured on your system) or upload the
signatures to a public keyserver directly:

  gpg --sign-key D28DA8DC
  gpg --armor --export D28DA8DC | mail -s 'OpenPGP Signatures' \
    ypwong@ypwong.org
  gpg --keyserver pgp.mit.edu --send-key D28DA8DC

Please let me know if there is any trouble, and sorry for the
inconvenience.

[1] https://launchpad.net/~anthonywong/+archive/ppa

Thanks,
Anthony Wong

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQQcBAEBCAAGBQJRqdnMAAoJEGBapT3SjajciuYf/isb8jX/+SXxnxr6veAS4VTL
rKG2Up6cGjAPsfNX4AmKUojiiGzWyqXOa3qTfRReXf2Xl+NroI6jMdN2zBEe4D7J
UZYDzvEVWDhhATbWIqL2mcxZYbInX4sd18UW5cw58Tx+uBATAIHSJdHiLMjLDbvc
uKpbIqwZDC1zgJIV1+vosgtR3KQFO0bJyeqEBMpjvzJ2Zy60ZFgBycNOZL2aIdav
HnbCBSttqdfUE+TuXMgHSSTgx0WUromRa9d4X/OaT+1veX2CRD/K9X+Qt4ac6dop
Tze43U4ns2Ijz8Mwb603rlkh5e+FFfI7VxehCpIPv2oYlpNHYt5BoJbNj5Hl8LYn
6DMmbJSugz5Ov94Q5/QLFPRprnwXRVmHuvm1sbxznQGrCDgJhxVLJuTTTSNjuUiL
5VVwqKJ8RrG8gFcqmcZhG1+U2tQvBfJU/NR7BF6LMqB9FrJNyBTvg0w460XnUiOc
2ZKubdDn7qj7iEhUru4Mmu1yz9tANjYP7ObCxsvWvMOt8basOn1EobgUywIVsQsz
QdCR5SYNvWT7fgKjbpMM6RwTOa7mqOvk+IF9SCgZcSy4KVkLr6PrB0VSNueMy7bz
iJqF+j06ur79/0IHK33iFj02OqXIXG99g6hGm86NgSIlBM0EZwX52O7XmfSfTcg0
DlKuAy/ZBZOltxujkTaUSECWmfRzT2FtGgDsNj9PGZ+RbSTIT0J3/P46Umg55IBq
ptNFg5e0VNSewVT5sQVGxZDesPmrcVH6KjmFI0HYihZp2SUjshoT9TmyeKIdZcGa
PtLKSlhuwVAgEEsMPKQb4x/8xwZa2D5ZuiSOWP0NWptzdE2g2qAMGVttgu19B1+X
Iv9FUts5BYI5xiocbBq0t/MSioEwQG4Q4fBjMltMKu062lXnHNj8bP2W0Z1lBOTy
/GxHRrRnvfdmsWjn5DTfY3Cg5LlOHWMxd0JnCYMrEMrdQb78+1sc/qpbhfW3cKeR
vL8nGw7GZthOLOTVHUMtMthVSvWcymWfuJ4pfwP+Im6PqHmV+aC8GfwsSBDoLjPn
6AkdoinofPTh7RziNK/bJ36qS5QVL4bITeIw5qBYG9cXGSyuX15clK8CiuMG6RqG
1ztp3rQbLp1a0/1fW4xuhZUfUo4kXPYwR5Tm8Emx9dnS1IDk6avbUYMw+30JUqPO
KLCSGQnsjXyBPD6Z+qxENUgk1046JNxUFZLoc9mbbU9CXWKlGDDAoF+moLcWmR4D
BX9JbQAkmTQkuvPuH9x7aYoFSJKcmJ7Zz6PTdD8PHAT0vm92Kg3viIu/2BhoyvyD
oKLf/yQ1d8y1/iZYuyOrnM7eT56BtcvSHlSXQSmyACiB4bR3YYz6SNS2KDjmWWCI
RgQBEQIABgUCUanZzAAKCRA0s6mge9IvdMFJAKDlnUHGERTnlUGTZTB5SH4IREKb
qwCeJy3k6qi6uxBlZqds/4AG6vDmtfE=
=UVhj
-----END PGP SIGNATURE-----

Running QQ on Ubuntu Phone

Courtesy to an open source project called uqq on Github, it is now possible to not only run QQ on Linux, but also on Ubuntu Phone! As Ubuntu phone is still under heavy development, and the current SDK is limited for QML-only apps, it is not an easy task to deploy uqq onto the phone, as it uses C++ for its backend. If you are adventurous, you can follow my steps below.

Continue reading Running QQ on Ubuntu Phone