Chris Boyle

So, hands up if your notification bar looks like this:

I see similar levels of clutter when other people show me their phones, and this just isn't a good user interface. These icons mean all kinds of different things, and the level of urgency of each one is really not clear. They're just ordered by most recent update first. On CyanogenMod I can turn off the clock to fit all 9 of them in, but really, a lot of them shouldn't be here.

There's an excellent Android Design article on notifications, highlighting many of the common reasons for notification clutter. If you see one of those anti-patterns, you should definitely bug the app developer about it (by email please! - again, we can't reply to Market comments), but that's not what's happening here. Let's look through them shall we? (I clicked in a text box, so now there are 10.)

Now we can see: 3 I really want, 3 system ones that aren't really notifying me of very much, and the other 4 are just background services that I really want to keep running. Apps like that have to show a notification, even if they've nothing to say, to avoid being killed, in line with an API change that was made in Android 2.0 (and here's the relevant API doc).

I think that was the right decision for Android at the time. As that article says, there was very much a tragedy of the commons problem with background services before then, with every app declaring itself to be high priority, the platform unable to make good decisions about what to kill, and users getting the short straw with slow phones where the apps they wanted to keep running would keep getting killed. So with that change, apps had more incentive to limit their use of background services, and users were much more likely to notice and take action when an app they didn't want was hogging resources.

The problem now is that this phone has a dual-core 1.2GHz chip and 1GB of RAM, and can happily run much more stuff in the background than can sensibly be prominently displayed to the user all the time (a nice problem to have). It means there really needs to be a better UI for managing background functionality. I don't have a definite answer to this problem, let alone a plan to migrate all the existing background apps to it, but here are a couple of ideas:

• Ditch these notifications in favour of the "Running" tab under "Manage apps", which could perhaps be surfaced as a second tab of the notification drawer? Or just a shortcut to it on that top line with the date?
• Remove their icons from the status bar, and keep them in the drawer, at the bottom, in a dedicated section which by default is collapsed to just icons. Have an expand/collapse button, and persist that state.

A big difficulty here is identifying which notifications serve no other purpose. Often a notification on which startForeground() is called is also displaying useful information, or acting as a shortcut for something that the user will frequently want to switch to without using the home screen. The second idea above seems safe-ish to apply to ongoing notifications from existing apps where a custom layout is not used, but you might have one app whose detail text you really want to see, and then you have to keep the whole thing expanded all the time. Still, even putting non-ongoing notifications first seems like an improvement to me.

For myself, I might put a band-aid on it by adding a CyanogenMod option to hide the icons of ongoing notifications from the status bar. Even then, I'd like the pull-down drawer to at least prioritise notifications that are, y'know, notifying me.

Better ideas, anyone? Bueller?

Far out in the uncharted backwaters of the unfashionable end of the Western Spiral arm of the Galaxy lies a small unregarded yellow sun. Orbiting this at a distance of roughly ninety-eight million miles is an utterly insignificant little blue-green planet whose ape-descended life forms are so amazingly primitive that they still think digital watches are a pretty neat idea.

(Douglas Adams, The Hitchhiker's Guide to the Galaxy)

Until quite recently, probably every 5 minutes on average, every waking hour, I would take my phone out of its holster (or wake it up, if docked) and glance at it, to make sure I hadn't missed any attempts to contact me. Frequently, I found I had: even with a belt holster, the phone's vibration is easy to miss. Now, I just look at my watch instead, and almost never miss any such notification in the first place: a watch vibrating is much more noticeable.

…so I never actually see the watch full of notifications like this. :-) The watch in question is a MetaWatch (digital), and I think it's a pretty neat idea. In contrast to previous smart watches I've seen, it's not trying to be much of a computer in its own right, just an add-on for an existing smartphone (or other Bluetooth device), providing an always-on, glanceable 96x96 LCD display and 6 buttons. The battery life is quite reasonable: about two days on mine, which I'm constantly messing with, some people report four days and five should be possible. It's built around a TI MSP430 and you can modify the firmware if you want, but currently that requires non-free tools and, in any case, I find there's more fun to be had hacking the supplied Android app ("MetaWatch Manager" or MWM), where you have a net connection and the data on your phone to play with. The firmware defines a few modes: idle (showing the time), full-screen application and full-screen notification.

I've not seen anything else quite like this: the LiveView seems much less hackable, the inPulse has one button as its only form of input (why?!), the WIMM One (no detail there, so have a review) is very shiny but is trying to be an independent smart device (running everything on the watch, with its own wifi connection) and is correspondingly more expensive, and all of the above seem like they'd have trouble achieving a four-day battery life.

I should mention at this point that it took a few heinous hacks in MWM and a modified build of Android to get to the photo above. The default UI out of the box is rather less information-dense:

I wanted to make better use of the pixels on mine. I suppose the stock layout does give you something pretty to show people in the absence of any notifications: mine is just the time, date and temperatures in that situation.

You can look through my commits on github to see what else I'm up to. Earlier today I mapped a button to act like a Bluetooth headset button: play/pause music and answer/hangup calls. I also finally got the Remember The Milk API to cooperate, so I can press a button as I'm walking into the supermarket and see my shopping list on the watch. The next step would be a UI to flip through the items and mark them as completed.

I'm also looking to improve the UI on the phone screen: the settings screen is about 10 screenfuls long and should be split, and I think the main screen could be much more useful. Currently it's just a text dump of various bits of status, but what if it showed you the current watch display (a feature I liked in Cicada) and let you remap buttons using dropdowns? Something like this…

Food for thought, at least. Meanwhile, quite a few others seem to be joining in, a few weird and wonderful projects have been going for some time and the forums seem to have a friendly and helpful community of users and developers. I think I'm going to have fun with this. :-)

The story so far

Over the past month or so, the "nymwars" have become the thing Google+ is most known for among my circle of friends. This is a problem of Google's own making: they are suspending profiles based on naive heuristics about "real names" (actually typical two part western names), and demanding government ID to reinstate them. The names policy they claim to be enforcing says you should use the name your friends, family or co-workers usually call you, but it is both possible and common for this to be entirely different from the name the government calls you. Furthermore, the verifiers apparently lack even basic training in ID verification, which allows frivolous reporting of profiles and trivial reinstatement for the true trolls/imposters (who will think nothing of faking an ID).

Why are they bothering?

This is not an effective defence against trolls as was initially claimed; they're more concerned with ideas about G+ as an "identity service" and a way to "improve our products" than about the wishes of their users or the fact that they're perpetuating the exclusion of minorities. I realise they are a business, but this is markedly worse than anything I've previously seen from them, it has directly affected and excluded several of my dearest friends, and when a player this big gets it this wrong, it's all the more important to hold them to account. This will be difficult, because apparently they are committed enough to their current course of action to ignore the wrath of numerous developers and employees, and to spend substantial extra resources employing an army of ID verifiers. This from Google, who famously don't do support because it doesn't scale.

"Improving our products": the Tentacles of Doom

This isn't just about being excluded from the latest trendy Facebook replacement. The first "identity service"-related feature went live before G+ did (using old-style Google profiles), and means connecting your content elsewhere on the web to a Google profile will make that content more eye-catching (even if no higher up) in search results, with a head shot and author name.

I would be very interested to hear what competition regulators have to say about this use of a dominant position in search to bribe people onto an "identity service", ostensibly a different product in which Google is still far from a market leader. They don't appear to offer this for any profile/identity pages but their own. This is something I intend to ask the Open Rights Group about, and it may be worth someone's while to make an EU competition complaint, as it seems superficially similar to cases in which the EU has acted against Microsoft in the past (use of dominance in the OS market to get into the media player software market).

Zen and the art of identity maintenance

I want to explicitly call out the dependency that Google is encouraging people to introduce by using them as an "identity service". If you already don't want to link your online identities together, or don't want to tell Google about them, then I very much understand that position, but there's not much I can suggest here to help you, except helping with any competition complaint. If you do want some level of identity linkage, then in order to minimise the damage if your Google profile is taken down, I recommend linking together your profile pages on other sites, rather than only linking everything to your Google profile.

A quick primer:

• rel=author is an HTML5 feature for content to link to its author (be that a profile page, a direct email address, or otherwise). It applies to the <article> element containing it, or else to the whole page. Example:

  1 ...by <a rel="author" href="/people/cmb">Chris Boyle</a>.

  Or on a single-author site, in the <head>:

  1 <link rel="author" href="/" type="text/html" />                                

• rel=me came from XFN in 2004 and allows a profile page to mark another profile page as being the same person. Example:

  1 ...I'm also <a rel="me" href="http://example.com/profiles/foo">foo on example.com</a>.

• Google has instructions old and new on this; I recommend the old instructions, as only those include rel=me, and the new ones are based on an extra URL parameter that only works for G+.
  • They will follow author links pointing either to a G+ profile directly, or to an author page on the same domain (as long as there's some sort of link path from G+ back to the content).
  • In the same-domain case, they will follow rel=me to find a G+ profile if present (but only if there's a mutual rel=me link back from the G+ profile).
  • They have not said whether they'll follow more than one hop of rel=me.

Putting all this together:

My recommendation: don't just do what's on the left here, because all those associations will be lost when your G+ profile is taken down. If you do something more like what's on the right, other identity services / social networks and other search engines will have a better chance of presenting what you want them to present. The warning in the bottom-right is a reminder that Google won't follow a content-to-author link to a different domain (they haven't said why). You may still want each profile to additionally link to G+, until it's established whether they'll follow more than one hop of rel=me for this purpose.

Over the bank holiday weekend I found time for some maintenance on the Android port of Simon Tatham's Puzzles. For users, this mostly means a few interesting new grid types in Loopy now that I've caught up with upstream. As always, you can update on Android Market, or manually on the project website.

Developers might find it more interesting: you can now use Eclipse or Ant to build and run the entire project out of the box (as in, immediately after a git clone, if you have the prerequisites installed), instead of the previous Perl/Makefile-based system. This gives you all of the debugging tools from Eclipse and the NDK, and the ndk-gdb script has already been worth the effort by itself. It also means I'm not using horrible unsupported methods to build the package…

The project is still mostly in C, and that won't change, but now that you can build and run it just like any other Android project, you really don't have to worry about that unless you're actually making a substantive change to the C code.

Anyone who would like to get involved can find all the gory details on github, specifically in the README file. That file and the issue tracker between them have plenty of ways you can lend a hand.

World IPv6 Day is 1 week away, and for what it's worth, I'm joining in. On Wednesday 8th June, if you visit my site or any of the other participants, such as Google, you'll be helping to test whether mass deployment of IPv6 is likely to break anything.

Here's a good explanation of the experiment. The tl;dr version: you probably won't notice any difference; you can try it out here; if you discover on Wednesday that you can get to Google but not to my site, then it's more important than usual that you tell me.

Oh, and if the experiment doesn't appear to cause anyone any significant problems here or elsewhere, then I may well leave it in place (plenty of sites have already been doing the same for years).

I'm now the proud owner of a Crypto Stick, an OpenPGP smartcard implementation in USB stick form. It is apparently capable of generating and using 4096-bit keys, but GnuPG has a limit of 3072 bits (quick tests and git master show this is still the case). Fixing this doesn't sound that hard, from that message; I'll be taking a look at this when I get time and energy. Given my track record on getting time and energy to do such things, don't let that stop anyone else from working on it. :-)

For my latest ugly hack, I wanted to give any/every machine in the house a globally-routable IPv6 address, and I didn't want to have to babysit the setup every time the router's IPv4 address changed. I actually already had 6to4 up and running on one machine (but failing these two requirements) since this post a couple of years ago. To extend it, I've borrowed heavily from Michael Wensley's howto, but used a different approach which I find more readable. (Danger, automatic modification of config files ahead…)

The idea here is for one machine on your network to set up a tunnel to your nearest 6to4 relay, thereby receiving an IPv6 prefix of the form 2002:wwxx:yyzz::/48, which it can then share among the other hosts by autoconfiguration using radvd. The machine on which you do this doesn't have to be your router, but it will be the gateway for all IPv6 traffic, so pick something that's usually powered on. Don't pick a machine on which you want to run Steam, because you'll want it to be the router's DMZ destination, and Steam bizarrely refuses to run in a DMZ. It also doesn't tell you that this is the problem. Yes, I did find this out the hard way. :-)

Having chosen a machine, add something like this tun6to4 stanza in /etc/interfaces, but replace abc with the internal address of this machine, and replace both instances of wwxx:yyzz with your router's current external IPv4 address, in hex. A quick converter: echo 11.22.33.44 |sed 's/\./ /g' |xargs printf %02x%02x:%02x%02x\\n

Before bringing that interface up, be sure this machine has a firewall for both IPv4 and IPv6, because you're about to have some shiny new globally reachable attack surface of your very own. :-) This should involve ip6tables -P INPUT DROP; ip6tables -P FORWARD DROP (or words to that effect) and poking holes as needed. There must be far better examples around, but this is the general idea. When you're ready, set your router's DMZ setting (or whatever it calls the machine to which to pass inbound traffic by default) to this machine. You actually only need IP protocol 41, but I don't expect any home router can be that specific. :-) Run ifup tun6to4. You can now try ping6 ipv6.google.com (or browsing there).

Sharing the fun with the rest of the house isn't any harder: apt-get install radvd and use this /etc/radvd.conf or something similar (replace wwxx:yyzz again). When you restart radvd, machines should begin to acquire IPv6 addresses in this subnet. If not, see what they say to sysctl -a |grep accept_ra and dmesg. Note that my firewall example above deliberately prevents inbound traffic to them until you add something like the commented FORWARD line.

Now for the ugly hack to update all of this. It's quite simple, just substituting the prefix as necessary. You'll need to edit it for your particular router; you may need to authenticate and might not find the IP at its root URL for example. Your firewall arrangements may also differ. Here it is: update6to4.sh, and you'll want to add this to /etc/crontab, perhaps every few minutes (if the IP hasn't changed, it exits without writing anything).

Happy hacking. :-)

LiveJournal and other sites using the same software don't provide a feed of your friends/reading page, something that has irritated me since I signed up. As more of your friends migrate to Dreamwidth or elsewhere (here's the latest reason) and more people thus suddenly want to keep up with multiple LJ-like sites, I'm guessing this will be an increasingly common irritation. These sites provide feeds of individual friends, but when you're reading a couple of hundred (making your feed reader's update cycle take forever), and/or you frequently edit the list, this is not ideal.

In late 2008 I thought well, why don't I just put a feed merger on a cron job? and so I did. I've been using it since then, and have finally got around to tidying it up and releasing it. On the project page, you'll find the script and instructions. It's by no means a complete or universal solution: you need to have your own web hosting and be able to set up authentication. Still, a few people have kept asking me for this, so for them at least, here it is: Amaljamate.

I recently bought a Bluetooth keyboard to use on Android; unfortunately some keys weren't recognised at all and others weren't quite mapped as I'd like. This is fairly easy to fix as an end user, as long as you have a rooted device (adb remount must succeed). Here's a quick howto, since I couldn't find one.

It shouldn't need saying, but you do this at your own risk. You can easily break your device's response to keys, and possibly make your device unbootable too if the file is invalid.

It's pretty much as described under Keymaps and Keyboard Input in the porting documentation. What we're going to do is edit one file: /system/usr/keylayout/qwerty.kl. This file maps scancodes to Android keycodes. For example: key 30 A means scancode 30 (decimal) maps to KEYCODE_A (29), the A key. As the porting docs say, a separate Key Character Map file deals with what characters, if any (e.g. "a", "A", "#"), should be generated given the keycode and the state of the modifiers (Shift, Fn/Sym, etc). A different KCM can be loaded for Bluetooth keyboards depending on their device name, but at least by default, the same key layout file (including your changes) is used for Bluetooth and the built-in keyboard (if any).

So, first we need to fetch your current key layout, and it's a good idea to keep backups (both on the device and locally).

1 adb pull /system/usr/keylayout/qwerty.kl qwerty.kl
2 cp qwerty.kl qwerty-orig.kl
3 adb shell cp /system/usr/keylayout/qwerty.kl /sdcard/qwerty-orig.kl

Now, edit qwerty.kl as you like. It should look initially look similar to this Nexus One example. You might find it helpful at this point to see what scancodes your keyboard is sending. There really ought to be a tool for this in Dev Tools or Spare Parts, but it seems there isn't. I've made a dirt-simple app to show you KeyEvent.toString(), which was the work of 5 minutes and could certainly be improved.

My particular keyboard is a white roll-up 87-key variety, that you can still find on eBay. It has F-keys doubling as media keys: if you get this model, be aware that the Fn key is really F-lock (toggles the state rather than acting as a modifier). Here is my layout file in case it's helpful. This layout mostly follows the principle of least surprise, except that the button with "Home" printed on it is Call, to mirror "End" being Hangup. F1 with F-lock on (Home symbol) is the Android "Home" keycode (goes to the home screen).

Now copy the layout back onto the device:

1 adb remount
2 adb push qwerty.kl /system/usr/keylayout/qwerty.kl

(If the remount fails, you probably don't have a rooted device.) It seems you need to reboot for the changes to take effect, so go ahead and do that, and your keys should then behave as you've specified. :-)

Now that my various projects seem to be mostly behaving themselves, it's time to decide what to write next. I've had a few requests and a few ideas, so here's a rundown of possibilities.

• Locale condition: ping. Let Locale ask "is that machine awake?" It could ping something in a private IP range, deduce that you're at home and your PC is on, and stop notifying you about things you're already likely to see on the PC. There seems to be a usable ping command in Android's busybox, but for bonus points, allow a TCP ping, maybe check the banner on a service, to guard against collisions with private IPs in other places.
• Locale action: K9 settings. It's been done.
• Locale action: ConnectBot. You can already configure a ConnectBot session to immediately run a command. Launching a remote shell script securely from Locale would add some nice flexibility: aggressive notification using PCs around the house, X10 home automation, garage doors… you could do this from ASE with an ssh client binary, but that's a little harder to install, especially without root.
• Locale condition: Timer. My timer, not yet formally released, could trigger the aforementioned aggressive notification if alarms have gone unanswered for half an hour. Update: Done.
• Port this site to Rails 3. This can wait until v3 is actually out.
• Puzzles improvements. Some ideas languishing in the bug tracker, including multiplayer.
• Set up mpd (with neompc on an old Palm Treo) at home. Probably no new code needed, but included for completeness.

Most of these will probably happen eventually, but it's always useful to know which ideas are useful to a wider audience, so feel free to comment or contact me if you're keen to see one of these created, or if you can think of improvements or existing projects I ought to be aware of. I'll also be writing up any interesting aspects of these projects as I implement them, and suggestions for posts of that kind that you'd like to see are also welcome.