Roo Reynolds - What's Next?

MeeTimer and myware and SQLite

Posted by Roo - 31/03/09 at 11:03:06 pm

I’m interested in the idea of self-interested self-surveillance. Long before we had PMOG (the Passively Multiplayer Online Game, now called The Nethernet) to make a game of it, Seth Goldstein was calling the idea ‘myware’ and building the (short-lived) AttentionTrust site. As Fred Wilson said at the time, “If someone is going to spy on you, it’s probably best if its you.”

With that in mind, I installed MeeTimer over the weekend. It’s a Firefox plugin which…

records where you spend your time online. It does it in a rather useful way, by allowing you to group websites into activities … so you can make sense of where your time is going. Finally, it accumulates time spent on a site over the course of a day…

I’ve been using it for 3 days and it’s giving some interesting food for thought.

You can even optionally set up ‘tab warnings’ on specific groups (sites you’ve labeled ‘Procrastination’, say) which will pop up with a nice overlay telling you exactly how much time you’ve wasted in this site, and others in the same category (though allows you to click through and ignore the warning just this once or for the current browsing session if you still want to). I’m already finding this feature useful on the handful sites whose feed I’m subscribed to but for some reason still find myself visiting out of habit. (For me, it’s Waxy links and Boing Boing. I love them, but I’d rather be reminded to enjoy them as part of my feed reading routine rather than browsing out of habit. I bet you have your own which make you ask is this really what you want to be doing right now?). A little reminder is really useful for habit-breaking here.

Mostly MeeTimer is just quietly keeping track of a bunch of per-site accumulators, cleverly based on whether Firefox has focus and which is the currently active tab. The results are already interesting. I realised that I was spending a bit less time on Twitter and Flickr, and a bit more time on work webmail, than I thought.

This is all very well, but I want more. Specifically, I wanted to get at the data. Not just the accumulated weekly/daily/monthly (etc) totals and averages, but the number of visits to each site per day. The raw visits. In as much detail as possible. I want CSV exports, or an API, or something. If I’m spending a daily average of 21 minutes on Twitter, how many visits comprise that time? MeeTimer simply doesn’t tell me.

Or does it?

Digging around my Firefox profile, I find a very interesting file at /Library/Application Support/Firefox/Profiles/{profile-id}/meetimer.sqlite. Ooh, I bet I know what that is. So I open up SQLite and start poking.

Sorry. It’s about to get a bit dull from here on in. Unless you get excited about the idea of being able to manipulate this data you’ll probably want to scroll down to the end. Honestly, I won’t mind.

They’ve gone? Right. Let’s get hacking.

$ sqlite
SQLite version 3.6.12
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .restore meetimer.sqlite
sqlite> .tables
deterrent_stats  groups           log
deterrentlinks   groups_urls      url_maps
deterrents       ignored_urls     urls

Excellent. We’ve got tables with sensible names and everything. Let’s see what log looks like.

sqlite> .headers on
sqlite> select * from log limit 3;
url_id|startdate|duration|day|week
4|1238324612508|3|200987|200913
5|1238324617244|44|200987|200913
6|1238324647668|17|200987|200913
sqlite> select * from urls limit 3;
id|url
1|mail.google.com
2|www.google.com
3|www.google.co.uk

Lovely. Easy enough then. The groups and groups_urls tables do what you’d expect too. For now, let’s make url_id more meaningful by doing a join with the url table.

sqlite> select
  url_id, duration, day, week, url
from log
left join urls on log.url_id=urls.id
limit 5;
url_id|duration|day|week|url
4|3|200987|200913|google.co.uk
8|40|200987|200913|meetimer.productivefirefox.com
4|16|200987|200913|google.co.uk
11|10|200987|200913|technorati.com
12|14|200987|200913|google.com/reader/

What if we wanted to show the number of visits, the total duration, and the maximum length of duration for visits to Twitter…

sqlite> select
  count(url_id), sum(duration), max(duration), url
from log
left join urls on log.url_id=urls.id
where url = 'twitter.com';
count(url_id)|sum(duration)|max(duration)|url
34|2712|455|twitter.com

Excellent. I wonder what the top seven URLs when ordered by the number of visits?

sqlite> select
  url_id, count(url_id), sum(duration), max(duration), day, week, url
from log
left join urls on log.url_id=urls.id
group by url
order by count(url_id) desc
limit 7;
url_id|count(url_id)|sum(duration)|max(duration)|day|week|url
9|34|2712|455|200989|200914|twitter.com
10|30|1075|249|200989|200914|search.twitter.com
1|22|2505|928|200989|200914|mail.google.com
4|20|206|57|200989|200914|google.co.uk
17|18|476|114|200989|200914|flickr.com
21|10|2480|2125|200989|200914|bbc.co.uk
39|8|13152|10212|200989|200914|webmail.bbc.co.uk

Twitter, with 34 visits. Sheesh. And for comparison, the top 7 sites by total duration of visit?

sqlite> select
  url_id, count(url_id), sum(duration), max(duration), day, week, url
from log
left join urls on log.url_id=urls.id
group by url
order by sum(duration) desc
limit 5;
url_id|count(url_id)|sum(duration)|max(duration)|day|week|url
39|8|13152|10212|200989|200914|webmail.bbc.co.uk
9|34|2712|455|200989|200914|twitter.com
1|22|2505|928|200989|200914|mail.google.com
21|10|2480|2125|200989|200914|bbc.co.uk
12|6|1355|633|200989|200914|google.com/reader/

13152 seconds (3.6 hours) on my work webmail between Sunday morning and Wednesday aftenoon. And all done in 8 visits. Yuck.

Ok. Let’s start thinking about daily summaries. Grouping by day, and then by URL (since I’m not very good at SQL, and don’t know how to limit it to 5 per day, I’ll just manually snip out all but the top 5 for each day for now)…

sqlite> select
  url_id, count(url_id), sum(duration), max(duration), day, url from log
left join urls on log.url_id=urls.id
group by day, url
order by day, sum(duration) desc;
url_id|count(url_id)|sum(duration)|max(duration)|day|url
1|2|306|228|200987|mail.google.com
9|6|296|217|200987|twitter.com
12|2|225|211|200987|google.com/reader/
28|1|128|128|200987|hunch.com
21|1|66|66|200987|bbc.co.uk
[...]
39|3|10222|10212|200988|webmail.bbc.co.uk
21|3|2155|2125|200988|bbc.co.uk
9|18|1494|235|200988|twitter.com
1|12|1003|185|200988|mail.google.com
10|14|777|249|200988|search.twitter.com
[...]
39|5|2930|2667|200989|webmail.bbc.co.uk
1|8|1196|928|200989|mail.google.com
9|10|922|455|200989|twitter.com
12|1|394|394|200989|google.com/reader/
21|6|259|151|200989|bbc.co.uk
[...]

And returning to the original question of just how many visits do I make to Twitter

sqlite> select
  count(url_id) as visits,
  round(sum(duration) / 60.0, 2) as total,
  round(max(duration) / 60.0, 2) as longest
from log
left join urls on log.url_id=urls.id
where url = 'twitter.com'
group by day
order by day;
visits|total|longest
6|4.93|3.62
18|24.9|3.92
10|15.37|7.58

So it seems that on Sunday I made 6 visits for a total of about 5 minutes and a single longest session of 3 and a half minutes. On Monday it was 18 visits for a total of 25 minutes including one session of nearly 4 minutes, while today, 10 visits so far (including one of over 7 minutes) have already added up to over 15 minutes.

.mode csv

in SQLite is handy too, because it changes that list format to look like

visits,total,longest
6,4.93,3.62
18,24.9,3.92
10,15.37,7.58

so it’s trivial to open it in a spreadsheet.

Even better will be something cunning and programmatic. Maybe in PHP or Ruby or something. Even this exploratory manual approach is fun though. It will obviously be better once I’ve built up a bit more history but now I know that MeeTimer is storing my data in a way that I can access it, I’m even more excited about it. Thanks, MeeTimer. You rock.

More Microprinting

Posted by Roo - 23/02/09 at 11:02:20 pm

I’ve been experimenting a bit more with the thermal receipt printer I bought recently. Inspired by Tom’s daily digests I’ve been trying some of my own.

You only get 48 characters per line using the default font. The alternative font (font B) is much denser, with 64 characters per line. The second printout is only about an inch longer than the first one, yet has twelve additional lines of content.

The barcode at the bottom is a sort of physical permalink using a Code 39 barcode. I’m thinking that each daily digest could also exist in a (private) blog, and a barcode (complete with text date stamp) could be a handy way in. If you’re using this code, or something like it, you could do this…

setBarcodeTextPosition(barcodePrintBelow);
setBarcodeHeight(45);
setBarcodeWidth(barcodeMedium);
printBarcode("/2009/02/22/", barcodeModeCODE39);

I used EvoBarcode Scanner to test reading it back in.

More ideas for a daily digest:

• today’s calendar entries (like Tom’s)
• recent Twitter @ replies and Direct Messages
• recent emails
• friends birthdays via via Facebook
• Radio 4 schedule
• anything anyone wants to send @roo_printer
• …

Tonight I hooked it up to Twitter. Every minute to checks to see what my contacts are saying and prints whatever is new since it last checked (usually 2 or 3 updates).

30 minutes of my friends’ twitter updates equated to five feet of paper. I don’t think I’ll be running this all the time but it does feel reassuring to have it whirring away in the background.

Another use of the Microprinter: printing books. I took the text of Cory Doctorow’s Down and Out in the Magic Kingdom (mainly because I can). It has over 47,000 thousand words, and if you print it at 64 characters per line on standard 80mm thermal paper it’s about 60 feet long.

Printing time: about 40 minutes (pausing briefly after every paragraph to let the printer catch up). Rolling it back up again took nearly as long.

At Nick‘s very cunning suggestion there are perforations at every chapter (as well as every sub-chapter, which the ASCII text denotes using a ‘#’ character on its own). Together this divides the book into 59 perforated segments which are about 30cm long on average. Rather than needing a bookmark I’ll just tear off the sections as I finish them.

It’s a portable, recyclable, tear-and-shareable book.

Microprinter

Posted by Roo - 01/02/09 at 11:02:16 pm

Inspired by Tom Taylor’s microprinter project, I’ve bought a Citizen CBM-231 thermal reciept printer of my own. I picked it up for just £20 on eBay, including shipping. It’s great.

Tom uses his to print the weather, his diary, where his friends are (according to Dopplr) and more. As soon as I saw it, I wanted one of my own to hack with. Reciepts, printed on cheap and recyclable thermal paper, are perfect for directions, schedules, TODO lists and other impermanent bits and pieces you might want to carry while you’re offline. I also like the idea of it politely telling me what I’m up to as part of my morning waking-up ritual. I have a feeling that the soft sound printing and the ‘clunk’ of the auto-cutting blade will be a nice start to the day.

(More photos)

A few hours of soldering and programming later, and I’m quite a happy hacker. I’ve put an Arduino sketch on github which shows how to easily print text and barcodes to the printer from an Arduino. It’s just a sketch at the moment, but I’ll turn it into a reusable library soon.  With a few utility methods and constants, a “hello world” with two barcodes ends up looking as simple as this…

println("Hello, World!");
feed();
setBarcodeTextPosition(barcodePrintBelow);
setBarcodeHeight(200);
setBarcodeWidth(barcodeWide);
printBarcode("123456789012");
feed();
setBarcodeHeight(50);
setBarcodeWidth(barcodeNarrow);
printBarcode("123456789012");
feed();
cut();

I think it can print bitmaps too. With a bit of work it should be able to print sparklines and QR Codes.

I know Tom has inspired a lot of people, and there are quite a few of these Citizen CBM-231 printers being repurposed at the moment. If you’re interested in building your own microprinter, you’ll hopefully find the wiki at microprinter.pbwiki.com useful.

Update: more microprinting fun including a book and sparklines.

My Computer(s)

Posted by Roo - 08/12/08 at 10:12:22 pm

I have owned many computers, and not all of them have had names. The Commodore 64 and BBC Micro B I grew up with didn’t ever have names. Nor do I remember giving one to the first PC my parents had (a second hand 286 IBM PS/2).

I think I’ve forgotten a few, but here are the names of the computers I have owned, named and remembered:

• Patience (my first desktop, bought during the first year of my degree, and named after the attribute I felt I had demonstrated while selecting and purchasing it. I had a love/hate relationship with the shop, the name of which I’ve now forgotten Something Squared? M², maybe?)
• Portaroo (my first work laptop while at IBM, a ThinkPad 760, as seen on the International Space Station). I still love this name. like Portaloo, with my nickname built into it. Oh, you got that already? Sorry).
• Parity (so-called because I was catching up with my friend and then-housemate Cheesy, who had upgraded his machine at the same time. I think this is a photo of me building it, with Cheesy to my right and Mark sat behind me. We had a lot of fun in that house)
• Quiss (a work desktop, named after a character in an Iain Banks novel)
• Roochelmini (Not the best name, but Roo + Rachel’s Mini = roochelmini. The Mac Mini we had in our living room. It’s a bit poorly at the moment).
• Rupert (a ThinkPad T42p, and now returned to the big warm blue bosom of IBM)
• Shuttle (a Shuttle mini-ITA PC. Not very imaginative. I should have at least called it Apollo or something. It’s now significantly unwell, often taking ages to start up only to power itself down in the middle of playing a game. I have not touched it for a couple of years, but I expect that if I wanted to I’d need to replace the power supply and/or motherboard)
• Sebastian (MacBook Pro. Bought this January. I love it. Looks a bit like this)
• Tristan (MacBook Air. A work machine. It’s less powerful than the Pro, but so light that I love commuting with it. Looks like this, though I should grab a photo of the stickers on the lid. Notice the alphabetic sequence here? My next machine will probably have to begin with ‘U’)
• Moby (named not after the musician, but the great white whale. Strictly speaking, I didn’t actually own this, I just borrowed it for a few weeks.

Toshiba Qosmio G40

Posted by Roo - 06/12/08 at 12:12:49 am

I recently received a very generous and interesting offer from TalkToshiba; they offered to lend me a laptop on the condition that I write an honest review of it. I get to play with a nice toy for a few weeks, you (and they) get to hear how I got on with it. Sounds fair to me. Let me make that perfectly clear: if the offer had been on the condition that I write a positive review, I’d have said no. The fact that they asked me to “post up your thoughts about the laptop on your blog … whether they be good or bad” and being able to tell the truth about the machine is the only reason I even considered it.

Unpacking it (from a big, heavy box that I’d assumed would be mostly packing material. Oh no, it really is that size) my first reaction was that I had never seen a bigger, heavier laptop. Opening it, I was struck by the distinctive design. Shiny, intricate and odd. Over time, that wore off and I now think of it as odd, and more than a tiny bit irritating. That’s partly because this isn’t the right laptop for me. Commuting every day means I value portability. Don’t expect this to be portable. It truly is a desktop replacement. In fact, you’ll want to plug in a mouse and keyboard too, because the layout is pretty dreadful.

On the plus side, it is quite powerful, has every connection you’d ever need, and the sound quality is amazingly good. When it did sometimes feel sluggish, I blamed the fact it was running Windows Vista. Oh, how I hate Vista. That’s not Toshiba’s fault though, and I should have installed Linux really.

Here’s what it looks like. The speakers vents are huge, and the visual aesthetic here seems to be ‘turbine’.

It’s big. Here it is stacked up against my wife’s MacBook and my MacBook Pro. The two put together are almost exactly the same height as the G40.

And here it is up against my MacBook Air. Perhaps not a fair comparison, but look at it. Insanity.

It’s covered in unnecessarily bright and numerous blinkenlighten. Not very soothing on the eyes.

The biggest problem, especially given the machine’s generous proportions, is having a teensy-tiny trackpad with two teensy tiny buttons, with a fingerprint device right in the middle, just in the way. The design is, frankly, dreadful.

The MacBook Air, despite being a much smaller laptop, makes room for a good-sized trackpad. There’s no excuse for a monster like the Qosmio G40 to have me scratching around on a surface half the size.

Good points

• I liked having a fingerprint reader to log in. Probably my favourite thing about it, and the one feature I now miss on my MacBook Pro and Air
• Having 5 (!) USB ports, and good connectivity generally. HDMI, s-video, SD/Memory Stick etc, even coax TV-antenna, I was almost expecting to see a SCART socket on this thing
• Good speakers, nice and loud with the best and most sound quality I have ever heard on any laptop
• Reasonably powerful

Bad points

• Unnecessarily ugly with lots of wasted space. 17″ inch screen feels small
• The screen seemed quite dim too. Certainly dimmer than the Pro or Air, even when powered by mains and turned up all the way
• Dreadful layout: tiny little trackpad with tiny little mouse buttons and a fingerprint reader plonked in the middle of it making it even more uncomfortable to use. I like the fingerprint reader, it’s just in the wrong place. The whole layout somehow manages to feel sprawling and cramped at the same time; I kept pressing the navigation wheel thing on the right when reaching for Return (pressing the soft touch ‘back’ button)
• No way (that I found) of dimming the enormous numbers of decorative lights
• HD-DVD. Seriously. I think the battle between BluRay and HD-DVD has been decided, hasn’t it?

It’s doesn’t really matter though because, being over a year old now, Toshiba no longer sells this laptop. The G50 has an even bigger (and I hope brighter) screen, but I don’t think I’ll be buying on. I like my laptops to be something I can put on my lap without fear of injury, and I returned the G40 without being terribly sad to see the back of it. Thanks to TalkToshiba for the loan though.

(More photos on Flickr if you’re interested.)

Tasty Tag Pages

Posted by Roo - 09/11/08 at 03:11:54 am

I’ve been improving Watchification and Speechification tonight.

Part of the enjoyment of both sites is the idea that we’re not just curating our favourite stuff, we’re weaving links between it, with those links becoming increasingly fun to explore. Tonight’s hack was a small improvement in order to pull in a few things that the web knows about every tag, but also every presenter, director, editor, and in fact any search term. Here’s what happens when you search Speechification for shows in which Stephen Fry is the presenter.

It works for normal tags too, and they’re handled in the same way. Ever wondered what Speechification has on Malcolm X? or Chris Watson?

I took exactly the same approach at Watchification, though had a bit more room to play with in the layout.

Other examples: ‘politics’, ‘music’, and shows in which Simon Amstell is the presenter.

As well as the obvious embed of (Creative Commons licensed) photos from Flickr, I used Monitter for the realtime Twitter monitoring, and a delicious.com feed of relevant bookmarks.

I have lots more ideas for other things to usefully enhance the page too. Watch this spaces. Um. These spaces.

Update: at the suggestion of Dan Hil, I’ve moved the widgety stuff to the bottom of the page.

This could be heaven or this could be hell…

Posted by Roo - 05/11/08 at 05:11:32 pm

Roo Reynolds plays “Hotel California” from Tom Armitage on Vimeo.

Thanks to Tom for capturing this video. That was fun.

Tardis

Posted by Roo - 17/10/08 at 11:10:49 pm

I was fortunate enough to be given a tour of the BBC Cardiff set today. Dr Who, Torchwood, Sarah Jane… brilliant!

Generally my camera was firmly switched off (there’s a big agreement on the way in about not taking photos or video, to avoid revealing spoilers) but I was told it was OK to take some shots inside the console room of the Tardis.

It really is bigger on the inside.

This is why we can’t have nice things: TechCrunch

Posted by Roo - 14/10/08 at 08:10:26 am

I stopped reading TechCrunch a while ago, so it wasn’t until I saw it mentioned on Loic’s blog recently that Michael Arrington annoyed me all over again, this time by crowing about a holiday video. At the risk of feeding the troll, I want to make it clear how stupid and utterly irritating Michael Arrington’s post about it is.

I watched and enjoyed the video yesterday afternoon. I’ll embed the YouTube version which TechCrunch links to, but I’ll warn you now that it’s already been taken down.

It depicts a group of young people on holiday together, in a large poolside villa. They’re lip-syncing to ‘Don’t Stop Believing’ by Journey, in a surprisingly well-orchestrated performance. The original description of the video apparently said

“Twenty world Internet citizens met in the Turkish Republic of Northern Cyprus in October of 2008 for a week of reflections on life, love, and the Internet.”

Harmless and fun, right? Here’s what Michael Arrington, writing on TechCrunch, had to say about it on Friday:

…this went down at an unfortunate time for a score of Silicon Valley posterboys and girls as they partied 1999 style “the Turkish Republic of Northern Cyprus in October of 2008 for a week of reflections on life, love, and the Internet.” They leave behind an absurd video that would have gone unnoticed a month ago. But this week, with the walls tumbling down, they look like a bunch of jackasses who have no idea what’s going on back at home.

I’ve personally got a lot of sympathy for the partyers. I’ve not heard of any of them, but it sounds like a bunch of people who are tech sector friends, taking a (probably much needed) week off and having a good time on their own money. At worst, it’s bad timing, but it’s not exactly the recent half-million dollar AIG retreat is it? I’m pleased they were able to take a holiday together, and I’m pleased they took time out of their vacation to entertain us with this amusing and well-executed video. Let’s not be puritanical or sanctimonious about it, eh?

We’ll look back in later years and think of this most recent boom as the Web 2.0 period, when we were wowed by the magic of user generated content, copyright violations on a massive scale, and neat little widgety things that used Javascript and Flash to turn web pages into pretty close equivalents to the old desktop apps.

Really? I like to think we’ll look back at the early years of the 21st century as a time when the web evolved into something participatory. Isn’t that, rather than technology or investment, what ‘Web 2.0′ is all about? Here’s Tim O’Reilly, explaining the term, in a video which lasts less than a minute.

Worst of all though, and a sentence which gets repeated in an even more unnecessary follow-up post pointing out that the video has now been made private but has sprung up on YouTube:

this video will always be associated with the end of Web 2.0.

Honestly, if Michael had thought to include a couple of extra words in that sentence, I might almost have agreed with him. This week is not “the end of Web 2.0″. If anything did just burst it’s the financial bubble which the Economist discusses as ‘Bubble 2.0‘. You can’t confuse that with Web 2.0 though. They’re different things. Is this really all Web 2.0 means to you, Michael? Were you not listening when you asked all those people what Web 2.0 means in that 24 minute long documentary you made on Web 2.0 in 2006? Some highlights from what they told you:

• Web 2.0 is a marketing term, and a bubble is a financial event.
• it’s people
• empowering the little guy
• users manipulating, interacting with, and sharing information
• leveraging collective wisdom
• openness
• conversation, not a lecture
• participants, not consumers

In short, Web 2.0 is an attitude, not a technology.

But back to the video. I watched it two, maybe three times yesterday. I enjoyed it. The original video on Vimeo had already been made private, so I was watching the ‘mirror’ on YouTube. Today, that the YouTube version has been taken down as a copyright infringement (“This video is no longer available due to a copyright claim by a third party.” initiated, I’m guessing, by the creators?). I’m disappointed that, unless you’ve already watched it or some other public backup of it exists somewhere, you can’t watch it to decide for yourself.

Playful

Posted by Roo - 08/10/08 at 08:10:58 pm

Playful is a one-day conference about games, happening on Friday the 31st of October, 2008 in Conway Hall, Red Lion Square, London. Add it to Upcoming or Facebook or, you know, just register.

Focusing on the creative and cultural dimensions, Playful examines game design as both a discipline and craft, offering different perspectives on its current and future possibilities.

I’ll be presenting (I’m on second apparently, following the inimitable James Wallis) and I’ll be showing how Rock Band / Guitar Hero controllers can, with very little money, time and effort, become actual musical instruments. This will be the first conference for which ‘pack plastic guitar, real amp and leads’ are on my TODO list.