Sunday, May 25, 2014

Lazy Sunday Service

I've finally completed the index for The Data Revolution book.  In the end it took a fair bit longer than I anticipated to put together, c. 30 hours.  I could have rattled one out a lot more quickly but I wanted to produce a really comprehensive, useful one, and my sense is it's the best of the twenty or so I've done to date.  I just hope it gets used after all that effort!  I've also got the book's website up and running.  I'm hoping that's everything with respect to the book, but I suspect I'll also need to do a video abstract at some point.  I had to make two of these last week - one for a project and one for an academic paper.  It seems that's the trend these days, to supply supplementary material such as blog posts and videos that will help increase the main material's discoverability.  The one good thing from this is I learnt how to use some good video editing software and how to put together a professional(ish) video, and found that I enjoyed the process (despite the frustration of trying to do it on an underpowered machine that kept crashing).  Once the videos have been released I'll put up links to them.

My posts this week
Review of Another Case in Cowtown by Mel Healy
Workshop: Code and the City
Review of Keep Away From Those Ferraris by Pat Fitzpatrick


Anonymous said...

Hi Rob,

Sorry if this sounds really cheeky, but your travails with the hard slog of indexing struck a chord. I did it once for a friend's book. Never again.

Mind you, some people might call the whole process a tad ironic. There's you writing a book about the frontiers of data revolution yet when it comes to the indexing you don't download a free...

(Hey - hold on a sec - and bear with me, even if I'm about to use the word "yet" about a gazillion times.)

There's something really important about the key differences between tasks that software is really brilliant at, and certain other kinds of work that we humanoids have to take our time about yet in the end we're really not that bad at it.

Take indexing. Thirty hours, all that thought and hard graft, and many people would think there's a bit of software out there to do all that. Yet it's rubbish.

Don't get me wrong. Software can be great at so many things. Yet it has its limitations.

For example, I'm thinking of my friend Tony here, Tony a good mate from Inchicore who's a dab hand at programming. I guess Tony would take this indexing task as a mainly quantitative, brute-force problem: "Ah yeah, no problem Mel, that's the very kind of word-crunching straightforward task that computers are perfect for. Knock up a bit of code, point it at the data, hit ENTER and off you go..."

How would Tony define this particular problem? In the slippery language of some programmers? He'd probably talk mainly about solutions.

"Ah no listen Mel - no problemo, we knocked up this thing in C++ or Java the other day to build up a concordance thing - ah just look it up in Wikipedia or whatever - and, hold on... see the menu in the app? Just tap START. See? It creates this lovely colour-coded map and a nice tree of your book's contents. Magic."

Sorry, that's the theory. Or in some cases the sales pitch. Or in Tonyspeak (I really hope Tony doesn't see this or he'll kill me. OK, call him Barry.)

As far as I can see, though, so far those same computers seem to be still pretty useless at mapping out the themes of a real-life book.

Why is that?

Because making an index is a surprisingly intense form of writing, and a certain way of map-making.

It's far from number crunching and frequencies. In a way it's the author telling you a meta-story, about the main territories and routes through his or her storyL what the key themes are, the characters, the defining moments...

Yet all this requires a type of conceptualising and comprehension about those key themes, a certain mapmaking kind of skill about an overall project that even super-intelligent supercomputers - sorry Tony, even despite its current state of evolution - sadly seem to lack.

Sorry Rob. Rant over. I use computers to do all kinds of wonderful things every day, yet while we should celebrate the things that software can do that's far, far better than we can do, I also love taking time out to wonder about the surprising things that our human brains are quite good at. Like indexing.

So well done on the 30 hours.

Oh yeah, and all the best with the new book!

Rob Kitchin said...

Mel, thanks for the extended comment. Yes, it would be great if there was a bit of software that could do a decent job. As it is I would still need to go through the whole book to find the relevant terms I'd want to make a decent map of the book. Even then you couldn't just do a word search as the search has to be contextual given meaning can be ambiguous or words/themes be used interchangeably. Plus you might want reference entire sections or chunks of text in the index. So, yes, I think if you want a really good index you need to do by hand. If you want an okay one, you could probably get away with some indexing software.