Yesterday, I went to Apple’s iOS Tech Talk held in Seattle out of all places (how could I not?) and was excited to meet a slew of other developers with whom I previously only interacted online or via their apps.  It was quite a trip – I guess I should climb from underneath my “rock” more often.

Besides all the socializing, I sat through a number of lectures delivered by the Apple folks on the wonders of the iOS technology.  One of the more interesting sessions centered on profiling, whose overarching message was “Don’t guess – measure!”.

If you’ve ever written code with tight performance requirements, “thou shalt measure” is a well known commandment.  What I recently discovered, however, is that measuring is equally important in the development of interactive books and apps in general.  Specifically, knowing how your customers use your product is paramount to figuring out what features are important and which ones are not.

Let me give you an example.

When I was writing Bobo, certain pages seemed more important to me.  For example, the book wouldn’t seem complete if it didn’t mention Edison at some point.  In addition, certain other pages appealed to me personally more than others.  For example, I was in love with the imagery of the Jungle / Photosynthesis page and spent good four days tweaking countless little details – from the blooming flowers to the swaying vines, paying particular attention to dynamically recreate jungle sounds from a collection of animal calls, avoiding repetitiveness yet mimic the overall impression of vibrant life.  In short, I really geeked out.

Dean was similarly enamored with the Bioluminescence page.  It all started with him sketching a beautiful yet menacing-looking angler fish.  From that point on, he wouldn’t rest until I finally caved in and spent four days on that page as well.  There was plenty to keep me occupied – animated fins on the fishes, several particle systems, fading colors with water depth, Bobo’s swimming movement which was unlike his movement on any other page, bubbles, water sounds, chomping angler fish, … you name it.  It was another point where we geeked out because of our sheer excitement (mostly powered by Dean) about the topic at hand.

Once we released the book into the wild, however, we were surprised that our users responded with page preferences completely different from ours.  The Introduction to Photosynthesis (a.k.a. the Tomato page), for example, is among the more popular pages in the book, even though we slapped it together in a single day to provide a much needed transition between some of the other topics in the book.

If I order all the pages by how much each of the topics appealed to me as a developer/user, I get the list in the left column.  If I order them by how much time I spent creating each, the list looks a little different, but not entirely dissimilar (middle column).  However, if I order it by popularity of users from all around the world, the list looks completely different (right column):

The Tomato page says it all.

The moral of the story is that the time we spent on the different book parts is incongruous with the amount of time people spend using it and, if we paused and collected some of this data during development, we would probably have adjusted our internal schedules and priorities accordingly.  Live and learn but, most importantly, don’t guess – measure often and repeatedly.