Tom Chi  

Theoretical Limits to Data Enjoyment

September 17th, 2004 by Tom Chi :: see related comic

Listening to a certain Rap song, I get 3 mintes and 35 seconds of enjoyment using 3,448,832 bytes of data. This translates to a data rate of about 16K/sec. Given that this is the data rate that I enjoy most of my music at, within a waking lifetime, I can handle about 25TB (assuming 80 years of non-stop rocking out without repeats).

Today, plenty of computers ship with 200GB HDs, and given a doubling rate of disk space of about 15 months, it will take 9 years before HDs are shipping that can hold more than a lifetime of music. In practice, no one can really listen to 25TB of mp3s so we’ll reach the mark even sooner (if not already).Looking on the scale of data rates for enjoyable things, music sits somewhere in the middle. I can enjoy a 400KB JPG from my digital camera in about 5-10 seconds (a data rate between 40-80KB/s), while a 100KB short story in text format could easily use up an hour for a measly rate of 0.027KB/s. In either case, add or subtract couple years, and you’ll derive the point where your drive holds more data than you can ever reasonably enjoy. The reason this matters is because our drives are ostensibly there to hold personal things that we will eventually use, but very soon (if not already), we will reach a point where there is more personal data available to us than we can ever parse.

So the question of the week is: If a tree falls in the forest… I mean rather… if you store personal data which is never accessed again, ever, does it matter that it was ever stored?

For me, a great personal example is digital photographs. In the past 3.5 years, I’ve taken about 10 thousand of them. I keep meaning to go back (if only to pick out the better ones and put them in a folder together) but in practice the rate at which I consume those old photos is still outstripped by the rate that I produce new ones. If these rates do not change, I will never, ever get to revisit a huge number of the photos I’ve taken. And unlike music or movies, this is a set of data which would not be that interesting for others to wade through. So it’s really up to me.

Similarly, I have a ton of old music I’ve worked on which is in various multitrack formats, waiting patiently for remastering (or even initial mastering). Realistically I will probably never listen to 90% of what’s there. The rest of it might as well be in data heaven already.

I’ve been thinking about this because I feel that it may define our relationship with data in the future. Currently, much ado is being made about Search, but enjoyment bandwidth is an even more fundamental limitation. While search can be tweaked and scaled and improved, enjoyment bandwidth is just there. There really is truly a finite limit to how much data can be consumed, and an acknowledgement of this will speak toward how much personal data it is reasonable for us to create.

The absolute, absolute limit would be to record all the inputs to a human sensory system at sufficient resolution that there is no discernable difference between playback and actual experience. This is pretty close to the ultimate virtual reality, but it still begs the question of how much life should be used in the creation of media, and how much used for consuming it. If I spent my first 40 years creating all this data, it would take me 40 years just to review it once. Even reviewing just the good parts might take more years than I am willing to spend. Thus, one wonders about the point of collecting all that data in the first place. All the never reviewed pieces are like the tree in the forest. Without observers their existence and their purpose are called into question.

In the future our hard drives will be huge, but they may be populated largely with forgotten data spaces.

11 Responses to “Theoretical Limits to Data Enjoyment”
Eric Svoboda wrote:

Volumes have been written on syllogomania (an obsessive-compulsive trash-hoarding disorder) - maybe there’s some “digital parallel” to be found there?

We “digital collectors” need to think about the trash we’re keeping and put a stop to it! Call it Digital Conservationism or Digital Environmentalism (seriously… I think the parallel with the biological environment works here…), we need to stop keeping more than we can possibly consume.

In addition, data storage does lead to bio-pollution, doesn’t it? Pollution is generated in the production and of media storage devices and the resultant disposal of obsolete media storage devices.

David Heller wrote:

While I agree that there is a pollution of data out there. I mean regulatory commissions have just made this even worse. I want to say that while out of Tom’s 10,000 photos he will never get to all of them in the next 80 years and most are useless to the world, it is the unpredictability of use that makes packrattin’ so compelling. You don’t see a use for it today, but that doen’t mean there will never be one. I am not a packrat (digital or otherwise) and I have to say that I’m greatful my friends are. Every 2-5 years I need something that I decided to delete but since they save EVERYTHING they have. I don’t think its been urgent enough to get me to change my behavior but it is interesting to note and if the disk space is there … I say use it! And yes, we hae to come up w/ better ways so that the initial storage process actually helps in the searching later. — dave

Kevin Cheng wrote:

Digital Packrat. Is that a term that’s been used before? I like the sound of it (though I definitely don’t fit in the category).

On a related note, on the ID Discuss list, an article on Automatic Icons Organizing was linked.

Gabriel Mihalache wrote:

Usually, my collection keeps getting larger and larger until it prevents me from getting new stuff, and at that point I delete eveything except those things I can identify, right there and then, and being useful.

I go through this cycle once every few months, more frequent now because I have old, small, HDDs.

A particular folder or file might make it through 2-3 such purges, but if it’s useless, it will eventually get deleted.

I’m *affraid* of the time when 5.1 music format files will get popular because I’ll need to get 3 times more storage space for music :-)

Dan wrote:

And what about DVDs?

Bob Salmon wrote:

My father-in-law said that for each roll of film he shot he’d go back and throw out all but 5 of them. Having physical space in your house be used up by printed photographs is a good way to influence your behaviour. I presume there won’t be this back-pressue over digital data as the device miniaturisation and data backup trends will keep up with the volume trend. (If not, we could farm this out to someone else i.e. a service provider, but then we trust their backup etc.)

What might be old rubbish to you might not be so to others, so then we have the concept of pooling these huge digital libraries. This doesn’t add any net enjoyment bandwidth over the community as a whole, but a given bit of information has a greater chance of popularity given more people. (Although the same could be said of the web, and check out Clay Shirky’s comments on the uneven distribution of traffic to blogs.)

A short jump from here to LifeLog type applications - one way of filling this vast storage. What are the privacy implications of this? The legal ones? (Could your own log be used against you in court? Could you edit it?)

another pink world wrote:

it has a name

unexpectedly discovered in an HCI (human-computer interaction) themed blog that i recently started reading, “OK/Cancel”… in an entry on enjoyment bandwith, ie, the amount of data a human can reasonably enjoy within a lifetime, in the first comment is…

Graham Storrs wrote:

Actually, I find that half the pleasure - maybe more - is in the creation of all this ‘junk’ rather than all of it being in the consumption. I too take many, many photos but each one is an act of creation - and a moment of hope and expectation - that I really enjoy. If I ever look at the photo again, that’s an added bonus! The same could be said for making and mixing music, writing of all kinds, programming, designing and so on.
Do you think I’ll ever read this note again? Not likely - but I enjoyed writing it :-)

Ian Hooper wrote:

Thanks Bob for the link to the LifeLog app. It led me to a series of links including Gordon Bell’s MyLifeBits, which I was trying to remember the name of as I read this article. The interesting point of MyLifeBits is just that - the challenge of access and recall. These were much greater problems than simply recording and storing. I think that no one expects to review a whole life, but the goal is to extend our memories to include perfect recollections of all our creations and experiences. So after 40 years recording, I don’t need to replay the whole thing, but if I want to know what was the fantastic meal I ate on the last day of my vacation to Nova Scotia 15 years ago, I could get that, along with perhaps some pictures or videos of me enjoying it. Like recalling a memory, I can spend as long or little time as I want recollecting it.

Jens Reineking wrote:

Once or twice a year, I do sort out my files and programs - with one exception: e-mail. What’s not deleted in the inbox will probably stay with me for years to come. Having the ability to backtrack communications is of real value to me.

Likewise, texts and layouts I created myself won’t get deleted, but moved to CD. I don’t need them on my drive, but I won’t let them go.

And all this not because of storage space (have enough of it), but because it simply _feels_ better to have an oderly environment. And I get to review what I have really done over the last year or so.

One special problems are link collections, mostly because of lack of metadata/contextual data. Sometimes I simply cannot figure out why on earth I bookmarked a certain page or site.

For me, having better metadata/contextual data would help a lot.
And some system helping me with sorting out my old files and programs. E.g., displaying on demand what files and softwares haven’d been opened and/or changed in the last six months - and then giving me the option to move them into some corner of my storage system or burn them to CD; deinstalling software, but giving me the opportunity for a quick reinstall.

Well, you get the idea, I hope.

But the most important thing is the “mounting psychological excess mass” that keeps me from being nimble on my mind’s feet. If there is no way to cope with this, my 60 gig of harddrive will be more than enough for a couple of more years. Or until I get into doing video myself :-)

Tom Chi wrote:

Here’s a Wired Article ’sort of’ about this topic…

Still, it mainly talks about limitations in finding pictures, and less about the fundamental lack of time to enjoy them.

Leave a Reply

OK/Cancel is a comic strip collaboration co-written and co-illustrated by Kevin Cheng and Tom Chi. Our subject matter focuses on interfaces, good and bad and the people behind the industry of building interfaces - usability specialists, interaction designers, human-computer interaction (HCI) experts, industrial designers, etc. (Who Links Here) ?