To housekeep or not to housekeep?

It is quite common to see IT services being sold on the basis of unlimited supply of some resource – the current favourite seems to be email storage. At last, an end to the inconvenience of being barred from sending new email unless your stored email remains below some level, which is arbitrary and clearly insufficient for your needs. Of course, the “inbox full” popup always appears at the most inconvenient time, so what could be better than providing more space, keeping everyone happy. In a world of unlimited storage, where the cost per byte of storage is so low, what other solution could make sense?

Well, according to research by ICD / EMC, at some time in 2010 the total amount of digital data stored in the digital world will exceed one zettabyte (that’s 8 followed by 21 zeros bits). I’m not aware of anyone doing any research on the proportions of different data types (email; web pages; video; photos; tweets …) but it all has to go somewhere and “somewhere”, in most cases, will be a server. If we go on adding (and copying) our data, then clearly the storage space it occupies has to increase as well – and unless there’s a very clever method for compressing data about to be announced, the trend can only go one way. More data will – at some point – mean that a given server’s capacity is “full”, at which point the next step has to be to install more capacity, either by adding storage to an existing device or installing a new one. In many cases, this will mean more energy being consumed, (there will be cases of replacing inefficient devices, but again the overall trend is clear).

Can this continue? If not, how do we address it? If we assume the magic new compression algorithm is NOT about to happen, we are left with the need to reduce the stored data. We could do this automatically, perhaps by automatically deleting data which has not been accessed for some time period, or we could ask our users to maintain their own data (or perhaps to employ a virtual clutter organiser).

While it is trite to say that somewhere, someone will generate the email which will mean a new server, it has to make sense that as part of the “something we just do” (see my previous posting) we apply some concepts of tidiness and order into our digital storage.

Leave a Reply