February, 2007

...now browsing by month

 

OpenStreetMap constructs maps from GPS tracks!

Wednesday, February 21st, 2007

Sources and uses of digital information are in-scope for this blog, and a great example just showed up in my RSS reader today.

OpenStreetMap is a wiki-like project to build a world map using contributed GPS tracks [OpenGeoData pointed me there]. Their map of Baghdad is here.

This project is truly a product of the early 21st century: it requires GPS satellites, cheap but accurate GPS receivers, the World Wide Web, inexpensive computers with fast color graphics, and so forth.

And like all modern geographic applications, it also exploits a special property of GPS’s information domain: everyone agrees on the meaning of geographical location; only dates and times have a similar level of standardization. In relational-database terminology, this means that any table with a date or location column has a meaningful join with any other.

This doesn’t work with most data. I’ve had driver’s licenses in four U.S. states, but you can’t aggregate my driving record from the state records because they all use different ID numbering schemes (nice for my privacy in this case).

Also noteworthy is the fact that GPS information can be used to put a time dimension into maps, since we can tell when the street is used as well as where it is. There are some very pretty examples at Cabspotting.

Posted in Areas of application, Geographic, Information Philosophy, Information usage patterns | No Responses »

Should metadata be stored in the file it describes? Jon Udell wonders…

Tuesday, February 20th, 2007

In “Who’s got the tag? Database truth versus file truth, part 3″, Jon Udell contrasts the Microsoft Vista and Mac OS X ways of associating metadata tags with image files: Vista tends to store them into the image files, and Mac OS X tends to leave the files untouched and use a separate database to store the tags (or at least Jon was under this impression).

There’s a great discussion about the relative advantages of the two approaches on the blog. Basically, storing the tags in the file makes the association harder to lose as you move the file around, and storing the tags separately avoids modifying the user’s data file. Neither one is obviously in accord with the user’s intention in all cases.

I think the issue has whole extra layers of subtlety. We perceive metadata that is stored within a data file as being what Jon Udell calls “file truth”. Since there’s only one set of metadata stored in the file, it becomes the One True Metadata. On the other hand, metadata stored in a separate database reads as the opinion of the maintainer of the database. This is exactly what social bookmarking systems such as del.icio.usdo: each attribution of a tag to a URL is also associated with a user making that attribution.

A pluralistic society requires a separate metadatabase!

This isn’t just another engineering tradeoff, though. The truth about “file truth” is that it’s still an opinion—the opinion of the last agent to modify the metadata within the file. When there’s One True Metadata, we can only represent disagreements by obliterating the last guy’s assertion.

Imagine trying to tag a scan of a photo taken at your parents’ wedding of someone you don’t recognize. You think it’s Dad’s college roommate, but your sister thinks it’s Mom’s second cousin. You have one “person depicted” slot: do you fight over it? Do you leave it blank and explain the situation in a semantically bland catch-all description field? Or do you each tag it as you will in your respective databases?

Not only is it unrealistic to allow for only one true description of a file, it’s also time we stopped regarding metadata as lost forever just because it’s not stored in the file. We could set up a distributed database that works like Gracenote’s CD identification database, but for all files instead of just music files. As with CDs, the lookup key for a file can be generated by anyone who possesses the file (by applying a secure hash), but the particular metadata obtained depends on which tagger’s part of the repository is consulted. It’s all doable, and it would eliminate blogstorms about how evil application X erases user metadata.

Posted in Information Philosophy, Metadata | 1 Response »

Information Patterns: series introduction

Friday, February 2nd, 2007

Every time a new data format spec hits my inbox, I get a little twinge of dread.

Such documents are often enormous. They’re written in standardese (often badly). They’re usually written by committees. They go through a maze of twisty little revisions, all different.

But worst of all, they often bury their novelty in a sea of details that resemble those in the last spec I reviewed.

I’d like to do for data formats and other information representations what the Gang of Four book does for programs: call out and label the patterns that come up over and over again so that I can classify details into bigger chunks for mental processing.

You can expect to see several different kinds of post in this series:

  • Case studies. I have to look at lots of actual data formats in order to discern the patterns!
  • Data format patterns. Most posts will be about patterns I find in data formats…
  • Information usage patterns. …but some posts will be about how information is generated, stored, and used.
  • Other. I’ll probably think of some other topics as well.

I expect to look at simple cases, such as comma-separated values, as well as fiendishly complex cases, such as PDF. Programming-language syntaxes are fair game; database index disk structures are right out. In between, I’ll draw the boundary as interest dictates.

This series will be open-ended as long as people keep inventing data formats faster than I can look at them.

Posted in Information Patterns, Information Philosophy | No Responses »