Understanding how Windows Live Photo…

Since writing Tassography.com tags have been an important part of my workflow. For as long as I can remember I have been using Microsoft’s Digital Image Pro 2006. With the release of the new version of Windows Live Photo Gallery (WLPG), I took this Christmas the time to switch over and take advantage of some of the cool new features. The most compelling new feature for me is the new people tagging capabilities. Not only does it store who is in a photo but also where they are. Unfortunately there’s no real standard for where to store people, let alone their location in the picture. Facebook does something similar but doesn’t store it in the photo and most applications just store the name in a keyword or the People XMP field. So the decision was made to store this in a private XMP namespace, which at least provides a standard way of accessing the data. The good news is, Microsoft channels lots of this thinking through a single group, so Windows 7 also uses the same format. In this posting, I’m going to explain how the data is stored and what it is. How is this data stored? Using the WICExplorer it’s possible to see how the data is store in XMP. When people are tagged in the file, WLPG will create a RegionInfo Struct which you can see below: View Image

Which will then contain one or more Regions:

Each Region containing up to four string values: View Image

It’s possible for a combination of these strings to be in each region depending on how much data you’ve given WLPG. There’s a couple of scenarios I have encountered to far:

Scenario	PersonDisplayName	Rectangle	PersonEmailDigest	PersonLiveCID
Descriptive Tag migrated to People Tag but person not identified yet.	yes	no	no	no
Person identified but they are not in your address book	yes	yes	no	no
Person identified, they are in your address book but not in your Windows Live network	yes	yes	yes	no
Person identified, they are in your address book and in your Windows Live network	yes	yes	yes	yes
Face detection found 1+ person but no one identified, so no data will be saved for any person.	no	no	no	no
2+ people found using face detection with 1+ person identified in the photo. This is what you’ll see for the person not identified.	no	yes	no	no

There’s some more permutations that I haven’t documented but this explains most of the ones I have encountered. So what is the data? Rectangle and PersonDisplayName are the two fields that we really care about. Here’s what they look like in WICExplorer:

PersonDisplayName is pretty obvious, it’s the name you specified in WLPG. Rectangle is four decimals stored in a comma delimited string. They represent the scale-invariant portion of the region in the file. That might sound like a mouth full but it’s pretty straight forward to use, it’s basically the percentage position of the rectangle in the file, plus the height and width. So a file of size 1000 x 1000 with a rectangle of 0.1, 0.1, 0.2, 0.2 would represent a rectangle with the top left point at 100, 100 with a width and height of 200. If you take the example data above you’d get a rectangle that nicely covers my Dad’s face. It’s ~45% of the way across the photo and ~24% down: View Image

PersonEmailDigest & PersonLiveCID are also kind of obvious but I don’t believe they are usable by anything but WLPG. PersonEmailDigest is the email address of the person encrypted by WLPG, I may be wrong but I don’t think you can decrypt this yourself. PersonLiveCID is the unique ID generated for everyone with a Windows Live ID, it might be possible to use this and I’ll be playing with some of the Azure Services sometime to see if you can resolve this to a contact. That could create some very interesting possibilities. That’s it for now, in my next post I’m going to explain how to use Windows Imaging Component to read and use the data. Oh and in the interests of full disclosure, I am a Program Manager in Windows Live but I don’t work on WLPG directly. So 90% of the information below I worked out by playing with WLPG and WICExplorer. The WLPG team were very helpful in answering some questions I had that helped round out my research.

Understanding how Windows Live Photo Gallery’s People Tags are Stored