Problems using the ‘sharedUserCertificates’ capability in the Windows Phone 8.1 Developer Preview

Windows Phone 8.1 introduces the long awaited support for certificate management. From the very first version of Windows Phone 7 it was possible to install certificates to the phone, e.g., by downloading them using Internet Explorer or by opening them from an e-mail message. But once installed, the only way to view and/or remove those certificates was to wipe your phone. Not very convenient.

As you may know from my earlier articles I did some sniffing of Windows Phone traffic using Fiddler. To enable the sniffing of SSL-traffic you need to install a Fiddler-generated root certificate, the infamous DO_NOT_TRUST_FiddlerRoot, to the phone. This enables Fiddler to perform man-in-the-middle attacks against the encrypted SSL connection.

Fiddler Root CA

Yesterday I tried to remove the Fiddler certificate from my phone, but was surprised I couldn’t find a certificate management settings app anywhere on the phone. As it turns out the Windows Phone 8.1 Developer Preview doesn’t come with a built-in certificate management app! Yes, the OS does now support certificate management, but only through the use of MDM software (i.e. Mobile Iron or Microsoft’s Intune/SCCM) or custom apps using the new certificate API’s.

Hey JoeB, was it that hard to add a simple Settings app for this? 🙂

Anyway, this gave me a good reason to check out those new certificate management API’s and I decided to write my own little certificate manager app. I inspected the Windows 8.1 ‘Cryptography and Certificate’ sample code and created a simple Windows Phone app using parts of that code (don’t you just love the converging of the Windows and Windows Phone API’s? 😉 ). To enable access to the certificate API’s I added the following capability to the package.appxmanifest:

<Capabilities>
    <Capability Name="sharedUserCertificates" />
</Capabilities>

Then I added the following lines of code (copied straight from said sample) to enumerate all certificates installed on the phone, so I could see if the Fiddler root certificate I had installed on the phone would show up:

private void Page_Loaded(object sender, RoutedEventArgs e)
{
    var task = CertificateStores.FindAllAsync();
    task.AsTask().Wait();
    var certlist = task.GetResults();
    Debug.WriteLine("Cert count: {0}", certlist.Count);
    LoadCertList(certlist);
}

private void LoadCertList(IReadOnlyList<Certificate> certificateList)
{
    CertListBox.Items.Clear();

    foreach (Certificate cert in certificateList)
    {
        CertListBox.Items.Add(cert.Subject);
    }
}

I fired up one of the Windows Phone emulators and installed the FiddlerRoot certificate by e-mailing it to myself and opening it from the e-mail message. Then I ran my test app. To my surprise I got zero results. I installed the FiddlerRoot cert again and reran my app. Again nothing… :-/

Well, maybe this was just emulator weirdness. So I deployed the app to a real phone, my trusty Lumia 920. This time another surprising thing happened. The deployment of the app failed and the following error message appeared in Visual Studio’s Output Window:

Error : DEP0001 : Unexpected Error: Package could not be registered. (Exception from HRESULT: 0x80073CF6)

I tried multiple times. I added some extra capabilities to the Package.appxmanifest, but it always failed with the above error message. A quick search turned up a StackOverflow topic of someone who experienced the exact same behavior. There Claus Jørgensen, a Skype developer, said the ‘sharedUserCertificates‘ capability was only available for first party developers (i.e. Microsoft, OEM’s and carriers).

So I contacted my buddy Matthijs Hoekstra, who happens to be a product marketing manager for Windows Phone and does a lot of the enterprisy stuff. Coincidentally, last week during the ‘Building Apps for Windows Phone 8.1 Jump Start’ he had talked about just this topic in his session ‘Windows Phone 8.1 for the Enterprise Developer’ and I was pretty sure this should be a supported scenario.

Building Apps for Window Phone 8.1 Jump Start Session 18

To cut a long story short: it’s a bug in the Windows Phone 8.1 Developer Preview!

Matthijs was able to confirm the behaviour. In fact, his Jump Start sample app (Session 18 Demos) also currently cannot be deployed to a real phone, as it also uses the 'sharedUserCertificates' capability.

Matthijs told me the ‘sharedUserCertificates‘ capability should be available to all Windows Phone developers, not just first party developers. 1 The fix for this bug will probably be included in the post-RTM updates that I expected to be released before General Availability of Windows Phone 8.1.

This still leaves me with the question why my app, when successfully deployed to the emulator, didn’t find any certificates. The code works fine in a Windows 8.1 WinRT app. Maybe it’s related to the deployment bug. Or maybe the phone is more restrictive about giving access to root certificates that haven’t been installed by the querying app itself? Heck, I may even be using the wrong piece of code 😀 Anyway,I’ll look into it. So as soon as I’ve got the answer, I’ll update this article.

1 Note that apps that require the ‘sharedUserCertificates‘ capability can only be deployed by sideloading or through a MDM solution. If you want to release such an app through the Windows Phone Store, you need special permission from Microsoft.

New and *undocumented* URI Schemes for Windows Phone 8.1

As you may know I’ve been busy lately inspecting the Windows Phone 8.1 internals. Last week I discovered some new Cortana personas. Now I’ve uncovered some new and, as far as I know, currently undocumented URI schemes. These can be used to launch certain apps and System Settings pages directly from within your own app using the LaunchUriAsync() method.

Here are the ones I’ve uncovered so far:

URI scheme Description
ms-battery Launches the Usage tab of the Battery Saver app.
ms-wallet Launches Wallet app.
ms-settings-camera Launches the Photos + Camera Settings app.
ms-settings-networkprofileupdate Launches the Network Profile Update app.
ms-settings-nfctransactions Launches the NFC Settings app.1
ms-settings-notifications Launches the Notifications + Actions Settings app.
ms-settings-proximity Launches the NFC Settings app.1
ms-settings-uicctoolkit Launches a SIM Applications related app. UICC appears to stand for Universal Integrated Circuit Card.2
ms-settings-workplace Launches the Workplace Settings app.

1 The URI scheme names seems to imply they lead to more specific pages within the NFC Settings app. However, during my testing they both opened the main NFC Settings page. This could be due to the fact that the phone I tested this on wasn’t fully in service (i.e. no SIM card inserted, no NFC trusted apps, etc.)

2 I didn’t insert a SIM card into my the Windows Phone 8.1 phone I was testing this with, so all I got was a page containing a ‘SIM applications aren’t available right now’ message.

Uri Scheme Tester App

Disclaimer: As these URI Schemes haven’t (yet) been publicly documented by Microsoft, you probably aren’t supposed to use them. They could have been provided for internal/carrier/OEM usage or maybe they will be documented at a later date. In short: no guarantees! In fact, using these may even cause your app to be rejected from the Store.

Customizing the Windows Phone 8.1 ‘Project my Screen’ app

I find I’m really liking the new Windows Phone 8.1 ‘Project my Screen’ feature. This feature enabled the projection of the handset to a secondary screen via wireless Miracast (newer handsets only, as it requires special hardware) or a wired USB connection. This is great for presentations and demos. It even works in the opposite direction: you can remote control your phone by interacting with the projection using your mouse or your touch screen! Also great is that it has support for displaying multiple touch points, so your audience can actually see where you touch your device. Great stuff!

If you haven’t already, I suggest you give the feature a try yourself. Microsoft’s Cliff Simpkins has done a great writeup on his blog on getting this thing to work.

In order to use the ‘Project my Screen’ feature on your Windows PC, you have to download and install the official ‘Project my Screen’ app. By default the app shows a pretty boring image of a nondescript Windows Phone, much like the Windows Phone SDK emulators do. Fortunately Microsoft included the ability to customize the appearance of the app, so it may suit our own needs. Here’s a short video that shows a quick customization I made using the application:

As you can see I’m using a custom background image featuring the Cortana logo. Also the projected phone’s skin matches my real phone (a yellow Nokia Lumia 920). Let me tell you how I did that:

If you look inside the ProjectMyScreenApp’s installation folder, which by default is located at C:\Program Files (x86)\ProjectMyScreenApp, you’ll notice there’s a file called config.xml. This is a XML configuration file that can be used to alter the appearance of the app. By default it’s not used by the app, though. To use it you need to explicitly launch the application with the configuration file passed in as a parameter. To complicate things a little further: in its original state the configuration file doesn’t work. It refers to images that don’t exist in the installation folder (they do exist, but are embedded as resources inside the application’s executable). So you HAVE to alter the config.xml (or create your own file based on it) to get it to work.

So, how do we configure this thingy? Fortunately the config.xml file itself contains instructions on how to use it. The basic structure of the file looks like this:

<?xml version="1.0" encoding="utf-8"?>

<Config>
  <PhoneConfigurations>

    <PhoneConfiguration>

      <Background />

      <PhoneImage />

      <VideoOutput />

      <!-- <TouchDots /> -->

    </PhoneConfiguration>

    <PhoneConfiguration>
    .
    .
    .
    </PhoneConfiguration>

  </PhoneConfigurations>
</Config>

The supplied config.xml file contains PhoneConfiguration elements for five different resolutions: WXGA (768×1280), HD1080 (1080×1920), HD720 (720×1280), WVGA (480×800) and FWVGA (480×854 with the new on-screen soft buttons). I suppose you can add additional resolutions as devices are released, but I’m currently not able to test this. The application will start up with a default configuration (which you can define; see below). When you connect an actual device and allow screen projection the application will switch to the configuration that matches the connected device’s resolution.

Each PhoneConfiguration element has three attributes. The resolutionX and resolutionY values are used to specify the resolution that this configuration belongs to. The default attribute is used to specify whether this is the default configuration to use when no device is connected.

<PhoneConfiguration
    resolutionX="768"
    resolutionY="1280"
    default="true">

For each phoneConfiguration section we can configure three settings: the background image, the phone’s skin and the size and placement of the projection. Actually, there is a fourth element, touchDots, that’s present in the file, but I couldn’t get it to work. I suspect it should be used for configuring the appearance of the dots that appear when you touch your device, but it didn’t seem to respond to changes I made to it.

Let’s go into detail about each element:

The Background element controls the application’s background. The entire app background will be filled with the specified color. On top of that you can place an image that will also be scaled to fit. Note that using a background image is optional. If you only want to use the solid background color, you still have to add the imageRelPath attribute, but just give it an empty (“”) value.

<Background color="0x00000000" imageRelPath="Cortana.png" />

Using the PhoneImage element you can control the appearance of the on-screen phone (i.e. the skin). The imageRelPath attribute specifies the path to the image to use. For my demo video I used one of the official Nokia Lumia 920 press images that I downloaded from the Nokia Press media library. I cropped the image, so it didn’t have any border, and saved it as a PNG. You need to specify the path to the image relative to the ProjectMyScreenApp’s executable file. The easiest way to do this is by adding your image file to the application’s installation folder, so you only need to supply the filename without any additional path information.

Lumia 920 skin

The scaleFactor attribute specifies the scale factor of the phone image as a fraction of the background image’s width or height, whichever is more constrained. If no background image was specified, the app window’s dimensions are used instead. I used a value of 0.9, so the phone is nice and large, but doesn’t go edge to edge.

Using the centerOffsetX and centerOffsetY values you can position the phone image relative to the background image. I’ve used this to place my phone on the right side of the screen, so it doesn’t obscure the Cortana logo of my background image. Interestingly, if you didn’t supply a background image, these values are ignored and the phone is always displayed in the center of the application’s window.

For devices that have physical buttons we can configure the location of the Back, Windows and Search buttons. For each button you need to supply the coordinates of its bounding box. This way the application can determine which button to press when you click on the skin image with your mouse/touch screen. The backButton, windowsButton and searchButton attributes accept a value that consists of four numbers that represent the left, top, right and bottom coordinates of the hardware back, Windows and search buttons.

Here’s my complete phoneImage section:

<PhoneImage
    imageRelPath="Lumia920-2.png"
    scaleFactor="0.9"
    centerOffsetX="350"
    centerOffsetY="0"
    backButton="128,2357,400,2600"
    windowsButton="610,2357,900,2600"
    searchButton="1160,2357,1370,2600"
          />

Next up we need to specify the location of the video output using the videoOutput element. Using the topLeftOffsetX and topLeftOffsetY you specify the top left coordinates of the video image. Unfortunately you cannot specify the width of the video window. Instead you have to do some math and calculate the weight value, which is defined as being ‘the width of the video output as a fraction of the phone image’s width’. So we need determine the width (in pixels) of the phone’s image display as it appears in the skin image (i.e. the bounding box where you want to projection to appear) and divide that by the total width of the skin image. For my Lumia image that results in the following formula: 1273 / 1535 = 0.8293159609120521.

Here it is:

<VideoOutput
     weight="0.8293159609120521"
     topLeftOffsetX="125"
     topLeftOffsetY="233"
          />

So the complete phoneConfiguration section for my Lumia 920 skin looks like this:

<PhoneConfiguration
    resolutionX="768"
    resolutionY="1280"
    default="true">

  <Background color="0x00000000" imageRelPath="Cortana.png" />

  <PhoneImage
      imageRelPath="Lumia920-2.png"
      scaleFactor="0.9"
      centerOffsetX="350"
      centerOffsetY="0"
      backButton="128,2357,400,2600"
      windowsButton="610,2357,900,2600"
      searchButton="1160,2357,1370,2600"
        />

  <VideoOutput
      weight="0.8293159609120521"
      topLeftOffsetX="125"
      topLeftOffsetY="233"
        />

</PhoneConfiguration>

Now launch the ProjectMyScreenApp using the config.xml file as an input parameter. One way to do this is starting it from a Command Prompt window like this:

C:\Program Files (x86)\ProjectMyScreenApp>ProjectMyScreenApp.exe Config.xml

You can also create a Shortcut with the parameter passed in. The ProjectMyScreenApp should start in full-screen mode. You can press the [ESC] key to switch to windowed mode.

ProjectMyScreenLumia

You can download my sample Lumia 920 files here. Just unzip the files into the ProjectMyPhoneApp’s installation folder (consider making a backup of the original config.xml file, just in case). As a bonus I’ve extracted the default phone images that are contained within the ProjectMyPhoneApp executable and added them too.

Please note that if you connect using a device that has a different resolution than my Lumia 920, you won’t get to see the custom stuff I mentioned above. Instead the ProjectMyScreenApp will use one of the other phone configurations and I didn’t bother changing those.

Maybe someday someone will create a nice tool that assists in creating the config.xml and/or the image files, just like Geert vd Cruijsen has done in the past with his Windows Phone 7 Emulator Skin Switcher application. Or maybe we can build a repository of compatible skin images. I’m even willing to host/link to them from here.

In any case: have fun! You know I am 😉

Hacking Cortana: meet *all* of Cortana’s personas!

Last week after Microsoft had released the Windows Phone 8.1 update there was a lot of buzz around Cortana, Windows Phone’s brandnew personal digital assistant. People found out Cortana had all kinds of funny responses to questions they asked her. But one of them was special. It turned out Cortana had an easter egg that triggered the return of the infamous Clippy. If you asked Cortana “Do you like Clippy?”, an animated Clippy would show up instead of the usual Cortana logo.

PersonaAssets720x1280_CLIPPY1_10

I wondered how this Clippy easter egg was triggered. So I fired up Fiddler and set it up so I could spy on my Windows Phone’s HTTP communication. It turned out that when you ask Cortana that question about Clippy, she will initiate a HTTP request to Bing’s servers. Bing responds with a bunch of HTML that seems to consist of two parts. One part is some plain HTML that Cortana uses to render the actual response in her window. So if you ask Cortana about the weather, Bing will return a piece of HTML that renders into a nice weather table. This is a very flexible solution, because Microsoft can add new Cortana response types by only changing some server-side code. Some day the following little Windows Weekly in-joke I crafted may even become reality 😉

The second part of the response I found more interesting. It consisted of JavaScript and SSML, which are used to control Cortana’s vocal response. When I combed through the JavaScript, I noticed it contained the following DIV-tag:

<div data-emot="clippy1">

The accompanying JavaScript seems to use the ‘data-emot’ attribute’s value to control Cortana’s emotion setting:

t.SetEmotionBySelector("div[data-emot]")

So there it was. I had found the piece of code that triggered the Clippy easter egg animation! I was quickly able to verify this, by using Fiddler’s AutoResponder feature. I set it up so arbitrary Cortana searches would get a response that contained the ‘clippy1‘ emotion. I posted a small video of this feat some days ago on YouTube:

One thing the above also showed, was that the Clippy animation had to reside on the Phone itself. I checked all HTTP traffic and it didn’t seem to contain the Clippy animation. I wondered if there were more easter eggs / emotion values I could use. Some further HTTP-sniffing showed that normal Cortana responses used the ‘CALM’ emotion. So I tried ‘SAD‘, ‘ANGRY‘, etc. Unfortunately those didn’t seem to have any effect. I also tried ‘clippy2‘, ‘clippy3‘, ‘bob1‘, ‘billgates1‘, etc. But they all didn’t seem to work.

Then I tried a different approach. I knew the new Windows Phone 8.1 Developer SDK emulator images included a working Cortana. And because Windows Phone 8.1 is just Windows 8.1 and the emulator images are basically Hyper-V images, it’s possible to mount their VHD’s and get access to their file system (you DID know that, right?). After searching the file system for a while I found Cortana’s image resources were stored in a couple of DLL’s located in the \Windows\System32 directory of the phone’s main partition; their filenames matching the pattern ‘PersonaAssets*.dll‘ (i.e. PersonaAssets720x1280.dll)

Resources

Looking at the resource list, I immediately noticed that ‘clippy1‘ didn’t seem to be the only easter egg. The file also contained a ‘CLIPPYRETIRED1‘ resource. I went back to Fiddler and changed the AutoResponder so that it would return ‘clippyretired1‘ instead of ‘clippy1‘. And when I performed a Cortana search I was greeted by an animation of a seemingly older and retired Clippy, wearing a pair of reading glasses! Brilliant 🙂

Retired Clippy

Excited by this, I extracted all resources from the file and inspected them with a HEX editor. I noticed it were basically animated GIF images that had a little XML header stuck to the front. Using the HEX editor I stripped the XML header from the file, so they could be viewed as normal animated GIFs.

Cortana Persona HEX

And that was when I first met all of Cortana’s current personas! Let me introduce them to the rest of the world:

 

I’m not sure if all of the resources extracted above can be used as an actual Cortana emotion (i.e. replace the default Cortana logo). I tried a couple of them and those all worked. It looks like the two Clippy animations are the only real Easter Eggs currently available. However, one of the resources is named ‘CIRCLE_SIRI1′, which must be a direct reference to Apple’s Siri.

Cortana Siri

Now that I’ve found these personas, I think it’s time to find the accompanying Cortana questions. What could you ask her that might trigger the retired Clippy animation. What question will trigger the Siri animation?

I think the fun has only just begun… 😉

Update: After I published this article, some members of the Cortana team have reached out to me! Marcus Ash, a Group Program Manager for Cortana, sent me the following tweet. Pretty cool!

I’m looking forward to future Cortana updates!

Update 2: Looking for Cortana’s sounds? They’re WAVE resources, located in \\MainOS\Windows\System32\SpeechUXRes.dll on the phone. Use your favorite resource extractor to get them 🙂

Cortana Sounds

Prerequisite components error when creating a Visual Studio 2012 SharePoint 2013 App project

Yeah, I know… it’s a horrible blog title. But I hope Google/Bing/whatever brings you here in case you experience this error.

So you’re ready for doing some Office 2013 / SharePoint 2013 App development and happily installed Visual Studio 2012 and the Microsoft Office Developer Tools for Visual Studio 2012. You then quickly fire up Visual Studio 2012 (remember: using administrative privileges!), select the new “App for SharePoint 2013” template, work your way through the App for SharePoint settings wizard and finally press “Finish”….

BANG! The following error dialog appears:

An error occurred while trying to load some required components. Please ensure that the following prerequisite components are installed

For SEO purposes I will repeat the dialog text here:

“An error occurred while trying to load some required components. Please ensure that the following prerequisite components are installed:

Microsoft Web Developer Tools
Microsoft Exchange Web Services”

Very weird, especially since the Web Platform Installer should  have taken care of these two prerequisites. Since I encountered this error yesterday, I spent some time investigating it. I actually found some references to it here and here. As you can read in those threads the culprit seems to be a pre-installed instance of the Exchange Web Services API. I remembered having installed that API a week ago to do some testing. Apparently the Web Platform Installer doesn’t correctly handle a pre-installed version of the Exchange Web Services API very well.

The solution was simple: just uninstall BOTH the Exchange Web Services API and the Microsoft Office Developer Tools for Visual Studio 2012. Then reinstall (only) the Office Developer tools. This time the Web Platform Installer will ensure the correct Exchange Web Services API is also installed. If all went well you should be able to create a new “App for SharePoint 2013” project without any problems!

The Return of the Search Application Topology Component Health State Error

Today I was prepping a SharePoint 2013 VM and noticed the Search Service Administration page in Central Admin didn’t display the Search Application Topology health indicators. Instead it showed an error I hadn’t seen in a while:

“Unable to retrieve topology component health states. This may be because the admin component is not up and running”

I had encountered this error several times during the SharePoint 2013 preview phase. Then it turned out to be a genuine bug and fixing it required the installation of several hotfixes as detailed in the SharePoint Server 2013 Known Issues article. However, on this particular VM I had installed the RTM versions of Windows Server 2012, SQL Server 2012 and SharePoint 2013. I also had installed the latest updates and the SharePoint 2013 Prerequisites Installer should have taken care of other missing stuff. So this bug shouldn’t have happened again.

Just to be sure I tried installing the four hotfix packages, but (of course) this failed as they were either already installed or not relevant anymore. Now, I was pretty sure I had seen fully working Search Application Topology health indicators the night before (yeah, we SharePointies are nightly creatures). So what could have changed?

Then I remembered I had performed a little tweak to minimize SharePoint 2013’s memory usage. As you may know SharePoint 2013 has some very high memory requirements, especially around Search. Specifically the Search Engine’s NodeRunner.exe processes are memory hungry beasts. A SharePoint 2013 server configured for Search typically has five such NodeRunner.exe instances running, each consuming hundreds of megabytes of memory. For production environments this may be just fine, but on my development and demo VM’s I like them a little less greedy.

From earlier blog posts I learned it is possible to limit the amount of memory a NodeRunner.exe instance uses, by adjusting the memoryLimitMegabytes parameter in the C:\Program Files\Microsoft Office Servers\15.0\Search\Runtime\1.0\NodeRunner.exe.config XML file. In fact, in the SharePoint 2013 Preview version the NodeRunners suffered from a memory leak and adjusting this config was a quick hack to fix this leak. One of the last things I did last night was change this value from 0 (unlimited) to 100 MB. However, I hadn’t actually come around to testing the outcome of this change (yes, I do need some sleep).

...
<nodeRunnerSettings memoryLimitMegabytes="100" />
...

So the first thing I did was starting SysInternals Process Explorer (which I always use instead of the built-in Task Manager) and searched for the NodeRunner.exe processes.

To my surprise there weren’t any…

The NodeRunner.exe instances are spawned by SharePoint’s Search Host Controller Service, so the first thing I did was go to the “Manage Services on Server” page in Central Admin and restart that service. Thanks to Process Explorer’s handy ‘Difference Highlighting’ feature it was clearly visible that several NodeRunner instances got started, but almost immediately stopped running.

To get a little more information I started SysInternals Process Monitor (another indispensable tool) and logged the system while restarting the Search Host Controller Service. From Process Monitor’s ‘Process Tree’ dialog it was clear (too) many NodeRunner instances were started, but somehow weren’t able to survive for long.

Process Explorer also showed me the NodeRunner processes were started using the following command line parameter:

"C:\Program Files\Microsoft Office Servers\15.0\Search\Runtime\1.0\NodeRunner.exe" --noderoot "C:\Program Files\Microsoft Office Servers\15.0\Data\Office Server\Applications\Search\Nodes\9E4952\IndexComponent1" --addfrom "C:\Program Files\Microsoft Office Servers\15.0\Data\Office Server\Applications\Search\Nodes\9E4952\IndexComponent1\Configuration\Local\Node.ini" --tracelog "C:\Program Files\Microsoft Office Servers\15.0\Data\Office Server\Applications\Search\Nodes\9E4952\IndexComponent1\Logs\NodeRunner.log"

From the bold piece above I learned NodeRunners keep a trace log. When I checked that file, the cause for the failing of the NodeRunners became pretty clear:

Caught exception in node activator: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at Microsoft.Ceres.CoreServices.Node.NodeActivator.ActivateNode(IDictionary`2 configuration)
--- End of inner exception stack trace ---

Since I figured this must be due to me changing NodeRunner’s memory limits, I reset the NodeRunner’s config file (hey, always keep a backup!) and again restarted the Search Host Controller Service. This time the NodeRunner instances continued to run just fine. So it became clear to me there apparently is some minimum amount of memory a NodeRunner instance needs to be able to run and in my case it needed more than 100 MB.

With the NodeRunner instances running just fine, the Topology Health indicators also worked again, showing my farm’s Search stuff was in perfect health. I experimented with the NodeRunner memory limits some more and finally settled for a 250 MB limit. Not as small as I’d liked them to be, but still a significant decrease when compared to their original memory footprint.

I do want to make clear that even with this 250 MB limit I experienced some NodeRunner crashes. The general advice is to NOT change the NodeRunner memory limit configuration. And NEVER EVER do this in a production environment!

Hopefully this blog post helps out other people who might stumble upon this problem.

The new HTC 7 Mini or: How to fake WP Bench

I am so evil… Whoahahahahaha…

Earlier today I sent out the following tweet:

LMAO reading today's #wp7 news. I'll explain later... #wp7nl

LMAO reading today's #wp7 news. I'll explain later... #wp7nl

So I fired up this old blog of mine to tell you why I just had to laugh:

Last night I was playing with the latest version of the popular Windows Phone 7 benchmarking app WP Bench, made by Robert Varga. Some time ago Robert added the ability to upload your benchmark results to his server and compare them with others.

However, WP Bench does not only publish the benchmarking results. It also publishes information about the phone models and the Windows Phone OS versions that are used. And these two lists have become an important source for people to scout for new Windows Phone models and OS versions. In fact, every new device/OS version that is listed in those stats is sure to get some headlines at popular news sites like WMPowerUser.com.

That’s when my evil plan starting taking shape. What if I could alter some of the data that WP Bench uploaded to its server and pretend I was using some new, unknown (and fake) Windows Phone model? Surely WP Bench had built in some mechanism that would prevent this, wouldn’t they? But I just had to try. Maybe it was possible to generate some buzz amongst the Windows Phone community and even get featured on some news sites…

First thing to do was to actually inspect what data and in what format is being sent back to the server. So I quickly hooked my Windows Phone to Fiddler, the popular HTTP debugging proxy. I started WP Bench on my phone and ran a simple speed test. I then uploaded the results to the WP Bench server.

The Fiddler Web Sessions list showed that benchmark results are uploaded to the server by using a single HTTP GET request that has all relevant result data specified as query string parameters, like this (I’ve removed my phone’s unique device ID from the URL):

http://www.computemark.com/wpb/index.php?deviceID=...&deviceManufacturer=HTC&deviceName=7+Mini+T86861&
os=Microsoft+Windows+CE+7.11.1131&publishSpeed=true&cpu1=3%2c67&
cpu2=3%2c67&data1=14%2c98&data2=7%2c26&gpuFPS=18&speedScore=58%2c3

What was even more interesting was that there didn’t seem to be any verification mechanism in place, to make sure this data wasn’t tampered with. No verification hashes or any of that kind. So this meant I could easily tamper with such a request and pass my own, fake, data.

I activated Fiddler’s Automatic Breakpoints feature, so it would intercept each request and allow me to edit it, before it was passed on to the WP Bench server. I again ran the WP Bench speed test and uploaded the results. This triggered the Fiddler breakpoint and I made the following changes to the request:

  • I slightly changed the deviceID parameter, so the results wouldn’t be related to results I had previously uploaded using the same device.
  • I changed the deviceName parameter from ‘7 Trophy T8686’ to ‘7 Mini T86861’.
    The fun part of this was that I came up with the name ‘Mini’, because just days ago HTC had announced a new and very large Windows Phone device, called the ‘Titan’. I really didn’t know that HTC had actually once shipped a device called the ‘HD Mini’, which ran Windows Mobile 6.5. So I guess by accident this ‘Mini’ name added more credibility to my fake model name.
  • I changed the OS version from ‘7.10.7712’ to ‘7.11.1131’, hoping that a previously unseen change from 7.10 to 7.11 would generate some buzz. Also, the 1131 part refers to the birthday of someone very close to me 🙂

I didn’t change any of the performance scores, but of course could have easily done so. After that I allowed Fiddler to send the altered request to WP Bench’s server and went to bed.

Fiddler screenshot of the altered WP Bench request

Fiddler screenshot of the altered WP Bench request

So today I visited WMPowerUser.com and sure enough there it was: a nice new article titled ‘HTC 7 Mini running OS 7.11.1131 shows up in WP Bench’. It talked about the fact that WP Bench stats showed this mysterious new model, the HTC 7 Mini, complete with a totally new OS version in the 7.11.xxxx range. There was talk about the fact that this could be the first evidence of Windows Phone Apollo running on a handset. It was also mentioned by many other news sites and of course there was some buzz on Twitter.

WMPowerUser article

WMPowerUser article

So that’s how easy it was to create some fake Windows Phone news. Just to be clear: as far as I know the HTC 7 Mini does NOT exist. Also there is no Windows Phone OS version 7.11.1131 (yet…). It was all made up by me. Whoahahahahaha…

Lastly, as a suggestion to WP Bench’s creator Robert Varga, I would advice him to add some sort of protection scheme, so that uploaded WP Bench results cannot easily be tampered with. I also wouldn’t list a new device/model in the WP Bench stats, unless it has been seen more than once, preferably with result uploads coming from a variety of IP addresses. Remember it took only one little request to get my fake phone to show up in the WP Bench stats.

Update: It appears Surur from WMPowerUser.com wasn’t happy about this and has banned me from posting comments to WMPowerUser. This is a JOKE, guys! Why so serious… !?

Update 2: WMPowerUser.com today has posted an article about another new phone. This one is probably legit, but notice the screenshot that accompanies the article. There is a device called ‘NA NA’ in the list. I know for sure that device was already present before I conducted my little experiment. So I have a strong feeling I wasn’t the first to submit fake data to WP Bench…

A device called 'NA NA'!?

A device called 'NA NA'!?

The iPad Backup that took a long time…

This saturday I decided to create a full backup of my iPad. To cut a long story short: it took a long time. A very long time. A very very very very long time. In total the backup my iPad 16GB, which was filled with about 7GB of data, took a shocking:

18 hours, 22 minutes and 19 seconds!

Unbelievable. If you Google with Bing you can find a lot of users experiencing similar long backup times. As it turns out Apple has devised a backup mechanism, whereby backup duration is heavily dependent on the number of files it has to backup. Each file apparently takes several seconds to process, so backing up several thousand files can easily take hours.

I just had to investigate which apps caused these huge backup times. Analysis of the directory where iTunes had placed my backup showed that it basically contains pairs of two types of files: *.mdinfo and *.mddata. As these extensions imply, the *.mddata file contains the actual backup data and the related *.mdinfo file contains some meta data about that data file.

The *.mdinfo files are binary, but when opened with a text editor like NotePad they contain some readable elements. One of the more interesting things in this file is a text string beginning with the word "AppDomain". This looks like a unique identifier for an iPad app. I noticed that many files have the same AppDomain identifier in them, which leads to the conclusion that they all belong to the same app.

Based on this analysis I came up with the following scheme:

  1. Collect the AppDomain identifier for every *.mdinfo file in the iPad backup directory.
  2. Group all these files based on their AppDomain identifier.
  3. Sort these groupings, based on the total file count.

I wrote a quick C# console application for this and below are the results. First a little pie chart:

iPad App File Count

As you can see from the pie chart there are two apps that have claimed a really large number of files:

The winner (or better: loser) is the Wired app, the iPad version of Wired Magazine. It somehow needs 4947 files to operate. For the record: I installed the Wired app and did an in-app purchase of the issue featuring Toy Story 3. I didn’t buy any other issue.

The runner-up is the NU.nl app, which is an app that shows news content from the popular Dutch news website www.nu.nl. It’s a fantastic app, but I really haven’t got the faintest clue why it would need 3596 files to operate, since it should fetch its content live from the website.

The third place actually surprised me. For these files no AppDomain identifier could be identified. They did contain the string "MediaDomain", but I didn’t investigate in its meaning. However, since this group of files finished third, I’ve definitively needed to include them in my list.

Below you’ll find the full breakdown. Deleting this list’s top five apps from my iPad dramatically decreased backup times.

So, in the end I guess both app developers and Apple can learn something from this. App developers need to make sure their apps only use a limited number of files. And Apple needs to seriously fix their stupid backup mechanism.

AppDomain File Count
AppDomain-com.condenet.wiredmag 4947
AppDomain-nl.sanomadigital.nuhd 3596
*Unknown* 824
AppDomain-nl.veronica.veronicamagazine 659
AppDomain-se.illusionlabs.labyrinth2 504
AppDomain-com.nytimes.editorschoice 367
AppDomain-com.linkedin.LinkedIn 338
AppDomain-com.outfit7.talkingtomipad 314
AppDomain-com.vsallaccess.vsaaipad 240
AppDomain-com.tunaverse.iPadCinq 199
AppDomain-com.chillingo.cuttheropehd 154
AppDomain-com.isignz.aanbieding 121
AppDomain-uk.co.guardian.Eyewitness 113
AppDomain-nl.tvgids.app.ipad 86
AppDomain-org.pogi.dp 77
AppDomain-com.NewspaperDirect.PRiphone 73
AppDomain-com.atebits.Tweetie2 70
AppDomain-se.illusionlabs.touchgrind 64
AppDomain-com.concretesoftware.acestrafficpackipad 43
AppDomain-com.vitotechnology.StarWalkiPad 36
AppDomain-com.apple.Keynote 30
AppDomain-com.melodis.soundhound.free 24
AppDomain-com.dawikihoodcom 21
AppDomain-com.apple.Numbers 20
AppDomain-BibleReader 19
AppDomain-com.ikmultimediaus.amplitubepadfull 19
AppDomain-com.flipboard.flipboard 18
AppDomain-com.skype.skype 18
AppDomain-com.firemint.flightcontrolipad 17
AppDomain-com.davidme.soundpaper 16
AppDomain-com.goodiware.GoodReaderIPad 16
AppDomain-com.spotify.client 15
AppDomain-com.trautvetter.atomicbrowser 15
AppDomain-com.mobilepioneers.tvgidslite 13
AppDomain-com.smule.magicpiano 13
AppDomain-wefwef 13
AppDomain-com.chillingo.angrybirdsipad 12
AppDomain-com.evernote.iPhone.Evernote 12
AppDomain-com.amazon.Lassen 11
AppDomain-com.apple.iBooks 9
AppDomain-com.apple.Pages 9
AppDomain-com.booking.BookingApp 9
AppDomain-com.mobilepond.ipadphotogene 9
AppDomain-pocketinformantiPad 9
AppDomain-osfoora.hd 8
AppDomain-ch.graphicarteservice.newspaper.nrc 7
AppDomain-com.com2us.sliceit 7
AppDomain-com.ea.recklessracing.ipad.bv 7
AppDomain-com.fotopedia.heritage 7
AppDomain-com.layar.Layar 7
AppDomain-com.tmsoft.FartCushion 7
AppDomain-com.truphone.ipad 7
AppDomain-com.ebay.core.ipad 6
AppDomain-com.marcoarment.instapaperpro 6
AppDomain-com.teamviewer.teamviewerhd 6
AppDomain-com.ted.TED 6
AppDomain-com.aemotion.magworld 5
AppDomain-com.mythpeople.SparkleHD 5
AppDomain-com.natenai.glowhockey4ipad 5
AppDomain-com.sqrt3innovations.calculatorpad 5
AppDomain-nl.hyves.hyves 5
AppDomain-nl.telegraaf.krant.ipad 5
AppDomain-nl.vpro.3voor12.luisterpaal 5
AppDomain-at.co.hlw.itaprdpclient 4
AppDomain-ca.jubilo.ipadflickr 4
AppDomain-com.bryandev.GearedHD 4
AppDomain-com.Buienradar.BuienradarHD 4
AppDomain-com.gameprom.software.ThePinball 4
AppDomain-com.getdropbox.Dropbox 4
AppDomain-com.idealbinary.rumpel 4
AppDomain-com.imdb.imdb 4
AppDomain-com.jellyfilledstudios.AmzngDiceHD 4
AppDomain-com.laya.osiris.wikipanion.ipad 4
AppDomain-com.tarsin.Dilbert 4
AppDomain-com.thinkfun.rushhour 4
AppDomain-com.wolframalpha.wolframalpha 4
AppDomain-org.pogi.tigmatching 4
AppDomain-com.appshopper.app 3
AppDomain-com.europeandirectories.dtgipad 3
AppDomain-com.handypadsoft.deviceinfo 3
AppDomain-com.johnmoff.PocketPondHD 3
AppDomain-com.lucasarts.miseipad 3
AppDomain-com.vitotechnology.SolarWalk 3
AppDomain-com.wireddevelopments.rocketeeripad 3
AppDomain-de.meteogroup.WeatherProHD 3
AppDomain-fr.febo.multipong 3
AppDomain-com.appbc.touchable 2
AppDomain-nl.mangrove.Nozzman 2
AppDomain-com.tiogadesigns.bigcalculatorfree 1

Googling with Bing

I laughed out loud when I read this quote from Scott Hanselman’s latest blog post:

I googled with Bing

How will Microsoft ever win this…

A Little Fun

I remember reading this funny text several years ago. I just stumbled on it again and decided to archive it here. I think it’s pretty funny 🙂

Woman are like computers, because:

  1. No one but their creator understands their internal logic.
  2. The native language they use to communicate with other computers is incomprehensible to everyone else.
  3. Even the smallest mistakes are stored in long term memory for possible later review.
  4. As soon as you make a commitment to one, you find yourself spending half your paycheck on accessories for it.

Men are like computers, because:

  1. In order to do anything with them, you have to turn them on.
  2. They have a lot of data but still can’t think for themselves.
  3. They are supposed to help you solve problems, but half the time they ARE the problem.
  4. As soon as you commit to one, you realize that if you had waited a little longer you could have gotten a better model.