SharePoint 2010 Profile Sync - Retrieving custom AD user properties and mapping them to User Profile Properties in SharePoint... and a gratuitous FIM overview

First blog! I've been thinking of doing this for a long time after finding many issues with different admin and dev tasks in SharePoint 2010, but this one really annoyed me so here, I've finally walked the walk...


So let's first recap on 2007...


I came from MOSS (2007), so if you're like me you were used to SharePoint 2007's simple way of mapping a User Profile Property to a custom (cannot stress enough here that we are talking about properties which have been created by your AD team that are not in the default AD schema!) AD one... in your SSP > User profiles and properties > Add profile property > Create your SharePoint property as you want it then at the bottom of the page in the 'Property Mapping' section, select your AD data source, then rather than select a field to map to, you would simply type the name of the attribute you wish to map to because it doesn't exist by default in an AD schema... SHIMPLES!!!


Well guess what?!?! (though I'm guessing if you're reading this you already know...) That text box has disappeared in 2010!!! (Everyone go "oohhhhhh" in a disappointed tone...) But fear not!! This functionality does exist in 2010 ("Hurray!!!").... but you have to use ForeFront Identity Manager in order to do it. ("......oh?"). 


Right... so what is ForeFront and what's it sticking it's nose in for?


First off - if you need a guide of how to set up profile sync from scratch in 2010, check this amazing blog post from the legendary SharePoint MVP, Spencer Harbar - http://www.harbar.net/articles/sp2010ups.aspx - an essential guide for setting up and learning how this profile sync malarkey actually works under the hood in 2010.


In his blog he talks about how when you start the User Profile Synchronization Service it takes a long time because it is provisioning a 'trimmed' copy of ForeFront Identity Manager (FIM) onto the server and setting up the connection from the user profile service to the FIM metaverse (this connection is AKA "Management Agent" in FIM). He then talks about setting up the connection to your directory service for doing an import, when you do this it creates another Management Agent in FIM which gets data from your AD and plonks it in the FIM Metaverse. Jibberish at all? If so, this incredible diagram I drew up should help out (feel free to email me for artwork contracts)...

Neat huh? I'm going on a bit here I know but I'm trying to cover as many bases as I can and it's important to understand how this works if you're to do anything other than what MS suggest.... did I mention that it is not supported to use the FIM tool? Hence it being hidden away in C:\Program Files\Microsoft Office Servers\14.0\Synchronization Service\UIShell\miisclient.exe.


So these two FIM management agents that were created... one is called "MOSS-GUID-of-your-UPS" (Type: Extensible Connectivity) and the other is called "MOSSAD-NameYouGaveOfYourSyncConnectionInTheUPS" (Type: Active Directory Domain Services)... the former representing the outbound connection between the FIM metaverse and your SharePoint UPS, and the latter being the inbound connection between the metaverse and AD. So let us delve into FIM...


Finally, how to set it up...


I will assume that you are at the stage where you have successfully crawled AD and returned all of your desired profiles into your SharePoint User Profile Service (UPS)...


So I have a custom property in AD called 'roomNumber' that I want to map into a field in my UPS called 'roomNumber'. In order to achieve this 


Open miisclient.exe from the above mentioned location. First we need to tell the AD connection to return the custom property. To do that, go to the 'Management Agents' tab, click the Active Directory Domain Services Agent (ADDS Agent from now on) and click on 'Properties' in the tool bar on the right...

...now click on 'Select Attributes' and then tick the 'Show All' tick box at the top right to expose all of the properties from your AD. Select the properties you want to add in...

Click 'OK. Now if you're mapping to a field that already exists in your UPS by default you needn't do this step, but I wasn't, so I needed to create a 'Room Number' field in the UPS... and if I do that, FIM needs to know about it. So I created the string property 'roomNumber' in my UPS (you shouldn't need telling how that's done...), then back in the main FIM window, go to the 'Metaverse Designer' tab, click on 'person', then in the bottom part of the window, click 'Add Attribute' and add the 'roomNumber' attribute in. For simplicity, we'll call it the same as we did in the 'Name' field of the property we created in the UPS...

Ok so you're all with me again now, so go back to the 'Management Agents' tab, click back on your ADDS Agent and click the 'Properties'... we're going to connect the property in AD to the attribute in you created (or not) in your FIM Metaverse. Click on 'Configure Attribute Flow', expand the node where 'Object Type: user' is mapping from the source to the 'Object Type: person' in the Metaverse...

Now it's time to experience the strangest UI logic in a Microsoft product... in order to create a new mapping, you would think you would first click 'New'... WRONG! Of course that's wrong you idiot, why would you think that?..... anyway, thanks for that MS. So the first thing to do is to build the link in the selection boxes above, before clicking 'New'. So find and select your custom 'user' attribute in the data source selection (the one from AD), then find and select your custom (or standard) person attribute in the Metaverse, leave the Mapping Type as 'Direct' and the 'Flow Direction' as 'Import', THEN click on the 'New' button to establish it as a new connection...

Excellent! Well done! We now have the custom AD property in our FIM Metaverse!! ^_^ Oh wait no we don't... Click 'OK'............ now we have!


So next we need to set up the export from the FIM Metaverse to the SharePoint UPS. On the 'Management Agents' tab, click on the 'Extensible Connectivity' agent and click 'Properties'. Click on 'Configure Attributes' on the left, then you need to add in a new attribute, so click 'New...'. IMPORTANT: the 'Name' field must match the 'Name' field you specified in the property in your UPS (if you did create a new one for this) otherwise it will not work! So this is how I created mine...

Note: I set the max character length to be 25 because that's what it's set to in my UPS. Click OK. Next we need to associate that new property with the 'user' object type. The 'user' object type here represents the Extensible Connectivity agent's 'virtual' equivalent of a user object type in the SharePoint UPS, so all we're doing here is telling FIM to present this information to the UPS. Next, click on 'Define Object Types', click on 'user', then click on 'Edit'. You then need to add your attribute in to the 'May have attributes', shown below...

So click the attribute, then click the 'Add' on the bottom to move it into the 'May have attributes'. Obviously, if this is a mandatory field, put it into the 'Must have attributes' but I wouldn't really recommend this because you're essentially trying to police AD by saying this AD attribute can't be empty, which you can do elsewhere, here is not the place, so I recommend putting it in the 'May have attributes' selection. Click OK once done.


FIM doesn't have an 'Apply' button so click ok in the 'Properties' dialog box then re-open the properties for the 'Extensible Connectivity' agent. Now click on the 'Configure Attribute Flow' option, and expand the 'Object Type: user' node. The method to create a new mapping is the same as before, so go ahead and select your custom 'user' attribute you just created, and then select your Metaverse  'person' property we created in the Metaverse designer, or the out the box one if you're using one, set the 'Mapping Type' to 'Direct', and the 'Flow Direction' to export...

And there we have it! FIM now does the magic work, if you followed these instructions you should now be able to do a full sync in your UPS, then find that the properties have magically picked up the sync, a la...

And your properties will map successfully! Whey!


Ok I did it but it doesn't work... help!


Here are a few possible problems that you may encounter, as well as a gotchya. I'll update these as more complaints come if I can...

• When I created the field in the ADDS Agent, it didn't show up in the configure attribute flow bit, why?

Click 'OK' in the properties window then go back in... annoyingly FIM doesn't have an apply button!

• When setting up the 'Extensible Connectivity' agent, I've added my custom attributes in configure attributes, but I can't see them under 'user' in the attribute flow, why?

Have you done the 'Define Object Types' step of this blog? If not, do that! If you have... have you clicked 'OK' then gone back into the properties window again? The attributes will not show up in the attribute flow section until you've done that.

• Something's not right, I've done everything but it's not bringing my properties in! I can see that it's being mapped in the UPS, but no entries are there!

I had this one, do what I did... in FIM, go to the 'Metaverse Search' tab, set the scope to be 'person' leave collation as '<default>' then click search... you should see all of your user profiles in there. If you don't, stop reading this bit and (unfortunately) delete your SharePoint User Profile Service Application and recreate it from scratch - there is something awry with the profiles in there, which is possibly caused by what I'm about to explain. But first, if there is, click on a user and verify whether the property is even being brought in from AD. If it's not, redo these instructions to make sure you've not missed a step. However if the properties are being brought in by FIM, I know no other resolution than to rebuild your UPSA as mentioned above. I have had both of these things happen to me - empty metaverse and FIM showing the properties but my UPS not, and here's how I broke it... I set up my UPS, then set up all my custom properties, then rather than importing the profiles from AD, I used the SharePoint 2010/2007 Administration Toolkit to import the profiles from my 2007 instance (as we were doing a migration), then set it all up through FIM as in these instructions. This will not work! If like me you want to do a migration from 2007 or indeed a 2010 environment, set all of this stuff up first! Then, use the admin toolkit mentioned above (http://technet.microsoft.com/en-us/library/cc508851.aspx) to do an import of only the properties you need to bring over from the other environment... i.e. "About me", "Interests", don't bring in ones that are coming from AD because that's only going to upset things! I'll do another short post on how to use the toolkit to do an import because I don't think it has great coverage on the web.

So there we have it! I hope this has helped any other frustrated people, I'd have much rather read this than have to go through the pain that I did! There is a blog post out there which claims that in order to do this you just need to go into FIM, click the ADDS Agent and click 'Update Schema' then all your custom properties will magically appear in the properties drop down in the UPS... trust me, that doesn't work.