Thoughts about migrating content into Yammer from forums

YammerMig

This article hopes to cover the considerations and possible approach for planning and migrating existing forum content into Yammer. Yammer doesn’t provide any form of ‘migration’ or ‘import’ for other system content. They take the stance that the working paradigms created by Yammer are unique to the platform and that managing the transition from old system to Yammer is where the investments should be made. The Yammer article here describes their high level thoughts on the subject, key for me was the statement:

2. Start fresh with content

Our customers’ experience has shown that technical migrations of content are not generally recommended, as information on Yammer tends to be more ephemeral in nature – not necessarily needed as material for future reference. There is better potential for users to be invested in and engaged with the new network if they are empowered to shape its structure and content themselves.

The realities of Client engagements tends to be slightly less black and white, where most want to consider some form of carry across information into Yammer where they have existing knowledge management forums established. Many of our clients have established forums woven into the day to day behaviours, the information within them can’t be wilted on the vine so to speak.

One key takeaway before diving into the more detailed thoughts….

Consider why you are migrating, unless its aligned to a business benefit realisation or business goal then the effort of the migration is mis-aligned to its value to the organisation. Also consider what your new operating model is, don’t migrate an old operating behaviour along with the content, you MUST migrate the content into the new behaviours and approaches you aim to have using Yammer integrated within your organisation.

Understanding the source forum information

Whatever the technology platform involved whether it is SharePoint forums or another forum technology the key element to a successful migration is auditing and understanding the content within the current forums. Key elements to understand are as follows:

  • Users
  • Forums metadata such as Title, description and logo image
  • Security model
  • Thread content
    • Styles of thread, such as single origin, multi-thread
    • Thread parenting, which forum the earliest multi-origin thread comes from
    • Mentioning users
    • Mentioning metadata
    • Attachments
    • Links
  • Value add activities like adding additional tagging or additional metadata content

From there you need to plan the content destinations in Yammer. Not always will there be a one forum to group mapping. In fact, now may be a sensible time to ‘refactor’ the approach to segregation of the conversations to something that is more relevant or easier to understand.

Users

The source system is unlikely to hold user accounts in the same fashion as Yammer. Therefore a mapping exercise is needed. Yammer users are keyed off the email address (the primary key as such).

A full listing of the users who have interacted with the forum needs to be generated. Alongside this a full extract of the Yammer users information should also be requested. In order to have all users within the Yammer Network you must establish the first date your network existed and pull the data from that date. The Yammer data export will only pull data where a user was interactive for that time period. This means you need to go a long way back for the first extract to get all the required user records. Subsequent users can be pulled in more recent extracts if you need to refresh and append data.

From this point you have the source and destination user lists. The next step is to produce a mapping between them. In general a good approach is to manipulate the records in the source user list to append the email address if possible. If you can get the source users to have a unique email address then you can map them directly against the Yammer user list.

From the mapping exercise you’ll be left with users in both lists who are not matched. Ignore the non-matches in Yammer side as they’re not really going to be a problem for migration as they have never interacted with the source forum. The non matched users in the source user list is where the focus should be placed. You have the following options:

  • Map all non-matched forum users to a generic account in Yammer. Something like Company Alumni, this has the advantage that it’s simple to map, and you can tie everything to one central user. During your Yammer rollout you can then inform the user base about the Alumni user and promote this as historic facts. The disadvantage of this approach is if there is value in knowing who made every thread post. Consider a situation where the discipline for the forum is more detailed and technical, users may still require to know who made which comment in order to trace that individual even outside of the organisation.
  • Map all non-matched forum users to newly created Yammer users. The advantage of this is that the user to thread relationship is maintained. Disadvantages is that you will have to consider how users are managed. If DIRSync is being performed this may be more complex. Also these users need to be active during migration and then suspended once the threads are added.

For the purposes of forum migration it is unlikely you need to consider the values within the user profiles and migrating those, but again this is worth discussing for your scenario.

Groups

Consider how you will be mapping in the existing forums. Do the existing forums contain threads which span wide topics? Could it be beneficial to split the existing forum into smaller segments, in which case the thread mapping to Yammer Groups needs to establish this. Also consider that the groups may not yet exist, so you might have to work out the creation and admin assignment as well. Be careful not to orphan any threads which are topics that no longer fit into the Yammer Group structure.

Also the big difference between Yammer Groups and many forum based technologies is that the Yammer Groups lack any form of hierarchy, so where a source forum may be grouping several sub sections the Yammer Groups don’t behave in this fashion. My thoughts here is that you should structure the Yammer Network to support the new working patterns you are creating. It is likely that the Yammer rollout is in support of other initiatives. So consider if the new Groups are topic/subject centric now, and how you might have to map threads into these topics. Many of the implementations we have been rolling out use a SharePoint site to aggregate the groups into a form of hierarchy or associations. This is a loose coupling though and a sensible approach, the association to hierarchy is only appropriate if the user scenario warrants it, such as a knowledge network.

Group metadata such as name, logo and description are also key for discovery by users within the Yammer Network. Consider also telling users which the primary language is for the group. Yammer has a pretty good translation mechanism using Bing Translate, but some groups naturally centre around a specific language.

Consider User membership for new Yammer Groups, do you want to add the users from the existing forums to the new Yammer Group? Sometimes this is a straight mapping one to one, sometimes if you re-jig the structure it becomes more complex. A simple algorithm is to capture all the users involved in the group threads and make sure they get added to the Group members. Again this in principal goes against what I’d normally be proposing, Users really shouldn’t be forcibly added to group it’s bad practice for good engagement and really annoys people.

Threads and Content

Threads and content form the cornerstone of the forum experience, in Yammer these map roughly to ‘Conversations’ and individual ‘Yams’. The Yammer paradigm is quite straight forward structure wise.

  • Network – The overall container for all the content
  • Groups – The segmentation of the content within the network into groupings of people who generate content
  • Conversation (aka a thread) – The conversations which are started and replied too
  • Yam (aka a message) – An individual post within the conversation thread.

Basics

Each thread message needs to be analysed for the following:

  • It’s parent message, so that it can be correctly associated into a new Yammer conversation
  • Any forum members mentioned in the content, so that the correct @mentions can be inserted into the new Yam
  • Any tagging used, so that the correct @mentions can be inserted into the new Yam (I know it sounds weird to say @ mentions again, but the user and tags are effectively the same with some meta data differences)
  • Any attachments being referenced by the message, so that the correct handling approach can be used (more about attachments in a minute)
  • Any Hrefs within the content, so that you can decide how to deal with the link (more about that in a few more lines)

Links

Handling Hrefs can take a couple of formats:

  • Keep them as is and just ensure they map into the content specification of Yammer
  • Modify the url against a mapping rule set, creating a permanent redirection to other migrated content
  • Modify the url against a mapping service, creating a permanent redirection but to something that can manage the redirection as content moves around in supporting platforms

If you go for an external link redirection service consider how long this will be in place for. Consider the overall benefits of redirection and watch the analytics tracing to ensure that users are still getting benefits from the service.

I did consider using SharePoint query rules as a way to manage redirection. Imagine that you create key threads in Yammer, you can create the links within them to point to a SharePoint site, each site can have its own query rules for content promotion. So where you’re knowledge management community also has a SharePoint site collection you can implement a local collection of search schema search query rules. If you take this approach you can effectively ‘map’ a Href into a search based url with specific terms. Then it fires across to SharePoint and the relevant promoted results would be presented.

Attachments

Dealing with attachments can take several forms:

  • Uploading the content into Yammer directly and effectively storing that content in Yammer
  • Making reference to external sources such as a web page where the content lives

Depending on your requirements may mean choosing the option most relevant almost on a message by message basis. My recommendation would be to store information outside of Yammer where it better aligns with collaboration systems such as Office365 sites, and upload directly where it makes more sense as social content. The obvious examples are a rule set which treats all Office document formats as upload to SharePoint and things such as images go into Yammer. You may also decide to treat some messages as ‘OpenGraph’ and have Yammer treat them as more like a notification than a message.

Thread sharing

Yammer allows a thread to be shared once created to another group. When sharing the thread sharer can choose to add additional text to the conversation share. Once shared the thread is effectively a new thread with its own content. So with this in mind you need to analyse the forum sharing paradigm in the current forums and decide how to map them into Yammer.

In situations where a thread appears across multiple forums. For example:

  • Original post goes into forum A, then is shared to forum B and C
  • When a user view either forum A,B or C the entire thread appears, thus capturing all the conversation once, but displaying it in multiple places

You need to consider how this maps into the Yammer sharing paradigm. I think the most sensible approach is to decide which Yammer Group the thread is most suited for, this is probably a manual decision in most cases. Once targeted for a specific group post the whole thread to that group, then share the complete thread into other groups. It might also be work retrospectively adding ‘tags’ to the thread.

Topics

Yammer has a tagging concept called ‘Topics’. Each thread can be tagged with topics to help discoverability. A user can elect to follow a specific Topic. So when migrating content analyse the content for potential topics and ensure they are tagged in the Yam with Topics. This is a value add activity and would require quite a lot of intelligence to identify topics in existing threads.

Dates

As you add new Yams to the Yammer threads you can’t influence the date stamp. So if you require date stamps to be known the only real solution is to append the information into the Yam text.

Security

Always a thorny issue, the conflicting paradigms of information security and engagement and openness working like a network.

There is definitely no right answer about how to handle security, so these thoughts are where I’d propose you started.

Yammer’s paradigm really needs Yammer Groups to remain publically visible and readable to reap the benefits from working like a network. So you need to plan how to ensure information which might have been secure remains so. Often the existing forums are restricted security wise by ‘membership’ as a way to keep a tight control of who can edit, often driven by reporting and funding concerns about membership numbers rather than true data sensitivity. For this scenario in most cases adopting the public group paradigm is fine.

Where you need a private group, say for example the Board, then using a Private group is perfectly acceptable.

You will need to audit the information in every thread for its compliance if you want to be 100% sure that you’ve not shown restricted information from the existing forums in the new Yammer Groups.

User engagement and roll out

Although not strictly migration in the technical sense don’t under estimate how you need to deal with users moving them from the old world into the new. Thinking about how to help users find threads that have moved.

One possible solution to this is to add a new reply in the existing forums which contains information about the thread location within Yammer. This helps anyone who has the old thread bookmarked come away with a smile as they can navigate to the Yammer conversation and continue the discussions. It also allows for a time of dual running where some but not all threads have made it into Yammer.

Another aspect to consider is publishing the whole process for the end user to read. Sounds little bonkers at first, why should users care or be told about the internals of an IT project. Well if you think about what Yammer has as a central pillar… it’s openness. So help users get into the correct mind set.

Yammer Technical

App Registration

The processing app must be registered within Yammer. Instructions for the App Registration process can be found in the Yammer Developer Center: https://developer.yammer.com/introduction/#gs-registerapp

Once the migration processing app is registered the Client Id and Client secret can be used by the app to make calls into the Yammer Network.

Authentication

The processing app must authenticate to Yammer in order to make calls the API. Information about the authentication process and options can be found in the Yammer Developer Center: https://developer.yammer.com/authentication/

OAuth Impersonation

To interact with the Yammer API on behalf of another user you will need to use a Verified Admin account for the processing app and have it call the relevant REST API endpoint to collect the user token for the user in question. Information about impersonation can be found in the Yammer Developer Center: https://developer.yammer.com/authentication/#a-impersonation

Users

The Yammer API allows you to retrieve, create, update and suspend (soft delete) users. Information about the exact API call format can be found here: https://developer.yammer.com/restapi/#rest-users

Rate limits

The migration processing app must have logic to throttle the calls made to Yammer as to not exceed these thresholds. It should also monitor the responses and act accordingly to retry and ensure data integrity in the event of API calls being blocked. Information about the Yammer REST API rate limits can be found on the Yammer Developer Center: https://developer.yammer.com/restapi/#rest-ratelimits

The content at the time of writing is as follows:

API calls are subject to rate limiting. Exceeding any rate limits will result in all endpoints returning a status code of 429 (Too Many Requests). Rate limits are per user per app. There are four rate limits:

Autocomplete: 10 requests in 10 seconds.

Messages: 10 requests in 30 seconds.

Notifications: 10 requests in 30 seconds.

All Other Resources: 10 requests in 10 seconds.

These limits are independent e.g. in the same 30 seconds period, you could make 10 message calls and 10 notification calls

Pre-reqs

Also useful during the process is the availability of an Enterprise Yammer network, mainly so you can get the admin functions like data export and impersonation working. The question over having a discreet ‘dev’ network came up, my personal thoughts are as follows. If you’re not ‘live’ with Yammer while doing the migration work then why not simplify the moving parts by keeping just the main Network. At the end of the day you can always try things in a private group first to write your processing code, then open it up once the code is tested. Worst case you add messages you need to delete later. This keeps costs down. If however you’re already using Yammer, then best to go for multiple Networks. Again it’s your choice, but Yammer is a service that you can’t actually customise, so your options are safer in my opinion.

NB: This information is my pre-execution thinking and hopefully after the work is complete I’ll come back and make any amendments to the information.