Thoughts about migrating content into Yammer from forums

YammerMig

This article hopes to cover the considerations and possible approach for planning and migrating existing forum content into Yammer. Yammer doesn’t provide any form of ‘migration’ or ‘import’ for other system content. They take the stance that the working paradigms created by Yammer are unique to the platform and that managing the transition from old system to Yammer is where the investments should be made. The Yammer article here describes their high level thoughts on the subject, key for me was the statement:

2. Start fresh with content

Our customers’ experience has shown that technical migrations of content are not generally recommended, as information on Yammer tends to be more ephemeral in nature – not necessarily needed as material for future reference. There is better potential for users to be invested in and engaged with the new network if they are empowered to shape its structure and content themselves.

The realities of Client engagements tends to be slightly less black and white, where most want to consider some form of carry across information into Yammer where they have existing knowledge management forums established. Many of our clients have established forums woven into the day to day behaviours, the information within them can’t be wilted on the vine so to speak.

One key takeaway before diving into the more detailed thoughts….

Consider why you are migrating, unless its aligned to a business benefit realisation or business goal then the effort of the migration is mis-aligned to its value to the organisation. Also consider what your new operating model is, don’t migrate an old operating behaviour along with the content, you MUST migrate the content into the new behaviours and approaches you aim to have using Yammer integrated within your organisation.

Understanding the source forum information

Whatever the technology platform involved whether it is SharePoint forums or another forum technology the key element to a successful migration is auditing and understanding the content within the current forums. Key elements to understand are as follows:

  • Users
  • Forums metadata such as Title, description and logo image
  • Security model
  • Thread content
    • Styles of thread, such as single origin, multi-thread
    • Thread parenting, which forum the earliest multi-origin thread comes from
    • Mentioning users
    • Mentioning metadata
    • Attachments
    • Links
  • Value add activities like adding additional tagging or additional metadata content

From there you need to plan the content destinations in Yammer. Not always will there be a one forum to group mapping. In fact, now may be a sensible time to ‘refactor’ the approach to segregation of the conversations to something that is more relevant or easier to understand.

Users

The source system is unlikely to hold user accounts in the same fashion as Yammer. Therefore a mapping exercise is needed. Yammer users are keyed off the email address (the primary key as such).

A full listing of the users who have interacted with the forum needs to be generated. Alongside this a full extract of the Yammer users information should also be requested. In order to have all users within the Yammer Network you must establish the first date your network existed and pull the data from that date. The Yammer data export will only pull data where a user was interactive for that time period. This means you need to go a long way back for the first extract to get all the required user records. Subsequent users can be pulled in more recent extracts if you need to refresh and append data.

From this point you have the source and destination user lists. The next step is to produce a mapping between them. In general a good approach is to manipulate the records in the source user list to append the email address if possible. If you can get the source users to have a unique email address then you can map them directly against the Yammer user list.

From the mapping exercise you’ll be left with users in both lists who are not matched. Ignore the non-matches in Yammer side as they’re not really going to be a problem for migration as they have never interacted with the source forum. The non matched users in the source user list is where the focus should be placed. You have the following options:

  • Map all non-matched forum users to a generic account in Yammer. Something like Company Alumni, this has the advantage that it’s simple to map, and you can tie everything to one central user. During your Yammer rollout you can then inform the user base about the Alumni user and promote this as historic facts. The disadvantage of this approach is if there is value in knowing who made every thread post. Consider a situation where the discipline for the forum is more detailed and technical, users may still require to know who made which comment in order to trace that individual even outside of the organisation.
  • Map all non-matched forum users to newly created Yammer users. The advantage of this is that the user to thread relationship is maintained. Disadvantages is that you will have to consider how users are managed. If DIRSync is being performed this may be more complex. Also these users need to be active during migration and then suspended once the threads are added.

For the purposes of forum migration it is unlikely you need to consider the values within the user profiles and migrating those, but again this is worth discussing for your scenario.

Groups

Consider how you will be mapping in the existing forums. Do the existing forums contain threads which span wide topics? Could it be beneficial to split the existing forum into smaller segments, in which case the thread mapping to Yammer Groups needs to establish this. Also consider that the groups may not yet exist, so you might have to work out the creation and admin assignment as well. Be careful not to orphan any threads which are topics that no longer fit into the Yammer Group structure.

Also the big difference between Yammer Groups and many forum based technologies is that the Yammer Groups lack any form of hierarchy, so where a source forum may be grouping several sub sections the Yammer Groups don’t behave in this fashion. My thoughts here is that you should structure the Yammer Network to support the new working patterns you are creating. It is likely that the Yammer rollout is in support of other initiatives. So consider if the new Groups are topic/subject centric now, and how you might have to map threads into these topics. Many of the implementations we have been rolling out use a SharePoint site to aggregate the groups into a form of hierarchy or associations. This is a loose coupling though and a sensible approach, the association to hierarchy is only appropriate if the user scenario warrants it, such as a knowledge network.

Group metadata such as name, logo and description are also key for discovery by users within the Yammer Network. Consider also telling users which the primary language is for the group. Yammer has a pretty good translation mechanism using Bing Translate, but some groups naturally centre around a specific language.

Consider User membership for new Yammer Groups, do you want to add the users from the existing forums to the new Yammer Group? Sometimes this is a straight mapping one to one, sometimes if you re-jig the structure it becomes more complex. A simple algorithm is to capture all the users involved in the group threads and make sure they get added to the Group members. Again this in principal goes against what I’d normally be proposing, Users really shouldn’t be forcibly added to group it’s bad practice for good engagement and really annoys people.

Threads and Content

Threads and content form the cornerstone of the forum experience, in Yammer these map roughly to ‘Conversations’ and individual ‘Yams’. The Yammer paradigm is quite straight forward structure wise.

  • Network – The overall container for all the content
  • Groups – The segmentation of the content within the network into groupings of people who generate content
  • Conversation (aka a thread) – The conversations which are started and replied too
  • Yam (aka a message) – An individual post within the conversation thread.

Basics

Each thread message needs to be analysed for the following:

  • It’s parent message, so that it can be correctly associated into a new Yammer conversation
  • Any forum members mentioned in the content, so that the correct @mentions can be inserted into the new Yam
  • Any tagging used, so that the correct @mentions can be inserted into the new Yam (I know it sounds weird to say @ mentions again, but the user and tags are effectively the same with some meta data differences)
  • Any attachments being referenced by the message, so that the correct handling approach can be used (more about attachments in a minute)
  • Any Hrefs within the content, so that you can decide how to deal with the link (more about that in a few more lines)

Links

Handling Hrefs can take a couple of formats:

  • Keep them as is and just ensure they map into the content specification of Yammer
  • Modify the url against a mapping rule set, creating a permanent redirection to other migrated content
  • Modify the url against a mapping service, creating a permanent redirection but to something that can manage the redirection as content moves around in supporting platforms

If you go for an external link redirection service consider how long this will be in place for. Consider the overall benefits of redirection and watch the analytics tracing to ensure that users are still getting benefits from the service.

I did consider using SharePoint query rules as a way to manage redirection. Imagine that you create key threads in Yammer, you can create the links within them to point to a SharePoint site, each site can have its own query rules for content promotion. So where you’re knowledge management community also has a SharePoint site collection you can implement a local collection of search schema search query rules. If you take this approach you can effectively ‘map’ a Href into a search based url with specific terms. Then it fires across to SharePoint and the relevant promoted results would be presented.

Attachments

Dealing with attachments can take several forms:

  • Uploading the content into Yammer directly and effectively storing that content in Yammer
  • Making reference to external sources such as a web page where the content lives

Depending on your requirements may mean choosing the option most relevant almost on a message by message basis. My recommendation would be to store information outside of Yammer where it better aligns with collaboration systems such as Office365 sites, and upload directly where it makes more sense as social content. The obvious examples are a rule set which treats all Office document formats as upload to SharePoint and things such as images go into Yammer. You may also decide to treat some messages as ‘OpenGraph’ and have Yammer treat them as more like a notification than a message.

Thread sharing

Yammer allows a thread to be shared once created to another group. When sharing the thread sharer can choose to add additional text to the conversation share. Once shared the thread is effectively a new thread with its own content. So with this in mind you need to analyse the forum sharing paradigm in the current forums and decide how to map them into Yammer.

In situations where a thread appears across multiple forums. For example:

  • Original post goes into forum A, then is shared to forum B and C
  • When a user view either forum A,B or C the entire thread appears, thus capturing all the conversation once, but displaying it in multiple places

You need to consider how this maps into the Yammer sharing paradigm. I think the most sensible approach is to decide which Yammer Group the thread is most suited for, this is probably a manual decision in most cases. Once targeted for a specific group post the whole thread to that group, then share the complete thread into other groups. It might also be work retrospectively adding ‘tags’ to the thread.

Topics

Yammer has a tagging concept called ‘Topics’. Each thread can be tagged with topics to help discoverability. A user can elect to follow a specific Topic. So when migrating content analyse the content for potential topics and ensure they are tagged in the Yam with Topics. This is a value add activity and would require quite a lot of intelligence to identify topics in existing threads.

Dates

As you add new Yams to the Yammer threads you can’t influence the date stamp. So if you require date stamps to be known the only real solution is to append the information into the Yam text.

Security

Always a thorny issue, the conflicting paradigms of information security and engagement and openness working like a network.

There is definitely no right answer about how to handle security, so these thoughts are where I’d propose you started.

Yammer’s paradigm really needs Yammer Groups to remain publically visible and readable to reap the benefits from working like a network. So you need to plan how to ensure information which might have been secure remains so. Often the existing forums are restricted security wise by ‘membership’ as a way to keep a tight control of who can edit, often driven by reporting and funding concerns about membership numbers rather than true data sensitivity. For this scenario in most cases adopting the public group paradigm is fine.

Where you need a private group, say for example the Board, then using a Private group is perfectly acceptable.

You will need to audit the information in every thread for its compliance if you want to be 100% sure that you’ve not shown restricted information from the existing forums in the new Yammer Groups.

User engagement and roll out

Although not strictly migration in the technical sense don’t under estimate how you need to deal with users moving them from the old world into the new. Thinking about how to help users find threads that have moved.

One possible solution to this is to add a new reply in the existing forums which contains information about the thread location within Yammer. This helps anyone who has the old thread bookmarked come away with a smile as they can navigate to the Yammer conversation and continue the discussions. It also allows for a time of dual running where some but not all threads have made it into Yammer.

Another aspect to consider is publishing the whole process for the end user to read. Sounds little bonkers at first, why should users care or be told about the internals of an IT project. Well if you think about what Yammer has as a central pillar… it’s openness. So help users get into the correct mind set.

Yammer Technical

App Registration

The processing app must be registered within Yammer. Instructions for the App Registration process can be found in the Yammer Developer Center: https://developer.yammer.com/introduction/#gs-registerapp

Once the migration processing app is registered the Client Id and Client secret can be used by the app to make calls into the Yammer Network.

Authentication

The processing app must authenticate to Yammer in order to make calls the API. Information about the authentication process and options can be found in the Yammer Developer Center: https://developer.yammer.com/authentication/

OAuth Impersonation

To interact with the Yammer API on behalf of another user you will need to use a Verified Admin account for the processing app and have it call the relevant REST API endpoint to collect the user token for the user in question. Information about impersonation can be found in the Yammer Developer Center: https://developer.yammer.com/authentication/#a-impersonation

Users

The Yammer API allows you to retrieve, create, update and suspend (soft delete) users. Information about the exact API call format can be found here: https://developer.yammer.com/restapi/#rest-users

Rate limits

The migration processing app must have logic to throttle the calls made to Yammer as to not exceed these thresholds. It should also monitor the responses and act accordingly to retry and ensure data integrity in the event of API calls being blocked. Information about the Yammer REST API rate limits can be found on the Yammer Developer Center: https://developer.yammer.com/restapi/#rest-ratelimits

The content at the time of writing is as follows:

API calls are subject to rate limiting. Exceeding any rate limits will result in all endpoints returning a status code of 429 (Too Many Requests). Rate limits are per user per app. There are four rate limits:

Autocomplete: 10 requests in 10 seconds.

Messages: 10 requests in 30 seconds.

Notifications: 10 requests in 30 seconds.

All Other Resources: 10 requests in 10 seconds.

These limits are independent e.g. in the same 30 seconds period, you could make 10 message calls and 10 notification calls

Pre-reqs

Also useful during the process is the availability of an Enterprise Yammer network, mainly so you can get the admin functions like data export and impersonation working. The question over having a discreet ‘dev’ network came up, my personal thoughts are as follows. If you’re not ‘live’ with Yammer while doing the migration work then why not simplify the moving parts by keeping just the main Network. At the end of the day you can always try things in a private group first to write your processing code, then open it up once the code is tested. Worst case you add messages you need to delete later. This keeps costs down. If however you’re already using Yammer, then best to go for multiple Networks. Again it’s your choice, but Yammer is a service that you can’t actually customise, so your options are safer in my opinion.

NB: This information is my pre-execution thinking and hopefully after the work is complete I’ll come back and make any amendments to the information.

SharePoint Evolution Roadshow 2014 session If apps are the answer what was the solution

SP Evo Conf

Wednesday June 11th saw the SharePoint Evolution roadshow roll into Cambridge. Bringing with it the weather Smile and a collection of SharePoint experts from around the globe.

The full days agenda can be seen here: https://www.sharepointevolutionconference.com/abstracts.html#camt1

My session kicked off the days ‘Technical’ track. The session described the evolution in thinking required when you move from the traditional SharePoint full trust solution model towards the SharePoint App model. Having been working with SharePoint apps since the pre-release program of SharePoint 2013 the session aimed to share my experiences and lessons learnt around the solution design approaches. The slides from the session are below.

 

Spotted new UI elements at SPC14

This short post gathers together a few things I spotted with the various demo’s which might give us a clue of some UI changes.

SuiteBar UI

Interesting to see what I assume is a version of the SuiteBar we can expect to see later this year.

image

Notice that Projects, Tasks and Oslo have appeared. Projects probably implies Project Online which currently hides in a context menu like in the picture of my tenant SuiteBar.

image

 

Sites page

Looks like the sites page might have gained some thumbnail images and a context menu fly out. Funny how this screen grab probably highlights nicely the issue most users will face. The site logo is either the standard SP version or will be set in the branding and thus be identical. Hopefully they will use the algorithms used inside Oslo to bring through something more engaging.

image

 

How Office Graph may change the OneDrive views

Below is a grab from the Oslo session which shows how the ‘Shared with me’ might benefit from the Office Graph enhancements.

image

SP.RequestExecutor cross domain calls using REST gotcha

CrossDomain

Recently I was building a prototype SharePoint hosted app to add items into the Host web. The basic operation of the app is that it queries the Host web for specific list types, then allows a user to add a collection of new items to the selected list. So read and write operations.

When dealing with the Host web it is important to remember that you are then subject to ‘cross domain’ calls and the restrictions in place for them. The browser protects users from cross site scripting and specifies that the client code can only access information within the same URL domain.

Thankfully SharePoint comes with some inbuilt options for these calls. The Cross Domain library is the primary option in either JSOM or REST forms.

I’ve been leaning towards REST mainly at the moment primarily as a focus for learning so I could get used to this method of data interactions.

So the first code sample is to get the host web task lists:

NB: This is a cut down extract of the function just to highlight the core request.

var executor;

// Initialize the RequestExecutor with the app web URL.
executor = new SP.RequestExecutor(appWebUrl);

//Get all the available task lists from the host web
executor.executeAsync(
{
url:
appWebUrl +
“/_api/SP.AppContextSite(@target)/web/lists/?$filter=BaseTemplate eq 171&$select=ID,Title,ImageUrl,ItemCount,ListItemEntityTypeFullName&@target=’” + hostWebUrl + “‘”,

method: “GET”,

headers: {  “Accept”: “application/json; odata=verbose” },
success: successHandler,
error: errorHandler
}
);

Note from this sample how the host and app web urls are used within the url combined with the SP.AppContextSite. This is the key to invoking a cross domain call in REST using the SP.RequestExecutor

The second snippet of code is the one which adds the new item to the host web list:

NB: This is a cut down extract of the function just to highlight the core request.

var executor;

// Initialize the RequestExecutor with the app web URL.

executor = new SP.RequestExecutor(appWebUrl);

var url = appWebUrl +
“/_api/SP.AppContextSite(@target)/web/lists(guid’” + hostWebTaskList.Id + “‘)/items?@target=’” + hostWebUrl + “‘”;

//Metadata to update.
var item = {
“__metadata”: { “type”: hostWebTaskList.ListItemEntityTypeFullName },
“Title”: item.title,
“Priority”: item.priority,
“Status”: item.status,
“Body”: item.body,
“PercentComplete”: item.percentComplete
};

var requestBody = JSON.stringify(item);

var requestHeaders = {
“accept”: “application/json;odata=verbose”,
“X-RequestDigest”: jQuery(“#__REQUESTDIGEST”).val(),
“X-HTTP-Method”: “POST”,
“content-length”: requestBody.length,
“content-type”: “application/json;odata=verbose”,
“If-Match”: “*”
}

executor.executeAsync({
url: url,
method: “POST”,
contentType: “application/json;odata=verbose”,
headers: requestHeaders,
body: requestBody,
success: addPrimeTasksToHostTaskListSuccessHandler,
error: addPrimeTasksToHostTaskListErrorHandler
});

Ok so at this point you’re probably wondering what is the gotcha mentioned in the title. Well here it comes and it’s one of those cut and paste horror stories which costs developers all over the land huge amounts of wasted effort.

So if you take a look at the following block of code

var url = appWebUrl +
“/_api/SP.AppContextSite(@target)/web/lists(guid’” + hostWebTaskList.Id + “‘)/items?@target=’” + hostWebUrl + “‘”;

 

//TODO: find out why this works but the below fails with not actually performing the update
//$.ajax({
executor.executeAsync({
url: url,
type: “POST”,
contentType: “application/json;odata=verbose”,
headers: requestHeaders,
data: requestBody,
success: addPrimeTasksToHostTaskListSuccessHandler,
error: addPrimeTasksToHostTaskListErrorHandler
});

You’ll notice i copied over the structure of the method from a normal $.ajax call. THIS IS MY MISTAKE!!!!

As with many things the devil is in the details. By using this ajax snippet I’d introduced a bug which took nearly 4 hours to work out (very little found on the popular search engines about this). The worst part is that the call fires and comes back with a 200 success and even enters the success handler, BUT the action is not performed.

So what is the cause? Well basically there are subtle differences in signature.

  • The ajax call ‘type’ should be ‘method’ in the SP.RequestExecutor
  • The ajax call ‘data’ should be ‘body’ in the SP.RequestExecutor

So there you have it, two word typo’s which throw no errors but cause a logical failure in the code.

I hope this helps someone else avoid the pain Open-mouthed smile

Some really useful information about this capability can be read at:

Chris’ app series covers using this library in anger – http://www.sharepointnutsandbolts.com/2012/11/access-end-user-data-in-host-web-from.html

Apps for Office and SharePoint blog article discussing the inner workings and options for cross domain versus cross site collection – http://blogs.msdn.com/b/officeapps/archive/2012/11/29/solving-cross-domain-problems-in-apps-for-sharepoint.aspx

Using REST in SharePoint apps – http://msdn.microsoft.com/en-us/library/office/jj164022.aspx

One final comment, MSDN Code has this sample: http://code.msdn.microsoft.com/SharePoint-2013-Get-items-7c27024f/sourcecode?fileId=101390&pathId=1361160678 which doesn’t really demo cross domain at the time of writing as it isn’t using the code correctly against the host web in my opinion.

Awarded Microsoft MVP 2013 for SharePoint Server

620MVP_Horizontal_FullColor

Literally minutes before leaving work on 1st  October I received the following email:

MVPEmail

This left me utterly speechless, which is not something that normal happens to me. After re-reading it about ten times the news finally sank in. I’m ecstatic that Microsoft have awarded me the MVP award for my contributions to the SharePoint community.

So I wanted to share a number of thank you’s for everyone who has supported me along the way.

Most importantly my beautiful wife who has supported my many many hours chained to my PC in the office at home beavering away on CKSDev or Presentations instead of snuggling up in front of a movie on the sofa. Without her understanding and support I definitely wouldn’t have found time to make the contributions I have. Also sitting through many hours of practice presentations before my speaking events.

Matt Smith for four years ago helping me to understand how to start to become involved in the SharePoint community.

Waldek Mastykarz for his friendship, support and advice and those midnight coding adventures before releasing a new version of CKSDev. Thanks buddy :)

Chris O’Brien for his friendship, advice and putting up with some seriously deep technical conversations over the years when I’m trying to get my head around a SharePoint feature and Visual Studio API.

The CKSDev team past and present, Wouter, Matt, Waldek, Todd, Paul, Carsten and Wictor who all at some point over the last few years have advised, written some cool code and generally helped ensure CKSDev was meeting everyone’s needs.

The event organisers for SUGUK, Evolutions/Best Practices London, SPSUK and SPSNL (Steve Smith and Combined Knowledge, Brett, Tony and Nick, Mirjam and the DIWUG peeps) for providing me opportunities to present and be part of the event teams.

My employers Content and Code and especially people like David Bowman and Tim Wallis who gave me time and support to contribute to the community. Not to mention Salvo di Fazio, Tristan Watkins and Ben Athawes.

To conclude what has been quite an emotional post to write I just want to say thanks to everyone in the community and MVP community who have helped me all these years. There are too many to list, but you know who you are :)

Yammer SharePoint App Installation Error

YammerError

Yammer and Microsoft released a new SharePoint app available in the Office Store here: Yammer App details

This is a great first step towards better Yammer and SharePoint integration Smile

However on installing it on some sites I encountered an error:

image

If this was an on-prem installation I’m sure the ULS would be giving some huge clues as to what is up, but I was lucky enough to be using Office365 (I love the fact bugs are Microsoft’s problem to diagnose Smile). So the only way forward was to raise Service Request. So after some problem investigation with the Support guy we got to the answer.

The key reason this App fails to install is the supported locales from Microsoft. As you can see from the screenshot below, only US English is a supported locale.

image

Ok so nothing massively unusual there? Nope in SP2010 this is the only English available, but with SP2013 Microsoft finally worked out the UK uses ‘proper’ English with all it’s quirky spellings for things like ‘colour’. So with this in mind all of our tenant site collections are set to UK English as the locale, as you can see in the screenshot below.

image

So when you add the app you think everything would be fine…. oh how wrong you’d be… So the add new app pops the ‘trust’ dialog. My first comment here is that it also includes the language options, so not really only about ‘trust’ is it.

image

Second bad user experience here is that the languages selection is hidden by default. So being a typical user I didn’t read the information and clicked ‘trust it’. And that’s when the install error happens.

So what should you be doing?

image

So the killer ‘feature’ is that the ‘Trust it’ dialog is picking up the current sites default locale (in this case English UK) and installing the app with that locale. Now if you remember the Yammer App only supports English US locale, so you need to select this locale from the dropdown.

image

So now the app is installing with its supported English US locale into our English UK sites. So our users get the language they want in most of SharePoint and Yammer works in it’s supported language.

image

image

I think this is pretty bad that it defaults to install in a locale it doesn’t support and provides no feedback to the user. So this experience is littered with badly designed UX and errors which would be very easy to avoid, and thus wasting the time of both the user and MS Support. Sad smile Lets hope someone fixes this in some later releases.

Further fixes and information can be found here: http://blogs.msdn.com/b/ragnarh/archive/2013/07/02/yammer-app-for-sharepoint-amp-office-365-tips-amp-tricks.aspx

I hope this saves someone the hassle of raising a ticket for something so simple.

CKSDev for Visual Studio 2012 version 1.2 Released

CKSLogo
The 1.2 version has just been released. It contains loads more features you had available in VS2010. All of the other features will be coming over the coming weeks as personal time allows. As you can imagine it’s no mean feat to port all of the existing features and support multiple SharePoint versions. The code base is also on a diet as code bloat was getting a little bit crazy as the tools evolved over the past 4 years.
 
You can find CKS: Development Tools Edition on CodePlex at http://cksdev.codeplex.com.
 
Download the extension directly within VS2012 from the extension manager or visit the Visual Studio Gallery.
We’re still looking for extension ideas so please post in the CKS Dev Discussions anything you think would make life better for SharePoint developers.
 

CKS Dev Feature highlights

Environment:
  • Copy Assembly Name – From the context menu of a SharePoint project you can now get the full assembly name copied to the clipboard.
  • Cancel adding features – Automatic cancellation of new SPIs being added to features. You can turn this off via the CKSDev settings options.
  • Find all project references – Find where a SPI/Feature is being used across the project from the project context menu.
  • Activate selected features – Setup which package features you want to auto-activate from the project context menu.
Exploration
  • No new features
Content
  • ASHX SPI template – Produces a full trust ASHX handler.
  • Basic Service Application template – Produces a full trust basic service application.
  • Banding SPI Template – Produces a full collection of SP2010 branding elements baseline items.
  • Contextual Web Part SPI template – Produces a contextual ribbon enabled web part.
  • WCF Service SPI template – Produces a full trust WCF Service endpoint.
  • Web template SPI template – Produces a SP2010 web template.
Deployment
  • Improvements to Quick Deploy – Performance improvements with a switch from calling into GACUtil.exe and returning to direct GAC API calls to improve performance. Also removal of ‘custom tokenisation’ for now until a more performant version is tested.
  • Quick deploy GAC/Bin deployment configuration – A deployment configuration which runs the pre-deployment command line, recycles the application pool, copies the assemblies to either the BIN or GAC depending on their packaging configuration, and runs the post-deployment command line.
  • Quick deploy Files deployment configuration – A deployment configuration which runs the pre-deployment command line, recycles the application pool, copies the SharePoint artefacts to the SharePoint Root, and runs the post-deployment command line.
  • Quick deploy all deployment configuration – A deployment configuration which runs the pre-deployment command line, recycles the application pool, copies the assemblies to either the BIN or GAC depending on their packaging configuration, copies the SharePoint artefacts to the SharePoint Root, and runs the post-deployment command line.
  • Upgrade deployment configuration – A deployment configuration which runs the pre-deployment command line, recycles the application pool, upgrades the previous version of the solution, and runs the post-deployment command line.
  • Attach To IIS Worker Processes Step – Which attaches the debugger to the IIS Worker process during deployment.
  • Attach To OWS Timer Process Step – Which attaches the debugger to the OWS Timer process during deployment.
  • Attach To SPUC Worker Process Step – Which attaches the debugger to the User Code process during deployment.
  • Attach To VSSPHost4 Process Step – Which attaches the debugger to the Visual Studio deployment process during deployment.
  • Copy Binaries Step – Copies the assemblies during deployment to either Bin or GAC.
  • Copy To SharePoint Root Step – Copies the files during deployment to the SharePoint Root.
  • Install App Bin Content Step – Copies the files to the App Bin folder during deployment.
  • Install Features Step – Installs the features during deployment.
  • Recreate Site Step – Recreates the deployment targeted site during deployment.
  • Restart IIS Step – Restarts IIS during deployment.
  • Restart OWS Timer Service Step – Restarts The SharePoint Timer service during deployment.
  • Upgrade Solution Step – Upgrades the solution during deployment.
  • Warm Up Site Step – Calls the site to get SharePoint to warm it up during deployment.
To see more details and the full feature list visit http://cksdev.codeplex.com/documentation.
 

Visit the CodePlex site for more information about the features and release.

Share and Enjoy and thanks for the continued support

SPSNL 2013 Presentation

SPSNL

Saturday 29th June saw the 2013 SharePoint Saturday Holland. Another great event organised with so many quality speakers and companies in attendance. It was a privilege to be invited to speak Smile

I presented a session on Apps for Office 2013 and SharePoint 2013, the slides can be seen below. I hope everyone found the session useful Smile I certainly enjoyed presenting to such an interactive audience.

I think Apps for Office is one of the coolest new features of Office and SharePoint 2013 and my session gives a really quick overview of the Apps for Office solution space. Then the hook up between SharePoint and Office that is now possible through the demo solution.

Over the next few months I’ll be publishing a dedicated series for Apps for Office so stay tuned for more soon.

Thanks to everyone who helped organise the event.

SPC125 – Hybrid and Search in the Cloud – Brad Stevenson

SPCLogo

Brad Stevenson talks about creating a search experience which spans an On-Premises and Office365 SharePoint 2013 environment.

Search in the cloud

 

So what is the story about the cloud search capability?

Comparing SP2010 online to SP2013 online

 

Area

2010

2013

Crawl Freshness

1-2 hours <15 minutes

Query Latency

good better

Scale

was limited now much greater

Manageability

was limited more extensive

User Experience

ok big UX improvements

Extensibility

very little some new stuff

 

  • Crawl freshness is important for any search system as its important that a user trusts the results they are given. Part of this trust is that they are seeing the latest content within a timely window.
  • Query latency is already really quite snappy in SP2010 online. SP2013 moves to client side rendering approaches to improve this snappiness perception even further. This approach allows the server to share some of the rendering load with the client device improving performance for the end user.
  • Scale in SP2013 originates from the FAST technologies so brings those benefits to bear, making it a powerful and scalable platform solution.
  • Manageability within SP2013 allows more control over the schema, examples are the control over managed properties and result sources. A lot of the features which were part of the service application have now been brought down to the site collection and tenant administration levels.
  • User Experience is dramatically different with new capabilities such as hover panels, visual refinements etc. This helps a user to establish the relevance of a result without leaving the results page or downloading the documents.
  • Extensibility is improved without writing code with such elements such as the rendering templates replacing the complex XSL.

Search extensibility in the cloud

 

No code:

  • Managed properties
  • Result sources
  • Query rules
  • Result types
  • Display templates
  • Dictionaries (via the Term Store)

Code:

  • CSOM
  • REST

Packaging:

  • Import/Export search settings
  • SharePoint apps

You manage your ‘global search’ via the tenant admin interface. The only major piece of the service application settings you have no control over is the crawl scheduling. In a multi-tenant environment this really makes sense.

Journey to the cloud

 

Definition of the cloud?

 

  • Public cloud – Office 365, allows you to focus on just the software services.
  • Private cloud – Windows Azure, allows you to offload the OS and hardware to the cloud provider.
  • Traditional – All managed by the internal organisation

Moving to the cloud

 

What to move? (not just everything) and should it be everything including customisations and settings.

When to move it? How do you plan the move? Is it an all or nothing or staged co-existence.

How to move it? What tools are available?

The migration lifecycle

 

Early – 90% on-prem 10% cloud

Mid – 50% on-prem 50% cloud

Late – 10% on-prem 90% cloud

How hybrid search can help

 

User want to easily find content, they just want to find things they’re looking for and not have to think about understanding the systems structure. It is about getting their job done efficiently.

Users don’t care about migration. So don’t force users to track what’s being moved and when.

Realise that most users will never move EVERYTHING to the cloud.

Hybrid Search User Experience

 

Demo environment details:

  • On-Premises SP2013 crawling a mixture of SharePoint data and file shares
  • Office365 indexing all of its content
  • Firewall between

So the idea is that from within either environment the user can get results from either. They use query rules to ‘cross-pollinate’ results from the other environment as a block of results. Personally I’m not sure this is a great user experience. It gives a false impression to a user of which results are most important. So I remain to be convinced about using result blocks.

A neat thing to know is that the refinement panel operates over ALL returned results rather than just the local SharePoint items. Also the hover panels are dependant on the sources WOPI configuration.

Configuring Hybrid Search

 

Configuration steps:

  • Choose one-way or bi-directional
  • Deploy and configure pre-reqs
  • Configure search data settings
  • Configure search UI settings

If you are early on in your migration lifecycle a one-way where on-premises indexes Office365 might suit your needs. Or late on a one-way works for Office 365 to use on-premises. Mid-life is definitely bi-directional where the experience should be the same.

Environment Configuration

 

image

One-way or Bi-directional

 

Where will users go to search?

  • Just on-premises
  • Just Office365
  • Both

Hybrid pre-requisites

 

Non-SharePoint:

  • Reverse proxy and certificate authentication
  • Identity provider (ADFS or Shibboleth for Office365)
  • MSOL Tools
  • SSO with Office365
  • DirSync

SharePoint:

  • New SharePoint STS Token signing certificate
  • Configure a trust relationship between SharePoint on-premises and ACS
  • Configure secure store
  • Configure UPA

Configure Data Settings

 

Result source (equivalent to a federated location and scope in SP2010) pointing at:

  • URL of remote location
  • Secure Store (for client certificate)

Configure UI Settings

 

  • Query rule to show remote results in ‘Everything’ tab
  • Search vertical which ‘only’ displays results from remote location (Office365 or on-premises)

Search Centre On-Premises: Data Flow

 

image

Scenario One:

User logs into on-premises and issues a search query. It actually issues two queries. First is to the local on-premises index. The second is issued to Office365. The second query is issued through the CSOM endpoint within Office365. Identity mappings take place where the on-premises identity is mapped to the Office365 identity. Office365 then performs the query and issues the results response.

Search Centre in Office365: Data Flow

 

image

Scenario Two:

Basic flow is a reverse of on-premises except there is the introduction of the revers proxies at the perimiter to route the request back to the on-premises SharePoint. Identity is mapped from the Office365 user to the on-premises user.

This means that in both scenarios there is correct data security trimming.

Beyond the Basics

Design considerations

 

What did Microsoft consider when designing the Office365 service:

  • Crawl versus Query – Chosen to go the query root as the crawl infrastructure within Office365 was limiting. Also hundreds of thousands of tenants need to have a consistent performance maintained. Query helps to provide the best and most consistent user experience.
  • UI Presentation – Users felt it was really important to see all the results on the same page. They didn’t want to have to switch between different pages. (I’m not sure I agree with this UX being the most optimum, users are used to choosing the ‘type’ of data they want on Bing/Google eg. Images/Shopping etc)
  • Relevance and clicks – Search learns over time. Search watches which results are clicked for specific queries and adjusts over time.
  • Background compatibility – Should provide other MOSS and SP2010 on-premises hybrid. This was not possible due to some significant infrastructure and services which would have required significant investment in extending the previous SharePoint versions. One major element was the challenge of identity mapping.

Alternative Hybrid User Experience

 

The demonstration showed the example of a glossary. This is stored in a SharePoint list on-premises. It is very useful information, but is not something users query for all the time. From on-premises the demo pulls ‘people’ results from Office365 via a dedicated search vertical.

To get the on-premises ‘people’ page to use the Office365 profiles it is as simple as pointing the web part settings to use the remote result source. This means when the users click the people results it goes to the Office365 profile page. Question is what happens to the links to people in normal item results?

Hit highlighting works.

Question about multiple people stores. Answer was the suggested best practice is to host in just one location.

Result Click-Through: Options

 

image

For users on-premises accessing on-premises links are fine. Once the user is on the Office365 search results the results now have internal only urls. The click through doesn’t route through the reverse proxy anymore. So users must have that external access to the internal system.

One solution is to have a VPN or Direct Access or leveraging the reverse proxy for url re-writing.

Microsoft recommend using VPN or Direct Access as it is easier to maintain over time.

The WOPI previews are operating where-ever the content is being served from.

CKSDev for Visual Studio 2012 version 1.0 Released

CKSLogo

Since 2009 CKSDev has been available to SharePoint 2010 developers to aid the creation of SharePoint solutions. The project extended the Visual Studio 2010 SharePoint project system with advanced templates and tools. Using these extensions you were able to find relevant information from your SharePoint environments without leaving Visual Studio. You experienced greater productivity while developing SharePoint components and had greater deployment capabilities on your local SharePoint installation.

The Visual Studio 2012 release of the MS SharePoint tooling supports both SharePoint 2010 and 2013. Our aim for CKSDev was to follow this approach and provide one tool to support both SharePoint versions from within Visual Studio 2012.

The VS2012 version has just been released. It contains the core quick deployment and server explorer features you had available in VS2010. All of the other features will be coming over the coming weeks as personal time allows. As you can imagine it’s no mean feat to port all of the existing features and support multiple SharePoint versions. The code base is also on a diet as code bloat was getting a little bit crazy as the tools evolved over the past 4 years.

You can find CKS: Development Tools Edition on CodePlex at http://cksdev.codeplex.com.

Download the extension directly within VS2012 from the extension manager or visit the Visual Studio Gallery.

We’re still looking for extension ideas so please post in the CKS Dev Discussions anything you think would make life better for SharePoint developers.

CKS Dev Feature highlights

Environment:
  • Copy assembly name so no more need to use reflector to get the full strong name.
Exploration
  • New Server Explorer nodes to list feature dependancies, site columns, web part gallery, style library, theme gallery.
  • Enhanced import functionality for web parts, content types, themes.
  • Create page layout from content type (publishing).
  • Copy Id.
Content
  • Coming Soon…
Deployment
  • Quick deploy to copy artefacts into the SharePoint root without the need for a build.
  • Deployment steps Coming Soon..

To see more details and the full feature list visit http://cksdev.codeplex.com/documentation.

Visit the CodePlex site for more information about the features and release.

Share and Enjoy and thanks for the continued support