Cumulative Update 7 for AppFabric 1.1 KB3092423 – CU7

Microsoft has released Cumulative Update (CU) 7 for App Fabric 1.1 – KB3092423. The fine SharePoint folk out there know App Fabric as the SharePoint Distributed Cache. It is an important prerequisite that SharePoint 2013 & SharePoint 2016 require prior to installation. In fact, SharePoint security, performance and some social features heavily rely on the Distributed Cache working properly.

Cumulative Update 7 for App Fabric 1.1 has a fix for intermittent crashes and unexpected behaviour when managing the service.

Microsoft has reported the following three issues have been fixed with this Cumulative Update 7.

CU 7 is a requirement for SharePoint 2016. This is what I got when I tried to install SharePoint 2016:2015-11-25_18-55-08

App Fabric Cumulative Update 7 fixes the following issues:

Issue 1

The AppFabric Caching service crashes intermittently with the following exception:

{System.Runtime.CallbackException: Async Callback threw an exception. —> System.NullReferenceException: Object reference not set to an instance of an object.
at System.Runtime.AsyncResult.End[TAsyncResult](IAsyncResult result)
at System.ServiceModel.Channels.CommunicationObject.EndOpen(IAsyncResult result)
at Microsoft.ApplicationServer.Caching.WcfServerChannel.OnOpen(IAsyncResult result)
at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
— End of inner exception stack trace —
at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
at System.Runtime.AsyncResult.AsyncCompletionWrapperCallback(IAsyncResult result)
at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
Issue 2

Note – May not apply in SharePoint as we use Stop-SPDistributedCacheServiceInstance –Graceful to gracefully stop a cache host.

When you run the Stop-CacheHost AppFabric PowerShell command together with the -Graceful switch (in SharePoint, we use), you can’t stop the cache host, and you receive the following error message:

Exception type: System.Collections.Generic.KeyNotFoundException
Message: The given key was not present in the dictionary.
InnerException: <none>
StackTrace (generated):
SP IP Function
000000001F28DAA0 000007FEF611C96F mscorlib_ni!System.Collections.Generic.Dictionary`2[[System.__Canon, mscorlib],[System.Boolean, mscorlib]].get_Item(System.__Canon)+0x4098ff

000000001F28DAD0 000007FF007A860B Microsoft_ApplicationServer_Caching_Management!Microsoft.ApplicationServer.Caching.AdminApi.CacheAdmin.GetNoOfNodesInServiceState(System.Collections.Generic.List`1<Microsoft.ApplicationServer.Caching.IHostConfiguration> ByRef, Microsoft.ApplicationServer.Caching.AdminApi.ProgressDelegate, Microsoft.ApplicationServer.Caching.AdminApi.ServiceStatus)+0xfb 000000001F28DBD0 000007FF007A83F1 Microsoft_ApplicationServer_Caching_Management!
Issue 3
Note – May not apply to how we work in SharePoint as we use Add-SPDistributedCacheServiceInstance to register a Cache host.

When you run the Register-CacheHost AppFabric PowerShell command, the command fails, and you receive the following error message:

Register-CacheHost : ErrorCode<PortAlreadyInUseError>:SubStatus<ES0001>:TCP port 22234 is already in use.
Note This failure generally occurs if the cache ports are bound to any IP address.

Source: https://support.microsoft.com/en-us/kb/3092423#/en-us/kb/3092423

 

To check AppFabric version on your server, run this in PowerShell:

(Get-ItemProperty "C:\Program Files\AppFabric 1.1 for Windows Server\PowershellModules\DistributedCacheConfiguration\Microsoft.ApplicationServer.Caching.Configuration.dll" -Name VersionInfo).VersionInfo.ProductVersion


So how do you go about applying Cumulative Updates to the SharePoint Distributed Cache (App Fabric 1.1)?

I have documented the manual steps here as there are many misleading blog posts out there. Feel free to copy this article and use this in your organisations Standard Operating Procedure (SOP) documents: http://www.jeremytaylor.net/2015/09/17/how-to-patch-sharepoint-distributed-cache-the-proper-way/

 

AppFabric 1.1 CU versions index:

RTM: 1.0.4632 (29-Nov-2011)

CU1: 1.0.4639 (20-Feb-2012) – KB2671763

CU2: 1.0.4644 (29-May-12) – KB2716015

CU3: 1.0.4652.2 (27-Nov-2012) – KB2787717

CU4: 1.0.4653.2 (28-Mar-2013) – KB2800726

CU5: 1.0.4655.2 (03-Mar-2014) – KB2932678

CU6: 1.0.4656.2 (27-Feb-2015) – KB3042099

CU7: 1.0.4657.2 (01- Sept-2015) – KB3092423

 


Cumulative Update 6 for AppFabric 1.1 KB3042099 – CU6

Once again, an AppFabric 1.1 Cumulative Update has been released without a notification from the AppFabric / Windows Azure / Distributed Services Support Team.

Cumulative Update 6 for AppFabric 1.1:

Contains bug fixes for both the caching and hosting services provided in AppFabric 1.1.

 

Cumulative Update 6 File publish date:

21st April 2015

KB3042099 publish date:

1st May 2015

 

Issues that are fixed:

Issue 1: Caching issue

Under certain rare conditions, a cache item could end up in an inconsistent state. Therefore, the AppFabric Cache service may crash because of an unhandled exception. Examples of such an exception include the following.

Exception 1

System.NullReferenceException at Microsoft.ApplicationServer.Caching.MultiDirectoryHashtable.PreProcess(), with the error message “Object reference not set to an instance of an object.”

Exception 2

Microsoft.Fabric.Common.ReleaseAssertException at Microsoft.Fabric.Common.ReleaseAssert.Fail(), with the error message “Operation of type COMMIT_DELETE is not expected to fail during postoperation – System.NullReferenceException: Object reference not set to an instance of an object.”

 

 

Since I haven’t come across this issue, I wasn’t sure if these ‘rare conditions’ would apply to our Distributed Cache service in SharePoint 2013. Here are some questions that I hope I could answer:

 

Question: Have other users experienced this?

Answer: According to this post, this was a known issue way back in May 2011 and a Microsoft rep answered that it ‘will be fixed in coming release of appfabric caching’ – May 25, 2011.

https://social.msdn.microsoft.com/Forums/en-US/1e5b050f-d365-495d-bbd2-011313f4a250/appfabric-service-crashing-frequently?forum=velocity

 

Question: Does this apply to SharePoint 2013 Distributed Cache?

Answer: Yes it does! Because SharePoint Updates do not update the AppFabric software running on the SharePoint 2013 server. After AppFabric and SharePoint are installed on the server, AppFabric will continue to be updated independently from SharePoint. Reference: https://support.microsoft.com/en-au/kb/2843251

 

AppFabric can be installed on the following operating systems:

  • Windows Server 2012 R2
  • Windows Server 2012
  • Windows Server 2008 R2
  • Windows Server 2008 Service Pack 2

AppFabric can be installed for development and testing on the following operating systems:

  • Windows 8
  • Windows 7
  • Windows Vista Service Pack 2

Supported Architectures:

  • 32-bit (x86)
  • 64-bit (x64)

Software Requirements:

  • To apply this Cumulative Update package, AppFabric 1.1 for Windows Server must already be installed on the computer.
  • Additionally, you must have the Microsoft .NET Framework 4.5 installed.

 

ATTENTION AppFabric support:

On 2nd April 2015, Microsoft announced end of support for Microsoft AppFabric 1.1 starting 2nd April 2016. However, we learned that SharePoint Server 2016 will utilise AppFabric, so what will happen to AppFabric support? At the Ignite Conference, Bill Baer announced that Microsoft will support technologies such as SharePoint 2016 that have a dependency on AppFabric.

 

 AppFabric 1.1 cache dll versions Microsoft has released:

RTM: 1.0.4632 (29-Nov-2011)

CU1: 1.0.4639 (20-Feb-2012) – KB2671763

CU2: 1.0.4644 (29-May-12) – KB2716015

CU3: 1.0.4652.2 (27-Nov-2012) – KB2787717

CU4: 1.0.4653.2 (28-Mar-2013) – KB2800726

CU5: 1.0.4655.2 (03-Mar-2014) – KB2932678

CU6: 1.0.4656.2 (27-Feb-2015) – KB3042099

 

To check AppFabric version on your server, run this in PowerShell:

(Get-ItemProperty “C:\Program Files\AppFabric 1.1 for Windows Server\PowershellModules\DistributedCacheConfiguration\Microsoft.ApplicationServer.Caching.Configuration.dll” -Name VersionInfo).VersionInfo.ProductVersion

 

How to patch the Distributed Cache in SharePoint 2013

Don’t forget to gracefully stop the SharePoint Distributed Cache Service Instance before applying the Cumulative Update.

http://www.wictorwilen.se/how-to-patch-the-distributed-cache-in-sharepoint-2013

 

To read more about this Cumulative Update 6, visit:

https://support.microsoft.com/en-us/kb/3042099

 

 To search for other Cumulative Updates for AppFabric, visit:

https://support.microsoft.com/en-us/search?query=Cumulative%20Update%20Microsoft%20AppFabric%201.1&p=

 

 

 

I highly recommend you to visit & bookmark my page on SharePoint, SQL Update Centers:

http://www.jeremytaylor.net/2014/04/12/latest-cumulative-updates-service-packs-hotfixes-sharepoint-sql/

 

 

 


SharePoint 2013 Distributed Cache service

This post is meant to give SharePoint Administrators an deeper look into the Distributed Cache along with some links to other resources on the internet.

Background
SharePoint 2013 uses the AppFabric (Caching) by Microsoft. Read here for a background on AppFabric. AppFabric Caching stores serialised managed objects in a ‘cache cluster’. The cache cluster consists of one or more machines (Cache Hosts) that pool their available physical memory. This pooled memory is presented  and used by SharePoint as a single source of caching memory.

How is it used in SharePoint
SharePoint uses the Distributed Cache to store data for very fast retrieval across all entities. The Distributed Cache service provides in-memory caching services to several features in SharePoint Server 2013. Some of the features that use the Distributed Cache service include:
· Newsfeeds
· Authentication
· OneNote client access
· Security Trimming
· Page load performance

In SharePoint Server 2013, there are several caches that exist, all of which depend on the Distributed Cache service.
Different caches that depend on the Distributed Cache service as indicated in the following table:

Name Cache name
Login Token Cache DistributedLogonTokenCache
Feed Cache DistributedActivityFeedCache This cache stores activities and conversations for use by the feeds on a user’s My Site.
Last Modified Time Cache DistributedActivityFeedLMTCache This cache stores time stamp information for all Feed Cache entities
OneNote Throttling DistributedBouncerCache
Access Cache DistributedAccessCache
Search Query Web Part DistributedSearchCache
Security Trimming Cache DistributedSecurityTrimmingCache
App Token Cache DistributedServerToAppServerAccessTokenCache
View State Cache DistributedViewStateCache
Default Cache DistributedDefaultCache This cache can be used by any feature.

Source: http://technet.microsoft.com/en-us/library/jj219700.aspx

Installation
AppFabric is a required component for SharePoint 2013 and is installed as part of the prerequisite installer. Thus each server automatically becomes part of the ‘cache cluster’. There is nothing to configure here as SharePoint does this automatically.

Farm Architecture – modes
Here is some terminology that will be useful when talking SharePoint architecture and topology:
There are two ‘modes’ for Distributed Cache – A collocated mode or a dedicated mode. The Distributed Cache is started and run on all WFE and APP servers by default. If you have over 10000 users, you should look into a dedicated server (dedicated mode) Distributed Cache. Dedicated Mode simply means all other services are turned off and more memory is allocated to the Distributed Cache.

Farm design
While designing your topology, could consider starting the Distributed Cache service on your WFE servers (for redundancy) and stop the Distributed Cache Service on the APP server if you are concerned with ‘overhead’ caused by all the other services running on the APP server. You can consider a scale up plan if you are low on resources or scale out (by starting the Distributed Cache on other servers in the farm).
Finally it boils down to memory allocation at the end of the day. By default, the Distributed memory allocation size defaults to a value of 10 percent of total physical memory when SharePoint Server 2013 installs. You can change the memory allocation with the Update-SPDistributedCacheSize cmdlet.

Here is a link that may help on the ‘right approach’: http://technet.microsoft.com/library/jj219572%28office.15%29.aspx (note the ‘Capacity planning for the Distributed Cache service’ section).

Management
The Distributed Cache service can be stopped or started via the Central Administration or via PowerShell.

Servers can be added or removed from the ‘Cache cluster’ too. When removing the server, the Distributed Cache service is stopped, then unregistered from the server. Unregistering the Distributed Cache service means that you will not see the Distributed Cache service listed on the Services on Server page in Central Administration. Adding a server means that you will see the Distributed Cache service listed on Services on Server page in Central Administration.

To add a server and start the service in PowerShell: Add-SPDistributedCacheServiceInstance

To remove a server and stop the service in PowerShell:
Remove-SPDistributedCacheServiceInstance

To stop (not remove) the Distributed Cache service by using Central Administration: In Central Administration, click Application Management. In Service Applications, click Manage Services on Server. On the Services on Server page, locate the Distributed Cache service. If the Distributed Cache service is started and you want to stop the service, under Action, click Stop.

To stop (not remove) the Distributed Cache service by using Windows PowerShell: At the Windows PowerShell command prompt, run the following command: $instanceName =”SPDistributedCacheService Name=AppFabricCachingService” $serviceInstance = Get-SPServiceInstance | ? {($_.service.tostring()) -eq $instanceName -and ($_.server.name) -eq $env:computername} $serviceInstance.Unprovision()

To check existing memory allocation of the Distributed Cache Host:
Use-CacheCluster
Get-AFCacheHostConfiguration -ComputerName ComputerName -CachePort “22233”

To reconfigure the cache size of the Distributed Cache Service:
Update-SPDistributedCacheSize -CacheSizeInMB CacheSize


Firewall configuration considerations

The Distributed Cache service uses the following communication ports:
22233
22234
22235
22236

Monitoring
The Distributed Cache can be monitored using below performance counters in Perfmon:

SharePoint2013DistCachecounters1
SharePoint Distributed Cache Counters
Cache Data Transferred Per Sec
Cache Hit Count
Cache Hit Ratio
Cache Miss Count
Cache Read Requests per sec
Cache Write Requests per sec
Total Cache Read Requests
Total Cache Write Requests

SharePoint2013DistCachecounters2
Additionally, there are three groups of ‘AppFabric’ counters available. Note there are multiple instances – indicating the different caches present in the Distributed Cache.
AppFabric Caching:Cache
AppFabric Caching:Host
AppFabric Caching:Secondary Host

Important Distributed Cache resource
Manage the Distributed Cache service in SharePoint Server 2013
Plan for feeds and the Distributed Cache service in SharePoint Server 2013

The above link shows how you can:

  • Change the memory allocation of the Distributed Cache service
  • Add or remove a server in a Distributed Cache cluster
  • Perform a graceful shutdown of the Distributed Cache service
  • Change the service account
  • Repair a cache host

Further reading
5 Minute Cheat Sheet on SharePoint 2013 Distribution Cache
Leveraging AppFabric for custom caching in SharePoint 2013
AppFabric Distributed Caching and SharePoint 2013
AppFabric Caching and SharePoint: Concepts and Examples (Part 1)
AppFabric Caching (and SharePoint): Configuration and Deployment (Part 2)
How to get Distributed Cache size (SharePoint 2013)
Leveraging AppFabric for custom caching in SharePoint 2013
Cache cluster is down, restart the cache cluster and Retry