There are many challenges when migrating from SharePoint on-premises to SharePoint Online and one of the more common ones that I’ve encountered is handling email enabled libraries. These are libraries in SharePoint on-premises which are configured to allow a user to send an email with an attachment to a specified email address and the attachment is then stored in the library as a file. When you enable the functionality SharePoint will automatically create several fields to capture information about the original email; these include E-Mail Sender, E-Mail To, E-Mail Cc, E-Mail Subject, and E-Mail Headers. These fields are special built-in fields that are hidden from most pages such as the list settings page and the new/edit/display form pages but you can see them when editing a list view so that you can add them to any views that you create.
When migrating to SharePoint Online a popular tool of choice is Sharegate. With Sharegate you can migrate an entire site collection or a single list and everything in between. If you’re migrating a complete site collection, libraries with these special email fields added will be migrated completely, meaning that the email fields will come over as expected; however, if you’re migrating just a single library then the fields will be skipped. The reason for this is because the fields belong to a “special” group with a value of _Hidden. (I believe they are skipped intentionally by Sharegate but I’ve not been able to confirm this with them at this time). Of course, SharePoint Online doesn’t support email enabled lists so for most people this might not be a concern if they migrate and lose these fields. For some though, the historical data is critical to maintain and though email enabled lists aren’t supported it is possible to use Power Automate or a Logic App to replicate the functionality and for those who deam the original email details as important it would be useful to be able to continue to populate the same fields without having to manually create a new set of fields.
In order to essentially trick Sharegate into migrating the fields we need to change the group associated with the fields to something other than _Hidden (such as Custom). The trick with this is that the fields are all marked as Sealed so you can’t just change the properties - doing so will result in an error or nothing at all happening depending on which property you’re attempting to set. So to get around this you need to temporarily set the Sealed property to False. Normally, using PowerShell, you would be able to just do something like this:
1$web = Get-SPWeb "https://example/"
2$list = $web.Lists["Email Enabled Library"]
3$field = $list.Fields["E-Mail Sender"]
4$field.Sealed = $false
5$field.Update()
The problem with this is that because the field is a built-in system field it will throw an error stating that the operation is not valid due to the current state of the object. To get around this we have to use a bit of reflection to call the internal methods to set the property and trigger the update.
To figure out exactly what needs to be done it’s helpful to look at the Sealed property itself using a disassembler. The following image shows the property - I’ve highlighted the important bit:
From this we can tell that we need to call the internal SetFieldBoolValue(string, bool) method and then call the internal UpdateCore(bool) method to commit the change. We can achieve this in PowerShell by gtting an instance of the System.Reflection.MethodInfo object for the methods and then calling the Invoke() method passing in the relavent arguments:
1# Get the MethodInfo object for the SetFieldBoolValue property
2$setFieldBoolValue = $field.GetType().GetMethod("SetFieldBoolValue", [System.Reflection.BindingFlags]::NonPublic -bor [System.Reflection.BindingFlags]::Public -bor [System.Reflection.BindingFlags]::Instance -bor [System.Reflection.BindingFlags]::InvokeMethod)
3
4# Get the MethodInfo object for the UpdateCore property
5$updateCore = $field.GetType().GetMethod("UpdateCore", [System.Reflection.BindingFlags]::NonPublic -bor [System.Reflection.BindingFlags]::Public -bor [System.Reflection.BindingFlags]::Instance -bor [System.Reflection.BindingFlags]::InvokeMethod)
6
7# Call the Invoke() method to set the Sealed property to False
8[void]$setFieldBoolValue.Invoke($field, @("Sealed", $false))
9
10# Change the group (no reflection needded)
11$field.Group = "Custom"
12
13# Call the Invoke() method to update the properties (pass in True to force the Sealed property to update)
14[void]$updateCore.Invoke($field, @($true))
Once you’ve set the Group property and committed the changes then you can reset the Sealed property:
1[void]$setFieldBoolValue.Invoke($field, @("Sealed", $true))
2[void]$updateCore.Invoke($field, @($true))
With the Group property now set to something other than _Hidden Sharegate will now “see” the field and be able to migrate it and any data associated with the field.
To make this easier to work with I’ve taken the steps above and packaged them up into a handle utility function which takes in a list and applies the change to all five of the email fields. Might be worth adding some error handling to make sure the fields exist and whatnot but I was only ever running this on lists where I knew for a fact that the fields were present - you could probably wrap this in a simple function which searches for all email enabled lists and makes the changes globally but personally I prefer to do things like this one list at a time so I’m very deliberate about what I’m doing and what I’m migrating (again, this only applies if you’re migrating a single list and not an entire site collection in which case Sharegate will migrate the fields successfully).
1<#
2.Synopsis
3 Change the Group property of E-Mail fields from _Hidden to Custom to allow Sharegate to migrate.
4.DESCRIPTION
5 Change the Group property of E-Mail fields from _Hidden to Custom to allow Sharegate to migrate.
6 The following fields will be updated by changing the field to not be Sealed, setting the Group property, and then resetting to Sealed:
7 "E-Mail From", "E-Mail To", "E-Mail Cc", "E-Mail Headers", "E-Mail Sender", "E-Mail Subject"
8.EXAMPLE
9 Set-SPEmailEnabledFieldsGroup -Web "https://example/" -ListTitle "Email Enabled Library"
10.EXAMPLE
11 Set-SPEmailEnabledFieldsGroup -List (Get-SPWeb "https://example/").Lists["Email Enabled Library"]
12#>
13function Set-SPEmailEnabledFieldsGroup
14{
15 [CmdletBinding()]
16 [OutputType([int])]
17 Param
18 (
19 [Parameter(Mandatory=$true, Position=0, ParameterSetName="SPList", ValueFromPipeline=$true)]
20 [ValidateNotNull()]
21 [Microsoft.SharePoint.SPList]$List,
22
23 [Parameter(Mandatory=$true, Position=0, ParameterSetName="SPWeb")]
24 [ValidateNotNull()]
25 [Microsoft.SharePoint.PowerShell.SPWebPipeBind]$Web,
26
27 [Parameter(Mandatory=$true, Position=1, ParameterSetName="SPWeb")]
28 [ValidateNotNull()]
29 [string]$ListTitle
30 )
31
32 Process
33 {
34 switch ($PsCmdlet.ParameterSetName) {
35 "SPWeb" {
36 $spWeb = $Web.Read()
37 $spList = $spWeb.Lists[$ListTitle]
38 try {
39 if ($spList -eq $null) { throw "Unable to locate a list with the title `"$ListTitle`"" }
40 Set-SPEmailEnabledFieldsGroup -List $spList
41 } finally {
42 $spWeb.Dispose()
43 }
44 return
45 }
46 }
47
48 $fields = @("E-Mail From", "E-Mail To", "E-Mail Cc", "E-Mail Headers", "E-Mail Sender", "E-Mail Subject")
49 foreach ($fieldName in $fields) {
50 Write-Host "[$(Get-Date)] Updating field `"$fieldName`"..."
51 $field = $List.Fields[$fieldName]
52
53 # Get the MethodInfo object for the SetFieldBoolValue property
54 $setFieldBoolValue = $field.GetType().GetMethod("SetFieldBoolValue", [System.Reflection.BindingFlags]::NonPublic -bor [System.Reflection.BindingFlags]::Public -bor [System.Reflection.BindingFlags]::Instance -bor [System.Reflection.BindingFlags]::InvokeMethod)
55
56 # Get the MethodInfo object for the UpdateCore property
57 $updateCore = $field.GetType().GetMethod("UpdateCore", [System.Reflection.BindingFlags]::NonPublic -bor [System.Reflection.BindingFlags]::Public -bor [System.Reflection.BindingFlags]::Instance -bor [System.Reflection.BindingFlags]::InvokeMethod)
58
59 # Call the Invoke() method to set the Sealed property to False
60 [void]$setFieldBoolValue.Invoke($field, @("Sealed", $false))
61 $field.Group = "Custom"
62
63 # Call the Invoke() method to update the properties (pass in True to force the Sealed property to update)
64 [void]$updateCore.Invoke($field, @($true))
65
66 # Call the Invoke() method to set the Sealed property back to True
67 [void]$setFieldBoolValue.Invoke($field, @("Sealed", $true))
68
69 # Call the Invoke() method to update the properties (pass in True to force the Sealed property to update)
70 [void]$updateCore.Invoke($field, @($true))
71 }
72 }
73}
If for some reason all you care about is getting the fields added to the target list but you’re not concerned about migrating the data then you could achieve this by migrating the structure first and then using a little bit of PnP PowerShell to add the missing fields to the list:
1$conn = Connect-PnPOnline -Url "https://<tenant>.sharepoint.com/" -UseWebLogin
2$fields = @("E-Mail From", "E-Mail To", "E-Mail Cc", "E-Mail Headers", "E-Mail Sender", "E-Mail Subject")
3$list = Get-PnPList -Identity "My List" -ThrowExceptionIfListNotFound -Connection $conn
4foreach ($field in $fields) {
5 $f = Get-PnPField -Identity $field -InSiteHierarchy -Connection $conn
6 Add-PnPField -List $list -Field $f -Connection $conn
7}
Again, this won’t help you with getting the content moved over as Sharegate will still not see the fields so it won’t copy the data. I ran into a need for this with one client where we used a “dev” site as a staging area for the structure while we reworked the workflows associated with the list so we went from on-prem to a dev site in SharePoint Online to the production site in SharePoint Online but when going from SPO to SPO there’s no way to get the fields copied so rather than do another structural migration from on-prem to the production site and risk overwritten field schema changes and workflows we had to make we just ran this little bit of script to add the missing fields and then did a content only migration from on-prem to the production online site. A bit convulated in my opinion but it achieved the goal.