Update 10/25/2017: This article originally appeared on ITUnity.com but as that site has been decommissioned I’ve republished it here.
In the article, Using the SharePoint REST Service with Windows PowerShell, I detailed how to use Windows PowerShell to make simple RESTful calls against your SharePoint Online Site Collection. In this article I want to build upon the example that I showed by abstracting out all of the various settings into a common utility function that you can use to make all of your REST service calls. This utility function, which I’ll name Invoke-SPORestMethod, will include parameters that will allow us to set all the necessary details required for the examples that I will discuss in the next article, Working with Lists and List Items using the SharePoint REST Service and Windows PowerShell.
Managing Credentials
Before creating the main function it might be helpful to create a couple utility functions to help with the management of your credentials. You have to provide the credentials every time you make a RESTful call so if can store the credentials in a global variable, you can have the main function check for that whenever it’s executed. In the following code snippet I’ve created two functions: Set-SPORestCredentials and Clear-SPORestCredentials (these functions are defined as global so that you don’t have to dot-source your script when loading).
In the Set-SPORestCredentials cmdlet I’m making sure that you have a valid credential object save and I’m also making sure that the Microsoft SharePoint Online Management Shell has been installed by looking for the presence of the associated PowerShell Module. This is necessary because I’m using the Microsoft.SharePoint.Client.SharePointOnlineCredentials object to avoid having to deal with getting the security token and cookies. In figure 1 I’ve loaded the functions into memory and then execute the Set-SPORestCredentials function, which in turn prompts me for credentials because I did not provide any via the -Credentials parameter:
Figure 1: Use the Set-SPORestCredentials custom function to store your SharePoint Online credentials.
If you no longer wish to have the credentials stored in the global variable, you can run the Clear-SPORestCredentials function and the variable will be set to null (or, of course, you can just explicitly set the variable directly).
The Invoke-SPORestMethod function
The main function, Invoke-SPORestMethod, is considerably more complex than these two little helper functions. For the Invoke-SPORestMethod function, I’ve tried to account for as many scenarios as possible in terms of the operations you might complete against your SharePoint Online tenant. To this end, you’ll find numerous parameters (eight to be precise) that you can provide the function and several bits of code to deal with things such as JSON to PowerShell incompatibilities and saving binary objects versus output JSON data.
The following code snippet shows the complete Invoke-SPORestMethod. Skim through it and then I’ll walk you through a couple of the key points.
To better understand what this function is doing, let’s pick it apart parameter by parameter. In the following sections I’ll describe the purpose of each parameter and then show the snippet of code that relates to the parameter.
Url parameter
The URL parameter represents the actual REST endpoint that you are targeting. The expected type is System.Uri but because PowerShell is able to convert a String value to this type, you don’t have worry about creating an explicit System.Uri object. The Url parameter is used when creating the System.Net.WebRequest object as shown in the previous article:
1$request = [System.Net.WebRequest]::Create($Url)
Method parameter
The Method parameter allows you to specify the HTTP verb to use when completing the web request. The default value is Get. Valid values are Get, Head, Post, Put, Delete, Trace, Options, Merge and Patch. When setting the System.Net.WebRequest object’s Method property I’ve found that casing matters so I make sure that the value is always uppercased:
1$request.Method = $Method.ToUpper()
Metadata parameter
The Metadata parameter is basically the data to include as part of the body of the request (I could have just as easily called this parameter Body). You’ll typically use this when you wish to add or update an existing artifact such as when creating a new list or uploading a file. To set the Metadata parameter as the body of the request, we must convert the value to a byte array if it isn’t one already. The byte array is then written to the request stream after setting the content length of the request. In the snippet below, I’m first checking if the data is a string and if so then I convert it to a byte array; otherwise, if it’s already a byte array then I use what was provided:
1if ($Metadata -is [string] -and ![string]::IsNullOrEmpty($Metadata)) {
2 $body = [System.Text.Encoding]::UTF8.GetBytes($Metadata)
3 $request.ContentLength = $body.Length
4 $stream = $request.GetRequestStream()
5 $stream.Write($body, 0, $body.Length)
6} elseif ($Metadata -is [byte[]] -and $Metadata.Count -gt 0) {
7 $request.ContentLength = $Metadata.Length
8 $stream = $request.GetRequestStream()
9 $stream.Write($Metadata, 0, $Metadata.Length)
10} else {
11 $request.ContentLength = 0
12}
RequestDigest parameter
When completing tasks that make changes it is required that you provide a form digest string. Use of this string improves the security of the request. When making REST calls from within a browser context getting this form digest variable is easy as it’s stored in a hidden input field within the page. But from PowerShell you’ll have to make a REST call to retrieve it. You can use the Invoke-SPORestMethod as follows to retrieve the form digest:
1$formDigest = (Invoke-SPORestMethod -Url "https://contoso.sharepoint.com/_api/contextinfo" -Method "Post").GetContextWebInformation.FormDigestValue
To include the form digest value as part of the request you have to set the X-RequestDigest header value:
1if (![string]::IsNullOrEmpty($RequestDigest)) {
2 $request.Headers.Add("X-RequestDigest", $RequestDigest)
3}
ETag parameter
When updating items you want to make sure that other users have not changed the item since you first retrieved it. The ETag parameter allows you to specify the ETag value you received when you retrieved the original item. If the item you are updating has a different value on the server then what you are submitting then that means that someone else has modified the item since you last retrieved it and your attempted update will fail. You can override this behavior by providing an asterisks (*) for the value. To include the ETag value as part of the request you have to set the If-Match header value:
1if (![string]::IsNullOrEmpty($ETag)) {
2 $request.Headers.Add("If-Match", $ETag)
3}
XHTTPMethod parameter
To work around the fact that some network firewalls block HTTP verbs other than GET and POST, you can use the XHTTPMethod parameter to provide the intended verb to use. Valid values are the same as those for the Method parameter. To include the XHTTPMethod value as part of the request, you have to set the X-HTTP-Method header value:
1if ($XHTTPMethod -ne $null) {
2 $request.Headers.Add("X-HTTP-Method", $XHTTPMethod.ToUpper())
3}
Like the Method parameter, I’ve found that case matters so you want to make sure you uppercase the value so the caller doesn’t have to worry about such details.
JSONVerbosity parameter
The JSONVerbosity parameter allows you to indicate how much metadata you wish included with the response. Valid values are Verbose, MinimalMetadata, and NoMetadata. There are two places where this is important – the first place is when you set up the request as you want to make sure that the Accept and ContentType properties are properly set:
1$odata = ";odata=$($JSONVerbosity.ToLower())"
2$request.Accept = "application/json$odata"
3$request.ContentType = "application/json$odata"
The second place where the JSONVerbosity parameter is used is when looking at the return value so that you can output the right information to the caller:
1if ($JSONVerbosity -ne "verbose") {
2 Write-Output $results
3} else {
4 Write-Output $results.d
5}
While I’m covering JSON output, one thing worth noting is the lines within the function right before converting the JSON data to a PSCustomObject:
1if ($data.Contains("`"ID`":") -and $data.Contains("`"Id`":")) {
2 $data = $data.Replace("`"ID`":", "`"ID-dup`":");
3}
4
5$results = ConvertFrom-Json -InputObject $data
There are several cases, mostly when working with list items, when a JSON data structure is returned with two ID properties, one as “ID” and the other as “Id”. While having two properties with the same name but different casing is valid with languages like C# and JavaScript, it is not valid with PowerShell which is, for the most part, case agnostic. So if you were to try and convert the JSON data to a PSCustomObject without addressing this issue then you would receive an error about duplicate key entries. So this little snippet of code just does a brute force replace on the JSON data string to change the duplicate value to “ID-dup”. This approach isn’t by any stretch a perfect solution but it will handle the majority of the cases without issue.
OutFile parameter
If the response contains a binary stream you can specify this parameter to have the response saved to the file path you specify. To know if the response contains a binary file I’m simply checking to see if the content type of the response equals “application/octet-stream.” If it does and if a value for the OutFile parameter was specified then I save the stream to the specified file. If no OutFile parameter value is specified than I simply output the file to the caller of the function:
1if ($response.ContentType.Contains("application/octet-stream")) {
2 if (![string]::IsNullOrEmpty($OutFile)) {
3 $fs = [System.IO.File]::Create($OutFile)
4 try {
5 $streamReader.BaseStream.CopyTo($fs)
6 } finally {
7 $fs.Dispose()
8 }
9 return
10 }
11 Write-Output $streamReader.ReadToEnd()
12 return
13}
Example
With the Invoke-SPORestMethod function you can now make simple or complex calls to the REST service with very little effort. In the following screenshot I’m retrieving the details about the Shared Documents document library within my site. Outputting the returned object allows you to see that you can work with the returned data just as though you were working with a typical structured object:
Figure 2: Use the Invoke-SPORestMethod function to execute RESTful queries against your SharePoint Online tenant.
I won’t bother with other examples here as that is the focus of the next article, Working with Lists and List Items using the SharePoint REST Service and Windows PowerShell.
Bonus Section – An alternate Invoke-SPORestMethod function
In this article and the previous article, Using the SharePoint REST Service with Windows PowerShell, I mentioned that by using the REST Service you can eliminate the dependency on things like the SharePointOnlineCredentials class and then I proceeded to create several examples, all of which were dependent on this particular class. Of course, I did mention in the previous article that doing so saves us a lot of extra work so now I want to show you a reworked version of the Invoke-SPORestMethod function but this time without the dependency. I’m not going to detail all the code as I’m only including this as an academic example and for those that, for one reason or another, just absolutely need pure implementation with no dependencies.
In the code above, notice that the Set-SPOCredentials function is now simplified so that all it is doing is using the Get-Credential cmdlet to store credentials via the $global:spoCred variable – again, I’m not using the SharePointOnlineCredentials class so there’s no need to do more than this.
The next change is the important part – I’ve now got two functions internal to the Invoke-SPORestMethod function (they’re internal to the function so that you can’t call them outside of the function). The first function, Get-SPOSecurityToken, takes in the credentials you provided and uses them to construct a SOAP-based web request to the https://login.microsoftonline.com/rst2.srf endpoint. The response contains a security token which basically validates that the credentials provided were in fact valid. This security token is then passed to the Get-SPOSecurityCookie function where it is used to make a call to the https://[site]/_vti_bin/idcrl.svc endpoint. The response from the call contains a cookie that you can then use when making the REST request. Notice that as part of the request, I’m no longer setting the Credential property of the WebRequest object and am instead adding the cookie to the request:
1$token = Get-SPOSecurityToken $global:spoCred
2$cookieValue = Get-SPOSecurityCookie $Url $token
3$cookieContainer = New-Object System.Net.CookieContainer
4$uri = [uri]$Url
5$cookie = New-Object System.Net.Cookie "SPOIDCRL", $cookieValue.Substring(9), "", $uri.Authority
6$cookieContainer.Add($cookie)
7$request.CookieContainer = $cookieContainer
One thing to note is that you don’t have to re-authenticate and retrieve a new cookie on every request as I’m doing in this example. Ideally you’d want to store the cookie in a variable and only retrieve a new cookie if the current one is expired. I didn’t include that code because the function is already pretty long and my intention was simply to show you the core pieces required to remove the SharePointOnlineCredentials dependency.
Summary
In this article I detailed a custom Windows PowerShell function that you can utilize to make working with the SharePoint REST Service and Windows PowerShell much easier and more repeatable. I definitely recommend using the first version of the function as it abstracts away all of the security management by leveraging the SharePointOnlineCredentials class; however, if you have a particular need where you simply cannot use this class, then the alternate version should get you going in the right direction. In the next article, Working with Lists and List Items using the SharePoint REST Service and Windows PowerShell, I will use the Invoke-SPORestMethod function extensively so having this function available in your PowerShell toolbox will definitely be advantageous if you’re following along with the article series.