HTTP Signature Authentication using CData Arc

This article provides a brief overview of how to use HTTP Signature Authentication with CData Arc and will use CyberSource as an example of that integration.

Overview of APIs

The acronym API stands for "Application Programming Interface." This may sound complicated, but what it really boils down to is that an API is a software that allows two different entities to communicate with one another. Since their development, APIs have quickly become adopted as a common method of communication used by businesses, industries and services across the globe.

Knowing this, CData Arc has been built with APIs in mind. Using dedicated connectors within the application, users can design and create flows that send and receive data from public API endpoints or REST based APIs. In practice, most - if not all - RESTful APIs currently use HTTP as a transport layer, since the infrastructure, servers and client libraries for HTTP are already widely used and available.

These flows can be accomplished by using a few of Arc's many dedicated connectors. These connectors allow users to specify which API endpoint they wish to target in order to POST some JSON data or to submit a GET request to receive a particular report. However, it can become a little tricky when an authentication method such as HTTP Signature Authentication is required by the API. Sending a proper request in Arc using this method requires a bit more than just drag, drop and go, as will be explained in more detail later on.

HTTP Signature Authentication. What is it and how is it used?

Commercial APIs typically require some form of security so that not just anyone can access the data that they can provide. The most common approach for protecting APIs is by using shared secret keys. Every time the API is called, this secret key must be presented. However, this method does not provide a way to ensure the integrity of the data being transferred. In these scenarios, HTTP Signature Authentication allows for an increase in the security of the APIs.

HTTP Signature Authentication is used commonly in two different ways.

• Allow a sender to request access to a resource by proving that they possess a secret key.
• Allow a sender to confirm the contents of the message sent are correct and have not been altered during transmission or storage.

Take the CyberSource API as an example here. The API documentation requires that REST HTTP requests must contain a Signature request header that you send to CyberSource. This signature is encrypted by the sender and decrypted when the API receives it to make sure that the user sending the request has access to the specified resource.

Within Arc, this can be accomplished by using the REST Connector along with a Script Connector that contains a custom script that generates the required headers and encryption for the HTTP Signature. The Script Connector then sends those message headers to the REST Connector which then uses those headers, with the Signature included, to process the REST request.

The REST Connector

The REST Connector allows users to send or receive data with trading partners using the HTTP protocol, and operates very similarly to the popular application, Postman. Both users and their client can establish a public API endpoint where both parties can either POST or GET data from that API.

When dragging and dropping a REST Connector into the workflow, the settings page will open and display something that looks like this.

Here is where details should be provided about the API that is to be interacted with, such as the request method and unique API endpoint URL, Authentication as well as any custom headers that are required. Server Authentication credentials can also be provided if that is something that the API requires.

However, what if the API requires the use of HTTP Signature Authentication in all requests? Enter the example, CyberSource, whose API requires that all requests be sent using HTTP Signature Authentication.

Using HTTP Signature Authentication within the REST Connector is a little bit more involved than just creating your message headers within the connector settings, providing a URL and then submitting the request. The main difference is how to get the encrypted signature string passed as a message header within the REST Connector. This is where the custom script and Script Connector come into play.

CyberSource API

The CyberSource API is a very large business API developed for the company that operates as an E-commerce credit card payment system management company. Their API allows users to do things such as process payments, manage invoicing and in this use case, reporting.

All API references made within this article can be found in the CyberSource API documentation which is located here.

If using this article to implement the CyberSource API with Arc, please take this time to make sure you have all of the necessary credentials and keys created and in order.

The Script Connector

With an understanding of the requirements of CyberSource and how Arc can handle them, it is time to introduce the Script Connector - the brains behind this operation. This connector allows you to generate the signature string and pass it as a message header to the REST Connector in the flow. At a high level, the process within the Script Connector looks something like this:

• Create a byte array of the decoded Secret Key (generated in the CyberSource Business Center).
• Create an HMACSHA256 object that is based on the decoded Secret Key byte array then use this HMACSHA256 object to compute an HMACSHA256 hash that is based on the string byte array.
• Finally, generate a base-64 encoded string from the byte array of the HMACSHA256 object from the previous step.

What makes this tricky is that the encrypted signature string, along with key-id, signing algorithm and header names compose the actual value of the signature header. However, all of this can be accomplished using ArcScript within a Script Connector.

Example Flow

Having covered the general API information, HTTP signatures and how Arc can be used in this process, now it is time to turn to the sample HTTP Signature Authentication Flow using the CyberSource API. It is strongly recommended to install this Flow before continuing.

The sample Flow consists of a Script Connector and a REST Connector.

Breaking it Down

The CyberSource API requires specific HTTP headers to be passed with requests. These headers are:

• v-c-merchant-id
• Date
• Host
• Signature

The v-c-merchant-id, Date and Host are straight forward. However, the Signature header is where it becomes a little more involved.

The Signature header is made up of multiple name-value pairs or parameters, consisting of the keyid, algorithm, headers and signature. The headers name-value pair is simply just a string of all the required headers while the signature name-value pair is the signature hash, which is a base-64 encoded hash of the header fields and their values.

By concatenating these HTTP Headers together to form a string and, obtaining an encrypted hash over that string using a symmetric algorithm (hmac-sha256) with a shared secret key, the digital Signature for the HTTP request is produced. Finally, the encrypted bytes are base-64 encoded and the signature string is obtained.

The below code block shows an example of what a generated HTTP signature header looks like when using the CyberSource API. As mentioned above, the Signature header is made up of multiple name-value pairs with the signature name-value pair containing the above-mentioned base-64 encoded signature string.

v-c-merchant-id: mymerchantid
Date: Fri, 12 Jul 201900:44:13 GMT
HOst: apitest.cybersource.com
Signature: keyid="6d75ffad-ed36-4a6d-85af-5609185494f4", algorithm="HmacSHA256", headers="host date (request-target) v-c-merchant-id", signature="eQkzhzu8UHEQmpBTVMibdNpPw1aLunmY41ckyLKoOjs="

The Script

With an understanding of the Script Connector and how it can utilize HTTP Signature Authentication, it is time to dig deeper into the script itself and understand what is happening during each step of the scripting process.

The HTTP Signature Authentication script that was designed for the CyberSource API can be broken down into four main parts.

• Decoding the Encoded Secret Key
• Staging the Hex Key as a Base-64 Encoded HMACSHA256 Hash
• Setting the Message Headers
• Generating the Signature Hash

The subsequent four sections will go into more detail about each one of these steps.

Decode the Encoded Secret Key

The first step within the script is to set the decoded secret key, obtained from the CyberSource Business Center, as an item within the script. That encoded secret key is then decoded into a hex value, using the encDecode operation.

When using the encEncode or encDecode operations, input and output format can be specified using the "informat" and an "outformat" attributes. The default informat is base64, which is what the original secret key is encoded with here, so there is no need to specify the informat in this case. The outformat however, has been set to "HEX".

This operation, in this case, also requires an "in" item and then a specified "out" item. This refers to what item is being passed into the operation and the desired name of the output item once the operation has finished processing it.

The final step in this process is to set the decoded secret key (now as a hex value) to a global item so that it can be made available anywhere within the scope of the script. The hex value must be prefixed with "0x" to tell the parsing engine that it is dealing with a constant, not an identifier or reserved word. This hex value is then later used to create a base-64 encoded HMACSHA256 hash.

<!-- encoded secret key in base64 encoding to HEX --> 
<arc:set attr="encodedKey.data" value="JIVAFb/fO0WmocDuc3EvSjNiye7tif/aj+STWdFi/sU="/> 
<arc:set attr="encodedKey.outformat" value="HEX"/> 
<!-- setting decodedGlobal and setting value to hex key to be available globally --> 
<arc:set attr="decodedGlobal.data" value="" /> 
<arc:call op="encDecode" in="encodedKey" out="decodedKey"> 
<arc:set attr="decodedGlobal.data" value="0x[decodedKey.decodeddata]" /> 
</arc:call>

Staging the Encoded Secret Key as a Base-64 Encoded HMACSHA256 Hash

The next step within the script is a preparation for a later step. At the end of the script, the string of each header field name and its associated value along with the hex key must be encoded to base-64. In this step the parameters of the input data that is going to be encoded (encIn), must be set. The input format, hmackey, algorithm, number of bits and the out format must be specified here.

This data will then later be processed by the encEncode operation where it will be encoded in Base64.

<!-- encoding the hex key to HMACSHA256 --> 
<arc:set attr="encIn.format" value="HMAC" /> 
<arc:set attr="encIn.hmackey" value="[decodedGlobal.data]" /> 
<arc:set attr="encIn.hmacalgorithm" value="SHA" /> 
<arc:set attr="encIn.hmacbits" value="256" /> 
<arc:set attr="encIn.outformat" value="BASE64" />

Setting message headers

With the decoded secret key and the encoding parameters configured, it is now time to add the message headers to the message that is being created in the Script Connector so that they will be able to be evaluated in the REST Connector.

One thing to note, the API key is defined in this section of the script, however it can also be defined in any previous step since it will not be used until the final section.

Using the arc:set keyword here, the required headers such as MerchantID, HeaderDate and HeaderHost can be defined.

<!-- API secret key. this is the secret key in UUID format --> 
<arc:set attr="apiSecret.key" value="b84ba2d7-1a4b-4814-b757-2f747ccab086" />  
<arc:set attr="Report.Date" value=[_|utcnow(yyyy-MM-dd)] />

<!-- Setting custom headers. Make sure you replace the merchant ID with your ID --> 
<arc:set attr="Fileload.header:MerchantID" value="nsoft_test1" /> 
<arc:set attr="Fileload.header:myCustomHeader" value="?organizationId=[Fileload.header:MerchantID]&reportDate=[Report.Date]&reportName=test" /> 
<arc:set attr="Fileload.header:HeaderDate" value= [_|date(r)] /> 
<arc:set attr="Fileload.header:HeaderHost" value="apitest.cybersource.com" />

There is also a myCustomHeader defined here that contains the MerchantID, Report Date and a Report Name. This header is created so that it can be referenced in the URL of the REST Connector as part of the query string, like this:

https://apitest.cybersource.com/reporting/v3/report-downloads[_message.header:MyCustomHeader]

Once these headers have been defined, they can be referenced in the REST Connector by using the _message.header syntax within the header value fields.

Generating the Signature Hash

The final step is creating the signature hash. This is what is used to verify the authenticity of the message contents or the entity submitting the request. As was mentioned earlier, the digital signature is obtained by concatenating all the HTTP headers together to form a string. An encrypted hash is then obtained over that string, with the decoded secret key, using the hmac-sha256 algorithm that was setup in the second step. Finally, everything is base-64 encoded, thus producing the signature string.

This process can be broken up into two different parts:

• Setting and concatenating the headers
• Encoding the header string to produce the signature string

Using the arc:set keyword, concatenate all the headers and their values together as a string. In this case, that is being done and set as the data attribute for the encIn item (the item to be encoded). In this step, with the CyberSource API in particular, the header fields must be placed in the same order as they were passed in the message header.

One thing to note here is that there is a header included called "request-target". The request-target header value is the HTTP verb in lowercase followed by a space, then the resource path (minus the host). The following example shows a GET request to the "/reporting/v3/report-downloads" resource. Include query strings and request IDs in the request-target value. In this case, the query strings have already been defined within the myCustomHeader value.

(request-target): get /reporting/v3/report-downloads[Fileload.header:myCustomHeader]

It is also important in this case to include a new line for each individual header, however do not include any line break characters (\n) at the end of the string.

<arc:set attr="encIn.data">host: [Fileload.header:HeaderHost] 
(request-target): get /reporting/v3/report-downloads[Fileload.header:myCustomHeader] 
date: [Fileload.header:HeaderDate] 
v-c-merchant-id: [Fileload.header:MerchantID]</arc:set>

This block accomplishes the final two processes within the script. First, the encEncode operation is called and all of the data that was defined earlier as encIn, is passed in as the input parameter of the operation. The output, defined as encOut in this case, contains all the input data but now it is base64 encoded.

The arc:set keyword is then used to create the signature header that consists of the keyid, algorithm, headers and the signature. In order to access the encoded data from the encEncode operation, the "encodeddata" attribute of the output item must be referenced.

In this case, the signature header is set to [encOut.encodeddata] which evaluates to the actual encoded signature string.

A successful signature header should look something like this:

Signature: keyid="6d75ffad-ed36-4a6d-85af-5609185494f4", algorithm="HmacSHA256", headers="host date (request-target) 
v-c-merchant-id", signature="eQkzhzu8UHEQmpBTVMibdNpPw1aLunmY41ckyLKoOjs="

<!-- generating signature hash --> 
<arc:call op="encEncode" in="encIn" out="encOut"> 
<arc:set attr="Fileload.header:signature">keyid="[apiSecret.key]", algorithm="HmacSHA256", headers="host (request-target) date v-c-merchant-id", signature="[encOut.encodeddata]"</arc:set>  
</arc:call> 

<!-- push the output --> 
<arc:push item="Fileload" />

Finally, the entire item, containing all the headers and signature is pushed out of the Script Connector using the arc:push keyword. This file is then sent to the REST Connector where the HTTP request can be issued, which in this case is a GET request.

Conclusion

The goal of this article is to aide in the understand of how Arc is built around APIs and how it can integrate with them using pre-built connectors and custom scripting in addition to how you can use Arc to adhere to specific authentication requirements of an API such as HTTP Signature Authentication.

This script contains ArcScript that has been developed to integrate specifically with the CyberSource API, however we have provided it in a way that it can be easily manipulated to fit many other use cases that might require HTTP Signature Authentication.

More information about the Script Connector, that is provided here. If you are looking for more information regarding the REST Connector, you can view that here.

If you would like a more in-depth review of ArcScript and the fundamentals of how our scripting system works, we have written an ArcScript QuickStart Guide that can be found here.