Skip to content

Make HTTP REST JSON Requests

Info

This how-to assumes you are familiar with setting up a project either manually or via VoltScript Dependency Management. If you are not, follow the Intro to VoltScript Tutorials.

Note

For specific details of APIs, see API Docs

Introduction

Although REST doesn't have to be over HTTP or transfer content as JSON, typically this is the case. This how-to covers this typical use case and requires the use of two VoltScript Extensions, which are WebVSE and JsonVSE.

VoltScript dependencies

Incorporating WebVSE and JsonVSE is straightforward. You just need to add the following JSON objects to the vsesDependencies element in your atlas.json.

        "WebVSE": {
            "library": "25286",
            "version": "1.0.1",
            "module": "25888/20853",
            "repository":"volt-mx-marketplace"
        },
        "JsonVSE": {
            "library": "25265",
            "version": "1.0.1",
            "module": "25877/20841",
            "repository":"volt-mx-marketplace"
        }

You'll need to add to your repositories object in the atlas.json of your project:

        {
            "id": "volt-mx-marketplace",
            "type": "marketplace",
            "url": "https://community.demo-hclvoltmx.com/marketplace/asset"
        }

To use the extension in your script, enter UseVSE "*JsonVSE" and UseVSE "*WebVSE".

WebServer object

The WebServer is the entrypoint for HTTP requests via WebVSE and is instantiated using the New constructor. Unlike other internally-focussed extensions (KeepVSE and CouchVSE), WebServer permits defining proxy settings as properties. Proxy servers aren't currently supported, VPN access is recommended.

If making multiple requests to the same base URL, you can re-use the same WebServer instance.

Private server as WebServer

Function getWebServer() as WebServer
    If (server is Nothing) Then
        Set server = new WebServer()
        server.protocol = "HTTPS"
        server.HostUrl = "httpstat.us"
    End If
    Return server
End Function

The HostURL shouldn't include the protocol and may just be the hostname of the server (for example "google.com") or may be to a more specific URL (for example "api.github.com/orgs/hcl-tech-software"). The HostURL shouldn't end with a path separator ("/").

SSL Certificate Handling

To use HTTPS, the web-based extensions expect a properly verifiable SSL certificate. For well-known sites, these may be loaded by your operating system.

For sites with self-signed certificates WebVSE has an option to disable this with WebServer.SSLHostValidationOn = False. This is equivalent to the libcurl command curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE); or the curl command line option -k/--insecure.

It's the responsibility of the systems administrator to ensure the relevant self-signed certificates are in the relevant cert store or, for Windows, appended to the curl-ca-bundle.crt in VoltScript executable directory.

Alternatively, WebServer.certificatePath property can be passed a path to a specific self-signed certificate.

Cookies

Although REST APIs are intended to be stateless, certain web interactions may require persisting of cookies between requests. This can be done by setting PreserveCookies=True.

Encoding URL content

When including content from variables, for example for including in querystring parameters, it can include invalid content, such as " ", "/". The receiving web server will require these values to be URL encoded. The WebServer class provides utility functions to URL encode and decode strings - WebServer.URLencode() and WebServer.URLdecode(), each taking a string to encode / decode.

WebRequest

The WebRequest class handles a single specific request to a specific endpoint on a REST API or web server. The key parts of a web request are target URL, a verb (e.g. "GET", "POST"), one or more HTTP headers, cookies, potentially a request body and timeout.

Basic request

The default verb for a request is "GET". So the most basic request will just need a target URL. Defining timeoutSeconds is recommended for best practice. This will be appended, after a "/", to the WebServer's HostURL.

Function makeBasicRequest() as WebResponse
    Dim request as WebRequest

    Set request = getWebServer().createRequest()
    request.target = "200"
    request.timeoutSeconds = 5
    Return request.send()
End Function

The request is sent to:

Property Value
WebRequest.Verb GET (default value)
WebServer.Protocol HTTPS
WebServer.HostURL httpstat.us
/
WebRequest.Target 418

The full request is GET https://httpstat.us/418, equivalent to the curl command:

curl --location 'https://httpstat.us/418'

HTTP headers

There are helper properties for common headers:

  • ContentType for data type of data being sent, for example "application/json"
  • AcceptHeader for expected return data type, for example "application/json"
  • AuthorizationHeader for credentials beginning with type, for example "Basic ", "Bearer ".

Additional headers can be added through AddHeader().

Content can be sent for PUT, PATCH, or POST requests via RequestBody property.

A sample request for a POST, with JSON body content and custom headers would be:

Function makeJsonRequest() as WebResponse
    Dim request as WebRequest

    Set request = getWebServer().createRequest()
    request.Verb = "PUT"
    request.ContentType = "application/json"
    request.AcceptHeader = "application/json"
    Call request.addHeader("X-HttpStatus-Response-Foo", "Bar")
    request.target = 418
    request.timeoutSeconds = 5
    request.RequestBody = |{"hello":"world"}|
    Return request.send()
End Function

Encode body content

In some cases, for example when sending images or sending basic authentication credentials, content may need to be Base64 encoded. This can be done via the utility function WebRequest.Base64Encode(), which will return the encoded value.

File uploading and downloading

File uploading and downloading can be done synchronously or asynchronously. When performed synchronously, it returns a WebResponse object corresponding to the response. When performed asynchronously, it returns a Completion object.

Completion objects have an IsComplete property which can be polled periodically, for example in a loop with a Sleep function. It also has a CompletionCode property for determining success or failure.

Once completed it also has a WebResponse object to check the response.

Note

A file will be created at the destination path regardless of the HTTP response code. If the HTTP response code is not 200, the file will contain the response information, and can be checked for troubleshooting purposes. If you are subsequently processing the file, you may wish to check WebResponse,ResponseCode and move / delete the file accordingly.

Create a WebRequest with URLBuilder

Building the URL manually from strings may not be a preferable approach. The URLBuilder class provides functionality to build a URL and pass it to a WebServer.

Function makeRequestFromUrlBuilder() as WebResponse
    Dim server as New WebServer()
    Dim request as WebRequest
    Dim builder as New UrlBuilder()

    builder.isHttps = True
    builder.targetHost = "httpstat.us"
    builder.targetPath = "200"
    Call builder.addUrlParameter("sleep", "1000")
    Set request = server.createRequestWithUrl(builder.toString(False)) '(1)!
    request.timeoutSeconds = 5
    Return request.send()
End Function
  1. toString() parameter is for whether or not to URL encode the URL.

Note

When using URLBuilder.toString() with multiple URL parameters, they're outputted in alphabetical order.

WebResponse

The WebResponse object is the result of a WebRequest.send() or file upload / download. It contains all information relating to the web server's response to your request. ContentType contains the data type of data received, for example "application/json". ContentLength can be checked for the size of the response as a Long in bytes. AllHeaders and AllCookies properties give string arrays of headers and cookies.

Query response code

WebResponse.ResponseCode gives the HTTP response code. These fall into five groups:

  • 1xx informational response while additional processing continues.
  • 2xx successful - the request was successfully received, understood, and accepted.
  • 3xx redirection - additional action needs to be taken to complete the request.
  • 4xx client error - you have sent incorrect or insufficient information, your request needs amending.
  • 5xx server error - the recipient has encountered some unexpected error. This may include unanticipated bad data in your request.

It's recommended to test the response code and act accordingly:

Function validateResponseCode(responseCode as Integer) as Boolean
    Dim request as WebRequest
    Dim response as WebResponse

    Set request = getWebServer().createRequest()
    request.target = "418"
    Set response = request.send()
    Return response.responseCode = 200
End Function

Process JSON response

ContentBody is always a string. In some cases, this may be a string of JSON. It's always recommended to verify. But such content can be easily parsed using JsonVSE.

Function parseResponse(json as String) as JsonObject
    Dim parser as New JsonParser()
    Dim obj as JsonObject

    Call parser.loadFromJSON(json)
    Return parser.getRootObject
End Function

Note

The sending server may encode characters, for example apostrophe as "\u0027". Loading into a JSON object will convert this. If in doubt about the response, cross-reference with Postman checking View > Postman Console or Raw tab in the UI. The "Pretty" tab does the same conversion that JsonVSE will do.

Decode and request body content

In some cases, for example when receiving images or parsing cookies/headers, values may be Base64 encoded. These can be decoded via the utility function WebResponse.Base64Decode(string), which will return the decoded value.

Troubleshoot requests

Using the web-based extensions will generate a libcurl.log in the current working directory from which the script is run. This is analogous to setting the notes.ini variable Debug_NotesHTTPRequest=1.

Note

Bear in mind this will contain credentials, so should be sanitized before sharing with third parties.

The complete implementations of the code snippets are available on GitHub.