Explore REST Capabilities for RESTful Workflows

Workflows are powerful as they can not only invoke REST services but also be deployed as REST Services on a KNIME Server. Workflows to be tailored to almost any of our REST based needs and make the KNIME stack a powerful integration tool. We can use them to enhance existing third party REST endpoints, create brand new REST services to score models, or expose several systems as one REST interface. We can even use these workflows to manage the KNIME Server itself.

We’ve seen how the KNIME Server’s REST API has been designed, and how to start invoking our deployed workflows in the blog posts, Giving The KNIME Server (a) REST and The KNIME Server REST API. In this article we now want to explore some of the REST capabilities for RESTful workflows and gain some inspiration on how we could apply these tools elsewhere. We’ll need a KNIME Server to invoke the workflow via REST but can invoke external services with the Analytics Platform.

Example REST APIs

There are probably millions of example REST APIs on the internet. We could use a tool like Apiary to create some quick mockups, explore several of the ones Google offers to us, build our own with some Python & Flask, or even use KNIME Server’s own REST endpoints. Today we’ll be using one from OpenWeather to get some weather related data. We can create a free account and get an API key relatively quickly.

Make a note of the API Key as we’ll need it to call the REST Service. If you are following along with this blog, you will have your own unique API Key.

Fig. 1. Current weather API with an endpoint of https://api.openweathermap.org/data/2.5/weather

We’ll be using a GET Request to invoke the Current Weather API with an endpoint of https://api.openweathermap.org/data/2.5/weather. The endpoint requires a query parameter for the location and returns the current weather for that location. When we pass in the following URL to our browser or API tool we would get back the weather for Austin, Texas. Keep in mind you’ll need to update API Key to be your own. Here's the URL: https://api.openweathermap.org/data/2.5/weather?q=Austin,us&APPID=9258b2a6603d28f00f5268209dfe3225

Fig. 2. This URL, passed to our browser or API tool, gets back the weather for Austin, Texas, US.

Build an Example RESTful Workflow

Let’s start building our example workflow so that it:

Accepts and validates a JSON Input for location
Accepts an API Key
Returns some basic weather information

This is a simple example, but it shows a lot of the KNIME REST functionality. Download the final workflow from the KNIME Hub.

Fig. 3. A simple example workflow to demonstrate KNIME REST functionality. The workflow is publicly available on the KNIME Hub.

Call REST Services

To call REST Web Services KNIME has a handful of nodes under the ‘Tools & Services à REST Web Services’ menu. We can also find the GET, POST, PUT, DELETE, and PATCH Request nodes on the KNIME Hub.

Fig. 4. The Tools & Services --> REST Web Services menu in KNIME Analytics Platform.

To invoke our REST endpoint for weather we’ll use the GET Request Node and add in our URL in the configuration. We are using the API Key as a query parameter to authenticate here, but if you needed to add any Authentication, Request Headers, or Response Headers those are all configurable on the GET Request Node as well. The same applies for the other REST nodes and include inputs for data where applicable.

Fig. 5. The GET Request node configuration dialog.

Adding a JSON Path node after our GET Request lets us parse specific JSON elements or queries from our response. Rather than writing our queries, we can highlight the paths of the values we want and add them as outputs. Here the node will grab the temperature, description of the weather, and what it feels like outside.

Fig. 6. The JSON Path node configuration dialog. This node grabs the temperature, weather description, and feels-like description for outside.

Create JSON Inputs and Outputs

On occasion applications must adhere to specific interface requirements. The Container Input (JSON) and Container Output (JSON) nodes will enhance our workflows to ingest and return tailored JSON bodies respectively. All we need to do is define our input and output payloads in each node. City and Country should be dynamically passed on the input to make our workflow more flexible.

Fig. 7. Defining the input payload in the Container Input (JSON) node.

Another JSON Path node is required to extract out the ‘city’ and ‘country’ values. A String Manipulation node can take our input values and dynamically build the request URL for our OpenWeather GET Request.

The output is even simpler. A column filter to remove unwanted columns, a Table to JSON node that will convert the data into JSON, and end with our JSON Output Node. The JSON Output node will let us type in our JSON, but it’s much easier to let it fill In from the input.

Fig. 8. The JSON Output node will allow us to type in our JSON, but it's much easier to let it fill in from the input.

We can see inputs and outputs by deploying the workflow, navigating to the Swagger API Definition on the KNIME Server, and trying it out from there.

Fig. 9. See the inputs and outputs by deploying the workflow and navigating to the Swagger API Definition on the KNIME Server.

Validate Schemas

We can enhance our simple passthrough workflow by validating our JSON Schema. The JSON Schema Validator will prevent incorrect data types, and values, and set what fields are required to improve the overall quality of our service. You can get in-depth with the schema validation and can check out the how the validation works on the json-schema webpage, here. This configuration will make our ‘country’ and ‘city’ parameters strings that are required on input.

Fig. 10. The JSON Schema Validator prevents incorrect data types and values and sets which fields are required to improve overall service quality.

If we redeploy our workflow and try out the endpoint again in our Swagger UI, we’ll see that the workflow returns an error if we don’t submit our required fields.

Fig. 11. Redeploy our workflow and try out the endpoint again in our Swagger UI. Here we see that it returns an error if we don't submit our required fields.

Configure Inputs

KNIME has several ways to input data, and those familiar with configuration nodes will find comfort in their reuse here. Some of our existing workflows might already be using top-level configurations: These will automatically be available as JSON Inputs.

As an example, we can add a String Configuration node to our workflow to accept the APPID required by our weather’s GET Request and see that it appears in the request body. Configurations are only available for us to change if we use the :jobs endpoints for our workflow!

Fig. 12. The String Configuration node dialog.

Fig. 13. Use the :jobs endpoints for our workflow to change the configuration.

Use Container Inputs

If you don’t want to work with JSON Data in the workflow or are already using another form of the Container Inputs, they will also appear as inputs in our REST Interface. This is particularly useful when we want to build one workflow that can be used with and beyond REST use cases.

If we wanted our workflow to pull in data from a database, but still be configurable to use REST we can use the Container Input (Table) or Container Input (Row). This workflow will read data from the database if we execute normally. If we invoke the workflow via REST, it will bypass the database reading and accept the REST inputs. This means we only need to build the workflow once for two separate execution types: reading data from the database or receiving data from somewhere else.

Fig. 14. More input types.

We’ll see all these inputs on our request:

Fig. 15. Here, we can see all of these inputs on our request.

KNIME Server for Robust RESTful Workflows

KNIME Server allows us to create robust RESTful workflows. Here’s a quick overview of what we can do:

GET, POST, PUT, DELETE and PATCH Nodes to work with REST Services.
JSON Path node to parse our JSON data.
JSON Inputs and Outputs to specify our interface.
JSON Schema Validator to validate our JSON.
Configuration and other Container nodes to enhance reusability and versatility of our data choice.

You can invoke external REST Services using KNIME Analytics Platform. The KNIME Server REST API is available as part of KNIME Server. For more information or to arrange a trial, contact sales@knime.com.