KNIME Big Data Connectors

KNIME® Big Data Connectors allow easy access to Apache Hadoop data from within KNIME Analytics Platform and KNIME Server. This extension offers a set of KNIME nodes for accessing Hadoop/HDFS via Hive or Impala and ships with all required libraries.

Nodes Included: HDFS Connection, webHDFS Connection, HttpFS Connection, HDFS File Permission, Hive Connector, Hive Loader, Impala Connector, Impala Loader

  • Move data between KNIME Analytics Platform or KNIME Server and Hive/Impala
  • Write Hive/Impala SQL queries using the standard KNIME Database Query node
  • Process SQL queries directly in Hive and Impala using standard KNIME database nodes

Try Now

Compatibility

KNIME Big Data Connectors are certified by Cloudera for CDH 5.x, by Hortonworks for HDP 2.1 and 2.2, by MapR for MapR 4.1. and have also been tested against Hadoop 2.4.0 and Hive 0.13.

 

 

Usage example: Querying data in Hive

The Hive Connector node establishes a connection to a Hive database. Once executed, the node returns a database connection that can be used with almost any of KNIME Analytic Platform's standard database nodes.

Usage example: Loading data into Hive

Hive itself does not allow modification of data – only inserting and appending data are supported. Since data import does not follow standard SQL/JDBC procedures, KNIME Big Data Connectors provide a special Hive Loader node. The node makes use of the File Handling extensions and first copies the data onto the Hive server, using HDFS, webHDFS, HttpFS, SSH, FTP or any other supported protocol. A Hive command is then executed to import the data into Hive. The node's output is a database connection operating on the imported table. The Hive Loader node allows you to specify the directory in the remote file system into which the data are uploaded, the table's name, whether existing tables should be dropped, or whether data should be appended, and by which columns data should be partitioned:

 

 

Installation steps

For KNIME Analytics Platform 3.4 and KNIME Server 4.5

Step 1: Obtain License  The KNIME Big Data Connectors require a license which you can purchase via the KNIME Store. To give it a try, please request a free 30 day trial license.

 

Step 2: Install Software  The KNIME Big Data Connectors can be installed as an extension in the KNIME Analytics Platform in one of the following ways:

  • Via the KNIME Update Manager: 
  1. Go to File → Install KNIME Extensions...
  2. Open the category KNIME.com Big Data Extensions (licenses required).
  3. Select the KNIME Big Data Connectors extension.
  4. Click on Next and follow the subsequent dialog steps to install the extension.
  • Via a zipped update site: If you don't have direct internet access you can also download a zipped update site and register it in File → Preferences → Install/Update → Available Software Sites. Then follow the steps above.

 

Step 3: Download License  Finally, to be able to use the installed extension, you need to download your previously obtained license into the KNIME Analytics Platform as described here.

For KNIME Analytics Platform 3.3 and KNIME Server 4.4

Step 1: Obtain License  The KNIME Big Data Connectors require a license which you can purchase via the KNIME Store. To give it a try, please request a free 30 day trial license.

 

Step 2: Install Software  The KNIME Big Data Connectors can be installed as an extension in the KNIME Analytics Platform in one of the following ways:

  • Via the KNIME Update Manager: 
  1. Go to File → Install KNIME Extensions...
  2. Open the category KNIME.com Big Data Extensions (licenses required).
  3. Select the KNIME Big Data Connectors extension.
  4. Click on Next and follow the subsequent dialog steps to install the extension.
  • Via a zipped update site: If you don't have direct internet access you can also download a zipped update site and register it in File → Preferences → Install/Update → Available Software Sites. Then follow the steps above.

 

Step 3: Download License  Finally, to be able to use the installed extension, you need to download your previously obtained license into the KNIME Analytics Platform as described here.

For KNIME Analytics Platform 3.2 and KNIME Server 4.3

Step 1: Obtain License  The KNIME Big Data Connectors require a license which you can purchase via the KNIME Store. To give it a try, please request a free 30 day trial license.

 

Step 2: Install Software  The KNIME Big Data Connectors can be installed as an extension in the KNIME Analytics Platform in one of the following ways:

  • Via the KNIME Update Manager: 
  1. Go to File → Install KNIME Extensions...
  2. Open the category KNIME.com Big Data Extensions (licenses required).
  3. Select the KNIME Big Data Connectors extension.
  4. Click on Next and follow the subsequent dialog steps to install the extension.
  • Via a zipped update site: If you don't have direct internet access you can also download a zipped update site and register it in File → Preferences → Install/Update → Available Software Sites. Then follow the steps above.

 

Step 3: Download License  Finally, to be able to use the installed extension, you need to download your previously obtained license into the KNIME Analytics Platform as described here.

For KNIME Analytics Platform 3.1 and KNIME Server 4.2

Step 1: Obtain License  The KNIME Big Data Connectors require a license which you can purchase via the KNIME Store. To give it a try, please request a free 30 day trial license.

 

Step 2: Install Software  The KNIME Big Data Connectors can be installed as an extension in the KNIME Analytics Platform in one of the following ways:

  • Via the KNIME Update Manager: 
  1. Go to File → Install KNIME Extensions...
  2. Open the category KNIME.com Big Data Extensions (licenses required).
  3. Select the KNIME Big Data Connectors extension.
  4. Click on Next and follow the subsequent dialog steps to install the extension.
  • Via a zipped update site: If you don't have direct internet access you can also download a zipped update site and register it in File → Preferences → Install/Update → Available Software Sites. Then follow the steps above.

 

Step 3: Download License  Finally, to be able to use the installed extension, you need to download your previously obtained license into the KNIME Analytics Platform as described here.

For KNIME Analytics Platform 2.12 and KNIME Server 4.1

Step 1: Obtain License  The KNIME Big Data Connectors require a license which you can purchase via the KNIME Store. To give it a try, please request a free 30 day trial license.

 

Step 2: Install Software  The KNIME Big Data Connectors can be installed as an extension in the KNIME Analytics Platform in one of the following ways:

  • Via the KNIME Update Manager: 
  1. Go to File → Install KNIME Extensions...
  2. Open the category KNIME.com Extension Store (licenses required).
  3. Select the KNIME Big Data Connectors extension.
  4. Click on Next and follow the subsequent dialog steps to install the extension.
  • Via a zipped update site: If you don't have direct internet access you can also download a zipped update site and register it in File → Preferences → Install/Update → Available Software Sites. Then follow the steps above.

 

Step 3: Download License  Finally, to be able to use the installed extension, you need to download your previously obtained license into the KNIME Analytics Platform as described here.

Availability

KNIME Big Data Connectors are available in the KNIME Store. Please have a look at the KNIME Big Data Extensions product sheet or contact us at bigdata@knime.com.