Author: Rosaria Silipo

Collaborative filtering (CF)^[1] based on the alternating least squares (ALS) technique^[2] is another algorithm used to generate recommendations. It produces automatic predictions (filtering) about the interests of a user by collecting preferences from many other users (collaborating). The underlying assumption of the collaborative filtering approach is that if a person A has the same opinion as a person B on an issue, A is more likely to have B’s opinion on a different issue than a randomly chosen person. This algorithm gained a lot of traction in the data science community after it was used by the team winner of the Netflix Prize.

Authors: Andisa Dewi and Tobias Koetter

The focus today is to show how to perform data exploration and visualization on a large dataset using KNIME Big Data Extensions and make the whole process interactive via the KNIME WebPortal. The data that we will use is the hugely popular NYC taxi dataset.

The idea of this workflow is to explore the taxi dataset step by step. We start with a general overview of the entire dataset and then, in the following step, we filter directly right on the interactive view, e.g select the specific years we want information on, or choose a particular taxi type, then zoom in on the particular subset of data that we are most interested in. The next step involves visualizing the selected subset subsequently. The last step shows visualizations of taxi trips of a certain taxi type in a specific certain NYC borough over during certain years. All the visualizations are accessible via the KNIME WebPortal and the computation is done on a Hadoop cluster using the KNIME Big Data Extension.

In this blog series we’ll be experimenting with the most interesting blends of data and tools. Whether it’s mixing traditional sources with modern data lakes, open-source devops on the cloud with protected internal legacy tools, SQL with noSQL, web-wisdom-of-the-crowd with in-house handwritten notes, or IoT sensor data with idle chatting, we’re curious to find out: will they blend? Want to find out what happens when IBM Watson meets Google News, Hadoop Hive meets Excel, R meets Python, or MS Word meets MongoDB?

     KNIME user: “You got your notebook in my workflow!”

     Jupyter Python user: “You got your workflow in my notebook!”

     Both: “Oooo, they work great together!” ¹

KNIME Analytics Platform has had good integration with Python for quite a while. Since we think it’s important, we continue to invest in making improvements. There are two particularly exciting new Python-related features in the recent 3.7 release of KNIME Analytics Platform:

1. You can now use the Python code found in Jupyter notebooks from the Python scripting nodes in KNIME
2. You can now execute KNIME workflows directly from within Python. If you are working within a Jupyter notebook you can also get a (static) view of the workflow in the notebook.

Since the release of KNIME Server 4.7 operations with KNIME Server have been simplified thanks to the new feature of being able to manage client preferences. This feature allows KNIME Server administrators to define profiles for KNIME Analytics Platform users and makes it easy to enable KNIME Analytics Platform to support a wide range of technologies and databases across all major operating systems.

When we opened the door for December 6 on the Advent Calendar at KNIME HQ this year we discovered new releases of KNIME Analytics Platform and KNIME Server. Yeah! This happens every year, but it’s always exciting to release a bunch of cool new functionality for our community and customers to start using. We’ve added new views; better integration with Google Drive; even more new database nodes; integrations with Jupyter notebook, PySpark, and XGBoost; and many other things.

We’ve put together a couple of videos showing some of the highlights of this release:

Author: Rosaria Silipo.

There are many declinations of data science projects: with or without labeled data; stopping at data wrangling or involving machine learning algorithms; predicting classes or predicting numbers; with unevenly distributed classes, with binary classes, or even with no examples at all of one of the classes; with structured data and with unstructured data; using past samples or just remaining in the present; with requirements for real-time or close to real-time execution or with acceptably slower performances; showing the results in shiny reports or hiding the nitty gritty behind a neutral IT architecture; and with large budgets or no budget at all.

Authors: Chris Baddeley and Rosaria Silipo.

What is a Metanode?

Before we start, what is a metanode? Metanodes are gray nodes that contain sub-workflows. They play the role of functions or macros in script based tools.

They look like a single node, although they can contain many nodes and even more metanodes.

Recurrent Neural Networks (RNN) are the state of the art for sequence analysis ^{5 6}. With the release of KNIME Analytics Platform 3.6, KNIME extended its set of deep learning integrations, adding the Keras integration to the DL4J Integration. This adds considerably more flexibility and advanced layers, like RNN Layers.

In this article, we want to find out what Recurrent Neural Networks are in general, and LSTMs in particular. Let’s see where they are useful and how to set up and use the Keras integration in KNIME Analytics Platform to implement them.

As a use case for this particular application of RNNs and LSTMs we want to focus on automatic text generation. The question is: Can we teach KNIME to write a fairy tale?