KNIME general.

Data Privacy and KNIME

Mon, 10/08/2018 - 10:00 Peter

Author: Peter Ohl

We are often asked two things about data privacy in KNIME:

  • How are data handled in the open source KNIME Analytics Platform?
  • Who has access to the data that are processed?

Before diving into the details, let’s first put this into context:

KNIME Analytics Platform is open source and has a huge community contributing to its functionality by developing many Community Extensions.

KNIME Meets KNIME – Will They Blend?

Mon, 10/08/2018 - 10:00 admin

In this blog series we’ll be experimenting with the most interesting blends of data and tools. Whether it’s mixing traditional sources with modern data lakes, open-source devops on the cloud with protected internal legacy tools, SQL with noSQL, web-wisdom-of-the-crowd with in-house handwritten notes, or IoT sensor data with idle chatting, we’re curious to find out: will they blend? Want to find out what happens when IBM Watson meets Google News, Hadoop Hive meets Excel, R meets Python, or MS Word meets MongoDB?

Follow us here and send us your ideas for the next data blending challenge you’d like to see at willtheyblend@knime.com.

Today: KNIME Meets KNIME - Will They Blend?

Author: Phil Winters

The Challenge

Imagine you have been happily getting and using a new version of KNIME Analytics Platform with all of its additional features and functionality twice a year for many, many years.

But one day, you are required by your organization to pull out something from your distant KNIME past – something that at the time was very, very important and that needs to work EXACTLY the same way today.

But you’ve heard all the horror stories of other data science tools and platforms that have changed so fundamentally between versions (sometimes yearly) that a time-consuming migration (or even a rewrite) is required to get the old code to work. And of course, there is no guarantee from these vendors of backward compatibility nor that the results will be the same even when you do get it to work again. But what about KNIME? Will the old easily blend with the new? That is our challenge today!

Topic. Backward compatibility of KNIME Workflows

Challenge. Reuse the oldest KNIME workflow available in today’s current KNIME version

Four Techniques for Outlier Detection

Mon, 10/01/2018 - 10:00 admin

Authors: Maarit Widmann and Moritz Heine

Ever been skewed by the presence of outliers in your set of data? Anomalies, or outliers, can be a serious issue when training machine learning algorithms or applying statistical techniques. They are often the result of errors in measurements or exceptional system conditions and therefore do not describe the common functioning of the underlying system. Indeed, the best practice is to implement an outlier removal phase before proceeding with further analysis.

But hold on there! In some cases, outliers can give us information about localized anomalies in the whole system; so the detection of outliers is a valuable process because of the additional information they can provide about your dataset.

There are many techniques to detect and optionally remove outliers from a dataset. In this blog post, we show an implementation in KNIME Analytics Platform of four of the most frequently used - traditional and novel - techniques for outlier detection.

Beauty and the Monster: A Casual Story of Model Management

Mon, 09/24/2018 - 10:00 daria.goldmann

Author: Daria Goldmann

About a year ago we told a beautiful story about how KNIME Analytics Platform can be used to automate an established modeling process using the KNIME Model Factory. Recently our Life Science team faced an exhausting and frightening exercise of building, validating, and scoring models for more than 1500 data sets.

Model Deployment with KNIME Server and Amazon API Gateway

Mon, 09/17/2018 - 10:00 admin

Author: Jim Falgout

You’ve built a predictive model using KNIME Analytics Platform. It’s a very good model. Maybe even an excellent model. You want others to take advantage of your hard work by applying their data to your model. Let’s build an API for that!

An API is an Application Programming Interface. It’s a way to programmatically (i.e. write some code) interface with a computer program. A REST API is a specific sort of API that is used in the world of web service development. REST APIs pass around data in a format known as JSON.

Here are a few reasons for building a REST API for the application of your model:

  • Integrate the application of your model with your company’s web site
  • Integrate the application of your model with business processes in your company
  • Share the application of your model with the outside world (with some controls on top)
  • Sell the application of your model as a service

As you can see from these example usages, APIs are all about sharing and integrating.

Fun With Tags

Mon, 09/10/2018 - 10:00 Jeany

Author: Jeanette Prinz

In a previous blog post, I discussed visualizations in KNIME Analytics Platform. Having recently moved to Berlin, I have been paying more attention to street graffiti. So today, we will be learning how to tag.

...just kidding. Sort of.

Our focus will be on tagging, but the text-mining (rather than street art) variety: We will learn how to automatically tag disease names in biomedical literature.

Introduction

The rapid growth in the amount of biomedical literature becoming available makes it impossible for humans alone to extract and exhaust all of the useful information it contains. There is simply too much there. Despite our best efforts, many things would fall through the cracks, including valuable disease-related information. Hence, automated access to disease information is an important goal of text-mining efforts1. This enables, for example, the integration with other data types and the generation of new hypotheses by combining facts that have been extracted from several sources2.

In this blog post, we will use KNIME Analytics Platform to create a model that learns disease names in a set of documents from the biomedical literature. The model has two inputs: an initial list of disease names and the documents. Our goal is to create a model that can tag disease names that are part of our input as well as novel disease names. Hence, one important aspect of this project is that our model should be able to autonomously detect disease names that were not part of the training.

To do this, we will automatically extract abstracts from PubMed and use these documents (the corpus) to train our model starting with an initial list of disease names (the dictionary). We then evaluate the resulting model using documents that were not part of the training. Additionally, we test whether the model can extract new information by comparing the detected disease names to our initial dictionary.

Productionizing Data Science with KNIME Server

Mon, 09/03/2018 - 10:00 admin

Author: Vincenzo Tursi

KNIME Analytics Platform is the open source software for creating data science. It allows you to design and implement data science workflows with added leverage from KNIME Integrations, KNIME Extensions, Community Extensions, and Partner Extensions.

Moving one step further to now put these data science applications into production, a number of requirements need to be taken into account.

Intelligently Automating Machine Learning, Artificial Intelligence, and Data Science

Mon, 07/30/2018 - 10:20 admin

Authors: Christian Dietz, Paolo Tamagnini, Simon Schmid, Michael Berthold

In recent months a wealth of tools has appeared, which claim to automate all or parts of the data science cycle. Those tools often automate only a few phases of the cycle, have a tendency to consider just a small subset of available models, and are limited to relatively straightforward, simple data formats.

At KNIME we take a different stance: automation should not result in black boxes, hiding the interesting pieces from everyone; the modern data science environment should allow automation and interaction to be combined flexibly. If the data science team works on a well defined type of analysis scenario, then more automation may make sense. But more often than not, the interesting analysis scenarios are not that easy to control and a certain amount of interaction with the users is actually highly desirable.

Get on Board and Navigate the Learning Options at KNIME

Mon, 07/23/2018 - 10:00 admin

Authors: Maarit Laukkanen, Rosaria Silipo, Heather Fyson

There are all kinds of resources here at KNIME to learn more about using data science with our tools: KNIME Analytics Platform or KNIME Server. There are courses, website articles, Innovation Notes, YouTube videos, noding and development guidelines, our SDK Setup on Github, etc… We even have a range of ebooks! The latest one in the series is about Text Mining, for example. But how can you find out which resource is best matched to what you need to know? What if you’re the kind of person interested in more structured learning and you want to check out our course schedule? Or you live in a faraway place and there’s no course scheduled near you? Some of our courses are in a classroom with teachers, some are run online by the same teachers, and some courses involve videos recorded by these teachers. Some courses cover KNIME Analytics Platform, some KNIME Server; some cover basic functionalities, some are more advanced, some cover data analytics, some text processing, or big data. But, hold on, there… what if you work in a company that produces a lot of data and want to use data analytics to find out more about your business’s impact, but you’re not a data scientist yourself? Well, then our Innovation Notes, a small series of use cases would be good. But where are they? And what if you are a more socially inclined kind of person, you might fancy some networking opportunities and are looking for our list of meetup events or even better, the KNIME Summit?

Do you feel overwhelmed? Is it hard to decide which learning option is the best for you? Maybe the flow chart below can help you navigate the different learning options, according to your needs and inclinations.

Summertime and KNIMEing is easy with the latest release of KNIME Analytics Platform

Mon, 07/16/2018 - 10:19 Vincenzo

In case you haven’t yet booked your summer holiday (or winter getaway depending which hemisphere you live in), not to worry - we’ve got the perfect thing to keep you busy! KNIME Analytics Platform 3.6 and KNIME Server 4.7 have been released and there’s plenty of new things to try out!

 

Have a go at creating your own fully functional local big data environment from within a KNIME workflow - thanks to this neat little node, this is now possible! Or, if you’re interested in deep learning, check out the newest enhancements to our deep learning integrations. There’s also plenty of utility nodes to try out as well as many new UI improvements.

On the KNIME Server side, there are new features to check out, too. Like an option that makes it easier for IT to centrally manage KNIME Analytics Platform client preferences, to display job views, and run a workflow faster on distributed executors.

Subscribe to KNIME Blog: general