How Do I Filter Data In Data Stream?

Updated Jul 09, 2019

K3 uses distributed indexing to help the user quickly filter data.  As data enters the K3 ecosystem it is immediately Indexed for searching in Data Streams.  Data Streams provides multiple ways of filtering data for a given route. Below we outline the filtering options.

Field Level Chip Filtering

The search input in Data Streams provides typeahead autocomplete for both fields in a given route and data values for a given field. This makes the process of filtering down to a given set of data super simple. Avery filter that is active is then displayed in a Chip above the dataset which can be easily removed or edited.

  1. At the top of the screen begin typing for the data you want to see.  For example, in the screenshot below cl... was typed. 
  2. K3 shows data meeting that typeahead value.
  3. Select what you are looking for.  K3 renders a filtered view for that criteria.  For example, we can see that there are 320 "Few" s in the data store.
  4. Select the criteria you wish and K3 Filters the result set in the K3 Data Stream.

Below you will see an example of the Chips describing the current filters in place.

Yes, K3 supports wild card searches.  For example, if you were looking for the term "Overcast" you could use the wild card search similar to the below:

  • over*
  • cloud_layer:over*

How Do I Distinguish Between Source and Target Data in Filtering?

When a value is used both in the Source and the Target it is prefixed.  K3 Source is prefixed by src.  K3 Target Data is prefixed by tgt.  For example, "tgt.description" is the post ETL or transformed column of description.

If the value is unique, meaning there is only one then it is not prefixed.

How Do I Filter by Date?

Date filtering in Data Stream is done by simply enter a From and To dates.  The typing date format is: yyyy-mm-dd

Video Walkthrough

Previous Article How Do I Distinguish Between Source and Target Data in Filtering?
Next Article How Do I See The State Of a Process in K3 Data Stream?