Notebooks in Mach5

Mach5 notebook is an interactive workspace that lets one query, analyse and visualize results in one place. This document provides a step-by-step guide to creating a notebook in Mach5. It explores the MQL cell search and Discover cell features in detail. MQL is Mach5’s query language, inspired by the Kusto Query Language (KQL).

Prerequisites

• This document assumes that Mach5 is deployed and running successfully. Lets assume it’s running at http://localhost:8888/
• Store, store route and warehouse localwarehouse are created successfully. Refer to Quickstart document for help
• Go to the Warehouse list and select localwarehouse. In the details section, click the Dashboards link, which opens the dashboard page at /warehouse/default/localwarehouse/dashboards/
• Click on Add Data to add Sample flights data. opensearch_dashboards_sample_data_flights index is created in Mach5 with flight records

Notebooks

Mach5 notebook is divided into cells i.e. MQL cell and Discover cell:

• One can run cells individually to query data, compute results or visualise data
• Query results appear right below each cell
• One can add any number of cells to query, analyze and visualise data in Mach5 UI
• Any cell can be deleted not affecting other cell execution
• Saving a notebook, saves the individual cell queries

Create a Notebook

• From the Mach5 Administrative UI, navigate to the Notebooks page
• Click on + icon on the top right corner to add a notebook
• Fill in the following details:
  • Name: Name of the notebook E.g., test_query_notebook
• Click on Save

Add cells in the Notebook

• Click on the notebook test_query_notebook from the Notebooks page
• Click on the + sign in the middle of the page
• Select either Add MQL Cell to run MQL queries or Add Discover Cell to use the discover functionality

MQL Cell

• Following are some example MQL queries on the sample flight data that can be executed in the MQL cell
• One can add and run multiple MQL queries in same notebook easily. Let’s see how that is done

MQL Query - Projection

• Click on + sign to add an MQL cell and type in the following query
• Click on Play button to view results

opensearch_dashboards_sample_data_flights
| take 100

• This query lists all columns of sample flight data and shows only first 100 rows
• Use the paging feature, Previous/Next buttons to navigate through the pages easily

MQL Query - Filters

• Click on + sign to add one more MQL cell and type in the following query
• Click on Play button to view results

opensearch_dashboards_sample_data_flights
| project Carrier, OriginAirportID, OriginCityName, Dest, DestCityName, DestRegion
| where OriginCityName == 'Zurich'

• This query finds all records with projected columns with OriginCityName as Zurich

MQL Query - Aggregations

• Click on + sign to add one more MQL cell and type in the following query
• Click on Play button to view results

opensearch_dashboards_sample_data_flights
| where AvgTicketPrice > 800.0
| summarize value = count() by Carrier, OriginCityName, DestWeather
| sort by Carrier desc, OriginCityName asc
| take 100

• This query groups the flight records by Carrier name, Origin City Name and Destination weather: sorts as per the given criteria and filters by the average ticket price. It only gives first 100 records

MQL Query - Visualizations with render

• Mach5 allows rendering of visualizations in notebook using the render operator. Different type of charts can be rendered so that one can visualize their data appropriately
• Click on + sign to add one more MQL cell and type in the following query
• Click on Play button to view results

opensearch_dashboards_sample_data_flights
| where AvgTicketPrice > 800.0
| summarize value = count() by OriginWeather
| render piechart

• Visualize how the data is distributed over the type of weather at the origin cities of the flights

Discover Cell

Discover functionality is a data exploration interface for Mach5 indices. It lets one interactively visualise indexed data, inspect fields, run customised time based searches. Discover does progressive query processing in the background and loading results in chunks so that a user sees query results early. This prevents long wait times and gives a responsive feel even for huge indices (billions of documents)

Data Distribution over time

• Click on Settings icon to select the Mach5 index name opensearch_dashboards_sample_data_flights
• Select a time field for chart plotting, in this case timestamp

• Keep the Hits Size be default value
• Let Enable Chunked be true. This gives the results in chunks so that user sees the query results early (instead of waiting for completion of full query and then viewing the results)
• Click on Close
• Click on Play button to view the chart and analyse the results

Filters

• The filter is an Elasticsearch/OpenSearch query filter and multiple filters can be added in the list
• Click on the Filter icon
• In the Add Filters pop-up, type in following filter query inside the existing braces [ ]

[
 {
    "match_phrase": {
      "OriginWeather": "Clear"
    }
  }
]

• Let’s filter flight data where Origin Weather is Clear, so that flights can take off in safe weather
• Click on Save

• Click on Play button to see the chart and results

• Note that Discover highlights the Clear keyword in results matching the filter criteria for OriginWeather
• Click on Expand below any result. Each field of that result is given in detail with its type, field name and value

• Click on the chart to drill down/up further. Use the slider below the chart to navigate as needed
• Use the different time based options (Relative, Absolute, Now) on the chart to query the data in detail
• Calendar interval is set to Auto by default in the charts. Select different options from the dropdown like milliseconds, seconds, minutes, hours, months, years, etc to visualize data as per the interval needed

Delete an MQL/Discover cell

• Click on Delete button of an MQL/Discover cell to delete the specific cell that is no longer needed

Close and Reopen a Notebook

• When a notebook is closed and later accessed again, MQL/Discover cell queries are preserved