Machine Learning with Qlik Sense: Utilize different machine learning models in practical use cases by leveraging Qlik Sense

Machine Learning with Qlik Sense

Copyright © 2023 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Ali Abidi

Publishing Product Manager: Sanjana Gupta

Book Project Manager: Farheen Fathima

Content Development Editor: Joseph Sunil

Technical Editor: Devanshi Ayare

Copy Editor: Safis Editing

Proofreader: Safis Editing

Indexer: Hemangini Bari

Production Designer: Prashant Ghare

DevRel Marketing Coordinator: Vinishka Kalra

First published: October 2023

Production reference: 1290923

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK.

ISBN 978-1-80512-615-7

www.packtpub.com

To my parents for the support and encouragement during my life. To Essi for being my dive buddy in life.

-Hannu Ranta

Contributors

About the author

Hannu Ranta is a data and cloud professional with wide technical knowledge. He has worked with big data, IoT, and analytics solutions with the largest enterprises across the globe. He always enjoys finding innovative solutions to build a better future with data, while helping customers to deliver value.

Hannu obtained his Master of Science degree with distinction in 2015 and has worked for leading data companies like Qlik, Microsoft and Cubiq Analytics since then. Currently, he is a Principal Enterprise Architect for Nordic and Baltic region at Qlik.

When not working, Hannu is usually scuba diving, snowboarding, or traveling. Originally from Tammela, Finland, he now lives in Helsinki, the capital of Finland, with his girlfriend.

I want to thank my girlfriend, Essi, for the support and encouragement during the writing process and my parents for everything. I would also like to thank my colleagues from Qlik for all the help and support, especially Troels, Mikko, and the Finnish team. Thanks also to my friends, for inspiring conversations, and everyone else who helped me during my career.

About the reviewers

Rohan Chikorde is an accomplished AI Architect professional with a post-graduate in Machine Learning and Artificial Intelligence. With almost a decade of experience, he has successfully developed NLP, Deep Learning and Machine Learning models for various business applications. Rohan’s expertise spans multiple domains, and he excels in programming languages such as R and Python, as well as analytics techniques like regression analysis and data mining. In addition to his technical prowess, he is an effective communicator, mentor, and team leader. Rohan’s passion lies in machine learning, deep learning, and computer vision.

Thank you so much to the Packt team for the opportunity.

Pablo Labbe is a seasoned consultant working on Business Intelligence (BI) projects over 25 years. During his journey he was always challenged to help organizations to be more data-driven. He is currently a Principal Solution Architect at iMaps Intelligence, a data and analytics company located in Brazil South Region.

Pablo has leveraged his expertise by directly working within industries such as government, retail, healthcare, agriculture, and manufacturing.

Pablo is the co-author of two books related to Qlik Sense: Qlik Sense Cookbook, 2nd edition and Hands-On Business Intelligence with Qlik Sense.

Clever Anjos is a Principal Solutions Architect at Qlik, a data analytics and data integration software company. He has been working for Qlik since 2018 but has been around the Qlik Ecosystem as a Partner and Customer since 2009. He is a Business Discovery professional with several years of experience working with Qlik, AWS, Google Cloud, Databricks, and other BI technologies.

He is a highly active member of the Qlik Community, with over 8,000 posts and 4.5K page views. In May 2022, he was named the Qlik Community’s Featured Member. Clever is also a writer and has published a book called Hands-On Business Intelligence With Qlik.

Table of Contents

Preface

Part 1: Concepts of Machine Learning

1

Introduction to Machine Learning with Qlik

Introduction to Qlik tools

Insight Advisor

Qlik AutoML

Advanced Analytics Integration

Basic statistical concepts with Qlik solutions

Types of data

Mean, median, and mode

Variance

Standard deviation

Standardization

Correlation

Probability

Defining a proper sample size and population

Defining a sample size

Training and test data in machine learning

Concepts to analyze model performance and reliability

Regression model scoring

Multiclass classification scoring and binary classification scoring

Feature importance

Summary

2

Machine Learning Algorithms and Models with Qlik

Regression models

Linear regression

Logistic regression

Lasso regression

Clustering algorithms, decision trees, and random forests

K-means clustering

ID3 decision tree

Boosting algorithms and Naive Bayes

XGBoost

Gaussian Naive Bayes

Neural networks, deep learning, and natural-language models

Summary

3

Data Literacy in a Machine Learning Context

What is data literacy?

Critical thinking

Research and domain knowledge

Communication

Technical skills

Informed decision-making

Data strategy

Summary

4

Creating a Good Machine Learning Solution with the Qlik Platform

Defining a machine learning problem

Cleaning and preparing data

Example 1 – one-hot encoding

Example 2 – feature scaling

Preparing and validating a model

Visualizing the end results

Summary

Part 2: Machine learning algorithms and models with Qlik

5

Setting Up the Environments

Advanced Analytics Integration with R and Python

Installing Advanced Analytics Integration with R

Installing Advanced Analytics Integration with Python

Setting up Qlik AutoML

Cloud integrations with REST

General Advanced Analytics connector

Amazon SageMaker connector

Azure ML connector

Qlik AutoML connector

Summary

6

Preprocessing and Exploring Data with Qlik Sense

Creating a data model with the data manager

Introduction to the data manager

Introduction to Qlik script

Important functions in Qlik script

Validating data

Data lineage and data catalogs

Data lineage

Data catalogs

Exploring data and finding insights

Summary

7

Deploying and Monitoring Machine Learning Models

Building a model in an on-premises environment using the Advanced Analytics connection

Monitoring and debugging models

Summary

8

Utilizing Qlik AutoML

Features of Qlik AutoML

Using Qlik AutoML in a cloud environment

Creating and monitoring a machine learning model with Qlik AutoML

Connecting Qlik AutoML to an on-premises environment

Best practices with Qlik AutoML

Summary

9

Advanced Data Visualization Techniques for Machine Learning Solutions

Visualizing machine learning data

Chart and visualization types in Qlik

Bar charts

Box plots

Bullet charts

Distribution plots

Histogram

Maps

Scatter plots

Waterfall charts

Choosing visualization type

Summary

Part 3: Case studies and best practices

10

Examples and Case Studies

Linear regression example

Customer churn example

Summary

11

Future Direction

The future trends of machine learning and AI

How to recognize potential megatrends

Summary

Index

Other Books You May Enjoy

Preface

Machine Learning with Qlik Sense is a book for anyone who wants to master machine learning and expand their use of analytics into predictive use cases. You will learn the key concepts of machine learning using practical examples, enabling you to create better analytics applications and get the most out of your data.

Qlik Sense is a world-leading data analytics platform with comprehensive capabilities in machine learning. This book will guide you to build machine learning enabled analytics solutions using both Qlik Cloud Analytics with AutoML and Qlik Sense Client-Managed.

Who this book is for

If you are interested in data and analytics with a will to extend your skillset to machine learning, this book is for you. In order to learn from this book, you should have basic knowledge of working with data, preferably with Qlik tools. This book is an excellent guide for everyone willing to take the next step on their journey and start using machine learning as a part of their data analytics journey.

What this book covers

Chapter 1, Introduction to Machine Learning with Qlik, will introduce you to the world of machine learning with the Qlik platform. This chapter covers all the basic concepts for implementing machine learning with Qlik, like R2, F1 and SHAP.

Chapter 2, Machine Learning Algorithms and Models with Qlik, will provide information about the essential algorithms and models in machine learning focusing on ones important in the Qlik platform. You will get a basic understanding of how the algorithms behind Qlik’s ML solution work and how to pick the right one for specific problems.

Chapter 3, Data Literacy in a Machine Learning Context, will cover how data literacy can be utilized in a machine learning context. You will learn and utilize data literacy skills to get the most out of the data that ML models are using.

Chapter 4, Creating a Good Machine Learning Solution with the Qlik Platform, covers the essential knowledge to create a good machine learning solution with the Qlik platform. You will learn all the steps needed to utilize automated solutions for model building.

Chapter 5, Setting Up the Environments, teaches how to set up the environments for machine learning using Qlik tools. You will get hands on examples for setting up and initializing different environments and also cover any problems that might occur during the setup, and how to fix them.

Chapter 6, Preprocessing and Exploring Data with Qlik Sense, will cover the techniques needed to preprocess the data in Qlik Sense. This chapter will guide you through all the important steps for preprocessing and exploring data. You will learn how to validate data and make data exploration efficient.

Chapter 7, Deploying and Monitoring Machine Learning Models, will cover how to deploy and monitor machine learning models in both cloud and client-managed environments. It will also cover what to consider before deploying to production.

Chapter 8, Utilizing Qlik AutoML, covers the use of Qlik AutoML tool in both cloud and on-premise environments. This chapter will guide you with the best practices and features of AutoML using real-world examples. You will also learn the features of AutoML and models that can be deployed using the tool.

Chapter 9, Advanced Data Visualization Techniques for Machine Learning Solutions, provides examples and best practices about visualizing machine learning related data with Qlik tools. This chapter covers Qlik charts and advanced features and functions to fully utilize the charts. It will also cover how to use Insight Advisor to help visualization tasks and provide insights about data.

Chapter 10, Examples and Case Studies, guides you through real world examples and use cases with Qlik’s machine learning portfolio. Each example is described in detailed level and also the information about the business value is provided.

Chapter 11, Future Direction, will give you an idea of the future development and trends of machine learning. You will get information about overall trends and how the Qlik portfolio will develop to support the adoption of new trends.

To get the most out of this book

You should have basic knowledge of Qlik tools and data analytics to get the most out of this book. Also, basic knowledge of Qlik Cloud and AutoML and understanding the basic machine learning concepts and statistics is helpful.

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Machine-Learning-with-Qlik-Sense. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: Mount the downloaded WebStorm-10*.dmg disk image file as another disk in your system.

A block of code is set as follows:

iris:

LOAD

RowNo() as id,

      sepal_length,

      sepal_width,

      petal_length,

      petal_width

FROM [lib:///iris_test.csv]

(txt, utf8, embedded labels, delimiter is ',', msq);

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

[predictions]:

LOAD * EXTENSION endpoints.ScriptEval('{RequestType:endpoint,

endpoint:{connectionname:ML demos:Iris}}', iris);

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: If we would like to change our experiment, we can select Configure v2.

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Machine Learning with Qlik Sense, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere? Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/978-1-80512-615-7

Submit your proof of purchase

That’s it! We’ll send your free PDF and other benefits to your email directly

Part 1:Concepts of Machine Learning

This section will provide the background for the remaining parts of the book. The section covers the basics of machine learning with the Qlik platform and provides an understanding of important concepts and algorithms used in machine learning and statistics. It also covers the use of data literacy in the area of machine learning. Finally, this section provides the essentials of building a good machine learning solution with the Qlik platform. These concepts will be utilized during section 2 of this book.

This section has the following chapters:

Chapter 1: Introduction to Machine Learning with Qlik

Chapter 2: Machine Learning Algorithms and Models with Qlik

Chapter 3: Data Literacy in a Machine Learning Context

Chapter 4: Creating a Good Machine Learning Solution with the Qlik Platform

1

Introduction to Machine Learning with Qlik

Machine learning and artificial intelligence are two of the most powerful technology trends in the 21st century. Usage of these technologies is rapidly growing since the need for faster insights and forecasts has become crucial for companies. Qlik is a leading vendor in the analytics space and has heavily invested in machine learning and AI tools.

This first chapter will introduce the different machine learning tools in the Qlik ecosystem and provide basic information about the statistical models and principles behind these tools. It will also cover the concepts of correct sample size and how to analyze model performance and reliability.

Here is what we will cover in this first chapter:

An overview of the Qlik tools and platform

The basic statistical concepts of machine learning

Proper sample size and the defining factors of a sample

How to evaluate model