E-guide

AI tools for
software application
developers
Part One
 AI tools for software application developers: Part One

In this e-guide
In this e-guide:
Codementor on AI tooling vs
The Computer Weekly Developer Network is in the engine
theory room, covered in grease and looking for Artificial Intelligence
(AI) tools for software application developers to use.
Shutterstock captures the
moment of AI evolution & With so much AI power in development and so many new
democratisation
neural network brains to build for our applications, how should
programmers ‘kit out’ their AI toolbox?
Mathworks on deep learning

How much grease and gearing should they get their hands dirty
Applitools on AI-assisted root with... and, which robot torque wrench should we start with?
cause analysis

Adrian Bridgwater, ComputerWeekly.com

Page 1 of 18
 AI tools for software application developers: Part One

In this e-guide
AI developer toolset series: Codementor
Codementor on AI tooling vs on AI tooling vs theory
theory
Adrian Bridgwater, guest contributor
Shutterstock captures the
moment of AI evolution &
Interest in AI has surged in the last decade. Along with this, AI tooling — which
democratisation we'll regard broadly here as anything that helps engineers build AI more
efficiently — has found itself in the spotlight.
Mathworks on deep learning
But how important is tooling?

Applitools on AI-assisted root Will it be tooling that allows us to advance AI further, or is tooling simply a
cause analysis distraction from ‘real’ work?

Tooling vs theory

The question comes down to the assessing the balance of tooling vs theory —
and asking which will truly lead us to the AI revolution?

Consider the tooling around Deep Learning, the poster child of recent AI
advancements.

Page 2 of 18
 AI tools for software application developers: Part One

Google, Facebook, Microsoft and Amazon have developed or adopted

In this e-guide TensorFlow, PyTorch, Microsoft Cognitive Toolkit and MXNet respectively,
highlighting the importance of these tools.
Codementor on AI tooling vs
theory A decade ago, we would have expected these companies to develop most of
their tooling as proprietary closed-source software, similarly to early versions of
Shutterstock captures the Microsoft Visual Studio. Instead, we now see tools developed under open
moment of AI evolution & source licenses, with strong inputs from a global community of developers.
democratisation
These tools undoubtedly help developers to work more efficiently.
Mathworks on deep learning
Instead of having to spend mental effort on backpropagation formulas, AI
engineers can work with abstract, high-level building blocks. This makes the
Applitools on AI-assisted root engineering feedback loop much faster and it is comparable to how writing
cause analysis
computer code in a high-level programming language is easier than writing code
in an assembly language.

The AI industrial revolution

The industrial revolution reduced the need for repetitive human labour and,
equivalently, AI tooling has reduced the need for low-level mathematics in
building AI solutions.

Many programmers today do not need to know how a compiler functions to write
useful computer software. With advanced enough tooling, developers can build
useful AI models without the need to study calculus and linear algebra.

Page 3 of 18
 AI tools for software application developers: Part One

But tooling can also be a distraction, or even a net negative. Frameworks can
In this e-guide suffer from depreciation over time. As Google pushes out new versions of
TensorFlow, models that were written last year are already obsolete.
Codementor on AI tooling vs
theory Most documentation, tutorials, books, presentations and academic papers
become obsolete within months.
Shutterstock captures the
moment of AI evolution &
Developers spend time becoming experts in specific tools only to find that they
democratisation need to start over a few months later.

Furthermore, not understanding the fundamental concepts can cause people to

Mathworks on deep learning
drastically overstate their results. Researchers today regularly claim to have
solved a problem with AI, when in fact they have simply uncovered random
Applitools on AI-assisted root results that look interesting due to careful selection from trillions of random
cause analysis
simulations.

Hammer to fall

While advanced tooling can help us, we shouldn't ignore simple alternatives.

A hammer in 2019 is not so different from a shaped rock that humans started
using 3.3 million years ago. Similarly, a modern computer keyboard is not that
different from those found on typewriters, and many professional engineers still
use old code editors such as Vim and Emacs.

Page 4 of 18
 AI tools for software application developers: Part One

It remains to be seen how much tooling will help us develop the next generation
In this e-guide of AI, and how much of this advancement will rather be done using older and
simpler methods.
Codementor on AI tooling vs
theory In conclusion, definitely don't ignore developments in tooling, but also make
sure you own your tools, instead of vice-versa.
Shutterstock captures the
moment of AI evolution &
democratisation Next Article

Mathworks on deep learning

Applitools on AI-assisted root

cause analysis

Page 5 of 18
 AI tools for software application developers: Part One

In this e-guide
AI developer toolset series: Shutterstock
Codementor on AI tooling vs captures the moment of AI evolution &
theory
democratisation
Shutterstock captures the Adrian Bridgwater, guest contributor
moment of AI evolution &
democratisation The following text is written by Peter Silvio in his capacity as vice president of
engineering for platform solutions at Shutterstock — the company hosts millions
Mathworks on deep learning of stock images, photos, videos and music on its portal website.

Silvio writes on the evolution and democratisation of AI/Deep Learning

Applitools on AI-assisted root
cause analysis development as follows…

When it comes to the works of AI, much of the focus has been in the realm of
pure science and mathematics, as well as researching, developing and training
models.

Many businesses are attempting to leverage AI and deep learning in their

applications… however, engineering and executing a production-ready build
which is scalable and performant is a difficult challenge.

This presents interesting challenges as the applications move to the cloud.

Page 6 of 18
 AI tools for software application developers: Part One

Some microservices architectures have robust infrastructure capabilities and

In this e-guide tooling, such as Docker and Kubernetes; however the same toolset support for
deep learning based applications, is still nascent.
Codementor on AI tooling vs
theory Offline Tasks & Online Tasks

In computer vision application development for example there are generally two
Shutterstock captures the
types of tasks to complete application, ‘Offline Tasks' & ‘Online Tasks'.
moment of AI evolution &
democratisation
Offline tasks are those which do not impact current production environments,
including model design, development, training and even initial testing and model
Mathworks on deep learning
validation. In terms of tooling, this is the most mature space within deep learning
with diverse languages from TensorFlow, Keras to PyTorch and a growing
Applitools on AI-assisted root ecosystem of supported libraries.
cause analysis
Additionally model development and training is beginning to see
democratization through the availability of commercial software and managed
services to reduce the overhead of the model development lifecycle.

A key tool many data scientists have used for years are Jupyter notebooks. In
recent years, many vendors including major cloud providers have focused their
attention on providing managed services which aim to make machine
learning/deep learning design and development more efficient as well as
accessible to data scientists and engineers alike.

Page 7 of 18
 AI tools for software application developers: Part One

Amazon Sage Maker, Google Data Lab and Azure ML Studio all provide fully
In this e-guide managed cloud Jupyter notebooks for designing and developing machine
learning and deep learning models by leveraging serverless cloud engines.
Codementor on AI tooling vs
theory By leveraging these platforms businesses can take advantage of a full spectrum
of capabilities, allowing developers to focus on building truly differentiated
Shutterstock captures the models where the business need or opportunity requires. As well as the ability
moment of AI evolution & to leverage pre-trained models or managed APIs for more standard,
democratisation commoditised capabilities or in between by leveraging transfer learning.

Transfer learning
Mathworks on deep learning

Given the time and resource required to build and train deep learning models,
Applitools on AI-assisted root transfer learning is an incredibly efficient and powerful method, which has
cause analysis become popular in deep learning.

Transfer learning allows us to use a pre-trained model on new problems.

Leveraging models such as Inception V3 for image recognition reduces overall
development time while still producing highly accurate results.

Online tasks consist of the productionisation and operations of AI / deep

learning capabilities into business applications and platforms. At Shutterstock
we have applied our research in image recognition to develop a platform which
offers our customers rich features such as ‘Reverse Image Search' and ‘Similar
Image Search'.

Page 8 of 18
 AI tools for software application developers: Part One

When it comes to executing on a production build for deep learning services

In this e-guide such as custom computer vision models the tool-chain available to developers
is still fairly nascent and relies heavily on more generalized tools and custom
Codementor on AI tooling vs development.
theory
Up until recently deploying Graphics processing unit (GPU) and Input/output
Shutterstock captures the (IO) heavy deep learning applications generally require expensive, intricate
moment of AI evolution & configurations and do not take full advantage of modern capabilities such as
democratisation containerization, autoscaling and tools such as Kubernetes.

This will change rapidly, there is already great language support across Python,
Mathworks on deep learning
Tensor, MxNet, etc, and as the offline tasks are being simplified so too will
online tasks. Now with the continued evolution of open source projects such as
Applitools on AI-assisted root
Kubernetes and Istio; building, deploying, testing and scaling are becoming
cause analysis
streamlined.

About the author

Peter Silvio is a passionate technology leader with over 20 years of experience,

he has spent the past decade architecting and building distributed service
platforms and designing data architecture not just for internal applications, but
also defining API products to power future company solutions.

Next Article

Page 9 of 18
 AI tools for software application developers: Part One

In this e-guide
AI developer toolset series: Mathworks on
Codementor on AI tooling vs deep learning
theory
Adrian Bridgwater, guest contributor
Shutterstock captures the
moment of AI evolution &
Senior engineering manager at data analysis and simulation software company
democratisation MathWorks is Jos Martin.

Martin says that when it comes to deep learning, which is probably the most
Mathworks on deep learning
prevalent form of AI at the moment, much of the complexity has already been
abstracted.
Applitools on AI-assisted root
cause analysis "As a user of deep learning you don't usually need to go and write
training algorithms, since ‘stochastic gradient descent with ****' is
usually what you want. You might want to think about your loss
function, but beyond that you don't really need to think how you
will train your network," said Martin.

He explains that in addition, it is important to remember that there are many

different intelligence layer types already implemented in openly available
libraries, so developers should really view any single layer of intelligence as a
whole set of computing neurons connected together.

Page 10 of 18
 AI tools for software application developers: Part One

Architectural responsibility
In this e-guide
"So, as a user of deep learning you really are piecing together
Codementor on AI tooling vs layers, thinking about the data you will train them on… and then
theory doing the training. I'm not going to underestimate the difficulty of
coming up with appropriate network architectures for the layers –
Shutterstock captures the but since that is problem-dependent, it seem reasonable that
moment of AI evolution & someone who wants to focus on building smarter applications
democratisation needs to think about their deep network architecture," said Martin.

Martin points out that much of the time and effort in developing new deep
Mathworks on deep learning
learning models involves data cleansing, data labelling and data marshalling.

Applitools on AI-assisted root He insists that the training data part of the total equation is something that
cause analysis hugely affects the overall performance of any model and is something that must
be considered a first class citizen alongside the network architecture and
training algorithms.

So talking of responsibilities, is there a responsibility to use open source

frameworks to share the machine knowledge made so that it's a case of deep
learning for all?

Martin says that there may be a responsibility to make sure your model can be
shared across different frameworks, but not a responsibility to use any particular
framework whilst developing your model.

Page 11 of 18
 AI tools for software application developers: Part One

"There is software and deep learning tools which allow both import
In this e-guide and export of our deep networks to the Open Neural Network
Exchange Format (ONNX – https://onnx.ai/). This allows different
Codementor on AI tooling vs frameworks to use models developed in others. Developers also
theory now have the ability to import some model developed in
TensorFlow and reuse the network architecture and some parts of
Shutterstock captures the the model when undertaking transfer learning on a new dataset,"
moment of AI evolution & he said.
democratisation
Martin concludes by saying that the interesting thing that is developed is the
Mathworks on deep learning model and its architecture. He asserts that ONNX is an excellent way for all the
different frameworks to interconnect to allow the right tools to be used in the
Applitools on AI-assisted root right situation.
cause analysis

Next Article

Page 12 of 18
 AI tools for software application developers: Part One

In this e-guide
AI developer toolset series: Applitools on
Codementor on AI tooling vs AI-assisted root cause analysis
theory
Adrian Bridgwater, guest contributor
Shutterstock captures the
moment of AI evolution &
The following text is written by Al Sargent in his role as VP of product marketing
democratisation at Applitools — the company is known for its AI-powered visual testing and
monitoring software capabilities.
Mathworks on deep learning
Sargent discusses AI-assisted root cause analysis…

Applitools on AI-assisted root Visual testing is the act of verifying that a User Interface (UI) appears correctly
cause analysis to its users.

Sometimes confused with functional testing, visual testing helps

ensure that each UI element appears in the right colour, shape,
position and size across any and all viewports, browsers and
device types. Visual testing also ensures that UI elements don't
hide or overlap with each other on the screen.

Because visual testing is difficult to automate, oftentimes these tests are done
manually.

Page 13 of 18
 AI tools for software application developers: Part One

Front-end debugging
In this e-guide
A visual test is comprised of capturing a screenshot, which serves as a baseline
Codementor on AI tooling vs image against which all other changes are compared. Once the baseline is
theory established, developers run test code and whenever a change is detected, an
image of the change is captured. The test runner then compares these images
Shutterstock captures the to the baseline image for this area of the code and if differences are detected
moment of AI evolution & between images, the test fails.
democratisation
After the test code is run and a report is generated, developers need to review
the images that are different than the baseline. Automated visual testing will
Mathworks on deep learning
clearly highlight the visual differences between images – in our own tool
[Applitools Eyes] for example, the visual differences are highlighted in pink. If
Applitools on AI-assisted root
the difference was caused by a bug, developers can fix the bug and rerun the
cause analysis
test to check that the bug was, indeed, fixed. If the difference was caused by a
real change in the UI, developers can review the screenshot and update the
baseline so that future test runs will be passed.

Developers also need to know the differences in their application's

Document Object Model (DOM) and Cascading Style Sheet (CSS)
rules that underpin the visual differences in their app, which is not
always easy to determine. By using automated visual testing
combined with existing developer tests, detecting visual bugs in
the early stages of a development cycle is made possible.

Page 14 of 18
 AI tools for software application developers: Part One

Line by line sifting

In this e-guide
When a developer discovers a visual bug in an app, they have to sift through
Codementor on AI tooling vs line after line of DOM and CSS rules to find the root cause. Getting the DOM
theory and CSS rules for the current version of an app is straightforward, but finding
the baseline DOM and CSS rules can be hard. Developers need to access the
Shutterstock captures the source code management system, fire up the baseline version of their app…
moment of AI evolution & and then once the app builds, they then need to get it into exactly the right state.
democratisation
Only then can a developer grab their baseline DOM and CSS rules, in order to
run their diffs.
Mathworks on deep learning

But doing a simple diff of DOM and CSS rules will turn up many
Applitools on AI-assisted root differences, many of them have nothing to do with a visual bug.
cause analysis Oftentimes, devs will be chasing dead-end leads, which is a
tedious, time-consuming process. Meanwhile, if a developer is
expected to release multiple times per day or week, they have less
time (but more pressure) to fix the bug before the next release.

What if an automated visual testing solution could show the specific DOM
differences likely to cause a visual bug and also show the relevant CSS rule
differences? This would be crucial since it's often CSS that's the root cause of a
visual bug.

Page 15 of 18
 AI tools for software application developers: Part One

[Ed spoiler alert: it sounds like Sargent is lining up to promote and extol the
In this e-guide virtues of his own product here — a good point is being well made here, but the
reader will please note that the writer is explaining a possible (albeit effective)
Codementor on AI tooling vs solution predominantly within the realms of his firm’s own product set.]
theory
AI-assisted root cause analysis
Shutterstock captures the
Applitools Root Cause Analysis shows what DOM and CSS differences
moment of AI evolution &
democratisation underpin each visual difference between a baseline screenshot and a test
screenshot and comprise the ‘root cause’ of the bug. Rather than digging
through potentially thousands of lines of DOM and CSS to find the root cause,
Mathworks on deep learning
developers typically only need to look at a handful of lines.

Applitools on AI-assisted root Relatedly, Applitools has extended its user interface version control so that it
cause analysis now includes DOM and CSS associated with each screenshot. This helps
development teams to see not only how the visual appearance of a web
application has evolved over time, but also how its underlying DOM and CSS
have changed – making it easier to roll back new features.

Applitools presents DOM differences in a similar way to Google Developer

Tools and CSS differences similar to GitHub, so you don't need to waste any
time figuring anything out.

Visual differences are displayed in Applitools Eyes Test Manager where the
click on a visual difference is highlighted and developers can instantly see what

Page 16 of 18
 AI tools for software application developers: Part One

DOM and CSS rules are related to that change. From here, devs will receive a
In this e-guide link to the exact view you're looking at - sort of like a blog post's permalink,
which you can add to your Jira bug report, Slack, or email.
Codementor on AI tooling vs
theory

Shutterstock captures the

moment of AI evolution &
democratisation

Mathworks on deep learning

Applitools on AI-assisted root

cause analysis

Page 17 of 18
 AI tools for software application developers: Part One

In this e-guide
Getting more CW+ exclusive content
Codementor on AI tooling vs
As a CW+ member, you have access to TechTarget’s entire portfolio of 140+
theory
websites. CW+ access directs you to previously unavailable “platinum members-
only resources” that are guaranteed to save you the time and effort of having to
Shutterstock captures the
track such premium content down on your own, ultimately helping you to solve
moment of AI evolution &
democratisation
your toughest IT challenges more effectively—and faster—than ever before.

Mathworks on deep learning

Take full advantage of your membership by visiting
Applitools on AI-assisted root
www.computerweekly.com/eproducts
cause analysis Images; stock.adobe.com

© 2019 TechTarget. No part of this publication may be transmitted or reproduced in any form or by any means without
written permission from the publisher.

Page 18 of 18