3/3/2019 What were the most significant machine learning/AI advances in 2018?

- Quora

Search for questions, people, and topics Sign In

Technology Trends Artificial Intelligence Computer Science Machine Learning Related Questions

What were the most significant machine learning/AI What would be the most significant machine
advances in 2018? learning/AI advances in 2019?

What have been the most significant machine

Ad by Lambda Labs learning advancements of this year (2014)?
ML workstations — fully configured. Let us save you the work.
What is the most advanced machine in the
Our machine learning experts take care of the set up. We are trusted by Amazon, world?
Tencent, and MIT.
What are AI and machine learning specialists?
Learn more at lambdal.com
Is Google Assistant the most advanced AI?

6 Answers Where can I learn machine learning and AI for

free?

Thia Kai Xin, Head of Data (Tech In Asia), Co-Founder of DataScience SG. What's the most comprehensive training on
Updated Dec 30, 2018 · Upvoted by Jeremy Hadfield, B.S. Philosophy & Computer Machine Learning?
Science, Dartmouth College (2021) and Vishal Sharma, PhD Candidate Computer
Science, Utah State University (2020) · Author has 60 answers and 606.4k answer views Are the AI/Machine learning posts at Amazon well
2018 was an exciting year for ML/AI. We see “smarter” AI, real-world applications, paid?

improvements in underlying algorithms and a greater discussion on the impact of AI

on human civilization. Here are some highlights: In other languages

AlphaZero, now peer-reviewed and working across three different board games , En français : Quelles ont été les avancées les plus
showcases the flexibility of deep reinforcement learning . We have been building significatives de l'intelligence artificielle/machine
learning en 2018 ?
models to follow historical trends and make predictions on what will happen based
on past data. Now, we have machines that can observe the environment, learn the
“unspoken rules” of the environment and adapt its actions to explore & exploit
the environment like a human ; just don’t hold your breath for artificial general
intelligence (yet).

AI in 2018 is not just about games and we now see real-world applications. In
healthcare, deep learning models can perform as well as a human expert in analyzing
electron microscopy or in detecting eye diseases . For the environment and
climate, AI is helping to build better climate models , mapping millions of solar
roofs in the US , detecting hyperspectral signatures of underwater objects to
monitor ocean health & many other animal conservation works. The open source
community is also contributing to interesting projects .

The common theme across the applications above is the drastic improvements in
computer vision (CV) and natural language processing (NLP) in 2018.

Deep learning has worked very well for the ImageNet classification challenge
where we improved from 26.2% error rate with SIFT (late 90s) to 15.3% error rate on
AlexNet (2012) and recently to 2.25% with SE-ResNet (2017). Now, object detection
(pictured above), a classic problem can be handled in real time by You Only Look

https://www.quora.com/What-were-the-most-significant-machine-learning-AI-advances-in-2018 1/10
3/3/2019 What were the most significant machine learning/AI advances in 2018? - Quora
Once (YOLO) and its derivative models or more accurately by Single-Shot Detector
Search for questions, people, and topics Sign In
(SSD) .

Image segmentation (pictured above) used to be a challenging problem, especially for

complex images where the boundaries of objects overlap. The state of the art deep
learning models like Path Aggregation Network (PANet), Context Encoding Network
(EncNet) builds on many previous ideas, with an improved feature extractor and
context information.

Of course, we need to include BigGAN ; it is like GAN , but bigger and more than 2x
better than the previous state of the art in image synthesis. AI can now generate
realistic fake faces , fake videos (DeepFakes) and is accurate enough to catch a
criminal among a crowd of 50,000 people in the middle of a concert.

It was also a big year for NLP , beginning with Jeremy Howard and Sebastian
Ruder’s ULMFiT that brought transfer learning at scale to NLP; this is a big deal,
as current language models are mostly language specific. While I was working in a
regional eCommerce firm in Southeast Asia, my team had to build a model to handle
English, another to handle Thai, Vietnamese, Bahasa…it gets exponentially more
difficult and costly as we try to cater to the linguistics of different languages. Transfer
learning allows us to bring what the model learned about English into Bahasa, saving
time and money.

ELMo and BERT bring in the idea of context into word embeddings. In the image
above, the word “play” is a noun in this particular context and carries a different
meaning from “playing” or “to play” in other contexts. ELMo and BERT will be able to

https://www.quora.com/What-were-the-most-significant-machine-learning-AI-advances-in-2018 2/10
3/3/2019 What were the most significant machine learning/AI advances in 2018? - Quora
separate and keep track of the various meanings of the same word and is thus far
Search for questions, people, and topics Sign In
more precise than the vector-based models that assign a single meaning to the word.

But what does all these AI development mean? Now that there are legal AIs that can
beat lawyers from Goldman Sachs, does it mean that AI will eventually replace all of
us? Interestingly, the lawyers are relieved that AI can take care of the mundane,
repetitive work like NDAs so that they can focus on more complex, value-adding
work. Even so, AI will likely replace many mundane tasks soon and people are
debating policies to help those affected by the AI future. Furthermore, we also see
the rise of data privacy and AI issues with Europe’s GDPR , Cambridge Analytica &
Facebook and Uber’s self-driving car crash . AI is powerful and had definitely
grown in 2018; however, it is still far from maturity. We may see explosive growth in
areas like IOT + AI, blockchain + AI and more in 2019; I can’t wait to see what we will
create next.

20.8k Views · View 181 Upvoters · View Sharers

Sponsored by Springboard
Build your data science skills with advice from expert mentors.
Kickstart your data science career. Learn from a curriculum designed by industry
experts.

Learn more at datascience.springboard.com

Related Questions More Answers Below

What would be the most significant machine learning/AI advances in 2019?

What have been the most significant machine learning advancements of this year (2014)?

What is the most advanced machine in the world?

What are AI and machine learning specialists?

Is Google Assistant the most advanced AI?

Viral Shah, Co-creator

Answered Dec 23, 2018

In 2017, in the Julia community, we surveyed machine learning/AI and wrote a blog
post on our findings - On Machine Learning and Programming Languages . We
concluded in the post:

Machine learning models have become extremely general information-processing

systems that build ever higher-level and more complex abstractions; recurrence,
recursion, higher-order models, even stack machines and language interpreters, all
implemented as compositions of basic components. ML is a new programming
paradigm, albeit a strange one that’s heavily numerical, differentiable and parallel.

In 2018, we focussed on building capabilities in Julia to support machine learning

natively. We articulated our progress in a blog post - Building a Language and
Compiler for Machine Learning .

We presented our progress in 4 posters at the MLSYS workshop at NeurIPS 2018. A

summary of our progress in 2018 is as follows:

1. A focus on differentiable programming by making AD work across the

language. A lot of this work was carried out in Julia compiler improvements
for 1.0 and in Cassette.jl , for injecting the Julia compiler with new,
context-specific behaviors. Packages such as Zygote.jl leverage these
capabilities to implement source-to-source AD in Julia .

2. Targeting hardware accelerators natively. Julia can natively run on GPUs

by generating PTX through the CUDAnative.jl package. Dynamic
automatic differentiation of Broadcast GPU kernels demonstrated

https://www.quora.com/What-were-the-most-significant-machine-learning-AI-advances-in-2018 3/10
3/3/2019 What were the most significant machine learning/AI advances in 2018? - Quora
significant performance improvements over existing tools. On TPUs, Julia
Search for questions, people, and topics Sign In
leverages the XLA compiler through the XLA.jl package.

3. Packaging these capabilities in Julia frameworks for researchers and users.

The Fashionable modeling with Flux paper describes the Flux.jl
framework, which is Julia all the way down. The Knet.jl framework
leverages a few hand written CUDA kernels, and is one of the fastest deep
learning frameworks.

2.8k Views · View 20 Upvoters · View Sharers

Sponsored by O'Reilly Media

Do you want to meet the world's leading data experts?
Join thousands of top data scientists, engineers, and executives March 25-28 at Strata
in San Francisco.

Learn more at conferences.oreilly.com

David Vandegrift, Co-Founder / CTO at 4Degrees (2017-present)

Answered Jan 5, 2019 · Upvoted by Anand Prashar, Masters Data Informatics &
Computer Science, University of Southern California (2018) · Author has 312 answers and
623.3k answer views
OpenAI Five - OpenAI fielded a reinforcement learning AI that competed against top
professional teams in the game of Dota2. Although OpenAI did not win, it did beat
slightly less good players and made a good showing. This marks the next significant
step in AI’s gameplaying abilities (one of the most watched benchmarks of AI’s
general intelligence). Chess was a watershed moment in 1997. Go was huge in
2015/2016. Competitive strategic videogames promises to be another such step
change improvement. This event was also significant because it marks the entrance of
OpenAI to the real world of releasing AI. Up until now, they had only developed
ecosystems and published papers.

Google DuplexAI - With great publicity, Google unveiled its DuplexAI, a spoken
language bot that can call up businesses and conduct transactions for you (like
booking a hair appointment). DuplexAI represents substantial breakthroughs in
voice-to-text, text-to-voice, and agent action execution. DuplexAI’s announcement
sparked a furor of conversation about the ethics of AI posing as a human in a
conversation.

DeepFakes went mainstream - Generative models like GANs and VAEs created their
first scandal, when someone anonymously released a series of porn videos online
with famous actress’ faces stitched into scenes. This is the inflection point for the
technology that has been developing for 4 years now; everyone knew it was coming,
but this marks the first real public issue.

Corporates began focusing on ethics - Led by Google, the corporate tech world
appears to be pumping the breaks a little on their breakneck pace of AI development.
Google committed to having DuplexAI always identify itself as a bot. It also withdrew
a bid for a major government contract over employees’ concerns about its ethics. IBM
released a tool for identifying bias in algorithms. Uber brought its autonomous
vehicle testing to a halt after striking and killing a pedestrian.

Advances in generative models - Teams around the world continue to work on

improving GAN, VAE, and other model capabilities to generate images, video, and
audio from scratch. Generated media is not yet completely convincing at high fidelity,
but it seems that every week brings us one step closer.

Advances in transfer learning - The industry’s biggest success in tackling the big
data requirements of deep learning has been in transfer learning: training a model on
one set of data and then adapting it to another, hopefully with a lower data
requirement to achieve high levels of accuracy. This year saw significant progress in

https://www.quora.com/What-were-the-most-significant-machine-learning-AI-advances-in-2018 4/10
3/3/2019 What were the most significant machine learning/AI advances in 2018? - Quora
the technology, with applications proven across a number of different types of data
Search for questions, people, and topics Sign In
sets.

Advances in one-shot learning - One of the biggest criticisms of deep learning is

that it takes tens of thousands of data points (or more) to generate highly accurate
outputs. Critics compare this to the ability of humans to learn substantially after
observing the correct outcome a single time. Work continued this year on this
capability (know as “one shot learning” or “few shot learning”). While there were no
major breakthroughs, progress was made and there is cause for optimism that
researchers will be able to address this problem.

1.2k Views · View 7 Upvoters · Answer requested by Jim Elias

Xavier Amatriain, Former ML researcher, now leading Engineering teams

Updated Dec 25, 2018 · Featured on Forbes · Upvoted by Vishal Sharma, PhD Candidate
Computer Science, Utah State University (2020) and Dmitriy Genzel, Ph.D. Computer
Science, Brown University (2005) · Author has 303 answers and 6.5m answer views
If I had to summarize the main highlights of machine learning advances in 2018 in a
few headlines, these are the ones that I would probably come up:

AI hype and fear mongering cools down

More focus on concrete issues like fairness, interpretability, or causality

Deep learning is here to stay and is useful in practice for more than image
classification (particularly for NLP)

The battle on the AI frameworks front is heating up, and if you want to be
someone you better publish a few frameworks of your own

Let’s look at all of this in some more detail.

If 2017 was probably the cusp of AI fear mongering and hype (as I mentioned in last
year’s answer), 2018 seems to have been the year where we have started to all cool
down a bit. While it is true that some figures have continued to push their message of
AI fear, they have probably been too busy with other issues to make of this an
important point of their agenda . At the same time, it seems like the press and others
have come at peace with the idea that while self-driving cars and similar technologies
are coming our way, they won’t happen tomorrow. That being said, there are still
voices defending the bad idea that we should regulate AI instead of focusing on
regulating its outcomes.

It is good to see that this year though, the focus seems to have shifted to more
concrete issues that can be addressed. For example, there has been a lot of talk
around fairness and there are not only several conferences on the topic (see
FATML or ACM FAT ) even some online courses like this one by Google.

https://www.quora.com/What-were-the-most-significant-machine-learning-AI-advances-in-2018 5/10
3/3/2019 What were the most significant machine learning/AI advances in 2018? - Quora

Search for questions, people, and topics Sign In

Along these lines, other issues that have been greatly discussed this year include
interpretability, explanations, and causality. Starting with the latter, causality
seems to have made it back to the spotlight mostly because of the publication of
Judea Pearl’s “The Book of Why”. Not only the author decided to write his first
“generally accessible” book, but he also took to Twitter to popularize discussions
around causality. In fact, even the popular press has written about this as being a
“challenge” to existing AI approaches (see this article in The Atlantic , for example).
Actually, even the best paper award at the ACM Recsys conference went to a paper
that addressed the issue of how to include causality in embeddings (see “Causal
Embeddings for Recommendations ”). That being said, many other authors have
argued that causality is somewhat of a theoretical distraction, and we should focus
again on more concrete issues like interpretability or explanations. Speaking of
explanations, one of the highlights in this area might be the publication of the paper
and code for Anchor , a follow up to the well-known LIME model by the same
authors.

https://www.quora.com/What-were-the-most-significant-machine-learning-AI-advances-in-2018 6/10
3/3/2019 What were the most significant machine learning/AI advances in 2018? - Quora

Search for questions, people, and topics Sign In

While there are still questions about the Deep Learning as the most general AI
paradigm (count me in with those raising questions), while we continue to skim over
the nth iteration of the discussion about this between Yann LeCun and Gary Marcus,
it is clear that Deep Learning is not only here to stay, but it is still far from having
reached a plateau in terms of what it can deliver. More concretely, during this year
Deep Learning approaches have shown unprecedented success in fields different
from Vision, ranging from Language to Healthcare.

In fact, it is probably in the area of NLP, where we have seen the most interesting
advances this year. If I had to choose the most impressive AI applications of the year,
both of them would be NLP (and both come from Google). The first one is Google’s
super useful smart compose , and the second one is their Duplex dialog system.

A lot of those advances have been accelerated by the idea of using language models,
popularized this year by Fast.ai ’s UMLFit (see also “Understanding UMLFit ”).
We have then seen other (and improved) approaches like Allen’s ELMO , Open AI’s
transformers , or, more recently Google’s BERT , which beat many SOTA results
out of the gate. These models have been described as the “Imagenet moment for NLP”
since they show the practicality of transfer learning in the language domain by
providing ready-to-use pre-trained and general models that can be also fine-tuned for
specific tasks. Besides language models, there have been plenty of other interesting
advances like Facebooks multilingual embeddings , just to name another one. It is
interesting to note that we have also seen how quickly these and other approaches
have been integrated into more general NLP frameworks such as AllenNLP’s or
Zalando’s FLAIR .

Speaking of frameworks, this year the “war of the AI frameworks” has heated up.
Surprisingly, Pytorch seems to be catching up to TensorFlow just as Pytorch 1.0 was

https://www.quora.com/What-were-the-most-significant-machine-learning-AI-advances-in-2018 7/10
3/3/2019 What were the most significant machine learning/AI advances in 2018? - Quora
announced . While the situation around using Pytorch in production is still sub-
Search for questions, people, and topics Sign In
optimal, it seems like Pytorch is catching up on that front faster than Tensor Flow is
catching up on usability, documentation, and education. Interestingly it is likely that
the choice of Pytorch as the framework on which to implement the Fast.ai library
has played a big role. That being said, Google is aware of all of this and is pushing in
the right direction with the inclusion of Keras as a first-class citizen in the
framework or the addition of key developer-focused leaders like Paige Bailey . At the
end, we all benefit from having access to all these great resources, so keep them
coming!

Interestingly, another area that has seen a lot of interesting developments in the
framework space is reinforcement learning. While I don’t think RL research
advances have been as impressive as in previous years (only the recent Impala work
by DeepMind comes to mind), it is surprising to see that in a single year we have seen
all major AI players publish an RL Framework. Google published the Dopamine
framework for research while Deepmind (also inside of Google) published the
somewhat competing TRFL framework. Facebook could not stay behind and
published Horizon while Microsoft published TextWorld , which is more
specialized for training text-based agents. Hopefully, all of this open source goodness
will help us see a lot of RL advances in 2019.

Just to finish up on the frameworks front, I was happy to see that Google recently
published TFRank on top of Tensor Flow. Ranking is an extremely important ML
application that is probably getting less love than it deserves lately.

It might seem like Deep learning has ultimately removed the need to be smart about
your data, but that is far from true. There are still very interesting advances in the
field that revolve about the idea of improving data. For example, while data
augmentation has been around for some time and is key for many DL applications,
this year Google published auto-augment , a deep reinforcement learning approach
to automatically augment training data. An even more extreme idea is to train DL
models with synthetic data. This has been tried in practice for some time and is seen
as key to the future of AI by many. NVidia presented interesting novel ideas in their
Training Deep Learning with Synthetic Data paper. In our “Learning from the
Experts ”, we also showed how to use expert systems to generate synthetic data that
can then be used to train DL systems even after combining with real-world data.
Finally, also interesting is the approach of reducing the need to have large quantities
of hand-labelled data by using “weak supervision”. Snorkel is a very interesting
project that aims at facilitating this approach by providing a generic framework.

As far as more foundational breakthroughs in AI, it might be me and my focus, but

I haven’t seen many. I don’t entirely agree with Hinton when he says that this lack of
innovation is due to the field having “a few senior guys and a gazillion young guys ”
although it is true that there is a trend in science where breakthrough research is done
at a later age . In my opinion, the main reason for the current lack of breakthroughs
is that there are still many interesting practical applications of existing approaches

https://www.quora.com/What-were-the-most-significant-machine-learning-AI-advances-in-2018 8/10
3/3/2019 What were the most significant machine learning/AI advances in 2018? - Quora
and variations so it is hard to risk in approaches that might not be practical right
Search for questions, people, and topics Sign In
away. That is even more relevant when most of the research in the field is sponsored
by large companies. In any case, an interesting paper that does challenge some
assumptions is “An Empirical Evaluation of Generic Convolutional and Recurrent
Networks for Sequence Modeling ”. While being highly empirical and using known
approaches, it opens the door to uncovering new ones since it proves that the one that
is usually regarded as optimal is in fact not. Another highly exploratory paper is the
recent NeurIPS best paper award winner “Neural Ordinary Differential Equations ”,
which challenges a few fundamental things in DL including the notion of layers itself.

Interestingly, this last paper was motivated by a project where the authors were
looking into healthcare data (more concretely Electronic Health Records). I cannot
finish this summary without referring to the area of research in the intersection of AI
and Healthcare since that is where my focus at Curai is at. Unfortunately, so much is
going on in this space that I would need another post only for this. So, I will only point
you to the papers that were published at the MLHC conference and the ML4H
NeurIPS workshop . Our team at Curai managed to get papers accepted at both, so
you will find our papers there among many other interesting ones that should give
you an idea of what is going on in our world.

61.1k Views · View 1.1k Upvoters · View Sharers

Rajesh Sampathkumar, Envisioned, and helped build and implement AI

systems
Answered Jan 30, 2019 · Author has 1k answers and 1.3m answer views

If I were to summarize in a few lines:

1. Complex deep learning architectures: BERT, bidirectional models of various

kinds

2. Automated machine learning frameworks such as Adanet, which are an

improvement over AutoML and AutoKeras

3. Improvements in feature engineering automation and data augmentation

automation (specifically, AutoAugment)

70 Views

Peter Charlot, Wrote a book on Artificial Being (AB)

Answered Jan 24, 2019 · Author has 7.7k answers and 8.3m answer views

This blew my mind. It is well worth watching and will answer your question big time.
As to 2018? Obviously what is discussed was operant in 2018, yet I it seems to have
been evolving for sometime.

Peter Charlot's answer to What jobs are AI performing currently?

70 Views · View 1 Upvoter · Answer requested by Johirul Islam and Jim Elias

Related Questions

Where can I learn machine learning and AI for free?

What's the most comprehensive training on Machine Learning?

Are the AI/Machine learning posts at Amazon well paid?

Which tech company currently has the most advanced AI/machine learning?

Is it possible to write a computer program that would beat the Turing test and make people
think they were talking to humans not machines?

How advanced are we in AI right now?

Do you believe scientists will be able to read people's minds with machines in the future?

https://www.quora.com/What-were-the-most-significant-machine-learning-AI-advances-in-2018 9/10
3/3/2019 What were the most significant machine learning/AI advances in 2018? - Quora
Will machine learning and AI destroy the demand for CFAs?
Search for questions, people, and topics Sign In
Why are doctors necessary in the age of Google and AI?

How will AI/machine learning “help” society?

What were the most significant Natural Language Processing advances in 2018?

Can deep learning be called advanced machine learning?

Will Las Vegas become an extinct industry, when AI and machine learning burst the gaming
industry?

What are the most significant machine learning advances in 2017?

Will most Python jobs be about machine learning and AI?

About · Careers · Privacy · Terms · Contact

https://www.quora.com/What-were-the-most-significant-machine-learning-AI-advances-in-2018 10/10