Scribd
Upload
58 views12 pages

Deep Learning Methods For Automated Discourse

The document provides an overview of a class on developing dialog systems using deep learning methods. The class will focus on building a socialbot for the Amazon Alexa using TensorFlow and the Alexa Skills Kit. The class will cover interfacing with the Alexa SDK, training chatbots with deep learning, and recent papers on dialog systems. The class format will involve discussions of papers, programming assignments in groups, and a final group project to build a working dialog system. Computing resources like AWS and prerequisite knowledge of Python and machine learning are also discussed.

Uploaded by

Nilesh Chaudhary
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
58 views12 pages

Deep Learning Methods For Automated Discourse

The document provides an overview of a class on developing dialog systems using deep learning methods. The class will focus on building a socialbot for the Amazon Alexa using TensorFlow and the Alexa Skills Kit. The class will cover interfacing with the Alexa SDK, training chatbots with deep learning, and recent papers on dialog systems. The class format will involve discussions of papers, programming assignments in groups, and a final group project to build a working dialog system. Computing resources like AWS and prerequisite knowledge of Python and machine learning are also discussed.

Uploaded by

Nilesh Chaudhary
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 12

 Deep Learning Methods

For Automated Discourse


CIS 700-7
Fall 2017
http://dialog-systems-class.org/

João Sedoc with Chris Callison-Burch and Lyle Ungar


[email protected]

January 12th, 2017


Alexa Prize Motivation
Main arc of this class is to develop a socialbot which will be an Alexa skill
that allows a dialog between users and the Alexa device

The Amazon Alexa Prize defines it as “socialbot will be an Alexa skill that
converses coherently and engagingly with humans on popular topics and news
events. ”

This course is comprized of 3 parts:


-learning how to interface with Alexa SDK
-learning tensorflow and how to train (particularly also get the data and manipulate
it) and create a chatbot
-Developing a foundation in deep learning and dialog systems and subsequently
recent papers for dialog systems
About the class
• This class is in “alpha” form, so you will be the first to try out the
assignments
• We intend of the class to be group based and collaborative
• Discussion about papers and implementation problems is a large part
of this class, not just programming
• The syllabus is still a bit fluid
• The readings will be posted at the end of next week
• BRING YOUR COMPUTERS TO CLASS
Prerequisites for the class
• CIS 419/519 or CIS 520
• Proficiency in Python
Grading
• Credit for reading material
• 1 day before class post 3 or more sentences about the paper and 3 questions
on Piazza
• Leading a paper discussion
• Leading the discussion
• Writing a synopsis of the paper and how it relates, and what we discussed
• Write up of the algorithm from the paper in your own words
• Programming assignments – groups ~ 3 people
• Final Project [2 intermediate checkpoints] – group project
• 6 page paper
• Working systems
Class Format
• 50-60 minutes of discussion of papers
• Understanding the key concepts of dialog systems
• Learning to critically read papers and analyze results
• 20-30 minutes of active programming
• Group discussion of common problems
• Relating readings back to code and trying to incorporate concepts into
systems
• Debugging and making sure that we are all moving forward
Computing Resources
• AWS Lambda
• AWS GPU server
• Amazon Echo

• TensorFlow
• Alexa Skills Kit (ASK)

• Git!
Please fill out the
questionnaire at

http://www.seas.upenn.edu/~joao/q
c7.html
Taxonomy of Dialog Systems
• RETRIEVAL-BASED VS. GENERATIVE MODELS
• Retrieval-based models (easier) use a repository of predefined responses and some
kind of heuristic to pick an appropriate response based on the input and context.
• Generative models (harder) don’t rely on pre-defined responses. They generate new
responses from scratch. 
• LONG VS. SHORT CONVERSATIONS
• Short-Text Conversations (easier) where the goal is to create a single response to a
single input.
• long conversations (harder) where you go through multiple turns and need to keep
track of what has been said.
• OPEN DOMAIN VS. CLOSED DOMAIN
• open domain (harder) where the user can take the conversation anywhere. 
• closed domain (easier) where only a specific topic is covered. (Shopping assistant)

From: http://www.wildml.com/2016/04/deep-learning-for-chatbots-part-1-introduction/
Deep Learning Models
• Sequence to Sequence Model
• Attention and Memory augmentation
• Context Sensitive Models
• Hierarchical extension
• Reinforcement Learning
Basic Sequence to Sequence Model
Idea
OK, let’s start to sign up
for AWS

You might also like