Intelligent bot for HealthCare

Team Members:

1. Prakash Raj S     (710716205030)

2. Praveen D            (710716205031)
3. VishnuVarthan T(710716205058)
4. Ragul S                (15IT041RO19)
Guided By : Mr.T.VijayaKumar AP/IT
 TABLE OF CONTENTS:
2

 Introduction
 Existing System / Literature Survey
 Problem Statement
 Objectives
 Proposed System
 Result & Discussion
 Conclusion
 Future Enhancement
 Reference

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

3 Introduction

Machine Learning Vs Traditional Programming

 Traditional programming is a manual process—meaning
a person (programmer) creates the program. But without
anyone programming the logic, one has to manually
formulate or code rules.
      If we're writing a game we can usually figure out the
rules. For example, if the ball hits the brick then the brick
should vanish and the ball should rebound. 
      But if the ball falls off the bottom of the screen then
maybe the player loses their life

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

 Machine Learning
 Machine learning rearranges this diagram 
 where we put answers and data in and then
we get rules out. 

 So instead of us as developers figuring out

the rules, take bunch of examples for what we
want to see and then have the computer figure
out the rule.

• So consider this example, activity

recognition. 

• So consider this example, activity

recognition. If I'm building a device that
detects if somebody is say walking and I have
data about their speed,

•  I might write code like this and if they're

running well that's a faster speed so I could
adapt my code to this and if they're
biking, well that's not too bad either. But then
I have to do golf recognition too, now my Machine Learning
concept becomes broken.

4 Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

  So instead of trying to express the problem as rules when often that isn't even
5 possible.
 The new paradigm is that I get lots and lots of examples and then I have labels on
those examples and I use the data to say this is what walking looks like,this is what
running looks like, this is what biking looks like and yes, even this is what golfing
looks like.
 So, then it becomes answers and data in with rules being inferred by the machine. A
machine learning algorithm then figures out the specific patterns in each set of data
that determines the distinctiveness of each.

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

DeepLearning

 Deep Learning is a subfield of

machine learning concerned with
algorithms inspired by the structure
and function of the brain called
artificial neural networks
 Andrew Ng from Coursera and
Chief Scientist at Baidu Research
formally founded Google Brain
that eventually resulted in the
productization of deep learning
technologies across a large number
of Google services.

6 Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

7

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

 Existing System / Literature Survey
8
 Doc Product: Medical Q&A with Deep Language Models
 Quality medical information is valuable to everyone, but it's not always
readily available. Doc Product aims to fix that.
 Doc Product has you covered with up to date information and unique
AI-generated advice to address your medical concerns.
 We wanted to use TensorFlow 2.0 to explore how well state-of-the-art
natural language processing models like BERT and GPT-2 could
respond to medical questions by retrieving and conditioning on relevant
medical data, and this is the result.
 we drew inspiration for our design from various different areas of
machine learning. By combining the power of Transformer
architecture,latent vector search, negative sampling  and generative pre-
training within TensorFlow 2.0's flexible deep learning framework,

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

  700,000 medical questions and answers scraped from Reddit, Health
Tap, WebMD, and several other sites
9  Fine-tuned TF 2.0 BERT with pre-trainef BioBERT weights for
extracting representations from text
 Network heads for mapping question and answer embeddings to
metric space, made with a Keras . Model feedforward network
 Over a terabyte of  CSV data
 What it does
 BERT has been trained to encode medical questions and medical
information. A user can type in a medical question, and our model
will retrieve the most relevant medical information to that question.
 Data
 They created datasets from several medical question and answering
forums. The forums are WebMD, HealthTap, eHealthForums,
iClinic, Question Doctors, and Reddit.com/r/AskDocs

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

10

Architecture

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

11 Problem Statement
 Doctor Source is a biomedical app that lets you chat with an
automated general practice doctor and get a quick diagnosis and
treatment plan for whatever your ailment is.
 The App is built for both iOS and Android using the Flutter
framework, and enables you to use your voice to communicate
with the AI doctor and Book appointment near  by specialist
doctor.
 It's trained on lots of Reddit, HealthTap, and WebMD medical
questions and answers, over 700K of them.
 How does BioBERT work
 How to build a Flutter plugin (iOS + Android)
 How to chain together multiple machine learning models for a
robut result

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

12 Surveillance
 (12 Sep 2019) Our paper has been accepted to Bioinformatics (
link). Please update your BibTex.
 (19 June 2019) Check out our BioBERT based multi-type NER
and normalization model (BERN) with its paper!
 (16 May 2019) A new version of BioBERT (v1.1) is now
available.
 (15 May 2019) Updated code and dataset for BioASQ 7ed3d7a.

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

13 Objectives
 Voice Recognition plugin (custom channel, iOS Speech API,
Android Speech Recognizer) 
 Building a Flutter app
 Increases the accuracy of BERT Model
 Integrating pretrained model into our flutter app 
 How to Improve the App 
 Named Entity Recognition
 All Voice
 More animations in the interface
 Continous training pipeline, keep getting more data from users,
feed it back in
 Robust edge case detection
 Affiliate deals with drug companies 
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
14 Proposed System
 How does BioBERT work
 First of all, when Google released BERT, it was like the ImageNet
moment in computer vision, but this time for natural language processing.
It was a big deal. 
 Now everyone can use the weights file that BERT learned after training on
millions of words as a starting point for their specific NLP task, be it
Q&A, sentence classfication, sentence generation, sentiment analysis, etc.
 Published 5 months ago
 It's a pre-trained biomedical language representation model. The full name
is "BiDirection Encoder Reprensentions from Transformers for
Biomedical Text Mining". -BioBERT outperforms plain old BERT on 3
tasks; Biomedical Named Entity Recognition, Biomedical Relation
Extraction, and Biomedical Question Answering

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

15

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

16

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

17

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

18 Software & Framework

 IBM Watson Studio

 Google Colabs
 Flutter Plugin
 Speech Recognition API
 Visual Studio Code / Android Studio

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

  How to build a Flutter plugin (iOS + Android)
19  Tools Used
 
 App building (Flutter)

User Accounts (Firebase)

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

 Voice Recognition plugin (custom channel, iOS Speech
20 API, Android Speech Recognizer) 

https://pub.dev/packages/speech_recognition
Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22
21 Installation
 Depend on it Add this to your package's pubspec.yaml file:
 dependencies:
   speech_recognition: "^0.3.0"
  
 Install it You can install packages from the command line:
 $ flutter packages get
  
 Import it Now in your Dart code, you can use:
 import 'package:speech_recognition/speech_recognition.dart';


Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

22

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

23 Result & Discussion
https://colab.research.google.com/drive/1xZBvVG-
pQDy8HsEIqDyEKzLKWST4gAYy#scrollTo=Ul_tuaqe0u98

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

24 Conclusion
 AI Model(BioBERT , input question into retrained bioBERT to
convert into an embedding. Input the embedding into a fully
connected neural network, output an embedding for similarity
lookup. Most similar Q&A's used by GPT-2 to generate an
answer. )

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

 Future Enhancement

 Develop a model , which monitor

ECG level , glucose bottle level and
patient behaviour using Computer
vision 
 Develop a model, which
automatically create and update
patient details and treatment for
future references using
Image recognition

25 Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22

26 Reference
 BASE PAPER
 This repository provides fine-tuning codes of BioBERT, a
language representation model for biomedical domain: 
https://github.com/dmis-lab/biobert
 REFERENCE PAPER
 Visualizing and Understanding Neural Models in NLP
 Enriching BERT with knowledge Graph Embeddings for
Document Classification
 Build it for Android and iOS specifically, then call it from
Dartviacreatingamessagechannel.https://medium.com/joytunes/de
ploying-a-tensorflow-model-to-android-69d04d1b0cba

Dr.N.G.P.IT/IT/2019-2020/Project Presentation 3/31/22