Holistic Model
of Sound Perception
THOST
May 2015
Presented by: Vedanta Sutra dasa
Ideas by: HG Lila Purushottama dasa
HG Rasaraja dasa
Semantic Processing
Artificial general intelligence (AGI):
a machine can successfully perform any intellectual task that
a human being can.
from understanding language to writing award-winning books
from comprehending music to composing bandishes in different
ragas
Source: Wikipedia
Semantic Processing
What are our computers doing ??
At present, digital computers only perform token manipulation
(0/1 bits).
It does not lead to understanding or cognition.
The computers manipulate the symbols using a form of syntax
rules, without any knowledge of the symbol's semantics (that is,
their meaning).
Source: Gomatam R., 2009
State of the Art - what is lacking?
Computers can perform complex computations at lightening
speeds. E.g.,
finding logarithms
computing square roots
optimization using gradient algorithms
They do not feel exhausted or bored by this labour
State of the Art - what is lacking?
Computers can perform complex computations at lightening
speeds. E.g.,
finding logarithms
computing square roots
optimization using gradient algorithms
They do not feel exhausted or bored by this labour
But they are only performing bit operations, they way they
have been programmed
State of the Art - what is lacking?
What about AI ?
Automatic temperature control
Automatic OCR (optical character recognition)
Adaptive signal processing
Automatic music transcription
GPS navigation
State of the Art - what is lacking?
What about AI ?
Automatic temperature control
Automatic OCR (optical character recognition)
Adaptive signal processing
Automatic music transcription
GPS navigation
Only a bit more sophisticated programming
They are only doing as they have been programmed
State of the Art - what is lacking?
Automatic OCR (optical character recognition)
pixels of 0/1
learn features which are again 0/1
Source: CS, Boston University
State of the Art - what is lacking?
Hierarchy of simple computational models give rise to complex
understanding
State of the Art - what is lacking?
Hierarchy of simple computational models give rise to complex
understanding
This is REDUCTIONISM, an old argument
It has brought some empirically successful systems upto
certain accuracy, but no understanding of understanding
itself
State of the Art - what is lacking?
What about a computer program defeating a human
grandmaster in playing chess?
Source: Deutsch D., 2012
State of the Art - what is lacking?
What about a computer program defeating a human
grandmaster in playing chess?
The algorithms used by human and computer are not even
remotely similar.
The grandmaster can give exciting reasons for his steps and
can write a bestseller book on the game. But a computer does
not even know what it is doing apart from following a set of
instructions.
Source: Deutsch D., 2012
Semantic Processing
But no brain on Earth is yet close to knowing what brains do in
order to achieve any of that functionality. The enterprise of
achieving it artificially the field of artificial general intelligence
or AGI has made no progress whatever during the entire six
decades of its existence.
Source: Deutsch D., 2012
Is it possible?: two views
Machine can only act like it thinks and has a mind
AGI is not possible
Machine can think and have a mind
AGI requires a major breakthrough
I call the core functionality in question creativity: the
ability to produce new explanations
Source: Deutsch D., 2012
Goals of Semantic Processing
Creativity: the ability to produce new explanations
Causal Reasoning: for taking decisions
Current Sound Processing
We sing
Air column vibrates
Diaphragm of microphone
vibrates
Vibration converted to
electric signal
Electric signal moves the coil
Diaphragm of speaker
vibrates
Air column vibrates
We hear
Current Sound Processing
We are processing in the visual domain, not the sound domain
[Sir]
Even touch is playing an intermediary role
Corollary: frequency, phase are an artifact
Can we process sound in the sound domain?
Law of the Instrument:
if all you have is a hammer, everything looks like a nail
- Abraham Maslow
Quantifying Experiences
Experience is irreducible/holistic
Modern science tries to reduce it to elementary entities. Why?
Quantifying Experiences
Experience is irreducible/holistic
Modern science tries to reduce it to elementary entities. Why?
May be the main motivation for this is to be able to QUANTIFY
the experiences.
Quantification gives better
understanding (notes of music/raga)
portability (reproducing the experience)
control (manipulability)
E.g., the experience of seeing an elephant can be represented as numerical values
corresponding to color, shape, size, etc. Once this is done, it opens the world of unlimited ways
this information can be utilized, be it for image processing, object detection, tracking,
animation and so on.
A Gap in the Understanding
The machine can do speech recognition, then how are we
different?
Machine can do symbol manipulation, not semantic one.
It can recognize symbols/shapes, but it does not know what to
recognize; we have to tell it what to look for.
But humans can automatically detect what to recognize or
learn, because they know the meaning associated
E.g. One can speak a sentence in many ways - low pitch, high
pitch, whisper, alaryngeal etc. But no one has to train me how
to make sense out of it. But a system has to be trained
separately that this signal (symbol) corresponds to this
phoneme (another symbol)
Humans can reach meaning even with fragmentary input
A GAP in the Understanding
The modern sound processing methods can manipulate lower
level handles like frequency, phase, etc.
Music theory can manipulate/explain higher level concepts like
instrumentation, pitch, etc.
There is a big gap in between
No single theory can explain all the levels coherently
Goals of Semantic Processing
Creativity: the ability to produce new explanations
Causal Reasoning: for taking decisions
Single theory for explaining different levels of perception
E.g. a note is important because it gives a different form to the
composition, and not because of statistics of notes
How is it possible?
Form vs Substance
Every instrument has a form which identifies it. This form
manifests through the audio that we hear.
A message has a form which manifests through the alphabets
written on paper
Form is simultaneously same and different from the substance
Modern science is good in manipulating substance, but no idea
of form
Source: Gomatam R., 2014
How is it possible?
Hierarchy of form
Alphabet, word, sentence, article, idea
Note, composition, raga, idea/mood
Similar to the levels of sabda in Vedic understanding
The higher level guides the lower level, not the other way
round
Modeling the form
Let the form be modeled by a mathematical entity called infon
Infon carries the information from the source to the perceiver
Modeling the form
Let the form be modeled by a mathematical entity called infon
Infon carries the information from the source to the perceiver
SOURCE
INFON
PROJECTED FROM
PERCEPTUAL
SPACE
OBSERVER
GROSS MEDIA
CARRYING INFON
INFON
PROJECTED TO
PERCEPTUAL
SPACE
Modeling the form
An Infon is a relationship - between different objects, sense
experiences
Model (MQM?)
infon as a wave function
information transfer as projection operator
INFORMATION is a scalar measure associated with an INFON
Modeling the Experience
Earth, water, fire, etc. are gross media to carry infons
The medium has some infon (form)
The source puts its infon onto the medium, resulting in a
different infon
The observer has own perceptual space
The infon in medium gets projected onto the perceptual space
of the observer
How to define Information?
According to the present understanding, information is defined
w.r.t. the quantity the observer is interested in
E.g. in Shannons information, the probabilities are the ones
associated with the patterns the receiver is expecting
E.g. given a green tennis ball
Red vs Green
Tennis vs Ping-pong
Size
Further directions
Dimension of infon?
Can we model it completely?
Conclusion
Present computation (AI) is syntactic, not semantic
AGI requires creativity and causal reasoning
Distinction between substance from form
Model form as infon, which carries information
Experience as projection on perceptual space
Information depends on what the perceiver is interested in
Higher level in the hierarchy guiding the lower level
Bhagavat Sankhya approach of
H.G. Rasaraja Prabhu
Syntactic TOKENS to Semantic SYMBOLS
A different world view: we are interpreting things as TOKENS, but
they can also be conceived of as SYMBOLS
physically characterize macroscopic objects in terms of their
relational properties, as SYMBOLS with semantic content
Source: Gomatam R., 2009
