Advanced Data Analytics Using Python: With Architectural Patterns, Text and Image Classification, and Optimization Techniques [2nd Edition] Sayan Mukhopadhyay download pdf
Advanced Data Analytics Using Python: With Architectural Patterns, Text and Image Classification, and Optimization Techniques [2nd Edition] Sayan Mukhopadhyay download pdf
com
OR CLICK HERE
DOWLOAD NOW
https://ebookmeta.com/product/advanced-analytics-with-transact-sql-
exploring-hidden-patterns-and-rules-in-your-data-1st-edition-dejan-
sarka/
ebookmeta.com
https://ebookmeta.com/product/advanced-analytics-with-transact-sql-
exploring-hidden-patterns-and-rules-in-your-data-1st-edition-dejan-
sarka-2/
ebookmeta.com
https://ebookmeta.com/product/international-business-fourth-edition-
yadong-luo/
ebookmeta.com
Services Liberalization in ASEAN Foreign Direct Investment
in Logistics 1st Edition Tham Siew Yean
https://ebookmeta.com/product/services-liberalization-in-asean-
foreign-direct-investment-in-logistics-1st-edition-tham-siew-yean/
ebookmeta.com
https://ebookmeta.com/product/entranced-guardian-academy-1-1st-
edition-jessica-sorensen/
ebookmeta.com
https://ebookmeta.com/product/chocolate-truffles-and-trouble-pacific-
cove-cozy-mystery-03-samantha-silver-et-el/
ebookmeta.com
https://ebookmeta.com/product/eloquent-javascript-3rd-edition-a-
modern-introduction-to-programming-marijn-haverbeke/
ebookmeta.com
Advanced Data
Analytics Using
Python
With Architectural Patterns,
Text and Image Classification,
and Optimization Techniques
Second Edition
Sayan Mukhopadhyay
Pratip Samanta
Advanced Data Analytics Using Python: With Architectural Patterns, Text
and Image Classification, and Optimization Techniques
v
Table of Contents
Normal Forms�����������������������������������������������������������������������������������������������������31
First Normal Form�����������������������������������������������������������������������������������������31
Second Normal Form�������������������������������������������������������������������������������������32
Third Normal Form����������������������������������������������������������������������������������������33
Elasticsearch�������������������������������������������������������������������������������������������������������35
Connection Layer API�������������������������������������������������������������������������������������38
Neo4j Python Driver��������������������������������������������������������������������������������������������39
neo4j-rest-client�������������������������������������������������������������������������������������������������39
In-Memory Database������������������������������������������������������������������������������������������40
MongoDB (Python Edition)����������������������������������������������������������������������������������40
Import Data into the Collection����������������������������������������������������������������������41
Create a Connection Using pymongo�������������������������������������������������������������42
Access Database Objects������������������������������������������������������������������������������42
Insert Data�����������������������������������������������������������������������������������������������������43
Update Data���������������������������������������������������������������������������������������������������43
Remove Data�������������������������������������������������������������������������������������������������43
Cloud Databases�������������������������������������������������������������������������������������������������43
Pandas����������������������������������������������������������������������������������������������������������������44
ETL with Python (Unstructured Data)������������������������������������������������������������������45
Email Parsing�������������������������������������������������������������������������������������������������45
Topical Crawling��������������������������������������������������������������������������������������������48
Summary������������������������������������������������������������������������������������������������������������52
vi
Table of Contents
Semi-Supervised Learning���������������������������������������������������������������������������������65
Decision Tree�������������������������������������������������������������������������������������������������������66
Which Attribute Comes First?������������������������������������������������������������������������66
Random Forest Classifier������������������������������������������������������������������������������68
Naïve Bayes Classifier�����������������������������������������������������������������������������������������68
Support Vector Machine��������������������������������������������������������������������������������������69
Nearest Neighbor Classifier��������������������������������������������������������������������������������71
Sentiment Analysis���������������������������������������������������������������������������������������������71
Image Recognition����������������������������������������������������������������������������������������������73
Regression with Python���������������������������������������������������������������������������������74
Least Square Estimation��������������������������������������������������������������������������������75
Logistic Regression���������������������������������������������������������������������������������������76
Classification and Regression�����������������������������������������������������������������������������76
Intentionally Bias the Model to Over-Fit or Under-Fit������������������������������������������77
Dealing with Categorical Data�����������������������������������������������������������������������������78
Summary������������������������������������������������������������������������������������������������������������79
vii
Table of Contents
viii
Table of Contents
ix
Table of Contents
Index�������������������������������������������������������������������������������������������������243
x
About the Authors
Sayan Mukhopadhyay has more than 13 years
of industry experience and has been associated
with companies such as Credit Suisse, PayPal,
CA Technologies, CSC, and Mphasis. He has
a deep understanding of applications for
data analysis in domains such as investment
banking, online payments, online advertising,
IT infrastructure, and retail. His area of
expertise is in applying high-performance
computing in distributed and data-driven
environments such as real-time analysis, high-
frequency trading, and so on.
He earned his engineering degree in electronics and instrumentation
from Jadavpur University and his master’s degree in research in
computational and data science from IISc in Bangalore.
xi
About the Technical Reviewer
Joos Korstanje is a data scientist with more
than five years of industry experience in
developing machine learning tools, of which a
large part is forecasting models. He currently
works at Disneyland Paris where he develops
machine learning for a variety of tools.
xiii
Acknowledgments
Thanks to Labonic Chakraborty (Ripa) and Soumili Chakraborty.
xv
Introduction
We are living in the data science/artificial intelligence era. To thrive in
this environment, where data drives decision-making in everything from
business to government to sports and entertainment, you need the skills
to manage and analyze huge amounts of data. Together we can use this
data to make the world better for everyone. In fact, humans have yet to find
everything we can do using this data. So, let us explore!
Our objective for this book is to empower you to become a leader
in this data-transformed era. With this book you will learn the skills to
develop AI applications and make a difference in the world.
This book is intended for advanced user, because we have incorporated
some advanced analytics topics. Important machine learning models and
deep learning models are explained with coding exercises and real-world
examples.
All the source code used in this book is available for download at
https://github.com/apress/advanced-data-analytics-python-2e.
Happy reading!
xvii
CHAPTER 1
OOP in Python
In this section, we explain some features of object-oriented programming
(OOP) in a Python context.
The most basic element of any modern application is an object. To
a programmer or architect, the world is a collection of objects. Objects
consist of two types of members: attributes and methods. Members can be
private, public, or protected. Classes are data types of objects. Every object
is an instance of a class. A class can be inherited in child classes. Two
classes can be associated using composition.
Python has no keywords for public, private, or protected, so
encapsulation (hiding a member from the outside world) is not implicit in
Python. Like C++, it supports multilevel and multiple inheritance. Like Java,
it has an abstract keyword. Classes and methods both can be abstract.
© Sayan Mukhopadhyay, Pratip Samanta 2023 1
S. Mukhopadhyay and P. Samanta, Advanced Data Analytics Using Python,
https://doi.org/10.1007/978-1-4842-8005-8_1
Another Random Document on
Scribd Without Any Related Topics
Hester work so hard to feed me, maybe I ought not to
stay."
"Oh, no, for you see you haven't any relations like my Uncle
Sidney," returned Ruth. "And, besides, who would chop the
wood and do the errands, Billy? Then if you went away, who
would keep store and buy back the big house for Aunt
Hester? It would be foolish for you to go when I have an
uncle to take care of me."
"If I only wasn't another mouth," Ruth went on, "or if I were
a dog like Stray and could live on scraps, or if I were a cat
and could catch mice."
"No, he hasn't. He has only one little boy about five years
old. Billy, promise me on your sacred word and honor that
you will come and get me just as soon as you begin to keep
store, or, if Aunt Hester gets the claim, before that."
Billy nodded gravely. He wished it were not right to tell Ruth
that it would be best for her to leave them. He felt that he
would miss her sadly and that one small boy in the house
with a grave elderly woman would not have as agreeable a
time as when a youthful comrade like Ruth was on hand to
take an interest in small matters beneath the notice of their
elders.
But Ruth had been thinking, too. "I'll go, but I'm not going
to promise to stay. I'm coming back the first chance I get. If
I find a thousand dollars that nobody wants, or if I do
something like saving a train from running off the track, and
they give me a whole lot of money for it, or if—or if—the
claim comes out all right, I'll come straight back, so I just
won't think that I'm going for good, and I am going in now
to tell Aunt Hester so."
"Don't tell her it's because you know she can't afford to
keep you," charged Billy, bluntly.
"Of course not," returned Ruth. She jumped down from her
seat and went slowly back to the house. Miss Hester was
sitting at the window of her room which looked out upon the
street. She had her lap full of little garments upon which
she was sewing a missing button here, a tape there.
"He will probably buy all you need, but I don't want to have
you go away unprovided for. I suppose you must go, Ruth. I
should be doing you a wrong to encourage you to do
otherwise."
Ruth shook her head. "I'm not going to think I'll be gone
long. I will tell my uncle that I am going to stay only a little
while, that I am coming back to you and that I love you
better than him or anybody."
It was well for Miss Hester that the time for preparation was
short for Mr. Mayfield could wait only another day, and so
Ruth's belongings were hastily packed. That she might
make a good appearance, the store of clothing in the chest
up-stairs was drawn forth and all of Henrietta's things that
were in good order were packed in a small trunk. Hetty, too,
was given room, and Ruth begged that her box of pieces
might go in.
"It will make me feel like home to see all my doll rugs," she
said.
But, though Ruth would have liked dearly to have him, Miss
Hester decided otherwise.
Then, after whispering to Miss Hester, she left the room and
returned with a red silk handkerchief which Dr. Peaslee had
once brought her after a visit to the city.
Soon the carriage turned into the main street. The children
were on their way to school, and to Ruth's satisfaction, they
passed Nora Petty, to whom Ruth gave a condescending
nod. She was riding away into new splendors where Nora
could no longer twit and tease her.
As the train moved out of the station, there came over the
child an overpowering desire to jump out and run back to
Aunt Hester who loved her, to Billy and Stray, to the little
brown house which she might never see again. The big tear
drops rolled down her cheeks. She wiped them furtively
away as she kept her head turned as if looking out the
window. Her handkerchief became a damp little ball in her
hand and the telegraph poles, as they flashed by, were seen
through a watery mist. Her uncle wisely said nothing to her
for a time, but absorbed himself in his newspaper, but,
when the train boy came along, he bought some fine fruit
and a box of chocolates saying cheerfully, "Here, little girl,
don't you want to see what is in the box?"
Mr. Mayfield looked rather abashed. "To tell you the truth,
son, I didn't," he began.
Ruth heard the swish of silken skirts and down the stairs a
lady came swiftly. She was very fair and looked quite
young. Ruth had never seen any one dressed so
wonderfully, and she stared with all her eyes at the vision.
"Well, you see, Lillie, I had so much to think of. Oh, by the
way, Ruth, this is your Aunt Lillie. Did you get my telegram,
dear?"
Ruth obediently slipped the box into his hand and he gave
her a smile.
"Here, Bertie," he said, "see what Cousin Ruth has for you.
Papa didn't bring you any candy, but Cousin Ruth did."
"Nobody, miss," was the reply. "It's one of the spare rooms,
but it ain't often used except when there's more company
than common. But I sleep just down the hall in one of the
back rooms."
CHAPTER X
Homesick
"Ruth won't let me have that old doll and I'm going to get
it, I am. I'll smash it all to pieces," cried Bertie, dancing up
and down in a fury.
"Why don't you let him have it, Ruth?" said Mrs. Mayfield.
"I'll get you a better one."
"I don't want a better one," replied Ruth, fiercely. "I don't
want any but this. There isn't another like her and you
couldn't get me one that would be half so dear."
"Yes, I do, I do. It's ugly and I'm going to. Make her give it
to me, mamma."
But Ruth held steadfastly to her own. "I don't want any
other and I couldn't give this away to be broken up," she
said. "She's the only one of the family I have here to
remind me of my home, and I can't give her up."
Whereupon, Bertie burst into screams of anger and
disappointment, flinging himself upon the floor in one of his
fits of temper.
"After all your uncle has done for you, I think it is a very
little thing to ask," said Mrs. Mayfield in an offended way,
addressing Ruth. Then meeting no response, she took a
different tone. "You must give it to him, Ruth. I wish you to
obey me."
"Oh, Aunt Lillie, I can't." The tears came into Ruth's eyes as
she held Hetty more tightly.
"I won't," she said. Then she started for the door. "I reckon
you wouldn't give your child either, to be torn to pieces by a
—by a—wicked Thing," she cried as she reached the door.
"He is younger than you," was always the plea, and Ruth,
though not always with a good grace, yielded the point. But
here was an issue which she felt was a different one from
any that she had been called upon to meet.
"It isn't right; it isn't," she said over and over to herself as
she climbed the stairs. "Aunt Hester wouldn't make me do
it. I know she wouldn't. Why Aunt Hester loves Hetty and
Dr. Peaslee does and Billy, and—why they would think it as
bad as throwing a baby to the crocodiles like a heathen
mother. I'll have to hide you, Hetty darling, like Moses in
the bulrushes or like they had to hide the babies from
wicked old Herod. Bertie is just like Herod, so he is. I don't
love him one bit, and I am going to write to Aunt Hester,
and tell her all about it. Oh, where can I hide you, my
darling Hetty, so the wicked evil foe will not seek you?"
"Oh, but wouldn't you like some nice candy and a pretty
toy? Let Katie dress you and take you out to get you
something nice? We won't get Ruth anything, will we? She
shall not go with you and Katie."
At last Bertie was pacified and was led away by the long-
suffering nurse while Ruth remained in her room.
Mademoiselle was out for the afternoon, and when she had
completed the task the governess had set her to do, she
would be free to do as she chose.
She closed her door softly and looked around the room for a
hiding-place. None seemed possible at first, but at last Ruth
discovered a safe one. A small window seat had been placed
before the one window. It opened and shut like a box.
Between the back of this box and the wall under the window
there was a space over which a small board had been
placed to cover the space which was caused by a slight
jutting out of the window, making an irregular opening.
Ruth found that she could lift the board, shove it back in
place and cover it again with the cushion of the seat. She
gave Hetty a loving kiss and stowed her away in this
retreat.
Her aunt treated her with cold disdain when they next met
and Ruth gave her head a little defiant toss.
"If she 'spises me, I reckon I can 'spise her," she told
herself, and more than ever she kept out of the way.
She had been in her aunt's rooms but seldom, though the
magnificence of them charmed her. On the dressing-table
were such beautiful shining things; the soft couch was piled
high with wonderfully embroidered cushions, and the whole
place was always redolent with some faint sweet odor. The
costumes which Mrs. Mayfield wore, too, were such as Ruth
thought fit for a queen. Once or twice she had seen her
sweeping down the stairway in exquisite evening dress and
she wondered what Nora Petty would say if she knew Ruth
were living in the same house as such a fairylike being.
Bertie did not forgot the doll in spite of the candy and the
new toy with which his mother had provided him, and the
very next day he climbed the stairs to the top floor bent on
finding Hetty. It was sufficient for Bertie to be denied a
thing for him to want it beyond anything else. He looked
around the room. No doll was in sight, but on Ruth's
washstand stood the little flowery mug, Billy's parting gift.
Possessing himself of this, he went down to the nursery
where Ruth was reciting her "J'ai, tu as, il a."
"You may play wis har, of course. Go get zis doll, zis poupée
but say first what is doll. It is poupée, poupée. Repeat."
"Say, zen to your cousin, 'I give you my doll—Je vous donne
ma poupée.' Repeat."
Ruth faced her, very pale. "What will you do?" she asked
slowly.
"Then I will tell my uncle and I will ask him, too, if I must
give my precious doll to Bertie to break up. I will tell him all
about my Hetty and I know he will not make me give her
up."
For answer Bertie dashed out in the hall, held aloft the mug
for a moment and then flung it down over the baluster. It
went crashing into a hundred bits upon the marble tiling in
the hall below. Having thus spent his fury, Bertie dashed
away with an impish look over his shoulder.
"Oh, Martin," said Ruth, "it is my dear little mug. Bertie got
it from my room and threw it down here. Can it be mended,
do you think?"
The man held out the dust-pan and Ruth selected a piece
upon which a rose still showed entire. "I'll keep this
forever," she said. "Thank you, Martin."
The man shook his head as he looked after her making her
way to the upper floor. "That spoiled young un," he
muttered. "I'd just like to see him get one good spanking."
As his step rang upon the tiled floor, she went to meet him.