A Tour Of Data Science: Learn R And

Python In Parallel Nailong Zhang

Visit to download the full and correct content document:
https://textbookfull.com/product/a-tour-of-data-science-learn-r-and-python-in-parallel-
nailong-zhang/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

A Python Data Analyst’s Toolkit: Learn Python and

Python-based Libraries with Applications in Data
Analysis and Statistics Gayathri Rajagopalan

https://textbookfull.com/product/a-python-data-analysts-toolkit-
learn-python-and-python-based-libraries-with-applications-in-
data-analysis-and-statistics-gayathri-rajagopalan/

Practical Data Science Cookbook Data pre processing

analysis and visualization using R and Python
Prabhanjan Tattar

https://textbookfull.com/product/practical-data-science-cookbook-
data-pre-processing-analysis-and-visualization-using-r-and-
python-prabhanjan-tattar/

Python for Teenagers: Learn to Program Like a

Superhero! James R Payne

https://textbookfull.com/product/python-for-teenagers-learn-to-
program-like-a-superhero-james-r-payne/

Hands-on Scikit-Learn for machine learning

applications: data science fundamentals with Python
David Paper

https://textbookfull.com/product/hands-on-scikit-learn-for-
machine-learning-applications-data-science-fundamentals-with-
python-david-paper/
Graph Data Science with Neo4j: Learn how to use Neo4j 5
with Graph Data Science library 2.0 and its Python
driver for your project Scifo

https://textbookfull.com/product/graph-data-science-with-
neo4j-learn-how-to-use-neo4j-5-with-graph-data-science-
library-2-0-and-its-python-driver-for-your-project-scifo/

Data Science with Python 1st Edition Coll.

https://textbookfull.com/product/data-science-with-python-1st-
edition-coll/

Learn Data Science Using SAS Studio: A Quick-Start

Guide Engy Fouda

https://textbookfull.com/product/learn-data-science-using-sas-
studio-a-quick-start-guide-engy-fouda/

Learn Python in One Day and Learn It Well 2nd Edition

Jamie Chan

https://textbookfull.com/product/learn-python-in-one-day-and-
learn-it-well-2nd-edition-jamie-chan/

A Tour of C++ Stroustrup

https://textbookfull.com/product/a-tour-of-c-stroustrup/
A Tour of Data Science
Chapman & Hall/CRC Big Data Series
Series Editor: Sanjay Ranka

A Tour of Data Science

Learn R and Python in Parallel
Nailong Zhang

Big Data Computing

A Guide for Business and Technology Managers
Vivek Kale

Big Data Management and Processing

Kuan-Ching Li, Hai Jiang, Albert Y. Zomaya

Frontiers in Data Science

Matthias Dehmer, Frank Emmert-Streib

High Performance Computing for Big Data

Methodologies and Applications
Chao Wang

Big Data Analytics

Tools and Technology for Effective Planning
Arun K. Somani, Ganesh Chandra Deka

Smart Data
State-of-the-Art Perspectives in Computing and Applications
Kuan-Ching Li, Qingchen Zhang, Laurence T. Yang, Beniamino Di Martino

Internet of Things and Secure Smart Environments

Successes and Pitfalls
Uttam Ghosh, Danda B. Rawat, Raja Datta, Al-Sakib Khan Pathan

For more information on this series please visit: https://www.crcpress.com/Chapman--HallCRC-Big-Data-

Series/book-series/CRCBIGDATSER
A Tour of Data Science
Learn R and Python in Parallel

Nailong Zhang
First edition published 2021
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742

and by CRC Press

2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN

© 2021 Taylor & Francis Group, LLC

CRC Press is an imprint of Taylor & Francis Group, LLC

Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot as­
sume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have
attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders
if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please
write and let us know so we may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or
utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including pho­
tocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission
from the publishers.

For permission to photocopy or use material electronically from this work, access www. copyright.com or contact the
Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are
not available on CCC please contact [email protected]

Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for
identification and explanation without intent to infringe.

Library of Congress Cataloging-in-Publication Data

Names: Zhang, Nailong, author.

Title: A tour of data science : learn R and Python in parallel / Nailong Zhang.
Description: First edition. | Abingdon, Oxon ; Boca Raton, FL : CRC Press,
2021. | Series: Chapman & Hall/CRC data science series | Includes
bibliographical references and index.
Identifiers: LCCN 2020025166 | ISBN 9780367895860 (paperback) | ISBN
9780367897062 (hardback) | ISBN 9781003020646 (ebook)
Subjects: LCSH: Data mining. | R (Computer program language) | Python
(Computer programlanguage)
Classification: LCC QA76.9.D343 Z458 2021 | DDC 006.3/12--dc23
LC record available at https://lccn.loc.gov/2020025166

ISBN: 9780367895860 (hbk)

Typeset in Computer Modern font

by Cenveo Publisher Services
To my parents
Contents

Preface ix
0.1 ASSUMPTIONS ABOUT THE READER’S BACKGROUND ix
0.2 BOOK OVERVIEW ix

Chapter 1 • Introduction to R/Python Programming 1

1.1 CALCULATOR 1
1.2 VARIABLE AND TYPE 3
1.3 FUNCTIONS 4
1.4 CONTROL FLOWS 6
1.5 SOME BUILT-IN DATA STRUCTURES 9
1.6 REVISIT OF VARIABLES 21
1.7 OBJECT-ORIENTED PROGRAMMING (OOP) IN R/PYTHON 30
1.8 MISCELLANEOUS 35

Chapter 2 • More on R/Python Programming 37

2.1 WORK WITH R/PYTHON SCRIPTS 37

2.2 DEBUGGING IN R/PYTHON 40
2.3 BENCHMARKING 47
2.4 VECTORIZATION 49
2.5 EMBARRASSINGLY PARALLELISM IN R/PYTHON 57
2.6 EVALUATION STRATEGY 61
2.7 SPEED UP WITH C/C++ IN R/PYTHON 63
2.8 A FIRST IMPRESSION OF FUNCTIONAL PROGRAMMING 65
2.9 MISCELLANEOUS 69

Chapter 3 • data.table and pandas 71

3.1 SQL 71
3.2 GET STARTED WITH DATA.TABLE AND PANDAS 74

vii
viii • Contents

3.3 INDEXING & SELECTING DATA 76

3.4 ADD/REMOVE/UPDATE 86
3.5 GROUP BY 91
3.6 JOIN 93

Chapter 4 • Random Variables, Distributions & Linear Regression 99

4.1 A REFRESHER ON DISTRIBUTIONS 99

4.2 INVERSION SAMPLING & REJECTION SAMPLING 102
4.3 JOINT DISTRIBUTION & COPULA 106
4.4 FIT A DISTRIBUTION 110
4.5 CONFIDENCE INTERVAL 111
4.6 HYPOTHESIS TESTING 114
4.7 BASICS OF LINEAR REGRESSION 122
4.8 RIDGE REGRESSION 127

Chapter 5 • Optimization in Practice 133

5.1 CONVEXITY 133

5.2 GRADIENT DESCENT 134
5.3 ROOT-FINDING 141
5.4 GENERAL PURPOSE MINIMIZATION TOOLS IN R/PYTHON 143
5.5 LINEAR PROGRAMMING 149
5.6 MISCELLANEOUS 153

Chapter 6 • Machine Learning - A gentle introduction 161

6.1 SUPERVISED LEARNING 161

6.2 GRADIENT BOOSTING MACHINE 169
6.3 UNSUPERVISED LEARNING 188
6.4 REINFORCEMENT LEARNING 195
6.5 DEEP Q-NETWORKS 197
6.6 COMPUTATIONAL DIFFERENTIATION 200
6.7 MISCELLANEOUS 202

Bibliography 203

Index 205
Preface

0.1 ASSUMPTIONS ABOUT THE READER’S BACKGROUND

It is hard to give a clear definition of data science because it is not clear where the bor­
der is. Actually, many data scientists are working in different industries with different
skill sets. Generally speaking, data science requires broad knowledge in statistics, ma­
chine learning, optimization and programming. It is impossible to cover even one of
these four areas in depth in a short book, and it is also beyond the scope of this book.
However, I think it might be useful to have a book that talks about data science with
a broad range of topics and a moderate amount of technical detail. And that is one of
the motivations for writing this book. I chose to make the book short with minimum
mathematical theories introduced, and hope reading through the book could help the
readers get a sense of what data science is about.
I assume the readers have basic knowledge in statistics and linear algebra, and
calculus, such as normal distribution, sample size, gradient, matrix inversion, etc.
Previous programming knowledge is not required. However, as the title of the book
says, we will learn R and Python in parallel, if you’re already familiar with either R
or Python it would help learning the other based on side-by-side comparison.
Besides the comparison of the two popular languages used in data science, this
book also focuses on the translation from mathematical models to codes. In the
book, the audience could find applications/implementations of some important al­
gorithms from scratch, such as maximum likelihood estimation, inversion sampling,
copula simulation, simulated annealing, bootstrapping, linear regression (lasso/ridge
regression), logistic regression, gradient boosting trees, etc.

0.2 BOOK OVERVIEW

The book consists of six chapters designed to introduce core ideas in data science.
Chapter 1: Introduction to R/Python Programming - R and Python are
the two most popular programming languages used in data science. In this chapter,
we go through the basics of R and Python programming in parallel with examples.
Specifically, the topics of this chapter include variable, type, function, control flow,
data structures, and object-oriented programming. The chapter is designed for both
beginners and intermediate audience.
Chapter 2: More on R/Python Programming - Following the first chapter,
the topics of this chapter include debugging, vectorization, parallelism, working with
C++ in R/Python, and functional programming. These topics are chosen to help
the audience to get familiar with some intermediate/advanced topics in R/Python

ix
x • Preface

programming. Mastering these topics will greatly help with coding skills. Like the first
chapter, in this chapter, I try to emphasize the differences between R and Python
with coding examples.
Chapter 3: data.table and pandas - In the first two chapters, we focus on
general-purpose programming techniques. In this chapter, we introduce the very ba­
sics of data science, i.e., data manipulation. For the audience with little experience
in data science, we start from a brief introduction to SQL. The major part of this
chapter focuses on the two widely used data.frame packages, i.e., data.table in R and
pandas in Python. Side-by-side examples using the two packages not only enables the
audience to learn basic usages of these tools but also can be used as a quick reference
manual.
Chapter 4: Random Variables, Distributions & Linear Regression - In
this chapter, we focus on statistics and linear regression, which is the foundation of
data science. To better follow this chapter, I recommend any introductory level statis­
tics course as a prerequisite. The topics of this chapter include random variable sam­
pling methods, distribution fitting, joint distribution/copula simulation, confidence
interval calculation, and hypothesis testing. In later sections, we also talk about lin­
ear regression models from scratch. Many textbooks introduce the theories behind
linear regressions but still don’t help much on the implementation. We will see how
the linear regression is implemented as a toy example in both R and Python with the
help of linear algebra. I will also show how the basic linear regression model can be
used for L2 penalized linear regression, i.e., ridge regression.
Chapter 5: Optimization in Practice - Most machine learning models rely
on optimization algorithms. In this chapter, we give a brief introduction to optimiza­
tion. Specifically, we will talk about convexity, gradient descent, general-purpose op­
timization tools in R and Python, linear programming and metaheuristic algorithms,
etc. Based on these techniques, we will see coding examples about maximum likeli­
hood estimation, linear regression, logistic regression, portfolio construction, traveling
salesman problem.
Chapter 6: Machine Learning – A gentle introduction - Machine learning
is a huge topic. In this chapter, I try to give a very short and gentle introduction to
machine learning. It starts with a brief introduction of supervised learning, unsuper­
vised learning and reinforcement learning, respectively. For supervised learning, we
will see the gradient boosting regression with a pure Python implementation from
scratch, from which the audience could learn the translation from the mathematical
models to object-oriented programs. For unsupervised learning, the finite Gaussian
mixture model and PCA are discussed. And for reinforcement learning, we will also
use a simple game as an example to show the usage of deep Q-networks. The goal
of this chapter is two-fold. First, I would like to give the audience an impression of
what machine learning looks like. Second, reading the code snippets in this chapter
could help the audience review/recap the topics in previous chapters.
 CHAPTER 1

Introduction to R/Python
Programming

In this chapter, I will give an introduction to general R and Python programming

in a parallel fashion.

1.1 CALCULATOR
R and Python are general-purpose programming languages that can be used for
writing softwares in a variety of domains. But for now, let us start with using them
as basic calculators. The first thing is to have them installed. R1 and Python2 can be
downloaded from their official website. In this book, I will be using R 3.5 and Python
3.7.
To use R/Python as basic calculators, let’s get familiar with the interactive mode.
After the installation, we can type R or Python (it is case insensitive so we can also
type r/python) to invoke the interactive mode. Since Python 2 is installed by default
on many machines, in order to avoid invoking Python 2 we type python3.7 instead.

2 ~ $R
3

4 R version 3.5.1 (2018 −07 −02) −− " Feather Sp\ footnoteray "
5 Copyright (C) 2018 The R Foundation for Statistical Computing
6 Platform : x86_64−apple−darwin15 .6.0 (64− bit)
7

8 R is free software and comes with ABSOLUTELY NO WARRANTY .

9 You are welcome to redistribute it under certain conditions .
10 Type ’license ()’ or ’licence ()’ for distribution details .
11

1
https://www.r-project.org
2
https://www.python.org

1
2 • A Tour of Data Science: Learn R and Python in Parallel

12 Natural language support but running in an English locale

13

14 R is a collaborative project with many contributors .

15 Type ’contributors () ’ for more information and
16 ’citation ()’ on how to cite R or R packages in publications .
17

18 Type ’demo () ’ for some demos , ’help () ’ for on−line help , or

19 ’help. start () ’ for an HTML browser interface to help.
20 Type ’q() ’ to quit R.
21

22 >

Python

1 ~ $python3 .7
2 Python 3.7.1 (default , Nov 6 2018 , 18:45:35)
3 [ Clang 10.0.0 (clang − 1000.11.45.5)] on darwin
4 Type "help", " copyright ", " credits " or " license " for more
information .
5 >>>

The messages displayed by invoking the interactive mode depend on both the
version of R/Python installed and the machine. Thus, you may see different messages
on your local machine. As the messages said, to quit R we can type q(). There are 3
options prompted by asking the user if the workspace should be saved or not. Since
we just want to use R as a basic calculator, we quit without saving workspace.
To quit Python, we can simply type exit().

1 > q()
2 Save workspace image ? [y/n/c]: n
3 ~ $

Once we are inside the interactive mode, we can use R/Python as a calculator.

1 > 1+1
2 [1] 2
3 > 2 ∗ 3+5
4 [1] 11
5 > log (2)
 Introduction to R/Python Programming • 3

6 [1] 0.6931472
7 > exp (0)
8 [1] 1

Python

1 >>> 1+1
2 2
3 >>> 2 ∗ 3+5
4 11
5 >>> log (2)
6 Traceback (most recent call last):
7 File "<stdin >", line 1, in <module >
8 NameError : name ’log ’ is not defined
9 >>> exp (0)
10 Traceback (most recent call last):
11 File "<stdin >", line 1, in <module >
12 NameError : name ’exp ’ is not defined

From the code snippet above, R is working as a calculator perfectly. However,

errors are raised when we call log(2) and exp(2) in Python. The error messages are
self-explanatory - log function and exp function don’t exist in the current Python
environment. In fact, log function and exp function are defined in the math module in
Python. A module3 is a file consisting of Python code. When we invoke the interactive
mode of Python, a few built-in modules are loaded into the current environment by
default. But the math module is not included in these built-in modules. That explains
why we got the NameError when we try to use the functions defined in the math
module. To resolve the issue, we should first load the functions to use by using the
import statement as follows.

Python

1 >>> from math import log , exp

2 >>> log (2)
3 0.6931471805599453
4 >>> exp (0)
5 1.0

1.2 VARIABLE AND TYPE

In the previous section we have seen how to use R/Python as calculators. Now, let’s
see how to write real programs. First, let’s define some variables.
3
https://docs.python.org/3/tutorial/modules.html
4 • A Tour of Data Science: Learn R and Python in Parallel

R Python

1 > a=2 1 >>> a=2

2 > b=5.0 2 >>> b=5.0
3 > x=’hello world ’ 3 >>> x=’hello world ’
4 > a 4 >>> a
5 [1] 2 5 2
6 > b 6 >>> b
7 [1] 5 7 5.0
8 > x 8 >>> x
9 [1] " hello world " 9 ’hello world ’
10 > e=a ∗ 2+b 10 >>> e=a ∗ 2+b
11 > e 11 >>> e
12 [1] 9 12 9.0

Here, we defined 4 different variables a, b, x, e. To get the type of each variable,

we can utilize the function typeof() in R and type() in Python, respectively.

R Python

1 > typeof (x) 1 >>> type(x)

2 [1] " character " 2 <class ’str ’>
3 > typeof (e) 3 >>> type(e)
4 [1] " double " 4 <class ’float ’>

The type of x in R is called character, and in Python is called str.

1.3 FUNCTIONS
We have seen two functions log and exp when we use R/Python as calculators.
A function is a block of code which performs a specific task. A major purpose of
wrapping a block of code into a function is to reuse the code.
It is simple to define functions in R/Python.

R Python

1 > fun1= function (x){ return (x 1 >>> def fun1(x):

∗ x)} 2 ... return x ∗ x # n ote the
2 > fun1 indentation
3 function (x){ return (x ∗ x)} 3 ...
4 > fun1 (2) 4 >>> fun1 (2)
5 [1] 4 5 4

Here, we defined a function fun1 in R/Python. This function takes x as input and
returns the square of x. When we call a function, we simply type the function name
 Introduction to R/Python Programming • 5

followed by the input argument inside a pair of parentheses. It is worth noting that
input or output are not required to define a function. For example, we can define a
function fun2 to print Hello World! without input and output.
One major difference between R and Python codes is that Python codes are
structured with indentation. Each logical line of R/Python code belongs to a certain
group. In R, we use {} to determine the grouping of statements. However, in Python
we use leading whitespace (spaces and tabs) at the beginning of a logical line to
compute the indentation level of the line, which is used to determine the statements’
grouping. Let’s see what happens if we remove the leading whitespace in the Python
function above.

Python

1 >>> def fun1(x):

2 ... return x ∗ x # note the indentation
3 File "<stdin >", line 2
4 return x ∗ x # note the indentation
5 ^
6 IndentationError : expected an indented block

We got an IndentationError because of missing indentation.

R Python

1 > fun2= function (){print(’ 1 >>> def fun2 (): print (’Hello
Hello World!’)} World !’)
2 > fun2 () 2 ...
3 [1] " Hello World !" 3 >>> fun2 ()
4 Hello World !

Let’s go back to fun1 and have a closer look at the return. In Python, if we
want to return something we have to use the keyword return explicitly. return in
R is a function but it is not a function in Python and that is why no parenthesis
follows return in Python. In R, return is not required even though we need to return
something from the function. Instead, we can just put the variables to return in the
last line of the function defined in R. That being said, we can define fun1 as follows.

1 > fun1= function (x){x ∗ x}

Sometimes we want to give a default value to an argument for a function, and

both R and Python allow functions to have default values.

R Python
6 • A Tour of Data Science: Learn R and Python in Parallel

1 > log_fun = 1 >>> def log_fun (x, base =2):

function (x, base =2){ 2 ...
2 + return (log(x, base)) return math.log(x, base)
3 + } 3 ...
4 > log_fun (5, base =2) 4 >>> log_fun (5 ,2)
5 [1] 2.321928 5 2.321928094887362
6 > log_fun (5, 2) 6 >>> log_fun (5, base =2)
7 [1] 2.321928 7 2.321928094887362
8 > log_fun (base =2, 5) 8 >>> log_fun (base =2, 5)
9 [1] 2.321928 9 File "<stdin >", line 1
10 > 10 SyntaxError : positional
argument follows keyword
argument

In Python we have to put the arguments with default values at the end, which
is not required in R. However, from a readability perspective, it is always better to
put them at the end. You may have noticed the error message above about positional
argument. In Python there are two types of arguments, i.e., positional arguments
and keyword arguments. Simply speaking, a keyword argument must be preceded
by an identifier, e.g., base in the example above. And positional arguments refer to
non-keyword arguments.

1.4 CONTROL FLOWS

To implement a complex logic in R/Python, we may need control flows.

1.4.1 If/else
Let’s define a function to return the absolute value of input.

R Python

1 > fun3= function (x){ 1 >>> def fun3(x):

2 + if (x >=0){ 2 ... if x >=0:
3 + return (x)} 3 ... return x
4 + else{ 4 ... else:
5 + return(−x)} 5 ... return −x
6 + } 6 ...
7 > fun3 (2.5) 7 >>> fun3 (2.5)
8 [1] 2.5 8 2.5
9 > fun3(−2.5) 9 >>> fun3( −2.5)
10 [1] 2.5 10 2.5

The code snippet above shows how to use if/else in R/Python. The subtle
 Introduction to R/Python Programming • 7

difference between R and Python is that the condition after if must be embraced by
parenthesis in R but it is optional in Python.
We can also put if after else. But in Python, we use elif as a shortcut.

R Python

1 > fun4= function (x){ 1 >>> def fun4(x):

2 + if (x==0){ 2 ... if x==0:
3 + print (’zero ’)} 3 ... print(’zero ’)
4 + else if (x >0){ 4 ... elif x >0:
5 + print (’positive ’)} 5 ... print(’positive ’)
6 + else{ 6 ... else:
7 + print (’negative ’)} 7 ... print(’negative ’)
8 + } 8 ...
9 > fun4 (0) 9 >>> fun4 (0)
10 [1] "zero" 10 zero
11 > fun4 (1) 11 >>> fun4 (1)
12 [1] " positive " 12 positive
13 > fun4(−1) 13 >>> fun4(−1)
14 [1] " negative " 14 negative

1.4.2 For loop

Similar to the usage of if in R, we also have to use parenthesis after the keyword for
in R. But in Python there should be no parenthesis after for.

R Python

1 > for (i in 1:3){ print(i)} 1 >>> for i in

2 [1] 1 range (1 ,4):print(i)
2 ...
3 [1] 2
3 1
4 [1] 3
4 2
5 3

There is something more interesting than the for loop itself in the snippets above.
In the R code, the expression 1:3 creates a vector with elements 1, 2 and 3. In the
Python code, we use the range() function for the first time. Let’s have a look at
them.

R Python

1 > typeof (1:3) 1 >>> type( range (1 ,4))

2 [1] " integer " 2 <class ’range ’>

range() function returns a range type object, which represents an immutable

8 • A Tour of Data Science: Learn R and Python in Parallel

sequence of numbers. range() function can take three arguments, i.e., range(start,
stop, step). However, start and step are both optional. It’s critical to keep in mind
that the stop argument that defines the upper limit of the sequence is exclusive. And
that is why in order to loop through 1 to 3 we have to pass 4 as the stop argument
to range() function. The step argument specifies how much to increase from one
number to the next. The default values of start and step are 0 and 1, respectively.

1.4.3 While loop

R Python

1 > i=1 1 >>> i=1

2 > while (i <=3){ 2 >>> while i <=3:
3 + print (i) 3 ... print(i)
4 + i=i+1 4 ... i+=1
5 + } 5 ...
6 [1] 1 6 1
7 [1] 2 7 2
8 [1] 3 8 3

You may have noticed that in Python we can do i+=1 to add 1 to i, which is not
feasible in R by default. Both for loop and while loop can be nested.

1.4.4 Break/continue
Break/continue helps if we want to break the for/while loop earlier, or to skip a
specific iteration. In R, the keyword for continue is called next, in contrast to continue
in Python. The difference between break and continue is that calling break would
exit the innermost loop (when there are nested loops, only the innermost loop is
affected); while calling continue would just skip the current iteration and continue
the loop if not finished.

R Python

1 > for (i in 1:3){ 1 >>> for i in range (1 ,4):

2 + print (i) 2 ... print(i)
3 + if (i==1) break 3 ... if i==1: break
4 + } 4 ...
5 [1] 1 5 1
6 > for (i in 1:3){ 6 >>> for i in range (1 ,4):
7 + if (i==2){next} 7 ... if i==2: continue
8 + print (i) 8 ... print(i)
9 + } 9 ...
10 [1] 1 10 1
11 [1] 3 11 3
 Introduction to R/Python Programming • 9

1.5 SOME BUILT-IN DATA STRUCTURES

In the previous sections, we haven’t seen much difference between R and Python.
However, regarding the built-in data structures, there are some significant differences
we will see in this section.

1.5.1 Vector in R and list in Python

In R, we can use function c() to create a vector; A vector is a sequence of elements
with the same type. In Python, we can use [] to create a list, which is also a sequence
of elements. But the elements in a list don’t need to have the same type. To get the
number of elements in a vector in R, we use the function length(); and to get the
number of elements in a list in Python, we use the function len().

R Python

1 > x=c(1 ,2 ,5 ,6) 1 >>> x=[1 ,2 ,5 ,6]

2 > y=c(’hello ’,’world ’,’!’) 2 >>> y=[ ’hello ’,’world ’,’!’]
3 > x 3 >>> x
4 [1] 1 2 5 6 4 [1, 2, 5, 6]
5 > y 5 >>> y
6 [1] " hello " " world " "!" 6 [’hello ’, ’world ’, ’!’]
7 > length (x) 7 >>> len(x)
8 [1] 4 8 4
9 > z=c(1,’hello ’) 9 >>> z=[1 ,’hello ’]
10 > z 10 >>> z
11 [1] "1" "hello" 11 [1, ’hello ’]

In the code snippet above, the first element in the variable z in R is coerced from
1 (numeric) to "1" (character) since the elements must have the same type.
To access a specific element from a vector or list, we could use []. In R, sequence
types are indexed beginning with the one subscript. In contrast, sequence types in
Python are indexed beginning with the zero subscript.

R Python

1 > x=c(1 ,2 ,5 ,6) 1 >>> x=[1 ,2 ,5 ,6]

2 > x[1] 2 >>> x[1]
3 [1] 1 3 2
4 >>> x[0]
5 1

What if the index to access is out of boundary?

10 • A Tour of Data Science: Learn R and Python in Parallel

R Python

1 > x=c(1 ,2 ,5 ,6) 1 >>> x=[1 ,2 ,5 ,6]

2 > x[−1] 2 >>> x[−1]
3 [1] 2 5 6 3 6
4 > x[0] 4 >>> x[len(x)+1]
5 numeric (0) 5 Traceback (most recent call
6 > x[ length (x)+1] last):
7 [1] NA 6 File "<stdin >", line 1,
8 > length ( numeric (0)) in <module >
7 IndexError :
9 [1] 0 list index out of range
10 > length (NA)
11 [1] 1

In Python, negative index number means indexing from the end of the list. Thus,
x[−1] points to the last element and x[−2] points to the second-last element of
the list. But R doesn’t support indexing with negative numbers in the same way as
Python. Specifically, in R x[−index] returns a new vector with x[index] excluded.
When we try to access with an index out of boundary, Python would throw an
IndexError. The behavior of R when indexing out of boundary is more interesting.
First, when we try to access x[0] in R we get a numeric(0) whose length is also
0. Since its length is 0, numeric(0) can be interpreted as an empty numeric vector.
When we try to access x[length(x)+1] we get an NA. In R, there are also NaN and
NULL.
NaN means "Not A Number" and it can be verified by checking its type - "double".
0/0 would result in an NaN in R. NA in R generally represents missing values. And
NULL represents a NULL (empty) object. To check if a value is NA, NaN or NULL, we
can use is.na(), is.nan() or is.null, respectively.

R Python

1 > typeof (NA) 1 >>> type(None)

2 [1] " logical " 2 <class ’NoneType ’>
3 > typeof (NaN) 3 >>> None is None
4 [1] " double " 4 True
5 > typeof (NULL) 5 >>> 1 == None
6 [1] "NULL" 6 False
7 > is.na(NA)
8 [1] TRUE
9 > is.null(NULL)
10 [1] TRUE
11 > is.nan(NaN)

In Python, there is no built-in NA or NaN. The counterpart of NULL in Python is

 Introduction to R/Python Programming • 11

None. In Python, we can use the is keyword or == to check if a value is equal to

None.
From the code snippet above, we also notice that in R the boolean type value
is written as "TRUE/FALSE", compared with "True/False" in Python. Although in
R "TRUE/FALSE" can also be abbreviated as "T/F", I don’t recommend using the
abbreviation.
There is one interesting fact that we can’t add a NULL to a vector in R, but it is
feasible to add a None to a list in Python.

R Python

1 > x=c(1,NA ,NaN ,NULL) 1 >>> x=[1, None]

2 > x 2 >>> x
3 [1] 1 NA NaN 3 [1, None]
4 > length (x) 4 >>> len(x)
5 [1] 3 5 2

Sometimes we want to create a vector/list with replicated elements, for example,

a vector/list with all elements equal to 0.

R Python

1 > x=rep (0, 10) 1 >>> x =[0] ∗ 10

2 > x 2 >>> x
3 [1] 0 0 0 0 0 0 0 0 0 0 3 [0, 0, 0, 0, 0, 0, 0, 0, 0,
4 > y=rep(c(0 ,1) , 5) 0]
5 > y 4 >>> y=[0 , 1] ∗ 5
6 [1] 0 1 0 1 0 1 0 1 0 1 5 >>> y
6 [0, 1, 0, 1, 0, 1, 0, 1, 0,
1]

When we use the ∗ operator to make replicates of a list, there is one caveat - if
the element inside the list is mutable then the replicated elements point to the same
memory address. As a consequence, if one element is mutated other elements are also
affected.

Python

1 >>> x=[0] # x is a list which is mutable

2 >>> y=[x] ∗ 5 # each element in y points to x
3 >>> y
4 [[0] , [0], [0] , [0] , [0]]
5 >>> y [2]=2 # we point y[2] to 2 but x is not mutated
6 >>> y
7 [[0] , [0], 2, [0] , [0]]
8 >>> y[1][0]= − 1 # we mutate x by changing y [1][0] from 0 to −1
12 • A Tour of Data Science: Learn R and Python in Parallel

9 >>> y
10 [[−1], [−1], 2, [−1], [−1]]
11 >>> x
12 [−1]

How to get a list with replicated elements but pointing to different memory ad­
dresses?

Python

1 >>> x=[0]
2 >>> y=[x[:] for _ in range (5)] # [:] makes a copy of the list x;
another solution is [list(x) for _ in range (5)]
3 >>> y
4 [[0] , [0], [0] , [0] , [0]]
5 >>> y [0][0]=2
6 >>> y
7 [[2] , [0], [0] , [0] , [0]]

Besides accessing a specific element from a vector/list, we may also need to do

slicing, i.e., to select a subset of the vector/list. There are two basic approaches of
slicing:
• Integer-based

1 > x=c(1 ,2 ,3 ,4 ,5 ,6)

2 > x[2:4]
3 [1] 2 3 4
4 > x[c(1 ,2 ,5)] # a vector of indices
5 [1] 1 2 5
6 > x[seq (1 ,5 ,2)] # seq creates a vector to be used as
indices
7 [1] 1 3 5

Python

1 >>> x=[1 ,2 ,3 ,4 ,5 ,6]

2 >>> x [1:4] # x[start :end] start is inclusive but end is
exclusive
3 [2, 3, 4]
4 >>> x [0:5:2] # x[ start :end:step]
5 [1, 3, 5]
 Introduction to R/Python Programming • 13

The code snippet above uses hash character # for comments in both R and
Python. Everything after # on the same line would be treated as comment (not
executable). In the R code, we also used the function seq() to create a vector.
When I see a function that I haven’t seen before, I might either google it or
use the built-in helper mechanism. Specifically, in R use ? and in Python use
help().

R Python

1 > ?seq 1 >>> help( print )

• Condition-based
Condition-based slicing means to select a subset of the elements which satisfy
certain conditions. In R, it is quite straightforward by using a boolean vector
whose length is the same as the vector to be sliced.

1 > x=c(1 ,2 ,5 ,5 ,6 ,6)

2 > x[x %% 2==1] # %% is the modulo operator in R; we select
the odd elements
3 [1] 1 5 5
4 > x %% 2==1 # results in a boolean vector with the same
length as x
5 [1] TRUE FALSE TRUE TRUE FALSE FALSE

The condition-based slicing in Python is quite different from that in R. The

prerequisite is list comprehension which provides a concise way to create new
lists in Python. For example, let’s create a list of squares of another list.

Python

1 >>> x=[1 ,2 ,5 ,5 ,6 ,6]

2 >>> [e ∗∗ 2 for e in x] # ∗∗ is the exponent operator , i.e.,
x ∗∗ y means x to the power of y
3 [1, 4, 25, 25, 36, 36]

We can also use if statement with list comprehension to filter a list to achieve
list slicing.

Python
14 • A Tour of Data Science: Learn R and Python in Parallel

1 >>> x=[1 ,2 ,5 ,5 ,6 ,6]

2 >>> [e for e in x if e %2==1] # % is the modulo operator in
Python
3 [1, 5, 5]

It is also common to use if/else with list comprehension to achieve more

complex operations. For example, given a list x, let’s create a new list y so
that the non-negative elements in x are squared and the negative elements are
replaced by 0s.

Python

1 >>> x=[1, −1,0,2,5, −3]

2 >>> [e ∗∗ 2 if e >=0 else 0 for e in x]
3 [1, 0, 0, 4, 25, 0]

The example above shows the power of list comprehension. To use if with list
comprehension, the if statement should be placed in the end after the for loop
statement; but to use if/else with list comprehension, the if/else statement
should be placed before the for loop statement.

We can also modify the value of an element in a vector/list variable.

R Python

1 > x=c(1 ,2 ,3) 1 >>> x=[1 ,2 ,3]

2 > x[1]=−1 2 >>> x[0]= −1
3 > x 3 >>> x
4 [1] −1 2 3 4 [−1, 2, 3]

Two or multiple vectors/lists can be concatenated easily.

R Python

1 > x=c(1 ,2) 1 >>> x=[1 ,2]

2 > y=c(3 ,4) 2 >>> y=[3 ,4]
3 > z=c(5 ,6 ,7 ,8) 3 >>> z=[5 ,6 ,7 ,8]
4 > c(x,y,z) 4 >>> x+y+z
5 [1] 1 2 3 4 5 6 7 8 5 [1, 2, 3, 4, 5, 6, 7, 8]

As the list structure in Python is mutable, there are many things we can do with
list.
 Introduction to R/Python Programming • 15

Python

1 >>> x=[1 ,2 ,3]

2 >>> x. append (4) # append a single value to the list x
3 >>> x
4 [1, 2, 3, 4]
5 >>> y=[5 ,6]
6 >>> x. extend (y) # extend list y to x
7 >>> x
8 [1, 2, 3, 4, 5, 6]
9 >>> last=x.pop () # pop the last element from x
10 >>> last
11 6
12 >>> x
13 [1, 2, 3, 4, 5]

I like the list structure in Python much more than the vector structure in R. list
in Python has a lot more useful features which can be found from the python official
documentation4 .

1.5.2 Array
Array is one of the most important data structures in scientific programming. In R,
there is also an object type "matrix", but according to my own experience, we can
almost ignore its existence and use array instead. We can definitely use list as array
in Python, but lots of linear algebra operations are not supported for the list type.
Fortunately, there is a Python package numpy off the shelf.

1 > x=1:12
2 > array1 =array (x, c(4 ,3)) # convert vector x to a 4 rows ∗ 3
cols array
3 > array1
4 [,1] [ ,2] [ ,3]
5 [1,] 1 5 9
6 [2,] 2 6 10
7 [3,] 3 7 11
8 [4,] 4 8 12
9 > y=1:6
10 > array2 =array (y, c(3 ,2)) # convert vector y to a 3 rows ∗ 2
cols array
11 > array2
12 [,1] [ ,2]
4
https://docs.python.org/3/tutorial/datastructures.html
16 • A Tour of Data Science: Learn R and Python in Parallel

13 [1,] 1 4
14 [2,] 2 5
15 [3,] 3 6
16 > array3 = array1 % ∗ % array2 # % ∗ % is the matrix multiplication
operator
17 > array3
18 [,1] [ ,2]
19 [1,] 38 83
20 [2,] 44 98
21 [3,] 50 113
22 [4,] 56 128
23 > dim( array3 ) # get the dimension of array3
24 [1] 4 2

Python

1 >>> import numpy as np # we import the numpy module and alias it

as np
2 >>> array1 =np. reshape (list(range (1 ,13)) ,(4 ,3)) # convert a list
to a 2d np.array
3 >>> array1
4 array ([[ 1, 2, 3],
5 [ 4, 5, 6],
6 [ 7, 8, 9],
7 [10, 11, 12]])
8 >>> type( array1 )
9 <class ’numpy. ndarray ’>
10 >>> array2 =np. reshape (list(range (1 ,7)) ,(3 ,2))
11 >>> array2
12 array ([[1 , 2],
13 [3, 4],
14 [5, 6]])
15 >>> array3 =np.dot(array1 , array2 ) # matrix multiplication using
np.dot ()
16 >>> array3
17 array ([[ 22, 28],
18 [ 49, 64] ,
19 [ 76, 100] ,
20 [103 , 136]])
21 >>> array3 .shape # get the shape ( dimension ) of array3
22 (4, 2)

You may have noticed that the results of the R code snippet and Python code
snippet are different. The reason is that in R the conversion from a vector to an array
 Introduction to R/Python Programming • 17

is by-column; but in numpy the reshape from a list to a 2D numpy.array is by-row.

There are two ways to reshape a list to a 2D numpy.array by column.

Python

1 >>> array1 =np. reshape (list(range (1 ,13)) ,(4 ,3) ,order =’F’) # use
order =’F’
2 >>> array1
3 array ([[ 1, 5, 9],
4 [ 2, 6, 10] ,
5 [ 3, 7, 11] ,
6 [ 4, 8, 12]])
7 >>> array2 =np. reshape (list(range (1 ,7)) ,(2 ,3)).T # use .T t o
transpose an array
8 >>> array2
9 array ([[1 , 4],
10 [2, 5],
11 [3, 6]])
12 >>> np.dot(array1 , array2 ) # now we get the same result as using
R
13 array ([[ 38, 83],
14 [ 44, 98] ,
15 [ 50, 113] ,
16 [ 56, 128]])

To learn more about numpy, the official website5 has great documentation/tutori­
als.
The term broadcasting describes how arrays with different shapes are handled
during arithmetic operations. A simple example of broadcasting is given below.

R Python

1 > x = c(1, 2, 3) 1 >>> import numpy as np

2 > x+1 2 >>> x = np. array ([1 , 2, 3])
3 [1] 2 3 4 3 >>> x + 1
4 array ([2, 3, 4])

However, the broadcasting rules in R and Python are not exactly the same.

R Python

1 > x = array (c(1:6) , c(3 ,2)) 1 >>> import numpy as np

2 > y = c(1, 2, 3) 2 >>> x = np. array ([[1 , 2],
3 > z = c(1, 2) [3, 4], [5, 6]])
5
http://www.numpy.org
18 • A Tour of Data Science: Learn R and Python in Parallel

4 # point−wise multiplication 3 >>> y = np. array ([1 , 2, 3])

5 > x ∗ y 4 >>> z = np. array ([1 , 2])
6 [,1] [ ,2] 5 >>> # point−wise
7 [1,] 1 4 multiplication
6 >>> x ∗ y
8 [2,] 4 10
7 Traceback (most recent call
9 [3,] 9 18
last):
10 > x∗z
8 File "<stdin >", line 1,
11 [,1] [ ,2] in <module >
12 [1,] 1 8 9 ValueError : operands could
13 [2,] 4 5 not be broadcast together
with shapes (3 ,2) (3 ,)
14 [3,] 3 12
10 >>> x ∗ z
11 array ([[ 1, 4],
12 [ 3, 8],
13 [ 5, 12]])

From the R code, we see the broadcasting in R is like recycling along with the
column. In Python, when the two arrays have different dimensions, the one with fewer
dimensions is padded with ones on its leading side. According to this rule, when we
do x ∗ y, the dimension of x is (3, 2) but the dimension of y is 3. Thus, the dimension
of y is padded to (1, 3), which explains what happens when x ∗ y.

1.5.3 List in R and dictionary in Python

Yes, in R there is also an object type called list. The major difference between a
vector and a list in R is that a list could contain different types of elements. list in R
supports integer-based accessing using [[]] (compared to [] for vector).

1 > x=list (1,’hello world!’)

2 > x
3 [[1]]
4 [1] 1
5

6 [[2]]
7 [1] " hello world !"
8

9 > x[[1]]
10 [1] 1
11 > x[[2]]
12 [1] " hello world !"
13 > length (x)
14 [1] 2
Another random document with
no related content on Scribd:
 If the analysis of the relation of the individual to society is correct,
we are justified in claiming that any adequate statement of the aim of
education must point unmistakably to the idea of the common good.
Education aims so to adjust the individual to the group that the
welfare of society as a whole may be advanced. This adjustment can
be brought about only through participation in social activities, and
thus the aim is constantly realized in the process.
In our democratic society, which makes possible free education for
all of its members, there can be no question of the right of society to
demand that education aim to develop men and women who work
for the common good. It is necessary, then, to analyze this aim of
social efficiency in terms of our society. The equality of opportunity
which we profess to offer is to be thought of in terms of possible
service which may be rendered.
In any community the contribution to the general welfare which
may be made by any one of its members is conditioned by the
interests which the individual has in the general good. The unsocial
individual, the one who seldom responds to the needs of the group,
is out of sympathy with social problems, and contributes little to
social welfare.
But it is not enough that the individual be interested in the common
welfare. Interest may lead him to do that which is harmful rather than
helpful, or it may be that his interest may have no result except to
give him certain pleasurable emotions. There must be added to
sympathy, knowledge. Interest or sympathy in the welfare of society
may furnish the propelling force, but knowledge is necessary for
effective action. The world is full of men and women with the best
intentions who hinder rather than advance the common good.
Since each is responsible not only for his own conduct, but also for
the welfare of the whole group, it is necessary that our education
provide opportunity for growth in intelligence. Our schools have
always emphasized this element in education. We have often
defined the aim of education in terms of the development of
citizenship. Usually the chief qualification of the citizen has been
interpreted to be that knowledge which would enable him to exercise
the right of suffrage with intelligence. We do well, however, to
remember that intelligence must be exercised in all of the activities of
life. Our education must strive constantly to develop men and
women who will be rational at all times. But we may not forget that
our schools have been so much concerned with the intellectual side
of education that they have tended to neglect other elements which
are equally significant from the standpoint of social welfare.
There is still another element which must be added, the habit of
acting on behalf of the group. We all know people who know just
what is demanded in a given social situation; they profess to be
interested in the welfare of the group; but they never act. When their
own private interests are involved they are quick to seize the
opportunity for improving their condition; but in social matters they
are inactive. It is in this particular, rather than in any other, that our
schools fall short. We do much to arouse the sympathy of children in
the general welfare; we give them the knowledge by which their
action may be guided; but we give them little opportunity to form the
habit of social service. This is due to the fact that we so often think of
adult social activities as the only ones that are worth while, forgetting
that for the child the important thing is social activity now and in his
society, that the only way to prepare for adult social effectiveness is
to secure social efficiency on the part of the child.
These questions still remain: how can we, through education,
produce the individual who, because of social sympathy, knowledge,
and activity, will tend to advance the welfare of all; and what kinds of
education meet the demands of the aim which we have set up.
First of all, we must endeavor to produce the individual who is
sound physically. Modern education recognizes the fact that a man’s
usefulness is conditioned by his bodily condition, and is also coming
to find that physical activity is not without its effect on the mental
development and life of the individual. There is, therefore, one large
division of our work which we may call physical education.
On the side of mental development, education consists in
preserving and stimulating the child’s interest in the materials and
processes with which he may come in contact. Intellectual training
aims to develop the man or woman who is mentally alert, active in
investigation, and controlled by reason. It is to this intellectual
education that our schools have devoted the larger part of their time.
The school is the agency set aside by society for transmitting culture,
and the teacher must always concern herself largely with the
intellectual life of children.
Our modern view of education is leading us to stress, along with
physical and intellectual education, a kind of training which aims to
develop the individual whose moral standards are positive rather
than negative. Moral-social education should establish ideals of
social service as well as standards of individual righteousness.
Along with physical, moral-social, and intellectual-cultural
education, there is need for that type of training which will enable
each individual to do some particular work with a high degree of
efficiency. This type of education we commonly call vocational. It is
only recently that we have come to realize that it is not enough to
train an individual with respect to general intelligence and morality,
but that it is also just as fundamental that our education provide the
training necessary for success in the particular calling which each
individual is to enter. For the preparation of clergymen, doctors,
lawyers, teachers, and engineers, whose vocations require a
maximum of intellectual achievement, it is true that we have long had
our vocational schools. We are coming now to appreciate the fact
that equality of opportunity demands that special training be given to
those who are to enter the industries. Indeed, our vocational schools
must multiply until there is training offered for each and every calling
before we can claim to provide that training which is essential for
social efficiency.
Another problem is that of the training for leisure. In society as at
present constituted, it is possible for many individuals, and it should
ultimately be possible for all, to have a considerable amount of
leisure time. The contribution of each individual in his special line of
work, and his general interest in the whole community, will depend in
a considerable degree upon the proper use of leisure time. Our
education must, therefore, attempt to equip men with interests and
ideals which make for the nobler enjoyments.
 Keeping in mind the sympathetic, wise, active social individual,
made so by the process of acquiring experience or making of
adjustments, both physical and mental, we have yet to reduce our
aim to the terms of schoolroom practice. What can a teacher hope to
do in this hour, with this group of children to work with?
First of all the teacher can work for the formation of habits which
are socially desirable and for the inhibition of those which are
undesirable. “Education is for behavior, and habits are the stuff of
which behavior consists.”[1] The school may be a very important
factor in the formation of habits in each of the fields of education
mentioned above. If the school is organized on a rational social
basis, it must continually present opportunities for actions which
should become habitual, and the future efficiency of the learner
depends upon gaining such control of much of the knowledge which
we teach that the response desired becomes habitual. The social
virtues of promptness, regularity, helpfulness, industry, fidelity,
honesty, truthfulness, cleanliness, both physical and mental,
patriotism, and the like, should be made habitual in connection with
the situations which demand their exercise. The physical habits
acquired in childhood are of the utmost significance throughout life.
Much of arithmetic, spelling, writing, geography, history, and even of
literature and art, will be significant in proportion as we have reduced
our knowledge to the automatic basis of habit. One cannot stop to
reason everything out; life is too short. We gain time and energy for
the higher activities of life in proportion as we reduce the responses
which occur frequently to the basis of habits. In vocational schools
one of the chief aims is the formation of habits of skill. Later we shall
want to discuss in detail the methodology of habit formation.
Every teacher recognizes that one of the ends which must be
achieved by the school is knowledge. We shall not here enter into
the discussion of the problem of what knowledge is of most worth,
since for the teacher this choice is usually made and prescribed in
the course of study. One cannot, however, refrain from suggesting
that much that is taught would be eliminated, if we kept constantly in
mind the end for which we strive. The following criteria, proposed by
Professor Frank M. McMurry, will be suggestive from the standpoint
of teaching, whether the teacher determines the curriculum or not.
“We hold to the following propositions in the rejection of subject
matter.[2]
“1. Whatever cannot be shown to have a plain relation to some
real need of life, whether æsthetic, ethical, or utilitarian in the
narrower sense, must be dropped.
“2. Whatever is not reasonably within the child’s comprehension.
“3. Whatever is unlikely to appeal to his interest; unless it is
positively demanded for the first very weighty reason.
“4. Whatever topics and details are so isolated or irrelevant that
they fail to be a part of any series or chain of ideas, and therefore fail
to be necessary for the appreciation of any large point. This
standard, however, not to apply to the three R’s and spelling.”
These criteria indicate clearly that knowledge can never be in itself
an end of teaching. It is not that the child may have knowledge
merely, but that he shall have knowledge which will function. This
knowledge which we seek to have the child master will concern his
physical life, his social relationships, his vocation; and in each field
the knowledge he possesses will limit his intellectual activity.
The school must keep alive, or, in some cases, awaken those
interests which are socially desirable. It is not enough that habits
have been formed and knowledge acquired. Much of the usefulness
of the individual after he leaves school will depend on his interests
which lead him to acquire new knowledge, or to attempt some new
activity. It has sometimes been asserted that the school, as at
present organized, tends to kill rather than to preserve those
interests which are common to little children. It is probable that the
passing interests in things due to curiosity must disappear,
regardless of the education which we give; but it is a poor sort of
education which leaves the child without abiding interests which will
help him not only in making a living, but also in enjoying his life.
Here, as elsewhere in education, we may be satisfied with the result
only when we get the corresponding action. That child has an
interest in good literature who reads good literature. We can be sure
that the boy is interested in natural phenomena when he is willing to
spend his leisure time finding out more about nature’s ways. The
only test that we have of an abiding interest in the welfare of others
is the fact that the child is now active on behalf of others. In like
manner are we to judge of our success in arousing and maintaining
those other interests which are desirable.
Judgments of fact are called for constantly in acquiring knowledge
and in our everyday activity; but no less important in the life of
individuals are judgments of worth. Education must concern itself
with the ideals, purposes, and standards which should be acquired
by children. There is no field in which greater skill is demanded in
teaching than in bringing children to appreciate those things which
are good, true, and beautiful. Ideals, or, for those who do not agree
with them, prejudices, will always be of tremendous importance.
They determine the course of action a man will take. Because of
their ideals men have been willing to labor incessantly for a cause
which they considered just, to give up personal good in the
pursuance of public duty, to lose all, if they might but retain their
honor, yes, even to lose their lives because they felt that this
extreme service was demanded of them. The awakening and
nurturing of ideals of work (or industry), of honor, of duty, of purity, of
service is the greatest contribution of the best teacher.
There is one other aim which the teacher should have constantly
in mind, included possibly in the above, but which needs to be stated
separately for the sake of emphasis, i.e. that children should be
taught how to work independently. The best teacher is the one who
is constantly striving to render her services unnecessary. There is
nothing that the school can do which will take the place of giving the
child knowledge of the most economical means to be employed in
achieving desirable ends. Is it a matter of knowledge, the child
should be made conscious of the methods whereby truth may be
established; is it the need of establishing a new habit, or the
breaking up of the old one, we should make available for the pupil
the principles of habit formation so that he may apply them to his
own case; in matters of right and wrong, the school should have
supplied standards of reference which will help in the difficult
situation. Possibly the great weakness of many teachers in imparting
this knowledge of methods of work is best illustrated by citing the
well-known fact that children of high school, or even college age, are
found very frequently who do not know how to read a book, or study
a lesson assigned. This problem will be treated in considerable detail
when we come to consider the study lesson.
Pupils at work forming habits of thought, feeling, and action;
acquiring knowledge of nature and of society; forming ideals which
make for social well-being; and learning in all of this work to act
independently, to function in the society of which they are a part: this
is education, and these are the goals which we should strive to
achieve every day and every hour that we teach.

For Collateral Reading

Nicholas Murray Butler, The Meaning of Education, Chapter I.
W. C. Bagley, The Educative Process, Chapter III.

Exercises.
1. How would you hope to contribute to the realization of the aim of education in
the teaching of English, arithmetic, cooking, geography, or other school subjects?
2. How would you determine whether or not the children in your grade are
socially efficient?
3. What are the most important subjects, or parts of subjects, which you teach?
Why?
4. How would an application of the aim of education as discussed in this chapter
modify the work commonly done in arithmetic? In nature study?
5. It has been claimed that education should provide for the harmonious
development of all of the powers. Criticize this statement of aim.
6. Could you defend the statement that “the aim of education is to produce
socially efficient men and women,” and at the same time deny that the greatest
individual good comes from working for the general welfare?
7. Why should education be free in a democracy?
8. Is society justified in offering special education to the deficient and the
delinquent? To the especially capable? Why?
 9. Is the excessive rivalry which we sometimes foster in our schools compatible
with the aim of social efficiency?
10. Of the several types of education, physical, intellectual, moral-social,
vocational, and education for leisure, which is most neglected?
11. How do you account for the fact that many children cease to inquire, to
investigate, or even to ask questions, although they are regularly taught in our
schools?
12. Why do you teach school? What do you hope to accomplish?
13. Can you name specific instances of changes brought about in children under
your instruction which justify you in believing that you have fulfilled the aim of
education in your teaching?
14. What justification is there for music, drawing, or literature in the curriculum?
15. State briefly the aim of education.
 CHAPTER II

T H E FA C T O R S C O N D I T I O N I N G T H E T E A C H I N G P R O C E S S

If it is essential that the teacher approach her work with a clear

view of the ends which it is desirable for her to achieve, it is quite as
necessary that she be conscious of the factors which condition the
teaching process. The school, with its limitations and its advantages,
the community and home life of the child, and, above all else, the
child himself, his instincts, impulses, and abilities must be the subject
of most careful study. Much progress has been made in recent years
because of a better understanding and a more sympathetic attitude
toward children. Teachers are beginning to see that education has its
beginning in, and that it is always conditioned by, the life of the child
outside of the school building. The possibilities of the school as an
institution for the education of children are just beginning to be
realized.
While it is true that the school shares with the home, the church,
and the community at large the education of children, no one can fail
to recognize the fact that the responsibilities and the activities of the
school have been very greatly augmented during the past few
decades. Where other institutions have lost or have become less
effective, the school has gained, or has been forced to accept new
responsibilities. Changed industrial conditions and life in cities have
made it impossible for the home to continue to hold the important
place which it once occupied in preparing its members for efficient
participation in the productive activities. Whether we like it or not, we
are forced to admit that the church no longer exerts the power over
the lives and conduct of men that it once did. Along with the
specialization of function which is so characteristic of our modern
life, citizenship in our democracy has come to require less of that
type of participation in public affairs which was once a great
educative factor in our community life.
As these changes in the effectiveness of other institutions have
taken place, men have looked to the schools to make good the
deficiency. The schools have responded to the demand made upon
them. Our curriculum no longer consists of the three R’s. Cooking,
sewing, gardening, and many other kinds of manual work, music,
physical training, and fine art are already found in our courses of
study. We are coming to recognize the need for more systematic
training in morals and civics, and vocational training is being
introduced.
What is the significance of these changes for teachers? Is it not
true that they must teach whatever is demanded by the course of
study; and is not this the only difference in the teacher’s function
brought about by changed conditions? The answer is, most
emphatically, no. The situation which has already made necessary
the change in curriculum demands also changes in method quite as
revolutionary. It is more essential to-day than ever before that the
school present opportunities for coöperation and for group work, a
chance for pupils to work together for common ends, because there
is so much less demand of this sort made upon children outside of
school than was formerly the case. We ought to do more than we do
to develop the independence and the self-reliance which were so
characteristic of the boy and girl who lived in an environment which
constantly made heavy demands upon their strength, skill, and
ingenuity. The responsibility for taking the initiative, and of measuring
the success of one’s efforts by the results produced, is all too
uncommon in the lives of our children. The school must, if it is to
adequately meet its enlarged responsibility, develop those habits of
thought and action which enable one to get along with his fellows.
The school life of the child must, in so far as this is possible, present
such opportunities, make such demands, and judge results by
standards essentially social. The child must learn in school to serve,
to accept responsibility, and to produce results socially valuable. We
could do much to increase the efficiency of the school if we planned
more carefully to have schoolroom activities find their application in
the homes of children.
School education begins not with the ignorance of children, but
with their knowledge. Children come to us with a great wealth of
experience. Our work as teachers is to enlarge and to interpret this
experience, to give it greater meaning and significance. Can any one
question, then, the necessity for acquaintance with the life of the
child outside of school? And this study of the out-of-school
environment must continue as long as the child is in school, if the
teacher’s work is to be most effective. It makes a great deal of
difference when you wish to teach nature study that your children
have always lived in the city, at a considerable distance from a park.
The problem of teaching a great commercial center to children living
on farms presents some difficulty. But it is not alone these more
gross differences in the lives of children which demand our attention.
There are differences in ideals, differences in social custom, in short,
in ways of thinking, feeling, and acting, which one must know if one
would claim any adequate knowledge of the child to be taught.
Probably the best opportunity to gain this intimate knowledge of the
lives of children whom we teach is to be had in the work with parents
and older brothers and sisters which should be carried on in the
school building when the smaller children are not present. The
school which is a center of community life, a place for study, for
recreation, for physical development, and for social intercourse is the
school that is fulfilling its mission in the life of the people; and the
teacher who works in such a school will know her children.
There is one other responsibility which we as teachers must
acknowledge which again leads us beyond the schoolroom. We
should work for the welfare of our children during the time that they
are not with us. No other body of men and women knows the needs
of these children better than we do. Our work is conditioned by the
life of the child before he comes under our influence. Our work is
ofttime of no effect because of the adverse conditions outside of the
school. What does it matter that we try to develop morality in
children, when the forces of immorality in the streets more than
counteract our influence? what does it matter that we strive earnestly
to provide hygienic conditions for work during five hours of the day,
when filth and disease are doing their deadly work outside of the
school for nineteen hours a day? Who knows better than we that
children with starved bodies cannot do great things intellectually? If
we were only organized to improve these conditions, we could do
much for the welfare of the community. The time is coming when it
will be considered as legitimate for a body of teachers to discuss the
problems of impure food supply, of relief for the poor, of means for
the suppression of vice, and of better hygienic conditions for the
children of our cities, as it is to discuss the problems of method or
the organization of school work. What we need, if we are to be
effective in the work, is better organization, more craft
consciousness. We now possess potentially great power for social
betterment. We are exercising this power in the school, and, as
individuals, outside of the school. We will, let us hope, in time,
recognize the larger social demand and perform the larger social
service.
The children with whom we work come to us equipped with many
native reactions or tendencies to behave. In any situation the child
will react in accordance with some native tendency or habit which
has grown out of the original tendency. Success in teaching depends
upon a recognition of these instinctive tendencies, the development
of some, the grafting of new but similar reactions on others, and the
inhibition of the native reaction and substitution of another in still
other cases. The instincts which are of importance in education have
been variously named; among these those of greatest significance
for the work of the teacher are play, constructiveness, imitation,
emulation, pugnacity, curiosity, ownership, including the collecting
instinct, sympathy, wonder. We shall deal briefly with each of these in
relation to the work of the teacher.
Play: Possibly the lesson which teachers need most to learn is that
play has real educative value. Before the school age has been
reached, the child has learned chiefly by playing. In play the child
gets his first experience in those activities which are later to make
possible a happy, useful life in the community. The number of
possible reactions possessed by a child of six is largely determined
by the opportunity he has had to play. This is why we value so much
a life free from restraint, and in contact with nature, for little children.
Contact with the trees, the rocks, the birds, the flowers, and
association with other children mean possibilities of learning for the
child which no amount of instruction or exercise of authority can
equal. The child plays now with this object and again with that; and
in consequence comes to know not only the objects, but his own
power. In an imaginative way he experiences all of the adult activities
about him, sowing, reaping, building, cooking, cleaning, hauling,
fighting; and he is wiser and better prepared for the period of
struggle, which must come later, because of these activities.
Nor should this period of play end when the child enters school.
The skillful teacher makes a game of many of the exercises of the
school, which might be otherwise drudgery. The desire to win is
common to children six years of age, and many a hard task will
become play, if the element of competition is introduced and
sufficient variety in procedure is provided for. By playing, children
may learn to work. To achieve the ends desired in a game may
involve the overcoming of difficulties which require the most earnest
effort. There can be no better preparation for life than the playing of
games where team work, self-restraint, and fairness are demanded.
We need more careful study on the part of teachers of children’s
games, and more planning that all may secure the benefits which
come from this sort of activity. In the schoolroom, wherever it is
possible, the spirit of play should pervade the work. There will be
cases enough where results will depend upon the exercise of
authority. Let us never forget that the reaction of play may mean just
as valuable results as the reaction of necessity, and that the ideal life
is the one in which all work is play.
Constructiveness: Closely connected with the play instinct is the
instinct to make out of the material at one’s command that which will
represent some element in the play. In the beginning, gestures,
sounds, and whatever objects are present suffice in the make-
believe world of the child. But soon the materials are rearranged or
shaped into some new form in order to represent the object desired.
Materials become to the child just what he can make out of them.
And it is not simply in power to construct or to represent that the
child grows because of this activity. To make something, to work out
in materials one’s idea, means growth in definiteness and control of
ideas. The one adequate test of ideas must always be some sort of
expression; and, for the adult as well as for the child, construction is
one of the most important forms of expression. We would gain much
in all of our school work in clearness and definiteness, if we resorted
oftener to construction as a test. Of course, construction is not to be
limited to the making of things of three dimensions. The map, plan,
or artistic representation belongs to the same group, and is
developed from the same instinctive tendency.
Just one more word of caution needs to be given with regard to
work of this kind. In constructive work, whether with wood or clay, or
with pencil or brush, the point of departure should be the child’s idea,
not the model or pattern provided by an adult. After the child has
made his attempt, then let him see where he has failed by reference
to the object which he has tried to represent. And we can afford to be
satisfied in the beginning with a crude product, so long as it satisfies
the child. As for technique, there will come a time when the desire for
a better product will call for greater skill and will furnish the very best
possible motive for the necessary practice.
Imitation: In both play and constructive work a most important
element is the instinct to imitate. The child constantly imitates adult
activities in play, and in construction he represents the objects about
him. As has already been indicated, it is in this way that he clarifies
his ideas, that he gains experience. In imitation, which is truly
instructive, the child does not consciously plan to imitate; it is enough
that the model is present. This kind of imitation is sometimes called
spontaneous imitation, in contradistinction to the other type of
imitation, in which the individual persistently tries to reproduce the
activity of another. In the latter case he is conscious of the process;
and this type is sometimes called voluntary imitation. This distinction
is important for teachers in many phases of school work. There are
cases where the only satisfactory response is that which accords
with the model, the standard which society imposes. We do not want
a child to try to spell a word without being conscious of the form
commonly accepted. He will succeed in spelling because he has
studied this word, or is able to build it up from his knowledge of its
constituent parts. On the other hand, wherever creative work is to be
done, wherever originality is required, the educational value of the
exercise is inversely proportioned to the degree in which conscious
imitation of a model has entered to produce the result. In such
subjects as English composition, constructive work, science work
involving observation and experiment, what we want above all else is
the attempt on the part of the learner to express his own ideas; and it
is only after this expression that any adequate appreciation of model
or of criticism can be hoped for.
There is one other factor in connection with imitation which is of
great importance in teaching; namely, that children persistently
imitate what they admire. This has a double significance for the
teacher. Those things which can be made less attractive will tend to
be less imitated; and, conversely, that which is held up as worthy of
great respect will be much imitated. If we were only wise, we would
devote our attention to the leader of the group, trying to secure the
appropriate or desired reaction upon his or her part, rather than
devoting ourselves equally to the whole group. We can depend upon
it, the crowd will follow the leader whom they admire. Our appeals
often mean little to children, and the models which we set up have
little effect, because, however admirable these standards may seem
to us, they are beyond the power of children to comprehend or
admire. Instead of giving a boy a letter of Jefferson as a model,
better give him the one written by his classmate. Do not expect the
girl to imitate the noblest women in history, but make your appeal on
the basis of the virtue of the girl she likes.
Emulation: Much that has been said above under imitation might
quite as well have been written under the head of emulation. As
social beings, we tend to do what others do. Consciousness of kind
compels us to lay great store upon our ability to do as others do.
When in Rome the difficult thing is not to do “as Romans do,” but to
do otherwise. The desire to do not only as well as others, but to
accomplish more, is responsible for much that is achieved in the
world. If we did not have others with whom we are constantly
comparing ourselves, few of us would do as well as we now do.
Rivalry will always be one of the greatest means of bringing about
improvement or advancement in social conditions. In school, as well
as in the world at large, rivalry, if kept free from jealousy and envy,
will justify its existence by the results produced. The boy or girl who
is anxious to distance his fellows in school is apt to be the man of
ambition and of success in later life.
Pugnacity: More prominent in boys than in girls, but present in
some degree in every individual, is the instinct to fight, the desire not
to be overcome either by persons or conditions which surround us.
In so far as this instinct leads to physical encounter, for all except the
unusually strong physically, the correction comes by way of defeat.
For all, the substitution of games which involve physical prowess for
fighting, and the substitution of victories of intellect for the victories of
physical combat, point to the utilization of this instinct in education. It
is sometimes possible to appeal to this instinct when
discouragement and defeat in school tasks seem inevitable. No boy
likes to be told that he has been downed by the task in long division,
or that he has failed to make good in spelling or geography. The
whole world hates a quitter, and normal, healthy children are no
exception to the rule.
Curiosity: Children are proverbially curious about things. They
want to know more, to enlarge and make more definite their
experience. This desire shows itself in their actions in handling
materials, in making and unmaking, in questions asked, in
reasoning, in play, and in imitating others. The most striking
characteristic in the mental life of children is the breadth of their
interests, due to this instinct of curiosity. Most adults think along very
narrow and restricted lines; not so with children. While it is true that
they do little abstract thinking, there is scarcely an object or an action
which comes within the range of their senses that is not followed by
the desire to find out more.
Children have the spirit of inquiry, have many problems, in short,
are mentally active to a degree most uncommon among adults. The
problem of the teacher is how to keep alive this spirit of inquiry, how
to insure a continuance of this mental alertness. Much of our school
work has certainly tended in the opposite direction. Reciting what is
written in books, without thought or question, has too often been
characteristic of recitations. The appeal to authority, whether of the
teacher or of the book, instead of the appeal to experience, to
observation and experiment, or to other methods of establishing
truth, tends to kill rather than to strengthen the spirit of inquiry. We
should place greater value upon the intelligent question than upon
the parrot-like answer. Respect for the problems of children, even
when they seem of little account to us, rather than ridicule or
evasion, will tend to keep alive this most precious heritage. Of
course it is not wise to encourage the scatter-brained boy or girl who
never thinks about the same thing for two minutes in succession.
One great function of the teacher is to help children to concentrate
upon the main issue, to show a child that his question is irrelevant to
the problem under consideration, and to guide him on the path which
makes thinking pleasant and profitable.
It would be a good thing for every teacher to ask herself whether
while under her direction the children whom she teaches are usually
mentally alert, thinking, asking questions, or whether they concern
themselves only with repeating the thoughts of others. If there be
any doubt with regard to the children’s natural aptitude, let her
observe them when out of school and contrast the result. Mental
laziness is a habit acquired in spite of our initial advantage, in spite
of our desire for knowledge and the pleasure which comes from
thinking. The school and the teacher must always be judged by their
success in keeping children awake mentally; for it is power to learn
rather than knowledge which counts in later years, and learning is
most of all dependent upon the initial impulses toward inquiry.
Ownership: Very early in the life of the child the idea of personal
ownership develops. There can be no doubt concerning the
importance of this instinct in its effect upon the achievements of
men, but we are concerned chiefly, in dealing with children, with one
aspect of this tendency which is commonly known as the collecting
instinct. This desire to have the most complete collection of buttons,
postage stamps, pictures, birds’ eggs, shells, arrowheads, or
whatever else it may be, may often be utilized to great advantage.
Illustrative material for work in history, geography, nature study, and
to some degree for other subjects can be had in this way. Such a
collection will mean not only a much greater interest in the work, but
also a livelier appreciation of the subject, more images upon which to
base its generalizations. I have never seen a class that learned more
geography in a short time than was mastered by a class who
followed the American fleet around the world, collecting pictures,
products, and stamps for each of the countries visited, and writing a
full account of the country visited to accompany these illustrations.
Another class made most interesting collections in connection with
their study of colonial history. It is a mistake to suppose that ready-
made collections will answer the same purpose. They may illustrate
better, but the added interest and enthusiasm growing out of the
exercise of the collecting instinct will be wanting.
The collecting instinct may be utilized in work which deals with
ideas rather than things. Children may be just as keen in collecting
ideas about a subject in which they are much interested as in making
their collection of stones, or birds. The transition from the one type of
collecting to the other is apparent, in collections which are interesting
mainly for the ideas which they suggest.
The Social Instinct: The school has often overemphasized the
individualistic point of view. Competition is a legitimate motive; but if
all of school life centers around this motive, the child has lost much
in the non-exercise of that peculiarly human instinct which demands
coöperation and sympathy. At the foundation of our society is the
idea of working together for the common good. Boys and girls who
are to be most useful to their fellows, who are to do the most for
society, i.e. those who are truly educated, must have kept alive and
developed this spirit, more than altruistic, which sees in the good of
society the greatest individual gain. In a later chapter this topic will
be dealt with in considerable detail; suffice it to say here that many
opportunities should be found for group projects, for service on the
part of each member of the group of the sort that he is particularly
qualified to render.
 Wonder: The instinct of wonder or awe, closely related to or
possibly identified with the religious instinct, is one that our modern
critically scientific attitude tends to discourage. No one who has had
the experience can doubt the value of this element in mental life. To
wonder at the glory of the heavens will doubtless make more
difference in the lives of most men and women than the smattering
of astronomy they may acquire. The man who wonders at the
manifestation of the power of the forces of nature may get more real
joy out of life than he who feels that he has solved all of her
mysteries. We are not as a people remarkable for our reverence. It
may be well urged that our schools have often been responsible for
the opposite attitude. This instinct of wonder will thrive only in a
sympathetic atmosphere. No teacher can directly inculcate or
develop it. Only that teacher who has preserved and nurtured the
instinct in her own life can hope to be effective in keeping alive the
same spirit in children.
In the first chapter it was claimed that teachers should work to
develop the socially sympathetic, intelligent, and active individual,
and that the ends to be expected from any exercises might be
classified as habits, knowledge, interests, ideals or appreciations,
and methods of work. In our discussion of the native reactions of
children, we have endeavored to show that the possibilities of such
accomplishment are the common possession of normal children. It is
for the teacher who would accomplish these ends most economically
to discover the instinctive basis for the habit to be formed, the
knowledge to be acquired, interest to be awakened, or appreciation
to be aroused. The instinctive interests of children will furnish the
most powerful motives, and will serve as a basis for the most lasting
results. Even when the native reaction is undesirable, the successful
process may depend not merely upon negation, but upon a grafting
upon the original tendency of one that is socially desirable; or, in
other cases, the substitution of another reaction based upon some
other instinctive tendency. We may not always follow where instinct
seems to lead, but we can never ignore these native tendencies.
Whether we blindly ignore or attempt to work against nature, or
wisely utilize the instincts, the fact remains that all of our work is
conditioned by the native equipment.
 It has become more or less the fashion in recent years to decry
the theory of those who discuss the teaching process from the
standpoint of the child’s native tendencies, and with due regard to
his interests. The reactionary who continually harks back to the good
old times is still with us. The term of ridicule most commonly used in
lieu of argument is “soft pedagogy.” We are told that the only way to
develop men and women of strength is to begin by making sure that
we make our appeal on the basis of our superior authority, or even
brute strength, instead of finding the foundation for our work in the
instinctive curiosity and tendency to mental activity with which
children come to us. It is presumed by those who argue on the side
of the importance of authority that, unless children are compelled by
others to do hard tasks, they will never attempt anything that
involves effort. Again, they interpret interest to mean the blind
following of the child’s instinctive tendencies.
In our previous discussion we endeavored to show that education
concerns itself quite as much with the inhibition of undesirable
tendencies as with the encouragement of those which lead to
desirable activity. The process is not one of following where children
lead, but rather of availing ourselves of the native tendencies in
order that the ends we desire to achieve may be accomplished with
the least waste of time or energy. In reality, the choice between the
two positions is not whether we will have regard for childish instincts
and capacities, but rather whether we shall approach our task from
the standpoint of one who has faith in an appeal to the lower motive
of fear, or whether we believe that children are best prepared for
later activity who work out their own problems.
The best teaching can never consist in driving pupils to tasks
which they do not understand and which have little significance for
them. The standard of efficiency is found in ability to present to the
child a need, a purpose, or a problem which solicits his attention. It
may be that we shall be but imperfectly able to accomplish this
result, but, nevertheless, this must be our ideal. And it is not for
reasons of sentiment that we adopt it. The learning process is
explained in this way only. We make a new adjustment, reconstruct
our experience only in a situation which makes such a demand upon