2/1/24, 8:47 PM 4.

1 Introduction

C⚬MP⚬SING PR⚬GRAMS TEXT PROJECTS TUTOR ABOUT

Chapter 4
Hide contents Chapter 4: Data Processing

4.1 Introduction 4.1 Introduction

Modern computers can process vast amounts of data representing many aspects of the world. From
4.2 Implicit Sequences these big data sets, we can learn about human behavior in unprecedented ways: how language is
used, what photos are taken, what topics are discussed, and how people engage with their
4.2.1 Iterators
surroundings. To process large data sets efficiently, programs are organized into pipelines of
4.2.2 Iterables
manipulations on sequential streams of data. In this chapter, we consider a suite of techniques
4.2.3 Built-in Iterators
process and manipulate sequential data streams efficiently.
4.2.4 For Statements
4.2.5 Generators and Yield In Chapter 2, we introduced a sequence interface, implemented in Python by built-in data types such
Statements as list and range. In this chapter, we extend the concept of sequential data to include collections that
4.2.6 Iterable Interface have unbounded or even infinite size. Two mathematical examples of infinite sequences are the
4.2.7 Creating Iterables with Yield positive integers and the Fibonacci numbers. Sequential data sets of unbounded length also appear
4.2.8 Iterator Interface in other computational domains. For instance, the sequence of telephone calls sent through a cell
4.2.9 Streams tower, the sequence of mouse movements made by a computer user, and the sequence of
4.2.10 Python Streams acceleration measurements from sensors on an aircraft all continue to grow as the world evolves.
Continue: 4.2 Implicit Sequences
4.3 Declarative Programming

4.3.1 Tables
4.3.2 Select Statements
4.3.3 Joins
4.3.4 Interpreting SQL
4.3.5 Recursive Select
Statements
4.3.6 Aggregation and Grouping

4.4 Logic Programming

4.4.1 Facts and Queries

4.4.2 Recursive Facts

4.5 Unification

4.5.1 Pattern Matching

4.5.2 Representing Facts and
Queries
4.5.3 The Unification Algorithm
4.5.4 Proofs
4.5.5 Search

4.6 Distributed Computing

https://www.composingprograms.com/pages/41-introduction.html 1/2
2/1/24, 8:47 PM 4.1 Introduction

4.6.1 Messages
4.6.2 Client/Server Architecture
4.6.3 Peer-to-Peer Systems

4.7 Distributed Data

Processing

4.7.1 MapReduce
4.7.2 Local Implementation
4.7.3 Distributed Implementation

4.8 Parallel Computing

4.8.1 Parallelism in Python

4.8.2 The Problem with Shared
State
4.8.3 When No Synchronization
is Necessary
4.8.4 Synchronized Data
Structures
4.8.5 Locks
4.8.6 Barriers
4.8.7 Message Passing
4.8.8 Synchronization Pitfalls
4.8.9 Conclusion

Composing Programs by John DeNero, based on the textbook Structure and Interpretation of Computer Programs by Harold Abelson and Gerald Jay Sussman, is licensed under a Creative
Commons Attribution-ShareAlike 3.0 Unported License.

https://www.composingprograms.com/pages/41-introduction.html 2/2