0% нашли этот документ полезным (0 голосов)

638 просмотров206 страниц

Informatics Rescue Notes

Загружено:

Ivan Chernetsky

Авторское право

Мы серьезно относимся к защите прав на контент. Если вы подозреваете, что это ваш контент, заявите об этом здесь.

Доступные форматы

Скачать в формате PDF или читать онлайн в Scribd

0% нашли этот документ полезным (0 голосов)

638 просмотров206 страниц

Informatics Rescue Notes

Загружено:

Ivan Chernetsky

Авторское право

Доступные форматы

Скачать в формате PDF или читать онлайн в Scribd

Вы находитесь на странице: 1/ 206

Informatics Rescue

,
7 2012 .

1
1.1 O- . . . . . . . . . . . . . . . . . .
1.1.1 . . . . . . . . . . . . .
1.1.2 . . . . . . . . . . . . .
1.2 . . . . . . . . . . . . . . .
1.3 . . . . . . . . . . . .
1.4 . . . . . . . . . . .
1.5 . . . . . . . . . . . . . . . . . . . .
1.6 . . . . . . . . .
1.7 . . . . . . . . . . . .
1.8 . . . . . . . .
1.9 . . . . . . . . . . .
1.10 - . . . . . . . . . . . . . . . . .
1.10.1 - . . . . .
1.10.2 . . . . . . . .
1.10.3 . . . . . . . . . . . . . .
1.11 . . . . .
1.11.1 . . . . . . . . . .
1.11.2 . . . . . . . . . . . . .
1.11.3 . . . . . .
1.12 C . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

6
6
6
7
7
8
9
10
12
14
16
16
18
18
18
19
20
21
22
24
26

2 Python
2.1
2.1.1 NoneType . . . . . . . . . . . . . . . . . . .
2.1.2 . . . . . . . . . . . . . . . . . . .
2.1.3 . . . . . . . . . . . . . . .
2.1.4 . . . . . . . . . . . . .
2.1.5 . . . . . . . . . . . . . . . . . . . .
2.1.6 . . . . . . . . . . . . . . . . . .
2.2 . . . . . . . . . . . . . . . . . . . . . .
2.3 . . . . . . . . . . . . . . . . . . . . . . .
2.4 . . . . . . .
2.4.1 . . . . . . . . .
2.4.2 . . . . . . . . . . . . . . . . . . .
2.4.3 . . . . . . . . . . . . . . . . . .
2.4.4 . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.

28
29
29
29
30
34
40
43
45
49
52
52
54
54
55

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

55
56
58
59
62
64
65
66
66
67
68
72
72
75
78

3
3.1 . . . . . . . . . .
3.1.1 . . . . . . . . .
3.1.2 . . . . . . . . . . . . .
3.2 . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.1 . . . . . . . . . . . . . . . . . . .
3.2.2 : . . . . . . . . . .
3.2.3 :
3.2.4 . . . . . . . . . . . . . . . . . . . .
3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1 . . . . . . . . . . . . . . . . . . . . . .
3.3.2 . . . . . . . . . . . . . . . . . . .
3.3.3 . . . . . . . . . . . . . . . .
3.3.4 . . . . . . . . . . . . . . . .
3.3.5 . . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.

81
81
81
83
84
84
85
88
93
93
93
93
93
93
93

4
4.1 TCP . . . . . . . . . . . . . . . . . . . . . . .
4.1.1 . . . . . . . . . . . .
4.1.2 TCP . . . . . . . . . . . . . . . . . .
4.1.3 TCP . . . . . . . . . .
4.1.4 TCP . . . . . . . . . . .
4.1.5 TIME_WAIT . . . . . . . . . . . . .
4.2 UDP . . . . . . . . . . . . . . . . . . . . . . .
4.2.1 UDP TCP . . . . . .
4.3 POSIX API . . . . . . . . . . . . . . . . . .
4.3.1 DAYTIME . . . . . . . . .
4.3.2 DAYTIME ,
4.3.3 DAYTIME . . . . . .
4.3.4 / . . . . . .
4.3.5 .
4.4 . . . . . . . . . . . . .
4.4.1 . . . . . . . . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

94
94
94
95
95
96
96
98
98
98
98
102
104
107
111
117
118

2.5
2.6
2.7

2.8
2.9
2.10
2.11
2.12
2.13

2.4.5 . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
2.6.1 . . . . . . . . . .
-
2.7.1 . . . . . . . . . . . . . . . . . .
2.7.2 MRO . . . . . . . .
2.7.3 . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
2.10.1 . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .

2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

3
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.

119
120
120
121
121
122
124
124
127
127
127
127
128

5
5.1 - . . . . . . . . . . . . . . . . . . . .
5.1.1 . . . . . . . . . . . . . . . . . . . . . . .
5.2 . . . . . . . . . . . . . . . . . . . . .
5.2.1 . . . . . . . . . . . . . . . . . . . . . . .
5.3 . . . . . . . . . . . . . . . . . . . . . . . . .
5.3.1
5.3.2 . . . . . . . . . . . . . . . . . . . . . . .
5.4 B- . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.1 B- . . . . . . . . . . . . . . .
5.4.2 . . . . . . . . . . . . . . . . . . . . . . .
5.5 . . . . . . . . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.

132
132
132
134
134
136
136
137
138
138
139
140

6 Common Lisp
6.1 . . . . . . . . . . . . . . . . . . . . . . .
6.1.1 . . . . . . . . . . . .
6.1.2 . . . . . . .
6.1.3 . . . . . . . . .
6.1.4 . . . . . . . . . . . .
6.2 . . . . . . . . . . . . . . . . . . . . .
6.2.1 . .
6.2.2 ()
6.2.3 . . . . . . . . . . . . . . . . . .
6.2.4 . . . . . . . . . . . . . . . .
6.3 . . . . . . . . . . . . . . . . . . . . . . .
6.3.1 . . . . . . . . . . . . . .
6.3.2 . . . . . . . . . . . . . . . . . .
6.3.3 . . . . . . . . . . . .
6.3.4 . . . . . . . . . . . . .
6.4 . . . . . . . . . . . .
6.4.1 . . . . . . . . . . . . . . . . . . . . .
6.4.2 . . . . . . . . . . . . . . . . . . . . .
6.4.3 . . . . . . . . . . . . . . . . . . . .
6.5 . . . . . . . . . . . . . . . . . . . . . .
6.5.1 . . . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

144
144
145
147
147
148
148
149
149
150
150
151
152
152
153
154
154
154
156
156
156
157

4.5

4.6

4.4.2 . . . . . . . . . . . . . . . . . . . . . .
4.4.3 . . . . . . . . . . . . . . . . . . . . .
4.4.4
4.4.5 NAT . . . . . . . . . . . . . .
4.4.6 ARP . . . . . . . . . . . . . . . . . . . . . . . .
4.4.7 ICMP . . . . . . . . . . . . . . . . . . . . . . .
4.4.8 DHCP . . . . . . . . . . . . . . . . . . . . . . .
4.4.9 DNS . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
4.5.1 - . . . . . . . . . . . . .
4.5.2 . . . . . . . . . . . . . . . .
4.5.3 . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

6.6

6.7

6.8

6.9

6.10

6.11
6.12

6.13

6.5.2 . . . . .
6.5.3 - . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . .
6.6.1 Cons- . . . . . . . . . . . . . . . . . .
6.6.2 . . . . . . . . . . . . .
6.6.3 . . . . . . . . . . . . . . . . . .
6.6.4 . . . . . . . . . . . . . .
6.6.5 . . . . . . . . . . . .
6.6.6 . . . . . . . . . . . .
6.6.7 Destructuring-bind . . . . . . . . . . . . . . .
Loop . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.7.1 . . . . . . . .
6.7.2 . . . . . . . . . . . . . .
Format . . . . . . . . . . . . . . . . . . . .
6.8.1 . . . . . . . . . . . . . . . . . . .
6.8.2 . . . . . . . . . .
6.8.3 . . . . . . . . . . . . . . . . . . . .
6.8.4 . . . . . . . .
CLOS . . . . . . . . . . . . . . . . . . . . . . . . . .
6.9.1 . . . . . .
6.9.2 . . . . . . . . . .
6.9.3 . . . . . . . . . . . . . . . . . . . . .
6.9.4 . . . . . . . . . . . . . .
6.9.5 . . . . . . . . . . .
6.9.6 . . . . . . . . . . . . . . .
6.9.7 . . . . . . . . . . . . . . . . . .
6.9.8 . . . . . . . . . . . .
6.9.9 . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . .
6.10.1 . . . . . . . . . . . . . . . .
6.10.2 . . . . . . . . . . . .
6.10.3 . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . .
6.11.1 . . . . . . . . . . . . .
:
6.12.1 . . . . . . . . . . . . .
6.12.2 . . . . . . . . . . . . . . . . . . .
6.12.3 . . . . . . . .
6.12.4 . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . .

7
7.1 . . . . . . . . . . . . . . . .
7.1.1 . . . . . . . . .
7.2 . . . . . . . . . . . . . . . . . . .
7.2.1 k . . . . . . . . . . .
7.2.2 . . . . . . . . . . . . . . . . . .
7.2.3 k . . . .
7.2.4 :

.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

157
159
160
160
160
161
162
163
163
164
164
165
168
171
171
173
174
174
174
174
176
179
179
179
181
182
182
183
183
183
184
184
185
186
186
186
187
188
189
189

.
.
.
.
.
.
.

194
194
195
197
197
198
200
201

7.3

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
7.3.1 k . . . . . . . . . . . . . . . . . . . . . . . . . 201
7.3.2 kd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

1.1

O-

.
. .
, .
an2 + bn + c, a, b c .
n ,
n2 . ,
(n2 ).
, .

1.1.1

f (n) O(g(n))

f g ( )
f g ( )
f g
g f
f g

f (n) (g(n))
f (n) (g(n))
f (n) o(g(n))
f (n) (g(n))

1.1.2

f (n) O(g(n))

= c > 0, n0

n > n0

f (n) c g(n),

(1.1)

f (n) (g(n))

= c > 0, n0

n > n0

c g(n) f (n),

(1.2)

f (n) (g(n))

= c1 > 0, c2 > 0, n0

f (n) o(g(n))

= > 0 n0

n > n0

f (n) < g(n),

(1.4)

f (n) (g(n))

= c > 0 n0

n > n0

c g(n) < f (n).

(1.5)

n > n0

c1 g(n) f (n) c2 g(n),

1.2

(1.3)

.
log n.
:
, .
, .
:
, , ,
.
, , , .
Listing 1.1:
1

int binary_search ( int elem , int array [] , size_t length )

{
int mid , min = 0 , max = length - 1;
do {
mid = min + ( max - min ) / 2;
if ( elem > array [ mid ])
min = mid + 1;
else
max = mid - 1;
} while ( min <= max && array [ mid ] != elem ) ;

if ( array [ mid ] == elem )

return mid ;
/* elem is not found */
return -1;

Listing 1.2:

int b i n a r y _ s e a r c h _ a u x ( int elem , int array [] , int low , int high )

{
int mid ;
if ( high < low )
/* elem is not found */
return -1;

mid = low + ( high - low ) / 2;

if ( array [ mid ] > elem )
return b i n a r y _ s e a r c h _ a u x ( elem , array , low , mid - 1) ;
if ( array [ mid ] < elem )
return b i n a r y _ s e a r c h _ a u x ( elem , array , mid + 1 , high ) ;

return mid ;
}
17

int b i n a r y _ s e a r c h _ r e c ( int elem , int array [] , size_t length )

{
return b i n a r y _ s e a r c h _ a u x ( elem , array , 0 , length - 1) ;
}

1.3

quicksort. :
;
:
1. , ;
2. , ;
.

(n2 )
(n log n)
(n log n)

,
, .
Listing 1.3:

static inline void swap ( int a [] , int i , int j )

{
int tmp = a [ i ];
a [ i ] = a [ j ];
a [ j ] = tmp ;
}
void quicksort ( int a [] , int n )
{

int i , last = 0;
if ( n <= 1)
return ;
14

swap (a , 0 , rand () % n ) ;
for ( i = 1; i < n ; i ++)
if ( a [ i ] < a [0])
swap (a , ++ last , i ) ;
swap (a , 0 , last ) ;

quicksort (a , last ) ;
quicksort ( a + last + 1 , n - last - 1) ;
}

1.4

, .

(1)
(n)
(1)
(n)

. (n),
n . n1 , ,
1 .
Listing 1.4:

struct ea {
int count ;
int max ;
int * a ;
};
enum {
EA_INIT = 1 ,
EA_GROW = 2
};
int add ( struct ea * ea , int value )
{
if ( NULL == ea - > a ) {
ea - > a = ( int *) malloc ( EA_INIT * sizeof ( int ) ) ;
if ( NULL == ea - > a )
return -1;
ea - > max = EA_INIT ;
ea - > count = 0;
} else if ( ea - > count == ea - > max ) {
int * a = ( int *) malloc ( EA_GROW * ea - > max * sizeof ( int ) ) ;
if ( NULL == a )
(amortized analysis).

return -1;
memcpy (a , ea - >a , ea - > count * sizeof ( int ) ) ;
free ( ea - > a ) ;

ea - > max *= EA_GROW ;

ea - > a = a ;
}
32

ea - > a [ ea - > count ] = value ;

return ea - > count ++;
}
37

int del ( struct ea * ea , int value )

{
for ( int i = 0; i < ea - > count ; i ++)
if ( value == ea - > a [ i ]) {
memmove ( ea - > a + i , ea - > a + i + 1 ,
( ea - > count - i - 1) * sizeof ( int ) ) ;
ea - > count - -;
return 1;
}

return 0;
}

Listing 1.5:
1

struct ea array ;
( void )
( void )
( void )
( void )
( void )

add (& array ,

add (& array ,
add (& array ,
add (& array ,
add (& array ,

5) ;
3) ;
2) ;
7) ;
6) ;

for ( int i = 0; i < array . count ; i ++)

printf ( " % d " , array . a [ i ]) ;
printf ( " \ n " ) ;
( void ) del (& array , 2) ;
( void ) del (& array , 7) ;

1.5

for ( int i = 0; i < array . count ; i ++)

printf ( " % d " , array . a [ i ]) ;
printf ( " \ n " ) ;

. . , - ;
.

(n)
(n)
(1)

.
Listing 1.6:

struct list {
int data ;
struct list * next ;
};
struct list * make_item ( int data )
{
struct list * item = ( struct list *) malloc ( sizeof ( struct list ) ) ;
item - > data = data ;
item - > next = NULL ;

return item ;
}

struct list * add_front ( struct list * head , struct list * item )

{
item - > next = head ;
return item ;
}
struct list * add_back ( struct list * head , struct list * item ) {
struct list * p ;
if ( NULL == head )
return item ;

for ( p = head ; p - > next != NULL ; p = p - > next )

;
p - > next = item ;
return head ;
32

}
# define for_each (n , head )
for ( n = ( head ) ; n != NULL ; n = n - > next )

\
\

struct list * remove_item ( struct list * head , int data )

{
struct list * current , * prev = NULL ;
for_each ( current , head ) {
if ( data == current - > data ) {
if ( NULL == prev )
head = current - > next ;
else
prev - > next = current - > next ;

free ( current ) ;
return head ;
}

prev = current ;

}
return head ;
}
57

void print_ forwar ds ( struct list * head )

{
struct list * next ;
for_each ( next , head )
printf ( " % x " , next - > data ) ;

void p ri nt _ ba ck w ar d s ( struct list * item )

{
if ( NULL == item )
return ;
p ri nt _ ba ck w ar ds ( item - > next ) ;
printf ( " % x " , item - > data ) ;

Listing 1.7:
struct list * head = NULL ;
2

head
head
head
head
head

=
=
=
=
=

add_front ( head , make_item (1) ) ;

add_front ( head , make_item (2) ) ;
add_front ( head , make_item (3) ) ;
add_back ( head , make_item (4) ) ;
add_back ( head , make_item (5) ) ;

pri nt_for wards ( head ) ;

printf ( " \ n " ) ;
p ri nt _ ba ck w ar ds ( head ) ;
printf ( " \ n " ) ;
head = remove_item ( head , 3) ;
head = remove_item ( head , 1) ;
head = remove_item ( head , 5) ;

pri nt_for wards ( head ) ;

printf ( " \ n " ) ;
p ri nt _ ba ck w ar ds ( head ) ;
printf ( " \ n " ) ;

1.6

. 0..2 . , ,
. .
, :
1. .
2. .

3. .

(log n)
(n)
(log n)
(n)

(n)
(n)

, 1..10, .
Listing 1.8:

struct tree {
int data ;
struct tree * left ;
struct tree * right ;
};
struct tree * make_node ( int data )
{
struct tree * node = ( struct tree *) malloc ( sizeof ( struct tree ) ) ;
node - > data = data ;
node - > left = NULL ;
node - > right = NULL ;

return node ;
}

struct tree * insert ( struct tree * root , struct tree * node )

{
if ( NULL == root )
return node ;
if ( root - > data < node - > data )
root - > right = insert ( root - > right , node ) ;
else if ( root - > data > node - > data )
root - > left = insert ( root - > left , node ) ;
/* skipping items that are already in the struct tree */

return root ;

void a p p l y _ f n _ p r e o r d e r ( struct tree * root , void (* fn ) ( struct tree *) )

{
if ( NULL == root )
return ;
fn ( root ) ;
a p p l y _ f n _ p r e o r d e r ( root - > left , fn ) ;
a p p l y _ f n _ p r e o r d e r ( root - > right , fn ) ;

void print_node ( struct tree * node ) {

printf ( " % d \ n " , node - > data ) ;
}
void print_tree ( struct tree * root )

{
a p p l y _ f n _ p r e o r d e r ( root , & print_node ) ;
49

Listing 1.9:
1

struct tree * root = NULL ;

root
root
root
root
root

=
=
=
=
=

insert ( root ,
insert ( root ,
insert ( root ,
insert ( root ,
insert ( root ,

make_node (8) ) ;
make_node (3) ) ;
make_node (1) ) ;
make_node (6) ) ;
make_node (10) ) ;

print_tree ( root ) ;

1.7

, :
2 .
d 1, d.
.
.

max
max

(1)
(log n)
(log n)

.
, .
( , ). ,
, . , , ,
, .
Listing 1.10:
1

# define HEAP_SIZE 50

struct binheap {
int data [ HEAP_SIZE ];
int size ;
};
static inline int topos ( int i )

2 , , , ,
.

1.
{
return i + 1;
11

static inline int toindex ( int i )

{
return i - 1;
}

static inline int parent ( int i )

{
return toindex ( topos ( i ) / 2) ;
}

static inline int left ( int i )

{
return toindex (2 * topos ( i ) ) ;
}

static inline int right ( int i )

{
return toindex (2 * topos ( i ) + 1) ;
}

static inline void swap ( int a [] , int i , int j )

{
int tmp = a [ i ];
a [ i ] = a [ j ];
a [ j ] = tmp ;
}
void max_heapify ( struct binheap * heap , int i )
{
int l = left ( i ) ;
int r = right ( i ) ;
int largest = i ;

if ( l < heap - > size && heap - > data [ l ] > heap - > data [ i ])
largest = l ;
if ( r < heap - > size && heap - > data [ r ] > heap - > data [ largest ])
largest = r ;

if ( largest != i ) {
swap ( heap - > data , i , largest ) ;
max_heapify ( heap , largest ) ;
}
}

void build_ max_he ap ( struct binheap * heap )

{
if ( heap - > size <= 1)
return ;
61

for ( int i = toindex ( heap - > size / 2) ; i >= 0; i - -)

max_heapify ( heap , i ) ;
}

Listing 1.11:
1

int i ;
struct binheap heap = {

. data = { 9 , 1 , 11 , 13 , 7 , 3 , 15 , 17 , 5 } ,
. size = 9
};
6

bui ld_max _heap (& heap ) ;

for ( i = 0; i < heap . size ; i ++)
printf ( " % d " , heap . data [ i ]) ;

1.8

heapsort. :
in-place.
() . [1..(n 1)]
. .

(n log n)
(n), O(n log n)
(n log n)

Listing 1.12:
1

void heapsort ( struct binheap * heap )

{
int size = heap - > size ;
bui ld_max _heap ( heap ) ;
for ( int i = size - 1; i > 0; i - -) {
swap ( heap - > data , 0 , i ) ;
heap - > size -= 1;
max_heapify ( heap , 0) ;
}

heap - > size = size ;

}

1.9

, :
Insert(S, x) x S;
M aximum(S) ;
ExtractM ax(S) , S;
IncreaseKey(S, x, k) x k, .

. .
heap_maximum , , .
heap_extract_max , heap_maximum, . :
.
.
max_heapif y.
heap_increase_key , . :
.
,
.
heap_insert k .
:
.
heap_increase_key k.
Listing 1.13:

int heap_maximum ( struct binheap * heap )

{
return heap - > data [0];
}
int h e a p _ e x t r a c t _ m a x ( struct binheap * heap )
{
int max = heap - > data [0];
heap - > size -= 1;
heap - > data [0] = heap - > data [ heap - > size ];
max_heapify ( heap , 0) ;

return max ;
}
17

int h e a p _ i n c r e a s e _ k e y ( struct binheap * heap , int i , int k )

{
if ( k < heap - > data [ i ])
return -1;
22

heap - > data [ i ] = k ;

while ( i > 0 && heap - > data [ parent ( i ) ] < heap - > data [ i ]) {
swap ( heap - > data , i , parent ( i ) ) ;
i = parent ( i ) ;
}
return 0;

}
32

void heap_insert ( struct binheap * heap , int k )

{
heap - > data [ heap - > size ] = INT_MIN ;
heap - > size += 1;
h e a p _ i n c r e a s e _ k e y ( heap , heap - > size - 1 , k ) ;
}

Listing 1.14:

int i ;
struct binheap heap = {
. data = { 9 , 1 , 11 , 13 , 7 , 3 , 15 , 17 , 5 } ,
. size = 9
};
bui ld_max _heap (& heap ) ;
heap_insert (& heap , 8) ;
heap_insert (& heap , 19) ;
( void ) h e a p _ e x t r a c t _ m a x (& heap ) ;
( void ) h e a p _ i n c r e a s e _ k e y (& heap , 6 , 12) ;
for ( i = 0; i < heap . size ; i ++)
printf ( " % d " , heap . data [ i ]) ;

1.10

,
. -.
- .
.

O(1)
(n)
O(1)

(n)

1.10.1

- () .
. k
k m, . h(k) = k mod m.
- m. , m = 2p , h(k) p k.
- .
. h(k) = bm(kA mod 1)c, 0 < A < 1.

m.

51
1
A = 2 0.6180339887...

1.10.2

. - . , , .

. -.
, .
, - ;

,
- .
.
-, [0.7; 0.75],
-, > 0.75 ( , ).
:
0

. h(k, i) = (h (k) + i) mod m, h

-, i [0, m 1].
0

. h(k, i) = (h (k) + c1 i + c2 i2 ) mod m, h -, i [0, m1], c1 c2 ,

0.
. h(k, i) = (h1 (k) + ih2 (k)) mod m, h1 h2 -, i [0, m 1].
, h(k, 0), h(k, 1), ..., h(k, m 1)
0, 1, ..., m 1, - .

1.10.3

- m.
.
Listing 1.15:

# define HTABLE_SIZE
# define MULTIPLIER

20
31

struct item {
char * name ;
int value ;
struct item * next ;
};
struct item * make_item ( char * name , int value )
{
struct item * item = ( struct item *) malloc ( sizeof ( struct item ) ) ;
item - > name = name ;
item - > value = value ;
item - > next = NULL ;

return item ;
}

unsigned int hash ( char * str )

{
unsigned int h = 0;
unsigned char * p = ( unsigned char *) str ;

for (; * p != \0 ; p ++)
h = MULTIPLIER * h + * p ;
return h % HTABLE_SIZE ;

struct item * lookup ( struct item * htable [] , char * name , bool create , int value
)
{
int h = hash ( name ) ;
struct item * i = htable [ h ];
for (; i != NULL ; i = i - > next )
if ( strcmp ( name , i - > name ) == 0)
return i ;

if ( create ) {
i = make_item ( name , value ) ;
i - > next = htable [ h ];
htable [ h ] = i ;
}
return i ;

}
Listing 1.16:
struct item * htable [ HTABLE_SIZE ] = { NULL };

char * strings [] = { " string1 " , " string2 " , " string3 " ,
" string4 " , " string5 " };
int i ;
struct item * it ;
for ( i = 0; i < 3; i ++)
lookup ( htable , strings [ i ] , true , i + 1) ;
for ( i = 0; i < 5; i ++) {
it = lookup ( htable , strings [ i ] , false , 0) ;
if ( it )
printf ( " % s in hash table , value = % d \ n " , it - > name , it - > value ) ;
else
printf ( " % s somewhere else \ n " , strings [ i ]) ;
}

1.11

, ee
ee , , , , .
.
:
1. .
2. , .
3. ,
.

4. , .

1.11.1

,

.
Fn :

n = 0
0
1
n = 1
Fn =

Fn1 + Fn2

. F5 :
F5 = F4 + F3
= F3 + F2 + F2 + F1
= F2 + F1 + F1 + F0 + F1 + F0 + 1
= F1 + F0 + 1 + 1 + 0 + 1 + 0 + 1
=1+0+1+1+0+1+0+1
= 5,
, F5 , F4 1 , F3 2 , F2 3
, F1 5 , F0 3 .
Listing 1.17:

unsigned long fib_slow ( unsigned long n )

{
if ( n <= 1)
return n ;
return fib_slow ( n - 2) + fib_slow ( n - 1) ;
}

F100 ,
.
(n),
(n) . t,
ti = ti1 + ti2 , 2 i n.
Listing 1.18:
unsigned long fib_fast ( unsigned long n )
{
unsigned long t [ n + 1];
4

if ( n <= 1)
return n ;

t [0] = 0;
t [1] = 1;

for ( unsigned long i = 2; i <= n ; i ++)

t [ i ] = t [ i - 2] + t [ i - 1];
return t [ n ];
14

(n) , ,
Fi Fi1 Fi2 .
Listing 1.19:
1

unsigned long fib_fastest ( unsigned long n )

{
unsigned long t [] = { 0 UL , 1 UL };
unsigned long tmp ;
if ( n <= 1)
return n ;

for ( unsigned long i = 2; i <= n ; i ++) {

tmp = t [0];
t [0] = t [1];
t [1] = t [1] + tmp ;
}

return t [1];
16

1.11.2

K = {k0 , k1 , ..., kk }, ki N, k0 = 1 .
. : k0 = 1 ., k1 = 5 ., k2 = 10
., k3 = 25 .
, K , kk > kk1 > ... > k0 .
.
,
n .
. Ci,j k0 , k1 , ..., ki , j. Ci,j
:
1. , ki ( ),
Ci,j = Ci1,j .
2. ki
,
Ci,j = 1 + Ci,jki .

Ci,j = min(Ci1,j , 1 + Ci,jki ).

.
, , j j 1 .:

j < 0

0
j = 0
Ci,j =
i = 0
j

min(Ci1,j , 1 + Ci,jki )
j 1
.
Ci,j , .
# define array_size ( a )
( sizeof ( a ) / sizeof (( a ) [0]) )

\
\

# define min (a , b )
({ __typeof__ ( a ) _a = ( a ) ;
__typeof__ ( b ) _b = ( b ) ;
_a < _b ? _a : _b ; })

\
\
\

void change_coins ( unsigned int n , unsigned int k , unsigned int coins [ k ] ,

unsigned int solution [ k ][ n + 1])
{
unsigned int i , j ;

for ( i = 0; i < k ; i ++)

solution [ i ][0] = 0;
for ( j = 1; j <= n ; j ++)
solution [0][ j ] = j ;

for ( i = 1; i < k ; i ++)

for ( j = 1; j <= n ; j ++)
if ( j < coins [ i ])
solution [ i ][ j ] = solution [ i - 1][ j ];
else
solution [ i ][ j ] = min ( solution [ i - 1][ j ] ,
1 + solution [ i ][ j - coins [ i ]]) ;

solution[k - 1][n] ,
.
. Ck,n ,
.
void recover_coins ( unsigned int n , unsigned int k , unsigned int coins [ k ] ,
unsigned int solution [ k ][ n + 1] , unsigned int counts [ k ])
{
unsigned int i , j ;
5

for ( i = 0; i < k ; i ++)

counts [ i ] = 0;

i = k - 1;
j = n;
while ( true ) {
if (0 == j )
break ;

1.
15

if (0 == i ) {
counts [0] += j ;
break ;
}

if ( solution [ i - 1][ j ] < (1 + solution [ i ][ j - coins [ i ]]) )

i -= 1;
else {
counts [ i ] += 1;
j -= coins [ i ];
}

}
}

Listing 1.20:

unsigned
unsigned
unsigned
unsigned
unsigned
unsigned

int
int
int
int
int
int

i, j;
n = 8;
coins [] = { 1 , 4 , 6 };
k = array_size ( coins ) ;
solution [ k ][ n + 1];
counts [ k ];

change_coins (n , k , coins , solution ) ;

for ( i = 0; i < k ; i ++) {
for ( j = 0; j <= n ; j ++)
printf ( " % d " , solution [ i ][ j ]) ;
printf ( " \ n " ) ;
}
if (0 == solution [ k - 1][ n ])
printf ( " No solution \ n " ) ;
else {
recover_coins (n , k , coins , solution , counts ) ;
for ( i = 0; i < k ; i ++)
printf ( " % d kop .: % d \ n " , coins [ i ] , counts [ i ]) ;

1.11.3

, ,
, , .
. S1 S2 m n . d(m, n).
d(i, j) i 0 m j 0 n.

d(i, 0) = i

d(0, j) = j,
, S1 [1..i] , i .
S2 [1..j] .

. d(i, j) . , ,
( S1 [i] = S2 [j]).
, S2 (j) .
, ( ), Si [1..i]
S2 [1..j 1], . , d(i, j) = d(i, j 1) + 1.
: , S1 [i].
S1 [1..i 1] S2 [1..j]. d(i, j) = d(i 1, j) + 1.
, S1 [i] S2 [j]. S1 [1..i 1] S2 [1..j 1], .
: d(i, j) = d(i 1, j 1) + 1 .
S1 [i] = S2 [j], , d(i, j) = d(i 1, j 1),
S1 [1..i] S2 [1..j] S1 [1..i 1] S2 [1..j 1].
. , d(i, j) :

0
i = 0, j = 0

i
i > 0, j = 0

j
i = 0, j > 0

min(
,
d(i, j) =
d(i, j 1) + 1,

d(i 1, j) + 1,

d(i
1, j 1) + t(S1 [i], S2 [j])

)
i > 0, j > 0
t(a, b) a = b, .
. . distances[m][n] .
# define min (a , b )
({ __typeof__ ( a ) _a = ( a ) ;
__typeof__ ( b ) _b = ( b ) ;
_a < _b ? _a : _b ; })

\
\
\

void distance ( char * word1 , char * word2 , unsigned int * distances )

{
unsigned int m = strlen ( word1 ) + 1;
unsigned int n = strlen ( word2 ) + 1;
unsigned int i , j ;
unsigned int add , del , change ;
for ( i = 0; i < m ; i ++)
distances [ i * n ] = i ;
for ( j = 1; j < n ; j ++)
distances [ j ] = j ;
for ( i = 1; i < m ; i ++)
for ( j = 1; j < n ; j ++) {
add = distances [ i * n + ( j - 1) ] + 1;
del = distances [( i - 1) * n + j ] + 1;
change = ( word1 [ i - 1] == word2 [ j - 1]) ? 0 : 1;

1.
change += distances [( i - 1) * n + ( j - 1) ];
distances [ i * n + j ] = min ( min ( add , del ) , change ) ;

}
}

Listing 1.21:

char * word1 = " cat " ;

char * word2 = " kate " ;
unsigned int m = strlen ( word1 ) + 1;
unsigned int n = strlen ( word2 ) + 1;
unsigned int distances [ m ][ n ];
unsigned int i , j ;
distance ( word1 , word2 , ( unsigned int *) distances ) ;
for ( i = 0; i < m ; i ++) {
for ( j = 0; j < n ; j ++)
printf ( " % u " , distances [ i ][ j ]) ;
printf ( " \ n " ) ;
}

1.12

right-left.
2["abc"]. , ?
. # ##.

[1] . , . , . , . , : . , 2- ., 2005.
[2] . , , . .: ; -, 2003.
[3] . , . , . , 2005.
[4] . , . , . , 5- ., 2009.
[5] . , Linux. , 2- ., 2008.
[6] . , . , . , 2- ., 2008.
[7] . , : . , 2006.
[8] . , . . , 3- ., 2007.

Python
Python, REPL ( Read, Eval, Print, Loop),
python ( ipython). Python,
.
.
Listing 2.1:
% ipython
Python 2.7.2 ( default , Jun 29 2011 , 15:07:32)
Type " copyright " , " credits " or " license " for more information .
5

IPython 0.10.2 -- An enhanced Interactive Python .

?
-> Introduction and overview of IPython s features .
% quickref -> Quick reference .
help
-> Python s own help system .
object ?
-> Details about object . ? object also works , ?? prints more .

In [1]: 2 ** 32
Out [1]: 4294967296

In [2]: dir ( None )

Out [2]:
[ __class__ , __delattr__ , __doc__ , __format__ , __getattribute__ ,
__hash__ , __init__ , __new__ , __reduce__ , __reduce_ex__ ,
__repr__ , __setattr__ , __sizeof__ , __str__ , __subclasshook__ ]
In [3]: map ?
Type :
builtin_function_or_method
Base Class : < type builtin_function_or_method >
String Form : < built - in function map >
Namespace : Python builtin
Docstring :
map ( function , sequence [ , sequence , ...]) -> list
Return a list of the results of applying the function to the items of
the argument sequence ( s ) . If more than one sequence is given , the
function is called with an argument list consisting of the corresponding
item of each sequence , substituting None for missing values when not all
sequences have the same length . If the function is None , return a list
of the items of the sequence ( or a list of tuples if more than one
sequence ) .

2. PYTHON

2.1

Python . , . , . Python
. . - , , , TypeError
AttributeError.
. ,
.
(mutable immutable ).

2.1.1

NoneType

None. , , , . None
1 ..

2.1.2

False , True .
( !)
if while. :

None;
False;
;
;
();
, , __nonzero__
() __len__() , False .
class Zero ( object ) :
def __nonzero__ ( self ) :
return False
5

class ZeroLen ( object ) :

def __len__ ( self ) :
return 0
>>> bool ( Zero () )
False
>>> bool ( ZeroLen () )
False

>>> bool ({})

False

Python 2.x , 3.x .

2. PYTHON

>>> bool ([])

False

.

x or y. x , x, y. , .
>>> 0 or 2 or 1 or 0
2
x and y. x , , y
. y,
.
>>> 1 and 2 and 0 and 3
0
not x.

2.1.3

x. , False, .

Python : int, long2 , float complex. int long

C. long Python . float Python
double C.
, double C.
x + yj, x y
.

>>> z = 1.3 + 0.5 j

>>> z
(1.3+0.5 j )
>>> z . real
1.3
>>> z . imag
0.5
>>> z . conjugate ()
(1.3 -0.5 j )

, int, long, float
complex.

>>> int (2)

2
>>> int (2.3)
2
>>> float (2)
2.0
>>> complex (2 , 4)
(2+4 j )
Python 3.x , int .

2. PYTHON

: +x, -x, ~x. + . - .
~ , -(x + 1).

>>> x = 111
>>> + x
111
>>> -x
-111
>>> bin ( x )
0 b1101111
>>> bin (~ x )
-0 b1110000

: x + y, x - y, x * y, x // y, x / y, x % y, x ** y. . / //
. ,
/ 3 . // b x
y c. %
; . ** x
y.
:
int, long.

>>> 7 + 5
12
>>> 7 - 5
2
>>> 7 * 5
35
>>> 7 / 5
1
>>> 7.0 / 5
1.4
>>> 7.0 // 5
1.0
>>> 7 % 5
2
>>> 3.14 % 0.7
0.3400000000000003
>>> 2**100
1267650600228229401496703205376 L

: x & y, x ^ y, x | y. . int long. & (AND); ^ (XOR); |

(OR).

>>> hex (0 xc4 & 0 x72 )

0 x40
>>> hex (0 xc4 ^ 0 x72 )
0 xb6
>>> hex (0 xc4 | 0 x72 )
0 xf6

3 Python 3.x . / float. ,

Python 2.x, from __future__ import division. .
PEP 238 : http://www.python.org/dev/peps/pep-0238/

2. PYTHON

: x << y x >> y. . int long. x y

. << x * pow(2, y), >>
x / pow(2, y).
>>> 1 << 5 == 1 * pow (2 , 5)
True
>>> 256 >> 5 == 256 / pow (2 , 5)
True

Python 4 .
,
.
1

class Id e nt it y El em e nt ( object ) :
def __repr__ ( self ) :
return " Id e nt it y El em e nt "
def __str__ ( self ) :
return self . __repr__ ()
def __add__ ( self , other ) :
return other
def __radd__ ( self , other ) :
return other
def __mul__ ( self , other ) :
return other
def __rmul__ ( self , other ) :
return other
>>> identity = Id en t it yE l em ent ()
>>> identity
I de nt i ty E le me n t
>>> 5 + identity
5
>>> identity * 5
5

Python ,
, , .
math , , , , ..
, ,
cmath.
random , , . .
>>> random . random ()
0.37444887175646646
>>> random . uniform (1 , 10)
1.1800146073117523
4 http://docs.python.org/reference/datamodel.html#special-method-names
, .

2. PYTHON
5

>>> random . randint (1 , 10)

7
>>> random . choice ( abcdefghij )
c

>>>
>>>
>>>
[7 ,

items = [1 , 2 , 3 , 4 , 5 , 6 , 7]
random . shuffle ( items )
items
3 , 2 , 5 , 6 , 4 , 1]

>>> random . sample ([1 , 2 , 3 , 4 , 5] ,

[4 , 1 , 5]

fraction .

>>> from fractions import Fraction

>>> x = Fraction (16 , -10)
>>> x
Fraction ( -8 , 5)
>>> y = Fraction ( 3/7 )
>>> y
Fraction (3 , 7)
>>> z = Fraction (2.25)
>>> z
Fraction (9 , 4)
>>> x + y + z
Fraction (151 , 140)
>>> from decimal import Decimal
>>> Fraction ( Decimal ( 1.1 ) )
Fraction (11 , 10)

decimal (decimal
floating point numbers).
:
.
, 1.2
, .
>>> 1.1 + 2.2
3.3000000000000003

. 0.1 +
0.1 + 0.1 0.3 0, :
>>> 0.1 + 0.1 + 0.1 - 0.3
5 . 5 5 1 1 1 5 1 2 3 1 2 5 7 8 3 e -17

(precision).
decimal:

>>> from decimal import *

>>> D = Decimal
>>> D ( 1.1 ) + D ( 2.2 )
Decimal ( 3.3 )
>>> D ( 0.1 ) + D ( 0.1 ) + D ( 0.1 ) - D ( 0.3 )
Decimal ( 0.0 )

2. PYTHON

>>> D (2) ** D ( 0.5 )

Decimal ( 1 . 4 1 4 2 1 3 5 6 2 3 7 3 0 9 5 0 4 8 8 0 1 6 8 8 7 2 4 )
>>> D (1) . exp ()
Decimal ( 2 . 7 1 8 2 8 1 8 2 8 4 5 9 0 4 5 2 3 5 3 6 0 2 8 7 4 7 1 )
>>> getcontext () . prec = 6
>>> D ( 3.1415926535 ) + D ( 2.7182818285 )
Decimal ( 5.85987 )
>>> getcontext () . rounding = ROUND_UP
>>> D ( 3.1415926535 ) + D ( 2.7182818285 )
Decimal ( 5.85988 )

2.1.4

Python : str (), unicode ( ),

(), tuple (), bytearray, buffer xrange.5
: foo, "foo
". ,
: [1, 2, 3]. , ;
: (1, 2, 3) 1, 2, 3
.
: (1,).
Python bytearray, buffer
xrange. bytearray(), buffer() xrange() .
xrange ,
. xrange , (len()); (slicing), (concatenation) (repetition);
in not in min() max() .
xrange :
list

>>> for i in xrange (1 , 10 , 2) :

...
print i ,
...
1 3 5 7 9

(mutable) (immutable). :
x in s. , s x;
. , x s.
>>> 2 in [1 , 2 , 3]
True
>>> boo in boozers
True
x not in s.

, . , x

s.
>>> 4 not in [1 , 2 , 3]
True
s + t.

s t.

5 , , .

2. PYTHON

>>> (1 , 2 , 3) + (4 , 5)
(1 , 2 , 3 , 4 , 5)
>>> foo + bar
foobar
s * n n * s. n . n <= 0,
. , shallow 6
.

>>>
[]
>>>
[]
>>>
[1 ,

[1 , 2 , 3] * -1
[1 , 2 , 3] * 0
[1 , 2 , 3] * 2
2 , 3 , 1 , 2 , 3]

>>> lists = [[]] * 3

>>> lists
[[] , [] , []]
>>> lists [0]. append (3)
>>> lists
[[3] , [3] , [3]]

s[i]. i- s. 0.
, . -1.

>>> s = list ( xrange (1 , 10 , 3) )

>>> s
1, 4, 7
>>> s [2]
7
>>> s [ -2]
4

s[i:j]. , ,
i j. . i,
; j, .
s[i:j:k]. , k.

>>>
>>>
[3 ,
>>>
[3 ,
>>>
[3 ,
>>>
[0 ,

len(s).

s = [0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9]
s [3: -1]
4 , 5 , 6 , 7 , 8]
s [3:]
4 , 5 , 6 , 7 , 8 , 9]
s [3: -1:2]
5 , 7]
s [:]
1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9]

>>> len ( xrange (10) )

10
min(s)

max(s). -

s.
6 shallow deep. ,
, .

2. PYTHON

>>> s = [4 , 1 , 9 , 5 , 6]
>>> min ( s )
1
>>> max ( s )
9

s.index(x[, start[, end]]). , x,

s. start end .
>>> s = [9 , 4 , 1 , 8]
>>> s . index (1)
2
s.count(x). s, x.
start end .
>>> s = [1 , 2 , 1 , 2 , 3]
>>> s . count (1)
2

, Python bytearray.
s[i] = x. i- x.
>>>
>>>
>>>
[1 ,

s = [1 , 2 , 3]
s [1] = 4
s
4 , 3]

s[i:j] = t.
>>>
>>>
>>>
[2 ,

s = [1 , 2 , 7 , 8]
s [:2] = [2 , 3]
s
3 , 7 , 8]

>>> s [2:2] = [4 , 5 , 6]
[2 , 3 , 4 , 5 , 6 , 7 , 8]

>>> s [1:6] = []
[2 , 8]

s[i:j:k] = t. , ,
.
>>>
>>>
>>>
[1 ,

s = [1 , 1 , 3 , 3]
s [1::2] = [2 , 4]
s
2 , 3 , 4]

del s[i:j]
>>>
>>>
>>>
[1 ,

del s[i:j:k]. s.

s = [1 , 2 , 3 , 4 , 5]
del s [ -1]
s
2 , 3 , 4]

>>> del [::2]

>>> s
[2 , 4]

2. PYTHON
s.append(x).
>>>
>>>
>>>
[1 ,

x s.

s = [1 , 2]
s . append (3)
s
2 , 3]

s.extend(t).

s.
>>>
>>>
>>>
[1 ,

s = [1 , 2]
s . extend ([3 , 4])
s
2 , 3 , 4]

s.insert(i, x).
>>>
>>>
>>>
[0 ,

x i.

s = [1 , 2]
s . insert (0 , 0)
s
1 , 2]

s.pop([i]). i .
, .

>>>
>>>
4
>>>
[1 ,
>>>
2
>>>
[1 ,

s = [1 , 2 , 3 , 4]
s . pop ()
s
2 , 3]
s . pop (1)
s
3]

s.remove(x).

x.
>>>
>>>
>>>
[1 ,

s = [1 , 3 , 2 , 3 , 4]
s . remove (3)
s
2 , 3 , 4]

s.reverse().
>>>
>>>
>>>
[3 ,

in-place.

s = [1 , 2 , 3]
s . reverse ()
s
2 , 1]

s.sort([cmp[, key[, reverse]]]). in-place. . cmp , .

, , , . key
,
. reverse .
True, .
>>> s = [ C , d , E , A , b , f ]
>>> s . sort ()
>>> s
[ A , C , E , b , d , f ]

2. PYTHON

>>> s = [ C , d , E , A , b , f ]
>>> s . sort ( key = lambda x : x . lower () )
>>> s
[ A , b , C , d , E , f ]
10

>>> s = [ C , d , E , A , b , f ]
>>> s . sort ( key = lambda x : x . lower () , reverse = True )
>>> s
[ f , E , d , C , b , A ]

.
. :
string "string".
, .
string """string""". .
.
>>> This
... makes
... sense .
This \ nmakes \ nsense .

:
r R.
7 . , , .
>>> print one \ ntwo
one two
>>> print r one \ ntwo
one \ ntwo

u U. 8 .
Python % format(*args, **kwargs)9 .
, . . PEP 3101 10 . ,
:
>>> My name is {0[ name ]} . format ( dict ( name = Fred ) )
My name is Fred

>>> %( language ) s has %( number ) 03 d quote types . % \

...
{ " language " : " Python " , " number " : 2 }
Python has 002 quote types .

7 http://docs.python.org/reference/lexical_
analysis.html#string-literals
8 Python 3.x ; unicode ,
str , unicode 2.x.
9 3.x format,
.
10 http://www.python.org/dev/peps/pep-3101/

2. PYTHON

>>> from datetime import datetime

>>> Today is : {0:% a % b % d % H :% M :% S % Y } . format ( datetime . now () )
Today is : Fri Nov 25 18:11:23 2011

Python UTF-8 . encode decode .

s.encode([encoding[, errors]]). s, encoding.
errros strict ( UnicodeEncodeError
), ignore, replace, ,
.
s.decode([encoding[, errors]]). s, encoding. , encode, ,
UnicodeDecodeError.
:
s.startswith(prefix[, start[, end]]). , s prefix.
start end . prefix
.
>>> substring . startswith ( sub )
True
s.endswith(suffix[, start[, end]]). , s suffix.
, startswith.
s.strip([chars]). s , chars.
>>> centered . strip ()
centered
s.lower()

s.upper(). -

.
>>> FOObar . lower ()
foobar
s.split([sep[, maxsplit]]). sep . sep , , ,
.
maxsplit.

>>> 1 ,2 ,3 . split ( , )
[ 1 , 2 , 3 ]
>>> 1 , ,3 . split ( , )
[ 1 , , 3 ]
>>> 1 2 3 . split ()
[ 1 , 2 , 3 ]
>>> 1 2 3 . split ( None , 1)
[ 1 , 2 3 ]

s.join(iterable).

, iterable,

s.
>>> , . join ( str ( i ) for i in xrange (10) )
0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9

2. PYTHON

Python . . ;
. ,
(1), , (n).

. , , . :

>>> tuple ([1 , 2 , 3])

(1 , 2 , 3)
>>> ( a , b , c )
( a , b , c )

(packing) , (unpacking):
1

>>> t = 1 , 2 , 3
>>> a , b , c = t
>>> b
2

2.1.5

. . :
:
1

>>> d = { one : 1 , two : 2 }

>>> d
{ one : 1 , two : 2}

:
2

>>> d = dict ( one =1 , two =2)

>>> d
{ one : 1 , two : 2}

, 2, , .
2

>>> d = dict ([( one , 1) , ( two , 1) ])

>>> d
{ one : 1 , two : 2}

(dict comprehension)11 .
2

>>> d = { k : v for k , v in enumerate ( ABCD ) if v not in CB }

>>> d
{0: A , 3: D }

Python 2.7. , .

2. PYTHON

, ,
, (,
). ,
, . :
2

>>> d = dict ()
>>> d [ list () ] = 1
Traceback ( most recent call last ) :
File " < stdin > " , line 1 , in < module >
TypeError : unhashable type : list
>>> class dummy ( object ) :
...
def __init__ ( self ) :
...
self . _list = list ()
...
>>> d [ dummy () ] = 1
>>> d
{ < __main__ . dummy object at 0 x7f922fb3a410 >: 1}

:
d[key]. , key. KeyError.
>>> d = dict ( one =1)
>>> d [ one ]
1
d[key] = value.

value key.

>>> d = dict ()
>>> d [ fruits ] = ( banana , apple , pear )
>>> d
{ fruits : ( banana , apple , pear ) }
del d[key].

, key.

>>> d = dict ( one =1 , two =2)

>>> del d [ two ]
>>> d
{ one : 1}
key in d

key not in d. key

>>> d = dict ( one =1 , two =2)

>>> one in d
True
>>> three in d
False

d.clear().

>>> d = dict ( one =1 , two =2)

>>> d . clear ()
>>> d
{}
d.items() d.iteritems(). items() (key, value).
iteritems() .
>>> d = dict ( one =1 , two =2)
>>> d . items ()
[( two , 2) , ( one , 1) ]

2. PYTHON
5

>>> d . iteritems ()
< dictionary - itemiterator object at 0 x7f922fb88f18 >
>>> for k , v in d . iteritems () :
...
print k , : , v
...
two : 2
one : 1

d.keys() d.iterkeys(). keys() . iterkeys()

.
>>> d = dict ( one =1 , two =2)

()

>>> d . keys ()
[ two , one ]
>>> d . iterkeys ()
< dictionary - keyiterator object at 0 x7f922fb88f18 >
>>> for k in d . iterkeys () :
...
print k ,
...
two one

d.values() d.itervalues(). values() . itervalues

.
>>> d = dict ( one =1 , two =2)

>>> d . values ()
[2 , 1]
>>> d . itervalues ()
< dictionary - valueiterator object at 0 x7f922fb88f18 >
>>> for v in d . itervalues () :
...
print v ,
...
2 1

d.update(other).

other.

>>> d = dict ( one =1 , two =2)

>>> d . update ({ three : 3 })
{ one : 1 , three : 3 , two : 2}
d.get(key[, default]). key, ,
default, , KeyError, .

>>> d = dict ( one =1)

>>> d . get ( one , None )
1
>>> d . get ( two , None )
None

d.setdefault(key[, default]). key,

default. default , None.
>>> d = dict ( furniture =[ chair ])
>>> d
{ furniture : [ chair ]}
5

>>> d . setdefault ( furniture , list () ) . append ( table )

>>> d
{ furniture : [ chair , table ]}

2. PYTHON

2.1.6

(set) Python ,
. .
: set frozenset. set , frozenset .
:
set() frozenset().
>>> basket = [ apple , orange , apple , pear , orange , banana ]
>>> fruits = set ( basket )
>>> fruits
set ([ apple , banana , orange , pear ])

:
1

>>> { apple , orange , apple , pear , orange , banana }

set ([ apple , banana , orange , pear ])

(set comprehension)12 .

>>> s = { v for v in ABCDABCD if v not in CB }

>>> s
set ([ A , D ])

: , , . . , ,
:
len(s). s.
>>> s = set ([1 , 2 , 3])
>>> len ( s )
3
x in s

x not in s. , x -

>>> s = set ([1 , 2 , 3])

>>> 2 in s
True
>>> 0 in s
False

s.isdisjoint(other). , s
other, s other .

>>> s1 , s2 , s3 = set ([1 , 2 , 3]) , set ([4 , 5 , 6]) , set ([3 , 4])
>>> s1 . isdisjoint ( s2 )
True
>>> s1 . isdisjoint ( s3 )
False

s.issubset(other)

s <= other. , s other.

>>> s = set ([1 , 2 , 3])

>>> s . issubset ( s )
True
s < other.
12

, s other, s != other.
Python 2.7. , .

2. PYTHON

>>> s = set ([1 , 2 , 3])

>>> s < s
False
s.issuperset(other)

s >= other. , other s.

>>> s = set ([1 , 2 , 3])

>>> s . issuperset ( s )
True
s > other.

, other s, other != s.

>>> s = set ([1 , 2 , 3])

>>> s > s
False
s.union(other)

s | other. , -

.
>>> s1 , s2 = set ([1 , 2]) , set ([2 , 3])
>>> s1 . union ( s2 )
set ([1 , 2 , 3])
s.intersection(other)

s & other. .

>>> s1 , s2 = set ([1 , 2]) , set ([2 , 3])

>>> s1 . intersection ( s2 )
set ([2])
s.difference(other)

s - other. .

>>> s1 , s2 = set ([1 , 2]) , set ([2 , 3])

>>> s1 . difference ( s2 )
set ([1])
s.symmetric_difference(other)

s ^ other. .

>>> s1 , s2 = set ([1 , 2]) , set ([2 , 3])

>>> s1 . s y m m e t r i c _ d i f f e r e n c e ( s2 )
set ([1 , 3])

:
s.add(x). x s.
>>> s = set ([1 , 2])
>>> s . add (3)
>>> s
set ([1 , 2 , 3])
s.remove(x).
KeyError.

x . ,

>>> s = set ([1 , 2])

>>> s . remove (1)
>>> s
set ([2])
s.discard(x).

x , .

>>> s = set ([1 , 2])

>>> s . discard (1)
>>> s . discard (1)
>>> s
set ([2])

s.clear().

2. PYTHON

>>> s = set ([1 , 2])

>>> s . clear ()
>>> s
set ([])

2.2

if
if 13 :
if expression: suite
( elif expression: suite) *
[ else : suite ]

expression ,
True. suite. , else ,
.
, ,
, - .
import os . path

def whatsit ( path ) :

if os . path . isdir ( path ) :
print directory
elif os . path . isfile ( path ) :
print file
else :
print something else

:
1

>>> whatsit ( / tmp )

directory
>>> whatsit ( / boot / grub / grub . conf )
file

while
while :
while expression: suite1
[ else : suite2 ]

while suite1 , expression . expression , else, .

>>> expression = [3 , 2 , 1]
>>> while expression :
...
print suite -1: , expression . pop ()
... else :
13 ,
: (x, y, ...) , ; * ,
; + ;
[x, y] , , ; x | y
x, y.

2. PYTHON
5

...
print suite -2
...
suite -1: 1
suite -1: 2
suite -1: 3
suite -2

break else.

>>> while True :

...
print suite -1
...
break
... else :
...
print suite -2
...
suite -1

continue .
for
for :
for variable in expression: suite1
[ else : suite2 ]

expression . . suite1 , ,
variable .
else.

>>> for i in xrange (3) :

...
print suite -1: , i
... else :
...
print suite -2
...
suite -1: 0
suite -1: 1
suite -1: 2
suite -2

break else.

>>> for i in xrange (3) :

...
print suite -1: , i
...
if i > 1:
...
break
... else :
...
print suite -2
...
suite -1: 0
suite -1: 1

continue ,
, else.
try
try :

2. PYTHON

try : suite1
( except [ expression [ ( as | , ) target ] ]: suite2 ) +
[ else : suite3 ]
[ finally : suite4 ]

try : suite1
finally : suite4

except .
suite1 , .

>>> try :
...
print Hello .
... except Ar i th me t ic E rr or :
...
print Arithmetic error occured .
... except :
...
print Error occured .
...
Hello .

except. expression , suite2 , expression

, -, ,
.

>>> try :
...
1 / 0
... except Z e r o D i v i s i o n E r r o r :
...
print Illegal operation performed .
...
Illegal operation performed .

, ArithmeticError, ZeroDivisionError.

>>> try :
...
1 / 0
... except Ar i th me t ic E rr or :
...
print Here we catch all arithmetic errors .
...
Here we catch all arithmetic errors .

except expression, , . ,
.

>>> try :
...
x
... except Ar i th me t ic E rr or :
...
print Here we catch all arithmetic errors .
... except :
...
print Here we are catching even NameError .
...
Here we are catching even NameError .

,
, finally . ,
, . , try
NameError, finally Exception.

2. PYTHON

>>> try :
...
x
... finally :
...
raise Exception
...
Traceback ( most recent call last ) :
File " < stdin > " , line 4 , in < module >
Exception

expression , ,
.
except , target, ,
expression, . ,
, try.

>>> try :
...
1 / 0
... except Exception , e :
...
print e
...
integer division or modulo by zero

else , try. , else, .

>>>
...
...
...
...
...
...
2
Sum

try :
1 + 1
except :
print Error occured .
else :
print Sum done well .

done well .

finally , : try, except else.

>>> try :
...
raise Exception
... finally :
...
print in finally
...
in finally
Traceback ( most recent call last ) :
File " < stdin > " , line 2 , in < module >
Exception

sys.exc_info(), (
exc_type, exc_value, exc_traceback), exc_type , exc_value ,
, exc_traceback traceback .
traceback.

Python Python:
import sys
import traceback

def run_user_code ( envdir ) :

source = raw_input ( " >>> " )
try :

2. PYTHON

exec source in envdir

except :
print " Exception in user code : "
print - * 60
traceback . print_exc ( file = sys . stdout )
print - * 60
envdir = {}
while True :
run_user_code ( envdir )

>>> 1 / 0
Exception in user code :
-----------------------------------------------------------Traceback ( most recent call last ) :
File " test . py " , line 7 , in run_user_code
exec source in envdir
File " < string > " , line 1 , in < module >
Z e r o D i v i s i o n E r r o r : integer division or modulo by zero
----------------------------------------------------------->>> Traceback ( most recent call last ) :
File " test . py " , line 16 , in < module >
run_user_code ( envdir )
File " test . py " , line 5 , in run_user_code
source = raw_input ( " >>> " )
EOFError

with
with :
with expression [ as target ]: suite

:
1. expression .
2. __enter__() expression.
3. as target, __enter__ target.
4. suite.
5. __exit__(). suite
__exit__() False, .
import threading
lock = threading . Lock ()
with lock : ...

2.3

2. PYTHON
1

def add (a , b ) :
return a + b

, def,
, , . return. , , None.
(
) .
, .

>>> type ( add )

< type function >
>>> add . __globals__
{ __builtins__ : < module __builtin__ ( built - in ) >,
__name__ : __main__ , add : < function add at 0 x7f4f81af8578 > ,
__doc__ : None , __package__ : None }

14 . ,
.
, .
,
.
.

def yes_or_no ( prompt , retries =4 , complaint = Please enter Yes or No . ) :

while True :
answer = raw_input ( prompt )
answer = answer . lower ()
if answer in ( y , ye , yes ) :
return True
if answer in ( n , no , nop , nope ) :
return False

retries -= 1
if retries < 0:
raise IOError ( User has not approved or declined an action . )
print complaint

:
1

>>> yes_or_no ( Delete the file : , retries =3)

...
>>> yes_or_no ( Will you marry me ? ,
complaint = " That s fine , but one more time ... " )
...

.
. (mutable) ,
, :
def append ( list =[]) :
list . append ( a )
return list
14

. 58.

2. PYTHON
5

>>> append ()
[ a ]
>>> append ()
[ a , a ]
>>> append ()
[ a , a , a ]

, :

def append ( list = None ) :

if list is None :
list = []
list . append ( a )
return list
>>> append ()
[ a ]
>>> append ()
[ a ]

*identifier, identifier
, , .
**identifier, identifier ,
.
def fargs (* args ) :
print args

def fkwargs (** kwargs ) :

print kwargs
>>> fargs (1 , 2 , 3)
(1 , 2 , 3)
>>> fkwargs ( one =1 , two =2 , three =3)
{ three : 3 , two : 2 , one : 1}

,
, TypeError. *
identifier, identifier ,
.
**identifier, identifier , ,
.
def fargs (a , b , c ) :
print a , b , c

def fkwargs (a , b =2 , c =3) :

print a , b , c
>>> args = [1 , 2 , 3]
>>> fargs (* args )
1 2 3

>>> kwargs = { a : 2 , b : 3 , c : 4 }
>>> fkwargs (** kwargs )
2 3 4

,
.

2. PYTHON

(docstring), . __doc__.
def func () :
""" Here goes docstring . """
pass
5

>>> func . doc

Here goes docstring .

() Python ,
. -:
lambda [ parameters ]: expression

-
(statements).
>>> map ( lambda i : i + 1 , [1 , 2 , 3])
[2 , 3 , 4]

2.4

Python .

2.4.1

,
. , .
def compose (g , f ) :
return lambda x : g ( f ( x ) )
3

def f ( x ) :
return x + 3

def g ( x ) :
return 2 * x
>>> z = compose (g , f )
>>> z (2)
10

, Python.
map
map ,
.
.

2. PYTHON

, None. . None,
, ;
, .
map ( function , sequence [ , sequence , ...]) -> list

>>> map ( lambda x , y : x + y , [1 , 2 , 3] , [2 , 3 , 4])

[3 , 5 , 7]
>>> map ( None , [1 , 2] , [3])
[(1 , 3) , (2 , None ) ]

filter
filter , , . ,
, . None, , .
filter ( function or None , sequence ) -> list , tuple , or string
>>> filter ( lambda x : x % 2 , [1 , 2 , 3 , 4 , 5])
[1 , 3 , 5]

reduce
reduce , , , , . : function, initial sequence.
reduce ( function , sequence [ , initial ]) -> value

4! , reduce:
>>> reduce ( lambda res , x : res * x , [2 , 3 , 4] , 1)
24

1
:
(((1 * 2) * 3) * 4)

2. PYTHON

2.4.2

Python ,
CPython , (tail
call optimization, TCO).
, , , . , ,
.
, TCO .
def infinite () :
infinite ()
4

>>> infinite ()
Traceback ( most recent call last ) :
File " < stdin > " , line 1 , in < module >
...
RuntimeError : maximum recursion depth exceeded

sys.getrecursionlimit() sys.setrecursionlimit(limit).

2.4.3

,
.
, ,
.
def make_adder ( y ) :
def add ( x ) :
return x + y
return add
5

>>> add5 = make_adder (5)

>>> add5 (5)
10
>>> add5 (7)
12

add make_adder. add y .

Python , 3.x
. 3.0 nonlocal, :

def make_counter () :
count = 0
def get () :
return count
def inc () :
count += 1
return get , inc

>>> get , inc = make_counter ()

>>> get ()
>>> 0

2. PYTHON

>>> inc ()
Traceback ( most recent call last ) :
File " < stdin > " , line 1 , in < module >
File " / tmp / py9637pml " , line 11 , in inc
U n b o u n d L o c a l E r r o r : local variable count referenced before assignment
def make_counter () :
count = 0
def get () :
nonlocal count
return count
def inc () :
count += 1
return get , inc

>>>
>>>
>>>
>>>
>>>
1

2.4.4

get , inc = make_counter ()

get ()
0
inc ()
get ()

n
nm (m < n), m
. .
Python:
1

from functools import partial

basetwo = partial ( int , base =2)
>>> basetwo ( 10010 )
18

2.4.5

(list comprehension) . , map filter. :

>>> [ i for i in range (10) ]
[0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9]

>>> [[ i
[[0 , 0 ,
[0 , 1 ,
[0 , 2 ,
[0 , 3 ,
[0 , 4 ,

* j for j in range (5) ] for i in range (5) ]

0 , 0 , 0] ,
2 , 3 , 4] ,
4 , 6 , 8] ,
6 , 9 , 12] ,
8 , 12 , 16]]

>>> [ i + j for i in range (3) for j in range (3) ]

[0 , 1 , 2 , 1 , 2 , 3 , 2 , 3 , 4]

>>> [ i + j for i in range (3) if i % 2 == 0 for j in range (3) if j % 2 == 1]

[1 , 3]

2. PYTHON

2.5

, , , ,
. .
import. import :
import module

from module import ( something [ as someothername ] ) +

(top-level)
, .
. dummy,
: .
Listing 2.2: dummy.py
print in dummy module

:
>>> import dummy
in dummy module
>>> import dummy

, .
, .
2

>>> import sys

>>> sys
< module sys ( built - in ) >
>>> sys . argv
[ / usr / bin / ipython ]

, .

>>> from sys import argv , modules

>>> argv
[ / usr / bin / ipython ]
>>> modules . popitem ()
( IPython . Prompts ,
< module IPython . Prompts from / usr / lib64 / python2 .7/ site - packages / IPython /
Prompts . pyc >)

, :
1. , .
2. ,
import.
, .
, . , sys.path.

2. PYTHON

>>> import sys

>>> sys . path . insert (0 , / var / www / site )
>>> sys . path [:3]
[ / var / www / site , , / usr / bin ]

,
(public) .
__all__ .

ImportError.

>>> import threading

>>> threading . _sleep
< built - in function sleep >
5

>>> from threading import *

>>> _sleep
Traceback ( most recent call last ) :
File " < stdin > " , line 1 , in < module >
NameError : name _sleep is not defined

>>> _sleep in threading . all

False

( ,
func_global ). m.x m.__dict__[x].
:
__name__ ;
__doc__ None, ;
__file__ , , .

>>> import threading

>>> threading . __name__
threading
>>> threading . __doc__ [:50]
" Thread module emulating a subset of Java s threadi "
>>> threading . __file__
/ usr / lib64 / python2 .7/ threading . pyc

.
, . ,
. ,
, , __init__.py.
Django:
django /

__init__ . py
db /
__init__ . py
backends /
models /
transaction . py
utils . py
forms /
__init__ . py
extras /
fields . py

2. PYTHON

forms . py
...
15

http /
__init__ . py
m ul ti p ar t pars e r . py
utils . py
template /
__init__ . py
loaders /
base . py
context . py
debug . py
...

...

import Django:
from django . db import models
from django . http import Http404 , HttpResponse
from django . template import Context
5

class Poll ( models . Model ) :

question = models . CharField ( max_length =200)
pub_date = models . DateTimeField ( date published )

. ,
, . ,
. . , from . import mod
pkg, pkg.mod. from ..subpkg2 import mod
pkg.subpkg1, pkg.subpkg2.mod.
from mod import * , _.
, __import__.

2.6

, . , __call__(self, f). , -
. , ,
:

def debug ( func ) :

def wrapped (* args , ** kwargs ) :
print " % s called with : %s , % s " % ( func . __name__ , args , kwargs )
ret = func (* args , ** kwargs )
print " % s returned : % s " % ( func . __name__ , ret )
return ret
return wrapped

:
def inc ( i ) :
return i + 1
3

inc = debug ( inc )

inc :
>>> inc (1)
inc called with : (1 ,) , {}
inc returned : 2

2. PYTHON

Python @- . :
@debug
def inc ( i ) :
return i + 1

2.6.1

(. currying)
, . ,

:
2

class curried ( object ) :

def __init__ ( self , func , * a ) :
self . func = func
self . args = a
def __call__ ( self , * a ) :
args = self . args + a
if len ( args ) < self . func . __code__ . co_argcount :
return curried ( self . func , * args )
else :
return self . func (* args )

:
@curried
def add (a , b ) :
return a + b

,
:
2

>>> add1 = add (1)

>>> add1 (2)
3

, . , . ,
(lock)
.
2

def synchronized ( lock ) :

def wrap ( f ) :
def sync (* args , ** kwargs ) :
with lock :
return f (* args , ** kwargs )
return sync
return wrap

,
synchronized :

2. PYTHON
import
import
import
lock =

time
threading
sys
threading . Lock ()

@synchronized ( lock )
def msg ( what , then ) :
print what ,
sys . stdout . flush ()
time . sleep (0.5)
print then ,
sys . stdout . flush ()

,
:
import thread
3

def start_threads () :
thread . s t a r t _ n e w _ t h r e a d ( msg , ( A , B ) )
thread . s t a r t _ n e w _ t h r e a d ( msg , ( C , D ) )
thread . s t a r t _ n e w _ t h r e a d ( msg , ( E , F ) )

>>>
E F
>>>
C D

start_threads ()
A B C D
start_threads ()
A B E F

, : A B . .
,
: , __name__
. ,
, .

def well_behaved ( decorator ) :

def new (f , * args , ** kwargs ) :
g = decorator (f , * args , ** kwargs )
g . __name__ = f . __name__
g . __doc__ = f . __doc__
g . __dict__ . update ( f . __dict__ )
return g
new . __name__ = decorator . __name__
new . __doc__ = decorator . __doc__
new . __dict__ . update ( decorator . __dict__ )
return new

debug:
@debug
def dummy () :
" Dummy s docstring "
pass
5

>>> dummy . name

wrapped
>>> dummy . __doc__

, dummy .
debug :

well_behaved

2. PYTHON

debug = well_behaved ( debug )

@debug
def dummy () :
" Dummy s docstring "
pass
>>> dummy . __name__
dummy
>>> dummy . __doc__
" Dummy s docstring "

! !

: , . :
.

@well_behaved
def accepts (* types ) :
def check_types ( f ) :
def wrapped (* args , ** kwargs ) :
for (a , t ) in zip ( args , types ) :
assert isinstance (a , t ) , \
" arg % r does not match % s " % (a , t )
return f (* args , ** kwargs )
return wrapped
return check_types
@well_behaved
def returns ( rtype ) :
def check_types ( f ) :
def wrapped (* args , ** kwargs ) :
result = f (* args , ** kwargs )
assert isinstance ( result , rtype ) , \
" return value % r does not match % s " % ( result , rtype )
return result
return wrapped
return check_types

,
, :

@accepts ( int , int )

@returns ( int )
def add (a , b ) :
return a + b
>>> add (2 , 2)
4
>>> add (2 , 2.0)
Traceback ( most recent call last ) :
File " < stdin > " , line 1 , in < module >
File " / tmp / py10971mml " , line 7 , in wrapped
Ass ertion Error : arg 2.0 does not match < type int >

2. PYTHON

2.7

-
I propose a new rule for discussions
of object-oriented programming: anyone
who brings up examples of Dog, Bike, Car,
Person, or other real-world objects, unless
they are talking about writing a clone of
The Sims or something, is immediately
shot.
I feel that this new policy will improve
the quality of the discourse enormously.
Once upon a time somewhere on the
Internet
Kragen Javier Sitaker

Python , . type. Python . , +

__add__ __radd__.
:
class klass ( object ) :
pass

, klass object. ,
, .
, .
:
def get_object () :
return object

class klass ( get_object () ) :

pass

Python ,
. :
class storage ( object ) :
i = 1
def __init__ ( self ) :
self . i = 3

>>>
>>>
>>>
>>>
2
>>>
4

s = storage ()
storage . i += 1
s . i += 1
storage . i
storage . i

__init__(self) storage.
self.i = 3 , 3.
-- :
class klass ( object ) :
def __init__ ( self ) :

2. PYTHON

self . __private = 1
5

>>> c = klass ()
>>> c . __dict__
{ _ kl as s __ pr i va t e : 1}

, name mangling: __*

_klass__*.
. ,
.

klass

class klass ( object ) :

def m ( self ) :
print self
5

>>> c = klass ()
>>> c . m ()
< __main__ . klass object at 0 x1188f10 >

, m
:
class klass ( object ) :
pass

def m ( self ) :
print self
>>> c = klass ()
>>> klass . m = m
>>> c . m ()
< __main__ . klass object at 0 x1225990 >

Python :
(static) (class methods). :
class klass ( object ) :
@classmethod
def clsmethod ( cls ) :
print class : , cls
5

>>> c = klass ()
>>> klass . clsmethod ()
class : < class __main__ . klass >
>>> c . clsmethod ()
class : < class __main__ . klass >

, ,
. :
class klass ( object ) :
@staticmethod
def staticmethod () :
print static
5

>>> c = klass ()
>>> klass . staticmethod ()
static
>>> c . staticmethod ()
static

, .

2. PYTHON

! init(). , new(cls[, ...]),

.
class complex ( object ) :
def __init__ ( self , real , imag ) :
self . real = real
self . imag = imag
5

>>> c = complex (5.1 , 3.0)

>>> c . real , c . imag
(5.1 , 3.0)

2.7.1

property,
:
property ( fget = None , fset = None , fdel = None , doc = None )

fget ; fset ; fdel del; doc ,

.
class klass ( object ) :
def __init__ ( self ) :
self . _x = 0
5

def getx ( self ) :

return self . _x
def setx ( self , val ) :
self . _x = val

def delx ( self ) :

del self . _x
x = property ( getx , setx , delx , " I m the x property . " )
15

>>> c = klass ()
>>> c . x = 3
>>> c . x
3
>>> klass . x . __doc__
" I m the x property . "

:
class klass ( object ) :
def __init__ ( self ) :
self . _x = 0
4

@property
def x ( self ) :
" I m the x property . "
return self . _x
9

@x . setter
def x ( self , val ) :
self . _x = val

2. PYTHON

@x . deleter
def x ( self ) :
del self . _x

doc , , . , , ,
property .

2.7.2

MRO

- super(type[, object-or-type]). super:

class C ( B ) :
def method ( self , arg ) :
super (C , self ) . method ( arg )

__mro__ type ,

super(). __mro__ , .
class A ( object ) :
def f ( self ) :
print A
5

class B ( A ) :
def f ( self ) :
print B
super (B , self ) . f ()

class C ( A ) :
def f ( self ) :
print C
super (C , self ) . f ()

class D (B , C ) :
def f ( self ) :
print D
super (D , self ) . f ()

>>> D . __mro__
( < class __main__ . D >,
< class __main__ . B >,
< class __main__ . C >,
< class __main__ . A >,
< type object >)

>>> D () . f ()
D
B
C
A

__mro__ ,
- , .
isinstance(object, class-or-typeor-tuple) issubclass(C, B).
>>> issubclass (D , D ) , issubclass (D , A ) , isinstance ( D () , A )
( True , True , True )

2. PYTHON

2.7.3

object.__getattr__(self, name). , .
AttributeError.
object.__setattr__(self, name, value). value
name.
object.__delattr__(self, name).
object.__getattribute__(self, name). . AttributeError
, object.__getattr__(self, name).

class.__instancecheck__(self, instance). True, instance .
class.__subclasscheck__(self, subclass). True, subclass .

2.8

raise, -
.

>>> raise NameError

Traceback ( most recent call last ) :
File " < stdin > " , line 1 , in < module >
NameError
>>> raise ValueError ( " Wrong value " )
Traceback ( most recent call last ) :
File " < stdin > " , line 1 , in < module >
ValueError : Wrong value

, Python, : http://docs.python.org/library/exceptions.html.
,
, ,
Exception.

class MyError ( Exception ) :

def __init__ ( self , value ) :
self . value = value
def __str__ ( self ) :
return repr ( self . value )
>>> try :
...
raise MyError (2 * 2)
... except MyError as e :
...
print My exception occurred , value : , e . value
...
My exception occurred , value : 4

2. PYTHON

>>> raise MyError ( oops ! )

Traceback ( most recent call last ) :
File " < stdin > " , line 1 , in ?
__main__ . MyError : oops !

try . 46.

2.9

, . __next__
. ,
StopIteration.
__iter__(),
__iter__() next(). __iter__() , ,
, , for.
-, limit:
()

class counter ( object ) :

def __init__ ( self , limit ) :
assert limit >= 0
self . _limit = limit
self . _value = 0
def __iter__ ( self ) :
return self
def next ( self ) :
if self . _value < self . _limit :
self . _value += 1
return self . _value - 1
else :
raise StopIteration

>>> [ i for i in counter (10) ]

[0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9]

Python itertools.
:
count(start=0, step=1). , start,
step.

>>>
>>>
>>>
1
>>>
2.5

import itertools
i = itertools . count (1 , 1.5)
i . next ()
i . next ()

cycle(iterable). ,
iterable.

2. PYTHON

>>> i =
>>> for
...
...
...
A B C D

itertools . cycle ( ABCD )

j , k in enumerate ( i ) :
print k ,
if j > 7:
break
A B C D A

repeat(object[, times]).

, times .

>>> i = itertools . repeat (1 , 5)

>>> list ( i )
[1 , 1 , 1 , 1 , 1]
chain(*iterables).

iterables.

>>> i = itertools . chain ( xrange (3) , xrange (3 , 6) , xrange (6 , 9) )

>>> list ( i )
[0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8]
dropwhile(predicate, iterable). , -
predicate . iterable.
>>> i = itertools . dropwhile ( lambda x : x < 5 , xrange (10) )
>>> list ( i )
[5 , 6 , 7 , 8 , 9]
takewhile(predicate, iterable). iterable , predicate, , .
>>> i = itertools . takewhile ( lambda x : x < 5 , xrange (10) )
>>> list ( i )
[0 , 1 , 2 , 3 , 4]
ifilter(function or None, sequence).

filter , -

.
>>> i = itertools . ifilter ( None , [1 , 0 , 2 , 0 , 3])
>>> list ( i )
[1 , 2 , 3]
imap(function, *iterables). map. : , iterables .
>>> i = itertools . imap ( lambda x , y : x + y , [1 , 2 , 3 , 4] , [2 , 3 , 4])
>>> list ( i )
[3 , 5 , 7]
izip(*iterables). , i-
i- iterables. , .
>>> i = itertools . izip ( xrange (5) , xrange (1 , 5) )
>>> list ( i )
[(0 , 1) , (1 , 2) , (2 , 3) , (3 , 4) ]

2.10

, .
, yield,
yield .

2. PYTHON

next() , yield, , , ,
, yield. next() . ,
yield ( ,
return, ).
, :

def cycle ( seq ) :

i = 0
l = len ( seq )
while True :
yield seq [ i ]
i = ( i + 1) % l
>>> g = cycle ([ John , James , Hannibal ])
< generator object cycle at 0 xb54640 >
>>> g . next () , g . next () , g . next ()
( John , James , Hannibal )
>>> g . next ()
John

- :
next(). - ,
. , yield None, .
yield, StopIteration.
send(value). value ,
value yield. , yield, StopIteration, . send(value) , value None.
throw(type[, value[, traceback]]). type , , yield :
raise type , value , traceback

, -.
, StopIteration. - , .
close(). GeneratorExit . -
StopIteration ( , ) GeneratorExit ( ), close.
yield, RuntimeError.
:
class Tree ( object ) :
def __init__ ( self , value , left = None , right = None ) :
self . value = value
self . left = left

2. PYTHON
5

self . right = right

@staticmethod
def from_list ( list ) :
n = len ( list )
if n == 0:
return None
i = n / 2
return Tree ( list [ i ] ,
Tree . from_list ( list [: i ]) ,
Tree . from_list ( list [ i + 1:]) )
@staticmethod
def inorder ( tree ) :
if tree :
for x in Tree . inorder ( tree . left ) :
yield x
yield tree . value
for x in Tree . inorder ( tree . right ) :
yield x

def iter ( self ) :

return Tree . inorder ( self )

>>> tree = Tree . from_list ( " A B C D E F G H I J K L M N O P Q R S T U V W X Y Z " )

>>> for x in tree :
...
print x ,
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

, tail -f.

def follow ( path ) :

f = file ( path )
f . seek (0 , 2)
while True :
line = f . readline ()
if not line :
continue
yield line
>>> for line in follow ( " / var / log / syslog " ) :
...
print line ,

, . , tail -f file | grep "error\|warning":

def consumer ( f ) :
def wrapper (* args , ** kwargs ) :
g = f (* args , ** kwargs )
g . next ()
return g
return wrapper
def follow ( path , target ) :
f = file ( path )
f . seek (0 , 2)
while True :
line = f . readline ()
if not line :
continue
target . send ( line )
@consumer

2. PYTHON

def grep ( string , target ) :

while True :
line = yield
if string in line :
target . send ( line )
@consumer
def echo () :
while True :
line = yield
print line ,

@consumer
def broadcast ( targets ) :
while True :
item = yield
for target in targets :
target . send ( item )

if __name__ == __main__ :
follow ( / tmp / fol ,
broadcast ([ grep ( error , echo () ) ,
grep ( warning , echo () ) ]) )

(coroutines). , , .
15 :
import collections
import sys
import types
5

class Trampoline ( object ) :

""" Manage com munica tions between coroutines . """

def init ( self ) :

self . running = False
self . queue = collections . deque ()
def add ( self , coroutine ) :
""" Request that a coroutine be executed . """
self . schedule ( coroutine )

def run ( self ) :

result = None
self . running = True
try :
while self . running and self . queue :
fn = self . queue . popleft ()
result = fn ()
return result
finally :
self . running = False
def stop ( self ) :
self . running = False

def schedule ( self , coroutine , stack =() , value = None , * exc ) :

def resume () :
PEP 342.

2. PYTHON

try :
if exc :
value = coroutine . throw ( value , * exc )
else :
value = coroutine . send ( value )
except :
if stack :
# send the error back to the " caller "
self . schedule ( stack [0] , stack [1] , * sys . exc_info () )
else :
raise

if isinstance ( value , types . GeneratorType ) :

self . schedule ( value , ( coroutine , stack ) )
elif stack :
self . schedule ( stack [0] , stack [1] , value )

self . queue . append ( resume )

2.10.1

, .
. , ;
, - ,
yield. , , ,
.
>>> sum ( i * i for i in range (10) )
285

2.11

, . owner,
-.
d.__get__(self, instance, owner). owner , d. instance , , None, owner. AttributeError.
d.__set__(self, instance, value). value
instance.
d.__delete__(self, instance). , instance.
x a : a.__dict__[x], type(a).__dict__[x],
, . , AttributeError.

2. PYTHON

x a , a.x
type(a).__dict__[x].__get__(a, type(a)), a , a.__dict__[x].__get__(None,
a), a .
:
class RevealAccess ( object ) :
def __init__ ( self , initval = None , name = var ) :
self . val = initval
self . name = name
5

def get ( self , obj , objtype ) :

print Retrieving , self . name
return self . val
10

def set ( self , obj , val ) :

print Updating , self . name
self . val = val
>>> class MyClass ( object ) :
x = RevealAccess (10 , x )
y = 5
>>> m = MyClass ()
>>> m . x
Retrieving x
10
>>> m . x = 20
Updating var " x "
>>> m . x
Retrieving var " x "
20
>>> m . y
5

, x
.
Python . :

class Property ( object ) :

def __init__ ( self , fget = None , fset = None , fdel = None , doc = None ) :
self . fget = fget
self . fset = fset
self . fdel = fdel
if fget is not None :
self . __doc__ = fget . __doc__
else :
self . __doc__ = doc
def __get__ ( self , obj , klass = None ) :
if obj is None :
return self
if self . fget is None :
raise AttributeError , " unreadble attribute "
return self . fget ( obj )
def __set__ ( self , obj , value ) :
if self . fset is None :
raise AttributeError , " unsetable attribute "
self . fset ( obj , value )

2. PYTHON

def delete ( self , obj ) :

if self . fdel is None :
raise AttributeError , " undeletable attribute "
self . fdel ( obj )

staticmethod

classmethod :

class StaticMethod ( object ) :

def __init__ ( self , fn ) :
self . fn = fn
5

def get ( self , obj , klass = None ) :

return self . fn
class ClassMethod ( object ) :
def __init__ ( self , fn ) :
self . fn = fn
def __get__ ( self , obj , klass = None ) :
if klass is None :
klass = type ( obj )
def m (* args ) :
return self . fn ( klass , * args )
return m

:
class WellTyped ( object ) :
of_type = object
3

def init ( self , default ) :

self . _check ( default )
self . _default = default
self . _value
= dict ()
8

def get ( self , obj , klass = None ) :

if obj is None :
return self
if obj in self . _value :
return self . _value [ obj ]
else :
return self . _default
def __set__ ( self , obj , val ) :
self . _check ( val )
self . _value [ obj ] = val
def _check ( self , val ) :
if not isinstance ( val , self . of_type ) :
raise ValueError
class Int ( WellTyped ) :
of_type = int

class Str ( WellTyped ) :

of_type = str
class PositiveInt ( Int ) :
def _check ( self , val ) :
super ( PositiveInt , self ) . _check ( val )
if val < 0:
raise ValueError

2. PYTHON

, , :

>>> t = Test ()
>>> t . i = 1.0
Traceback ( most recent call last ) :
File " < stdin > " , line 1 , in < module >
File " / tmp / py6835YLA " , line 19 , in __set__
File " / tmp / py6835YLA " , line 24 , in _check
ValueError
>>> t . i = 2
>>> t . i
2
>>> t . pi = -1
Traceback ( most recent call last ) :
File " < stdin > " , line 1 , in < module >
File " / tmp / py6835YLA " , line 19 , in __set__
File " / tmp / py6835YLA " , line 36 , in _check
ValueError

2.12

Python , .
.
>>> class klass ( object ) :
...
pass
4

>>> c = klass ()
>>> c
< __main__ . klass object at 0 xc14a90 >
>>> klass
< class __main__ . klass >
>>> id ( klass )
12861152

id , , ,
, .
,
. Python type.
>>> object . __class__
< type type >
>>> type ( object )
< type type >

type:

>>> int . class

< type type >
>>> str . __class__
< type type >
>>> tuple . __class__
< type type >
>>> list . __class__
< type type >

Python ( class), . -
.

2. PYTHON

type.
, class.
# prototype
type ( name , bases , dict )

name , bases (
), dict ,
, .

class MyClass ( BaseClass ) :

attribute = 42

MyClass = type ( " MyClass " , ( BaseClass ,) , { attribute : 42 })

__metaclass__.
() , , ,
( self),
. cls.
, , .

>>> class UselessType ( type ) :

...
def __new__ ( metacls , name , bases , dct ) :
...
print " Allocating memory for class " , name
...
return super ( UselessType , metacls ) . __new__ ( metacls , name , bases ,
dct )
...
...
def __init__ ( cls , name , bases , dct ) :
...
print " Initing ( configuring ) class " , name
...
super ( UselessType , cls ) . __init__ ( name , bases , dct )
...
...
def __call__ ( cls , * args , ** kwargs ) :
...
print " Calling % s " % cls . __name__
...
return super ( UselessType , cls ) . __call__ (* args , ** kwargs )
...
...
def whoami ( cls ) :
...
print cls . __name__
>>> class UselessClass ( object ) :
...
__metaclass__ = UselessType

Allocating memory for class UselessClass

Initing ( configuring ) class UselessClass
>>> u = UselessClass ()
Calling UselessClass

>>> UselessClass . whoami ()

UselessClass
>>> u . whoami ()
Traceback ( most recent call last ) :
File " < stdin > " , line 1 , in < module >
Att ribute Error : UselessClass object has no attribute whoami

, __new__ __init__ .
__call__ .
class, Python
. , -

2. PYTHON

. Python :
1. dict[__metaclass__] , .
2. , , ( __class__, ,
).
3. , __metaclass__, .
,
.
__metaclass__ ,
(callable) , .
def meta ( name , bases , dct ) :
# setting useless attr
dct [ foo ] = bar
return type ( name , bases , dct )
5

class Bar ( object ) :

__metaclass__ = meta

>>> Bar () . foo

bar

Singleton :
class S i n g l e t o n M e t a c l a s s ( type ) :
instance = None

def call ( cls , * args , ** kwargs ) :

if cls . instance is None :
cls . instance = super ( SingletonMetaclass , cls ) . __call__ (
* args , ** kwargs )
return cls . instance
class Foo ( object ) :
__metaclass__ = S i n g l e t o n M e t a c l a s s
>>> Foo () is Foo ()
True

Python, ,
, , . : , , ,
, , ORM / .
,
, , dump_file,
JSON.
import json

class I n i t F r o m D u m p M e t a ( type ) :
def __new__ ( metacls , name , bases , dct ) :
if dump_file in dct :
try :
dump_file = file ( dct [ dump_file ])

2. PYTHON

dump = json . load ( dump_file )

except ( IOError , ValueError ) :
pass
else :
dct . update ( dump )

return super ( InitFromDumpMeta , metacls ) . new ( metacls , name , bases ,

dct )

/tmp/dump.json :
{ " one " : 1 , " two " : 2 }

:
class I n i t F r o m D u m p C l a s s ( object ) :
__metaclass__ = I n i t F r o m D u m p M e t a
dump_file = / tmp / dump . json

:
1

>>> I n i t F r o m D u m p C l a s s . one
1
>>> I n i t F r o m D u m p C l a s s . two
2

2.13

Python .
. , ,
( , dict ) __dict__.
class klass ( object ) :
pass

>>> c = klass ()
>>> c . attr = some
>>> print c . __dict__
{ attr : some }
>>> print klass . __dict__
{ __dict__ : < attribute __dict__ of klass objects > ,
__module__ : __main__ ,
__weakref__ : < attribute __weakref__ of klass objects > ,
__doc__ : None }

, , :
class BaseA ( object ) :
pass
3

class BaseB ( object ) :

pass

class Child ( BaseA , BaseB ) :

pass
>>> c = Child ()
>>> c . __class__

2. PYTHON

< class main . Child >

>>> c . __class__ . __base__
< class __main__ . BaseA >
>>> c . __class__ . __bases__
( < class __main__ . BaseA >, < class __main__ . BaseB >)
>>> c . __class__ . __mro__
( < class __main__ . Child >,
< class __main__ . BaseA >,
< class __main__ . BaseB >,
< type object >)

inspect.
import inspect
def f (a , b , c =3) :
return a + b + c
5

>>> inspect . getargspec ( f )

ArgSpec ( args =[ a , b , c ] , varargs = None , keywords = None , defaults =(3 ,) )
>>> f . func_code . co_code
|\ x00 \ x00 |\ x01 \ x00 \ x17 |\ x02 \ x00 \ x17S

[1] M. Pilgrim, Dive into Python 3. Apress, 2nd ed., 2009.
[2] Official Python Tutorial. http://docs.python.org/tutorial/index.html.
[3] Python Library Reference. http://docs.python.org/library/index.html.
[4] Python Language Reference. http://docs.python.org/reference/index.html.

3.1

3.1.1

,
. () , . ;
.

, , .

, . , .
1

int y = 0 , z = 0;
co
x=y+z;
//
y = 1; z = 2;
oc ;

x = y + z y
z, x 0, 1, 2
3.
, .
( int) (,
), .
: , .

.
, .
( ).
, , ,
, , .
, e ,
, ,
. , e , e,
, , . , x = e
, (,
), .
, , , .
.
.
, . ,
,
. x = e
, e , x , e
, x.
,
,

. , e , x , , ,
. , e , , , ,
; .
int x = 0 , y = 0;
co x = x + 1; // y = y + 1; oc ;
int x = 0 , y = 0;
co x = y + 1; // y = y + 1; oc ;

:
, , . .
,
, ,
.

h i. , hei ,
e .
await:
hawait ( B ) S ;i

(delay condition), S , .
:
hawait ( s > 0) s = s - 1;i

await , . , await . , await .

await, .
, await ,
. await:
hS ;i

S ,
, S , S ; hS; i
, S.
await :
hawait ( B ) ;i

B , ,
hawait (B);i
while ( not B ) ;

(spin loop).
, B. , ,
. () , await,
, .
await B.
, B . B ,
. , ,
,
.

3.1.2

(fairness), ,
.

(unconditional fairness). , .
(weak fairness). , : 1) , 2)
,
, , .
(strong fairness). , 1) , 2)
, .

3.2

,
, . ,
.
( ),, , .

3.2.1

. n ,
. ,
. , ,
:

process CS [ i = 1 to n ] {
while ( true ) {
;
;
;
;
}
}

, .
, :
(1) (mutual exclusion). .
(2) ( ) (freedom from livelocks).
,
.
(3) . , ,
.

(4) ( , freedom from starvation). , , - .

, .
, . . await. (1) .
, , ,
, , ,
. ,
.
3.1.
Listing 3.1: :

bool in1 = false , in2 = false ;

## MUTEX : (in1 in2) --
process CS1 {
while ( true ) {
< await (! in2 ) in1 = true ; > /* */
;
in1 = false ; /* */
;
}
}

process CS2 {
while ( true ) {
< await (! in1 ) in2 = true ; > /* */
;
in2 = false ; /* */
;
}
}

3.2.2

, 3.1, . n n .
:
, . , ,
, .
lock , , . :
lock == (in1 in2)
lock in1 in2, 3.1 , 3.2.2.
, 3.2.2,
3.1 ,
.

Listing 3.2:
bool lock = false ;

process CS1 {
while ( true ) {
< await (! lock ) lock = true ; > /* */
;
lock = false ; /* */
;
}
}
process CS2 {
while ( true ) {
< await (! lock ) lock = true ; > /* */
;
lock = false ; /* */
;
}
}

-
lock in1 in2, 3.2.2,
, , ,
. ,
-.
- (test-and-set, TS)
lock .
TS lock,
, lock.
TS :
1

bool TS ( bool lock ) {

< bool initial = lock ; /* */
lock = true ; /* lock */
return initial ; > /* */
}

TS,
3.2.2 , 3.3. ,
, (spin lock),
, .
Listing 3.3:
bool lock = false ; /* */

process CS [ i = 1 to n ] {
while ( true ) {
while ( TS ( lock ) ) skip ; /* */
;
lock = false ; /* */
;
}
}

3.3 (1), (2) (3).

, (4) .
, , lock .
, , . ,
, . , 3.3 .
,
:
.
.
--
3.3 , ,
. , lock.
, ,
.
TS lock, .
, TS
, .
- , -. lock ,
TS.
Listing 3.4: --
bool lock = false ; /* */

process CS [ i = 1 to n ] {
while ( true ) {
while ( lock ) skip ; /* */
while ( TS ( lock ) ) {
while ( lock ) skip ;
}
;
lock = false ; /* */
;
}
}

await
hS;i,
. CSenter ; CSexit . hS;i :

CSenter ;
S;
CSexit ;

, , , S ( , S),
. , h i
CSenter CSexit.
hawait(B) S;i. , , B , S.
S, B .
, , S. B, ,
.
2

CSenter ;
while (! B ) { CSexit ; Delay ; CSenter ; }
S;
CSexit ;

, , ,
B S, , S,
.
, B . , B , . , ,
, , , , B , . B
S B . Delay , .
.

3.2.3

, ,
. (4) . , , , .
, , ,
, ,
. , ,
.
: , .
,
(round-robin) , , , , .

( 3.1).
, , ,
, . ,
, , . ,
, .
( ) , ,
. , , .
, 3.1.
.
- , ,
, ,
.
, ,
, ,
.
last , , CS1
CS2 . , . . it1 it2 , .
, 3.5.
Listing 3.5: :
1

bool in1 = false , in2 = false ;

int last = 1;
process CS1 {
while ( true ) {
last = 1; in1 = true ; /* */
< await (! in2 or last == 2) ; >
;
in1 = false ;
;
}
}
process CS2 {
while ( true ) {
last = 2; in2 = true ; /* */
< await (! in1 or last == 1) ; >
;
in2 = false ;
;
}
}

3.5 ,
await. , await
, .
, await 3.5 ,

.
. .
, CS1 ,
. CS1 , in2 , in2 .
CS2 last 2; ,
, in2 .
, last == 2 , ,
last , CS1 . ,
, CS1 ,
. ( CS2 .)
,
await while, ,
.
( 3.6).
Listing 3.6: :
bool in1 = false , in2 = false ;
int last = 1;
3

process CS1 {
while ( true ) {
last = 1; in1 = true ; /* */
while ( in2 and last == 1) skip ;
;
in1 = false ; /* */
;
}
}

process CS2 {
while ( true ) {
last = 2; in2 = true ; /* */
while ( in1 and last == 2) skip ;
;
in2 = false ; /* */
;
}
}

.
.
, n ,
n 1 .
, , .
, n 1 , ,
.
in[1:n] last[1:n] . in[i] , CS[i]. last[j] ,
j. 3.2.3. for n-1 . for CS[i]
. CS[i] ,
, CS[i] , j. j ,
CS[i] , CS[i]

. , n 1 , n 2 . ,
n .
Listing 3.7: n
int in [1: n ] = ([ n ] 0) , last [1: n ] = ([ n ] 0) ;
3

process CS [ i = 1 to n ] {
while ( true ) {
for [ j = 1 to n ] { /* */
/* , i j
*/
last [ j ] = i ; in [ i ] = j ;
for [ k = 1 to n st i != k ] {
/* , k
i j */
while ( in [ k ] >= in [ i ] and last [ j ] == i ) skip ;
}
}
;
in [ i ] = 0; /* */
;
}
}

n ,
. ,
,
, ,
.

, () .
number next 1, turn[1:n]
0. , CS[i] turn[i] number
number 1. () , . CS[i] ,
next .
CS[i] 1 next, .
, 3.2.3.
number next ,
.
1

TICKET : next > 0

(i : 1 i n :
( CS [ i ] )
( turn [ i ] == next ) ( turn [ i ] > 0)
(j : 1 j n, j 6= i: turn [ i ] 6= turn [ j ]) )

, turn .
, turn[i] next, . . .

. , , ,
, .
Listing 3.8: :
int number = 1 , next = 1 , turn [1: n ] = ([ n ] , 0) ;
## -- TICKET (. )

process CS [ i = 1 to n ] {
while ( true ) {
< turn [ i ] = number ; number = number + 1; >
< await ( turn [ i ] == next ) : >
;
< next = next + 1; >
;
}
}

3.2.3 . await
, . ,
next, , . ,
( number ) .
,
. , .
(fetch-and-add, FA), :

FA ( var , incr ) {
< int tmp = var ;
var = var + incr ;
return tmp ; >
}

Listing 3.9: :
int number = 1 , next = 1 , turn [1: n ] = ([ n ] , 0) ;

process CS [ i = 1 to n ] {
while ( true ) {
turn [ i ] = FA ( number , 1) ; /* */
while ( turn [ i ] != next ) skip ;
;
next = next + 1; /* */
;
}
}

, , .

. ,
:
turn [ i ] = number ; < number = number + 1; >

number ,
. ,
!
, .
: .
, . , CSenter , CSexit .
5.4.2 .

CSenter ;
turn [ i ] = number ;
number = number + 1;
CSexit ;

, ,
-.
- , ,
.
, . ,
, CSenter.
, next turn[i].

3.2.4

3.3

3.3.1

3.3.2

3.3.3

3.3.4

3.3.5

4.1

TCP

; Transmission Control Protocol.

,
. TCP , -, ..
, TCP . TCP
,
.
TCP , . .
, .
.
TCP (round-trip-time, RTT )
. TCP RTT.
TCP , . 2048 TCP,
TCP.
11024, 10252048. - ,
TCP , ,
. TCP , , , .
TCP (flow control ). TCP , .
(window ).

4.1.1

1. . (passive open).
2. (active open), SYN,
. IP, TCP TCP.
94

3. SYN
SYN, . SYN ACK
.
4. SYN .

socket

socket, bind, listen

connect,

connect

SYN J
CK J+1
SYN K, A
ACK K+1

accept,

accept
. 4.1: TCP
(c. . 4.1), TCP (TCP three-way handshake).

4.1.2

TCP

SYN TCP. :
MSS. (maximum
segment size, MSS) ,
TCP .
(Window scale option). ,
TCP, 65535, 16 .

. , TCP
014 ,
(65535 214 ).
(Timestamp option). , ,
, .

4.1.3

TCP

.
1. (active close). TCP
FIN.

2. , FIN, (passive close).

FIN TCP. FIN
.
3. , ,
. TCP FIN .
4. FIN.
FIN . , 2 3, . , 2 3 -
, , ,
. (half-close).

4.1.4

TCP

. 4.2 . 97.
4.1:

CLOSED
LISTEN
SYN_SENT
SYN_RECEIVED
ESTABLISHED
CLOSE_WAIT

.

;

;
, ;

, ;

;
, ,

FIN_WAIT_1
CLOSING
LAST_ACK
FIN_WAIT_2
TIME_WAIT

4.1.5

TIME_WAIT

. , , . MSL (maximum segment lifetime

). MSL ,
IP . 8-
(hop limit), 255. ,

Timeout after two maximum

segment lifetimes (2*MSL)

CLOSED

e
YN
/S
en
op

tiv
Ac

Passive open

Timeout/RST

LISTEN

Send /SYN

SYN/SYN + ACK
SYN/SYN + ACK
SYN_RCVD

SYN_SENT

SYN + ACK/ACK

ACK

Close/FIN
ESTABLISHED

FIN/ACK

Close/FIN

CLOSE_WAIT

FIN_WAIT_1

ACK
Close/FIN

ACK

FIN_WAIT_2

FIN +
ACK/ACK

LAST_ACK
CLOSING

ACK
FIN/ACK
TIME_WAIT

. 4.2: TCP

MSL .
TIME_WAIT:
TCP;

, .
, ACK . FIN, ,
ACK .
, RST, .
, ,
. TCP
.
, TCP ,
TIME_WAIT.

4.2

UDP

; User Datagram Protocol.

, .
.
UDP , IP . . - , .
, (record ).
TCP, , .
UDP , . , UDP
, .
UDP . UDP , UDP.

4.2.1

UDP TCP

UDP:
UDP , UDP , .
UDP .
. TCP 10 ,
.

4.3

POSIX API

4.3.1

DAYTIME

DAYTIME .
IPv4- .
# include < stdio .h >
# include < stdlib .h >

# include < string .h >

# include < error .h >

# include < errno .h >
# include < unistd .h >
# include < arpa / inet .h >
# include < sys / socket .h >
# define BUFFER_SIZE

1024

int main ( int argc , char * argv [])

{
char * server_ip ;
int server_port ;
int server_socket ;
struct sockaddr_in server_addr ;
char buffer [ BUFFER_SIZE + 1];
int n , result ;

/* extra byte for \0 */

if ( argc != 3)
error (1 , 0 , " usage : date - client - basic < server - ip > < port > " ) ;
25

server_ip = argv [1];

server_port = atoi ( argv [2]) ;
if ( server_port <= 0)
error (1 , 0 , " port number must be a number > 0 " ) ;
30

if (( server_socket = socket ( AF_INET , SOCK_STREAM , 0) ) < 0)

( error (1 , errno , " socket " ) ) ;
memset (& server_addr , 0 , sizeof ( server_addr ) ) ;
server_addr . sin_family = AF_INET ;
server_addr . sin_port = htons ( server_port ) ;

if (( result = inet_pton ( AF_INET , server_ip , & server_addr . sin_addr ) ) < 0)

error (1 , errno , " inet_pton " ) ;
else if ( result == 0)
error (1 , 0 , " IPv4 address must be specified " ) ;

if ( connect ( server_socket , ( struct sockaddr *) & server_addr ,

sizeof ( server_addr ) ) < 0)
error (1 , errno , " connect " ) ;

while (( n = read ( server_socket , buffer , BUFFER_SIZE ) ) > 0) {

buffer [ n ] = \0 ;
if ( printf ( " % s " , buffer ) < 0)
error (1 , 0 , " error of printing to stdout " ) ;
}

if ( n < 0)
error (1 , errno , " read " ) ;
55

if ( close ( server_socket ) < 0)

error (1 , errno , " close " ) ;
return 0;
60

100

. DAYTIME ,
1 :
% nc6 -l -- half - close -- continuous -- exec date -p 45678

:
% date - client - basic 127.0.0.1 45678
Fri Feb 3 01:22:43 FET 2012

Voil`
a! ! , .
socket
/ socket,
.
# include < sys / socket .h >
int socket ( int family , int type , int protocol ) ;

, -1 . family
.
4.2: family connect

AF_INET
AF_INET6
AF_LOCAL
AF_ROUTE
AF_KEY

IPv4
IPv6
Unix

4.3:

SOCK_STREAM
SOCK_DGRAM
SOCK_SEQPACKET
SOCK_RAW

()

protocol
, .
family type . socket
, . , . , .
1

man.

101

4.4: protocol

IPPROTO_TCP
IPPROTO_UDP
IPPROTO_SCTP

TCP
UDP
SCTP

connect
connect .
# include < sys / socket .h >
int connect ( int sockfd , const struct sockaddr * servaddr , socklen_t addrlen ) ;

sockfd , socket.
IP- .
bind connect,
, IP- .
TCP connect .
, . :
1. SYN, ETIMEDOUT.
2. SYN RST, , . ECONNREFUSED RST.
3. SYN ICMP
- . , SYN . ,
ICMP EHOSTUNREACH,
ENETUNREACH.
inet_pton inet_ntop
IP-, , , . p n .
# include < arpa / inet .h >
int inet_pton ( int family , const char * str , void * addr ) ;

# include < arpa / inet .h >

const char * inet_ntop ( int family , const void * addr , char * str ,
socklen_t len ) ;

family AF_INET, AF_INET6. family , errno, EAFNOSUPPORT

. len , <netinet/in.h>:

4.
# define
# define

4.3.2

I NE T_ A DD RS T RL EN
INET6_ADDRSTRLEN

102

16
46

DAYTIME ,

IPv4-, IPv6- ;
,
/etc/services.
# include < stdio .h >
# include < stdlib .h >
# include < string .h >
5

# include < error .h >

# include < errno .h >
# include < netdb .h >
# include < unistd .h >
# include < sys / socket .h >
# define BUFFER_SIZE

1024

int main ( int argc , char * argv [])

{
char * server_host ;
char * service ;
int server_socket ;
struct addrinfo hints , * addr , * addrs ;
char buffer [ BUFFER_SIZE + 1];
/* extra byte for \0 */
int n , result ;
if ( argc != 3)
error (1 , 0 , " usage : date - client < server - ip > < port > " ) ;

server_host = argv [1];

service = argv [2];

memset (& hints , 0 , sizeof ( hints ) ) ;

hints . ai_family = AF_UNSPEC ;
hints . ai_socktype = SOCK_STREAM ;
if (( result = getaddrinfo ( server_host , service , & hints , & addrs ) ) != 0)
error (1 , 0 , " getaddrinfo : % s " , gai_strerror ( result ) ) ;
addr = addrs ;
do {

if (( server_socket = socket ( addr - > ai_family , addr - > ai_socktype ,

addr - > ai_protocol ) ) < 0)
continue ;
if ( connect ( server_socket , addr - > ai_addr , addr - > ai_addrlen ) == 0)
break ;

if ( close ( server_socket ) != 0)
error (1 , errno , " close " ) ;
} while (( addr = addr - > ai_next ) != NULL ) ;
if ( addr == NULL )
error (1 , 0 , " was not able to connect to specified network address and
service " ) ;

103

freeaddrinfo ( addrs ) ;
while (( n = read ( server_socket , buffer , BUFFER_SIZE ) ) > 0) {
buffer [ n ] = \0 ;
if ( printf ( " % s " , buffer ) < 0)
error (1 , 0 , " error of printing to stdout " ) ;
}

if ( n < 0)
error (1 , errno , " read " ) ;

if ( close ( server_socket ) < 0)

error (1 , errno , " close " ) ;
return 0;

DAYTIME :
% date - client localhost 45678
Fri Feb 3 02:06:13 FET 201

getaddrinfo
. sockaddr. getaddrinfo
.
# include < netdb .h >
int getaddrinfo ( const char * node , const char * service ,
const struct addrinfo * hints ,
struct addrinfo ** res ) ;

res addrinfo.

struct addrinfo {
int
ai_flags ;
int
ai_family ;
int
ai_socktype ;
int
ai_protocol ;
size_t
ai_addrlen ;
struct sockaddr * ai_addr ;
char
* ai_canonname ;
struct addrinfo * ai_next ;
};

/*
/*
/*
/*
/*
/*
/*
/*

AI_PASSIVE , AI_CANONNAME */
AF_xxx */
SOCK_xxx */
0 or IPPROTO_xxx for IPv4 or IPv6 */
ai_addr length */
pointer to canonical name */
pointer to socket address structure */
pointer to next structure in list */

node , (-
IPv4 IPv6). service
, , .
hints , addrinfo, ,
.
ai_flags ( AI_xxx, );
ai_family ( AF_xxx);
ai_socktype ( SOCK_xxx);
ai_protocol.

104

ai_flags:
AI_PASSIVE , ;
AI_CANONNAME ;

4.3.3

DAYTIME

.
# include < stdio .h >
# include < stdlib .h >
# include < string .h >
5

# include < error .h >

# include < errno .h >
# include < netdb .h >
# include < unistd .h >
# include < sys / socket .h >
# include < time .h >

# define BUFFER_SIZE
# define BACKLOG

1024
5

int main ( int argc , char * argv [])

{
int listen_socket , client_socket ;
socklen_t socket_length ;
char * service ;
struct addrinfo hints , * addr , * addrs ;
struct s o c k a d d r _ s t o r a g e client_addr ;
char buffer [ BUFFER_SIZE + 1];
/* extra byte for \0 */
time_t ticks ;
int result ;
const int on = 1;
if ( argc != 2)
error (1 , 0 , " usage : echo - client - basic < port > " ) ;
service = argv [1];

memset (& hints , 0 , sizeof ( hints ) ) ;

hints . ai_flags = AI_PASSIVE ;
hints . ai_family = AF_UNSPEC ;
hints . ai_socktype = SOCK_STREAM ;
if (( result = getaddrinfo ( NULL , service , & hints , & addrs ) ) != 0)
error (1 , 0 , " getaddrinfo : % s " , gai_strerror ( result ) ) ;
addr = addrs ;
do {

if (( listen_socket = socket ( addr - > ai_family , addr - > ai_socktype ,

addr - > ai_protocol ) ) < 0)
continue ;
if ( setsockopt ( listen_socket , SOL_SOCKET , SO_REUSEADDR ,
& on , sizeof ( on ) ) != 0)
error (1 , errno , " setsockopt " ) ;

105

if ( bind ( listen_socket , addr - > ai_addr , addr - > ai_addrlen ) == 0)

break ;
if ( close ( listen_socket ) != 0)
error (1 , errno , " close " ) ;
} while (( addr = addr - > ai_next ) != NULL ) ;

if ( addr == NULL )
error (1 , 0 , " was not able to bind to specified network address and
service " ) ;
freeaddrinfo ( addrs ) ;

if ( listen ( listen_socket , BACKLOG ) != 0)

error (1 , errno , " listen " ) ;
65

for (;;) {
socket_length = sizeof ( client_addr ) ;
if (( client_socket = accept ( listen_socket ,
( struct sockaddr *) & client_socket ,
& socket_length ) ) < 0)
error (1 , errno , " accept " ) ;

ticks = time ( NULL ) ;

snprintf ( buffer , sizeof ( buffer ) , " %.24 s \ r \ n " , ctime (& ticks ) ) ;
if ( write ( client_socket , buffer , strlen ( buffer ) ) < 0)
error (1 , errno , " write " ) ;
/*
*
ignoring the fact that write can write less bytes
*
than specified
*/

if ( close ( client_socket ) < 0)

error (1 , errno , " close " ) ;
}
85

if ( close ( listen_socket ) < 0)

error (1 , errno , " close " ) ;
return 0;
90

, ,
,
. TIME_WAIT. setsockopt
.
:
% date - server 45678
# in another shell
% nc6 localhost 45678
Fri Feb 3 07:37:05 2012

: !
bind
bind , IP
.
# include < sys / socket .h >

106

int bind ( int sockfd , const struct sockaddr * addr , socklen_t addrlen ) ;

,
.
bind IP- . IP . IP-,
IP-.
: ,
IP-.
, bind . IP- ,
IP-,
.
IPv4 , INADDR_ANY.
struct sockaddr_in addr ;
servaddr . sin_addr . s_addr = htonl ( INADDR_ANY ) ;

listen
listen TCP .
1. socket, , ,
, connect. listen ,
.
2. ,
.
# include < sys / socket .h >
int listen ( int sockfd , int backlog ) ;

backlog, , :
1. (incomplete connection queue), SYN, ,
TCP.
2. (complete connection queue), ,
TCP.
,
. ,
.
accept,
, .
:

107

,
.
, SYN, TCP
SYN RST. ,
, TCP SYN,
, .
, , ,
accept, TCP-,
.
accept
accept TCP .
, (
).
# include < sys / socket .h >
int accept ( int sockfd , struct sockaddr * addr , socklen_t * addrlen ) ;

addr addrlen (). accept ,

, .
TCP . accept .

4.3.4

, - (, ). -.
-
:
(
), -.
.
TCP , ,
-.
TCP, UDP, -.
select
, ,
, , .

108

# include < sys / select .h >

int select ( int nfds , fd_set * readfds , fd_set * writefds ,
fd_set * exceptfds , struct timeval * timeout ) ;

, , , . timeval :
struct timeval {
long tv_sec ;
long tv_usec ;
};

/* seconds */
/* microseconds */

:
1. , . timeout NULL.
2. , -
- .
3. . timeout timeval, .
readfds, writefds exceptfds ,
. :
void
void
void
int

FD_ZERO ( fd_set * fds ) ;

FD_SET ( int fd , fd_set * fds ) ;
FD_CLR ( int fd , fd_set * fds ) ;
FD_ISSET ( int fd , fd_set * fds ) ;

/*
/*
/*
/*

clears a set */
add a descriptor to a set */
remove a descriptor from a set */
tests whether a descriptor is
part of a set */

FD_SETSIZE
fd_set.

, :
1.
(low water-mark).
2. .
.
3. , .
4. , .
, :
1.
,
, .
2. , , . SIGPIPE.
3. , .
, ,
, .

109

,
TCP

t
t

t
t
t
t

t
t

poll
poll select.
# include < poll .h >
int poll ( struct pollfd * fds , nfds_t nfds , int timeout ) ;

. pollfd, , fd.

struct pollfd {
int
fd ;
short events ;
short revents ;
};

/* file descriptor */
/* requested events */
/* returned events */

4.5: events revents

events

revents

POLLIN
POLLPRI

r
r

POLLOUT

r
r
r

POLLERR
POLLHUP
POLLNVAL

timeout , .

110

<0
0

epoll
epoll , Linux 2.5.44. , select poll.
kqueue FreeBSD.
# include < sys / epoll .h >
int epoll_create ( int size ) ;

epoll_create epoll- , . size ,

. ,
close.
# include < sys / epoll .h >
int epoll_ctl ( int epfd , int op , int fd , struct epoll_event * event ) ;

epoll-, epfd. op:

EPOLL_CTL_ADD. fd , event.
EPOLL_CTL_MOD. , , fd.
EPOLL_CTL_DEL. fd epoll-.
epoll_event :

typedef union epoll_data {

void
* ptr ;
int
fd ;
__uint32_t
u32 ;
__uint64_t
u64 ;
} epoll_data_t ;
struct epoll_event {
__uint32_t
events ;
epoll_data_t data ;
};

events

/* epoll events */
/* user data variable */

, :

EPOLLIN. .
EPOLLOUT. .
EPOLLRDHUP.
.
EPOLLPRI. .

111

EPOLLERR. . epoll_wait ,
events.
EPOLLHUP. . epoll_wait ,
events.
EPOLLET. edge-triggered . Level-triggered
.
EPOLLONESHOT. , epoll_wait, ,
epoll_ctl EPOLL_CTL_MOD.
:
1. rfd, , epoll.
2. 2kB .
3. epoll_wait, , rfd .
4. 1kB .
5. epoll_wait.
level-triggered epoll_wait , edge-triggered .
# include < sys / epoll .h >
int epoll_wait ( int epfd , struct epoll_event * events ,
int maxevents , int timeout ) ;

, epoll-, epfd
. events epoll_event, ,
maxevents. timeout timeout poll.

4.3.5

! ,
inetd.

! .
! .
% ,
.

% , , .

112

Python. , ,
, /.
echo-, . - :
import argparse

def parse_args ( description ) :

parser = argparse . Argum entPar ser ( description = description )
parser . add_argument ( port , type = int , help = port to bind )
return parser . parse_args ()

, ,
:
import socket
import sys
4

def l i s t e n i n g _ s o c k e t ( port ) :
host = None
family = socket . AF_UNSPEC
socket_type = socket . SOCK_STREAM
protocol = 0
flags = socket . AI_PASSIVE
server_socket = None
for addrinfo in socket . getaddrinfo ( host , port , family , socket_type ,
protocol , flags ) :
family , socket_type , protocol , canonical_name , addr = addrinfo
try :

server_socket = socket . socket ( family , socket_type , protocol )

except socket . error , msg :
server_socket = None
continue
server_socket . setsockopt ( socket . SOL_SOCKET , socket . SO_REUSEADDR , 1)
try :
server_socket . bind ( addr )
server_socket . listen (1)
except socket . error , msg :
server_socket . close ()
server_socket = None
continue
break

if server_socket is None :
print could not open socket
sys . exit (1)
return server_socket

: ,
.

def communicate ( client_socket ) :

while True :
data = client_socket . recv (1024)
if not data :
break

113

client_socket . sendall ( data )

aux.
:
import os
import signal
import aux
5

def main () :
args = aux . parse_args ( Echo server with fork processing model . )
server_socket = aux . l i s t e n i n g _ s o c k e t ( args . port )
10

signal . signal ( signal . SIGCHLD , signal . SIG_IGN )

while True :
client_socket , client_addr = server_socket . accept ()

if os . fork () == 0:
server_socket . close ()
aux . communicate ( client_socket )
client_socket . close ()
return
else :
client_socket . close ()
if __name__ == __main__ :
main ()

fork - , ,
. :
1. SIGCHLD,
( );
2. SIGCHLD, , , os.waitpid
(-1, os.WNOHANG).
FSM
FSM (Finite State Machine) ,
(select, poll ..).

! CPU.
% .
% FSM, ,
.

! , FSM .
% .
,
.

4.
Listing 4.1: select
import select
import aux
5

def main () :
args = aux . parse_args ( Echo server with select - processing model . )
server_socket = aux . l i s t e n i n g _ s o c k e t ( args . port )
soc ket_to _data = {}
cli ent_so ckets = []
all_sockets = [ server_socket ]
while True :
readable , _ , _ = select . select ( all_sockets , [] , [] , 0.05)

if server_socket in readable :
client_socket , _ = server_socket . accept ()
soc ket_to _data [ client_socket ] =
cli ent_so ckets . append ( client_socket )
all_sockets . append ( client_socket )

readable . remove ( server_socket )

for client_socket in readable :

data = client_socket . recv (1024)
if not data :
del socket _to_da ta [ client_socket ]
cli ent_so ckets . remove ( client_socket )
all_sockets . remove ( client_socket )
client_socket . close ()
else :
soc ket_to _data [ client_socket ] += data
c h e c k _ w r i t a b i l i t y = [ cs for cs in clie nt_soc kets
if so cket_t o_data [ cs ]]
_ , writable , _ = select . select ([] , check_writability , [] , 0.05)

for client_socket in writable :

data = soc ket_to_data [ client_socket ]
n = client_socket . send ( data )
if n > 0:
sock et_to_ data [ client_socket ] = data [ n :]

if __name__ == __main__ :
main ()
Listing 4.2: poll
import select
import aux
5

def main () :
args = aux . parse_args ( Echo server with poll - processing model . )
server_socket = aux . l i s t e n i n g _ s o c k e t ( args . port )
server_fileno = server_socket . fileno ()
fil eno_to _data = {}

114

115

f i l e n o _ t o _ s o c k e t = {}
read_poll = select . poll ()
read_poll . register ( server_fileno , select . POLLIN )
15

while True :
for fileno , eventmask in read_poll . poll (50) :
if fileno == server_fileno :
client_socket , _ = server_socket . accept ()
client_fileno = client_socket . fileno ()
fil eno_to _data [ client_fileno ] =
f i l e n o _ t o _ s o c k e t [ client_fileno ] = client_socket

read_poll . register ( client_fileno , select . POLLIN )

else :
client_socket = f i l e n o _ t o _ s o c k e t [ fileno ]
data = client_socket . recv (1024)
if not data :
read_poll . unregister ( fileno )
del fileno _to_da ta [ fileno ]
del f i l e n o _ t o _ s o c k e t [ fileno ]
client_socket . close ()
else :
fil eno_to _data [ fileno ] += data
c h e c k _ w r i t a b i l i t y = [ f for f , d in fi leno_t o_data . iteritems () if d ]
if not c h e c k _ w r i t a b il i t y :
continue
write_poll = select . poll ()
for fileno in c h e c k _ w r i t a b i l i t y :
write_poll . register ( fileno , select . POLLOUT )

for fileno , eventmask in write_poll . poll (50) :

if eventmask & ( select . POLLERR | select . POLLHUP ) :
read_poll . unregister ( fileno )
f i l e n o _ t o _ s oc k e t [ fileno ]. close ()
del fileno _to_da ta [ fileno ]
del f i l e n o _ t o _ s o c k e t [ fileno ]
continue
client_socket = f i l e n o _ t o _ s o c k e t [ fileno ]
data = fil eno_to_data [ fileno ]
n = client_socket . send ( data )
if n > 0:
file no_to_ data [ fileno ] = data [ n :]

if __name__ == __main__ :
main ()

select poll CPU .

, .

! .
! .

116

% .
% .
, . , fork - .
import threading
import aux
5

class Worker ( threading . Thread ) :

def __init__ ( self , client_socket ) :
super ( Worker , self ) . __init__ ()
self . client_socket = client_socket
self . daemon = True
def run ( self ) :
aux . communicate ( self . client_socket )
self . client_socket . close ()

def main () :
args = aux . parse_args ( Echo server with threaded - processing model . )
server_socket = aux . l i s t e n i n g _ s o c k e t ( args . port )
20

while True :
client_socket , client_addr = server_socket . accept ()
worker = Worker ( client_socket )
worker . start ()

if __name__ == __main__ :
main ()

FSM c

! -

, .
.

! .
% FSM.
Linux epoll.
Listing 4.3: epoll
import select
import aux
5

def main () :
args = aux . parse_args ( Echo server with epoll - processing model . )
server_socket = aux . l i s t e n i n g _ s o c k e t ( args . port )
server_fileno = server_socket . fileno ()
fil eno_to _data = {}
f i l e n o _ t o _ s o c k e t = {}

117

read_epoll = select . epoll ()

read_epoll . register ( server_fileno , select . EPOLLIN )
15

while True :
for fileno , eventmask in read_epoll . poll (0.05) :
if fileno == server_fileno :
client_socket , _ = server_socket . accept ()
client_fileno = client_socket . fileno ()
fil eno_to _data [ client_fileno ] =
f i l e n o _ t o _ s o c k e t [ client_fileno ] = client_socket

read_epoll . register ( client_fileno , select . EPOLLIN )

else :
client_socket = f i l e n o _ t o _ s o c k e t [ fileno ]
data = client_socket . recv (1024)
if not data :
read_epoll . unregister ( fileno )
del fileno _to_da ta [ fileno ]
del f i l e n o _ t o _ s o c k e t [ fileno ]
client_socket . close ()
else :
fil eno_to _data [ fileno ] += data
c h e c k _ w r i t a b i l i t y = [ f for f , d in fi leno_t o_data . iteritems () if d ]
if not c h e c k _ w r i t a b i l i t y :
continue
write_epoll = select . epoll ()
for fileno in c h e c k _ w r i t a b i l i t y :
write_epoll . register ( fileno , select . EPOLLOUT )

for fileno , eventmask in write_epoll . poll (0.05) :

if eventmask & ( select . EPOLLERR | select . EPOLLHUP ) :
read_epoll . unregister ( fileno )
f i l e n o _ t o _ s o c k e t [ fileno ]. close ()
del fileno _to_da ta [ fileno ]
del f i l e n o _ t o _ s o c k e t [ fileno ]
continue
client_socket = f i l e n o _ t o _ s o c k e t [ fileno ]
data = fil eno_to_data [ fileno ]
n = client_socket . send ( data )
if n > 0:
file no_to_ data [ fileno ] = data [ n :]

write_epoll . close ()
if __name__ == __main__ :
main ()

4.4

TCP/IP :
(application): HTTP, FTP . .;

118

(transport): TCP, UDP . .;

(network): IP, ICMP, IGMP . .;
(data link): Ethernet, Token Ring . .
, TCP UDP IP- . , IP IP-.
Ethernet MAC-.
IPv4- 32 . , , 17.52.86.120. IP- :
A. : 0. : 8.
B. : 10. : 16.
C. : 110. : 24.
D. : 1110. : .
E. : 1111. : .
D , E
.
.
, . 192.168.1.23 C
192.168.1 ( 24 ), 23.
A-C , :
IP-
ID ID

.? .?

hostid

127

loopback

-1

( )

netid

-1

netid

. IP-, , . 0 . , 192.168.1.0/24 192.168.1.255 , 0.0.0.0

. 0 . 127
. , 127, -.

4.4.1

H1 H2, , IP H2, .

119

H1 H3, H1
R1. , H1
, , 2,
R1, ,
H3.

R2
1

R1
2

. 4.3:
, .
. -, . , ,
, .
:
1

% netstat -i
Kernel Interface table
Iface
MTU ... Flg
eth1
1500 ... BMRU
lo
16436 ... LRU
ppp0
1400 ... MOPRU
% route -n
Kernel IP routing table
Destination
Gateway
0.0.0.0
0.0.0.0
81.25.32.68
192.168.1.1
91.187.0.1
0.0.0.0
127.0.0.0
127.0.0.1

Genmask
0.0.0.0
2 55 . 25 5. 2 55 . 25 5
2 5 5. 25 5 .2 5 5. 25 5
255.0.0.0

Flags
U
UGH
UH
UG

Metric
0
0
0
0

Ref
0
0
0
0

Use
0
0
0
0

Iface
ppp0
eth1
ppp0
lo

,
, ,
ppp0. ,
81.25.32.68 ( H) 192.168.1.1 ( G).
91.187.0.1 ( H) ,
ppp0. 127 127.0.0.1.

4.4.2

120

.
.
, ,
, .
.
( ), ,
.
0

15 16

23 24

. 4.4:
, , .
, , , .

4.4.3

. 255.255.255.255.
, , ,
.
.
, .
. ,
. , .
.
, . ,
.

4.4.4

Classless Inter-Domain Routing (CIDR). CIDR . CIDR IP-.

1000 IP-.
C 200.10.4.0 200.10.7.0.
22 ,
200.10.4.0. , IP-
. 255.255.252.0.

4.4.5

121

NAT

IP-.
RFC 1918 , :
10.0.0.010.255.255.255 ( 10/8);
172.16.0.0172.31.255.255 ( 172.16/12);
192.168.0.0192.168.255.255 ( 192.168/16).
,
, IP-.
, (NAT, Network Address Translation).
IP-. NAT
:
.
, .
. NAT IP ,
.
PAT (Port Address Translation, ). , . ,
,
.

4.4.6

ARP

ARP (Address Resolution Protocol) IP-

MAC- Ethernet. .
:
1. TCP gateway.
2. /etc/hosts/ 192.168.1.1 gateway.
3. gethostbyname("gateway"). , , 32- IP-.
4. TCP- IP-.
- ,
IP-.
5. , -,
MAC- . , IP-
. , MAC- .

122

6. ARP - ,
IP- . ,
MAC-.
7. , .
ARP- tcpdump:

% tcpdump - enqti eth1 arp

listening on eth1 , link - type EN10MB ( Ethernet ) , capture size 65535 bytes
00:25:22: b2 :46:86 > ff : ff : ff : ff : ff : ff , ARP , length 42: Request who - has
192.168.1.1 tell 192.168.1.234 , length 28
00:26:5 a : e0 :17: d9 > 00:25:22: b2 :46:86 , ARP , length 60: Reply 192.168.1.1 is at 00:26:5 a : e0 :17: d9 , length 46
00:26:5 a : e0 :17: d9 > 00:25:22: b2 :46:86 , ARP , length 60: Request who - has
192.168.1.234 tell 192.168.1.1 , length 46
00:25:22: b2 :46:86 > 00:26:5 a : e0 :17: d9 , ARP , length 42: Reply 192.168.1.234 is
- at 00:25:22: b2 :46:86 , length 28

ARP
ARP .
:
% arp -a
? (192.168.1.1) at 00:26:5 a : e0 :17: d9 [ ether ] on eth1

4.4.7

ICMP

ICMP (Internet Control Message Protocol) , , , , ,

, , - -, . . ,
, .
ICMP TCP/IP,
IP-. ICMP
TCP/IP.
ICMP - :
ICMP -;
IP - , ( );
, ;
ICMP - ;
;
IP -, , , ,
loopback-.

123

ICMP , ping traceroute.

traceroute. .
UDP- TTL 1. :
1. . TTL , TTL 0,
ICMP.
2. , . , . 1.
ICMP- : . ,
. , ,
TTL . ,
TTL.
Listing 4.4: traceroute
import argparse
import socket

icmp = socket . getp rotoby name ( icmp )

udp = socket . getp rotoby name ( udp )
def create _socke ts ( ttl ) :
recv_socket = socket . socket ( socket . AF_INET , socket . SOCK_RAW , icmp )
send_socket = socket . socket ( socket . AF_INET , socket . SOCK_DGRAM , udp )

send_socket . setsockopt ( socket . SOL_IP , socket . IP_TTL , ttl )

return recv_socket , send_socket

def main ( host , port , hops ) :

addr = socket . gethostbyname ( host )
ttl = 1
while True :
recv_socket , send_socket = cr eate_s ockets ( ttl )
recv_socket . bind (( " " , port ) )
send_socket . sendto ( " " , ( host , port ) )
curr_addr = None
curr_name = None
try :
_ , curr_addr = recv_socket . recvfrom (512)
curr_addr = curr_addr [0]
try :
curr_name = socket . gethostbyaddr ( curr_addr ) [0]
except socket . error :
curr_name = curr_addr
except socket . error :
pass
finally :
send_socket . close ()
recv_socket . close ()
if curr_addr is not None :
curr_host = " % s (% s ) " % ( curr_name , curr_addr )
else :
curr_host = " * "

124

print " % d \ t % s " % ( ttl , curr_host )

ttl += 1
if curr_addr == addr or ttl > hops :
break

if __name__ == __main__ :
parser = argparse . Argum entPar ser ( description = traces path to a network
host )
parser . add_argument ( host , metavar = host , type = unicode ,
help = host which is to trace path to )
parser . add_argument ( -p , -- port , dest = port , metavar = port ,
help = port to use for connection [ default : %( default )
d],
default =45678 , type = int )
parser . add_argument ( -m , -- max - hops , dest = hops , metavar = hops ,
help = max hops [ default : %( default ) d ] ,
default =30 , type = int )

4.4.8

args = parser . parse_args ()

main ( args . host , args . port , args . hops )

DHCP

Dynamic Host Configuration Protocol ( ).

, IP-.
, DHCP ,
. :
IP- ;
;
DNS;
.
DHCP -, UDP. 67 68.
DHCP-, 0.0.0.0,
255.255.255.255.

4.4.9

DNS

Domain Name System ( ). .

IP- .
.
DNS . :

125

arpa

org
wikipedia

in-addr
81

www

example
www

25
32
34
.
. , FQDN (Fully Qualified Domain Name). . , www.example.com
(), www.example.com. ().
. () ; , . .
:
. : .com, .org, .net, .biz.
. , .
.arpa. , .
DNS
. . ,
, . ,
,
- . , ,
.
, DNS, . , DNS- , .
DNS- , , .
. ,
, . .
DNS- . , DNS-
.
DNS- , ,
, .

126

. , , . .
DNS-, , /etc/resolv.conf. :
nameserver 81.25.32.34
nameserver 81.25.32.9

IP- . , ,
. ,
.
DNS
DNS. :

NAME
TTL (Time To Live)

,

DNS-

,

TYPE
CLASS
RDLEN
RDATA

:
A (address record ). IP.
AAAA (IPv6 address record ). IPv6.
CNAME (canonical name record ). . .
MX (mail exchange). .
NS (name server ). DNS- .
PTR (pointer ). IP- .

DNS- host
nslookup.

% host 81.25.32.34
34.32.25.81. in - addr . arpa domain name pointer ns1 . infonet . by .
% host -t mx adsl . by
adsl . by mail is handled by 10 mail . adsl . by .
% nslookup - type = ptr 34.32.25.81. in - addr . arpa
Server :
81.25.32.34

4.
Address :

127

81.2 5.32.3 4#53

34.32.25.81. in - addr . arpa

% nslookup - type = mx adsl . by
Server :
81.25.32.34
Address :
81.2 5.32.3 4#53

name = ns1 . infonet . by

adsl . by mail exchanger = 10 mail . adsl . by

4.5
4.5.1

- - :
1. alarm, SIGALRM, .
2. - select,
, read write.
3. SO_RCVTIMEO SO_SNDTIMEO.
/,
connect, TCP
( 75 ).

4.5.2

.
TCP 40 ,
4000%.
, . , . RFC 1122 , ,
. : .
TCP . , TCP
ACK ,
. ACK .
TP ,
, / RT T + TP .
,
, , , .
, , .

4.5.3

IANA (Internet Assigned Numbers Authority

) .

128

1. : 0 1023.
IANA.
2. : 1024 49151. IANA, IANA
.
3. , : 49152 65535.

4.6

ps
. , :
wchan , . -, . *, , ps
.
ping
ping ICMP ECHO_REQUEST
( ) ICMP ECHO_RESPONSE
.

% ping -c 3 informatics . by
PING informatics . by ( 9 1 . 1 4 9 . 1 5 7 . 1 5 2 ) 56(84) bytes of data .
64 bytes from vh24 . hoster . by ( 9 1 . 1 4 9 . 1 5 7 . 1 5 2 ) : icmp_req =1 ttl =59 time =403 ms
64 bytes from vh24 . hoster . by ( 9 1 . 1 4 9 . 1 5 7 . 1 5 2 ) : icmp_req =2 ttl =59 time =399 ms
64 bytes from vh24 . hoster . by ( 9 1 . 1 4 9 . 1 5 7 . 1 5 2 ) : icmp_req =3 ttl =59 time =267 ms
--- informatics . by ping statistics --3 packets transmitted , 3 received , 0% packet loss , time 1999 ms
rtt min / avg / max / mdev = 2 6 7 . 6 5 1 / 3 5 6 . 9 1 6 / 4 0 3 . 5 1 2 / 6 3 . 1 4 0 ms

traceroute
traceroute .
UDP c TTL 1 30 (
) TTL (ICMP
TIME_EXCEEDED).
1

% traceroute informatics . by
traceroute to informatics . by ( 9 1 . 1 4 9 . 1 5 7 . 1 5 2 ) , 30 hops max , 60 byte packets
1 10.10.224.1 (10.10.224.1) 459.626 ms 679.630 ms 683.505 ms
2 * * *
3 77.74.33.114 (7 7.74.3 3.114) 687.141 ms 1246.943 ms 1250.856 ms
4 as6697 -2. by - ix . net ( 1 9 5 . 1 3 7 . 1 8 0 . 2 5 3 ) 1254.780 ms 1282.746 ms 1287.441
ms
5 gi15 -0 -0. ne1 -1 -2. net . belpak . by (93.84.125.9) 1287.400 ms 1290.625 ms
1294.567 ms
6 gw2 . datacenter . beltelecom . by (9 3 .8 4. 1 25 .1 6 5) 1294.530 ms 255.403 ms
327.064 ms
7 vh24 . hoster . by ( 9 1 . 1 4 9 . 1 5 7 . 1 5 2 ) 334.976 ms 386.862 ms 762.791 ms

129

route
IP-.
1

% route add - net 192.56.76.0 netmask 255.255.255.0 dev eth0

netstat
, ,
, ..

% netstat - nr
Kernel IP routing table
Destination
Gateway
10.6.6.6
0.0.0.0
169.254.0.0
0.0.0.0
0.0.0.0
10.6.6.6

Genmask
2 5 5. 25 5 .2 5 5. 25 5
255.255.0.0
0.0.0.0

Flags
UH
U
UG

MSS
0
0
0

Window
0
0
0

irtt
0
0
0

Iface
ppp0
ppp0
ppp0

ifconfig
ifconfig .
, ifconfig . ,
; -a,
, . .
netcat
TCP UDP, ,
. cat /.
tcpdump
.
.
Listing 4.5: tcpdump

% tcpdump -i lo
tcpdump : verbose output suppressed , use -v or - vv for full protocol decode
listening on lo , link - type EN10MB ( Ethernet ) , capture size 65535 bytes
2 2: 02 : 04 .0 2 19 05 IP localhost . localdomain .43849 > localhost . localdomain .6600:
Flags [ P .] , seq 2463307991:2463307998 , ack 2463842528 , win 770 , options [
nop , nop , TS val 12929880 ecr 12929629] , length 7
2 2: 02 : 04 .0 2 28 45 IP localhost . localdomain .6600 > localhost . localdomain .43849:
Flags [ P .] , seq 1:156 , ack 7 , win 512 , options [ nop , nop , TS val 12929880
ecr 12929880] , length 155
2 2: 02 : 04 .0 2 28 78 IP localhost . localdomain .43849 > localhost . localdomain .6600:
Flags [.] , ack 156 , win 770 , options [ nop , nop , TS val 12929880 ecr
12929880] , length 0
^C
3 packets captured
6 packets received by filter
0 packets dropped by kernel

130

tcpreplay
, tcpdump.
.
iperf
TCP UDP . TCP, UDP. ,
.
sysctl
, ,
. , .
/etc/sysctl.conf, .

[1] . , TCP/IP. , 2001.
[2] . . , . , . . , UNIX. .
, 3- ., 2007.
[3] . . , . . , TCP/IP, 3.
/ Linux/POSIX. , 2002.
[4] W. Richard Stevens, TCP/IP Illustrated. Volume 1. The protocols. Addison-Wesley, 1994.

131

5.1

- . , . Haskell :
data Color = R
| B
deriving Show
5

data Tree e = E
| T Color ( Tree e ) e ( Tree e )
deriving Show

,
- ,
. :
1. .
2. .
3. .
4. .

5.1.1

. , :
empty :: Tree e
empty = E
3

member :: Ord e = > e -> Tree e -> Bool

member x E = False
member x ( T _ a y b ) | x < y = member x a
| x == y = True
| x > y = member x b

132

133
z

d
y

. 5.1:
, 5.1, .
5.1 ,
.
balance. , ,
.

balance
balance
balance
balance
balance
balance

:: Color -> Tree e -> e -> Tree

B (T R (T R a x b) y c) z d = T
B (T R a x (T R b y c)) z d = T
B a x (T R b y (T R c z d)) = T
B a x (T R (T R b y c) z d) = T
color a x b = T color a x b

e
R
R
R
R

->
(T
(T
(T
(T

Tree e
B a x b)
B a x b)
B a x b)
B a x b)

y
y
y
y

(T
(T
(T
(T

B
B
B
B

c
c
c
c

z
z
z
z

d)
d)
d)
d)

insert : , ,
, balance,
.
insert :: Ord e = > e -> Tree e -> Tree e

134

insert x t = makeBlack ( insert t )

where insert E = T R E x E
insert ( T color a y b ) | x
| x
| x
makeBlack ( T _ a x b ) = T B

< y = balance color ( insert a ) y b

== y = T color a y b
> y = balance color a y ( insert b )
a x b

, .
fromList :: Ord e = > [ e ] -> Tree e
fromList = foldl ( flip insert ) E
3

main :: IO ()
main = do
putStrLn $ show $ fromList [9 , 8 , 7 , 4 , 5 , 6 , 3 , 2 , 1 , 0]

, :
T B (T B (T R (T B E 0 E) 1 (T B E 2 E)) 3 (T B E 4 E)) 5 (T B (T B (T R E 6
E) 7 E) 8 (T B E 9 E))

5.2

(trie) ,
, , , . ,
, .
, .

.

t
o

i
n

te
a
tea

in
n

d
ted

ten

inn

, . , O(m), m
.

5.2.1

Data.Map,
, Data.Maybe:

135

import Prelude hiding ( lookup , null )

import qualified Data . Map as Map
import qualified Data . Maybe as Maybe

:
1

data Trie k v = Trie ( Maybe v ) ( Map . Map k ( Trie k v ) )

deriving Show

, :
empty :: Ord k = > Trie k v
empty = Trie Nothing Map . empty
3

null :: Ord k = > Trie k v -> Bool

null ( Trie Nothing children ) = Map . null children
null _
= False

member lookup.

lookup :: Ord k = > [ k ] -> Trie k v -> Maybe v

lookup [] ( Trie value _ )
= value
lookup ( k : ks ) ( Trie _ children ) = do
next <- Map . lookup k children
lookup ks next
member :: Ord k = > [ k ] -> Trie k v -> Bool
member k = Maybe . isJust . lookup k

- insert:

insert :: Ord k = > [ k ] -> v -> Trie k v -> Trie k v

insert [] value ( Trie _ children ) = Trie ( Just value ) children
insert ( k : ks ) value ( Trie v children ) =
let next = Map . f in dW i th De fau l t empty k children
node = insert ks value next in
Trie v $ Map . insert k node children

-, , .
fromList :: Ord k = > [([ k ] , v ) ] -> Trie k v
fromList = foldl (\ t (k , v ) -> insert k v t ) empty
4

main :: IO
main = do
let keys
let trie
putStrLn

()
= [ " tea " , " ted " , " ten " , " inn " , " te " , " " ]
= fromList [ (k , True ) | k <- keys ]
$ show trie

main :
Trie ( Just True ) ( fromList [( i , Trie Nothing ( fromList [( n , Trie Nothing (
fromList [( n , Trie ( Just True ) ( fromList []) ) ]) ) ]) ) ,( t , Trie Nothing (
fromList [( e , Trie ( Just True ) ( fromList [( a , Trie ( Just True ) (
fromList []) ) ,( d , Trie ( Just True ) ( fromList []) ) ,( n , Trie ( Just True )
( fromList []) ) ]) ) ]) ) ])

- :

136

delete :: Ord k = > [ k ] -> Trie k v -> Trie k v

delete [] ( Trie _ children ) = Trie Nothing children
delete ( k : ks ) trie@ ( Trie value children ) =
case Map . lookup k children of
Just next ->
let node = delete ks next in
if null node
then Trie value $ Map . delete k children
else Trie value $ Map . insert k node children
Nothing
-> trie

, , delete.
deleteAll :: Ord k = > [[ k ]] -> Trie k v -> Trie k v
deleteAll keys trie = foldl ( flip delete ) trie keys

main :: IO
main = do
let keys
let trie
putStrLn

()
= [ " tea " , " ted " , " ten " , " inn " , " te " , " " ]
= fromList [ (k , True ) | k <- keys ]
$ show $ deleteAll keys trie

, :
Trie Nothing ( fromList [])

5.3

(Bloom filter ) , . ,
, .
, , .

. .
m - h1 , ..., hk , . . -
.
e
h1 (e), ..., hk (e).
, h1 (e), ..., hk (e). , . , .

5.3.1

- , - , m . , - -,
:
1

1
.
m

137

, -, :

1
1
m

k
.

n , , :

nk
1
.
1
m
, :

nk
1
1 1
.
m
, . :
f=

nk !k

k
1
= 1 enk/m ,
1 1
m

1 . ,
m n .
, k -, m n,
f k. f
:
nk/m
)
f = ek ln(1e
.
g = k ln(1 enk/m ). :
g
nk enk/m
= ln(1 enk/m ) +
k
m 1 enk/m
g k = ln 2 (m/n) ' 0.6931 m
n . f = (1/2)k ' 0.6185m/n .

5.3.2

class BloomFilter ( object ) :

def __init__ ( self , size , hashers ) :
assert size > 0 ,
" size must be greater than zero "
assert len ( hashers ) , " at least one hash function must be provided "
self . size
= size
self . hashers = tuple ( hashers )
self . bitset = 0 L

def add ( self , item ) :

for hasher in self . hashers :
position = hasher ( item ) % self . size
1 lim

1 x
x

138

self . bitset |= 1 << position

def contains ( self , item ) :

for hasher in self . hashers :
position = hasher ( item ) % self . size
if not ( self . bitset & (1 << position ) ) :
return False
return True

def repr ( self ) :

return self . __str__ ()
24

def str ( self ) :

binary = bin ( self . bitset )
binary = binary [2:]
binary = 0 * ( self . size - len ( binary ) ) + binary
return binary

:
1

>>> bf = BloomFilter (64 , [ hash ])

>>> bf . add (20)
>>> bf
0000000000000000000000000000000000000000000100000000000000000000
>>> bf . add (34)
>>> bf
0000000000000000000000000000010000000000000100000000000000000000
>>> 34 in bf
True
>>> 20 in bf
True
>>> 21 in bf
False

5.4

B-

B- , , .
.
B- - , -. , B- , B- .
B-, - , O(lg n),
, - , .

5.4.1

B-

B- t , :
1. 2t .
2. , , t .
3. , .
4. , h.
5. k k 1 .

139

. ,
. , 2t 1 .

5.4.2

, t 1 . ,
, ,
.
open Batteries
2

type btree = { degree : int ; tree : tree }

and tree =
| Node of int list * tree list
| Leaf of int list
7

let create degree =

if degree < 2
then raise ( I n v a l i d _ a r g u m en t " degree < 2 " )
else { degree = degree ; tree = Leaf [] }
12

let find btree key =

let rec find = function
| Leaf [] -> None
| Leaf keys ->
if List . mem key keys then Some key else None
| Node ( keys , children ) ->
if List . mem key keys then Some key
else let rec move_right ks cs =
match ks , cs with
| [] , [ c ] -> find c
| k :: ks , c :: cs ->
if k > key then find c else move_right ks cs
| _ -> raise ( Failure " Hot we got here ?! " )
in move_right keys children
in find btree . tree
let is_full tree degree = match tree with
| Node ( keys , _ ) | Leaf keys ->
List . length keys = 2 * degree - 1

let s pl it _ at _c e nt e r list =
let center = ( List . length list ) / 2 in
let left , right = List . split_at center list in
let middle , right = List . hd right , List . tl right in
left , middle , right

let split_leaf = function

| Leaf keys -> let left , middle , right = sp li t _a t_ c en te r keys in
Node ([ middle ] , [ Leaf left ; Leaf right ])
| _ -> raise ( I n v a l i d _ a r g u m e n t " leaf " )

let split_node = function

| Node ( keys , children ) ->
let left , middle , right = sp l it _ at _c e nt er keys in
let i = ( List . length left ) + 1 in
let left_children , r ight_c hildre n = List . split_at i children in
Node ([ middle ] , [ Node ( left , left_children ) ;
Node ( right , right_ childr en ) ])

140

| _ -> raise ( I n v a l i d _ a r g um e n t " node " )

let split_full tree =

match tree with
| Leaf _ -> split_leaf tree
| Node _ -> split_node tree
57

let split_with key = List . partition ( fun k -> key > k )

5.5

let rec insert btree key =

let rec insert tree =
match tree with
| Leaf [] -> Leaf [ key ]
| Leaf keys ->
if is_full tree btree . degree
then let tree = split_full tree in insert tree
else let left , right = split_with key keys in
Leaf ( left @ key :: right )
| Node ( keys , children ) ->
if is_full tree btree . degree
then let tree = split_full tree in insert tree
else let left_keys , right_keys = split_with key keys in
let i = List . length left_keys in
let left_children , r ight_c hildre n = List . split_at i children in
let next = List . hd right_ childr en in
let right_ childr en = List . tl rig ht_chi ldren in
if is_full next btree . degree
then match split_full next with
| Node ([ median ] , [ left ; right ]) ->
let left , right =
if key <= median then ( insert left , right )
else ( left , insert right ) in
Node ( left_keys @ median :: right_keys ,
left_children @ left :: right :: r ight_c hildre n )
| _ -> raise ( Failure " How we got here ?! " )
else Node ( keys , left_children @ ( insert next ) ::
rig ht_chi ldren )
in { btree with tree = insert btree . tree }

-, , , , . , ,
.
- ,
-, / .
,

. - .
-
(static hash tables), , , .
-,

141

(extendible hashing).
:
1. . 2,
.
2. :
, .
3. - h k , k
(, 32).
d, . , n = 2d
d k.
m, , , -,
B.
, .
key
1. seq = h(key).
2. d .
3. d seq. i , .
4. B i- .
5. bkey, m seq,
(d + 1).
6. B bkey,
.
(key, data)
1. B, i.
2. B , (key, data) B.
.
3. , m < d, .
:
(a) .
(b) , B, ,
(d + m + 1)- -.
- B, .

142

(c) B: m = m + 1.
(d) , ,
B, B
d + m- -.
4. m = d, d.
. .
5. .
key
1. , key.
2. , .
3. .
4.
2m , :
(a) Q.
(b) . , ,
, .
(c) m = m 1 .
(d) .
(e) Q.
5. , :
(a) .
(b) .

[1] . , . , . , . , : . , 2- ., 2005.
[2] . -, . . , . , : .
, 2003.
[3] C. Okasaki, Purely functional data structures. Cambridge University Press, 1999.
[4] C. Okasaki, Red-black tree in a functional setting. Journal of Functional Programming,
Volume 9 Issue 4, July 1999.
[5] A. Broder, M. Metzenmacher, Network applications of Bloom filters: a survey. 2005.
[6] R. Fagin, J. Hievergelt, N. Peppenger, H. Raymond Strong, Extendible hashing a fast access
method for dynamic files. ACM Transactions on Database Systems 4 (3): 315344, 1979.

143

Common Lisp
, ,
:
;
;
;
;
;
;
( !);
, CLOS ;
.

6.1

DEFUN. :
( defun function-name ( parameter *)
" Optional documentation string "
body *)

, , .
.
,
, . , , , (keyword) .
, . , , ( ) DOCUMENTATION.

144

6. COMMON LISP

145

DEFUN CL. , , .
RETURN-FROM.
( defun verbose-sum ( x y )
" Sum any two numbers after printing a message . "
( format t " Summing ~ d and ~ d .~% " x y )
(+ x y ) ) ( defun foo ()
5

( documentation verbose-sum function ) = > " Sum any two numbers after printing
a message . "

6.1.1

.

( ), . ,
.
.
, , CL .

. .
( defun foo ( a b & optional c d )
( list a b c d ) )
4

( foo 1 2)
= > (1 2 NIL NIL )
( foo 1 2 3)
= > (1 2 3 NIL )
( foo 1 2 3 4) = > (1 2 3 4)

,
NIL.
, . , .
( defun foo ( a & optional ( b 10) )
( list a b ) )

( foo 1 2) = > (1 2)
( foo 1)
= > (1 10)

, , , .
( defun make -recta ngle ( width & optional ( height width ) )
...)

,
. , , .

6. COMMON LISP

146

, NIL
.
( defun foo ( a b & optional ( c 3 c-supplied-p ) )
( list a b c c-supplied-p ) )

( foo 1 2)
= > (1 2 3 NIL )
( foo 1 2 3) = > (1 2 3 T )
( foo 1 2 4) = > (1 2 4 T )

(rest)
,
, .
, .
( defun + (& rest numbers ) ...)

, , ,
, &key
.
( defun foo (& key a b c )
( list a b c ) )

( foo )
( foo : a
( foo : b
( foo : c
( foo : a
( foo : a
( foo : a

=>
1)
=>
1)
=>
1)
=>
1 : c 3)
=>
1 : b 2 : c 3) = >
1 : c 3 : b 2) = >

( NIL NIL NIL )

(1 NIL NIL )
( NIL 1 NIL )
( NIL NIL 1)
(1 NIL 3)
(1 2 3)
(1 2 3)

, supplied-p-.
, ,
, .
( defun foo (& key ( a 0) ( b 0 b-supplied-p ) ( c (+ a b ) ) )
( list a b c b-supplied-p ) )

( foo
( foo
( foo
( foo

:a
:b
:b
:a

1)
1)
1 : c 4)
2 : b 1 : c 4)

=>
=>
=>
=>

(1
(0
(0
(2

0
1
1
1

1
1
4
4

NIL )
T)
T)
T)

, , , ,
, , .
( defun foo (& key ((: apple a ) ) ((: box b ) 0) ((: charlie c ) 0 c-supplied-p ) )
( list a b c c-supplied-p ) )
3

( foo : apple 10 : box 20 : charlie 30) = > (10 20 30 T )

6. COMMON LISP

147

,
. ,
, : , , , .
, , . .
.
1

( defun foo ( x & optional y & key z )

( list x y z ) )
( foo 1 2 : z 3) = > (1 2 3)
( foo 1)
= > (1 nil nil )
( foo 1 : z 3)
= > ERROR
( defun foo (& rest rest & key a b c )
( list rest a b c ) )

( foo : a 1 : b 2 : c 3)

6.1.2

= > ((: A 1 : B 2 : C 3) 1 2 3)

, . .
RETURN-FROM,
.

( defun foo ( n )
( dotimes ( i 10)
( dotimes ( j 10)
( when ( > (* i j ) n )
( return-from foo ( list i j ) ) ) ) ) )

6.1.3

CL . DEFUN :
1. -.
2. , -.
- FUNCTION #.
( defun foo ( x ) (* 2 x ) )
( function foo ) = > # < FUNCTION FOO >
# foo
= > # < FUNCTION FOO >

- , FUNCALL APPLY. , , .
FUNCALL , , ,

6. COMMON LISP

148

. FUNCALL -,
- .
FUNCALL. -
, , min max step.
( defun plot ( fn min max step )
( loop for i from min to max by step do
( loop repeat ( funcall fn i ) do ( format t " * " ) )
( format t " ~% " ) ) )
5

CL-USER > ( plot # exp 0 4 1/2)

*
*
**
****
*******
************
********************
*********************************
******************************************************
NIL

APPLY -,
. APPLY - .
( apply # plot (# exp 0 4 1/2) )
( apply # plot # exp (0 4 1/2) )

6.1.4

LAMBDA.
( lambda ( parameters ) body )
( funcall # ( lambda ( x y ) (+ x y ) ) 2 3) = > 5
(( lambda ( x y ) (+ x y ) ) 2 3)
=> 5

6.2

CL : ().
CL . ,
,
.
, , LET.
:
( let ( variable *)
body-form *)

variable ,
, ( NIL).
( let (( x 10) ( y 20) z )
( list x y z ) )

= > (10 20 NIL )

6. COMMON LISP

149

, .
. body-form*.

( defun foo ( x )
( format t " Parameter : ~ a ~% " x )
( let (( x 2) )
( format t " External LET : ~ a ~% " x )
( let (( x 3) )
( format t " Internal LET : ~ a ~% " x ) )
( format t " External LET : ~ a ~% " x ) )
( format t " Parameter : ~ a ~% " x ) )
CL-USER > ( foo
Parameter : 1
External LET :
Internal LET :
External LET :
Parameter : 1
NIL

1)
2
3
2

LET: LET*. ,
LET LET, LET*
, .

6.2.1

( let (( count
( list
# ( lambda
# ( lambda
# ( lambda

( funcall
( funcall
( funcall
( funcall

0) )
() ( incf count ) )
() ( decf count ) )
() count ) ) )

( first * count-fns *) )
( first * count-fns *) )
( second * count-fns *) )
( third * count-fns *) )

=>
=>
=>
=>

1
2
1
1

(closure), , LET. , . ,
, .

6.2.2

()

CL : DEFVAR DEFPARAMETER.
, DEFVAR ,
DEFPARAMETER . ,
.
, .
( defvar
* count *
( defvar
* count *

* count * 0 " Number of created widgets so far . " )

=> 0
* count * 1)
=> 0

( defparameter * count * 2)
* count * = > 2

6. COMMON LISP

150

DEFVAR . (unbound).
-
, , , , ,
( ).
( defun hello-world ()
( format * st a nd ar d -o ut p ut * " Hello , world !~% " ) )

CL-USER > ( hello-world )

Hello , world !
NIL
( wi th-ope n-file ( stream # p " / tmp / scream . txt " : direction : output )
( let ((* st an d ar d- o ut pu t * stream ) )
( hello-world ) ) )
CL-USER > ( hello-world )
Hello , world !
NIL

/tmp/scream.txt :
1

% cat scream . txt

Hello , world !

(). , , ,
, . , .

6.2.3

DEFCONSTANT.
DEFCONSTANT DEFPARAMETER.
( defconstant name i n i t i a l - v a l u e - f o r m [ d o c u m e n t a t i o n - s t r i n g ])

.
.

6.2.4

, SETF, CL :
( setf place value )
SETF (expand) , place. place , SETQ,
, . SETF
.
( defvar * x *)
( defvar * y *)

( setf * x * 10) = > 10

( setf * x * 1 y 2) = > 2
( setf * x * ( setf * y * ( random 10) ) ) = > 6

6. COMMON LISP

151

. SETF
(place). CL , , .
( let (( x 1) )
( setf x 10)
x ) = > 10
5

( let (( x (2 2 3) ) )
( setf ( first x ) 1)
x ) = > (1 2 3)
( let (( x #(2 2 3) ) )
( setf ( aref x 0) 1)
x ) = > #(1 2 3)
( let (( x ( m ak e -h as h -t a bl e ) ) )
( setf ( gethash key x ) 10)
( gethash key x ) ) = > 10

DEFSETF DEFINE-SETF-EXPANDER.

6.3

, S-
CL.
- (embedded domain-specific languages).
Lisp , Lisp,
, , , ,
, , Lisp,
.
,
. ,
, , , . :
, . , , ,
.
, WHEN :
( defmacro when ( condition & rest body )
( if , condition ( progn , @body ) ) )

, FOO:
( defun foo ( x )
( when ( > x 10) ( print big ) ) )

, condition (> x 10), (print big)

, body. :
( if ( > x 10) ( progn ( print big ) ) )

DEFMACRO :

6. COMMON LISP

152

( defmacro name ( parameter *)

" Optional documentation string . "
body-form *)

6.3.1

.
.
-. .
, &body &rest. &body &rest .
Listing 6.1:
( defun primep ( number )
( when ( > number 1)
( loop for fac from 2 to ( isqrt number ) never ( zerop ( mod number fac ) ) ) ) )
5

( defun next-prime ( number )

( loop for n from number when ( primep n ) return n ) )
( defmacro do-primes (( var start end ) & body body )
( do (( , var ( next-prime , start ) ( next-prime (1+ , var ) ) ) )
(( > , var , end ) )
, @body ) )

6.3.2

,
() , ,
at (@).
. at ,
, .
, . ,
, .
6.1:

(a
(a
(a
(a
(a

(list a (+ 1 2) c)
(list a (+ 1 2) c)
(list a (list 1 2) c)
(list a (list 1 2) c)
(append (list a) (list 1 2))

(a
(a
(a
(a
(a

(+ 1 2) c)
,(+ 1 2) c)
(list 1 2) c)
,(list 1 2) c)
,@(list 1 2))

(+ 1 2) c)
3 c)
(list 1 2) c)
(1 2) c)
1 2 c)

: MACROEXPAND-1
MACROEXPAND-1 -

MACROEXPAND.

6. COMMON LISP

153

. MACROEXPAND .

6.3.3

.
, DO-PRIMES (random 100):
( do-primes ( p 0 ( random 100) )
( format t " ~ d " p ) )

MACROEXPAND-1, , (random
DO:

100)

CL-USER > ( macroexpand-1 ( do-primes ( p 0 ( random 100) ) ( format t " ~ d " p ) ) )

( DO (( P ( NEXT-PRIME 0) ( NEXT-PRIME (1+ P ) ) ) )
(( > P ( RANDOM 100) ) )
( FORMAT T " ~ d " P ) )
T

end
.

( defmacro do-primes (( var start end ) & body body )

( do (( ending-value , end )
( , var ( next-prime , start ) ( next-prime (1+ , var ) ) ) )
(( > , var ending-value ) )
, @body ) )

.
, ,
, end, ,
start. .
DO

( defmacro do-primes (( var start end ) & body body )

( do (( , var ( next-prime , start ) ( next-prime (1+ , var ) ) )
( ending-value , end ) )
(( > , var ending-value ) )
, @body ) )

ending-value. ,
.
do-primes - :
( do-primes ( ending-value 0 10)
( print ending-value ) )

MACROEXPAND-1 :
( do (( ending-value ( next-prime 0) ( next-prime (1+ ending-value ) ) )
( ending-value 10) )
(( > ending-value ending-value ) )
( print ending-value ) )

. GENSYM .
- . , GENSYM, .

6. COMMON LISP

154

( defmacro do-primes (( var start end ) & body body )

( let (( e n d i n g - v a l u e - n a m e ( gensym ) ) )
( do (( , var ( next-prime , start ) ( next-prime (1+ , var ) ) )
( , e n d i n g - v a l u e - n a me , end ) )
(( > , var , e n d i n g - v a l u e - n a m e ) )
, @body ) ) )

( do (( ending-value ( next-prime 0) ( next-prime (1+ ending-value ) ) )

(#: g2141 10) )
(( > ending-value #: g2141 ) )
( print ending-value ) )

6.3.4

GENSYM:

( defmacro do-primes (( var start end ) & body body )

( with-gensyms ( e n d i n g - v a l u e - n a m e )
( do (( , var ( next-prime , start ) ( next-prime (1+ , var ) ) )
( , e n d i n g - v a l u e - n a me , end ) )
(( > , var , e n d i n g - v a l u e - n a m e ) )
, @body ) ) )

DO-PRIMES, .
( defmacro with-gensyms ((& rest names ) & body body )
( let ,( loop for n in names collect ( , n ( gensym ) ) )
, @body ) )

, loop-:
CL-USER > ( loop for n in ( a b c ) collect ( , n ( gensym ) ) )
(( A ( GENSYM ) ) ( B ( GENSYM ) ) ( C ( GENSYM ) ) )

6.4

6.4.1

CL :
, -;
;
;
;
.

:
CL-USER > ( list +1 -1 2/1 3/2 6/4 # xA # b01001 # b1010 /1011 # o777 #36 rABFG 1.)
(1 -1 2 3/2 3/2 10 9 10/11 511 481372 1)

6. COMMON LISP

155

, . , 10. #B #b,
. #O #o ,
#X #x .
( 2 36) #nR, n (
).
A-Z a-z.
:
CL-USER > ( list 1.0 1 e0 1 d0 0.1 .1 1 e-3 )
(1.0 1.0 1.0 d0 0.1 0.1 0.001)

CL : short, single, double long. , ,

. : s, f, d, l short,
single, double long. e , .
:

CL-USER > ( list # c (2 1) # c (2/3 3/4) # c (2 1.0) # c (2.0 1.0 d0 )

# c (1/2 1.0) # c (3 0) # c (3.0 0.0) # c (1/2 0) # c (-6/3 0) )
s
(# C (2 1) # C (2/3 3/4) # C (2.0 1.0) # C (2.0 d0 1.0 d0 )
# C (0.5 1.0) 3 # C (3.0 0.0) 1/2 -2)

: #C #c, , .

(+)
(+ 1 2 3)
(- 2)
(*)
(/ 10 5 2)

=>
=>
=>
=>
=>

0
6
-2
1
1

(+
(((*
(/

1 2)
5 4)
10 3 5)
2 3 4)
2 3)

=>
=>
=>
=>
=>

3
1
2
24
2/3

(+ # c (1 2) 3) = > # c (4 2)

CL
( ) :
1. FLOOR ,
, .
2. CEILING ,
, .
3. TRUNCATE , FLOOR , CEILING .
4. ROUND .
,
.
MOD REM .
FLOOR TRUNCATE :
(+ (* ( floor
(/ x y ) ) y ) ( mod x y ) ) == x
(+ (* ( truncate (/ x y ) ) y ) ( rem x y ) ) == x

6. COMMON LISP

156

= ,
, . <, >, <= >=
.
(= 1 1.0 # c (1.0 0.0) # c (1 0) )

(/= 1 1)
= > NIL
(/= 1 2 3)
=> T
( <= 2 3 3 4) = > T

=> T

(/= 1 2)
=> T
(/= 1 2 3 1.0) = > NIL
( <= 2 3 4 3)
= > NIL

: ZEROP, MINUSP, PLUSP,

MAX.

EVENP, ODDP, MIN

6.4.2

CL , . : #\, . #\ .
:
#\ .
Lisp, Space Newline. , Tab, Page, Rubout, Linefeed, Return
Backspace.
, , : CHAR=
, CHAR=/, CHAR<, CHAR>, CHAR<= CHAR>=. , ,
: CHAR-EQUAL, CHAR-NOT-EQUAL, CHAR-LESSP, CHAR-GREATERP, CHAR-NOT-GREATERP
CHAR-NOT-LESSP.

6.4.3

. ,
. , ,
(") (\).
.
, , : STRING=,
STRING=/, STRING<, STRING>, STRING<= STRING>=. , , : STRING-EQUAL, STRING-NOT-EQUAL, STRING-LESSP, STRING-GREATERP, STRINGNOT-GREATERP STRING-NOT-LESSP. ,
.
: :start1, :end1, :start2
:end2.
( string = " foobarbaz " " quuxbarfoo " : start1 3 : end1 6 : start2 4 : end2 7) = > T

6.5

CL : -. : , .
,
, , .

6. COMMON LISP

6.5.1

157

. .
VECTOR #(...). , , , .
( vector )
= > #()
( vector 1)
= > #(1)
( vector 1 2) = > #(1 2)

MAKE-ARRAY . , ,
. . .
:initial-element , .
( make-array 5 : i n it ia l -e l em en t nil ) = > #( NIL NIL NIL NIL NIL )

(fill pointer).
:fill-pointer.
( make-array 5 : fill-pointer 0) = > #()
( make-array 5 : fill-pointer 3) = > #(0 0 0)
( make-array 5 : fill-pointer 3 : in it i al -e l em e nt nil ) = > #( NIL NIL NIL )

:adjustable.
, 5 ,
.
( make-array 5 : fill-pointer 0 : adjustable t ) == > #()

( defparameter * array * ( make-array 2 : fill-pointer 0) )

( vector-push 0 * array *) = > 0
( vector-push 1 * array *) = > 1
( vector-push 2 * array *) = > NIL
* array * = > #(0 1)
( defparameter * array *
( make-array 2 : fill-pointer 1 : in it i al -e l em en t 0 : adjustable t ) )
( vector-push 1 * array *) = > 1
( vector-push 2 * array *) = > NIL
* array * = > #(0 1)
( v e c t o r - p u s h - e x t e n d 2 * array *) = > 2
* array * = > #(0 1 2)

.
, , :
element-type.
( make-array 5 : fill-pointer 0 : adjustable t : element-type character ) = > " "
( make-array 3 : i n i t i a l - c o n t e n t s (0 1 0) : element-type bit ) = > #*010

6.5.2

6. COMMON LISP

158

6.2:

COUNT
FIND
POSITION

REMOVE

NIL

NIL
,
,

SUBSTITUTE

6.3:

:test

, ( , :key) .
, . NIL
.
()
.
(n 1) . NIL .
, .
, , NIL
( REMOVE SUBSTITUTE).

EQL

:key

:start
:end

:from-end
:count

NIL

0
NIL

NIL
NIL

Listing 6.2:

( length #(1 2 3) )
( elt 1 #(1 2 3) )
( count 1 #(1 2 1 2 3 1 2 3 4) )
( remove 1 #(1 2 1 2 3 1 2 3 4) )
( remove 1 (1 2 1 2 3 1 2 3 4) )
( remove #\ a " foobarbaz " )
( substitute 10 1 #(1 2 1 2 3 1 2 3 4) )
( substitute 10 1 (1 2 1 2 3 1 2 3 4) )
( substitute #\ x #\ b " foobarbaz " )
( find 1 #(1 2 1 2 3 1 2 3 4) )
( find 10 #(1 2 1 2 3 1 2 3 4) )

=>
=>
=>
=>
=>
=>
=>
=>
=>
=>
=>

3
2
3
#(2 2 3 2 3 4)
(2 2 3 2 3 4)
" foobrbz "
#(10 2 10 2 3 10 2 3 4)
(10 2 10 2 3 10 2 3 4)
" fooxarxaz "
1
NIL

6. COMMON LISP

( position 2 #(1 2 1 2 3 1 2 3 4) )
( position 9 #(1 2 1 2 3 1 2 3 4) )

159
=> 1
= > NIL

( count " foo " #( " foo " " bar " " baz " ) : test # string =)
=> 1
( find c #(( a 10) ( b 20) ( c 30) ( d 40) ) : key # first ) = > ( C 30)
18

( find a #(( a 10) ( b 20) ( a 30) ( b 40) ) : key # first )

= > ( A 10)
( find a #(( a 10) ( b 20) ( a 30) ( b 40) ) : key # first : from-end t ) = > ( A 30)
( remove #\ a " foobarbaz " : count 1)
= > " foobrbaz "
( remove #\ a " foobarbaz " : count 1 : from-end t ) = > " foobarbz "

CL , , , ,
. ,
, -IF. , , , - .
, -IF-NOT,
, .
Listing 6.3:
( count-if # evenp #(1 2 3 4 5) )
=> 2
( count-if-not # evenp #(1 2 3 4 5) )
=> 3
( position-if # digit-char-p " abcd0001 " ) = > 4
5

( remove-if-not # ( lambda ( x ) ( char = ( elt x 0) #\ f ) )

#( " foo " " bar " " baz " " foom " ) ) = > #( " foo " " foom " )
( count-if # evenp #((1 a ) (2 b ) (3 c ) (4 d ) (5 e ) ) : key # first )
=> 2
( count-if-not # evenp #((1 a ) (2 b ) (3 c ) (4 d ) (5 e ) ) : key # first ) = > 3

( remove-if-not # alpha-char-p
#( " foo " " bar " " 1 baz " ) : key # ( lambda ( x ) ( elt x 0) ) ) = > #( " foo " " bar " )

6.5.3

- MAKE-HASH-TABLE. : :test. : EQ,

EQL, EQUAL, EQUALP, .
GETHASH -.
( defparameter * h * ( m a ke -h a sh- ta bl e ) )
( gethash foo * h *) = > NIL
( setf ( gethash foo * h *) quux )
( gethash foo * h *) = > QUUX
GETHASH
MULTIPLE-VALUE-BIND.

( defun show-value ( key hash-table )

( m u l t i p l e - v a l u e - b i n d ( value present ) ( gethash key hash-table )
( if present
( format nil " ~ a is present " value )
( format nil " ~ a is not present " value ) ) ) )

6. COMMON LISP

160

Listing 6.4: -
( defparameter * h * ( m a ke -h a sh- ta bl e ) )
( setf ( gethash key1 * h *) value1 )
( setf ( gethash key2 * h *) value2 )
5

CL-USER > ( maphash # ( lambda ( k v ) ( format t " ~ a = > ~ a ~% " k v ) ) * h *)

KEY1 = > VALUE1
KEY2 = > VALUE2
NIL
( loop for k being the hash-keys in * h * using ( hash-value v )
do ( format t " ~ a = > ~ a ~% " k v ) )

6.6

CL ,
, .
.

6.6.1

Cons-

cons-. Cons- ( CONS,

) . ,
() , , .
( cons 1 nil )
= > (1)
( cons 1 ( cons 2 nil ) )
= > (1 2)
( cons 1 ( cons 2 ( cons 3 nil ) ) ) = > (1 2 3)

CAR CDR,
.

( defparameter * cons * ( cons 1 2) )

( car * cons *)
=> 1
( cdr * cons *)
=> 2
( setf ( car * cons *) 10) = > 10
* cons *
= > (10 . 2)
( setf ( cdr * cons *) 20) = > 20
* cons *
= > (10 . 20)

CAR CDR : FIRST REST.

6.6.2

LIST. APPEND ,
.
( defparameter * list-1 * ( list 1 2) )
( defparameter * list-2 * ( list 3 4) )
( defparameter * list-3 * ( append * list-1 * * list-2 *) )
5

( setf ( first * list-2 *) 0)

* list-2 * = > (0 4)
* list-3 * = > (1 2 0 4)

6. COMMON LISP

161

NCONC , , cons-
, .
CL SECOND TENTH, .
NTH : , n-
( ) . NTHCDR,
n- REST .
28 CAR/CDR .
4 A D C R. A
CAR, D CDR.
6.4:

LAST

cons- . n, n .
cons-.
n, n .
BUTLAST, ,
.
cons-.
T, cons-,
.
, , CDR cons- , LIST APPEND.
n . NIL
:initial-element.
REVERSE APPEND: ,
REVERSE . .
; ,
NREVERSE . .
, cons-.
, cons-.
, cons- NIL
, NIL.

BUTLAST
NBUTLAST
LDIFF
TAILP
LIST*

MAKE-LIST
REVAPPEND

NRECONC

CONSP
ATOM
LISTP
NULL

: REVERSE, NREVERSE, PUSH, POP,

PUSHNEW, DELETE, DELETE-IF, DELETE-IF-NOT, DELETE-DUPLICATED, NSUBSTITUTE, NSUBSTITUTE-IF NSUBSTITUTE
-IF-NOT.

6.6.3

MAP, . MAPCAR, MAPLIST, MAPCAN,

MAPCON, MAPC MAPL.

6. COMMON LISP

162

( mapcar # ( lambda ( x ) (* 2 x ) ) ( list 1 2 3) ) = > (2 4 6)

( mapcar # + ( list 1 2 3) ( list 10 20 30) )
= > (11 22 33)
( maplist # ( lambda ( x ) x ) ( list 1 2 3) ) = > ((1 2 3) (2 3) (3) )
5

( mapcan # ( lambda ( x ) ( and ( numberp x ) ( list x ) ) )

( a 1 b c 3 4 d 5) )
= > (1 3 4 5)
( mapcon # list (1 2 3 4) ) = > ((1 2 3 4) (2 3 4) (3 4) (4) )

6.6.4

, ,, ,
. , ((nil1nil)2(nil3nil))
,
2, 1 3.
CL .
COPY-TREE, , COPY-LIST, , .
TREE-EQUAL , ,
.
SUBST . SUBSTIF, NSUBST NSUBST-IF .
( subst 10 1 (1 2 (3 2 1) ((1 1) (2 2) ) ) ) = > (10 2 (3 2 10) ((10 10) (2 2) ) )

Listing 6.5:

( defun preorder ( node f )

( when node
( funcall f ( first node ) )
( preorder ( second node ) f )
( preorder ( third node ) f ) ) )
( defun inorder ( node f )
( when node
( inorder ( second node ) f )
( funcall f ( first node ) )
( inorder ( third node ) f ) ) )
( defun postorder ( node f )
( when node
( postorder ( second node ) f )
( postorder ( third node ) f )
( funcall f ( first node ) ) ) )
( defun level-order ( node f )
( loop with level = ( list node )
while level
do
( setf level ( loop for node in level
when node
do ( funcall f ( first node ) )
and collect ( second node )
and collect ( third node ) ) ) ) )
( defparameter * tree * (1 (2 (4 (7) )
(5) )
(3 (6 (8)

6. COMMON LISP

163
(9) ) ) ) )

( defun show ( t r a v e r s a l - f u n c t i o n )
( format t " ~&~(~ A ~) :~12 ,0 T " t r a v e r s a l - f u n c t i o n )
( funcall t r a v e r s a l - f u n c t i o n * tree * ( lambda ( value ) ( format t " ~ A " value ) ) )
)
( map nil # show ( preorder inorder postorder level-order ) )

preorder :
inorder :
postorder :
level - order :

6.6.5

1
7
7
1

2
4
4
2

4
2
2
3

7
5
5
4

5
1
1
5

3
8
8
6

6
6
6
7

8
9
9
8

9
3
3
9

CL , . ADJOIN PUSHNEW.
( defparameter * set * () )
( adjoin 1 * set *)
* set * = > NIL
5

( setf * set * ( adjoin 1 * set *) )

( pushnew 2 * set *)
* set * = > (2 1)
( pushnew 2 * set *)
* set * = > (2 1)

MEMBER, MEMBER-IF MEMBER-IF-NOT , .

, , ,
cons-, . ,
NIL.
INTERSECTION, UNION, SET-DIFFERENCE SET-DIFFERENCE-OR-EXCLUSIVE , . , N,
.
SUBSETP , .

6.6.6

, , . /, ,
, .
, ,
cons-.
(( a . 1) ( b . 2) ( c . 3) )

ASSOC ( ASSOC-IF ASSOC

-IF-NOT).

( assoc c (( a . 1) ( b . 2) ( c . 3) ) ) = > 3
( assoc d (( a . 1) ( b . 2) ( c . 3) ) ) = > NIL
( cdr ( assoc a (( a . 1) ( b . 2) ( c . 3) ) ) ) = > 1
( assoc " a " (( " a " . 1) ( " b " . 2) ( " c " . 3) ) : test # string =) = > ( " a " . 1)
( assoc a (( a . 10) ( a . 1) ( b . 2) ( c . 3) ) ) = > ( A . 10)

6. COMMON LISP

164

RASSOC, RASSOC-IF RASSOC-IF-NOT .

ACONS.
( acons new-key new-value alist )
( setf alist ( acons new-key new-value alist ) )
( push ( cons new-key new-value ) alist )

COPY-ALIST.
PAIRLIS,
.

6.6.7

Destructuring-bind

DESTRUCTURING-BIND
.
( d e s t r u c t u r i n g - b i n d ( parameter *) list
body-form *)

Listing 6.6:
( d e s t r u c t u r i n g - b i n d ( x y z ) ( list 1 2 3)
( list : x x : y y : z z ) ) = > (: X 1 : Y 2 : Z 3)
3

( d e s t r u c t u r i n g - b i n d ( x y z ) ( list 1 ( list 2 20) 3)

( list : x x : y y : z z ) ) = > (: X 1 : Y (2 20) : Z 3)

( d e s t r u c t u r i n g - b i n d ( x ( y1 y2 ) z ) ( list 1 ( list 2 20) 3)

( list : x x : y1 y1 : y2 y2 : z z ) ) = > (: X 1 : Y1 2 : Y2 20 : Z 3)
( d e s t r u c t u r i n g - b i n d ( x ( y1 & optional y2 ) z ) ( list 1 ( list 2 20) 3)
( list : x x : y1 y1 : y2 y2 : z z ) ) = > (: X 1 : Y1 2 : Y2 20 : Z 3)

( d e s t r u c t u r i n g - b i n d ( x ( y1 & optional y2 ) z ) ( list 1 ( list 2) 3)

( list : x x : y1 y1 : y2 y2 : z z ) ) = > (: X 1 : Y1 2 : Y2 NIL : Z 3)
( d e s t r u c t u r i n g - b i n d (& key x y z ) ( list : x 1 : y 2 : z 3)
( list : x x : y y : z z ) ) = > (: X 1 : Y 2 : Z 3)

( d e s t r u c t u r i n g - b i n d (& key x y z ) ( list : z 1 : y 2 : x 3)

( list : x x : y y : z z ) ) = > (: X 3 : Y 2 : Z 1)

6.7

( d e s t r u c t u r i n g - b i n d (& whole whole & key x y z ) ( list : z 1 : y 2 : x 3)

( list : x x : y y : z z : whole whole ) )
= > (: X 3 : Y 2 : Z 1 : WHOLE (: Z 1 : Y 2 : X 3) )

Loop

LOOP - (embedded domainspecific language, eDSL) . :

;
, , , ;
Lisp;

6. COMMON LISP

165

;
;
;
.

6.7.1

LOOP (clause), .
:

( loop for i from 1 to 10

while ( < i 7)
collect (* i i )
do ( format t " Working on ~ D now ~% " i )
when ( evenp i )
do ( format t " ~ D is non-odd number ~% " i )
finally ( format t " About to exit ...~% " ) )
Working on 1 now
Working on 2 now
2 is non - odd number
Working on 3 now
Working on 4 now
4 is non - odd number
Working on 5 now
Working on 6 now
6 is non - odd number
About to exit ...

, ,
. ,
(prepositions), , from to.

, . ,

return, go throw. :
,
.
initially progn.
.
finally.

:

6. COMMON LISP

166

for as .
, , , ,
. , and,
.
with let. for,
with .
. - .

collect , , . ,
.
append , , . , (. APPEND).
nconc , , . , (. nconc).
sum ,
. ,
.
count , . , .
minimize . ,
.
maximize . ,
.

loop-finish , . finally, .
for as , .
repeat .
while , . (if (not condition)(loop-finish)).
until , . (if condition (loop-finish)).
always , , NIL. , T
.

6. COMMON LISP

167

never ,
, NIL. , T
.
thereis ,
, .

do .
return , . do (return value).

if , ,
. , ;
and.
when if.
unless , when,
, .
else if, when
unless.
,
else.
end .

named loop.
, initially,
.
, finally, . finally.

LOOP

. .
. - .
. .
, .
:
return.
RETURN, RETURN-FROM .

6. COMMON LISP

168

always, nerver thereis

RETURN RETURN-FROM .
.

LOOP :

initial - final ::= initially | finally

variables ::= with | initial - final | for - as | repeat
main ::= unconditional | accumulation | conditional
| termination | initial - final
loop ::= ( loop [ named name ] { variables }* { main }*)

6.7.2

for- :
;
;
, cons-;
, , : ;
- -;
;
.
loop for-, ,
.
( loop
for item in (1 2 3)
for i from 1 to 4
collect i )

= > (1 2 3)

Listing 6.7:
1

( loop
( loop
( loop
( loop
( loop

for i upto 3 collect i )

for i from -1 downto -3 collect i )
for i from 1 to 3 collect i )
repeat 3 collect ( random 3) )
for i below 6 by 2 collect i )

=>
=>
=>
=>
=>

(0 1 2 3)
(-1 -2 -3)
(1 2 3)
(2 2 0)
(0 2 4)

6. COMMON LISP

169

( loop
( loop
( loop
( loop
( loop

for
for
for
for
for

i
i
x
x
x

in ( list 1 2 3 4) collect i ) = > (1 2 3 4)

in ( list 1 2 3 4) by # cddr collect i ) = > (1 3)
on ( list 1 2 3) collect x ) == > ((1 2 3) (2 3) (3) )
on ( list 1 2 3 4) by # cddr collect x ) == > ((1 2 3 4) (3 4) )
across " abcd " collect x ) == > (#\ a #\ b #\ c #\ d )

- , - , .
:
( loop for var being the things in ha sh - or - pa ck a ge ...)

- things hash-keys hash-values,

, var
-, . hash-or-package .
things symbols, present-symbols external-symbols.
var , , , , , .
- , ,
- using-.
( loop for k being the hash-keys in h using ( hash-value v ) ...)
( loop for v being the hash-values in h using ( hash-key k ) ...)
4

( length ( loop for s being the symbol in : cl-user collect s ) ) = > 1277

Equals-then
, , for ,
- , equals-then.
( loop for var = i n i t i a l - v a l u e - f o r m [ then step-form ] ...)

( loop repeat 5
for y = 1 then (+ x y )
for x = 0 then y
collect y ) = > (1 1 2 4 8)
( loop repeat 5
for x = 0 then y
and y = 1 then (+ x y )
collect y ) = > (1 1 2 3 5)

LOOP .
( loop for ( a b ) in ((1 2) (3 4) (5 6) )
append ( list a b ) )

= > (1 2 3 4 5 6)

( loop for ( a nil ) in ((1 2) (3 4) (5 6) ) collect a ) = > (1 3 5)

6. COMMON LISP

170

( defparameter * random * ( loop repeat 10 collect ( random 1000) ) )

( loop for i in * random *
counting ( evenp i ) into evens
counting ( oddp i ) into odds
summing i into total
maximizing i into max
minimizing i into min
finally ( return ( list min max total evens odds ) ) )
= > (34 950 5294 4 6)

do ( doing).
( loop for i from 1 to 10 do ( print i ) )

do, , ,
RETURN-FROM.

RETURN

( block outer
( loop for i from 0 return 100)
( print " This will print " )
200) = > 200
5

( block outer
( loop for i from 0 do ( return-from outer 100) )
( print " This won t print " )
200) = > 100

do , , , IF
WHEN .
( loop for i from 1 to 10 do ( when ( evenp i ) ( print i ) ) )

do , , ,
, - , :
( loop for i from 1 to 10 when ( evenp i ) sum i ) = > 30

, . ,
it.
( loop for key in some-list when ( gethash key some-hash ) collect it )

else-. and. , , :
( loop for i from 1 to 100
if ( evenp i )
minimize i into min-even and
maximize i into max-even and

6. COMMON LISP

171

unless ( zerop ( mod i 4) )

sum i into e v e n - n o t - f o u r s - t o t a l
end
and sum i into even-total
else
minimize i into min-odd and
maximize i into max-odd and
when ( zerop ( mod i 5) )
sum i into fives-total
end
and sum i into odd-total
do ( u pd at e -a na l ys is min-even
max-even
min-odd
max-odd
even-total
odd-total
fives-total
even-not-fours-total ))

6.8

Format

FORMAT (). ,
( loop for cons on list
do ( format t " ~ a " ( car cons ) )
when ( cdr cons ) do ( format t " , " ) )

( format t " ~{~ a ~^ , ~} " list )

FORMAT : ,
. :
T *STANDARD-OUTPUT*.
NIL. FORMAT .
, .
. .

6.8.1

(~) ( ),
. (
).
, , ,
.
:
v FORMAT
.
# .

6. COMMON LISP

( format
( format
( format
( format
( format

nil
nil
nil
nil
nil

" ~ $ " pi )
" ~5 $ " pi )
" ~ v$ " 3 pi )
" ~# $ " pi )
" ~ ,5 f " pi )

=>
=>
=>
=>
=>

172

3.14
3.14159
3.142
3.1
3.14159

@, .
( format
( format
( format
( format

nil
nil
nil
nil

" ~ d " 1000000)

" ~: d " 1000000)
" ~ @d " 1000000)
" ~: @d " 1000000)

=>
=>
=>
=>

1000000
1 ,000 ,000
+1000000
+1 ,000 ,000

~A . ~S
,
READ.
( format nil " The value is : ~ a " 10)
= > " The value is : 10 "
( format nil " The value is : ~ a " " foo " )
= > " The value is : foo "
( format nil " The value is : ~ a " ( list 1 2 3) ) = > " The value is : (1 2 3) "

~% ~& , ~% ,
~& .

( format
( format
( format
( format

nil
nil
nil
nil

" ~ c " #\ a )
" ~: c " #\ Space )
" ~ @c ~% " #\ a )
" ~: @c " ( code-char 0) )

=>
=>
=>
=>

"a"
" Space "
" #\ a "
" ^ @ ( Control @ ) "

( format nil " ~ d " 1000000)

= > " 1000000 "
( format nil " ~12 d " 1000000)
= > " 1000000 "
( format nil " ~12 , 0 d " 1000000) = > " 000001000000 "
10

( format nil " ~4 , 0 d- ~2 , 0 d- ~2 , 0 d " 2005 6 10) = > " 2005-06-10 "
( format nil " ~: d " 100000000)
= > " 100 ,000 ,000 "
( format nil " ~ , , . ,4: d " 100000000) = > " 1.0000.0000 "

( format
( format
( format
( format

nil
nil
nil
nil

" ~ x " 1000000) = > " f4240 "

" ~ o " 1000000) = > " 3641100 "
" ~ b " 1000000) = > " 1 1 1 1 0 1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 "
" ~3 r " 1000000) == > " 1212210202001 "

~F , , , . ~F ,
. ~E, , .
( format
( format
( format
( format
5

nil
nil
nil
nil

" ~ f " pi )
" ~ ,4 f " pi )
" ~ e " pi )
" ~ ,4 e " pi )

=>
=>
=>
=>

" 3 . 1 4 1 5 9 2 6 5 3 5 8 9 7 9 3 d0 "
" 3.1416 "
" 3 . 1 4 1 5 9 2 6 5 3 5 8 9 7 9 3 d +0 "
" 3.1416 d +0 "

6. COMMON LISP

173

( format nil " ~ $ " pi ) == > " 3.14 "

( format nil " ~2 ,4 $ " pi ) == > " 0003.14 "

~R
.
( format
( format
( format
( format

nil
nil
nil
nil

" ~ r " 1234)

" ~: r " 1234)
" ~ @r " 1234)
" ~: @r " 1234)

=>
=>
=>
=>

" one thousand two hundred thirty-four "

" one thousand two hundred thirty-fourth "
" MCCXXXIV "
" MCCXXXIIII "

~P s, 1.
( format nil " file ~ p " 1) = > " file "
( format nil " file ~ p " 10) = > " files "
( format nil " file ~ p " 0) = > " files "
5

( format nil " ~ r file ~: p " 1) = > " one file "
( format nil " ~ r file ~: p " 10) = > " ten files "
( format nil " ~ r file ~: p " 0) = > " zero files "
( format nil " ~ r famil ~: @p " 1) = > " one family "
( format nil " ~ r famil ~: @p " 10) = > " ten families "
( format nil " ~ r famil ~: @p " 0) = > " zero families "

~).
( format
( format
( format
( format

6.8.2

nil
nil
nil
nil

" ~(~ a ~) " " tHe Quick BROWN foX " )

" ~ @ (~ a ~) " " tHe Quick BROWN foX " )
" ~:(~ a ~) " p " tHe Quick BROWN foX " )
" ~: @ (~ a ~) " " tHe Quick BROWN foX " )

=>
=>
=>
=>

" the quick brown fox "

" The quick brown fox "
" The Quick Brown Fox "
" THE QUICK BROWN FOX "

~[ ~], , ~;. ~[ ,
FORMAT. , .
( format
( format
( format
( format

nil
nil
nil
nil

" ~[ zero ~; one ~; two ~] "

" ~[ zero ~; one ~; two ~] "
" ~[ zero ~; one ~; two ~] "
" ~[ zero ~; one ~; two ~] "

0)
1)
2)
3)

=>
=>
=>
=>

" zero "

" one "
" two "
""

~:; ~;, .
( format nil " ~[ zero ~; one ~; two ~:; many ~] " 3)

= > " many "

~[ .
,
NIL, .
( format t " ~:[ FAIL ~; pass ~] " test-result )

6. COMMON LISP

6.8.3

174

~{ FORMAT .
( format nil " ~{~ a , ~} " ( list 1 2 3) )
=> "1,2,3,"
( format nil " ~{~ a ~^ , ~} " ( list 1 2 3) ) = > " 1 , 2 , 3 "
( format nil " ~ @ {~ a ~^ , ~} " 1 2 3)
=> "1,2,3"

.
( , ~@{) ,
~:{.

6.8.4

~* .
. ( ),
.
( format nil " ~ r ~:*(~ d ) " 1) = > " one (1) "

~{ ~* .
( format nil " ~{~ s ~*~^ ~} " (: a 10 : b 20) ) = > " : A : B "

~* .
, , . @
, ,
( 0).

6.9

CLOS

CLOS Common Lisp Object System ( Common Lisp). CLOS . .

. , NUMBER STRING
, . , , ,
.
/ . , ( ). ,
. Common Lisp
T, ( ) . , Common Lisp . Common
Lisp .

6.9.1

( ),
, .

6. COMMON LISP

175
bank-account

checking-account

savings-account

. 6.1:
DEFGENERIC DEFUN, , .
DEFGENERIC ,
, .
( defgeneric withdraw ( account amount )
(: documentation " Withdraw the specified amount from the account .
Signal an error if the current balance is less than amount . " ) )

( ) , . :
( EQL).
, (&rest)
(&key, &allow-other-keys) , .
, . ,
, .
( defmethod withdraw (( account bank-account ) amount )
( when ( < ( balance account ) amount )
( error " Account overdrawn . " ) )
( decf ( balance account ) amount ) )

CALL-NEXT-METHOD
. ,
, , bank-account.
, ,
.
, .

( defmethod withdraw (( account c h e c k i n g - a c c o u n t ) amount )

( let (( overdraft (- amount ( balance account ) ) ) )
( when ( plusp overdraft )
( withdraw ( o v e r d r a f t - a c c o u n t account ) overdraft )
( incf ( balance account ) overdraft ) ) )
( call-next-method ))

, EQL-,
, DEFMETHOD.
Listing 6.8: EQL-

( defmethod withdraw (( account ( eql * a c c o u n t - o f - b a n k - p r e s i d e n t *) ) amount )

( let (( overdraft (- amount ( balance account ) ) ) )
( when ( plusp overdraft )
( incf ( balance account ) ( embezzle * bank * overdraft ) ) )
( call-next-method )))

6. COMMON LISP

6.9.2

176

,
. :
1. , .
2. . (
:argument-precedence-order DEFGENERIC).
.
3. ,
.
, , . , , . , checking-account
, ,
bank-account.
EQL- , . EQL ,
EQL-.
.

, . . ,
, ,
CALL-NEXT-METHOD.
( , , ).
: :before, :after :around. :
( defmethod withdraw : before (( account bank-account ) amount ) ...)

. :before ( ) .
, .

( defmethod withdraw : before (( account c h e c k i n g - a c c o u n t ) amount )

( let (( overdraft (- amount ( balance account ) ) ) )
( when ( plusp overdraft )
( withdraw ( o v e r d r a f t - a c c o u n t account ) overdraft )
( incf ( balance account ) overdraft ) ) ) )

:after , . :around ,
.
:

6. COMMON LISP

177

:around , .
.
:around CALL-NEXT-METHOD.
, - . CALL-NEXTMETHOD, , NO-NEXT-METHOD.
NEXT-METHOD-P ,
.
:around CALL-NEXT-METHOD, :
around . :around
:around ,
:
:before , . . CALL-NEXT-METHOD
.
. CALL-NEXT-METHOD . ,
- . CALL-NEXT-METHOD, , NO-NEXT-METHOD. NEXT-METHOD-P ,
. CALL-NEXT-METHOD ,
.
:after , . . CALL-NEXT-METHOD
.
:around , , . CALL-NEXT-METHOD,
:around ,
.
, , .
Listing 6.9:
; ;; integer = > rational = > real = > number = > t
( defgeneric speak ( arg ) )

( defmethod speak (( arg number ) )

( format t " number : primary method ~% " ) )
( defmethod speak (( arg integer ) )
( format t " integer : primary method ~% " )
( call-next-method ))

( defmethod speak : before (( arg number ) )

( format t " number : before method ~% " ) )

( defmethod speak : before (( arg integer ) )

( format t " integer : before method ~% " ) )

6. COMMON LISP

178

( defmethod speak : after (( arg number ) )

( format t " number : after method ~% " ) )
20

( defmethod speak : after (( arg integer ) )

( format t " integer : after method ~% " ) )
( defmethod speak : around (( arg number ) )
( format t " number : around method ~% " )
( call-next-method ))
( defmethod speak : around (( arg integer ) )
( format t " integer : around method ~% " )
( call-next-method ))

CL-USER > ( speak 1)

integer : around method
number : around method
integer : before method
number : before method
integer : primary method
number : primary method
number : after method
integer : after method
NIL

CL 9 : +, and, append, list,
progn. :
, ,
,
. :around .
, , + ,
, . , , and or ,
, .
, , .
, :method-combination .
max, min, nconc, or

Listing 6.10:
( defgeneric priority ( job )
(: documentation " Return the priority at which the job should be run . " )
(: m e t h o d - c o m b i n a t i o n + : m o s t - s p e c i f i c - l a s t ) )
5

( defmethod priority + (( job express-job ) ) 10)

. :
:around . , :around
. CALL-NEXT-METHOD NO-NEXT-METHOD-P.
, . CALL-NEXT-METHOD NO-NEXTMETHOD-P.

6. COMMON LISP

179

, DEFINE-METHOD-COMBINATION
.

6.9.3

, T.
: . , , INTEGER, STRING LIST.
, , .
DEFCLASS. DEFCLASS
, .
: , (
).
( defclass name ( d i r e c t - s u p e r c l a s s - n a m e *)
( sl ot-spe cifier *) )

,
. MAKE-INSTANCE.
, STANDARD-OBJECT.
STANDARD-OBJECT T.
( defclass bank-account () ...)
( defclass c h e c k i n g - a c c o u n t ( bank-account ) ...)
( defclass s av i ng s- a cc ou n t ( bank-account ) ...)

6.9.4

DEFCLASS . .
SLOT-VALUE SETF
SLOT-VALUE.
.
.
( defclass bank-account ()
( customer-name
balance ) )
5

( defparameter * account * ( make-instance bank-account ) ) = > * ACCOUNT *

( setf ( slot-value * account * customer-name ) " John Doe " ) = > " John Doe "
( setf ( slot-value * account * balance ) 1000)
= > 1000

6.9.5

.
:initarg ,
MAKE-INSTANCE. :initform , , MAKE-INSTANCE
:initarg.
( defclass bank-account ()
(( customer-name
: initarg : customer-name )

6. COMMON LISP

180

( balance
: initarg : balance
: initform 0) ) )
( defparameter * account *
( make-instance bank-account : customer-name " John Doe " : balance 1000) )

( slot-value * account * customer-name ) = > " John Doe "

( slot-value * account * balance )
= > 1000

( slot-value ( make-instance bank-account ) balance )

=> 0
( slot-value ( make-instance bank-account ) customer-name ) = > error

:initarg :initform .
( defvar * ac c ou nt - nu m be rs * 0)

INITIALIZE-INSTANCE. , STANDARD-OBJECT , :initarg :initform. :after

( defclass bank-account ()
(( customer-name
: initarg : customer-name
: initform ( error " Must supply a customer name . " ) )
( balance
: initarg : balance
: initform 0)
( ac count- number
: initform ( incf * a cc ou n t- nu m be rs *) )
account-type ) )
( defmethod i n i t i a l i z e - i n s t a n c e : after (( account bank-account )
& key o p e n i n g - b o n u s - p e r c e n t a g e )
( when o p e n i n g - b o n u s - p e r c e n t a g e
( incf ( slot-value account balance )
(* ( slot-value account balance ) (/ o p e n i n g - b o n u s - p e r c e n t a g e 100) ) )
)
( let (( balance ( slot-value account balance ) ) )
( setf ( slot-value account account-type )
( cond
(( >= balance 100000) : gold )
(( >= balance 50000) : silver )
( t : bronze ) ) ) ) )
( defparameter * acct * ( make-instance
bank-account
: customer-name " Sally Sue "
: balance 1000
: o p e n i n g - b o n u s - p e r c e n t a g e 5) )
( slot-value * acct * balance ) = > 1050

6. COMMON LISP

6.9.6

181

SLOT-VALUE . ,
, , .
:reader ,
. DEFCLASS , , , , . ,

( balance
: initarg : balance
: initform 0
: reader balance )

:
1

( defgeneric customer-name ( account ) )

( defmethod customer-name (( account bank-account ) )
( slot-value account customer-name ) )

( defgeneric ( setf customer-name ) ( value account ) )

( defmethod ( setf customer-name ) ( value ( account bank-account ) )
( setf ( slot-value account customer-name ) value ) )

:writer . SETF-,
, .

( customer-name
: initarg : customer-name
: initform ( error " Must supply a customer name . " )
: reader customer-name
: writer ( setf customer-name ) )

, :accessor.
( customer-name
: initarg : customer-name
: initform ( error " Must supply a customer name . " )
: accessor customer-name )

:
1

( defclass bank-account ()
(( customer-name
: initarg : customer-name
: initform ( error " Must supply a customer name . " )
: accessor customer-name
: documentation " Customer s name " )
( balance
: initarg : balance
: initform 0
: reader balance
: documentation " Current account balance " )
( ac count- number
: initform ( incf * a cc ou n t- nu m be rs *)
: reader acc ount-n umber
: documentation " Account number , unique within a bank . " )

6. COMMON LISP

182

( account-type
: reader account-type
: documentation " Type of account , one of : gold , : silver , or : bronze . " ) ) )

CL : WITH-SLOTS
WITH-ACCESSORS.
( with-slots ( slot *) instance-form
body-form *)

slot ,
, , ,
, .
, SLOTVALUE . :
( defmethod a s s e s s - l o w - b a l a n c e - p e n a l t y (( account bank-account ) )
( when ( < ( slot-value account balance ) * m in im u m- ba l an ce *)
( decf ( slot-value account balance ) (* ( slot-value account balance ) .01)
)))
5

( defmethod a s s e s s - l o w - b a l a n c e - p e n a l t y (( account bank-account ) )

( with-slots ( balance ) account
( when ( < balance * m in i mu m -b al a nc e *)
( decf balance (* balance .01) ) ) ) )

:accessor, WITH-ACCESSORS, ,
SLOT-VALUE.
( defmethod a s s e s s - l o w - b a l a n c e - p e n a l t y (( account bank-account ) )
( wi th-acc essors (( balance balance ) ) account
( when ( < balance * m in i mu m -b al a nc e *)
( decf balance (* balance .01) ) ) ) )

6.9.7

, , :allocation.
:instance ( ), :class. , .
. ,
, .

6.9.8

CL . ,
.

. .
,
:initform, .
:initarg , .

6. COMMON LISP

183

:reader, :writer :accessor

, , ,
. .
:allocation .
( defclass foo ()
(( a : initarg : a : initform " A " : accessor a )
( b : initarg : b : initform " B " : accessor b ) ) )
5

( defclass bar ( foo )

(( a : initform ( error " Must supply a value for a " ) )
( b : initarg : the-b : accessor the-b : allocation : class ) ) )

6.9.9

, . CL ,

DEFCLASS: , ,
, , . ,

( defclass m o n e y - m a r k e t - a c c o u n t ( c h e c k i n g - a c c o u n t s av in g s- ac c ou n t ) () )

( money-market-account
checking-account
s av in g s- a cc ou n t
bank-account
s ta nd a rd - ob je c t
t)

, ,
money-market-account bank-account.

6.10

, :
(property list). , .
. , .
. , .
. ,
. ,
.

6.10.1

,
(property list, plist). ,

6. COMMON LISP

184

( ), , , , ( ).
.
. , .
SYMBOL-PLIST.
GET GETF SETF. :
( defun get ( symbol indicator & optional default ) ...)
( defun getf ( place indicator & optional default ) ...)
( get x y ) == ( getf ( symbol-plist x ) y )

Listing 6.11:
( defvar foo ( list : bar t : baz 3 : hunoz " Huh ? " ) )
2

( getf foo : bar )

=> T
( getf foo : baz )
=> 3
( getf foo : hunoz ) = > " Huh ? "
7

( setf ( getf foo : baz ) 4)

( getf foo : baz ) = > 4

GET-PROPERTIES .
( defun get- proper ties ( place indica tor-li st ) ...)

REMPROP REMF.
( defun remprop ( symbol indicator ) ...)
( defun remf ( place indicator ) ...)
( remprop x y ) == ( remf ( symbol-plist x ) y )

Listing 6.12:
( defvar foo ( list color blue height 6.3 near-to bar ) )
2

( remf foo height ) = > ( COLOR BLUE NEAR-TO BAR )

6.10.2

.
SYMBOL-NAME.
( symbol-name xyz ) = > " XYZ "

6.10.3

.
, , .
. , READ
, .
, .
#:.
INTERN. MAKE-SYMBOL
. COPY-SYMBOL

6. COMMON LISP

185

. .
.
( defun make-symbol ( print-name ) ...)
( defun copy-symbol ( sym & optional copy-props ) ...)

GENSYM .
( defun gensym (& optional x ) ...)

*GENSYM-COUNTER*
X, ( ). G
X, ( ).
( gensym ) = > G7
( gensym " FOO- " ) = > FOO-8
( gensym 32) = > FOO-32
( gensym ) = > FOO-33

, , SYMBOL-PACKAGE.
, KEYWORD. , . ,
, KEYWORDP.
( keywordp : sym ) = > T
( eql : foo : foo ) = > T

6.11

. FIND-PACKAGE .
FIND-SYMBOL. ,
, , *PACKAGE*,
.
READ - , INTERN. ,
.
, package:symbol
package::symbol. package, .
, . ,
FIND-SYMBOL, .
. , . , .
, .
, , .
, ,
. .
.
.

. ,
.

6. COMMON LISP

6.11.1

186

( defpackage : email-app
(: use : common-lisp : db : network )
(: shadow : build-index )
(: s h a d o w i n g - i m p o r t - f r o m : network : save )
(: import-from : email : p a r s e - e m a i l - a d d r e s s )
(: export : make-email
: send-email
: save-email )
(: nicknames : eapp ) )

:use . :shadow
, :use.
, ,
:use. :shadowing-import-from ,
, . :export .
:nicknames .
.
:
1. :shadow :shadowing-import-from

2. :use
3. :import-from :intern
4. :export

6.12

CL , . ; , , .
CL , (), , ,
.
, , ( ) CONDITION. STANDARD-OBJECT.
, DEFCLASS, .
, SLOT-VALUE, :reader :accessor.
INITIALIZE-INSTANCE.
MAKE-CONDITION ( MAKE-INSTANCE).
ERROR.
( d e f i n e - c o n d i t i o n m a l f o r m e d - l o g - e n t r y - e r r o r ( error )
(( text : initarg : text : reader text ) ) )

6.12.1

, . -
,
. ERROR,

6. COMMON LISP

187

SIGNAL.
, . ERROR ,
.
( defun p a rs e- l og -e n tr y ( text )
( if ( w e l l - f o r m e d - l o g - e n t r y - p text )
( make-instance log-entry ...)
( error m a l f o r m e d - l o g - e n t r y - e r r o r : text text ) ) )

,
, .

.
, , .
.
, SIGNAL
, .
.
, , . HANDLER-CASE.
( handler-case expression
( co nditio n-type ([ var ]) code ) *)

expression , HANDLER .
, condition-type,
, HANDLER-CASE.
CASE. code

( defun pars e-log- file ( file )

( wi th-ope n-file ( in file : direction : input )
( loop for text = ( read-line in nil nil ) while text
for entry = ( handler-case ( p ar se - lo g -e nt r y text )
( m a l f o r m e d - l o g - e n t r y - e r r o r () nil ) )
when entry collect it ) ) )

6.12.2

PARSE-LOG-FILE .
, , , , ,
.
, HANDLER-CASE RESTART
-CASE. RESTART-CASE HANDLER-CASE , , .

( defun pars e-log- file ( file )

( wi th-ope n-file ( in file : direction : input )
( loop for text = ( read-line in nil nil ) while text
for entry = ( restart-case ( p ar se - lo g -e nt r y text )
( sk ip-log -entry () nil ) )
when entry collect it ) ) )

,
.

6. COMMON LISP

188

skip-log-entry. , PARSE-LOG-FILE
, ,
NIL.
. , .
( defun log-analyzer ()
( dolist ( log ( find-all-logs ) )
( analyze-log log ) ) )
5

( defun analyze-log ( log )

( dolist ( entry ( par se-log -file log ) )
( analyze-entry entry ) ) )

HANDLER
. HANDLERCASE HANDLER-BIND .
INVOKE-RESTART.
-BIND.

( defun log-analyzer ()
( handler-bind (( m a l f o r m e d - l o g - e n t r y - e r r o r
# ( lambda ( c )
( in voke-r estart sk ip-log -entry ) ) ) )
( dolist ( log ( find-all-logs ) )
( analyze-log log ) ) ) )

INVOKE-RESTART , CONTROL-ERROR. ,
:
( lambda ( c )
( let (( restart ( find-restart skip- log-en try ) ) )
( when restart ( invo ke-res tart restart ) ) ) )

6.12.3

, - /
.
, INVOKE-RESTART.

( defun p a rs e- l og -e n tr y ( text )
( if ( w e l l - f o r m e d - l o g - e n t r y - p text )
( make-instance log-entry ...)
( restart-case ( error m a l f o r m e d - l o g - e n t r y - e r r o r : text text )
( use-value ( value ) value )
( reparse-entry ( fixed-text ) ( pa r se -l o g- e nt ry fixed-text ) ) ) ) )
( defun log-analyzer ()
( handler-bind (( m a l f o r m e d - l o g - e n t r y - e r r o r
# ( lambda ( c )
( use-value
( make-instance m a l f o r m e d - l o g - e n t r y : text ( text c ) ) ) ) )
)
( dolist ( log ( find-all-logs ) )
( analyze-log log ) ) ) )

6. COMMON LISP

6.12.4

189

, .
SIGNAL , ;
,
SIGNAL NIL.
ERROR SIGNAL. , HANDLER-CASE , SIGNAL . SIGNAL , ERROR
, *DEBUGGER-HOOK*. , ERROR
, , .
WARN , . WARN
SIGNAL, , WARN
, *ERROR-OUTPUT* NIL. WARN MUFFLE-WARNING,
. MUFFLE-WARNING
, CONTROL-ERROR,
.

( defun count-down ( x )
( do (( counter x (1- counter ) ) )
((= counter 0) done )
( when (= counter 1)
( warn " Almost done " ) )
( format t " ~&~ D ~% " counter ) ) )
CL-USER > ( count-down 3)
3
2
Warning : Almost done
1
DONE
( defun i g n o r e - w a r n i n g s - w h i l e - c o u n t i n g ( x )
( handler-bind (( warning # ignore -warni ng ) )
( count-down x ) ) )
( defun igno re-war ning ( condition )
( declare ( ignore condition ) )
( mu ffle-w arning ) )
CL-USER > ( i g n o r e - w a r n i n g s - w h i l e - c o u n t i n g 3)
3
2
1
DONE

CERROR , . ERROR, CERROR . WARN, CERROR

CONTINUE SIGNLA. CONTINUE CERROR MUFFLE-WARNING
WARN.

6.13

6. COMMON LISP

190

Listing 6.13:

( defun comma-split ( string )

( loop : for start := 0 : then (1+ finish )
: for finish := ( position #\ , string : start start )
: collecting ( subseq string start finish )
: until ( null finish ) ) )
( defun w r i t e - w i t h - p e r i o d s ( strings )
( format t " ~{~ A ~^.~} " strings )

Listing 6.14:

( defun t o p o l o g i c a l - s o r t ( graph & key ( test eql ) )

" Graph is an association list whose keys are objects and whose
values are lists of objects on which the corresponding key depends .
Test is used to compare elements , and should be a suitable test for
hash-tables . T o p o l o g i c a l - s or t returns two values . The first is a
list of objects sorted toplogically . The second is a boolean
indicating whether all of the objects in the input graph are present
in the topological ordering ( i . e . , the first value ) . "
( let (( entries ( ma ke - ha s h- ta b le : test test ) ) )
( flet (( entry ( vertex )
" Return the entry for vertex . Each entry is a cons whose
car is the number of outstanding dependencies of vertex
and whose cdr is a list of dependants of vertex . "
( m u l t i p l e - v a l u e - b i n d ( entry presentp ) ( gethash vertex entries )
( if presentp entry
( setf ( gethash vertex entries ) ( cons 0 () ) ) ) ) ) )
; ; populate entries initially
( dolist ( vertex graph )
( d e s t r u c t u r i n g - b i n d ( vertex & rest dependencies ) vertex
( let (( ventry ( entry vertex ) ) )
( dolist ( dependency dependencies )
( let (( dentry ( entry dependency ) ) )
( unless ( funcall test dependency vertex )
( incf ( car ventry ) )
( push vertex ( cdr dentry ) ) ) ) ) ) ) )
; ; L is the list of sorted elements , and S the set of vertices
; ; with no outstanding dependencies .
( let (( L () )
( S ( loop for entry being each hash-value of entries
using ( hash-key vertex )
when ( zerop ( car entry ) ) collect vertex ) ) )
; ; Until there are no vertices with no outstanding dependencies ,
; ; process vertices from S , adding them to L .
( do * () (( endp S ) )
( let * (( v ( pop S ) ) ( ventry ( entry v ) ) )
( remhash v entries )
( dolist ( dependant ( cdr ventry ) ( push v L ) )
( when ( zerop ( decf ( car ( entry dependant ) ) ) )
( push dependant S ) ) ) ) )
; ; return (1) the list of sorted items , (2) whether all items
; ; were sorted , and (3) if there were unsorted vertices , the
; ; hash table mapping these vertices to their dependants
( let (( all-sorted-p ( zerop ( h a s h - t a b l e - c o u n t entries ) ) ) )
( values ( nreverse L )
all-sorted-p
( unless all-sorted-p
entries ) ) ) ) ) )
> ( defparameter * d e p e n d e n c y -g r a p h *

6. COMMON LISP

(( d es-sys tem-li b
ramlib ieee )
( dw01
( dw02
( dw03
( dw04
( dw05
( dw06
( dw07
( dware
( gtech
( ramlib
( std-cell-lib
( synopsys ) ) )
* DEPENDENCY-GRAPH *

191

std synopsys std-cell-lib des-s ystem- lib dw02 dw01

ieee dw01 dware gtech )
ieee dw02 dware )
std synopsys dware dw03 dw02 dw01 ieee gtech )
dw04 ieee dw01 dware gtech )
dw05 ieee dware )
dw06 ieee dware )
ieee dware )
ieee dware )
ieee gtech )
std ieee )
ieee std-cell-lib )

> ( t o p o l o g i c a l - s o r t * d e p e n d e n c y - g r a p h *)
( IEEE DWARE DW02 DW05 DW06 DW07 GTECH DW01 DW04 STD-CELL-LIB SYNOPSYS STD
DW03 RAMLIB D ES-SYS TEM-LI B )
T
NIL

Listing 6.15: -

( defun zigzag ( n )
( flet (( move ( i j )
( if ( < j (1- n ) )
( values ( max 0 (1- i ) ) (1+
( values (1+ i ) j ) ) ) )
( loop with a = ( make-array ( list n n )
with x = 0
with y = 0
for v from 0 below (* n n )
do ( setf ( aref a x y ) v )
( if ( evenp (+ x y ) )
( setf ( values x y ) ( move
( setf ( values y x ) ( move
finally ( return a ) ) ) )
0 1 5
2 4 7
3 8 12
9 11 17
10 18 19

6
13
16
20
23

j))
: element-type integer )

x y))
y x)))

14
15
21
22
24

Listing 6.16: loop

( defun sum- of-squ ares ( vector )
( loop for x across vector sum ( expt x 2) )

( defun s i e v e - o f - e r a t o s t h e n e s ( maximum )
( let (( composites ( make-array (1+ maximum ) : element-type bit
: in i ti al - el em e nt 0) ) )
( loop for candidate from 2 to maximum
when ( zerop ( bit composites candidate ) )
collect candidate
and do ( loop for composite from ( expt candidate 2) to maximum by
candidate
do ( setf ( bit composites composite ) 1) ) ) ) )

6. COMMON LISP

192

Listing 6.17: Quicksort

( defun quicksort ( list )

( if ( <= ( length list ) 1)
list
( let (( pivot ( first list ) ) )
( append ( quicksort ( remove-if-not # ( lambda ( x ) ( < x pivot ) ) list ) )
( remove-if-not # ( lambda ( x ) (= x pivot ) ) list )
( quicksort ( remove-if-not # ( lambda ( x ) ( > x pivot ) ) list ) ) ) ) ) )
( defun qs ( list )
( if ( < ( length list ) 2)
list
( macrolet (( pivot ( test ) ( remove ( first list ) list : test-not # , test ) )
)
( append ( qs ( pivot >) ) ( pivot =) ( qs ( pivot <) ) ) ) ) )

( defun quicksort ( sequence )

( labels (( swap ( a b ) ( rotatef ( elt sequence a ) ( elt sequence b ) ) )
( sub-sort ( left right )
( when ( < left right )
( let (( pivot ( elt sequence right ) )
( index left ) )
( loop for i from left below right
when ( <= ( elt sequence i ) pivot )
do ( swap i ( prog1 index ( incf index ) ) ) )
( swap right index )
( sub-sort left (1- index ) )
( sub-sort (1+ index ) right ) ) ) ) )
( sub-sort 0 (1- ( length sequence ) ) )
sequence ) )

[1] P. Seibel, Practical Common Lisp. Apress, 2005.
[2] G. Kiczales, J. des Rivieres, D. G. Bobrow, The Art of the Metaobject Protocol. The MIT
Press, 1991.
[3] P. Graham, ANSI Common Lisp. Prentice Hall, 1995.
[4] P. Graham, On Lisp: Advanced Techniques for Common Lisp. Prentice Hall, 1993.
[5] G. Steele, Common Lisp: The Language. Digital Press, 2nd ed., 1990.

193

7.1

z = sin(x)sin(y) (. . 7.1) , , , , z. z, : .

0.5
1
0
0

1
2

0.5

0
0

. 7.1: z = sin(x)sin(y)
:
, (, ). ,
:
j = j

J(),
j

J() .
.
.
194

195

Listing 7.1:
import operator
2

from numpy import *

def s in si n _g ra d ie n t ( vec ) :
x , y = vec
return array ([ cos ( x ) * sin ( y ) ,
sin ( x ) * cos ( y ) ])
def gdescent ( initial , gradient_fn , step_fn = operator . sub ,
maxiter =1000 , eps =0.01 , precision =0.00001) :

x = initial
for i in range ( maxiter ) :
old_x = x . copy ()
x = step_fn ( old_x , eps * gradient_fn ( x ) )
17

if sum (( x - old_x ) ** 2) < precision :

break
return x

[2; 1].

0.5
1
0
0

1
2

7.1.1

0.5

0
0

, ,
.
(2 )

(1000$)

120
90
...

4
3
...

157
120
...

196

-
. .
h (x) = 0 + 1 x1 + 2 x2 ,
i ( ), x0 = 1.
h(x) =

n
X

i xi = T x.

i=0

- i . h(x) . :
m

J() =

1X
(h (x(i) ) y (i) )2 .
2 i=1

, .
j :

1
(h (x) y)2
J() =
j
j 2
1

= 2 (h (x) y)
(h (x) y)
2
j
!
n
X

= (h (x) y)
i xi y
j i=0
= (h (x) y)xj .
, :
(i)
j = j (h (x)(i) y (i) )xj .
,
. m,
j = j

m
X
(i)
(h (x)(i) y (i) )xj .
i=1

, , .

. 0 1 .
def lg_gradient ( p ) :
def fn ( x ) :
return p [0] + p [1] * x
4

result = array ([0 for _ in range ( len ( p ) ) ])

for x , y in samples :

197

result += ( fn ( x ) - y ) * array ([1.0 , x ])

return result

>>> gdescent ( array ([20.0 , 0.1]) , lg_gradient , eps =0.00000001)

[ 19.99998493
0.06248649]

, 0 = 19.999 1 = 0.0624. price = 0 + 1 square . 7.2.

Price (in $1000)

400

300

200

100

0
0

1,000

2,000

3,000

4,000

Square (f t )
. 7.2: .

7.2

() . .

7.2.1

x(1) , . . . , x(m) , x(i) Rn . k , k . k

:
1. (1) , . . . , (k) , (i) Rn .
2. ( , ):
(a) i,
c(i) = arg min kx(i) j k2 .
j

(b) j,
Pm
(i)
= j}x(i)
i=1 1{c
j = P
.
m
(i) = j}
i=1 1{c

198

:
1. .
2. .
3. .
4. , 2.
:
.
.
.
,
.

7.2.2

(, )
- . , k . , .

d(p, q) p q. p =
(p1 , . . . , pn ) q = (q1 , . . . , qn ), :
v
u n
uX
d(p, q) = t (pk qk )2 .
k=1

, ,
, .
, .
, , , , ,
.
from math import sqrt
3

def sim_distance ( vec1 , vec2 ) :

assert len ( vec1 ) == len ( vec2 )
sum _of_sq uares = sum ( pow ( v1 - v2 , 2)
for v1 , v2 in zip ( vec1 , vec2 ) )
return 1 / (1 + sum_of _squar es )

199

X Y . 1 +1, .
:
X,Y =

cov(X, Y )
E[(X X )(Y Y )]
=
,
X Y
X Y

cov , E , X Y .
, , :
Pn
1

i=1 (Xi X)(Yi Y )

n1
q
=
r= q
P
P
n
n
1
1
2
2

X)
(Y

Y
)
i
i=1
i=1 i
n1
n1
Pn

i=1 (Xi X)(Yi Y )

qP
= qP
,
n
n
2

2
(X

X)
i
i=1
i=1 (Yi Y )
Y .
X
, :
n

1 X
r=
n 1 i=1

Xi X
sX

1
n

i=1

Yi Y
sY

1
n

i=1

Yi , .

, . ,
E[(X X )(Y Y )] = E(XY ) X Y ,

E[(X X )(Y Y )]
p
=
(E[(X X )2 ]) (E[(Y Y )2 ])
E(XY ) X Y
p
=p
.
2
(E[X ] 2X ) (E[Y 2 ] 2Y )
Pn
Pn
Pn
1
1
i=1 Xi Yi n2
i=1 Xi
i=1 Yi
n
q
r= q P
=
Pn
Pn
Pn
n
1
1
1
2
2 (1
2
2
X
X
)
Y
Y
)

(
i=1 i
i=1 i
i=1 i
i=1 i
n
n
n
n
Pn
Pn
Pn
X
Y
i
i
i=1
i=1
i=1 Xi Yi
n
r
.
=r
Pn
Pn
2
2
Pn
P
X
Y
(
(
)
)
i
i
n
i=1
i=1
2
2
i=1 Xi
i=1 Yi
n
n

X,Y = p

,
2 .. . 1 ,
:
, , . ,
.

200

7.1:

[-0.09; 0.0]
[-0.3; -0.1]
[-0.5; -0.3]
[-1.0; -0.5]

[0.0;
[0.1;
[0.3;
[0.5;

0.09]
0.3]
0.5]
1.0]

1 , ,
. , [1, 2, 3],
[1, 3, 5], - .
0 , .
Listing 7.2:

def sim_pearson ( vec1 , vec2 ) :

assert len ( vec1 ) == len ( vec2 )
n = len ( vec1 )
if n == 0:
return 0

sum1 , sum2 = sum ( vec1 ) , sum ( vec2 )

s um 1_ o f_ sq u ar es = sum ( pow (v , 2) for v in vec1 )
s um 2_ o f_ sq u ar es = sum ( pow (v , 2) for v in vec2 )
s um _o f _p ro d uc ts = sum ( v1 * v2 for v1 , v2 in zip ( vec1 , vec2 ) )
12

num = su m_ o f_ pr o du c ts - ( sum1 * sum2 / n )

den = sqrt (( s um 1 _o f_ s qu ar e s - pow ( sum1 , 2) / n ) *
( su m 2_ of _ sq uar es - pow ( sum2 , 2) / n ) )
if den == 0:
return 0

r = num / den
return r

7.2.3

from numpy import *

from metrics import sim_distance
4

def kmeans ( data , distance = sim_distance , k =3 , iternum =100 , err =0.0005) :

vecnum , veclen = data . shape
minima = data . min ( axis =0)
maxima = data . max ( axis =0)
centroids = random . rand (k , veclen ) * ( maxima - minima ) + minima
clusters = None
for t in range ( iternum ) :
distances = zeros ( shape =( k , vecnum ) )
for i in range ( k ) :

201

distances [ i ] = sum (( data - centroids [i ,:]) **2 , axis =1)

clusters = distances . argmin ( axis =0)
clusters . shape = ( vecnum , 1)
19

old_centroids = centroids . copy ()

for i in range ( k ) :
cluster = where ( clusters == i , 1 , 0)
if sum ( cluster ) > 0:
centroids [i ,:] = sum ( data * cluster , axis =0) / sum ( cluster )

if abs ( sum ( centroids - old_centroids ) ) < err :

break
return centroids , clusters

7.2.4

, ,
, , .
1

5.1 ,3.5 ,1.4 ,0.2 , setosa

...
7.0 ,3.2 ,4.7 ,1.4 , versicolor
...
6.3 ,3.3 ,6.0 ,2.5 , virginica
...

. 7.3 , .
. 7.4 k .

7.3

. , .

7.3.1

k (. k-nearest neighbor algorithm) ,

,
k . ,
.
, , .
.
:
Listing 7.3: kNN
def knn ( item , dataset ) :
klass = None
nearest = None
4

for row in dataset :

next = metrics . sim_distance ( item , row [ " features " ])

if nearest is None :
nearest = next

202

. 7.3: . setosa , versicolor

, virginica .

elif next > nearest :

klass = row [ " class " ]
nearest = next
return klass

:
1

>>> knn ([4.5 , 3.8 , 1.2 , 0.4] , dataset )

" setosa "

7.3.2

kd (. k-dimensional tree) ,
k- .
kd ,
k. , , .

203

. 7.4: .
,
.
: -
, . ,
x, , x ,
, (xconcrete , 0, . . . );
. ,
.
kd
, , , .
:
,
x, .

204

,
.
(2; 3), (5; 4), (9; 6), (4; 7), (8; 1), (7; 2). ,
. 7.5. , . 7.6.
7; 2

5; 4

2; 3

9; 6

4; 7

8; 1

. 7.5: kd .

10
8

6
4
2
0

x
. 7.6: kd .
kd .

class Node ( object ) :

def __init__ ( self , location , kind , left , right ) :
self . location = location
self . kind = kind
self . left = left
self . right = right
def kdtree ( dataset , depth =0) :
if not dataset :
return None
n = len ( dataset )
k = len ( dataset [0][ " features " ])

205

axis = depth % k
median = n // 2
dataset . sort ( key = lambda row : row [ " features " ][ axis ])
18

location = dataset [ median ][ " features " ]

kind = dataset [ median ][ " class " ]
left = kdtree ( dataset [: median ] , depth + 1)
right = kdtree ( dataset [ median + 1:] , depth + 1)
23

return Node ( location , kind , left , right )

:
n O(n log2 n).
O(log n).
O(log n).
1

O(log n) ; O(k n1 k ) .