Download the full version of the textbook now at textbookfull.

com

Big Data Computing A Guide for Business and

Technology Managers 1st Edition Vivek Kale

https://textbookfull.com/product/big-data-
computing-a-guide-for-business-and-technology-
managers-1st-edition-vivek-kale/

Explore and download more textbook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Digital Transformation of Enterprise Architecture 1st

Edition Vivek Kale

https://textbookfull.com/product/digital-transformation-of-enterprise-
architecture-1st-edition-vivek-kale/

textbookfull.com

Agile network businesses : collaboration, coordination,

and competitive advantage 1st Edition Vivek Kale

https://textbookfull.com/product/agile-network-businesses-
collaboration-coordination-and-competitive-advantage-1st-edition-
vivek-kale/
textbookfull.com

Big Data Analytics for Cloud IoT and Cognitive Computing

1st Edition Kai Hwang

https://textbookfull.com/product/big-data-analytics-for-cloud-iot-and-
cognitive-computing-1st-edition-kai-hwang/

textbookfull.com

Engineering Interventions in Agricultural Processing 1st

Edition Goyal

https://textbookfull.com/product/engineering-interventions-in-
agricultural-processing-1st-edition-goyal/

textbookfull.com
Smart Modeling for Engineering Systems Proceedings of the
Conference 50 Years of the Development of Grid
Characteristic Method Igor B. Petrov
https://textbookfull.com/product/smart-modeling-for-engineering-
systems-proceedings-of-the-conference-50-years-of-the-development-of-
grid-characteristic-method-igor-b-petrov/
textbookfull.com

Lonely Planet Scotland 2017 9th Edition Lonely Planet

https://textbookfull.com/product/lonely-planet-scotland-2017-9th-
edition-lonely-planet/

textbookfull.com

Superconductors in the Power Grid Materials and

Applications Woodhead Publishing Series in Energy 1st
Edition Rey
https://textbookfull.com/product/superconductors-in-the-power-grid-
materials-and-applications-woodhead-publishing-series-in-energy-1st-
edition-rey/
textbookfull.com

Go programming language The Ultimate Beginner s Guide to

Learn Go Programming Step by Step 3rd Edition John Bach

https://textbookfull.com/product/go-programming-language-the-ultimate-
beginner-s-guide-to-learn-go-programming-step-by-step-3rd-edition-
john-bach/
textbookfull.com

Growing Sanctuary Magical Mischief at the B B 4 1st

Edition Susi Hawke

https://textbookfull.com/product/growing-sanctuary-magical-mischief-
at-the-b-b-4-1st-edition-susi-hawke/

textbookfull.com
Regenerated Organs Future Perspectives 1st Edition Chandra
P Sharma

https://textbookfull.com/product/regenerated-organs-future-
perspectives-1st-edition-chandra-p-sharma/

textbookfull.com
Big Data
Computing
A Guide for Business
and Technology Managers
 Chapman & Hall/CRC
Big Data Series

SERIES EDITOR
Sanjay Ranka

AIMS AND SCOPE

This series aims to present new research and applications in Big Data, along with the computa-
tional tools and techniques currently in development. The inclusion of concrete examples and
applications is highly encouraged. The scope of the series includes, but is not limited to, titles in the
areas of social networks, sensor networks, data-centric computing, astronomy, genomics, medical
data analytics, large-scale e-commerce, and other relevant topics that may be proposed by poten-
tial contributors.

PUBLISHED TITLES
BIG DATA COMPUTING: A GUIDE FOR BUSINESS AND TECHNOLOGY
MANAGERS
Vivek Kale
BIG DATA OF COMPLEX NETWORKS
Matthias Dehmer, Frank Emmert-Streib, Stefan Pickl, and Andreas Holzinger
BIG DATA : ALGORITHMS, ANALYTICS, AND APPLICATIONS
Kuan-Ching Li, Hai Jiang, Laurence T. Yang, and Alfredo Cuzzocrea
NETWORKING FOR BIG DATA
Shui Yu, Xiaodong Lin, Jelena Mišić, and Xuemin (Sherman) Shen
Big Data
Computing
A Guide for Business
and Technology Managers

Vivek Kale
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2017 by Vivek Kale
CRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Printed on acid-free paper

Version Date: 20160426

International Standard Book Number-13: 978-1-4987-1533-1 (Hardback)

This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid-
ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright
holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may
rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti-
lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy-
ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.

For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.

Library of Congress Cataloging-in-Publication Data

Names: Kale, Vivek, author.

Title: Big data computing : a guide for business and technology managers /
author, Vivek Kale.
Description: Boca Raton : Taylor & Francis, CRC Press, 2016. | Series:
Chapman & Hall/CRC big data series | Includes bibliographical references
and index.
Identifiers: LCCN 2016005989 | ISBN 9781498715331
Subjects: LCSH: Big data.
Classification: LCC QA76.9.B45 K35 2016 | DDC 005.7--dc23
LC record available at https://lccn.loc.gov/2016005989

Visit the Taylor & Francis Web site at

http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
 To

Nilesh Acharya and family

for unstinted support on

references and research

for my numerous book projects.

This page intentionally left blank
Contents

List of Figures .............................................................................................................................. xxi

List of Tables .............................................................................................................................. xxiii
Preface .......................................................................................................................................... xxv
Acknowledgments .................................................................................................................... xxxi
Author .......................................................................................................................................xxxiii

1. Computing Beyond the Moore’s Law Barrier While Being More Tolerant of
Faults and Failures..................................................................................................................1
1.1 Moore’s Law Barrier .....................................................................................................2
1.2 Types of Computer Systems ........................................................................................4
1.2.1 Microcomputers ...............................................................................................4
1.2.2 Midrange Computers ......................................................................................4
1.2.3 Mainframe Computers ....................................................................................5
1.2.4 Supercomputers ...............................................................................................5
1.3 Parallel Computing .......................................................................................................6
1.3.1 Von Neumann Architectures ................................................................. 8
1.3.2 Non-Neumann Architectures ........................................................................9
1.4 Parallel Processing ........................................................................................................9
1.4.1 Multiprogramming........................................................................................ 10
1.4.2 Vector Processing ........................................................................................... 10
1.4.3 Symmetric Multiprocessing Systems .......................................................... 11
1.4.4 Massively Parallel Processing ...................................................................... 11
1.5 Fault Tolerance ............................................................................................................. 12
1.6 Reliability Conundrum .............................................................................................. 14
1.7 Brewer’s CAP Theorem .............................................................................................. 15
1.8 Summary ...................................................................................................................... 18

Section I Genesis of Big Data Computing

2. Database Basics ..................................................................................................................... 21

2.1 Database Management System .................................................................................21
2.1.1 DBMS Benefits ................................................................................................22
2.1.2 Defining a Database Management System.................................................23
2.1.2.1 Data Models alias Database Models ............................................ 26
2.2 Database Models .........................................................................................................27
2.2.1 Relational Database Model ...........................................................................28
2.2.2 Hierarchical Database Model .......................................................................30
2.2.3 Network Database Model .............................................................................32
2.2.4 Object-Oriented Database Models...............................................................32
2.2.5 Comparison of Models ..................................................................................33
2.2.5.1 Similarities ...................................................................................... 33
2.2.5.2 Dissimilarities ................................................................................. 35

vii
viii Contents

2.3 Database Components................................................................................................36

2.3.1 External Level ................................................................................................. 37
2.3.2 Conceptual Level ........................................................................................... 37
2.3.3 Physical Level ................................................................................................. 38
2.3.4 The Three-Schema Architecture ................................................................. 38
2.3.4.1 Data Independence ........................................................................ 39
2.4 Database Languages and Interfaces ......................................................................... 40
2.5 Categories of Database Management Systems .......................................................42
2.6 Other Databases ..........................................................................................................44
2.6.1 Text Databases ................................................................................................44
2.6.2 Multimedia Databases ..................................................................................44
2.6.3 Temporal Databases.......................................................................................44
2.6.4 Spatial Databases ........................................................................................... 45
2.6.5 Multiple or Heterogeneous Databases ........................................................ 45
2.6.6 Stream Databases ........................................................................................... 45
2.6.7 Web Databases ............................................................................................... 46
2.7 Evolution of Database Technology .......................................................................... 46
2.7.1 Distribution .....................................................................................................47
2.7.2 Performance ....................................................................................................47
2.7.2.1 Database Design for Multicore Processors .................................48
2.7.3 Functionality .................................................................................................. 49
2.8 Summary ...................................................................................................................... 50

Section II Road to Big Data Computing

3. Analytics Basics .................................................................................................................... 53

3.1 Intelligent Analysis ..................................................................................................... 53
3.1.1 Intelligence Maturity Model.........................................................................55
3.1.1.1 Data ..................................................................................................55
3.1.1.2 Communication ..............................................................................55
3.1.1.3 Information .....................................................................................56
3.1.1.4 Concept ............................................................................................56
3.1.1.5 Knowledge ......................................................................................57
3.1.1.6 Intelligence ......................................................................................58
3.1.1.7 Wisdom............................................................................................58
3.2 Decisions ...................................................................................................................... 59
3.2.1 Types of Decisions .........................................................................................59
3.2.2 Scope of Decisions .........................................................................................61
3.3 Decision-Making Process........................................................................................... 61
3.4 Decision-Making Techniques ....................................................................................63
3.4.1 Mathematical Programming ........................................................................63
3.4.2 Multicriteria Decision Making .....................................................................64
3.4.3 Case-Based Reasoning...................................................................................64
3.4.4 Data Warehouse and Data Mining ..............................................................64
3.4.5 Decision Tree...................................................................................................64
3.4.6 Fuzzy Sets and Systems ................................................................................65
Contents ix

3.5 Analytics.......................................................................................................................65
3.5.1 Descriptive Analytics .................................................................................... 66
3.5.2 Predictive Analytics ...................................................................................... 66
3.5.3 Prescriptive Analytics ................................................................................... 67
3.6 Data Science Techniques ............................................................................................ 68
3.6.1 Database Systems...........................................................................................68
3.6.2 Statistical Inference ........................................................................................68
3.6.3 Regression and Classification.......................................................................69
3.6.4 Data Mining and Machine Learning ...........................................................70
3.6.5 Data Visualization ..........................................................................................70
3.6.6 Text Analytics .................................................................................................71
3.6.7 Time Series and Market Research Models..................................................72
3.7 Snapshot of Data Analysis Techniques and Tasks ................................................. 74
3.8 Summary ......................................................................................................................77

4. Data Warehousing Basics .................................................................................................... 79

4.1 Relevant Database Concepts...................................................................................... 79
4.1.1 Physical Database Design .............................................................................80
4.2 Data Warehouse .......................................................................................................... 81
4.2.1 Multidimensional Model ..............................................................................83
4.2.1.1 Data Cube ........................................................................................84
4.2.1.2 Online Analytical Processing ........................................................84
4.2.1.3 Relational Schemas ........................................................................87
4.2.1.4 Multidimensional Cube.................................................................88
4.3 Data Warehouse Architecture ................................................................................... 91
4.3.1 Architecture Tiers ...........................................................................................91
4.3.1.1 Back-End Tier .................................................................................. 91
4.3.1.2 Data Warehouse Tier ..................................................................... 91
4.3.1.3 OLAP Tier ........................................................................................ 93
4.3.1.4 Front-End Tier ................................................................................. 93
4.4 Data Warehouse 1.0..................................................................................................... 93
4.4.1 Inmon’s Information Factory .......................................................................93
4.4.2 Kimbal’s Bus Architecture ............................................................................94
4.5 Data Warehouse 2.0 .................................................................................................... 95
4.5.1 Inmon’s DW 2.0 ..............................................................................................95
4.5.2 Claudia Imhoff and Colin White’s DSS 2.0 ................................................96
4.6 Data Warehouse Architecture Challenges .............................................................. 96
4.6.1 Performance ....................................................................................................98
4.6.2 Scalability ........................................................................................................98
4.7 Summary .................................................................................................................... 100

5. Data Mining Basics ............................................................................................................ 101

5.1 Data Mining ............................................................................................................... 101
5.1.1 Benefits .......................................................................................................... 103
5.2 Data Mining Applications ....................................................................................... 104
5.3 Data Mining Analysis .............................................................................................. 106
5.3.1 Supervised Analysis .................................................................................... 106
5.3.1.1 Exploratory Analysis ................................................................... 106
5.3.1.2 Classification ................................................................................. 107
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
x Contents

5.3.1.3 Regression ..................................................................................... 107

5.3.1.4 Time Series .................................................................................... 108
5.3.2 Un-Supervised Analysis ............................................................................. 108
5.3.2.1 Association Rules ......................................................................... 108
5.3.2.2 Clustering ...................................................................................... 108
5.3.2.3 Description and Visualization.................................................... 109
5.4 CRISP-DM Methodology ......................................................................................... 109
5.4.1 Business Understanding ............................................................................. 110
5.4.2 Data Understanding .................................................................................... 111
5.4.3 Data Preparation .......................................................................................... 111
5.4.4 Modeling ....................................................................................................... 112
5.4.5 Model Evaluation ......................................................................................... 113
5.4.6 Model Deployment ...................................................................................... 113
5.5 Machine Learning ..................................................................................................... 114
5.5.1 Cybersecurity Systems ................................................................................ 116
5.5.1.1 Data Mining for Cybersecurity .................................................. 117
5.6 Soft Computing ......................................................................................................... 118
5.6.1 Artificial Neural Networks ........................................................................ 119
5.6.2 Fuzzy Systems .............................................................................................. 120
5.6.3 Evolutionary Algorithms ............................................................................ 120
5.6.4 Rough Sets .................................................................................................... 121
5.7 Summary .................................................................................................................... 122

6. Distributed Systems Basics .............................................................................................. 123

6.1 Distributed Systems .................................................................................................. 123
6.1.1 Parallel Computing...................................................................................... 125
6.1.2 Distributed Computing............................................................................... 128
6.1.2.1 System Architectural Styles ........................................................ 129
6.1.2.2 Software Architectural Styles..................................................... 130
6.1.2.3 Technologies for Distributed Computing ................................. 135
6.2 Distributed Databases .............................................................................................. 138
6.2.1 Characteristics of Distributed Databases ................................................. 140
6.2.1.1 Transparency ................................................................................ 140
6.2.1.2 Availability and Reliability ......................................................... 140
6.2.1.3 Scalability and Partition Tolerance ............................................ 141
6.2.1.4 Autonomy ...................................................................................... 141
6.2.2 Advantages and Disadvantages of Distributed Databases.................... 142
6.2.3 Data Replication and Allocation ................................................................ 146
6.2.4 Concurrency Control and Recovery ......................................................... 146
6.2.4.1 Distributed Recovery ................................................................... 147
6.2.5 Query Processing and Optimization ........................................................ 148
6.2.6 Transaction Management ........................................................................... 149
6.2.6.1 Two-Phase Commit Protocol ...................................................... 149
6.2.6.2 Three-Phase Commit Protocol ................................................... 150
6.2.7 Rules for Distributed Databases ................................................................ 151
6.3 Summary .................................................................................................................... 152
Contents xi

7. Service-Oriented Architecture Basics ............................................................................153

7.1 Service-Oriented Architecture ................................................................................ 153
7.1.1 Defining SOA................................................................................................ 155
7.1.1.1 Services .......................................................................................... 155
7.2 SOA Benefits .............................................................................................................. 156
7.3 Characteristics of SOA.............................................................................................. 157
7.3.1 Dynamic, Discoverable, Metadata Driven ............................................... 157
7.3.2 Designed for Multiple Invocation Styles .................................................. 158
7.3.3 Loosely Coupled .......................................................................................... 158
7.3.4 Well-Defined Service Contracts ................................................................. 158
7.3.5 Standard Based ............................................................................................ 158
7.3.6 Granularity of Services and Service Contracts........................................ 158
7.3.7 Stateless ......................................................................................................... 159
7.3.8 Predictable Service-Level Agreements (SLAs) ........................................ 159
7.3.9 Design Services with Performance in Mind ............................................ 159
7.4 SOA Applications ...................................................................................................... 159
7.4.1 Rapid Application Integration ................................................................... 160
7.4.2 Multichannel Access.................................................................................... 160
7.4.3 Business Process Management .................................................................. 160
7.5 SOA Ingredients ........................................................................................................ 161
7.5.1 Objects, Services, and Resources ............................................................... 161
7.5.1.1 Objects............................................................................................ 161
7.5.1.2 Services .......................................................................................... 161
7.5.1.3 Resources ....................................................................................... 162
7.5.2 SOA and Web Services ................................................................................ 163
7.5.2.1 Describing Web Services: Web Services Description
Language ...................................................................................... 165
7.5.2.2 Accessing Web Services: Simple Object Access Protocol ...... 165
7.5.2.3 Finding Web Services: Universal Description, Discovery,
and Integration ............................................................................. 165
7.5.3 SOA and RESTful Services ......................................................................... 166
7.6 Enterprise Service Bus .............................................................................................. 167
7.6.1 Characteristics of an ESB Solution ............................................................ 170
7.6.1.1 Key Capabilities of an ESB .......................................................... 171
7.6.1.2 ESB Scalability .............................................................................. 174
7.6.1.3 Event-Driven Nature of ESB ....................................................... 174
7.7 Summary .................................................................................................................... 175

8. Cloud Computing Basics ...................................................................................................177

8.1 Cloud Definition........................................................................................................ 177
8.2 Cloud Characteristics ............................................................................................... 179
8.2.1 Cloud Storage Infrastructure Requirements ........................................... 180
8.3 Cloud Delivery Models ............................................................................................ 181
8.3.1 Infrastructure as a Service (IaaS)............................................................... 182
8.3.2 Platform as a Service (PaaS) ....................................................................... 182
8.3.3 Software as a Service (SaaS) ....................................................................... 183
xii Contents

8.4 Cloud Deployment Models .....................................................................................185

8.4.1 Private Clouds ..............................................................................................185
8.4.2 Public Clouds ...............................................................................................185
8.4.3 Hybrid Clouds..............................................................................................186
8.4.4 Community Clouds .....................................................................................186
8.5 Cloud Benefits ...........................................................................................................186
8.6 Cloud Challenges ......................................................................................................190
8.6.1 Scalability ......................................................................................................191
8.6.2 Multitenancy.................................................................................................192
8.6.3 Availability ....................................................................................................193
8.6.3.1 Failure Detection .......................................................................... 194
8.6.3.2 Application Recovery................................................................... 195
8.7 Cloud Technologies...................................................................................................195
8.7.1 Virtualization ................................................................................................196
8.7.1.1 Characteristics of Virtualized Environment ............................ 197
8.7.2 Service-Oriented Computing .....................................................................200
8.7.2.1 Advantages of SOA ...................................................................... 201
8.7.2.2 Layers in SOA ............................................................................... 202
8.8 Summary ....................................................................................................................203

Section III Big Data Computing

9. Introducing Big Data Computing ...................................................................................207

9.1 Big Data ......................................................................................................................207
9.1.1 What Is Big Data?.........................................................................................208
9.1.1.1 Data Volume ..................................................................................208
9.1.1.2 Data Velocity .................................................................................210
9.1.1.3 Data Variety................................................................................... 211
9.1.1.4 Data Veracity .................................................................................212
9.1.2 Common Characteristics of Big Data Computing Systems ...................213
9.1.3 Big Data Appliances ....................................................................................214
9.2 Tools and Techniques of Big Data ...........................................................................215
9.2.1 Processing Approach ...................................................................................215
9.2.2 Big Data System Architecture ....................................................................216
9.2.2.1 BASE (Basically Available, Soft State, Eventual
Consistency) .............................................................................. 217
9.2.2.2 Functional Decomposition ..........................................................218
9.2.2.3 Master–Slave Replication ............................................................218
9.2.3 Row Partitioning or Sharding ....................................................................218
9.2.4 Row versus Column-Oriented Data Layouts ..........................................219
9.2.5 NoSQL Data Management..........................................................................220
9.2.6 In-Memory Computing ...............................................................................221
9.2.7 Developing Big Data Applications ............................................................222
9.3 Aadhaar Project .........................................................................................................223
9.4 Summary ....................................................................................................................226
Contents xiii

10. Big Data Technologies .......................................................................................................227

10.1 Functional Programming Paradigm .....................................................................227
10.1.1 Parallel Architectures and Computing Models .................................. 228
10.1.2 Data Parallelism versus Task Parallelism............................................. 228
10.2 Google MapReduce.................................................................................................229
10.2.1 Google File System ..................................................................................231
10.2.2 Google Bigtable ........................................................................................ 232
10.3 Yahoo!’s Vision of Big Data Computing ..............................................................233
10.3.1 Apache Hadoop........................................................................................234
10.3.1.1 Components of Hadoop Ecosystem.....................................235
10.3.1.2 Principles and Patterns Underlying the Hadoop
Ecosystem ................................................................................ 236
10.3.1.3 Storage and Processing Strategies ........................................237
10.3.2 Hadoop 2 alias YARN ............................................................................. 238
10.3.2.1 HDFS Storage .......................................................................... 239
10.3.2.2 MapReduce Processing .......................................................... 239
10.4 Hadoop Distribution ..............................................................................................240
10.4.1 Cloudera Distribution of Hadoop (CDH) ............................................243
10.4.2 MapR..........................................................................................................243
10.4.3 Hortonworks Data Platform (HDP) ......................................................243
10.4.4 Pivotal HD................................................................................................. 243
10.5 Storage and Processing Strategies ........................................................................244
10.5.1 Characteristics of Big Data Storage Methods.......................................244
10.5.2 Characteristics of Big Data Processing Methods................................. 244
10.6 NoSQL Databases ....................................................................................................245
10.6.1 Column-Oriented Stores or Databases .................................................246
10.6.2 Key-Value Stores (K-V Stores) or Databases .........................................246
10.6.3 Document-Oriented Databases ..............................................................247
10.6.4 Graph Stores or Databases ...................................................................... 248
10.6.5 Comparison of NoSQL Databases ......................................................... 248
10.7 Summary .................................................................................................................. 249

11. Big Data NoSQL Databases ..............................................................................................251

11.1 Characteristics of NoSQL Systems .......................................................................254
11.1.1 NoSQL Characteristics Related to Distributed Systems and
Distributed Databases .............................................................................254
11.1.2 NoSQL Characteristics Related to Data Models and Query
Languages ................................................................................................. 256
11.2 Column Databases ..................................................................................................256
11.2.1 Cassandra.................................................................................................. 258
11.2.1.1 Cassandra Features.................................................................258
11.2.2 Google BigTable .......................................................................................260
11.2.3 HBase ......................................................................................................... 260
11.2.3.1 HBase Data Model and Versioning ......................................260
11.2.3.2 HBase CRUD Operations ...................................................... 262
11.2.3.3 HBase Storage and Distributed System Concepts ............. 263
xiv Contents

11.3 Key-Value Databases............................................................................................. 263

11.3.1 Riak .......................................................................................................... 264
11.3.1.1 Riak Features ........................................................................ 264
11.3.2 Amazon Dynamo .................................................................................. 265
11.3.2.1 DynamoDB Data Model ...................................................... 266
11.4 Document Databases ............................................................................................ 266
11.4.1 CouchDB ................................................................................................. 268
11.4.2 MongoDB ................................................................................................ 268
11.4.2.1 MongoDB Features ..............................................................269
11.4.2.2 MongoDB Data Model ........................................................270
11.4.2.3 MongoDB CRUD Operations .............................................272
11.4.2.4 MongoDB Distributed Systems Characteristics .............. 272
11.5 Graph Databases ................................................................................................... 274
11.5.1 OrientDB ................................................................................................. 274
11.5.2 Neo4j ........................................................................................................ 275
11.5.2.1 Neo4j Features ......................................................................275
11.5.2.2 Neo4j Data Model ................................................................ 276
11.6 Summary ................................................................................................................ 277

12. Big Data Development with Hadoop..............................................................................279

12.1 Hadoop MapReduce .............................................................................................284
12.1.1 MapReduce Processing .........................................................................284
12.1.1.1 JobTracker..............................................................................284
12.1.1.2 TaskTracker ........................................................................... 286
12.1.2 MapReduce Enhancements and Extensions ...................................... 286
12.1.2.1 Supporting Iterative Processing .........................................286
12.1.2.2 Join Operations .....................................................................288
12.1.2.3 Data Indices ..........................................................................289
12.1.2.4 Column Storage .................................................................... 290
12.2 YARN ......................................................................................................................291
12.3 Hadoop Distributed File System (HDFS)........................................................... 293
12.3.1 Characteristics of HDFS ........................................................................ 293
12.4 HBase ...................................................................................................................... 295
12.4.1 HBase Architecture ............................................................................... 296
12.5 ZooKeeper .............................................................................................................. 297
12.6 Hive .........................................................................................................................297
12.7 Pig ............................................................................................................................ 298
12.8 Kafka .......................................................................................................................299
12.9 Flume ......................................................................................................................300
12.10 Sqoop ......................................................................................................................300
12.11 Impala ..................................................................................................................... 301
12.12 Drill .........................................................................................................................302
12.13 Whirr .......................................................................................................................302
12.14 Summary ................................................................................................................ 302

13. Big Data Analysis Languages, Tools, and Environments ..........................................303

13.1 Spark ....................................................................................................................... 303
13.1.1 Spark Components ................................................................................305
Contents xv

13.1.2 Spark Concepts ......................................................................................306

13.1.2.1 Shared Variables ..................................................................306
13.1.2.2 SparkContext .......................................................................306
13.1.2.3 Resilient Distributed Datasets ...........................................306
13.1.2.4 Transformations...................................................................306
13.1.2.5 Action .................................................................................... 307
13.1.3 Benefits of Spark .................................................................................... 307
13.2 Functional Programming ......................................................................................308
13.3 Clojure....................................................................................................................... 312
13.4 Python ....................................................................................................................... 313
13.4.1 NumPy.................................................................................................... 313
13.4.2 SciPy ........................................................................................................ 313
13.4.3 Pandas ..................................................................................................... 313
13.4.4 Scikit-Learn ............................................................................................ 313
13.4.5 IPython ................................................................................................... 314
13.4.6 Matplotlib ............................................................................................... 314
13.4.7 Stats Models ...........................................................................................314
13.4.8 Beautiful Soup ....................................................................................... 314
13.4.9 NetworkX ............................................................................................... 314
13.4.10 NLTK....................................................................................................... 314
13.4.11 Gensim .................................................................................................... 314
13.4.12 PyPy ........................................................................................................ 315
13.5 Scala .......................................................................................................................... 315
13.5.1 Scala Advantages .................................................................................. 316
13.5.1.1 Interoperability with Java .................................................. 316
13.5.1.2 Parallelism............................................................................ 316
13.5.1.3 Static Typing and Type Inference ......................................316
13.5.1.4 Immutability ........................................................................316
13.5.1.5 Scala and Functional Programs .........................................317
13.5.1.6 Null Pointer Uncertainty.................................................... 317
13.5.2 Scala Benefits ......................................................................................... 318
13.5.2.1 Increased Productivity .......................................................318
13.5.2.2 Natural Evolution from Java .............................................318
13.5.2.3 Better Fit for Asynchronous and Concurrent Code ....... 318
13.6 R ................................................................................................................................. 319
13.6.1 Analytical Features of R ....................................................................... 319
13.6.1.1 General..................................................................................319
13.6.1.2 Business Dashboard and Reporting .................................320
13.6.1.3 Data Mining .........................................................................320
13.6.1.4 Business Analytics .............................................................. 320
13.7 SAS ............................................................................................................................ 321
13.7.1 SAS DATA Step...................................................................................... 321
13.7.2 Base SAS Procedures ............................................................................ 322
13.8 Summary .................................................................................................................. 323

14. Big Data DevOps Management .......................................................................................325

14.1 Big Data Systems Development Management .................................................... 326
14.1.1 Big Data Systems Architecture............................................................ 326
xvi Contents

14.1.2 Big Data Systems Lifecycle ..................................................................... 326

14.1.2.1 Data Sourcing ..........................................................................326
14.1.2.2 Data Collection and Registration in a Standard Format .......326
14.1.2.3 Data Filter, Enrich, and Classification..................................327
14.1.2.4 Data Analytics, Modeling, and Prediction ..........................327
14.1.2.5 Data Delivery and Visualization ..........................................328
14.1.2.6 Data Supply to Consumer Analytics Applications ............ 328
14.2 Big Data Systems Operations Management ........................................................ 328
14.2.1 Core Portfolio of Functionalities ............................................................ 328
14.2.1.1 Metrics for Interfacing to Cloud Service Providers ........... 330
14.2.2 Characteristics of Big Data and Cloud Operations ............................. 332
14.2.3 Core Services ............................................................................................ 332
14.2.3.1 Discovery and Replication ....................................................332
14.2.3.2 Load Balancing........................................................................333
14.2.3.3 Resource Management ...........................................................333
14.2.3.4 Data Governance .................................................................... 333
14.2.4 Management Services .............................................................................334
14.2.4.1 Deployment and Configuration ...........................................334
14.2.4.2 Monitoring and Reporting ....................................................334
14.2.4.3 Service-Level Agreements (SLAs) Management ................334
14.2.4.4 Metering and Billing ..............................................................335
14.2.4.5 Authorization and Authentication .......................................335
14.2.4.6 Fault Tolerance ........................................................................ 335
14.2.5 Governance Services ............................................................................... 336
14.2.5.1 Governance ..............................................................................336
14.2.5.2 Security.....................................................................................337
14.2.5.3 Privacy ......................................................................................338
14.2.5.4 Trust ..........................................................................................339
14.2.5.5 Security Risks ..........................................................................340
14.2.6 Cloud Governance, Risk, and Compliance .......................................... 341
14.2.6.1 Cloud Security Solutions .......................................................344
14.3 Migrating to Big Data Technologies .....................................................................346
14.3.1 Lambda Architecture ..............................................................................348
14.3.1.1 Batch Processing .....................................................................348
14.3.1.2 Real Time Analytics ............................................................... 349
14.4 Summary .................................................................................................................. 349

Section IV Big Data Computing Applications

15. Web Applications................................................................................................................353

15.1 Web-Based Applications ........................................................................................ 353
15.2 Reference Architecture ...........................................................................................354
15.2.1 User Interaction Architecture ................................................................ 355
15.2.2 Service-Based Architecture .................................................................... 355
15.2.3 Business Object Architecture ................................................................. 356
15.3 Realization of the Reference Architecture in J2EE ............................................. 356
15.3.1 JavaServer Pages and Java Servlets as the User Interaction
Components .............................................................................................. 356
Contents xvii

15.3.2 Session Bean EJBs as Service-Based Components ...............................356

15.3.3 Entity Bean EJBs as the Business Object Components .......................357
15.3.4 Distributed Java Components ................................................................357
15.3.5 J2EE Access to the EIS (Enterprise Information Systems) Tier .......... 357
15.4 Model–View–Controller Architecture ................................................................. 357
15.5 Evolution of the Web ............................................................................................... 359
15.5.1 Web 1.0.......................................................................................................359
15.5.2 Web 2.0....................................................................................................... 359
15.5.2.1 Weblogs or Blogs.....................................................................359
15.5.2.2 Wikis .........................................................................................360
15.5.2.3 RSS Technologies ....................................................................360
15.5.2.4 Social Tagging .........................................................................361
15.5.2.5 Mashups: Integrating Information .......................................361
15.5.2.6 User Contributed Content ..................................................... 361
15.5.3 Web 3.0....................................................................................................... 362
15.5.4 Mobile Web ...............................................................................................363
15.5.5 The Semantic Web .................................................................................... 363
15.5.6 Rich Internet Applications ......................................................................364
15.6 Web Applications ....................................................................................................364
15.6.1 Web Applications Dimensions............................................................... 365
15.6.1.1 Presentation .............................................................................365
15.6.1.2 Dialogue ...................................................................................366
15.6.1.3 Navigation ...............................................................................366
15.6.1.4 Process ......................................................................................366
15.6.1.5 Data ........................................................................................... 367
15.7 Search Analysis ....................................................................................................... 367
15.7.1 SLA Process .............................................................................................. 368
15.8 Web Analysis ........................................................................................................... 371
15.8.1 Veracity of Log Files Data ....................................................................... 374
15.8.1.1 Unique Visitors .......................................................................374
15.8.1.2 Visitor Count ...........................................................................374
15.8.1.3 Visit Duration .......................................................................... 375
15.8.2 Web Analysis Tools.................................................................................. 375
15.9 Summary .................................................................................................................. 376

16. Social Network Applications ...........................................................................................377

16.1 Networks .................................................................................................................. 378
16.1.1 Concept of Networks...............................................................................378
16.1.2 Principles of Networks ............................................................................ 379
16.1.2.1 Metcalfe’s Law ........................................................................379
16.1.2.2 Power Law ...............................................................................379
16.1.2.3 Small Worlds Networks ......................................................... 379
16.2 Computer Networks ............................................................................................... 380
16.2.1 Internet ......................................................................................................381
16.2.2 World Wide Web (WWW) ...................................................................... 381
16.3 Social Networks....................................................................................................... 382
16.3.1 Popular Social Networks ........................................................................ 386
16.3.1.1 LinkedIn................................................................................... 386
16.3.1.2 Facebook................................................................................... 386
xviii Contents

16.3.1.3 Twitter ................................................................................... 387

16.3.1.4 Google+ ................................................................................ 388
16.3.1.5 Other Social Networks........................................................... 389
16.4 Social Networks Analysis (SNA) .......................................................................... 389
16.5 Text Analysis ............................................................................................................ 391
16.5.1 Defining Text Analysis ............................................................................ 392
16.5.1.1 Document Collection .......................................................... 392
16.5.1.2 Document ............................................................................. 393
16.5.1.3 Document Features ............................................................. 393
16.5.1.4 Domain Knowledge ............................................................ 395
16.5.1.5 Search for Patterns and Trends .......................................... 396
16.5.1.6 Results Presentation ............................................................... 396
16.6 Sentiment Analysis ................................................................................................. 397
16.6.1 Sentiment Analysis and Natural Language Processing (NLP) ......... 398
16.6.2 Applications ..............................................................................................400
16.7 Summary ..................................................................................................................400

17. Mobile Applications...........................................................................................................401

17.1 Mobile Computing Applications .......................................................................... 401
17.1.1 Generations of Communication Systems ............................................. 402
17.1.1.1 1st Generation: Analog ....................................................... 402
17.1.1.2 2nd Generation: CDMA, TDMA, and GSM ..................... 402
17.1.1.3 2.5 Generation: GPRS, EDGE, and CDMA 2000 .............. 405
17.1.1.4 3rd Generation: wCDMA, UMTS, and iMode ................. 406
17.1.1.5 4th Generation ......................................................................... 406
17.1.2 Mobile Operating Systems ..................................................................... 406
17.1.2.1 Symbian ................................................................................ 406
17.1.2.2 BlackBerry OS ...................................................................... 407
17.1.2.3 Google Android ................................................................... 407
17.1.2.4 Apple iOS ............................................................................. 408
17.1.2.5 Windows Phone ......................................................................408
17.2 Mobile Web Services ...............................................................................................408
17.2.1 Mobile Field Cloud Services ................................................................... 412
17.3 Context-Aware Mobile Applications .................................................................... 414
17.3.1 Ontology-Based Context Model ............................................................. 415
17.3.2 Context Support for User Interaction .................................................... 415
17.4 Mobile Web 2.0 ........................................................................................................ 416
17.5 Mobile Analytics ..................................................................................................... 418
17.5.1 Mobile Site Analytics ...............................................................................418
17.5.2 Mobile Clustering Analysis ....................................................................418
17.5.3 Mobile Text Analysis ...............................................................................419
17.5.4 Mobile Classification Analysis ...............................................................420
17.5.5 Mobile Streaming Analysis .................................................................... 421
17.6 Summary .................................................................................................................. 421

18. Location-Based Systems Applications ............................................................................423

18.1 Location-Based Systems .........................................................................................423
18.1.1 Sources of Location Data ........................................................................ 424
18.1.1.1 Cellular Systems ..................................................................... 424
Contents xix

18.1.1.2 Multireference Point Systems ............................................ 426

18.1.1.3 Tagging..................................................................................... 427
18.1.2 Mobility Data ............................................................................................ 429
18.1.2.1 Mobility Data Mining ............................................................430
18.2 Location-Based Services ......................................................................................... 432
18.2.1 LBS Characteristics ..................................................................................435
18.2.2 LBS Positioning Technologies ................................................................436
18.2.3 LBS System Architecture .........................................................................437
18.2.4 LBS System Components ........................................................................439
18.2.5 LBS System Challenges ........................................................................... 439
18.3 Location-Based Social Networks .......................................................................... 441
18.4 Summary ..................................................................................................................443

19. Context-Aware Applications.............................................................................................445

19.1 Context-Aware Applications..................................................................................446
19.1.1 Types of Context-Awareness ..................................................................448
19.1.2 Types of Contexts .....................................................................................449
19.1.3 Context Acquisition .................................................................................450
19.1.4 Context Models ........................................................................................450
19.1.5 Generic Context-Aware Application Architecture ..............................452
19.1.6 Illustrative Context-Aware Applications .............................................. 452
19.2 Decision Pattern as Context ................................................................................... 453
19.2.1 Concept of Patterns ..................................................................................454
19.2.1.1 Patterns in Information Technology (IT) Solutions ........... 455
19.2.2 Domain-Specific Decision Patterns ....................................................... 455
19.2.2.1 Financial Decision Patterns ................................................ 455
19.2.2.2 CRM Decision Patterns .......................................................... 457
19.3 Context-Aware Mobile Services ............................................................................ 460
19.3.1 Limitations of Existing Infrastructure.................................................. 460
19.3.1.1 Limited Capability of Mobile Devices .............................. 460
19.3.1.2 Limited Sensor Capability .................................................. 461
19.3.1.3 Restrictive Network Bandwidth ........................................ 461
19.3.1.4 Trust and Security Requirements ...................................... 461
19.3.1.5 Rapidly Changing Context .................................................... 461
19.3.2 Types of Sensors .......................................................................................462
19.3.3 Context-Aware Mobile Applications ..................................................... 462
19.3.3.1 Context-Awareness Management Framework ...................464
19.4 Summary .................................................................................................................. 467

Epilogue: Internet of Things ................................................................................................... 469

References ................................................................................................................................... 473
Index ............................................................................................................................................. 475
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
This page intentionally left blank
List of Figures

Figure 1.1 Increase in the number of transistors on an Intel chip.........................................2

Figure 1.2 Hardware trends in the 1990s and the first decade .............................................3
Figure 1.3 Von Neumann computer architecture ...................................................................9
Figure 2.1 A hierarchical organization ...................................................................................30
Figure 2.2 The three-schema architecture ............................................................................ 37
Figure 2.3 Evolution of database technology......................................................................... 47
Figure 3.1 Characteristics of enterprise intelligence in terms of the scope of the
decisions ................................................................................................................... 61
Figure 4.1 Cube for sales data having dimensions store, time, and product and a
measure amount ......................................................................................................84
Figure 4.2 OLAP Operations. (a) Original Cube (b) Roll-up to the Country
level (c) Drill down to the month level (d) Pivot (e) Slice on
Store.City = ‘Mumbai’ (f) Dice on Store.Country = ‘US’ and Time.
Quarter = ‘Q1’ or ‘Q2’ .............................................................................................85
Figure 4.3 Example of a star schema....................................................................................... 87
Figure 4.4 Example of a snowflake schema ........................................................................... 88
Figure 4.5 Example of a constellation schema ....................................................................... 89
Figure 4.6 Lattice of cuboids derived from a four-dimensional cube................................90
Figure 4.7 Reference data warehouse architecture ............................................................... 92
Figure 5.1 Schematic of CRISP-DM methodology .............................................................. 110
Figure 5.2 Architecture of a machine-learning system...................................................... 115
Figure 5.3 Architecture of a fuzzy inference system.......................................................... 120
Figure 5.4 Architecture of a rough sets system ................................................................... 122
Figure 6.1 Parallel computing architectures: (a) Flynn’s taxonomy (b) shared
memory system, and (c) distributed system...................................................... 127
Figure 7.1 Web Services usage model ................................................................................... 164
Figure 7.2 ESB reducing connection complexity (a) Direct point-to-point
connections (n*n) and (b) Connecting through the bus (n) ............................ 168
Figure 7.3 Enterprise service bus (ESB) linking disparate systems and computing
environments ......................................................................................................... 169
Figure 8.1 The cloud reference model ................................................................................... 183
Figure 8.2 Portfolio of services for the three cloud delivery models ............................... 184

xxi
xxii List of Figures

Figure 9.1 4V characteristics of big data ...................................................................................... 208

Figure 9.2 Use cases for big data computing ........................................................................ 209
Figure 9.3 Parallel architectures .......................................................................................... 215
Figure 9.4 The solution architecture for the Aadhaar project .........................................225
Figure 10.1 Execution phases in a generic MapReduce application ................................. 230
Figure 10.2 Comparing the architecture of Hadoop 1 and Hadoop 2 ............................. 240
Figure 12.1 Hadoop ecosystem .............................................................................................. 282
Figure 12.2 Hadoop MapReduce architecture..................................................................... 285
Figure 12.3 YARN architecture ............................................................................................. 292
Figure 12.4 HDFS architecture .............................................................................................. 295
Figure 14.1 Big data systems architecture ............................................................................ 327
Figure 14.2 Big data systems lifecycle (BDSL) ..................................................................... 328
Figure 14.3 Lambda architecture ..........................................................................................348
Figure 15.1 Enterprise application in J2EE ........................................................................... 355
Figure 15.2 MVC and enterprise application architecture ................................................ 358
Figure 18.1 Principle of lateration.......................................................................................... 427
Figure 18.2 Trajectory mapping ............................................................................................. 432
Figure 19.1 Context definition ................................................................................................465
Figure 19.2 Conceptual metamodel of a context ................................................................. 466
List of Tables

Table 2.1 Characteristics of the Four Database Models ..................................................... 36

Table 2.2 Levels of Data Abstraction .................................................................................... 38
Table 3.1 Intelligence Maturity Model (IMM) ..................................................................... 55
Table 3.2 Analysis Techniques versus Tasks ....................................................................... 74
Table 4.1 Comparison between OLTP and OLAP Systems ............................................... 82
Table 4.2 Comparison between Operational Databases and Data Warehouses ............83
Table 4.3 The DSS 2.0 Spectrum ............................................................................................ 96
Table 5.1 Data Mining Application Areas.......................................................................... 105
Table 5.2 Characteristics of Soft Computing Compared with Traditional Hard
Computing ............................................................................................................. 118
Table 8.1 Key Attributes of Cloud Computing .................................................................. 178
Table 8.2 Key Attributes of Cloud Services ....................................................................... 179
Table 8.3 Comparison of Cloud Delivery Models............................................................. 185
Table 8.4 Comparison of Cloud Benefits for Small and Medium Enterprises
(SMEs) and Large Enterprises ............................................................................. 189
Table 9.1 Scale of Data ........................................................................................................... 210
Table 9.2 Value of Big Data across Industries .................................................................... 211
Table 9.3 Industry Use Cases for Big Data ......................................................................... 212
Table 10.1 MapReduce Cloud Implementations .................................................................. 233
Table 10.2 Comparison of MapReduce Implementations .................................................. 233
Table 12.1 Hadoop Ecosystem Classification by Timescale and General
Purpose of Usage .................................................................................................. 283
Table 15.1 Comparison between Web 1.0 and Web 2.0 ...................................................... 362
Table 17.1 Evolution of Wireless Networks ..........................................................................404
Table 17.2 Comparison of Mobile Operating Systems ....................................................... 407
Table 18.1 Location-Based Services (LBS) Classification ................................................... 424
Table 18.2 LBS Quality of Service (QOS) Requirements ....................................................434
Table 18.3 Location Enablement Technologies .................................................................... 437
Table 18.4 Accuracy and TIFF for Several Location Techniques ...................................... 438

xxiii
This page intentionally left blank
Random documents with unrelated
content Scribd suggests to you:
The Project Gutenberg eBook of The Mannion
Court-Martial
This ebook is for the use of anyone anywhere in the United States
and most other parts of the world at no cost and with almost no
restrictions whatsoever. You may copy it, give it away or re-use it
under the terms of the Project Gutenberg License included with this
ebook or online at www.gutenberg.org. If you are not located in the
United States, you will have to check the laws of the country where
you are located before using this eBook.

Title: The Mannion Court-Martial

Author: Randall Garrett

Release date: May 18, 2021 [eBook #65370]

Language: English

Credits: Greg Weeks, Mary Meehan and the Online Distributed

Proofreading Team at http://www.pgdp.net

*** START OF THE PROJECT GUTENBERG EBOOK THE MANNION

COURT-MARTIAL ***
The Mannion Court-Martial
 By Randall Garrett

Why would a Space Officer lead an android

rebellion? Even Lieut. Mannion believed he was
guilty as they gave him the supreme penalty....

[Transcriber's Note: This etext was produced from

Imagination Stories of Science and Fantasy
October 1957
Extensive research did not uncover any evidence that
the U.S. copyright on this publication was renewed.]
Lieut. Dan Mannion of the Earth Space Patrol stood in the prisoner's
dock in the courtroom, gripping the rail of his cubicle so hard his
fingers hurt.
Comdr. Edward Harkness of the SP, who was presiding, glared at him
sternly. "Lieutenant Mannion, the charges against you are severe.
You face the risk of total mnemonic erasure if found guilty. Is there
anything you care to say in your own defense before we proceed
with the trial?"
Mannion glanced around the military courtroom, seeing the pale,
tense, anxious face of his wife Virginia, the stern countenance of
Dubrow, his former commanding officer, the interested eyes of half a
hundred onlookers.
"No," he said. His voice was thin and dry. "There's nothing I can say.
Nothing at all."
He saw Virginia's pleading eyes. She was telling him silently, Please,
Dan. Tell them you're innocent. At least put up a defense!
"Call the witness," Commander Harkness ordered.
"Base Commandant Lee Dubrow will please take the witness stand."
While Dubrow was being sworn in, Mannion studied him. His former
commander on the Iapetus base was a tall, icy-faced man with
close-cropped gray hair and a stiff military mustache. Mannion had
never been particularly friendly with his commanding officer.
"Commander Dubrow, will you relate the events leading up to
Lieutenant Mannion's actions in the Android Rebellion?"
Dubrow cleared his throat. "Very well. As you know, the Space Patrol
established its base on Iapetus last year—no, two years ago, at the
end of 2365—as part of its program of preparing Saturn's moons for
colonization."
"How many members of the patrol were with you?"
"Fifteen, altogether. I was in command, naturally, and for most of
the period we were there Lieutenant Mannion was my second-in-
command."
"Isn't it fairly unusual for a Lieutenant to hold such a high position?"
the prosecutor asked.
"Major Dunphy was killed by a rebellious android seven weeks after
we arrived," Dubrow said. "Lieutenant Mannion was the next highest
ranking officer in my squadron and he took over."
"How many androids did you have with you?"
"Over a hundred," said Dubrow. "It was quite a time we had when
they mutinied."
"Had you any knowledge of the mutiny beforehand?"
"No."
"Did any member of your staff know about the mutiny before it took
place?"
"Yes."
"Who?"
"Lieutenant Mannion. He was in conspiracy with VZ-1972, the ring-
leader of the mutiny."

Mannion felt his face go bright red. He wanted to stand up and

shout, "That's a lie! I never knew anything about the mutiny!"
But he couldn't. Somewhere in the back of his mind lay a shadow of
doubt. He could not remember anything that had happened at the
time of the mutiny—and perhaps he had—perhaps—
The judge said, "Tell us about Lieutenant Mannion's part in the
mutiny."
"Yes, sir. The first we knew about it was on the morning of
November 9, 2366, when the androids we used to keep the
atmosphere-generators running refused to perform their regular
tasks. I ordered Lieutenant Mannion to go outside and discover what
the trouble was. He refused. I ordered him a second time, and he
struck me and threw open the airlock. All of the androids rushed in."
"What happened then?"
"I found myself wrestling with Lieutenant Mannion while the
androids destroyed all of the Project's equipment and apparatus. In
the struggle all 12 of my men were killed by the androids. Finally I
succeeded in subduing Lieutenant Mannion and bringing the
androids back under control—"
"How was that done?"
"The androids respond automatically to a direct command from the
superior officer, no matter what they're doing. Had I been free to
give that command the mutiny would never have taken place. But
Lieutenant Mannion prevented me from giving the command until it
was too late. All of our men were dead and the Project set back
more than a year. I placed Lieutenant Mannion under detention, put
the androids in permafreeze, and returned to Earth. And here I am."
"Is that the extent of your testimony?"
"It is."
"You may step down, then. Lieutenant Mannion?"
Mannion rose and faced the judge. "Sir?"
"You've now heard your commander testify that you wilfully
obstructed his attempt to end the android mutiny ... a mutiny which
cost 12 human lives and did over $5,000,000 worth of damage to
the Iapetus Project. Is there, again, anything you care to say in your
own defense?"
Mannion shook his head. "No, sir."
"Very well, then. The court will adjourn for 15 minutes while data is
programmed and fed to the computer, after which the verdict will be
announced and the sentence read."

Mannion left the stand and felt his wife Virginia come up to him and
hold him tightly.
"Dan, Dan, why don't you say something? Dubrow's testimony is
damning if you don't speak up!"
Mannion frowned. "But I don't remember, Virginia! My mind is a
blank for the entire period of the mutiny! For all I know I did do as
the Commander says!"
"Impossible, Dan! You were always so loyal to the Patrol—"
"I still am," he said. "And if I committed this crime I deserve to be
punished for it."
"Do you know what the punishment is?"
"Mnemonic erasure," Mannion said.
"No! Do you know what mnemonic erasure means? They'll strip
away all your memories, everything but the basic pattern of your
reflexes and reactions. Everything that is Dan Mannion will be
erased, discarded, thrown away." Tears appeared in the corners of
her eyes. "I'll be declared a widow, officially. And your body will be
given a new name, a different identity. You'll be re-educated as
someone else."
Mannion nodded bleakly. "I know. What can I do? Dubrow's my
Commander; he has to be telling the truth. I don't remember
anything. Perhaps I went temporarily out of my mind, did an insane
thing, and now my consciousness has blanked out that period. It
doesn't matter. I killed 12 men by my actions, Ginny."
"No! No!"
"I'm afraid so," Mannion said. "And I'll take my punishment for it
now."
He turned away, not wanting to see his wife's tearstreaked face. A
torrent of conflicting emotions raged within him despite the calm
exterior he maintained. All his life he had dreamed of the Patrol and
its glory; he had worked toward that one end. Four years at the
Academy, two more in apprentice-work, then finally the commission
and the assignment to Iapetus.
And what happened? A moment of insanity, perhaps—or downright
conspiracy with an android to overthrow the Project by violence? He
didn't know. He would never know. All he knew was he had done
some mad act and now he would pay for it. His marriage, his career,
even his identity itself, would be taken from him.
An orderly touched his arm. "The court's returning to order,
Lieutenant Mannion. Please resume your place."
"Sure. Sure, I'm going." He kissed his wife tenderly and started up
the row of steps to take his place in the prisoner's dock.

Commander Harkness was staring grimly at him. The verdict, when

it came, would be no surprise; from the nature of Mannion's lack of
defense, it would be a foregone certainty.
"Lieutenant Mannion, you're aware of the nature of the crime you're
charged with?"
"Yes, sir."
"The only witness against you has been your former Commander,
Lee Dubrow. You have not made any statements in your own
defense."
"No, sir."
"In view of this situation, the court has no recourse but to find you
guilty of insubordination in the highest degree, conspiracy, malicious
attack upon an officer with intent to aid in mutiny."
Mannion bowed his head. "Yes, sir," he said in a half-audible tone.
"The punishment for these crimes in necessarily severe," the judge
went on. "Naturally, we're unable to put into effect what would
normally have been the penalty 300 years ago. The death penalty is
obsolete. However, I hereby pronounce a sentence amounting to
execution upon the personality, mind, and accumulated memories of
the man formerly known as Daniel Mannion."
"You mean mnemonic erasure, sir?"
"Obviously. This sentence automatically carries with it loss of all
privileges, pensions, and honors that go with your high rank in the
Space Patrol. Your name will be wiped from its roster. After the
erasure, you will never have existed, Lieutenant Mannion. Your body
will be restrained under a new name and will make a fresh start in
life. It will even be possible for your new personality to enter the
Space Patrol, if it so chooses. No prejudice against your body will be
entertained for your mind's previous acts."
In the background, Mannion heard his wife's faint sobbing. "I hear
and accept the sentence, sir," he said quietly.
"The act of erasure will be carried out immediately, in the Space
Patrol's mnemonic laboratories on the 14th level of this building."
The gavel rapped three times. "The case of Earth versus Daniel
Mannion is hereby considered closed."
"No!" Virginia suddenly shouted. All eyes in the courtroom swivelled
to focus on her as she rose from the audience to protest. "No, don't
close the case yet!"
"This is highly irregular," said Judge Harkness. "Do you have
additional testimony, Mrs. Mannion?"
"Not—testimony, your honor. But can't you see that Dan's obviously
insane? He's allowing himself to be sentenced without even a
protest! Can't he enter a plea of insanity?"
"The plea of insanity would not alter the judgment in any way, Mrs.
Mannion. Rest assured that your husband's—ah—disturbed mental
state has been taken into account in the decision. Whether he was
insane or criminally possessed at the time of the mutiny makes no
difference; the crime has been committed, obviously, and the guilty
person is of no further value to society. Mnemonic erasure is not
merely a punishment, Mrs. Mannion. It's the gateway to
rehabilitation for a sick person."
"I—see. May I say goodbye to my husband before you—erase him?"

Going down in the lift tube from the courtroom on the 60th floor of
Patrol headquarters to the lab on Level Fourteen, Mannion felt
strangely numb inside.
Two Patrol members stood behind him, ready to go for blasters if he
made the slightest move toward escaping. But Mannion had no idea
of escaping.
He was on his way to be erased.
He wondered what erasure was like. Did it hurt? Did you feel the
pain as they stripped away layer after layer of your memory like
peelings from an onion? First 2367 would go, but the new year was
only two weeks old and he'd spent those two weeks in prison. Then
2366 would vanish—but 2366 was partly gone, at least for the few
hours of the Mutiny. Next would go 2365, the year they first landed
on Iapetus.
And so, ever backward, they would tear away more and more of the
accumulation of memories and experiences that was Dan Mannion.
2364, 2363.
2362. That was the year he met Virginia. They would take away his
courtship, his wedding, those wonderful early days of marriage—
The two years as a Patrol Apprentice would go. The four years at the
Academy.
Adolescence. Boyhood. Childhood.
Soon there would be nothing left of Dan Mannion but a few vague
memories of babyhood, and then even those would be gone. He
would emerge from the lab wiped blank, a fresh unmarked slate
ready to be given its new identity.
Suddenly, he found himself quivering.
I'm not guilty! I didn't do it! I couldn't have done it!
Too late, a voice said. He saw again the faces of Virginia, of
Commander Harkness, of stern-faced Dubrow giving the testimony
that damned him.
Too late. Too late to defend yourself.
"Fourteen," the robobrain of the elevator announced. The door slid
back. Mannion felt light pressure behind each of his arms as his two
guards shoved him gently forward.
A frosted glass door loomed up ahead of him. The sign on the door
read Mnemonics Laboratory.
Cold sweat drenched his body. Now that he was but feet away from
the room where the erasure would take place, he wanted out
desperately, wanted some chance to prove that he hadn't conspired
with the androids, hadn't aided in the revolt, hadn't helped to
murder 12 fellow Patrolmen and wreck the Iapetus project.
"You go in here," someone said to him.
The door marked Mnemonics Laboratory was swinging open to
receive him.
There was no way out.
Four gray-smocked technicians waited inside for him. One of the
guards with him said, "This is Mannion. He was just sentenced
upstairs."
"I know. The order came down the pneumotubes a minute ago.
Total erasure."
"That's right," the guard said. "He gets wiped clean."
"Will you lead him to the machines, please?"
Dan went forward and faced a complex angle of probes and dials,
"Is this the machine that does it?" he asked uneasily.
"That's right. It'll be over in a minute, Lieutenant Mannion. We
clamp the electrodes to your scalp and run preliminary tests with an
electoencephalograph—and then we use the Eraser."
"Will it be painful?"
"It'll be quick. There won't be anything more than a faint tickling
sensation, and then—"
"Then Dan Mannion ceases to exist." He stared appealingly at the
technician in charge and said, "Listen—does the sentence have to be
carried out at once?"
"The order says immediately. We have the machine all ready for
you."
Dan felt perspiration trickling down his body. "Can you wait a few
minutes? There's something I'd like you to do for me?"
"What's that?"
"Probe my mind. I'm suffering from amnesia—a short-range
blockage of the critical era around the time the android mutiny took
place. Couldn't you—?"
"Impossible. Not without a court order, at any rate. And the trial's
over."
Dan scowled. "But my life depends on it! My identity is going to be
taken away. The least you could have done was look!"
"Come on, Mannion," one of the guards growled. "The time to make
your pitch is during the trial, not after the sentence has been
pronounced." Dan felt himself shoved forward.
The machine loomed up before him—gigantic, monstrous, a
mindless instrument of horror. Within minutes he was going to
undergo mnemonic erasure, to have his mind blanked, his identity
removed—
For a crime I didn't commit!
Suddenly he felt sure of his innocence. Despite the evidence, despite
the testimony, he knew in his heart that he was innocent.
It was a frameup of some sort. It had to be.
He allowed himself to be led up to the machine. But abruptly, as
they were unhinging some apparatus to strap to his head, he spun
away from the guards who held him lightly, dove, grabbed at a
blaster that protruded from a black leather holster—
"Okay," he said. "Get against that wall, all of you. One move I don't
like and I'll destroy the whole lab."

His fingers were shaking with inner tension. All his life he had been
raised to obey authority, to accept the commands of his superior
officers—
And now he was rebelling. He was threatening the destruction of
one of Earth's most expensive pieces of equipment.
The threat worked. The four technicians and the two guards backed
against the wall.
"What do you want?" the head technician asked.
"I told you before. I want you to probe my mind, to look into that
period that's a blank for me. If you find that I'm guilty, I'll—I'll
submit to the erasure. If not, I'll demand a new trial. But I won't
allow myself to be wiped out without at least a look!"
"All right. We'll probe you," said the technician. "You'll have to be
under anesthetic, of course."
"How can I trust you? How do I know you won't put me through
mnemonic erasure the moment I submit to being anesthetized?"
The technician had no answer. "I'll tell you," Mannion said. "You're
all doctors, aren't you? All four?"
They nodded.
"All right, then. I'll rely on your oaths as medical men not to put me
through erasure until you've probed that mutiny fully. Well?"
"Okay, Mannion. We'll take a look. But if it's not as you say—"
"I'll take my chances," Mannion said. He felt cold and uncertain
inside. He didn't know what they'd find. He didn't even know
whether they'd keep their word and probe him before the erasure.
He put the gun down on a lab table. "Here," he said. "Here's my
gun. Now let's see how good your oath is."
The only trouble with that was he might never see how good it was.

"Just relax," the technician said. "The probe is entering your mind,
now. Just relax...."
Mannion sank downward into the soft, warm darkness that enfolded
him. He was moving back into his own past now, gently guided
along by the mind-probe—
WHAM!
It was like walking full-tilt into a mountainside. Some obstruction in
his mind, no doubt.
But the probe bored its way through, drilled through the hard barrier
of amnesia in his mind.
And suddenly he was back on Iapetus, in Project Headquarters.
He was saying, "Commander Dubrow, the androids running the
atmosphere-generators are lying down on the job. They don't seem
to be working."
Dubrow glared at him coldly. "Stick to your own job, Lieutenant
Mannion. Coleridge is supervising the androids out there."
"No, he isn't! Coleridge isn't there."
"He must be there, Lieutenant."
"Commander, I'm going out there to see what's wrong. Those
androids have been acting up strangely all day and I don't like it."
"I order you to stay here!" Dubrow snapped.
"But—"
Hesitantly Mannion took a few steps toward the airlock. The
androids outside were sauntering casually around like unemployed
thieves. It wasn't a natural way for androids to behave.
"Sir, I request special permission to go out there and investig—sir!"
Dubrow was throwing open the airlock—and the androids came
rushing in!
He's crazy, Lieutenant Mannion thought. I've got to take charge—
keep those androids from wrecking the Project—
"Get away from there, sir! Close the lock!"
"Don't give me orders, Mannion!"
Dan shook his head and started to run toward his superior officer.
But suddenly Dubrow charged him.
The abrupt assault bowled him over. Dan ducked and tried to land a
punch but Dubrow had his blaster out. A blow crashed into
Mannion's forehead. He tried to clear away the cobwebs but Dubrow
hit him again and all went dim.
He had a vague memory of Dubrow's directing the androids in a
methodical destruction of the Project. Then it was all over and the
androids were back where they belonged. Dubrow was holding a
hypnomech in front of his eyes, spinning around and around, a
dizzying sleep-inducing confusing blare of many colors, around and
around, around and around....
And then he was asleep.
"We owe you a great apology, Lieutenant Mannion," the technician
was saying. "If you hadn't forced us to probe your mind we would
have sent an innocent man to mnemonic erasure. But now we have
the record of what actually happened—"
"Hang on to it," Mannion said. "I've got to get upstairs and find
Dubrow before he gets out of here."
Without waiting for a word of protest, Dan threw off the mind-probe
apparatus, jumped off the table, and raced out into the hall.
He caught the lift tube going up. In all likelihood Dubrow, Virginia,
and the judge would still be in the courtroom, working out some
settlement of the former Lieutenant Mannion's private property.
He was right.
"Mannion! What are you doing—"
Dan ignored the judge's outcry. "Hello, Dubrow. I just had some of
my amnesia removed. That was a pretty clever story you told, wasn't
it?"
"I don't know what you're talking about, Mannion."
"The hell you don't! You don't know anything about the hypnomech
you used to block my mind and—"
Dan ducked suddenly as a spurt of energy from the proton-gun in
Dubrow's hand seared through the wall behind him. Dubrow was
aiming the gun, readying to fire again—
And Judge Harkness rose from the bench and hurled a heavy law-
book at him.
It struck Dubrow squarely on the side of the head; the bolt of
proton-force squirted toward the ceiling and Dan leaped forward.
He crashed into Dubrow and knocked the tall officer sprawling; the
proton-gun clattered to the floor. Dubrow squirmed and kicked but
Dan's fists thundered against his body.
"Hypnotize me, will you? And try to frame me for that mutiny? I'll—"
"All right, Mannion," a calm voice said from somewhere above him.
"You can get off him now. He's out cold."

Judge Harkness faced Dan and Virginia Mannion. "I don't understand
why you didn't speak up, son."
"I—I assumed I was wrong, sir. I've always been trained to respect
the word of an officer. If Commander Dubrow said I was guilty and I
didn't remember—well, sir, he had to be right!"
Harkness chuckled. "You know differently now. We've had a mind-
probe run on Dubrow. It seems he was bribed by a group of private
contractors to wreck the Patrol's project on Iapetus so they could get
the job instead. He figured he'd have you tried for the crime, leaving
him in the clear. So all he did was switch the action around and then
hypnotize you into forgetting it."
"What's going to happen to him now?" Mannion asked.
"What else? He's being erased now. Commander Dubrow no longer
exists."
Mannion shuddered. He remembered vividly that complex pile of
machinery on the 14th Level.
"I guess I'm free, then," he said.
Harkness nodded. "I guess you are, young man. And next time don't
be so ready to believe your own guilt."
"No, sir! I mean—yes, sir! I mean—"
It didn't matter. Mannion smiled at Harkness and took his wife in his
arms. The case was closed and he was a free man and an officer in
the Space Patrol.
And he was still Dan Mannion.
 *** END OF THE PROJECT GUTENBERG EBOOK THE MANNION
COURT-MARTIAL ***

Updated editions will replace the previous one—the old editions will
be renamed.

Creating the works from print editions not protected by U.S.

copyright law means that no one owns a United States copyright in
these works, so the Foundation (and you!) can copy and distribute it
in the United States without permission and without paying
copyright royalties. Special rules, set forth in the General Terms of
Use part of this license, apply to copying and distributing Project
Gutenberg™ electronic works to protect the PROJECT GUTENBERG™
concept and trademark. Project Gutenberg is a registered trademark,
and may not be used if you charge for an eBook, except by following
the terms of the trademark license, including paying royalties for use
of the Project Gutenberg trademark. If you do not charge anything
for copies of this eBook, complying with the trademark license is
very easy. You may use this eBook for nearly any purpose such as
creation of derivative works, reports, performances and research.
Project Gutenberg eBooks may be modified and printed and given
away—you may do practically ANYTHING in the United States with
eBooks not protected by U.S. copyright law. Redistribution is subject
to the trademark license, especially commercial redistribution.

START: FULL LICENSE

THE FULL PROJECT GUTENBERG LICENSE
 PLEASE READ THIS BEFORE YOU DISTRIBUTE OR USE THIS WORK

To protect the Project Gutenberg™ mission of promoting the free

distribution of electronic works, by using or distributing this work (or
any other work associated in any way with the phrase “Project
Gutenberg”), you agree to comply with all the terms of the Full
Project Gutenberg™ License available with this file or online at
www.gutenberg.org/license.

Section 1. General Terms of Use and

Redistributing Project Gutenberg™
electronic works
1.A. By reading or using any part of this Project Gutenberg™
electronic work, you indicate that you have read, understand, agree
to and accept all the terms of this license and intellectual property
(trademark/copyright) agreement. If you do not agree to abide by all
the terms of this agreement, you must cease using and return or
destroy all copies of Project Gutenberg™ electronic works in your
possession. If you paid a fee for obtaining a copy of or access to a
Project Gutenberg™ electronic work and you do not agree to be
bound by the terms of this agreement, you may obtain a refund
from the person or entity to whom you paid the fee as set forth in
paragraph 1.E.8.

1.B. “Project Gutenberg” is a registered trademark. It may only be

used on or associated in any way with an electronic work by people
who agree to be bound by the terms of this agreement. There are a
few things that you can do with most Project Gutenberg™ electronic
works even without complying with the full terms of this agreement.
See paragraph 1.C below. There are a lot of things you can do with
Project Gutenberg™ electronic works if you follow the terms of this
agreement and help preserve free future access to Project
Gutenberg™ electronic works. See paragraph 1.E below.
1.C. The Project Gutenberg Literary Archive Foundation (“the
Foundation” or PGLAF), owns a compilation copyright in the
collection of Project Gutenberg™ electronic works. Nearly all the
individual works in the collection are in the public domain in the
United States. If an individual work is unprotected by copyright law
in the United States and you are located in the United States, we do
not claim a right to prevent you from copying, distributing,
performing, displaying or creating derivative works based on the
work as long as all references to Project Gutenberg are removed. Of
course, we hope that you will support the Project Gutenberg™
mission of promoting free access to electronic works by freely
sharing Project Gutenberg™ works in compliance with the terms of
this agreement for keeping the Project Gutenberg™ name associated
with the work. You can easily comply with the terms of this
agreement by keeping this work in the same format with its attached
full Project Gutenberg™ License when you share it without charge
with others.

1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside the
United States, check the laws of your country in addition to the
terms of this agreement before downloading, copying, displaying,
performing, distributing or creating derivative works based on this
work or any other Project Gutenberg™ work. The Foundation makes
no representations concerning the copyright status of any work in
any country other than the United States.

1.E. Unless you have removed all references to Project Gutenberg:

1.E.1. The following sentence, with active links to, or other

immediate access to, the full Project Gutenberg™ License must
appear prominently whenever any copy of a Project Gutenberg™
work (any work on which the phrase “Project Gutenberg” appears,
or with which the phrase “Project Gutenberg” is associated) is
accessed, displayed, performed, viewed, copied or distributed:
 This eBook is for the use of anyone anywhere in the United
States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it away
or re-use it under the terms of the Project Gutenberg License
included with this eBook or online at www.gutenberg.org. If you
are not located in the United States, you will have to check the
laws of the country where you are located before using this
eBook.

1.E.2. If an individual Project Gutenberg™ electronic work is derived

from texts not protected by U.S. copyright law (does not contain a
notice indicating that it is posted with permission of the copyright
holder), the work can be copied and distributed to anyone in the
United States without paying any fees or charges. If you are
redistributing or providing access to a work with the phrase “Project
Gutenberg” associated with or appearing on the work, you must
comply either with the requirements of paragraphs 1.E.1 through
1.E.7 or obtain permission for the use of the work and the Project
Gutenberg™ trademark as set forth in paragraphs 1.E.8 or 1.E.9.

1.E.3. If an individual Project Gutenberg™ electronic work is posted

with the permission of the copyright holder, your use and distribution
must comply with both paragraphs 1.E.1 through 1.E.7 and any
additional terms imposed by the copyright holder. Additional terms
will be linked to the Project Gutenberg™ License for all works posted
with the permission of the copyright holder found at the beginning
of this work.

1.E.4. Do not unlink or detach or remove the full Project

Gutenberg™ License terms from this work, or any files containing a
part of this work or any other work associated with Project
Gutenberg™.

1.E.5. Do not copy, display, perform, distribute or redistribute this

electronic work, or any part of this electronic work, without
prominently displaying the sentence set forth in paragraph 1.E.1
with active links or immediate access to the full terms of the Project
Gutenberg™ License.

1.E.6. You may convert to and distribute this work in any binary,
compressed, marked up, nonproprietary or proprietary form,
including any word processing or hypertext form. However, if you
provide access to or distribute copies of a Project Gutenberg™ work
in a format other than “Plain Vanilla ASCII” or other format used in
the official version posted on the official Project Gutenberg™ website
(www.gutenberg.org), you must, at no additional cost, fee or
expense to the user, provide a copy, a means of exporting a copy, or
a means of obtaining a copy upon request, of the work in its original
“Plain Vanilla ASCII” or other form. Any alternate format must
include the full Project Gutenberg™ License as specified in
paragraph 1.E.1.

1.E.7. Do not charge a fee for access to, viewing, displaying,

performing, copying or distributing any Project Gutenberg™ works
unless you comply with paragraph 1.E.8 or 1.E.9.

1.E.8. You may charge a reasonable fee for copies of or providing

access to or distributing Project Gutenberg™ electronic works
provided that:

• You pay a royalty fee of 20% of the gross profits you derive
from the use of Project Gutenberg™ works calculated using the
method you already use to calculate your applicable taxes. The
fee is owed to the owner of the Project Gutenberg™ trademark,
but he has agreed to donate royalties under this paragraph to
the Project Gutenberg Literary Archive Foundation. Royalty
payments must be paid within 60 days following each date on
which you prepare (or are legally required to prepare) your
periodic tax returns. Royalty payments should be clearly marked
as such and sent to the Project Gutenberg Literary Archive
Foundation at the address specified in Section 4, “Information