Download Full Version ebookmass - Visit ebookmass.

com

Python Debugging for AI, Machine Learning, and

Cloud Computing: A Pattern-Oriented Approach 1st
Edition Vostokov

https://ebookmass.com/product/python-debugging-for-ai-
machine-learning-and-cloud-computing-a-pattern-oriented-
approach-1st-edition-vostokov/

OR CLICK HERE

DOWLOAD NOW

Discover More Ebook - Explore Now at ebookmass.com

 Instant digital products (PDF, ePub, MOBI) ready for you
Download now and discover formats that fit your needs...

Python Debugging for AI, Machine Learning, and Cloud

Computing: A Pattern-Oriented Approach 1st Edition Dmitry
Vostokov
https://ebookmass.com/product/python-debugging-for-ai-machine-
learning-and-cloud-computing-a-pattern-oriented-approach-1st-edition-
dmitry-vostokov/
ebookmass.com

Fundamentals of Trace and Log Analysis: A Pattern-Oriented

Approach to Monitoring, Diagnostics, and Debugging 1st
Edition Dmitry Vostokov
https://ebookmass.com/product/fundamentals-of-trace-and-log-analysis-
a-pattern-oriented-approach-to-monitoring-diagnostics-and-
debugging-1st-edition-dmitry-vostokov-2/
ebookmass.com

Fundamentals of Trace and Log Analysis A Pattern-Oriented

Approach to Monitoring, Diagnostics, and Debugging 1st
Edition Dmitry Vostokov
https://ebookmass.com/product/fundamentals-of-trace-and-log-analysis-
a-pattern-oriented-approach-to-monitoring-diagnostics-and-
debugging-1st-edition-dmitry-vostokov/
ebookmass.com

The Six-Word Secret to Success Earl Nightingale

https://ebookmass.com/product/the-six-word-secret-to-success-earl-
nightingale/

ebookmass.com
Diagnostic Imaging: Brain 4th Edition Miral D. Jhaveri

https://ebookmass.com/product/diagnostic-imaging-brain-4th-edition-
miral-d-jhaveri/

ebookmass.com

Hair Transplantation 5th Edition

https://ebookmass.com/product/hair-transplantation-5th-edition/

ebookmass.com

Over and Under the Waves Kate Messner

https://ebookmass.com/product/over-and-under-the-waves-kate-messner/

ebookmass.com

Civil Engineering PE Practice Exams: Breadth and Depth 2nd

Edition Indranil Goswami

https://ebookmass.com/product/civil-engineering-pe-practice-exams-
breadth-and-depth-2nd-edition-indranil-goswami/

ebookmass.com

Antioxidants Effects in Health: The Bright and the Dark

Side Seyed Mohammad Nabavi

https://ebookmass.com/product/antioxidants-effects-in-health-the-
bright-and-the-dark-side-seyed-mohammad-nabavi/

ebookmass.com
Security Awareness For Dummies (For Dummies
(Computer/Tech)) 1st Edition Winkler

https://ebookmass.com/product/security-awareness-for-dummies-for-
dummies-computer-tech-1st-edition-winkler/

ebookmass.com
Dmitry Vostokov

Python Debugging for AI, Machine

Learning, and Cloud Computing
A Pattern-Oriented Approach
Dmitry Vostokov
Dalkey, Dublin, Ireland

ISBN 978-1-4842-9744-5 e-ISBN 978-1-4842-9745-2

https://doi.org/10.1007/978-1-4842-9745-2

© Dmitry Vostokov 2024

Apress Standard

The use of general descriptive names, registered names, trademarks,

service marks, etc. in this publication does not imply, even in the
absence of a specific statement, that such names are exempt from the
relevant protective laws and regulations and therefore free for general
use.

The publisher, the authors, and the editors are safe to assume that the
advice and information in this book are believed to be true and accurate
at the date of publication. Neither the publisher nor the authors or the
editors give a warranty, expressed or implied, with respect to the
material contained herein or for any errors or omissions that may have
been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Apress imprint is published by the registered company APress

Media, LLC, part of Springer Nature.
The registered company address is: 1 New York Plaza, New York, NY
10004, U.S.A.
To Ekaterina, Alexandra, Kirill, and Maria
Introduction
Python is the dominant language used in AI and machine learning with
data and pipelines in cloud environments. Besides debugging Python
code in popular IDEs, notebooks, and command-line debuggers, this
book also includes coverage of native OS interfacing (Windows and
Linux) necessary to understand, diagnose, and debug complex software
issues.
The book begins with an introduction to pattern-oriented software
diagnostics and debugging processes that, before doing Python
debugging, diagnose problems in various software artifacts such as
memory dumps, traces, and logs. Next, it teaches various debugging
patterns using Python case studies that model abnormal software
behavior. Further, it covers Python debugging specifics in cloud native
and machine learning environments. It concludes with how recent
advances in AI/ML can help in Python debugging. The book also goes
deep for case studies when there are environmental problems, crashes,
hangs, resource spikes, leaks, and performance degradation. It includes
tracing and logging besides memory dumps and their analysis using
native WinDbg and GDB debuggers.
This book is for those who wish to understand how Python
debugging is and can be used to develop robust and reliable AI,
machine learning, and cloud computing software. It uses a novel
pattern-oriented approach to diagnosing and debugging abnormal
software structure and behavior. Software developers, AI/ML
engineers, researchers, data engineers, MLOps, DevOps, and anyone
who uses Python will benefit from this book.
Source Code: All source code used in this book can be downloaded
from github.com/Apress/Python-Debugging-for-AI-
Machine-Learning-and-Cloud-Computing.
Any source code or other supplementary material referenced by the
author in this book is available to readers on GitHub. For more detailed
information, please visit https://www.apress.com/gp/services/source-
code.
Table of Contents
Chapter 1:​Fundamental Vocabulary
Process
Thread
Stack Trace (Backtrace, Traceback)
Symbol Files
Module
Memory Dump
Crash
Hang
Summary
Chapter 2:​Pattern-Oriented Debugging
The History of the Idea
Patterns and Analysis Patterns
Development Process
Development Patterns
Debugging Process and Patterns
Elementary Diagnostics Patterns
Debugging Analysis Patterns
Debugging Architecture Patterns
Debugging Design Patterns
Debugging Implementation Patterns
Debugging Usage Patterns
Debugging Presentation Patterns
Summary
Chapter 3:​Elementary Diagnostics Patterns
 Functional Patterns
Use-Case Deviation
Non-Functional Patterns
Crash
Hang
Counter Value
Error Message
Summary
Chapter 4:​Debugging Analysis Patterns
Paratext
State Dump
Counter Value
Stack Trace Patterns
Stack Trace
Runtime Thread
Managed Stack Trace
Source Stack Trace
Stack Trace Collection
Stack Trace Set
Exception Patterns
Managed Code Exception
Nested Exception
Exception Stack Trace
Software Exception
Module Patterns
Module Collection
 Not My Version
Exception Module
Origin Module
Thread Patterns
Spiking Thread
Active Thread
Blocked Thread
Blocking Module
Synchronization Patterns
Wait Chain
Deadlock
Livelock
Memory Consumption Patterns
Memory Leak
Handle Leak
Case Study
Summary
Chapter 5:​Debugging Implementation Patterns
Overview of Patterns
Break-Ins
Code Breakpoint
Code Trace
Scope
Variable Value
Type Structure
Breakpoint Action
 Usage Trace
Case Study
Elementary Diagnostics Patterns
Debugging Analysis Patterns
Debugging Implementation Patterns
Summary
Chapter 6:​IDE Debugging in the Cloud
Visual Studio Code
WSL Setup
Cloud SSH Setup
Case Study
Summary
Chapter 7:​Debugging Presentation Patterns
Python Debugging Engines
Case Study
Suggested Presentation Patterns
Summary
Chapter 8:​Debugging Architecture Patterns
The Where?​Category
In Papyro
In Vivo
In Vitro
In Silico
In Situ
Ex Situ
The When?​Category
 Live
JIT
Postmortem
The What?​Category
Code
Data
Interaction
The How?​Category
Software Narrative
Software State
Summary
Chapter 9:​Debugging Design Patterns
CI Build Case Study
Elementary Diagnostics
Analysis
Architecture
Design
Implementation
Data Processing Case Study
Elementary Diagnostics
Analysis
Architecture
Design
Implementation
Summary
Chapter 10:​Debugging Usage Patterns
 Exact Sequence
Scripting
Debugger Extension
Abstract Command
Space Translation
Lifting
Gestures
Summary
Chapter 11:​Case Study:​Resource Leaks
Elementary Diagnostics
Debugging Analysis
Debugging Architecture
Debugging Implementation
Summary
Chapter 12:​Case Study:​Deadlock
Elementary Diagnostics
Debugging Analysis
Debugging Architecture
Exceptions and Deadlocks
Summary
Chapter 13:​Challenges of Python Debugging in Cloud Computing
Complex Distributed Systems
Granularity of Services
Communication Channels Overhead
Inter-Service Dependencies
Layers of Abstraction
 Opaque Managed Services
Serverless and Function as a Service
Container Orchestration Platforms
Continuous Integration/​Continuous Deployment
Pipeline Failures
Rollbacks and Versioning
Immutable Infrastructure
Diversity of Cloud Service Models
Infrastructure as a Service
Platform as a Service
Software as a Service
Evolving Cloud Platforms
Adapting to Changes
Staying Updated
Environment Parity
Library and Dependency Disparities
Configuration Differences
Underlying Infrastructure Differences
Service Variabilities
Limited Visibility
Transient Resources
Log Management
Monitoring and Alerting
Latency and Network Issues
Network Instabilities
Service-to-Service Communication
 Resource Leaks and Performance
Resource Starvation
Concurrency Issues
Race Conditions
Deadlocks
Security and Confidentiality
Debugger Access Control Restrictions
Sensitive Data Exposure
Limited Access
Cost Implications
Extended Sessions
Resource Provisioning and Deprovisioning
Data Transfer and Storage Fees
State Management
Stateful Services
Data Volume
Limited Tooling Compatibility
Versioning Issues
Deprecations and Changes
SDK and Library Updates
Real-time Debugging and User Experience
External Service Dependencies
Dependency Failures
Rate Limiting and Quotas
Asynchronous Operations
Flow Tracking
 Error Propagation
Scaling and Load Challenges
Load-Based Issues
Resource Contention
Multi-Tenancy Issues
Resource Contention
Data Security
Reliability and Redundancy Issues
Service Failures
Data Durability
Summary
Chapter 14:​Challenges of Python Debugging in AI and Machine
Learning
The Nature of Defects in AI/​ML
Complexity and Abstraction Layers
Non-Determinism and Reproducibility
Large Datasets
High-Dimensional Data
Long Training Times
Real-Time Operation
Model Interpretability​
Hardware Challenges
Version Compatibility and Dependency Hell
Data Defects
Inconsistent and Noisy Data
Data Leakage
 Imbalanced Data
Data Quality
Feature Engineering Flaws
Algorithmic and Model-Specific Defects
Gradients, Backpropagation, and Automatic Differentiation
Hyperparameter Tuning
Overfitting and Underfitting
Algorithm Choice
Deep Learning Defects
Activation and Loss Choices
Learning Rate
Implementation Defects
Tensor Shapes
Hardware Limitations and Memory
Custom Code
Performance Bottlenecks
Testing and Validation
Unit Testing
Model Validation
Cross-Validation
Metrics Monitoring
Visualization for Debugging
TensorBoard
Matplotlib and Seaborn
Model Interpretability​
Logging and Monitoring
 Checkpoints
Logging
Alerts
Error Tracking Platforms
Collaborative Debugging
Forums and Communities
Peer Review
Documentation, Continuous Learning, and Updates
Maintaining Documentation
Library Updates
Continuous Learning
Case Study
Summary
Chapter 15:​What AI and Machine Learning Can Do for Python
Debugging
Automated Error Detection
Intelligent Code Fix Suggestions
Interaction Through Natural Language Queries
Visual Debugging Insights
Diagnostics and Anomaly Detection
Augmenting Code Reviews
Historical Information Analysis and Prognostics
Adaptive Learning and Personalized Debugging Experience
Test Suite Integration and Optimization
Enhanced Documentation and Resource Suggestions
Problem Modeling
 Generative Debugging Strategy
Help with In Papyro Debugging
Summary
Chapter 16:​The List of Debugging Patterns
Elementary Diagnostics Patterns
Debugging Analysis Patterns
Debugging Architecture Patterns
Debugging Design Patterns
Debugging Implementation Patterns
Debugging Usage Patterns
Debugging Presentation Patterns
Index
About the Author
Dmitry Vostokov
is an internationally recognized expert,
speaker, educator, scientist, inventor, and
author. He founded the pattern-oriented
software diagnostics, forensics, and
prognostics discipline (Systematic
Software Diagnostics) and Software
Diagnostics Institute (DA+TA:
DumpAnalysis.org + TraceAnalysis.org).
Vostokov has also authored multiple
books on software diagnostics, anomaly
detection and analysis, software, and
memory forensics, root cause analysis
and problem-solving, memory dump
analysis, debugging, software trace and log analysis, reverse
engineering, and malware analysis. He has over thirty years of
experience in software architecture, design, development, and
maintenance in various industries, including leadership, technical, and
people management roles. In his spare time, he presents multiple topics
on Debugging.TV and explores software narratology and its further
development as narratology of things and diagnostics of things (DoT),
software pathology, and quantum software diagnostics. His current
interest areas are theoretical software diagnostics and its mathematical
and computer science foundations, application of formal logic, artificial
intelligence, machine learning, and data mining to diagnostics and
anomaly detection, software diagnostics engineering and diagnostics-
driven development, diagnostics workflow, and interaction. Recent
interest areas also include cloud native computing, security,
automation, functional programming, applications of category theory to
software development and big data, and artificial intelligence
diagnostics.
About the Technical Reviewer
Krishnendu Dasgupta
is currently the Head of Machine
Learning at Mondosano GmbH, leading
data science initiatives focused on
clinical trial recommendations and
advanced patient health profiling
through disease and drug data. Prior to
this role, he co-founded DOCONVID AI, a
startup that leveraged applied AI and
medical imaging to detect lung
abnormalities and neurological
disorders.
With a strong background in
computer science engineering,
Krishnendu has more than a decade of experience in developing
solutions and platforms using applied machine learning. His
professional trajectory includes key positions at prestigious
organizations such as NTT DATA, PwC, and Thoucentric.
Krishnendu’s primary research interests include applied AI for
graph machine learning, medical imaging, and decentralized privacy-
preserving machine learning in healthcare. He also had the opportunity
to participate in the esteemed Entrepreneurship and Innovation
Bootcamp at the Massachusetts Institute of Technology, cohort of 2018
batch.
Beyond his professional endeavors, Krishnendu actively dedicates
his time to research, collaborating with various research NGOs and
universities worldwide. His focus is on applied AI and ML.
© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2024
D. Vostokov, Python Debugging for AI, Machine Learning, and Cloud Computing
https://doi.org/10.1007/978-1-4842-9745-2_1

1. Fundamental Vocabulary
Dmitry Vostokov1

(1) Dalkey, Dublin, Ireland

Debugging complex software issues in machine learning and cloud computing

environments requires not only the knowledge of the Python language and its interpreter
(or compiler), plus standard and external libraries, but also necessary and relevant
execution environment and operating system internals. In this chapter, you will review
some necessary fundamentals from software diagnostics and debugging languages to
have the same base level of understanding for the following chapters. In this book, I
assume that you are familiar with the Python language and its runtime environment.

Process
A Python script is interpreted by compiling it into bytecode and then executing it, or it
can even be precompiled into an application program. In both cases, this interpreter file
or the compiled application is an executable program (in Windows, it may have a .exe
extension) that references some operating system libraries (.dll in Windows and .so in
Linux). This application can be loaded into computer memory several times; each time, a
separate process is created with its own resources and unique process ID (PID, also
TGID), as shown in Figure 1-1. The process may also have a parent process that created it,
with a parent process ID (PPID).
Figure 1-1 Two python3 processes with two different PIDs
To illustrate, I executed the code in Listing 1-1 on both Windows and Linux twice.

import time

def main():
foo()

def foo():
bar()

def bar():
while True:
time.sleep(1)

if __name__ == "__main__":
main()
Listing 1-1 A Simple Script to Model Running Python Code

Figure 1-2 shows two processes on Windows.

Figure 1-2 Two running python3.11.exe processes on Windows
On Linux, you can also see two processes when you execute the same script in two
separate terminals:

~/Chapter1$ which python3

/usr/bin/python3

~/Chapter1$ ps -a
PID TTY TIME CMD
17 pts/0 00:00:00 mc
60 pts/2 00:00:00 python3
61 pts/1 00:00:00 python3
80 pts/3 00:00:00 ps

Note The operating system controls hardware and processes/threads. From a high
level, it is just a collection of processes with the operating system kernel as a process
too.

Thread
From an operating system perspective, a process is just a memory container for a Python
interpreter, its code, and data. But the interpreter code needs to be executed, for example,
to interpret the Python bytecode. This unit of execution is called a thread. A process may
have several such units of execution (several threads, the so-called multithreaded
application). Each thread has its own unique thread ID (TID, also LWP or SPID), as shown
in Figure 1-3. For example, one thread may process user interface events and others may
do complex calculations in response to UI requests, thus making the UI responsive. On
Windows, thread IDs are usually different from process IDs, but in Linux, the thread ID of
the main thread is the same as the process ID for a single-threaded process.

Figure 1-3 Two python3 processes with different numbers of threads

To model multithreading, I executed the code in Listing 1-2 on both Windows and
Linux.

import time
import threading

def thread_func():
foo()

def main():
t1 = threading.Thread(target=thread_func)
t1.start()
t2 = threading.Thread(target=thread_func)
t2.start()
t1.join()
t2.join()
def foo():
bar()

def bar():
while True:
time.sleep(1)

if __name__ == "__main__":
main()
Listing 1-2 A Simple Script to Model Multiple Threads
Figure 1-4 shows that in Windows, you can see 11 threads at the beginning (this
number later changes to 7 and then to 5). You see that the number of threads may be
greater than expected.

Figure 1-4 The number of threads in the running python3.11.exe process on Windows

In Linux, you can see the expected number of threads – 3:

~/Chapter1$ ps -aT
PID SPID TTY TIME CMD
17 17 pts/0 00:00:00 mc
45 45 pts/2 00:00:00 python3
45 46 pts/2 00:00:00 python3
45 47 pts/2 00:00:00 python3
54 54 pts/1 00:00:00 ps
Stack Trace (Backtrace, Traceback)
I should distinguish Python source code tracebacks (which we call managed stack traces)
and unmanaged (native) ones from the Python compiler and interpreter that compiles to
and executes Python byte code. You will see this distinction in some chapters for several
case studies and how to get both traces. But, for now, I will just show the difference.
Listing 1-3 shows managed stack trace. Listing 1-4 shows the corresponding unmanaged
Linux stack trace with debugging symbols (the most recent call first). Listing 1-5 shows
the corresponding unmanaged Windows stack trace without debugging symbols (the
most recent call first).

Traceback (most recent call last):

File "process.py", line 14, in <module>
main()
File "process.py", line 4, in main
foo()
File "process.py", line 7, in foo
bar()
File "process.py", line 11, in bar
time.sleep(1)
Listing 1-3 Managed Stack Trace from the Execution of the Python Script from Listing 1-1

#0 0x00007f6bc84e6b97 in __GI___select (nfds=0, readfds=0x0,

writefds=0x0, exceptfds=0x0, timeout=0x7ffc60288fe0)
at ../sysdeps/unix/sysv/linux/select.c:41
#1 0x00000000004e8965 in pysleep (secs=<optimized out>) at
../Modules/timemodule.c:1829
#2 time_sleep (self=<optimized out>, obj=<optimized out>,
self=<optimized out>, obj=<optimized out>)
at ../Modules/timemodule.c:371
#3 0x00000000005d8711 in _PyMethodDef_RawFastCallKeywords
(method=0x82dbe0 <time_methods+288>,
self=<module at remote 0x7f6bc800dc78>,
args=0x7f6bc80c4550, nargs=<optimized out>, kwnames=<optimized
out>)
at ../Objects/call.c:644
#4 0x000000000054b330 in _PyCFunction_FastCallKeywords
(kwnames=<optimized out>, nargs=<optimized out>,
args=0x7f6bc80c4550, func=<built-in method sleep of module
object at remote 0x7f6bc800dc78>)
at ../Objects/call.c:730
#5 call_function (pp_stack=0x7ffc60289150, oparg=<optimized
out>, kwnames=<optimized out>) at ../Python/ceval.c:4568
#6 0x00000000005524cd in _PyEval_EvalFrameDefault (f=
<optimized out>, throwflag=<optimized out>)
at ../Python/ceval.c:3093
#7 0x00000000005d91fc in PyEval_EvalFrameEx (throwflag=0,
f=Frame 0x7f6bc80c43d8, for file process.py, line 11, in
bar ()) at ../Python/ceval.c:547
#8 function_code_fastcall (globals=<optimized out>, nargs=
<optimized out>, args=<optimized out>, co=<optimized out>)
at ../Objects/call.c:283
#9 _PyFunction_FastCallKeywords (func=<optimized out>, stack=
<optimized out>, nargs=<optimized out>,
kwnames=<optimized out>) at ../Objects/call.c:408
#10 0x000000000054e5ac in call_function (kwnames=0x0, oparg=
<optimized out>, pp_stack=<synthetic pointer>)
at ../Python/ceval.c:4616
#11 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=
<optimized out>) at ../Python/ceval.c:3124
#12 0x00000000005d91fc in PyEval_EvalFrameEx (throwflag=0,
f=Frame 0x7f6bc80105e8, for file process.py, line 7, in foo
()) at ../Python/ceval.c:547
#13 function_code_fastcall (globals=<optimized out>, nargs=
<optimized out>, args=<optimized out>, co=<optimized out>)
at ../Objects/call.c:283
#14 _PyFunction_FastCallKeywords (func=<optimized out>, stack=
<optimized out>, nargs=<optimized out>,
kwnames=<optimized out>) at ../Objects/call.c:408
--Type <RET> for more, q to quit, c to continue without paging-
-
#15 0x000000000054e5ac in call_function (kwnames=0x0, oparg=
<optimized out>, pp_stack=<synthetic pointer>)
at ../Python/ceval.c:4616
#16 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=
<optimized out>) at ../Python/ceval.c:3124
#17 0x00000000005d91fc in PyEval_EvalFrameEx (throwflag=0,
f=Frame 0x205ade8, for file process.py, line 4, in main ())
at ../Python/ceval.c:547
#18 function_code_fastcall (globals=<optimized out>, nargs=
<optimized out>, args=<optimized out>, co=<optimized out>)
at ../Objects/call.c:283
#19 _PyFunction_FastCallKeywords (func=<optimized out>, stack=
<optimized out>, nargs=<optimized out>,
kwnames=<optimized out>) at ../Objects/call.c:408
#20 0x000000000054e5ac in call_function (kwnames=0x0, oparg=
<optimized out>, pp_stack=<synthetic pointer>)
at ../Python/ceval.c:4616
#21 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=
<optimized out>) at ../Python/ceval.c:3124
#22 0x000000000054bcc2 in PyEval_EvalFrameEx (throwflag=0,
 f=Frame 0x7f6bc80ab9f8, for file process.py, line 14, in
<module> ()) at ../Python/ceval.c:547
#23 _PyEval_EvalCodeWithName (_co=<optimized out>, globals=
<optimized out>, locals=<optimized out>,
args=<optimized out>, argcount=<optimized out>,
kwnames=0x0, kwargs=0x0, kwcount=<optimized out>, kwstep=2,
defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x0,
qualname=0x0) at ../Python/ceval.c:3930
#24 0x000000000054e0a3 in PyEval_EvalCodeEx (closure=0x0,
kwdefs=0x0, defcount=0, defs=0x0, kwcount=0, kws=0x0,
argcount=0, args=0x0, locals=<optimized out>, globals=
<optimized out>, _co=<optimized out>)
at ../Python/ceval.c:3959
#25 PyEval_EvalCode (co=<optimized out>, globals=<optimized
out>, locals=<optimized out>) at ../Python/ceval.c:524
#26 0x0000000000630ce2 in run_mod (mod=<optimized out>,
filename=<optimized out>,
globals={'__name__': '__main__', '__doc__': None,
'__package__': None, '__loader__':
<SourceFileLoader(name='__main__', path='process.py') at remote
0x7f6bc803dfd0>, '__spec__': None, '__annotations__': {},
'__builtins__': <module at remote 0x7f6bc8102c28>, '__file__':
'process.py', '__cached__': None, 'time': <module at remote
0x7f6bc800dc78>, 'main': <function at remote 0x7f6bc80791e0>,
'foo': <function at remote 0x7f6bc7f69c80>, 'bar': <function at
remote 0x7f6bc7f69d08>},
locals={'__name__': '__main__', '__doc__': None,
'__package__': None, '__loader__':
<SourceFileLoader(name='__main__', path='process.py') at remote
0x7f6bc803dfd0>, '__spec__': None, '__annotations__': {},
'__builtins__': <module at rem--Type <RET> for more, q to quit,
c to continue without paging--
ote 0x7f6bc8102c28>, '__file__': 'process.py', '__cached__':
None, 'time': <module at remote 0x7f6bc800dc78>, 'main':
<function at remote 0x7f6bc80791e0>, 'foo': <function at remote
0x7f6bc7f69c80>, 'bar': <function at remote 0x7f6bc7f69d08>},
flags=<optimized out>, arena=<optimized out>) at
../Python/pythonrun.c:1035
#27 0x0000000000630d97 in PyRun_FileExFlags (fp=0x2062390,
filename_str=<optimized out>, start=<optimized out>,
globals={'__name__': '__main__', '__doc__': None,
'__package__': None, '__loader__':
<SourceFileLoader(name='__main__', path='process.py') at remote
0x7f6bc803dfd0>, '__spec__': None, '__annotations__': {},
'__builtins__': <module at remote 0x7f6bc8102c28>, '__file__':
'process.py', '__cached__': None, 'time': <module at remote
0x7f6bc800dc78>, 'main': <function at remote 0x7f6bc80791e0>,
'foo': <function at remote 0x7f6bc7f69c80>, 'bar': <function at
remote 0x7f6bc7f69d08>},
locals={'__name__': '__main__', '__doc__': None,
'__package__': None, '__loader__':
<SourceFileLoader(name='__main__', path='process.py') at remote
0x7f6bc803dfd0>, '__spec__': None, '__annotations__': {},
'__builtins__': <module at remote 0x7f6bc8102c28>, '__file__':
'process.py', '__cached__': None, 'time': <module at remote
0x7f6bc800dc78>, 'main': <function at remote 0x7f6bc80791e0>,
'foo': <function at remote 0x7f6bc7f69c80>, 'bar': <function at
remote 0x7f6bc7f69d08>}, closeit=1, flags=0x7ffc6028989c) at
../Python/pythonrun.c:988
#28 0x00000000006319ff in PyRun_SimpleFileExFlags
(fp=0x2062390, filename=<optimized out>, closeit=1,
flags=0x7ffc6028989c) at ../Python/pythonrun.c:429
#29 0x000000000065432e in pymain_run_file (p_cf=0x7ffc6028989c,
filename=<optimized out>, fp=0x2062390)
at ../Modules/main.c:427
#30 pymain_run_filename (cf=0x7ffc6028989c,
pymain=0x7ffc60289970) at ../Modules/main.c:1627
#31 pymain_run_python (pymain=0x7ffc60289970) at
../Modules/main.c:2877
#32 pymain_main (pymain=<optimized out>, pymain=<optimized
out>) at ../Modules/main.c:3038
#33 0x000000000065468e in _Py_UnixMain (argc=<optimized out>,
argv=<optimized out>) at ../Modules/main.c:3073
#34 0x00007f6bc841a09b in __libc_start_main (main=0x4bc560
<main>, argc=2, argv=0x7ffc60289ab8, init=<optimized out>,
fini=<optimized out>, rtld_fini=<optimized out>,
stack_end=0x7ffc60289aa8) at ../csu/libc-start.c:308
#35 0x00000000005e0e8a in _start () at ../Modules/main.c:797
Listing 1-4 Unmanaged Linux Backtrace from the Execution of the Python Script from Listing 1-1 with
Debugging Symbols

00 00000090`7e1ef0a8
00007ff9`8c44fcf9 ntdll!NtWaitForMultipleObjects+0x14
01 00000090`7e1ef0b0
00007ff9`8c44fbfe KERNELBASE!WaitForMultipleObjectsEx+0xe9
02 00000090`7e1ef390
00007ff8`ef943986 KERNELBASE!WaitForMultipleObjects+0xe
03 00000090`7e1ef3d0
00007ff8`ef94383d python311!PyTraceBack_Print_Indented+0x35a
04 00000090`7e1ef430
00007ff8`ef81a6b2 python311!PyTraceBack_Print_Indented+0x211
05 00000090`7e1ef460
00007ff8`ef82fa77 python311!PyEval_EvalFrameDefault+0x8f2
06 00000090`7e1ef670
00007ff8`ef82f137 python311!PyMapping_Check+0x1eb
07 00000090`7e1ef6b0
00007ff8`ef82d80a python311!PyEval_EvalCode+0x97
08 00000090`7e1ef730
00007ff8`ef82d786 python311!PyMapping_Items+0x11e
09 00000090`7e1ef760
00007ff8`ef97a17e python311!PyMapping_Items+0x9a
0a 00000090`7e1ef7a0
00007ff8`ef7e33a5 python311!PyThread_tss_is_created+0x53ce
0b 00000090`7e1ef810
00007ff8`ef8da620 python311!PyRun_SimpleFileObject+0x11d
0c 00000090`7e1ef880
00007ff8`ef8daaef python311!PyRun_AnyFileObject+0x54
0d 00000090`7e1ef8b0
00007ff8`ef8dab5f python311!Py_MakePendingCalls+0x38f
0e 00000090`7e1ef980
00007ff8`ef8db964 python311!Py_MakePendingCalls+0x3ff
0f 00000090`7e1ef9b0
00007ff8`ef8db7f5 python311!Py_RunMain+0x184
10 00000090`7e1efa20
00007ff8`ef8260d9 python311!Py_RunMain+0x15
11 00000090`7e1efa50
00007ff6`aefe1230 python311!Py_Main+0x25
12 00000090`7e1efaa0 00007ff9`8e1c26ad python+0x1230
13 00000090`7e1efae0
00007ff9`8ef6a9f8 KERNEL32!BaseThreadInitThunk+0x1d
14 00000090`7e1efb10
00000000`00000000 ntdll!RtlUserThreadStart+0x28
Listing 1-5 Unmanaged Windows Stack Trace from the Execution of the Python Script from Listing 1-1
Without Debugging Symbols

Note Each thread has its own stack trace (backtrace).

Symbol Files
Symbol files allow a debugger to map memory addresses to symbolic information such as
function and variable names. For example, if you download and apply symbol files to the
Windows example above, you get a much better and more accurate stack trace, as shown
in Listing 1-6.

00 00000090`7e1ef0a8
00007ff9`8c44fcf9 ntdll!NtWaitForMultipleObjects+0x14
01 00000090`7e1ef0b0
00007ff9`8c44fbfe KERNELBASE!WaitForMultipleObjectsEx+0xe9
02 00000090`7e1ef390
00007ff8`ef943986 KERNELBASE!WaitForMultipleObjects+0xe
03 00000090`7e1ef3d0
00007ff8`ef94383d python311!pysleep+0x11a
04 00000090`7e1ef430
00007ff8`ef81a6b2 python311!time_sleep+0x2d
05 00000090`7e1ef460
00007ff8`ef82fa77 python311!_PyEval_EvalFrameDefault+0x8f2
06 (Inline Function) --------`-------
- python311!_PyEval_EvalFrame+0x1e
07 00000090`7e1ef670
00007ff8`ef82f137 python311!_PyEval_Vector+0x77
08 00000090`7e1ef6b0
00007ff8`ef82d80a python311!PyEval_EvalCode+0x97
09 00000090`7e1ef730
00007ff8`ef82d786 python311!run_eval_code_obj+0x52
0a 00000090`7e1ef760
00007ff8`ef97a17e python311!run_mod+0x72
0b 00000090`7e1ef7a0
00007ff8`ef7e33a5 python311!pyrun_file+0x196b66
0c 00000090`7e1ef810
00007ff8`ef8da620 python311!_PyRun_SimpleFileObject+0x11d
0d 00000090`7e1ef880
00007ff8`ef8daaef python311!_PyRun_AnyFileObject+0x54
0e 00000090`7e1ef8b0
00007ff8`ef8dab5f python311!pymain_run_file_obj+0x10b
0f 00000090`7e1ef980
00007ff8`ef8db964 python311!pymain_run_file+0x63
10 00000090`7e1ef9b0
00007ff8`ef8db7f5 python311!pymain_run_python+0x140
11 00000090`7e1efa20
00007ff8`ef8260d9 python311!Py_RunMain+0x15
12 00000090`7e1efa50
00007ff6`aefe1230 python311!Py_Main+0x25
13 (Inline Function) --------`-------
- python!invoke_main+0x22
14 00000090`7e1efaa0
00007ff9`8e1c26ad python!__scrt_common_main_seh+0x10c
15 00000090`7e1efae0
00007ff9`8ef6a9f8 KERNEL32!BaseThreadInitThunk+0x1d
16 00000090`7e1efb10
00000000`00000000 ntdll!RtlUserThreadStart+0x28
Listing 1-6 Unmanaged Windows Stack Trace from the Execution of the Python Script from Listing 1-1
with Debugging Symbols
Module
Like the distinction between managed and unmanaged stack traces, there is a difference
between Python modules (which may correspond to files in traceback) and native
modules such as DLLs in Windows and .so files in Linux, which are loaded into memory
when you execute the Python compiler/interpreter. For example, the following shared
libraries are loaded in Linux for the simple multithreaded example from Listing 1-2:

~/Chapter1$ pmap 60
60: python3 process.py
0000000000400000 132K r---- python3.7
0000000000421000 2256K r-x-- python3.7
0000000000655000 1712K r---- python3.7
0000000000801000 4K r---- python3.7
0000000000802000 664K rw--- python3.7
00000000008a8000 140K rw--- [ anon ]
0000000001fff000 660K rw--- [ anon ]
00007f6bc7f69000 1684K rw--- [ anon ]
00007f6bc810e000 2964K r---- locale-archive
00007f6bc83f3000 12K rw--- [ anon ]
00007f6bc83f6000 136K r---- libc-2.28.so
00007f6bc8418000 1308K r-x-- libc-2.28.so
00007f6bc855f000 304K r---- libc-2.28.so
00007f6bc85ab000 4K ----- libc-2.28.so
00007f6bc85ac000 16K r---- libc-2.28.so
00007f6bc85b0000 8K rw--- libc-2.28.so
00007f6bc85b2000 16K rw--- [ anon ]
00007f6bc85b6000 52K r---- libm-2.28.so
00007f6bc85c3000 636K r-x-- libm-2.28.so
00007f6bc8662000 852K r---- libm-2.28.so
00007f6bc8737000 4K r---- libm-2.28.so
00007f6bc8738000 4K rw--- libm-2.28.so
00007f6bc8739000 8K rw--- [ anon ]
00007f6bc873b000 12K r---- libz.so.1.2.11
00007f6bc873e000 72K r-x-- libz.so.1.2.11
00007f6bc8750000 24K r---- libz.so.1.2.11
00007f6bc8756000 4K ----- libz.so.1.2.11
00007f6bc8757000 4K r---- libz.so.1.2.11
00007f6bc8758000 4K rw--- libz.so.1.2.11
00007f6bc8759000 16K r---- libexpat.so.1.6.8
00007f6bc875d000 132K r-x-- libexpat.so.1.6.8
00007f6bc877e000 80K r---- libexpat.so.1.6.8
00007f6bc8792000 4K ----- libexpat.so.1.6.8
00007f6bc8793000 8K r---- libexpat.so.1.6.8
00007f6bc8795000 4K rw--- libexpat.so.1.6.8
00007f6bc8796000 4K r---- libutil-2.28.so
00007f6bc8797000 4K r-x-- libutil-2.28.so
00007f6bc8798000 4K r---- libutil-2.28.so
00007f6bc8799000 4K r---- libutil-2.28.so
00007f6bc879a000 4K rw--- libutil-2.28.so
00007f6bc879b000 4K r---- libdl-2.28.so
00007f6bc879c000 4K r-x-- libdl-2.28.so
00007f6bc879d000 4K r---- libdl-2.28.so
00007f6bc879e000 4K r---- libdl-2.28.so
00007f6bc879f000 4K rw--- libdl-2.28.so
00007f6bc87a0000 24K r---- libpthread-2.28.so
00007f6bc87a6000 60K r-x-- libpthread-2.28.so
00007f6bc87b5000 24K r---- libpthread-2.28.so
00007f6bc87bb000 4K r---- libpthread-2.28.so
00007f6bc87bc000 4K rw--- libpthread-2.28.so
00007f6bc87bd000 16K rw--- [ anon ]
00007f6bc87c1000 4K r---- libcrypt-2.28.so
00007f6bc87c2000 24K r-x-- libcrypt-2.28.so
00007f6bc87c8000 8K r---- libcrypt-2.28.so
00007f6bc87ca000 4K ----- libcrypt-2.28.so
00007f6bc87cb000 4K r---- libcrypt-2.28.so
00007f6bc87cc000 4K rw--- libcrypt-2.28.so
00007f6bc87cd000 192K rw--- [ anon ]
00007f6bc8801000 28K r--s- gconv-modules.cache
00007f6bc8808000 4K r---- ld-2.28.so
00007f6bc8809000 120K r-x-- ld-2.28.so
00007f6bc8827000 32K r---- ld-2.28.so
00007f6bc882f000 4K r---- ld-2.28.so
00007f6bc8830000 4K rw--- ld-2.28.so
00007f6bc8831000 4K rw--- [ anon ]
00007ffc6026a000 132K rw--- [ stack ]
00007ffc60356000 16K r---- [ anon ]
00007ffc6035a000 4K r-x-- [ anon ]
total 14700K
The Windows version has the following loaded modules:

00007ff6`aefe0000 00007ff6`aeffa000 python python.exe

00007ff8`ef7e0000 00007ff8`efdad000 python311 python311.dll
00007ff9`62950000 00007ff9`6296b000 VCRUNTIME140
VCRUNTIME140.dll
00007ff9`7f1e0000 00007ff9`7f1ea000 VERSION VERSION.dll
00007ff9`8bce0000 00007ff9`8bd08000 bcrypt bcrypt.dll
00007ff9`8c3f0000 00007ff9`8c793000 KERNELBASE KERNELBASE.dll
00007ff9`8c840000 00007ff9`8c951000 ucrtbase ucrtbase.dll
00007ff9`8c960000 00007ff9`8c9db000 bcryptprimitives
bcryptprimitives.dll
00007ff9`8d150000 00007ff9`8d1c1000 WS2_32 WS2_32.dll
00007ff9`8d1d0000 00007ff9`8d2e7000 RPCRT4 RPCRT4.dll
00007ff9`8dd50000 00007ff9`8ddf7000 msvcrt msvcrt.dll
00007ff9`8ded0000 00007ff9`8df7e000 ADVAPI32 ADVAPI32.dll
00007ff9`8e1b0000 00007ff9`8e272000 KERNEL32 KERNEL32.DLL
00007ff9`8e280000 00007ff9`8e324000 sechost sechost.dll
00007ff9`8ef10000 00007ff9`8f124000 ntdll ntdll.dll

Memory Dump
A process memory can be saved in a memory dump file.

An undigested and voluminous mass of information about a problem or the state of a

system and most especially one consisting of hex runes describing the byte-by-byte
state of memory.
Eric S. Raymond, The New Hacker’s Dictionary, Third Edition

These memory dumps are also called core dumps in Linux. It is also possible to get a
kernel memory dump and a dump of physical memory (also called a complete memory
dump in Windows). Figure 1-5 shows different memory dump types.

Figure 1-5 Memory dump types

Memory dumps may be useful for debugging hard-to-reproduce intermittent

problems. This approach is called postmortem debugging. You will see some case studies
in the following chapters.

Crash
To fail suddenly. “Has the system just crashed?” “Something crashed the OS!” Also
used transitively to indicate the cause of the crash (usually a person or a program, or
both). “Those idiots playing SPACEWAR crashed the system.”
Eric S. Raymond, The New Hacker’s Dictionary, Third Edition.

When something illegal happens inside a process thread, such as when memory
outside its available range is accessed or you write to read-only memory, the operating
system reports the error and terminates the process. It may also save the process
memory into a memory dump file. The process then disappears from the list of available
processes.

Hang
1. To wait for an event that will never occur. “The system is hanging because it can’t
read from the crashed drive.”

2. To wait for an event to occur. “The program displays a menu and then hangs until
you type a character.”
Eric S. Raymond, The New Hacker’s Dictionary, Third Edition

Threads interact with other threads, including other processes’ threads. These
interactions can be viewed as sending messages and waiting for the responses. Some
processes may be critical because their threads process messages from many other
threads from other processes. If threads from such a critical process stop sending
responses, all other waiting threads are blocked. A deadlock is when two threads are
waiting for each other. When hanging, the process continues to be present in the list of
available processes. There are also processes (critical components) that, when their
threads hang, block threads from many other processes (noncritical components). Figure
1-6 depicts such components and their interaction abstracted via messages in the normal
scenario, and Figure 1-7 shows the abnormal scenario when noncritical components are
blocked and waiting for responses because the critical components are deadlocked.
Figure 1-6 Request and response interaction between critical and noncritical system components
Figure 1-7 Blocked and deadlocked components

Summary
In this chapter, you learned the fundamental vocabulary you will use in subsequent
chapters. The next chapter introduces a pattern-oriented debugging approach.
© The Author(s), under exclusive license to APress Media, LLC, part of Springer
Nature 2024
D. Vostokov, Python Debugging for AI, Machine Learning, and Cloud Computing
https://doi.org/10.1007/978-1-4842-9745-2_2

2. Pattern-Oriented Debugging
Dmitry Vostokov1

(1) Dalkey, Dublin, Ireland

This chapter introduces the pattern-oriented debugging process

approach and the pattern languages you will use in subsequent
chapters.

The History of the Idea

The idea of using patterns in debugging is not new1. Earlier, such
patterns came in two types: bug patterns2 and debug patterns3. Before
2000, only a few debugging-related patterns could be found, such as the
Debug Printing Method4.
Bug patterns are usually specific patterns for specific languages and
platforms. By bugs, we mean software defects. Usually, these are related
to the source code but can also be related to configuration and data
models.
Using source code as a starting point for debugging is only possible
for a limited number of scenarios, such as when you have a Python
stack trace. However, there are many cases when the starting point for
source code investigation is unknown. Here, a well-defined process may
benefit. A number of debugging processes were proposed in the past,
including multidisciplinary approaches5.
The phrase “pattern-oriented debugging” appeared around 1987-
1988 in the context of patterns of process interaction6. It is not the
same as the “pattern-oriented debugging process” proposed in 20147
as further development of unified debugging patterns that were
introduced in 20108. Since then, these patterns have been used for
successfully teaching Windows debugging of unmanaged (native,
Win64, C, C++) and managed (.NET, C#) code9 for almost a decade,
starting in 2013. Overall, this pattern-oriented approach can be traced
to our earlier presentation published as a book in 201110. In it, we
apply the same pattern-oriented process to Python debugging in cloud
and machine learning environments.

Patterns and Analysis Patterns

Before looking at the debugging process, a few words about patterns in
the context of diagnostics and debugging. By a pattern, we mean a
common recurrent identifiable set of indicators (symptoms, signs). By
an analysis pattern, we mean a common recurrent analysis technique
and method of pattern identification in a specific context. By pattern
language, we mean common names of patterns and analysis patterns
used for communication.

Development Process
Let’s first look at the traditional software development process stages.
Figure 2-1 abstracts them from several development processes,
including waterfall and iterative ones.
Figure 2-1 Stages of the typical software development process

Development Patterns
For each stage, there exists some pattern language such as a vocabulary
of solutions to common recurrent identifiable problems with grammar,
semantics, and pragmatics. Figure 2-2 also includes software usage and
presentation patterns for human-computer interaction. In this book, I
assume you have familiarity with such pattern languages (some
references are provided below).
Random documents with unrelated
content Scribd suggests to you:
the most general description of the wanted motor, the chance of its
discovery seemed of the slightest.
He came out of the telephone-box to find Sir Clinton and
Wendover waiting for him in Sir Clinton's car.
“Get in,” the chief constable ordered. “We've got to waste a
minute or two in going down the road to meet that gang of
constables and giving them orders to follow on. Put both feet on the
accelerator, squire, and do anything else short of spilling us in the
ditch. Every minute may count now.”
Wendover needed no urging. They flashed down the road
towards Lynden Sands, pulled up as they met the body of police,
and were off again as soon as Sir Clinton had given the constables
their orders to make direct for Foxhills. A very few minutes brought
the car to the Foxhills gate, where Wendover, at a sign from Sir
Clinton, stopped the car. The chief constable jumped out and
examined the road surface with his pocket flash-lamp.
“Thank the Lord! A car's gone up the avenue. We may be in time
to nab them yet.”
 Chapter XV.
The Method of Coercion
When Cressida received her uncle's note that afternoon, she was
both relieved and puzzled. Within less than a week she had been
subjected to shocks and strains of such acuteness that she had
almost lost the power of being surprised by anything that might
happen; and Paul Fordingbridge's letter caused her hardly any
astonishment, which she would certainly have felt had she been in a
more normal condition. All that she gathered from it was that, after
disappearing in a mysterious manner, he had returned and evidently
needed her assistance. She was not particularly attached to him; but
she was not the sort of person who would refuse her help to anyone
in an emergency, even though that person had shown her very little
sympathy in her own recent troubles.
She had a very fair idea of the rumours which had been running
through the hotel, and she had no desire to advertise her meeting
with her uncle. The final phrase in the note: “Come alone,” was quite
enough to suggest that he wished to keep the encounter secret. And
she knew well enough that a plain-clothes constable had been
detached to watch her; she had seen him once or twice when she
had been passing through the entrance-hall, and had no difficulty in
detecting the interest which he took in her movements. Unless she
could contrive to give him the slip, he would follow her out to the
Blowhole. Then she thought of the lady golfers' dressing-room, with
its convenient door to the outside of the hotel, and a method of
evasion suggested itself. She took the lift down; walked boldly past
the watcher; turned down the passage and entered the dressing-
room. Then, picking up her hat, blazer, and golfing-shoes, she
slipped out of the side-entrance and hurried down one of the paths
till she reached a place where she could change her slippers for her
outdoor shoes.
Leaving the slippers to be picked up on her way back, she
crossed the hotel gardens and made her way out on to the headland
where the Blowhole lay. The night was clear enough, but the moon
was still very low, and the light was dim. As she came up towards
the Blowhole, a figure came forward to meet her.
“Is that you, uncle?” she asked.
As soon as she spoke she was aware of someone who had risen
behind her from an ambush. An arm came round her from the rear,
pinning her hands to her sides; and a soft, wet pad was brought
down on her face. She felt a burning liquid on her lips, and, as she
gasped under the mask, a sickly, sweet-scented vapour seemed to
penetrate down into her lungs. As she struggled to free herself and
to cry out, the man before her stepped forward and helped his
companion to hold her.
“Don't choke her altogether, you fool!” she heard her new
assailant say, but his voice sounded faint; and in a minute she had
lost consciousness.
When she came to herself once more, it was to find herself lying
on a bed from which all the bed clothes had been removed. Her
head swam at the slightest movement, and she felt deadly sick. With
complete incuriosity, she noticed some figures in the room, and then
again she slipped back into unconsciousness.
The sound of voices roused her once more, after what seemed to
be a span of eternity, and she slowly began to recollect the events
which had led to her present condition. As she gained more control
over herself, she attempted to move, but she found that her wrists
and ankles were fettered, and her further vague attempts to get her
bearings satisfied her that some kind of gag had been thrust
between her teeth and lashed at the back of her head.
For a while she lay, feeling sick and dizzy and unable to think
clearly; but gradually, as the narcosis passed slowly off, she grew
better able to take in her surroundings. She had just reached the
stage when she could concentrate her attention when one of the
figures in the room came to the side of the bed, and stooped down
to examine her in the light of a candle. The features seemed faintly
familiar; but in her drugged condition it was some moments before
she could identify the man as Simon Aird, at one time valet at
Foxhills.
“Got your senses back, miss? You've been a longish while over it.
Better pull yourself together.”
Even in her bemused condition she recognised something in the
tone of his voice which told her that he was not friendly. She lay still,
fighting hard to recover her normal personality. Aird watched her
with cold interest, without making any attempt to disturb her. At last
the fumes of the anesthetic seemed to clear from her brain.
“Feelin' sick, miss?” Aird inquired callously. “That's the
chlorryform, I expect. You'll be all right in a jiffy or two.”
Her head still swam, but she managed to turn slightly so that she
could see the two other figures in the room. One of them, with his
back to her, was unrecognisable. The other, whose face she could
see, was a total stranger.
Aird saw her glance, and interpreted it aloud.
“Lookin' for your uncle, miss, I expect?”
His mean little eyes seemed to twinkle at some obscure joke.
“He couldn't come to meet you, miss, as arranged. He was
unexpectedly detained. Ain't that so, boys? Mr. Paul Fordingbridge
was unexpectedly detained, and couldn't come to meet 'is niece?”
The joke, whatever it was, seemed to be shared by the other
two, for they laughed coarsely. Aird was encouraged to proceed to
further flights of humour.
“You've got an expressive face, miss—always 'ad. Why, I can
read you like a book. You're worryin' your pretty 'ead to know 'ow
you came 'ere, isn't that it? Trust Simon Aird to understand what a
girl's thinkin' about. A pretty girl's as plain as print to me—always
was. But I'm keepin' you on tenterhooks, I see, an' that's not polite.
I'll soon tell you. We found you up yonder on the headland, near the
Blowhole, drunk and incapable. That was a dangerous thing to do,
miss. I can't think 'ow you came to be doin' it. Lord! In that state,
there's no knowin' what mightn't 'ave 'appened to you—it's dreadful
just to think of it! But you fell into good 'ands, miss. We took you up
an' lifted you into our car, which 'appened to be near by; and we
brought you 'ere with as much care as if you'd been worth your
weight in gold, miss. No pains spared, I assure you.”
Through the numbness engendered by the anæsthetic, fear had
been growing in Cressida's mind, until now it had overwhelmed
every other feeling. She knew she was completely in the hands of
these three men; and even what she had seen of them was enough
to fill her with the acutest dread. Aird's oily phrases went ill with the
expression in his little piggish eyes.
“It's a great thing to be a bit of a psycho-what-d'ye-call-it, miss. I
can tell to a dot just exactly what's passin' through your mind,” he
went on. “You're wonderin' where you are at this minute. I 'ave
much pleasure in enlightenin' you. You've 'ad the extraordinary good
luck to be brought to the 'eadquarters of Aird & Co., a purely
philanthropic syndikit formed for the good purpose of purifyin'
morals, and rescuin' heiresses from the clutch of their fancy men,
and teachin' them to lead saintly lives in future. Your case is the first
we've 'ad brought to our door, so we can give it our excloosive
attention. And we shall!”
He sniggered, apparently much amused by his own conceit.
Cressida felt the menace behind all this forced jocularity. Aird
brought the candle nearer to her face, and made a pretence of
studying her features.
“Ah!” he continued. “You're feelin' nervous, miss? It's as plain as
a pike-staff to the eye of the trained mind-reader like me. You're all
of a flutter, like. And no wonder, miss. You that's been livin' in sin
with that young Fleetwood for the best part of a year, with your own
true husband alive and mournin' all the while. Shockin'! Such goin's
on! But never fear, miss. You 'ave fallen into good 'ands, as I said
once before. Aird & Co., expert matrimonial agents, will take up your
case and make an honest woman of you yet.”
Behind his jovality, Cressida could feel some dreadful menace.
She turned her face away, so as to hide from herself his little
gloating eyes.
“That'll do, Aird,” said a fresh voice, quite unknown to her. “I'll
explain things. You talk too much.”
 The man who had his back turned to her came across to the bed
and took the candle from Aird. Then, stooping down, he let the light
play over his own features, while his hand forced Cressida's head
round so that she gazed straight up into his face. At the first glance
she thought she must still be under the influence of the chloroform,
for what she saw was torn almost out of human likeness.
“Allow me to introduce myself,” said the wreck. “Your future
husband, also your cousin: Derek Fordingbridge. Not recognise me
again? Well, I suppose I've changed since we said good-bye last.”
He let the candlelight play across his shattered face for some
moments, so that she might miss no detail of the horror. Then, as
she closed her eyes, he released his grip, and she turned her head
away to escape the sight of him.
“You'll get used to me in time,” was his only comment. “Here's
the situation. Our uncle chooses to keep me out of my money. If I
go to law over it, most of the cash will be wasted in legal expenses.
He won't suffer, but I shall. Now, you're the next in the line of
inheritance; so, if I drop out, it comes to you. And if you marry me,
then what's yours is mine—I'll see to that part of it. You understand
the idea? You marry me and I drop my claim; and between us we
collar the dibs. Uncle won't object, I'll guarantee; and dear Auntie
Jay will be delighted.”
He paused and examined the expression of loathing on Cressida's
face.
“I don't wish to go to extremes,” he said coldly, “but you're going
to do as you're told. Make no mistake about that.”
He drew back slightly, allowing Aird to come nearer.
“Aird will take the gag out and let you speak; but he'll keep his
hand on your throat, and the first attempt you make to cry out you'll
get throttled pretty sharply. Understand?”
Aird obeyed instructions, and Cressida passed her tongue over
her bruised lips. She was in deadly terror now, and her mind was
working swiftly. A glance at the three men bending over her was
sufficient to show her that she need expect no mercy from them.
She was completely in their power; and if she refused to give in to
them, they might—— But she thrust to the back of her mind all the
possibilities which she could read so clearly on the face of Aird.
Then, as a thought shot through her mind, she strove her
hardest to keep out of her expression the relief that she felt. If she
submitted immediately, and promised to carry out the order, that
would perhaps save her for the time being; and, when it came to
implementing her promise, the marriage ceremony would have to be
performed in public, or at least in the presence of some clergyman
or official; and there would be nothing to prevent her refusing then.
They could not coerce her in a church or before a registrar.
Nowadays forced marriages are found only in books.
The pressure of the gag had hurt her mouth, and she had some
difficulty in framing words in which to make her submission.
“I can't help myself. But you're not my cousin Derek.”
The faceless creature laughed.
“That's a quick courting!” he sneered. “But one doesn't need to
be a psycho-what-d'ye-call-it, as Aird says, to see what's in your
mind.”
His voice became tinged with a menace beside which Aird's
seemed childish.
“You think you've only got to say ‘Yes’ now; then, when it comes
to the point, you'll turn on us and give the show away? We're not
such fools as all that. I've got a string that I'm going to tie to your
leg. It'll bring you running back to me and no questions asked.”
He paused for a moment, as though expecting her to speak; but,
as she said nothing, he continued in the same tone:
“You think that the worst we could do to you would be to hand
you over to Aird, there, or to share you amongst us. You can make
your mind easy. It's not going to be that.”
The wave of relief which passed over Cressida at this hint was
followed by a chill of apprehension as she realised the full
implication of his words. He did not keep her on tenterhooks long.
“Ever heard of hydrophobia? Know much about it? No? Well,
then, I'll tell you something. You get bitten by a mad dog. First of all
you feel tired and restless; and naturally you can't help being
worried a bit. Then, after a day or two, things get a bit more
definite. You can't swallow, and you get a thirst that torments you.
Then, they say, you get spasms even at the thought of drinking; and
you get into a state of devilish funk—unspeakable terror, they say in
the books. After that you get fits—frothing at the mouth, and all the
rest of the jolly business. And, of course, eventually you die after
considerable agony, if you get the proper dose. I'd hate to see a
pretty girl like you afflicted in that way. Dreadful waste of good
material.”
He paused deliberately, letting this picture sink into her mind,
and scanning her face to see the effect which he had produced.
“No mad dogs here, of course; but they have them in France.
I've a French medical friend who's kindly supplied me with some
extract taken from one of them.”
Again he paused, to let anticipation do its work.
“If you get injected with this extract, or whatever it is, there's
only one hope. Within a certain number of days you've got to get to
a Pasteur Institute and put yourself under treatment there. Nothing
else is any good. And, if you overshoot the time, even the Pasteur
Institute can do nothing for you. You just go on till you froth at the
mouth, get cramp in the throat, and die that rather disgusting
death.”
He looked down at Cressida's face, with its eyes dark with horror;
and something which might have been a smile passed over his
shattered countenance.
“My French medical friend supplied me with both the bane and
the antidote—at least, enough of the antidote for a first dose. You
see the point? Perhaps I'd better be precise. Here's a hypodermic
syringe.”
He produced a little nickel case from his pocket, and drew from it
a tiny glass syringe, to which he fitted a hollow needle.
“I'm going to fill this with some of the mad dog extract and inject
it into your arm. Once that's done, your only chance is to get Pasteur
Institute treatment within a certain time or else rely on me to give
you a first dose of the antidote before the time's up. Once the time's
past without treatment, nothing can save you. I couldn't do it
myself, even with the antidote. You'd simply go through all the
stages I've told you about, and then die.”
He fingered the tiny syringe thoughtfully.
“Now do you see the ingenuity of my plan? I'm going to inject
some of the stuff into your veins now. Then we'll keep you here until
the very last moment of your safety. Then you'll come with me and
get spliced by special license. By that time it'll be too late to get to
an institute; you'll have no chance whatever except the dose of
antidote that I've got. And you won't get that from me until we're
safely married without any fuss. You'll stand up in public and say: ‘I
will!’ without any objection, because it'll be your one chance of
escaping the cramps and all the rest of it. Ingenious, isn't it? Shall I
repeat it, in case you've missed any of the points? It's no trouble, I
assure you.”
Cressida glanced from face to face in the hope of seeing some
signs of relenting; but none of the three showed the faintest trace of
pity.
“Be sensible, miss,” said Aird, with the air of one reasoning with a
wayward child. “A pretty girl like you wouldn't want to be seen
frothin' at the mouth and runnin' round bitin' people. It wouldn't be
nice.”
His unctuous tone brought up all Cressida's reserves of strength.
“You'd never dare do it,” she gasped.
“You think so?” the faceless man inquired indifferently. “Well,
you'll see in a moment or two.”
He rose with the hypodermic syringe in his hand and went out of
the room. She could hear him doing something with a sink, and the
sound of water. At that her nerve gave way.
“Oh, don't do it! Please, please don't! Anything but that! Please!”
For the first time she realised that this hideous scheme was
seriously meant; and the pictures which flashed through her mind
appalled her. To pass out of life was one thing; but to go out by the
gate of madness—and such a form of madness—seemed an
unbearable prospect. To die like a mad dog—anything would be
better than that!
“Oh, don't!”
 She gazed up at the faces of the two men who stood beside her
in the hope that in this last moment they might flinch from carrying
the foul business through. But there was no comfort in what she
saw. Aird was evidently drinking in her torment with avidity. It was
something which seemed to give him a positive pleasure. The
stranger shrugged his shoulders, as though suggesting that the
matter had been irrevocably settled. Neither of them made any
answer to her hysterical pleading.
The man with the hypodermic came back into the room; and she
hid her face as he crossed to the side of the bed. Her arm was
roughly grasped, and she felt him pinch her skin before he drove the
needle home. Then came a sharp pang as he injected the contents
of the syringe.
Then, just as she wavered on the edge of fainting from the
nervous strain she had undergone, the whole scene changed. There
was a crash of glass, and a voice which seemed faintly familiar
ordered sharply:
“Hands up!”
A scuffle, two shots, a cry of pain, and the fall of a heavy body to
the floor; more sounds of rapid movement in the room; a voice
shouting directions; another shot, outside the house—all these
impinged on her consciousness without her grasping exactly what
had happened. With a last effort of will she wrenched herself round
on the bed, so that she could see the room.
Sir Clinton, pistol in hand, was stooping over the third man, who
lay groaning on the floor. At the open window she could see
Wendover climbing into the room; and, as he jumped down,
Inspector Armadale dashed in through the open door. Rescue had
come just too late; and, as she realised this, her power of resistance
gave out, and she fainted.
Sir Clinton made a gesture to Wendover, putting him in charge of
the unconscious girl, while he himself turned back to his captive.
“I've smashed your shoulder with that shot, I think, Billingford,”
he commented. “You're safe enough, my man, now that I've taken
your gun away from you. You'll stay where you are until my
constables come for you. Mr. Wendover will keep an eye on you—
and he'll shoot you without the slightest compunction, I'm sure, if
you give trouble.”
Billingford seemed engrossed in more immediate afflictions.
“Oh! It hurts damnably!” he muttered.
“Glad to hear it,” Sir Clinton declared unsympathetically. “It'll
keep you quiet. Well, inspector?”
Armadale held up a bleeding hand.
“They got me,” he said laconically. “It's only a flesh-wound. But
they've cleared off in their car—hell-for-leather.”
Sir Clinton turned to Wendover.
“You look after that girl. The constables will be here in a few
minutes. Shoot Billingford in the leg, if he shows the slightest sign of
moving, though I don't expect he'll do much. I've got to get on the
track of those two who broke away.”
Followed by the inspector, he hurried out into the night.
 Chapter XVI.
The Man-Hunt on the Beach
Much to the inspector's surprise, Sir Clinton did not drive
furiously when they had got into his car, which had been left
standing at some distance from the cottage. It was only when they
almost ran into the approaching squad of police that he understood
his superior's caution.
“Two of you get on board,” said Sir Clinton, as he pulled up.
“Four more go up to the cottage; and the rest of you make the best
time you can down to the hotel and wait there for orders.”
When the two constables had got into the car, he drove off
again; and this time the inspector had no reason to complain of slow
speeds. His heart was in his mouth as Sir Clinton took the turn out
of the avenue into the main road.
“You've got the number of their car, haven't you?” the chief
constable demanded. “Then tell one of the constables to telephone a
warning about it to headquarters from the hotel. I'm going to drop
him there. And tell him to send a party with a car up at once to the
cottage to get Mrs. Fleetwood down comfortably. You'd better get
Billingford brought down also—not in the same car.”
The inspector transmitted these instructions just in time to allow
the constable to alight from the car as Sir Clinton pulled up at the
hotel gate. Without hesitation, the chief constable swung the car off
along the road to Lynden Sands and opened the throttle to its
fullest.
“Sure they're going this way, sir?” Armadale asked.
“No, just taking a chance. They'll want to get clear of the car as
soon as possible, I expect, since it's recognisable now that we've got
the number. I may be all wrong, of course.”
 The big car tore on in the moonlight, and the speed left the
inspector little inclination for talk. He gasped once or twice as they
swung round corners, and his main feeling was one of thankfulness
that at that hour of the night they were not likely to meet anything
on the road. One last turn, which made Armadale and the constable
grip frenziedly at the nearest hand-hold, and they came out on the
edge of the bay.
“Look!” the inspector ejaculated. “You've pulled them in, sir.”
Not three hundred yards ahead, the hunted car appeared in the
moonlight, travelling much slower than Armadale had expected, but
apparently gaining speed as it ran.
“They've parted company,” Sir Clinton snapped. “The car's slowed
down to let one man off. There's only the driver on board now.”
Suddenly, at a point where the road ran level with the beach,
their quarry left the highway and plunged down on to the sands.
“He's trying to gain something by cutting straight across the
beach, sir, instead of following the curve of the road.”
Armadale, expecting Sir Clinton to do the same, gripped the side
of the car in anticipation of the shock when they left the road; but
the chief constable held to the highway.
“He's making for Flatt's cottage, to get the boat and leave us
standing,” he said. “He'll get a surprise when he finds the oars
gone.”
The inspector had no time to admire his chief's forethought. The
hunted car was now running on a line which would bring it between
the old wreck and the edge of the incoming tide; and on the hard
sands it was making tremendous speed. Armadale, leaning forward
in the excitement of the chase, saw the long cones of its headlights
illuminate the hull of the wreck for a moment; then the beams
swung up into the air; the car seemed to halt for an instant, and
then rolled over sideways along the sands. And then it vanished as
though the ground had swallowed it.
“The quicksand!” ejaculated Armadale, as he realised what had
happened.
Sir Clinton shut the throttle and let his car slow down.
 “Hit some rock projecting slightly from the sand, I expect,” he
commented. “Probably the front axle or the steering-gear went, and
he came to smash. Well, that's one of 'em gone.”
He chose a place carefully and turned his own car on to the
sands, running down to near the wreck.
“Don't go too near,” he advised. “One can't be sure of the
danger-zone.”
They got out and went down to the scene of the disaster. A
glance at the car-tracks showed the correctness of Sir Clinton's
guess. The hunted car had struck a low projecting rock with its near
front wheel; and from that point the wheel-marks were replaced by
the trace of the whole vehicle, overturned and sliding along the
beach. The trail ended abruptly; and where the car had sunk they
saw an area of repulsive black mud.
“Ugh!” said the inspector, examining it with disgust. “Fancy going
down into that stuff and feeling it getting into your eyes and mouth.
And then choking in that slime! It gives me the creeps to think of it.”
He shuddered at the picture conjured up by what he saw before
him.
“Do you think there's any chance of recovering the body?” he
inquired after a moment or two.
Sir Clinton shook his head.
“I doubt it. You'll need to try, of course; best do it with grappling-
irons from a boat, I suppose. But I shouldn't think you're likely to
succeed. It doesn't matter much, anyhow. He's got his deserts. Now
for the other man. Come along!”
They went back to the car and got aboard. Sir Clinton seemed to
have decided on his next move, for he drove along the sands in the
direction of the hotel. Rather to the inspector's surprise, they did not
turn off on to the road at Neptune's Seat, but went still farther along
the shore, making for the headland on which the Blowhole was
situated.
Armadale was still in ignorance of much that had happened in
the last hour. When they had reached Peter Hay's cottage, Sir
Clinton had detached the inspector to search for the car which had
brought their quarry; and, as this had been carefully concealed,
Armadale had spent some time in hunting for it. In the meanwhile,
Sir Clinton and Wendover had gone cautiously to the cottage. The
next thing the inspector heard was the sound of shooting; and two
men had come upon him before he had time even to think of
disabling the fugitives' car. They had shot him in the hand, flung him
down, and escaped in the car before he had time to do anything to
hinder them. His entry into the cottage had failed to enlighten him
as to what had been going on; and Sir Clinton had hurried him off
again almost before he had time to get his bearings.
“That's as far as we can go with the car,” Sir Clinton announced,
opening the door and getting out.
The moon shone out just at that moment, as a passing cloud
slipped away from its face; and Sir Clinton, gazing along the shore,
uttered an exclamation of satisfaction.
“We're in luck, inspector! See him? Yonder, just under the cliff. He
hasn't been able to get far.”
He pulled out his automatic.
“I've often wondered how far these things carry. I don't want to
hurt him, and it seems safe enough at this range. A scare's all we
need, I think. He's making for the mouth of the cave below the
headland.”
He lifted the pistol and fired in the direction of the figure. At the
sound of the shot, the fugitive turned and, seeing his pursuers, ran
stumblingly over the rocks where the edge of the tide was washing
close up against the cliff.
“No hurry,” Sir Clinton pointed out, as Armadale and the
constable quickened their steps. “We've got him trapped by the tide.
There's only one bolt-hole—the cave. And I hope he takes it,” he
added, with something of sinister enjoyment in his tone which
surprised the inspector.
They moved leisurely in the direction of the cave-mouth; and, as
they did so, the fugitive gave one backward glance and then
splashed waist-deep through the water which was foaming into the
entrance. He ducked under the low arch and vanished. As he did so,
Sir Clinton halted, and then, after a careful inspection of the
incoming tide, he led the way back to the car.
 “It's as cheap sitting as standing,” he commented, settling
himself comfortably in the driving-seat. “We'll need to wait here until
the tide shuts the door on him by filling that tunnel he's gone
through. After that, I suspect he'll be the most anxious of the lot of
us.”
“But there's another exit from that cave,” Armadale pointed out.
“He's probably climbing up the tube of the Blowhole just now, sir. He
might get clean away by the top of the headland.”
Sir Clinton pulled out his case and lit a cigarette in a leisurely
fashion.
“I'm sure I hope he does,” he replied, much to the inspector's
surprise. “Just wait a moment and you'll see.”
He smoked for a minute or two without troubling to make his
meaning clear; and then the souffleur itself gave the answer.
Armadale's ear caught the sound of a deep gurgle from the heights
above their head; then came a noise like a giant catching his breath;
and at last from the Blowhole there shot up the column of spray,
towering white and menacing in the moonlight. As it fell, Sir Clinton
pressed the self-starter.
“That bolts the back door, you see, inspector. I only hope he's
been caught on the threshold. Now, I think, we can go back to the
hotel and see if we can pick up one or two useful things.”
He turned the car on the last strip of sand before the rocks and
swung it round towards Neptune's Seat. After a little searching, he
found a spot from which he could ascend to the road without
straining his springs.
“I had the curiosity to examine that Blowhole cave at low tide
once, inspector,” he explained as he drove up towards the hotel.
“The thing works this way. The entrance is low, and the tide fills it
soon. The air in the cave can still get out by a narrow tunnel leading
up to the Blowhole. But in a minute or two this second tunnel's
mouth gets filled up, and there's no escape from the cave. The sides
are smooth, and the tide rises quickly, so that fellow will either
drown or else he'll creep into the Blowhole tunnel to escape. The
tide rises a bit farther, and compresses the air in the cave. At that
stage the souffleur begins to work. Intermittently, you get the air-
pressure in the cave big enough to blow through the Blowhole
tunnel, carrying the layer of water there in front of it; and that
mixture of water and compressed air makes the jet. So, you see, if
that fellow's in the cave, he must be swimming round like a rat in a
pail; and if he's in the tunnel, he must be suffering agonies as the
jet comes up and tears at him. You know what sort of force it has.
And if he can't cling on to the rocks of the tunnel, he'll be battered
against the sides as the jet carries him before it, and he'll probably
be severely injured by the time it spits him out at the top.”
“Good Lord!” said the inspector, as the realisation of the thing
crept into his mind. “That's a nasty trap to fall into. He's going to get
his gruel, sure enough.”
They had reached the hotel, and Sir Clinton dispatched the
constable to bring ropes, if any were available.
“You don't seem eager to get him out, sir,” the inspector
ventured, as they were waiting.
“I don't know exactly what happened at Peter Hay's to-night,” Sir
Clinton returned, “but I saw enough to know it was something
uncommonly bad that they were trying to do to that girl, inspector. It
must have been something worse than the normal way of putting
the screw on a woman. Our friend in the Blowhole didn't mind doing
that. And, somehow, that makes me feel a bit indolent when it
comes to rescuing him. Let him go through it. Besides, the longer
he's there—if we happen to get him out alive—the more his nerves
will be shaken, and the easier it will be to wring some truth out of
him. You can tackle him at once, before the effect wears off. And I
shan't feel inclined to ask you to be moderate in your questioning
this time. We must get all we can out of him while he's got the
jumps. I've no doubt whatever that Billingford will turn King's
evidence if he gets half a chance—he's that sort. But the other
fellow was deeper in, and we may get more out of him if we can
catch him at the right moment. So I'm not really in much of a hurry.
This isn't a case where my humanitarian instincts are roused in the
very slightest.”
He broke off, seeing Wendover coming out of the hotel.
“Everything fixed up comfortably, squire?” he asked.
 Wendover nodded affirmatively; then, as Sir Clinton invited him
to join them, he amplified his news.
“We got Mrs. Fleetwood down here quite comfortably; and she's
upstairs now. Very shaken up, of course; but she's a plucky girl, and
she hasn't had any bad collapse of her nerves so far, though one
might have expected it.”
“I've a good mind to see her myself now,” Sir Clinton said
thoughtfully. “Did she say anything about what they'd done to her?”
“No. But she asked me to send for Rafford immediately. I didn't
like to worry her with questions.”
Sir Clinton's face darkened.
“It's a nuisance we have to go and fish that creature out of the
Blowhole. I'd much prefer to leave him there to go through it. He
deserves as long a spell as we can give him. But I suppose there
would be a howl if we left him to die. Besides, I want to hang him if
I can. By the way, what about his jovial colleague, Billingford?”
“He's here too,” Wendover explained. “We thought we'd bring
him to the hotel and wait for your instructions. He's safe enough.”
“That's all right. Now here's the constable with the ropes, so I
think we'll have to move on.”
Sir Clinton showed no desire to hurry; nor did Wendover when he
had learned the state of affairs. Both of them were in the mood to
prolong the agony so far as decency permitted. Wendover could not
get out of his mind the expression he had seen on Cressida's face at
Peter Hay's cottage; and when it came back to his memory he felt
that the man in the Blowhole tunnel was getting only a fair
retribution for his crime.
As they came near the mouth of the souffleur, the great fountain
shot up into the night air and broke in spray in the moonlight. Sir
Clinton hurried forward and bent down to listen to the orifice.
“He's there, all right, and still alive,” he reported. “A trifle
unnerved, to judge by his appeals. I suppose we'll have to yank him
out now.”
Armadale also had been listening to the cries from below.
“If we get him out in that state,” he said, with satisfaction, “there
won't be much that he'll keep back when we start questioning him.
He's all to pieces.”
Before they could do any more, the souffleur spouted again.
Wendover, whose imagination was keener than that of the inspector,
was suddenly appalled by the picture conjured up by that wild
fountain jetting from the ground. Down below their feet he could see
with his mind's eye the miserable wretch clinging for life to some
inequality in the tunnel, while the continual blasts of the souffleur
tore and battered at him, and the rush of water made him fight for
his breath. A rat in a trap would be happy compared with that.
“Oh, let's get him out!” he exclaimed. “It must be devilish down
there in the dark, waiting for the next spout.”
“If you're set on seeing him hanged, squire, we'll do our best,”
Sir Clinton conceded, with no sympathy in his tone.
But, even by doing their best, they had great difficulty in rescuing
their quarry from the grip of the death-trap. When at last they got
him to the surface, he was more dead than alive; and three ribs had
been cracked by the last torrent which had flung him against the
side of the conduit.
As they lifted him into safety, Sapcote hurried up from the hotel;
and, after a glance at the torn and haggard face, he recognised the
prisoner.
“That's Aird, sir. Used to be valet at Foxhills once.”
“Well, you can have Mr. Aird, inspector,” Sir Clinton intimated. “If
you give him some brandy, he'll probably wake up enough to part
with any information you want. Don't let your sympathy overcome
you. We must get enough out of him to hang him if we can; and it
depends on putting him through it while his nerve's gone.”
He moved away without another glance at the broken figure on
the ground, and, followed by Wendover, turned his steps towards
the hotel.
“I suppose he calculated on being able to climb to the top before
the jet began to play,” he continued. “Well, he seems to have paid
for his mistake,” he concluded grimly.
At the hotel door, Wendover expected that they would go straight
to the Fleetwood suite; but, rather to his surprise, Sir Clinton
summoned one of the constables and gave him some instructions in
a low voice. Then, accompanied by Wendover, he ascended the
stairs.
“I want to see Cargill for a moment,” he explained, as they
passed the first floor. “I've something to say to him.”
Rather puzzled, Wendover followed him to the Australian's room.
“I happened to be passing,” he said, as he entered in response to
Cargill's permission, “and I dropped in to see how you've been
getting on. Leg all right now?”
“It's a bit better,” Cargill replied. “Won't you sit down?”
“Got enough to read?” Sir Clinton inquired, stepping over a pile
of books which lay near Cargill's couch and picking up one of them.
“I've got one or two I can lend you.”
Wendover was taken completely by surprise; for, without altering
the tone of his voice, Sir Clinton bent suddenly forward and
imprisoned Cargill's wrists.
“See if you can find a pistol anywhere near, squire. It's as well to
be on the safe side.”
He whistled shrilly; and, before the Australian had recovered
from the surprise of the attack, two constables had rushed into the
room and made any attempt at a struggle impossible. Sir Clinton
relaxed his grip.
“I shouldn't kick about, if I were you, Cargill. All you'll succeed in
doing is to reopen that wound of yours. The game's up, you see;
and you may as well take it quietly. We've got some of your friends.”
Cargill's face showed an eagerness at the words.
“Has my brother got off?”
“You mean the pseudo-Derek, I suppose? Yes, he's gone to
ground”—Cargill's expression showed a relief which was quenched
as Sir Clinton continued—“in the same place as you put Paul
Fordingbridge.”
Cargill's head sank at the news.
“I'm afraid I can't stay,” Sir Clinton said, with almost ironical
politeness. “You've given me such a lot of work to do, you know,
lately. I shan't trouble you with questions, because I think we shall
get all we want from your confederates. If you need anything we
can give you, please ask the constables for it. Good evening.”
 In the corridor, Wendover broke into a flood of questions; but Sir
Clinton brushed them aside.
“There's time enough for that by and by,” he said brusquely. “I
must get to the bottom of this business first. We'll go along and ask
if Mrs. Fleetwood can see us for a moment or two.”
Wendover was glad to find, when they entered the Fleetwood
suite, that Cressida seemed to be getting over the worst of the
shock. Her face lighted up as she saw them come in, and she began
at once to thank them. Sir Clinton brushed the thanks aside.
“There's nothing in it,” he said. “I only wish we'd been sooner.”
At the words, Cressida's expression changed, as though some
dreadful thing had been recalled to her. Sir Clinton put his hand into
his pocket and drew out the glass syringe.
“What part did this thing play?” he asked gently.
The sight of it brought back all Cressida's terrors.
“Oh, you were too late!” she exclaimed despairingly. “I'm still
dazed by it all, and that brings it back.”
Under Sir Clinton's sympathetic interrogation, she was soon able
to tell them of the ordeal she had gone through. When she had
finished, the chief constable bent forward and took up the
hypodermic syringe from the table.
“You can sleep quietly to-night,” he said. “There was nothing in
this affair except tap-water. I saw the fellow filling it at the sink as I
passed the window. I'd have stopped him then, but there were only
two of us against three of them, and I had to wait till they were all
in one room. I must say the hypodermic puzzled me. I couldn't make
out what they were after, unless it was more drugging. But there
was nothing in the syringe. I saw him washing it out under the tap
before he filled it. At the worst you may have a sore arm; but the
only germs in the syringe were some that might be in tap-water. The
whole affair was a piece of bluff from start to finish. But it's no
wonder it took you in. They must have staged it well. Be thankful it's
no worse, Mrs. Fleetwood.”
“Oh, I am! You don't know what a relief it is, Sir Clinton. I meant
to go off first thing to-morrow to the Pasteur Institute for treatment.
I wasn't very frightened, once I got out of the hands of these
horrible men, because I knew I could be saved if I got treatment in
time.”
“That's very sensible of you. But you need have no fears about
hydrophobia, at any rate. It was simply a bluff and nothing more.”
Cressida thanked them again, and, in order to escape from her
gratitude, Sir Clinton said good night, promising to return next
morning to tell her anything that she might wish to know.
Wendover had been horrified by the story; and he began to wish
that after all they had left Aird to his fate in the tunnel.
“Brutes like that aren't fit to live,” he declared bitterly, when the
door had closed behind them.
“Some of them won't live much longer, squire, if I can manage
it,” Sir Clinton assured him, in a tone that left no doubt in the matter.
In the hall below, they encountered Mme. Laurent-Desrousseaux,
and at the sight of Sir Clinton her face showed something more than
the mere pleasure of meeting an acquaintance. She came forward
and intercepted them.
“I am most fortunate,” she explained, with a smile which
betrayed her real gratification at their meeting. “I depart to-morrow
morning by the first train, and I was fearing that I might not
encounter you to make you my adieux. That would have been most
impolite to friends so cordial as you have been. And, besides, I am
so very happy that I would wish to be very amiable to all the world.
All the embarrassments that I feared have been swept away, and
everything has arranged itself happily.”
Sir Clinton's face lost the hard expression which it had borne a
few moments before.
“I hope that it is my good fortune to be the first to congratulate
you on your approaching marriage, madame. You have all my wishes
for great happiness.”
Mme. Laurent-Desrousseaux's manners did not allow her to
throw up her hands in astonishment, but her face betrayed her
surprise.
“But it is marvellous!” she exclaimed. “One would need to be a
sorcerer to know so much! It is quite true, what you say. Now that
Staveley is dead, I can espouse such a good friend of mine, one who
will be kind to me and whom I have been adoring for so long. I can
hardly believe it, I am so happy.”
Sir Clinton smiled.
“And you would like everyone else to be happy too? Then you will
perhaps begin at once. Go upstairs, madame, and ask to see Mrs.
Fleetwood. Say that I sent you. And when you see her, tell her that
you married Staveley in 1915. You do not need to say any more.”
Rather puzzled, but quite anxious to do as he told her, Mme.
Laurent-Desrousseaux bade them both farewell, and they saw her
ascending the stair. Sir Clinton gazed after her.
“Easy enough to guess that riddle. One gets a reputation on the
cheap sometimes. Her association with Staveley; then her complete
separation for years; then this sudden need to meet him again in
order to side-track some ‘embarrassments’: obviously she had
married him, and needed a divorce if she was to marry again. I wish
most problems were as simple.”
“And, of course, if she married Staveley in 1915, as she seems to
have done, he committed bigamy in marrying Mrs. Fleetwood?”
“Which means that Mrs. Fleetwood is Mrs. Fleetwood, and that
she's legally married now. She won't be sorry to hear it. That's why I
sent Mme. Laurent-Desrousseaux up there now. First-hand evidence
is better than documents; and, of course, the documents will be
forthcoming if they're required in future. Evidently those three
scoundrels didn't know this latest twist in the affair, or they wouldn't
have tried the trick they did last night. They'd have done worse,
probably, when they got hold of her. If she'd been dead and out of
the way, there would have been no one except old Miss
Fordingbridge to contest that impostor's claim—and she was so
besotted with him that she'd never have dreamed of doing so.”
He paused for a moment or two, as though considering the case;
but when he spoke again it was on a different point.
“You sometimes jeer at me for playing the mystery-man and
refusing to tell you what I infer from the facts that turn up. It's
sometimes irritating, I admit; and now and again I suppose it makes
me look as if I were playing the superior fellow. But it's really
nothing of the sort. In affairs of this kind, one never can tell what
the next turn of the wheel may be; and one might quite well blurt
out something which would give the cue to the very people you
want to keep in the dark.”
“You do irritate me often enough, Clinton,” Wendover admitted.
“I can't see why you shouldn't put your cards on the table. A fact's a
fact, after all.”
“I'll give you just one example,” said Sir Clinton seriously.
“Suppose I had blurted out the fact which I'd inferred about Mme.
Laurent-Desrousseaux's marriage. It was implicit in the story she
told us; but luckily no one spotted the key except myself. Now, just
think what would have happened to-night if that had been common
property. These scoundrels would have known that Mrs. Fleetwood
was legally married to young Fleetwood, since the ceremony with
Staveley was illegal. Therefore, instead of trying the business of the
forced marriage, they'd simply have pitched her over the cliff at the
Blowhole. She'd have been dead by this time; for their only interest
in keeping her alive was to force this marriage with the claimant and
side-track difficulties in that way. Suppose I'd blurted out my
inference, and sent that girl to her death by my carelessness, how
should I be feeling at this moment? None too comfortable, so far as
I can see.”
Wendover had to admit that the secrecy policy had justified itself.
“It would have been a dreadful business,” he confessed.
Inspector Armadale's figure appeared from one of the corridors,
and, catching sight of Sir Clinton, he came over to where they were
standing. His face showed that he had good news to tell.
“I've got practically the whole business out of them, sir.
Billingford gave everything away that he knew about; and the other
chap's nerve was completely gone, so that he couldn't resist
questioning. It's as clear a case as one could wish for.”
He paused, as though puzzled by something, and then added:
“It beats me how you tumbled to the fact that Cargill was one of
the gang, though.”
Sir Clinton ignored the underlying inquiry.
“Was he the brains of the show?” he asked. “I've only a suspicion
to go on there.”
 “Yes, he did the planning for them.”
“And the gentleman with no face collaborated with Aird in the
actual murders? That's a guess, I may say, so far as the Staveley
affair's concerned, though I'm fairly sure of my ground in the other
cases.”
“You're right in that case too, sir. Aird and the impostor fellow
were the actual murderers. Aird'll hang for certain.”
“It'll make a very nice case for you, inspector; and I'm sure you'll
work it up well for the Public Prosecutor. I can seen a laurel wreath
somewhere in the background.”
“But it's you who did most of it, sir. Nobody understands that
better than I do,” the inspector objected, evidently afraid lest Sir
Clinton thought him capable of accepting the credit without protest.
“I came into the thing on a strict understanding that I was to be
a pure spectator, you remember. I'm afraid that at times I got a
shade too zealous, perhaps; but it's your case and not mine. If we'd
made a mess of it between us, you'd have had to stand the racket;
so obviously a success goes down to your account. The subject's
closed.”
Wendover, seeing the inspector's difficulty in framing a suitable
reply to this, intervened to change the subject.
“I see the main outlines of the affair easily enough, Clinton,” he
said, “but I'd like to hear just how you worked it out as you went
along. Any objections to telling me? It'll go no farther, of course.”
The chief constable's face betrayed a tinge of boredom.
“You've lived with this case for the best part of a week. Haven't
you had enough of it by this time?”
Wendover persisted in his demand; but Sir Clinton, instead of
complying, glanced at his watch.
“There's one detective story I'm very fond of, squire: The Hunting
of the Snark. I rank it high in the scale, especially on account of the
number of apt quotations one can make from it. Here's one:

The method employed I would gladly explain

While I have it so clear in my head,
If I had but the time and you had but the brain—
 But much yet remains to be said.

It's far too late to start a long story to-night. I'm dead sleepy. If
you remind me about it to-morrow, I'll do my best; but I will not sit
up all night even to please you.”
The inspector seemed as much disappointed as Wendover at his
superior's decision.
“I'd like to hear it too, sir, if you don't mind.”
Sir Clinton suppressed a yawn with difficulty.
“I don't mind, inspector. Meet us at Neptune's Seat at eleven
o'clock to-morrow morning. It'll be interesting to hear how far wrong
I've gone in some of my guesses; and you can tell me that, since
you've got so much out of these two precious scoundrels to-night.
And now I'll drive you into Lynden Sands—save you the trudge. After
that I really must get to bed.”
 Chapter XVII.
The Threads in the Case
“This hasn't been a tidy case, in the strict meaning of the words,”
Sir Clinton mused, as he chose a comfortable spot on Neptune's Seat
and settled down on it. “It's really an omnium gatherum. It began
long before we appeared on the scene; and the inspector has the
facts about the earlier stages, whilst I've nothing better than
guesswork.”
“What we're interested in, chiefly, is what you thought about it at
different stages in the game,” Wendover pointed out. “If you start
with the Peter Hay case and go on from there, you can tell us what
you saw and what we missed. And at the tail end you can give us
your guesses about the earlier stage. The inspector can check them
from the confessions he got.”
Sir Clinton agreed with a gesture, and began without more ado.
It was evident that he was by no means eager to recapitulate, and
was doing so merely out of good nature.
“The Peter Hay case was crystal clear so far as one side was
concerned. It required no marvellous insight to see what had
happened. It wasn't by any possibility a one-man murder. At least
two men must have been on the spot to overpower Peter and tie
him up. They—or at least one of them—was a better-class fellow, or
Peter Hay would have been in his shirt-sleeves instead of having his
jacket on. And the jacket implied that he knew they were coming
that evening, too. Further, the fact that they had amyl nitrite ready
in their pockets is enough to prove two things. They weren't casual
strangers, for they knew about his liability to cerebral congestion.
And they premeditated killing him in certain circumstances. We
worked out pretty definitely the course of events which led to his
death, so I needn't go over that again. I suppose we were right in
the main points, inspector?”
Armadale, primed with the information he had extracted from
Aird, was able to confirm this.
“They used surgical bandages because they hoped to leave no
marks on the skin, I suppose?” Sir Clinton inquired.
“That was the idea, Aird admitted, sir. He thought they'd
succeeded, and he was surprised to find they'd made a mess of it.”
Sir Clinton smiled, apparently at the thought of Aird's
discomfiture.
“Well, if they bungled one side of the affair, they certainly
managed to leave us in the dark about the second side: the motive
behind the murder. It wasn't robbery, obviously. It wasn't bad feeling
—everyone we examined had a good word for Peter Hay. It couldn't
be homicidal mania—not with two of them mixed up in it. That left,
so far as I could see, only one probable motive at the back of the
thing. Either they wanted to force Peter Hay into something he didn't
like, or else he knew something about them which they were afraid
of.
“Peter Hay wasn't the blackmailing type, from all we learned
about him. That notion wouldn't hold water for a moment. So, if he
had to be silenced, then the information he had must have been
something he'd come across quite innocently. What it was I couldn't
guess. In fact, the whole idea was very vague in my mind, since
there didn't seem anything definite to support it. That was the stage
I'd reached when we sat in the garden and discussed the thing.
“But there was one thing that seemed to lead on to something
fresh—the silver we found in Peter Hay's drawer. That silver was the
murderers' mistake—the silly thing that gives them away. No
sensible person would ever have tried to throw a suspicion of breach
of trust on Peter Hay. The thing was absurd on the face of it, from
what we learned of his character. But these fellows weren't looking
at it objectively. They would have abused their trust in Hay's shoes;
and naturally they saw nothing outré in faking a petty theft and
trying to throw it on his shoulders. That made me think.
 “Why suggest a robbery at Foxhills? For that was what they
evidently intended the police to swallow. As a matter of fact, they'd
have been much better to have left the thing alone. It was a mistake
to fake evidence; it always is. However, they'd done it, and I wanted
to know why they'd done it. So, when the Fordingbridges arrived, we
went off to Foxhills.
“I wasn't surprised to find that the murderers had tried to make
the thing convincing by filling that sack with silver odds and ends. I'd
almost expected something of the sort. But I don't suppose any of
us were misled to the extent the murderers hoped. They thought
we'd be delighted to find confirmation of Peter Hay's dishonesty—
stuff ready packed up for transport. But what one obviously asked
oneself was the question: ‘What's all this meant to cover?’ And the
answer was, so far as I could see: ‘The removal of some
inconspicuous object whose absence won't be noticed in the
excitement over the silver.’
“It wasn't long before I got an inkling of what that inconspicuous
object was. Miss Fordingbridge was able to tell us. But, notice, if
Miss Fordingbridge hadn't happened to come up to Foxhills when we
paid our visit, we'd never have known that the diary was missing.
We'd have missed the main clue in the whole affair. That was where
we were really lucky, and where Aird & Co. had very hard lines.
“There was the diary gone a-missing, anyhow; and the obvious
question to ask was: ‘Cui bono? Who stands to score by its removal?’
“You heard the story of the missing nephew and the power of
attorney given to Paul Fordingbridge. You could hardly help drawing
the obvious conclusion that here at last was a possible motive
appearing behind the Peter Hay case. The stake on the table was
the assets of the Fordingbridge estate; and that was quite big
enough to make murder worth while, if you happened to have a turn
for that sort of thing.
“But when you come to ask: ‘Cui bono?’ you find that the
problem's like one of those quadratic equations we used to get at
school, where there are two answers and one seems just as good as
the other.
 “Suppose the claimant was an impostor, and see where that
leads you. Derek Fordingbridge had spent a lot of time with Peter
Hay in earlier days. The chances were that Peter Hay was the only
man who could give evidence about their joint doings, and that
evidence might form the basis of a damaging examination of the
impostor. Further, the diary would be a priceless thing for an
impostor to lay his hands on. It would supply him with any amount
of irrefutable evidence which he could draw on for his sham
recollections. Clearly enough, if the claim was a fraudulent one, then
the theft of the diary and the silencing of Peter Hay would fit in very
neatly.
“On the other hand, suppose the boot's on the other foot.
Assume that Paul Fordingbridge had some very strong reason for
wishing to retain control over the funds, and see where that leads
you. Remember that he showed no desire whatever to investigate
this claim. He simply denied straight off that the claimant was his
nephew, without waiting for any evidence on the point. That seemed
to me a curious attitude in a trustee; perhaps it struck you in the
same way. And he appeared to be very little put out by Peter Hay's
death, if you remember—treated it very much as a matter of course.
It doesn't take much thinking to see that what holds good for a
fraudulent claimant would hold good for a fraudulent trustee also.
The diary and Peter Hay would be two weak spots for him too.
They'd help a genuine nephew to establish an almost irrefutable
case if he could pass tests applied both by the diary and by Peter
Hay's recollections.
“So, whichever way one looked at it, there seemed to be
something to be said. And, consequently, I got no further at that
stage than being able to say that three things were possible. First,
the claimant might be a fraud, and Paul Fordingbridge merely an
obstinate old beggar. Second, the claimant might be genuine, and
Paul might be a dishonest trustee. Or, third, both the claimant and
Paul might be wrong 'uns.
“Miss Fordingbridge had known her nephew intimately, and she
had identified him straight off, it's true. But we've heard of that kind
of thing before. You remember how Roger Tichborne's mother
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookmass.com