Chapter 5

DATA FLOW ARCHITECTURES

1
 Architectural Styles
The following architectural styles serve as
patterns of subsystem decomposition that can
be used in different systems.
Styles categories:
1. Data Flow Architectures
2. Data-Centered Software Architecture
3. Hierarchical Architecture
4. Interaction-Oriented Software Architectures
5. Distributed Architecture

2
Data Flow Architecture Styles
Data Flow Architecture Styles:
Views the entire software system as a series of
transformations on successive sets of data. Data and
operations on it are independent of each other.
Used in any application involving a well-defined series of
independent data transformations with defined input and
output, such as data streams.
E.g. compilers and business batch data processing; neither of
these require user interactions.

3
Example of Data Flow

4
Subcategories in the Data-flow style

Data Flow Architecture Styles:

Batch Sequential
Pipe and Filter
Process Control

5
 1. Batch Sequential
Traditional data processing model that was
widely used from 1950 to 1970.
Each subsystem cannot start its process until its
previous subsystem completes its computation.
Data flow carries a batch of data as a whole
from one subsystem to another.
A script is often used to make the batch
sequence of the subsystems in the system.

6
Batch Sequential Architecture

Validate Sort Update Report

Data flow data

transformation

7
 Batch Sequential in Business Data
Processing
Rejected
transaction

-123 U
Validated Sorted
Transaction File
transaction transaction

222 U
222 U 111 I
111 I
Validate 111 I sort 222 U
333 D
333 D 333 D
-123 U

100 ----
Update
111 ----
200 ----
222 ----
Generate Report 444 ----

Reports 100 ---

Updated 200 ---
Master file 222 ---
333 ---
444 ---

Master file
8
Implementing Batch Sequential

9
 Batch Sequential
Applicable domains of batch sequential
architecture:
Data are batched.
Intermediate file is a sequential access file.
Each subsystem reads related input files and writes out put files.

10
 Batch Sequential
Benefits:
Simple divisions on subsystems.
Each subsystem can be a stand-alone program

Limitations:
Implementation requires external control.
It does not provide interactive interface.
Concurrency is not supported and hence throughput remain slow
High latency.

11
 2. Pipe and Filter
Is a type of data flow architecture where the flow is
driven by data.
The subsystems are called “filters,” and the
associations between the subsystems are called
“pipes.”
Components are filter
Connectors are pipes
The particular property attribute of the pipe and filter
architecture is its concurrent and incremented
execution.
12
 Pipe and Filter Block and Sequence Diagrams
data source: filter1: pipe: filter2: data sink:
Data Source Filter pipe Filter Data Sink

read

Filter1

W
Filter2
R

write

Data Source Filter1 Filter2 Data Sink

pipe
13
 Pipe and Filter
The pipes move data from one filter to another.
A pipe can carry binary or character streams. An object-type data
must be serialized to be able to go over a stream.
Each filter knows only the content and the format of the data
received on the input pipes, not the filters that produced them.
Each filter is executed concurrently, and synchronization is
accomplished via the pipes.
An object-type data must be serialized to be able to go over a
stream.
Filters can run in separate threads of the same process.
A data stream is a first-in/first-out buffer which can be a stream of
bytes, characters, or even records of XML or any other type.
14
 Pipe and Filter
Filters are independent(no shared data)
Examples:
UNIX shell
Parallel programming
Distributed systems

15
 Pipe and Filer - Example
Unix shell command: ls invoices | grep -e
August | sort
The filters are : ls, grep and sort commands.
The vertical bar (|) represents the pipe.
ls lists the files in the present location the terminal is
in.

16
 Pipe and Filter
There are three ways to make the data flow:
Push only (Write only)
A filter may push data in a downstream.
Pull only (Read only)
a
A filter may pull data from an upstream.
Pull/Push (Read/Write)
D
I
A filter may pull data from an upstream and push transformed
data in a downstream.

1 write
only

17
 Pipe and Filter
There are two types of filters:
000000
An active filter pulls in data and pushes out the transformed
as
data(pull/push); it works with a passive pipe that provides read/write
mechanisms for pulling and pushing.
The pipe and filter mechanism in Unix adopts this mode.
A passive filter lets connected pipes push data in and pull data out. It
works with active pipes that pull data out from a filter and push data
into the next filter. The filter must provide the read/write mechanisms in
this case. This is very similar to data flow architecture.
Variations:
Pipelines: linear sequences of filters.
Bounded pipes: limited amount of data on a pipe.
Typed pipes: data strongly typed.

18
 Pipe and Filter Sequence Diagram
active passive active
Pull only Pull /push Pull / push Pull /push Push only

data source Filter 1 Pipe Filter 2 data sink

R ead R ead

( blocking )

Wr ite

R ead

R ead

Wr ite

Wr ite

R ead

19
Implementing Pipe and Filter
The Java PipedWriter and PipedReader classes work
together to synchronize the data stream pulling and the
data stream pushing between active filters.
Pipe
Filter
+Read()
+Write()

Data Source Data Sink

+read() +write() 20
 Pipe & Filter in Java
• The Java API provides the PipedWriter and PipedReader
classes in the java.io package; this allows for filters to
push data into downstream or to pull data from the
upstream.
• Filters can run in separate threads and they can be
synchronized in a parallel manner by PipedWriter and
PipedReader.
• The pipeFilter Java main program runs two threads in
addition to itself: Filter1 and Filter2.
• They are connected by a synchronized pipe (PipedWriter
and PipedReader).
• Filter1 writes to the pipe and Filter2 reads from the pipe.

21
Implementing Pipe and Filter

22
Implementing Pipe and Filter

23
Implementing Pipe and Filter

24
 Pipe and Filter
Applicable domains:
The system can be broken into a series of processing steps over
data streams, and at each step filters consume and move data
incrementally.
Suited for systems that apply transformations to streams of data
without interventions by users.
For example, batch processing and business data processing
systems.
Not suited for systems that require more complex interactions
between components such as an information management
system or an interactive system.

25
 Pipe and Filter
Benefits:
Concurrency: It provides high overall throughput for excessive data
processing.
Reusability: Encapsulation of filters makes it easy to plug and play.
Modifiability: Filters can be modified with ease as long as the I/O interfaces
are not changed.
Simplicity: It offers clear division between any two filters connected by a pipe
Flexibility: It supports both sequential and parallel execution.
Limitations:
It is not suited for dynamic interactions.
A low common denominator is required for data transmission in the
ASCII formats since filters may need to handle data streams in different
formats, such as record type or XML type rather than character type.
Overhead of data transformation among filters.

26
 3. Process Control
It is for the embedded system software design, where the
system is manipulated by a process control variable data.
It decomposes the whole system into subsystems (modules)
and connections between subsystems.
There are two types of subsystems:
an executor processing unit for changing process control variables
a controller unit for calculating the amounts of the changes.

27
 Process Control
E.g. the data flow of a feedback close-loop process
control system. The connections between the
subsystems are the data flow.
Input variables

C ontroller Process
Manipulated C ontrolled
Set Point ( s )
variables ( m) variable (c )

Æ= s Ğ c, m = f(Æ)

28
 Process Control
A process control system must have the following process
control data:
Controlled variable: a target controlled variable such as
speed in a cruise control system. It has a set point goal to
reach. The controlled variable data should be measured by
sensors as a feedback reference to recalculate manipulated
variables.
Input variable: a measured input data such as the
temperature of return air in a temperature control system.
Manipulated variable: can be adjusted by the controller.

29
 D
q The input variables and manipulated variables are
applied to the execution processor which results in a
controlled variable.
q The set point and controlled variables are the input data
to the controller, the difference
FIT of the controlled variable
value from the set point value is used to get a new
manipulated value.
q good examples:
Ø Cars cruise-control systems and building temperature control
systems.

30
 Process Control
Applicable domains of process control
architecture:
Embedded software systems involving continuing actions
Systems that need to maintain an output data at a stable level

a
The system can have a set point—the goal the system will reach
at its operational level.

31
 Process Control
Benefits of close-loop feedback process control
architecture over open forward architecture:
It offers a better solution to the control system where no precise
formula can be used to decide the manipulated variable.
The software can be completely embedded in the devices.

Devices

32
 Summary
• The data flow architecture decomposes a system into a
fixed sequence of transformations and computations.
• There is no direct interaction between any two
consecutive sub-systems except for the exchange of
data through data flow links.
• There is no data sharing among sub-systems in data
flow architecture.
• It is not suitable for interactive business processing.
• Three types of data flow architectures were discussed.
• The pipe & filter is an incremental data transformation
processing model and runs concurrently.

33
 Summary
• The data flow and the control flow in pipe & filter are
implicit.
• Reading and writing I/O files drive the data flow in batch
sequential architecture; its control flow is thus explicit.
• The batch sequential architecture may cause bottlenecks
because it requires batched data as input and output.
• The process control architecture is another type of data
flow architecture where the data is neither batched
sequential nor pipelined streamed.
• The mechanism to drive the data flow comes from a set
of variables that controls the process execution.

34
 Summary

The design guidelines for a data flow based

software system:
– Decompose the system into series of process steps;
each step takes the output of its previous step.
– Define the output and input data formats for each step.
– Define the data transformation in each step.
– Design pipelines if concurrency is necessary.

35