The Problem Space Theory

A problem space
o consists of states

States
contain partial knowledge about the problem
and about the solution

• Operators
o transform states

• Search control knowledge

o guides selection of operators

o guides selection of states

11
 Design Methods Observed

. Design occurs mainly in algorithm design space

I

. Schematic kernel idea quickly selected

. Successive refinement

. Adapt very general operators

o specific knowledge

o means ends analysis

. Symbolic and test-case execution of algorithms

(in the absence of knowledge)

• Discovery and problem-solving in task space

13
 Lessons about Design

. A data flow space for representing partially specified

algorithms

. A variety of design schema (generate and test, divide

and conquer)

• General operators instantiated with knowledge or by

means-ends analysis

• Symbolic and test-case execution to expose problems

and opportunities

. A task space for test-case execution and discoveries

. Discovery involves a prepared problem state

• Discovery requires recognition in the task space

27
 Goals for Algorithm Design

. Design algorithm sketches

from semi-formal specifications

Exploit knowledge if available

Flexible in the absence of knowledge

general problem solving and search
Alternative Automation Methods

. Expert systems
o successive refinement

o mostly data and control structures

o detailed and brittle

o no problem solving or learning

• Formal derivation
o small set of transformations

o guarantee correctness
o good local optimizations

o lack focus, choice, transparency

• Inductive learning
o input-output pairs or traces

o schema matching and heuristic search

o methods have not evolved much

o possibilities if language is logic-based
 Why Study Human Design?

• human thinking is flexible and robust

• programming details learned, not hand coded

. people are resources to start and increment design

knowledge

. interfaces (automatic programmers must

communicate with people)
 Scientific Issues

algorithm design (conquer new Al territory)

formalize algorithm design principles, optimization,
analysis

program synthesis (do better than existing systems)

• .

discovery (conquer new Al phenomenon)

. - •

visual reasoning (critical-path Al phenomenon)

shed light en
kno Isdge/soarch Issues in expert systems
•
*

interaction off domain and programming

knowledge in program synthesis

how humans design and dfecover (confirm cognitive

psychology theory)
The Protocol Analysis Approach

Select a specific task (convexhull construction)

Take protocols from appropriate subjects

o subjects: faculty and graduate students in
computer science

o transcripts: complete text from audio tapes

(videotaping would be better)

• Analyze protocols
o need theory as basis

o very detailed, phrase by phrase

o refine theory as analysis proceeds

 After the Analyses

)
• A simulation system (to test the theory

• An automated design system

An algorithm design assistant

A protocol analysis aid

Other programming tasks

6
 A Task

A point set and its convex hull

 Sample of Protocol

Episode 2 f
L21 [»Minute 2«] Let's start with some point.
L25 Either a point is on the hull or its not, right?
L27 And the question is how to make this decision.

Episode 3
Episode 3.1
L28 Let's take a few points here. (Draws 4 points.)
L29 Well, that's not a good example,
L30 because all four of them are on the convex hull.
[S draws figure with 5 points not all in hull.]
L35 OK, let's suppose I start with a point here.
L36 And I'll just draw a line to some other point, right.
L42 Now I can go in three directions from this point.
L43 [»Minute 3«] I conjecture that
L44 if it's the case that I can choose two points,
L45 such that I can go on either side of the given line,
L46 then this line can't be on the convex hull.
L47 And I had better retreat.
Episode 3.2
L63 Let's retreat, uh, back... back to A.
L65 And choose some other point.
L66 And this time we'll chose C.
L67 Right? So now I have a line from A to C.

8
 Protocol Sample continued

Episode 3.4
L113 [»Minute 6«] And I see that, urn,
L114 all the points are to one side of the line AC.
L115 So I've got a candidate.
L116 Now I'm at C and now I'll go again.
L117 Choose some other point.
L118 Suppose I choose B. [pause]
LI19 A goes to C goes to B.
L120 Urn, now I see that uh, [pause]
L121 there are points on either side of the line CB, right,
L122 there's E and there's A.
L123 [»Minute 7«] I guess I have to look at A
L124 even though I've already got a line segment from it.
L125 So I know that the line CB can't be on the hull.
L126 So I have to retreat back to C.
L128 It looks like I'm not going to come up with
a linear algorithm to do this.
 Selected Episodes

Episode Lines Description

El L1-L15 Acquire problem
E2 L16-L27 Design generate-and-test schema
E2.1 L22-L24 Interrupt(E2): specification of points
E3 L28-L179 Develop algorithm
E3.1 L28-L61 Find test
E3.1.1 L28-L32 Get example figure
E3.2 L62-L67 Decide how to handle test failure
E3.3 L68-L86 Find can discard interior start point
E3.4 L86-L140 Push algorithm all the way to find CH
E3.4.1 L86-L92 Return to previous state after E3.2
E3.4.2 L93-L108 Interrupt(S): Exclude segment not point
E3.4.3 L127-L128 Interrupt(S): Greater than linear
E3.6 L141-L160 Develop initialization
E3.6 L161-L179 Recap algorithm
E4 L180-L256 Analyze complexity
E6 L256-L261 Termination (algorithm is first try)
 A Problem-Space Model

. What are the problem spaces?

o an algorithm description space
o a task domain space (geometry)

. What design methods are used?

• What are the operators and representations

in the design space?

. What is the task-domain knowledge?

 Design Methods Observed

. Design occurs mainly in algorithm design space

• Schematic kernel idea quickly selected

. Successive refinement

. Adapt very general operators

o specific knowledge
o means ends analysis

. Symbolic and test-case execution of algorithms

(in the absence of knowledge)

• Problem-solving in task space

. Discovery

8
 Basic Design Steps

(occur as a result of productions firing)

Select a problem (exposed during symbolic execution)

Find kernel idea or solution plan

Lay down basic structure (components)

Elaborate details of structure

Verify solution (optional, symbolic execution)

Evaluate solution (e.g. time complexity)

14
 Representation in DFS

. Data-flow space (DFS) is main problem-solving space

. represents partial algorithm designs (incomplete

knowledge)
.o new information comes in small increments
o postpone commitments

. small vocabulary of basic components

o arbitrary symbols and assertions
(partially specified or alternative concepts)
o expert vocabulary built up on top

. components characterized by functional properties

(not formal input-output relationships)
o stable data dependencies in initial connections
o internal representation by assertions
o add assertions and additional inputs as needed
o component can be refined to configuration

10
DFS Descriptions of Algorithm

[pt] ,true
•>Generate———>Test—— '.

Assertions:,
on {x}: elements are points
on Test: predicate = is-on-hull(pt)

+------------------------"•----------------------------+
| delete |
V [x] [yx] | true
{x}———>Generate———>Draw———>{z| hull-so-far}———> Test
t | t | false
| y || delete |
+—————+ +——————————+

Assertions:
on {x}: elements are points
on Generate: ordering = random
on {z}: elements are segments
on Test: predicate =
not(points-both-sides(segment,{x}))

16
 D e sc ri p ti o n s of A lg o ri th m
DFS

(new-component 'aeex>ry)
(add-coaponent 'generator 'point-gen)
(add-cooponent 'test 'side-test)
(add-component 'memory nil 'add-elen)

(arid-assertion 'point-gen '(Input- order-next))

on '(te st -pre di cate (o n- s1de te st -I n x-ax1s left))
(add-assert1
'side-test)

•(display-configuration 'subset)
************§
************* f _ * §
* § f •-"> SIDE-TEST 4- *a MEMORY-2 f
i MEHORY-1 > > fOIMT-GEII > c f
f f f f *
* a f
*********#***

a has symbolic Hen S

and has description Item S
b has symbolic 1te« POINT-4
c has symbolic 1te« POINT-SET-3
Component GENERATOR-1 has assertions:
INPUT-ORDER-NEXT
Component TEST-1 has assertions:
TEST-PREDICATE-1: X-AXIS LEFT))
(TEST-PREDICATE (ON-SIDE MAIN-INPUT

16
Applying Operators in DPS

Edit DPS configuration - add,-modify or remove:

o process components

links

input and output ports

assertions

items

Map the operator onto the situation

o simple instantiation by specific task-space or
component rules

o means ends analysis with subgoaling

18
 I I

Review of Design Principles

. A data flow space for representing partially specified

algorithms

. A variety of design schema (generate and test, divide

and conquer)

• General operators instantiated with knowledge or by

means-ends analysis

. Symbolic and test-case execution to expose problems

and opportunities

. A task space for test-case execution and discoveries

. Discovery involves a prepared problem state

• Discovery requires recognition in the task space

23
 I I

Future Directions

. discoveries

• example construction

• low level MEA adaptation

. representations

. algorithm analysis

• code generation/ program synthesis

• algorithm memory

. analogy

• learning

• interactive assistant

protocol analysis aid

collect more protocols

24