,-TR-92-2129

954 AD-A254 IhIIIIIIII

INTRODUCTORY REVIEW OF TARGET DISCRIMINATION CRITERIA

John Donohue

Dynamics Research Corporation 60 Concord Street Wilmington, MA 03087

31 December 1991

DTIC ELECA
AUG 12 1992

SAU
Final Report 16 April 1991-31 December 1991

APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED

++

PHILLIPS LABORATORY AIR FORCE SYSTEMS COMIMAND HANSCOM AIR FORCE BASE, MASSACHUSETTS 01731-5000

92-22628 92
8 lo I- o

.. . .; : :- : ..- .

-- --

- . .

-.

..

.,. 7

-_--: ..

-.

.--

._.- ._--. 7,.____

_--.

-.--.-.

_ ,

.:K_'

. ,.-.

.- '

_.

.- '-.

.. .

-_

----------

"1hds technical report has been reviewed and is approved for publication"

Atmospheric Structure Branch

F4W A. M-LATCM, Director Atmospheric Sciences Division

This report has been reviewed by the ESD Public Affairs Office (PA) and is releasable to the National Technical information Service (NTIS).

Qualified requestors may obtain additional copies from the Defense Technical Information Center.

If your address has changed, or if you wish to be removed from the mailing list, or if the addressee is no longer employed by your This organization, please notify PL/IIh, Hanscom AN', MA 01731-5000. will assist us in maintaining a current mailing list. Do not return copies of this report unless contractual obligations or notices on a specific document requires that it be returned.

REPORT DOCUMENTATION PAGE ftfolopal. ~ ~ ~~ ~

40,0 .al
4a 40,610 fuI A fftd 41M d (Oqh.q.A 04a~lOAo ,.41 t M~ratton

"U4 ~qO ~ o q~"fQn~?n ~~M

OMB Nom 0A04-0ve

#S'U

o'o'.o."ce.n'ol tee t
ctfof, t

Wit! COMA14"tS

,#vqw4 9fq IM 00t14101. tfOi

1. AGENCY USE ONLY (Leave bn)2.REPORT DAT

J. REPORT T1PE ANDJ OATY4 CnIIERED

31 Dec 199
4. TITLE AND SUBTITLE

Scel 1t.i~i cinal

1~ 10

~~~

1 i91 Dan Q1 'S.FUNDING NUMBERS

INTRODUCTORY REVIEW OF TARGET DISCRIMINATION CRITERIA 6. AUTHOR(S) JOHN DNOHUEF-Y712191037714-

F19628--89-D-00ll
PR:
TA:

2688
268801

PLPR INos. :PY?1219203706

7. PERFORMING ORGANIZATION NAME(S) AND ADORESSME) I. PERFORMING ORGAN12ATION REPORT NUMBER

DYNAMICS RESEARCH CORPORATION 60 CONCORD STREET WILMINGTON, MA 03087

9. SPONSORING/MONITOIRING AGENCY NAME(S) AND ADORESS(ES)

E- 19290U

Phillips Laboratory
Itanscom AFB, KA 01731-5000

10. SPONSORING/ MONITORING AGENCY REPORT NUMBER

PL-TR-92--2129
Contract Manager: Capt Frank Hughea/GPM

_____________

11. SUPPLEMENTARY NJOTIS This report was funded by Philips Laboratory (CPAA) under the Electronic Systems Division TEMS contract with DRC. 12s. OISTRIBUTION /AVAILARIIIITY STATEMENT Approved for public release; Distribution unlimited 12b. LVISTRINUTION CODE

IJ. ABSTRACT (Maximum 200 wohis)

This report presents results of investigations to determin, the validity of certain assumptions and criteria used in the development of the Sensor Performance Modal (8PM), which is part of the Electro-Optical Tactical Decision Aids (EOTDA) software program. The investigations were based upon a literature review of works by John Johnson, Herschel Self, Lucien Biberman, David Schmieder and other authors knowledgeable in the electro-optical field. The investigations sought to answer several issues related to target
discrimination criteria and the role of Johnson's criteria in defining

target discrimination.

Among the issues to be addressed weret

1)

the

levels of target discrimination, 2) the definition of each target discrimination level, 3) the definition of Johnson's criteria, 4) the use of Johnson's

criteria in target discrimination, 5) the factors that influence the accuracy of Johnson's criteria and 6) the implication of Johnson's criteria inaccuracies on the application of these criteria to the Electro-Optical. Tactical Decision Aide. IS. NUMBER Of PAGES 14, SUMIEC? 19RAS DISCRIMINATION CRITE.RIA
JOHNSON CRITERIA
III 5URITY CASIPFICAIN $.

SENSOR PERFORMANCE MODEL ECURITYfLASSIFICATION UNCLASSIFIED It. SECURITY CLASSIFICATION UNCLASsIFIED

1.PIECD PIECo

34

20. LIMITATION OP AOSTRACT7

UNCLASSIFIED NYSN 7540C-l-2SO.SSOO

UNLIMITED Standard Form 298 (Rev 2.89)

oftf6 ovi 404% $JW.I

TABLE OF CONTENTS

INTRODUCTION BACKGROUND TECHNICAL DISCUSSION CONCLUSIONS AND RECOMMENDATIONS

16 18

REFERENCES APPENDIX APPENDIX A B

19 27

Acceston For

NTIS
DTIC

CRAWI
TAE;

ULij

UWaimouorod

Justitication ...............................

BY ................................. Di6t. ibutiol I

Availability Codes Dist Avail and I or Sptcial

iii,

This page intentionally left blank.

iv

----. -.r-1-~n

-.-

--

.........

-rY

'-

'r

'

,:

-" -

-"

'-

'

INTRODUCTORY REVIEW OF TARGRT DISCRIMINATION CRITERIA

INTRODUCTION

This report contains results of investigations undertaken by Dynamics Research Corporation (DRC) at the request of Phillips Laboratory, Geophysics Directorate, Hanscom AFB and is submitted in accordance with CDRL Item No. 212 of Contract F19628-89-D-0011. The purpose of the investigations was to determine the validity of certain assumptions and criteria used in the development of the Sensor Performance Model (SPM), which is part of the ElectroOptical Tactical Decision Aids (EOTDA) software program. The investigations were based upon a literature review of works by John Johnson, Herschel Self, Lucien Biberman, David Schmieder and other authors knowledgeable in the electro-optical field. The investigations sought to answer several issues related to target discrimination criteria and the role of Johnson's criteria in defining target discrimination. Among the issues to be addressed were: 1) the levels of target discrimination, 2) the definition of each target discrimination level. 3) the definition of Johnson's criteria, 4) the use of Johnson's criteria in target discrimination, 5) the factors that influence the accuracy of Johnson's criteria and 6) the implication of Johnson's criteria inaccuracies on the application of these criteria to the ElectroOptical Tactical Decision Aids.
BACKGROUND

DRC was initially requested to answer six questions related to the Electro-Optical Tactical Decision Aids Validation Plan. These questions were: 1. 2. What are the Johnson Criteria that have been applied to EOTDA range calculations? What are the definitions of detection, recognition, and identification? What is the role of "clutter" in these processes? Can we quantify clutter in a scene? What concepts do we need to define for measurr-ients and validation? Models of targets with uniform thermal characteristics could be used to test MRT criteria. How can we test MDT with various stages of clutter? What data parameters must be available or measured our analysis? for

3. 4.

Based on EOTDA sensitivity analyses, under what different weather events should the analysis be conducted (e.g. clear, fog, haze, precipitation, etc.?) Do any current data sets have sufficient validation? data for SP!'

5.

6.

What volume of data is needed to have a statistically representative analysis?

Questions 1, 2 and 4 were to be answered during the current fiscal year, while questions 3, 5 and 6, relating to TDA data, were to be deferred until FY'92. This report covers only questions I and 2. Question 1, which addresses the Johnson criteria is covered in the Technical Discussion section of this report. Question 2, while addressed in the Technical Discussion section, is also covered in Appendix A with respect to the definitions. Appendix A contains additional definitions than those requested, since many of these terms are used extensively in the EOTDA literature. Appendix B contains a list of acronyms which have been collected from a variety of sources in the EOTDA literature.
TECHNICAL DISCUSSION

In the mid- and late-1950's the military was developing electro-optical image intensifiers which provided enhanced visual surveillance capabilities under conditions of limited visibility. These intensifiers permitted significant increases in visual target acquisition range and image display capability. The complexity of these intensifiers and associated target acquisitions systems required a methodology for evaluating performance characteristics. John Johnson (1) of the U. S. Army Engineer Research and Development Laboratories (ERDL), Fort Belvoir, Virginia presented a paper at the October 1958 Image Intensifier Symposium which addressed "methods and procedures for the solution of problems involving military visual surveillance through image intensifier devices under low light level conditions." The paper, entitled "Analysis of Image Forming Systems" contained results of experiments conducted at ERDL to determine the resolution required of a system to perform certain target interpretation processes identified by Johnson as detection, shape orientation, shape recognition, detail recognition and target identification. He also referred to these as "decision responses". These were regarded as distinct "degrees of freedom" or "states" of an image intensifier system. These "decision responses" are dependent upon "the characteristics of the optical message, the properties of the intensifier device, and the physiological response of the human readout processes". The paper went on to develop a series of relationships between a nunber of variables in the "space domain" and the "space frequency domain". Johnson indicated that the "space domain" approach was tedious and cumbersome and had to be repeated for each view of each target of interest. The space frequency approach greatly simplified the analysis. However the abstract frequency spectra had to be related to real targets. Through a series of experiments using trained observers, Johnson was able to develop a rethod relating the "decisiun response" by normalizing resolve'd

 -.!

. ..

.. -..

. ..

. .

... .-.

_-- .

---

--

- .

,-. _ - - .

.. .

-_ .---

..-

.. .

line pairs for a critical target dimension. He found that the minimum resolution required for a particular "decision response" was nearly constant for a group of nine military targets. The results of these experiments are tabulated in Table 1. The data show that the minimum resolution required for a particular decision activity is a constant for nine military targets within a maximum error excursion of +25%.

TABLE 1. TARGET Broadside View

OPTICAL IMAGE TRANSFORMATIONS

Resolution - Line Pairs per Minimum Dimension Detection .90 .75 .75 .75 1.0 1.2 1.2 1.5 1.0
.25

Orientation 1.25 1.2 1.2 1.2 1.50 1.50 1.5 1.8 1.5
1.4 + .35

Recognition Identification 4.5 3.5 3.3 3.5 4.0 4.5 4.3 3.8 4.8
4.0 + .8 6.4

Truck M-48 Tank Stalin Tank Centurion Tank Half-track Jeep Command Car Soldier (Standing) 105 Howitzer
Average

8.0 7.0 6.0 6.0 5.0 5.5 5.5 8.0 6.0

1.5

1.0 +

-4-

These target transformations were found to be independent of contrast and scene signal to noise ratio as long as the contrast in the resolution chart was the same as the contrast in the complex target. These results indicated that complex military targets may be considered equivalent in a visual sense to repetitive resolution patterns of appropriate spatial. frequencies for each decision level. The results are general, at least for the limited group considered, and are independent of distance. They simplify considerably the determinations of decision level activity in any imaging system, since it is only necessary to determine the angular resolution characteristic as a function of a few parameters. These transformations, which provided target discrimination criteria based upon resolution, gained widespread acceptance within the industry and became the accepted criteria for performance

measurement of optical systems. the "Johnson Criteria".

These criteria were referred to as

The methodology developed by Johnson was simple and straight forward. A target was moved to a range where it was just barely detectable. A bar pattern was placed in the field of view and its spatial frequency was increased until it could barely be resolved at the same range, i.e. the number of lines on the bar pattern was increased until they could no longer be distinguished. The spatial frequency of the pattern was specified in terms of the number of lines in the pattern subtended by the object's minimum dimension as illustrated in Figure 1. The same methodology was used for orientation, recognition and identification.

~d'S

FIGXUR

1.

METHOD OF OPTICAL IKAGE TRKNSFORMATION

Since Johnson set the contrast in the resolution chart to the target his the target, in contrast as the same level transformations were independent of contrast and scene signal to These experiments were conducted under laboratory noise ratio. conditions, and involved only electro-optical sensors. One of the shortcomings of Johnson's paper was that he did not define the meaning of detection, orientation, recognition and Lucien Biberman (2) provided definitions as shown identification. in Table 2. TABLE 2. LEVELS OF OBJECT DISCRIMINATION

Discrimination Level Detection Orientation

Meaning An object is present

approximately is object The symmetric or asymmetric and its orientation may be discerned to which the class The belongs may be discerned house, truck, man, etc.) object (e.g.,

Recognition

Identification

The target can be described to the limit of the observer's knowledge truck, pickup motel, (e.g., policeman, etc.)

Other definitions of this terminology are used, but most For example, the Air Force conform closely to those of Table 2. (3) defines target detection as "an object is detected although no further target information can be determined"; target orientation as "target symmetry and dimensional shape are noted"; target as "the target can be placed (e.g., the target is a recognition house, a truck, or a tank)"; target identification as "the target can be described to the limit of the observer's knowledge (e.g.. Further the target is a motel, a pickup truck, or an enemy tank". definitions of these and other terms are contained in appendix A. Johnson did not explain all of the factors which influence target detection, orientation, recognition and identification. There are numerous factors involving not only the target, but the Herschel Self (4) has identified a observer and overall scene.

Self indicated that number of these factors, as shown in Table 3. not all factors listed in each group are independent of other is neither systematic factors under the same heading and the list However it does indicate the complexity involved in nor complete. Many of these target detection against a complex background. factors were downplayed or ignored in Johnson's writings.

TABLE 3.

FACTORS INFLUENCING TARGET DETECTION AND RECOGNITION

The scene 1. 2. 3. The size of the picture or displayed image. Numbers, sizes, shapes, and scene distribution of areas contextually likely to contain the target object. Scene objects: numbers, shapes and patterns, achromatic (hue, saturation, contrasts, colors, and color lightness), acutance, amount of resolved details, all both absolutely and relative to the target object. Scene distribution of objects. Granularity, noise. Total available information content and amount of each type of information. Average image brightness or lightness. Contextual cues to target object location. Location in the image format. Location in the scene Shape and pattern. acutance, lightness or resolution(s), Size, color, brightness. Type and degree of isolation from background and objects.

4. 5. 6. 7. 8. The target 1. 2. 3. 4. 5.

The observer 1. Training

2. 3. 4. Experience Native ability Instructions and task briefing

5. 6. 7. 8.

Search habits Motivation Compromise on speed versus accuracy Assumptions.

Another shortcoming of Johnson's paper is the abser"c of the Without the raw data which was used to compute the average values. raw data it is impossible to determine the accuracy of the averages used. Distribution curves of the laboratory readings should have been provided so that the reader could determine the deviations Also al .it the average value over each target by each observer. and experience of the observers are not the qualifications 6

presented. There are numerous characteristics of observers that can change the data dramatically, including the visual accuity of the observer, his training and experience, hia search habits in seeking targets, how well he was briefed, his understanding of the task, the amount of time available, his motivation and other physical and psychological factors. These factors have been found to be significant in experiments run by Self (4), Self states that "upon close examination it is seen that cany variables or factors influence detection and recognition of objects. The effects become especially apparent when the time to view an image is limited. Even the common image quality measures in use today turn out to be complex in application and in specification of the obtained values. For example, at different points in the image and in different directions at any given point, obtained image resolution varies. In making predictions of observer performance, it is clear that the effects of even the simple quality aspects depend upon the state of adaptation, visual capabilities, training, instructions, motivation, etc., of the observers. Even observer search patterns are important. Clues from briefing and/or the image context can make a very large difference in performance. Similarly, time to find targets or the probability of finding them within specified time limits is greatly influenced by "image complexity variables, several of which are The influence of target-background included in the term 'context'. interaction effects is clearly established." Self made the following observations: i. When a target is not quickly found, searchers tend to 'oversearch' (repeatedly search) likely areas and completely avoid areas dismissed as either unsuitable or as suitable but not containing the target. Frequently targets in contextually unlikely places are not found for minutes even though of adequate size, resolution, and contrast for quick recognition when examined. Despite instructions and training, few observers systematically search a scene until after initial rapid scene-appropriate search fails to find a target. Clearly, search is neither purely systematic nor purely random. Observers sometimes forget which areas have been searched and assume that they have searched an area when they have not. This leads to large time scores when the target is there. Other things being equal, target objects closer to the center of the picture tend to be found quicker. Numerous moving image studies show that subjects under high pressure do hurry to find targets much quicker than those under little or no pressure. 7

2.

3.

4. 5.

6.

Some observers quickly find targets that others with equal training find only after extended search time or do Chance factors, such as looking at the not find at all. right place early in search, are clearly important. However, some subjects are consistently as much as two to three times faster than others over dozens of targets and scenes, and across studies. identically-appearing Averaged across many subjects, target images vary drastically in the time required to detect and to recognize them in different backgrounds In other words, there is a strong target(scenes). background interaction. When briefing target pictures are rotated relative to the target in the scene, or are of a different size or lightness, target detection and recognition are slower.

7.

8.

3ohnsor's paper downplayed the significance of the oubjective nature of the data and implies a much greater objectivity and precision than was actually achieved. Soma other shortcomings of Johnson's original paper were: 1. 2. 3. 4.
-.

the sampie targets were relatively small, with sr'ill length to width tatios, including clutter, were not the targct backgrounds, defined, the characteristics of the targets were not provided, targets with highly recognizable features were not addressed, and the experiments involved optical systems only, with no application to infrared systems.

Johnson made a Despite the shortcomings of the paper, significant contribution to defining requirements for military target acquisition systems by enabling measurement of performance requirements in terms of bar target equivalent spacial frequency. ioh'son's experiments were very useful in providing criteria which -reatly simplified measurement and test of image intensifiers. Some sixteen years after publishing his original paper, .!ohnscn collaborated with Walter Lawson on another paper (5) which to cover infrared rodiiied the original work and extended it They added another decision response term, identified as systems. target "classification", which they defined as "the visual act corresponding to perception of the general class of military Much of the tracked versus wheeled vehicles". targets e.g. nomenclature in Johnson's original paper was changed in the second What was referrLd to as "decision response" is now referred rpiper. to -s "discrimination levels". The later paper eophasizes that the values for the various discrimination levels are "representative values, essentially average values required for 50% probability, and must not be construed as rigid values or optimum values for 8

specific targets and target aspects". They thus recognized the less than precise nature of the empirical data. Another change in nomenclature is 50% probability" from "resolution per later paper changes the average values for 50% probability, as shown in Table TABLE 4. the change to "cycles for minimum dimension". The shown in Table 1 to cycles 4.

DISCRIMINATION LEVEL DEPENDENCE ON CYCLES PER CRITICAL DIMENSION CYCLES FOR 50% PROBABILITY 1.0
1.4 2.5 3.0 6.4

DISCRIMINATION LEVEL DETECTION

ORIENTATION CLASSIFICATION RECOGNITION IDENTIFICATION

A comparison of Tables 1 and 4 shows that the values associated with the various discrimination levels remained the same, except for recognition, which was reduced from 4 to 3, and classification which was added. The 1974 paper also introduced the use of minimum resolvable temperature (MRT) for thermal viewers. MRT is used to determine the maximum subjectively resolvable frequency for the effective target temperature difference. The procedure developed by Johnson and Lawson for thermal viewers involved five steps, as shown diagrammatically in Figure 2. The first step requires determination of the effective target temperature difference and determination of the minimum dimension. is the area The temperature difference associated with the target weighted mean temperature difference calculated from actual target signaturas; atmospheric properties are then used together with this temperature difference to establish the effective target temperature (apparent temperature) difference at the observer station. The second step requires calculation of the device minimum resolvable temperature (as a function of spatial frequency); third, determination of the number of resolvable cycles across the target minimum dimension; fourth, determination of the recognition probability (or other discrimination level) from the number of resolvable cycles; fifth, construction of the recognition probability versus range function.

DETEMINZ TARGET SIGNAL AND HEIGHTO

EYlTSM CALCULATI

it

-F'

4t4

SISATY&A

FREQUENCY

34
WITS SIYSTEM IRESOLUTION( RESOLUTION/TARGET CALCULATE VSi RANGE UTILIZE TAROET TRANSFORIM PROI&SILZTY FUNCTION

5
CALCULATEC*AI?10O A$ FUNCTION PROIAB[ILTY Op RANIGE

5
.5101.1 RANGE
0

4
0

U.o.. 4
3.0
9

4.5 -1 HEIGET

RISOLUZOXI/tA*GET

PANGS

FIGURE 2.

GENERAL MIGHT VISION PREDICTION PROCEDURE

(N),

a target To determine the number of resolvable cycles across relationship: Johnson developed the N - To Ja
R

where:

TO = maximum resolvable frequency h = minimum target dimension

R = range to target

Transform He also developed a function he referred to as the Target from derived was function This Probability Function (TTPF). of ability the which in experiments laboratory psychophysical function a as observers to discern the nature of tactical targets 10

=N2

of

resolvable

cycles

across

the

target

minimur dimension

was

measured. The sensor used for these experiments was a low light The TTPF is a target detection and recognition thermal viewers. probability function and is derived from laboratory and field Figure 3 shows plots of data for target detection and experiments. recognition. The two plots to the left are derived from laboratory experiments and the plot to the right is derived from field data. Johnson considered the TTPF to be of fundamental importance in the The TTPF replaced the earlier optical image prediction process. transformations (Table 1) and became the new "Johnson Criteria". The new criteria had broader applicability since it was valid for ' oth optical and thermal viewers.

DETECTION

RECOGNITION

1.0
FIELD

C2

RESOLUTION/TARGET HEIGHT

FIGURE 3.

TARGET TRANSFORM PROBABILITY FUNCTIONS

Another significant contribution of Johnson's 1974 paper is the extension of the criteria to targets with large length to width paper dealt exclusively with targets Since the first ratios (L/W). questions had been raised concerning the with small L/W (2:1), to in particular the TTPF, applicability of the criteria, recognition and identification of ships and aircraft, i.e. targets Johnson ran experiments, similar to his earlier .. 'Kth large L/W.

11

work, using scale models of aircraft and ships which were assembled and mounted in a viewing studio to permit accurate light control. A repetitive square wave resolution pattern with a seven to one aspect ratio was mounted in the target plane. The contrast and

reflectance of the bar pattern precisely matched those of the scale models. An image intensifier equipped with a variable resolution "aperture plate" was used to view the scale models and the resolution chart simultaneously. A pilot experiment was run in which the ability of a single observer to recognize and identify
targets as a function of resolution and -aspect angle was

determined.

The resolution of the viewing device was set to zero

at the beginning of each run and then progressively increased until the target was correctly recognized. Table 5 shows the results of Johnson's experiments with large L/W targets which vary from 6-to-i for a fighter aircraft to 16-to1 for a battleship. Also included are two small L/W targets (2-to1) from his original studies. The table lists the average number of cycles for side view recognition for each target and is given in resolution per minimum target dimension. Johnson stated that "for aspect angle conditions which vary from side viewing (900) down to near frontal viewing (30"), a single target recognition criterion of 3 cycles would cover practically all targets with a maximur error of about 30%". Johnson noted that this data is based upon observations of a single observer in a limited number of trials.

TABLE S.

SIDE VIEW RESOLUTION CRITERIA OF TACTICAL TARGETS

TARGET

ASPECT RATIO

RESOLUTIONIMIN TARGET
DIMENSION FOR AECQOGION

Battleship Destroyer

16/1 12/1

2.8 4.5

Air:raft carrier P. T. Boat

15/1 12/1 12/1 6/1 9/1 2/1 2/1

2 3.0 3.0 3 cycle 2.5 cycle 3.5 3.5

Passenger Ship Fighter Jet Transport "lank APC

12

The experiments showed that while detection and recognition criteria for large L/W targets are remarkably close to small L/W targets when viewed from the side, they differ appreciably when For example, in the case of the viewed from the front or rear. battleship, the resolution required for viewing angles of 900 to 300 For viewing angles between 303 and 0 was approximately 3 cycles. (front view) the resolution required nearly doubled to 5.5 cycles. Johnson indicates that "the actual behavior at small aspect view The complex nature of angles is complex and rather unpredictable". small aspect angles was true for aircraft targets as well as ships. Johnson also noted that "recognition probability is determined The larger the perceptible by the perception of target features. feature set for a given target, and the more detailed the features the greater will be the recognition this set, comprising He indicated that in view of the multiplicity of probability". must be anticipated that "s, it target signatures and obh-, recognition probability wi!. .*?dnge as the perceptible feature set is altered. Since there w I.. e n approximate correlation between the visible feature set and the subjective resolution relative to the target (number of cycles across the target) the recognition probability must change as this relative resolution changes. While Johnson recognized that target fertures would impact recognition probability, he did not provide any indication of how this would impact the accuracy of the recognition criteria. Johnson's 1974 paper also addressed the issue of background clutter. He emphasized that the detection function is applicable under conditions which require some degree of target shape i.e. where the target, in order to detect discrimination significant background clutter is present. He also stated that the number of cycles required to attain a particular detection probability can vary significantly depending upon the nature of the background clutter. Unfortunately this is as far as the discussion of clutter went, and it raised a number of questions as to the significance of background clutter on the validity of the Johnson criteria, especially since Johnson did not provide a definition of clutter. (7) found that indeed the amount and David Schmieder, et al (6) nature of background clutter had a significant impact on the probability of target detection. The most common clutter measure was scene intensity standard deviation. However this measure has the deficiency of tending to give large clutter values to relatively uncluttered scenes when this Moreover, those scenes possess several intensity modes. definition, like many other amplitude measures, lacks a spatial weighting factor. Both amplitude and spatial measures appear to be Since existing definitions required to predict observed trends. appeared inappropriate, Schmieder, et al, undertook to redefine the term. The definition found to be successful was an "average" scene radiance (or equivalent) standard deviation computed by averaging

1.3

..

. . ..

. ..

.. .

..

. .

. .

. . . ... . . . .. .. ...

the variances of contiguous scene cells over the whole scene and taking the square root of the result. Each cell was square in
shape and had side dimensions of approximately twice height. This definition can be expressed as: the target

N1

clutte~r=
where or is

G/N)

the radiance standard deviation for the i1 cell and N is the number of contiguous cells in the scene.

The above definition was judged successful because it yielded higher values for scenes which subjectively appeared to be more complex and cluttered. However, it was also an intuitively satisfying definition because it included both spatial and intensity measures. This definition effectively introduces normalization of intensity on a cell by cell basis. Hence, unlike the conventional intensity standard deviation, it avoids yielding a large clutter value for relatively uncluttered multimodal scenes. In addition, the use of contiguous cells introduces a spatial weighting parameter, i.e., the cell size, which satisfies the intuitive feeling that clutter object sizes close to the target size ought to weigh more heavily in the clutter calculation. Based upon this definition, experiments performed by Schmieder showed that clutter could be categorized as high, moderate and low, where high clutter exhibited a signal to clutter ratio (SCR) of less than one, moderate clutter an SCR of 1 to 10, and low clutter an SCR greater than 10. The experiments yielded the results shown in Table 6. From this data it can be seen that for a detection probability of 0.50, the number of cycles (LP/TGT) varies between 0.5 for low clutter and 2.5 for high clutter, with moderate clutter at 1.0. Using the moderate clutter case as a reference, the low clutter condition requires 50% fewer cycles and the high clutter requires 250% more cycles for the same detection probability of 0.50. Complete curves of detection probability versus required resolution were generated for low, moderate, and high clutter conditions, as shown in Figure 4. The numbers next to the curves represent the measured SCRs, ranging from a high of 39 (very low clutter) to a low of 0.33 (high clutter).

14

TABLE 6.

$=myA~ OF D"3t3Ot00 CUIZYRIX FOR &ROAD 0ATBOOR1118 or CLUTTE Numerof Cygles ILP/TGT) Low Clutter (SCR>10) 1.7 1.0 0.9 0.75 0.5 0.3 0-4.5 0.05 0.0 Moderate Clutter -SCR-A -10) (1. 0 2.8 1.9 1.7 1.3 1.0 0.75 0.35 0.1 0.0 High Clutter 1 1.0) -(SCR

Detection
_---Probability.

1.0 0.95 0.90 0.80 0.50 0.30 0.10 0.02 0.0 *Est imated

7.0* 5.0 2.5 2.0 1.4 1.0 0.0

0.6-

Resolution in LP/TGT

FIGURE 4. MEASURED DATA FROM OBSERVER TESTING FOR INDICATED SIGNAL TO CLUTTER RATIO$ (SCR) AFTER CORRECTION YOR CHANCE

15

was concluded that Johnson's criteria From this data, it (identified as NVEOL criteria in Figure 4) yielded results closely Further it approximating the moderate clutter case (SCR = 1.2). was concluded that "detection performance is a strong function of Detection range performance clutter as well as resolution. Since prediction models must therefore include clutter effects. the number of line pairs per target subtense necessary for detection is inversely proportional to detection range, changes in SCR can be expected to significantly alter range performance." CONCLUSIONS AND RECONMENDATIONS While Johnson's criteria has proven to be a useful tool in is it sensor performance, of electro-optical determination questionable how accurate the results are when applied to a Johnson's criteria ignores sophisticated system such as the 'OTDA. or downplays many factors which other investigators have considered significant. Some of thl tactors ignored by Johnson's critera are: degree of background clutter, characteristics of the observers, target shape, target location in a meteorological conditions, scene, color effects, effects of rultiple targets in a scene, and affects of time constraints in the discrimination process. Johnson's criteria were developed based upon well controlled laboratory tests using models, but were not verified during field As any observer can testify, lab tests against actual targets. conditions and "real world" conditions can be vastly different.
I

Whlf tni' various lveas of discrimination appear straight forward, 'h,-y art not, neither in the definition of the terms nor The discrimination pe critical dimension (C/CD). ,-,.leu in thL levels -r? suljective and are dependent upon the experience of the A highly' trained, experienced observer will have a much observer. of detection, for example, than an inexperienced higher prcirtility These values Further, the C/CD are not precise values. observer. are derived through laboratory measurements and are average values over several measurements for several observers for each target These average values are thenlaveraged again over all the type. target typet;, as shown in Table 1. The resulting values for each discrimination level averaged over the nine targets are only Since data oA individual measurements is accurate to about +25%. not provided bl' Johnson it is not possible to determine overall Table 7 shows some of the accuracy of thn discrimination levels. inaccuracy of factors that 4ould contribute to the overall Since the EOT&A Statement of Operational heed Johnson's criterii. (SON) MAC 509-87 requires ar a'.curacy of 20% for range outputs, it is highly unlikely the GON requirement can be achieved using Johnson's criteria without significant modification.

16

..

t&

.c

,.

.i

_..

TrALE 7.

FACTORS CONTRIBUTING TO INACCURACY

OF JOHNSON'8 CRITIRIA

EACTOR
Reading-Average Per-Target
S.

PCCR~ 25%
25%

Average Over Small Targets

Average Over Large L/W Targets - Side Average Over Large L/W Targets - Front/Rear
Clutter Effects - Low Clutter Clutter Effects - High Clutter

30% 55%
50% 250%

Target Characteristics Observer Characteristics Time Limitations (Semi-Trained)

200% 100% 250%

Estimated impact on resolution accuracy for target detection.

Based upon the foregoing, it is recommended that the Sensor Model be reviewed to determine the degree of Performance As a inaccuracies due to the application of Johnson's criteria. step, it is recommended that data from field experience be first If there is a reviewed and compared to predicted values. significant difference between observed and predicted data, an analysis should be performed to determine what factors have the most significant impact on the results. This analysis may require a test and evaluation program conducted under field conditions which approximate as closely as possible actual conditions experienced by pilots using the system.

Separate field tests may be required using the SPM only, to separate the impact of this model from those of the other models

The analsis and test effort will determine comprising the EOTDA. how well the SPM works, and if there are problems, it will identify them so that a plan of action can be undertaken to make required improvements.

17

REFERRNCB8

(1) (2) (3)

Johnson, John, "Analysis of Image Forming Systems", Image Intensifier Symposium, Fort Belvoir, VA, 6-7 October 1958. Biberman, Lucien, "Perception Plenum Press, New York, 1973. USAF, "Electro-Optics", Chanute of Displayed Information", Center

Technical

Training

. . .(ATC)-,:31December-1990.

(4) -Self, Herschel, "Imaqe Evaluation for the Prediction of the Performance of a Human Observer", NATO Symposium on Image Evaluation, 18-22 August 1969. (5) Johnson, John and Lawson, Walter, "Performance Modeling Methods and Problems", Proceedings of IRIS Specialty Groups on Imagery, January 1974. Schmieder, D.E., et al, "Clutter and Resolution Effects on Observer Static Detection Performance", Wright Avionics Laboratory Report AFWAL-TR-82-1059, WPAFB, Dayton, Ohio, February 1982. Schmieder, D.E. and Weathersby, M.R., "Detection Performance in Clutter with Variable Resolution", IEEE Transaction in Aerospace and Electronic Systems, Vol AES-19, No. 4, July 1983.

(6)

(7)

18

APPENDIX A DEFINITIONS ACUTANCE: Herschel Self 1969 - "Acutance is a measure of edge gradient and is related to the subjective impression of sharpness, but is independent of resolution." Lucien M. Biberman- 1973 - "is a measure- of, the sharpness of edge expressed in terms of the mean square of the gradient of radiant or luminous flux or density (in a photographic image) with distance from the edge." ACUITY: See Visual Acuity

BRIGHTNESS: John Johnson 1958 functions. If C - input target Rs =intensifier BT

o -

for both circular and rectangular object contrast spatial response

output target

brightness

output background brightness

BsBT Co
0 -

input background brightness

input target image brightness

output image contrast intensifier background noise

B" 0

intensifier scatter co-efficient

is given by
4 FJI J(
2

The output image brightness

-BS-Re

for B,7> Bf
and 7, g-'--BO ,-B;) +B + 80 ARA I

or 19

;>

81

where F M G T CLASSIFICATION: John act visual "the 1974 and Walter Lawson Johnson corresponding to perception _of _t.he.:general class of military target e.g. tracked versus wheeled vehicles." dimension for 50 The number of cycles per critical percent probability is 2.5 (taken from field results). = = = = objective aperture ratio tube magnification gain tube light optical transmission

CLUTTER: D. E. Schmieder, M. R. Weathersby, W. M. Finlay and T. J. Doll R. Schmieder and Marshall and David E. 1982, June scene radiance (or Weathersby November 1982-"average" equivalent) standard deviation computed by averaging the over the whole scene variances of contiguous scene cells is Each cell and taking the square root of the result. square in shape and has side dimensions of approximately be This definition can the target height. twice expressed as

clutter=

1N)

where TI N = radiance standard deviation = number of contiguous cells for the ith cell. in the scene.

Air

Force EO Training 1990 - "A typical target/background target scene may contain in addition to a hot (cold) other "hot (or cold) spots" surrounding the target within These "hot (or of view of the guidance s.nsor. the field cold)" spots are called thermal clutter."

CONTRAST: John Johnson 1958 functions. for both circular and rectangular object

20

If

C = input target contrast Rs

BT =
0

intensifier spatial response output target brightness output background brightness input background brightness

Bo B

Br = input target image brightness

8 = output image contrast C
0

0= intensifier background noise

0

K then it may
given by

intensifier scatter co-efficient shown that the output image contrast Cs

0

be

is

TR( -

3 0

+ K (Br+ B)

Richard H. Blackwell Blackwell as C =

1946 (B1 -

by Contrast C was defined BQ)/E 0 for stimuli brighter than

the observation screen, and C= (BO - B)/B 0 for stimuli the darker than the observation screen, where B, is brightness of the observation screen (backgroun'd) and B 3 the brightness of the stimulus. Lucien H. Biberman 1973 the ratio - Usually contrast is taken to mean

(brightness of the brighter) - (brighbness

brightness David E. Schmieder and of the brighter R. Marshall

of the dqrker)
1982 The

Weathersby

fowmula for video contrast is

target background

Cv

background

Video contrast corresponds to the contrast input to the Display contrast, on the other hand, is the monitor. contrast apparent on the display and is a&so c-,mputed as above except that target and background values must first be converted to display brightness. 21

R.

P.

Fiegel, P.S. Gillespie and M. P. Bleiweiss t991 - For and television direct view optics, image intensifiers, devices the inherent contrast of the dust plume against the background surface is given by Rb

Cc0 t

C0

Rb

where R is the reflectance- of the dust plume and the reflectance of th, background. The inherent contrast for thermal imagers is

Rb is

given by

Co

Td-

Tb-u
the dust plume and Tb is

Where T is the temperature of the background temperature. The contrast

as observed some distance away

is

given by

Cr

where

is

the

sky-to-ground

ratio,

and

T.

is

the

atmosphJric DETECTION:

transmittance.

John Johnson 1958 By normalizing the resolved line pairs for a critical target dimension the minimum resolution required fcy detection is 1.0 + 0.25. John Johnson and Walter Lawson 1974 - The number of resolvable cycles per critical dimension for 50 percent probability of detection is 1.0. Lucien M. Biberman 1973 - "An object is present". Johnson's Criteria for the Resolution Required per Minimum Object Dimension versud Discrimination Level Detection:

2 -0,5 fl.6 TV lines, Weathersby, W. M. Finlay, and T. J.

D.

E.

Schmieder, Doll June

M.

R.

198222

1) Detection is the designation potentially of military interest.

of

point

as

2) Detection involves distinguishing the target from other confusing objects in the scene such as bushes and rocks. 3) Detection occurs called to a particular closer scrutiny. when the observer's attention is point and is singled out. for

4) Detection of targets does not involve discrimination between target types such as tank, truck, or jeep but is simply a determination of whether or not an object on the screen is natural clutter or a military target. 5) Detection is said to occur when an observer correctly indicates his decision that an object of interest exists in the field of view. 6) Hot spot detection is the perception of a spatially and temporally persistent spot on a spatially and temporally random noise field. David E. Schmieder and Marshall R. Weathersby November 1982 "Target detection probability is a joint function of both signal-to-clutter ratio and resolution." Air Force EO Training 1990 - "An object is detected although no further target information can be determined." DFFRACTION LIMIT of resolution: Lucien M. Biberman 1973 - is the spatial distribution of radiance down to angular spatial frequencies approaching 2DX, where D is the diameter of the receiving optics and N is the wevelength of the transmitted information. IDENTIFICATION: John Johnson 1958 - By normalizing the resolved line pairs for a critical target dimension the minimum resolution required for identification is 6.4 + 1.5. John Johnson and Walter Lawson 1974 - The number of resolvable cycles per critical dimension for 50 percent probability of identification is 6.4. Lucien M. Biberman 1973 - The target can be described to the limit of the observer's knowledge (e.g., motel, pickup truck, policeman, etc.) Johnson's Criteria Zor the Resolution Required per Minimum Object Dimension versus Discrimination Level

23

Identification:

12.8

,2

TV lines,

Air Force EO Training 1990 - "Target identification occurs when the target can be described to the limit of the a the target is a motel, observer's knowledge (e.g., pickup truck, or an enemy tank)." INFORMATION FLOW in space frequency domain:

S.....is

John Johnson 1958 - information flLw in space frequency domain the number-of resolved line pairs per foot of ta'-,get space.

344O
W a, L (resolution bits per foot)

L = distance to target in feet a, minimum anguler subtense of system per foot of target space W = resolution bits Air Force EO Training 1990 - "At lock-on, the sensor LOCK-ON: detects enough energy contrast between the target and its between the two and background for the seeker to differentiate the tracker to follow (or track) the target."

MINIMUM DETECTABLE TEMPERATURE (MDT):

Capt Paul T. Beaudoin 1990 - "known as the hot-spot or star detection, describes the situation in which the target is detected as an unresolved spot or blob standing out from contrast." the background by virtue of its

MINIMUM RESOLVABLE TEMPERATURE (MRT):

where the Capt Paul T. Beaudoin 1990- "describes the situation target is perceived as a definite shape, possibly mottled surroundings." in grey shade, distinguishable from its NUMBER OF RESOLVABLE CYCLES: John 1974 - The number Johnson and Walter Lawson resolvable cycles across the target is given by N - T hI
R

of

where T, is the maximum resolvable frequency, h is the range to minimum target dimension and R is target. 24

the the

ORIENTATION: John Johnson 1958 - By normalizing the resolved line pairs for a critical target dimension the minimum resolution required for orientation is 1.4 o.35 John Johnson and Walter Lawson 1974 - The number of resolvable cycles per critical dimension for 50 percent probability of orientation is 1.4 Lucien M. Biberman 1973 The object is approximately symmetric or asymmetric and its orientation may be discerned. Johnson's Criteria for the Resolution Required per Minimum Object Dimension versus Discrimination Level Orientation: 2.8 #0.1 TV lines, -0.4 "target symmetry and dimensional

Air Force EO Training 1990 shape are noted."

RECOGNITION:
John Johnson 1958 - Py normalizing the resolved line pairs for a critical target dimension the minimum resolution required for recognition is 4.0 0.8 John Johnson and Walter Lawson 1974 - The number of resolvable cycles per critical dimension for 50 percent probability of recognition is 3.0 Lucien N. Biberman 1973 - The class to which the object belongs may be discerned (e.g., house, truck, man, etc.) Johnson's Criteria for the Resolution Required per Minimum Object Dimension versus Discrimination Level Recognition:

8.0 41.1 -0.4 TV lines, 1990 - "the target can be placed a truck, or a tank)." (e.g.,

Air Force EO Training the target is RESOLUTION:

a house,

Lucien M. Biberman 1973 - The horizontal limiting resolution is the spacing at which one can no longer consistently tell adjacent resolution elements apart.

25

David E. Schameder 1988 per class object

"the number of resolvable line-pairs criticr] dimension provided by the

sensor
SIGNAL-TO-CLUTTER David

as

function

range

(the

present

TDA

definition)." RATIO (SCR)

E. Schmieder and Marshall R. Weathersby 1982 - the maximum difference between the target and background -radiance divided by-the ras clutter radiance for positive.. contrast targets: back round mean valume-. aximum target SCR: rms clutter while $CR -for negative contast targets: value inimum target r9s clutter - backiround meani

VISUAL ACUITY: Lucien M. Biberman 1973 - The detail discrimination threshold of the human eye. Visual acuity is the reciprocal of the angle subtended by the minimum size standard test object that can be resolved 50 percent of the time by a human observer. The angle resolved by a normal eye is approximately I minute of arc; normal acuity is therefore the receprocal of 1 minute of arc, i.e., one.

26

S..

F..F. W

"

wW

--.

-_

APPINDIX B TDA ACRONYN/ABBRIVIATION LIST

. .

ADPE AFATL AFGWC AFWAL AGM AH . AIM ALSPM APC ASL ATBSM ATCCS ATM AWS BCD BICi BMP" BFDF C CDLOR CDMDT CDMRT CFF CFLOS CI CLOS CM CMRL CNVEO COMBIC

Automated Data Processing Equipment Air Force Armaments Test Laboratory Air Force Global Weather Central -Air Force-Wright-Aeronautical Labo-ratories Air-Ground Missile Absolute Humidity Aerial Image Modulation Avionics Laboratory Sensor Performance Model Armored Personnel Carrier Atmospheric Sciences Lab/Air Superiority Lab Analytics Target/Background Signature Model Army Tactical Command & Control System Atmospheric Transmittance Model Air Weather Service Battelle Columbus Division Induced Contaminants Battlefield Target (Armored Personnel Carrier) Bidirectional Reflectance Distribution Function Command and Control Critical Dimension for Lock-On Range Dimension for Minimum Detectable Temperature Critical Dimension for Minimum Resolvable Temperature Critical Critical Flicker Frequency Cloud-Free Line-Of-Sight Image Contrast Clear Line-Of-Sight. Countermeasures Combat Material Research Lab (Army) Center for Night Vision EO Induced Combined Obstruction Model for Battlefield Contaminants Classical Target Acquisition Cycle CTAC Contingency Tactical Air Control System Automated CTAPS Planning System Video Contrast C Digital Equipment Corporation AC Detachment DET Defence Mapping Agency DMA Demand Moduation Function DMF Desired Mean Point of Impact DMPI Program DMSP Defense Meteorological Satellite Detection Range DR Dynamics Research Corporation DRC Deployable SMDPSIII D/SKDPSIJI Direct View TDA (Cats Eye Night Vision Goggles) DVTDA Differential Temperature AT 27

EBBT EM EMAC ENSCE EO EOAW EOCM EOSAEL EOTDA ETAC EXMRT FAC FL FLAPS FLIR FMX FNOC FOV GBM GD/PL GLINT GTR! HARM HE HRG HVT ICLTR ICOMPL ICO ICONTR ICSTL IFOV I'R IR [RTDA ITDA JOT&E KRC LANTIRN LA/LV/LO 1.GB LLLTV !.MRT
L."- I. N-L.V

Equivalent Black Body Temperature Electro Magnetic Environmental Monitoring And Control Enemy Situation Correlation Element Electro-Optical Electro-Optical Analyst Workstation Electro-Optical Counter Measures Electro-Optical Systems Atmospheric Effects Library Electro-Optical Tactical Decision Aids Environmental. Technical Applications__Center. .. Threshold for MRT Forward-Air Controller ... Foot Lamberts Force Level Automated Planning Systems Forward Looking Infrared Cut-off (Maximum) Frequency for Sensor Fleet Numerical Operations Center Field Of View Generic Building Model Geophysics Directorate/Phillips Laboratory Gated Laser Illuminator for Narrow TelevJsion Georgia Tech Research Institute, Electro-Optics Lab High-speed Anti-Radiation Missile High Explosive High Resolution Geometry High Value Target Clutter Index Index of Scene Complexity International Commission on Optics Index of Scene Contrast Air Mass Index Instantaneous Field-Of-View Imaging Infrared Infrared Infrared TDA Interim TDA Joint Operational Test and Evaluation Keweenaw Research Center, Michigan Technological University, Houghton Low Altitude Navigation and Targeting Infrared for Night Launch-Leave-Lockon Laser Guided Bomb Low Light Level Television Laboratory MRT
Lockon-Launch-Leave

LON
LOWTRAN LP/TGT LRCO LRD L/W

Lock-On-Range
Low Resolution Transmittance Model Line Pairs per Target Long Range Capability Objectives Laser Hangefinder Designator Length-to-Width Ratio 28

'

mm

--

"

m .m.m

mm

'mm

"m

.. m

mmm

m.

..

mm

LWIR
MAC MDS MDSV .MDT NFT MIDAS MIRL MLOR

Long Wave Infra Red

S....

Command Military Airlift Minimum Detectable Signal (joules/cm') Miniium Detectable Signal Value Minimum Detectable Temperature Multi-faceted Target Multispectral Imagery Data Analysis System Mobile Infra Red Laboratory Maximum Lock-On Range .MMAC-PAc:---Moble-Meteo ologica!l-Acqu-isition-and-Control- Package MMW Millimeter Wave Mini-MIRL Mini Mobi-le Infra7 Red LaboratoryMinimum Resolvable Contrast NRC MRS Minimum Resolvable Signal MRT Minimum Resolvable Temperature MSS Mission Support System MTF Modulation Transfer Function MWIR Middle Wave Infra Red NEDT Noise Equivalent Delta-T NEI Noise Equivalent Irradiance NETD Noise Equivalent Temperature Difference Xarrow Field of View NFOV NOARL Naval Oceanographic and Atmospheric Research Laboratories NRL Naval Research Laboratory NVEOL Night Vision & Electro-Optics Lab NVG TDA Night Vision Goggles TD t NVL Night Vision Lab (Now CNVEO) OATS Optical-Acquisition and Tracking System OMAC Orange Measurement And Control Package OTDA Operational TDA Probability of Detection PD pdf probability density function PGM Precision Guided Munition PL Phillips Laboratory Portable Non-Imaging Radiometer PNIR POC Point Of Contact (Tank) POL Petroleum, Oil, Lubricant PRESSURS Prestrike Surveillance and Reconnaissance System Pacific Sierra Research Corporation PSR PTDA Preliminary TDA ROTDA Research Grade TDA RH Relative Humidity RMSD Root Mean Square Difference Rr Lockon Range RR Rain Rate SAC Strategic Air Command SAI Science Applications Incorporated Silent Attack Warning System SAWS Signal-to-Clutter Ratio SCR SO Signal SIG Signal SITF Signal Intensity Transfer Function 29

..

...

. .

. . .

..

..

. .. ..

..

SMDPSIII SNR SOF SPM SRM STX TA TABILS TAC/IN TAF -TAMPS TARGAC TAS TCLT TCM TDA TOT TH TOT TQF TRAM TSCEN TSCF TTPF TV T-72 UARS UMBRELLA VIDEM VSA VTR WDA WFOV W/M'2 WL WRDC WS XSCALE YEFF YMAC ZIL

[II Strategic Mission Data Preparation System Phase Signal-to-Noise Ratio Special Operations Forces Sensor Performance Model Sensor Ranging Model ST Systems Corporation Target Acquisition Target and Background Information Library System Tactical Air Couuand/Intelligence Tactical Air Force Tact-ical Aircraft- Mission -Planning -System Target Acquisition Model Target Acquisition System Clutter Temperature Thermal Contrast Model Tactical Decision Aids Target Threshold Time Over Target Threshold Quality Factor Target Recognition and Attack Multisensor Scene Temp,:rature Targeting Systems Characterization Facility Target Transform Probability Function Television (Visible) Soviet Tank Model Unmanned Air Reconnaissance System Unified Measurement Blueprint for Rational Experiments Leading to Logical Analysis Visual Detection Model Vertical Structure Algorithm Video Tape Recorder Weather Decision Aids Wide Field Of View Watts per Meter Squared Wright Labs Wright Research and Development Center Wind Speed Army VSA Targets Projected Across-Track Dimension Yellow Measurement and Control Package Russian Tank or Truck

30