EVALUATING THE PERFORMANCE OF AUTOMATED PAVEMENT CRACKING

EQUIPMENT

Version December 2007

Technical Committee CT 4.2 on the Interactions Vehicle/Road

This report has been prepared by the working group D of the Technical Committee TC 4.2
“Interactions Vehicle/Road”.

We would like to thank the following organisations and their representatives that provided their time
and shared their precious expertise to the accomplishment of this World Road Association (PIARC)
document.

Working group D members of the PIARC Committee CT 4.2:

• Michel Boulet Laboratoire Central des Ponts et Chaussées (LCPC), (France)

• Pietro Bumma Ministère wallon de l'Equipement & des Transports, (Belgium)
• Steve Brown VIC Roads, (Australia)
• Amadou Cisse Groupe d'Ingénieurs Consultants (GIC), (Mali)
• Brian Ferne Transport Research Laboratory (TRL), (Great Britain)
• Mathieu Grondin Ministère des Transports du Québec, (Canada – Quebec)
• Paul Harbin Roadware Group Inc., (Canada)
• Lucien Heleven Ministerie van de Vlaamse Gemeenschap, (Belgium)
• Frank B. Holt Dynatest International, (United States of America)
• David Olodo Ministère des Travaux Publics et des Transports, (Benin)
• Paulo Pereira Universidade do Minho, Escola de Engenharia, (Portugal)
• Lily Poulikakos EMPA Dübendorf, (Switzerland)
• Filippo Pratico Mediterranean University of Reggio Calabria, (Italia)
• Yves Provencher Institut canadien de recherches en génie forestier (FERIC), (Canada – Quebec)

We also want to thank the persons who contributed directly or indirectly to the redaction of the paper:

• John Laurent Institut National d’Optique, INO, (Canada – Quebec)

• Claude Laberge Statex inc. (Canada – Quebec)
• Martin Boucher Ministère des Transports du Québec, (Canada – Quebec)
• Bert De Wit Rijkswaterstaat Dienst Weg- en Waterbouwkunde, (Nederland)

The translation into French of the original version was produced by the Quebec Ministry of
Transportation (Quebec-Canada).

Pietro Bumma (Belgium), Paul Harbin (Canada) and Mathieu Grondin (Canada-Quebec) were
responsible within the Technical Committee of the quality control for the production of this report.

The Technical Committee was chaired by Bjarne Schmidt (Danemark). Guy Descornet (Belgium) and
Mathieu Grondin (Canada – Quebec) were respectively the English and the French-speaking
secretaries.

-2-
TABLE OF CONTENTS
SUMMARY ....................................................................................................................................................... 4
INTRODUCTION.............................................................................................................................................. 5
1. EVALUATION METHODOLOGY CONTEXT..................................................................................................................................... 5
1.1 WHAT IS A CRACK? .......................................................................................................................................................................... 5
1.2 UNCONTROLLABLE VARIATIONS IN MEASURING (DETECTING) CRACKS ...................................................................... 5
1.3 TESTING PROCEDURES ................................................................................................................................................................... 6
2. INCREASING THE REPRODUCIBILITY OF THE CRACKS MEASUREMENTS....................................................................... 7
2.1 DELIMITATION OF THE ANALYZED ZONE................................................................................................................................. 7
2.2 CRACK DESCRIPTION AND TYPE ................................................................................................................................................. 9
2.3 CRACK SEVERITY LEVEL ............................................................................................................................................................. 11
2.4 CRACK EXTENT............................................................................................................................................................................... 12
2.4.1 Research level validation test..................................................................................................................................................... 12
2.4.2 Project level validation test........................................................................................................................................................ 12
2.4.3 Network level validation test ...................................................................................................................................................... 13
3. SETTING THE METHOD TO ASSESS THE AUTOMATED CRACK MEASUREMENT DEVICES PERFORMANCES.... 13
3.1 RESEARCH LEVEL VALIDATION TEST...................................................................................................................................... 13
3.2 PROJECT LEVEL VALIDATION TEST.......................................................................................................................................... 15
3.3 NETWORK LEVEL VALIDATION TEST....................................................................................................................................... 17
4.0 CONCLUSION AND RECOMMENDATIONS .................................................................................................................................. 20
5.0 BIBLIOGRAPHY.................................................................................................................................................................................... 21
APPENDICE 1 DETAILED METHODOLOGY FOR THE RESEARCH LEVEL VALIDATION TEST ......................... 22

-3-
SUMMARY

Crack type, severity and extent are one of the most important data to select maintenance and repair
actions. These data are in fact, the basic data of a pavement management system.

The conclusions of a worldwide survey carried out by the PIARC’s former Committee C1 in the
previous cycle (2004-2007) revealed that performance tests of the automated crack measurement
systems are only conducted infrequently, and these are very often incomplete and non-uniform. In
addition, all of the experiments involving comparison of the results obtained using automated
equipment with those obtained by the manual method faced the same definitional problems relating to
distresses. In light of this, Working Group D of Committee TC 4.2 “Road/Vehicle Interaction” set as
its principal objectives the development:

ƒ of a detailed method for identification of surface distress (and more specifically of pavement
cracking);
ƒ of a method allowing evaluation of the reliability of the automatic pavement cracking detection
equipment.

First, the paper presents a series of more objective and more detailed rules to measure and classify
cracks. Some have the objective of increasing the reproducibility of measurements and allowing more
reliable comparison of the cracking data gathered by different equipment. A monitored zone has been
delimited as well as a better definition of a crack and a methodology to measure the crack severity and
extent.

Similarly, three methods of evaluation of the reliability of automated crack measuring equipment are
also proposed. The research level validation test is used to rate the sensor’s resolution and sensitivity
regarding the capability to capture raw pavement data (images or 3D data). The project level
validation test allows road administrations to rate the equipment’s bias and repeatability to precisely
measure cracks on some small controlled reference track which presents specific and various
conditions. While the last validation test, entitled network level validation test, differs from the
preceding one by the scope of the measurement campaign and by the precision level sought.
Performed on 50-100 km of road, the equipment can be tested on varied surfaces that are more
representative of the network reality.

The objective of this technical report is to help road administrations choose and select automated
systems in regard to their needs, therefore increasing the quality of the crack data used in pavement
management systems. The concepts presented in this document are not international standards, only a
summary of the experiences exchanged between world experts.

-4-
INTRODUCTION

The document entitled "Automated Pavement Cracking Measurement Equipment – Worldwide

Progress Report", [1] which was prepared by the working group of the C1 committee of the PIARC
during the previous cycle, presented the conclusions of a worldwide survey concerning validation
testing and the use of automated pavement cracking measurement equipment. This survey revealed
that performance tests on this type of equipment are only conducted infrequently, and that the tests are
very often incomplete and non-uniform. In addition, all of the experiments involving comparison of
the results obtained using automated equipment with those obtained by the manual method faced the
same definitional problems relating to distresses. In light of this, the study recommended that further
work had to be conducted in order to develop more precise methods to identify pavement distresses
and to propose accepted methods for evaluating the reliability of automated pavement cracking
measurement equipment.

The present working group is undertaking to pursue this goal, which breaks down into two stages:

ƒ Preparing an inventory of pavement crack detection and identification methods in order to

improve the reproducibility of the measurements.
ƒ Designing an assessment methodology for evaluating and classifying automated pavement
cracking measurement equipment in terms of reliability (bias and repeatability).

This report presents the main conclusions of the work carried out by the members of the working
group D of the TC 4.2 committee on these two objectives. The objective of this technical report is to
help road administrations evaluate and select automated systems in regard of their needs and to
increase the quality of the crack data used in pavement management systems. The concepts presented
in this document are not international standards but only a summary of the experience exchanges
between world experts.

1. EVALUATION METHODOLOGY CONTEXT

The purpose of this section is to set the background for the discussion of issues with respect to
automated crack detection to be presented in this document. In particular, what it is and why it is, that
there is such concern over the apparent and often proven differences between the many systems that
claim the automated detection of cracks. What are the issues?

1.1 WHAT IS A CRACK?

One would think that this is a relatively simple question, however, there is no definitive definition of a
crack in objective engineering terms. Nowhere is it stated that a crack is a discontinuity in the road
surface that has a minimum length, width and depth.

Until such a common and widely accepted definition can be written, the real question to be answered
is: how do we define a crack? The answer will depend on the observer. Although this point may be
applied more to the classification of cracking than to the crack itself, the dilemma still remains the
same. For example, a series of small disjointed cracks could be considered as a series of cracks or only
one crack.

Although the context in which we are thinking is in terms of automated crack detection, what the
computer can see and how it is interpreted, one could easily argue that human pattern recognition is
also quite ambiguous. Different opinions will always exist.

1.2 UNCONTROLLABLE VARIATIONS IN MEASURING (DETECTING) CRACKS

There are many factors that can influence the capability and reliability of the automated crack
measurement systems. These factors can affect both the collection and the processing of the raw data.
Throughout the world there are many different pavement types and mixtures. A lot of road
construction depends on the local availability of the resources needed for road construction.

-5-
Geographic regions throughout the world, within a country or even a state or province will dictate the
requirements of road construction and surface properties. Because there is such a wide variety one
would ponder the feasibility of developing a singular test.

As there are different roads in different regions there will also be different maintenance programs for
these roads. As such, the requirements or information required from an agency for these roads from an
automated crack detection system will be different. The proposed testing and evaluation methods will
need to reflect these needs and ensure that the correct properties are being evaluated.

Road markings, whether they are intentional paint lines, oil patches, skid marks, bleeding or other
abnormal surface irregularities will cause havoc with the automated detection of cracking. Should
these be designated as false positives and be removed or is it sufficient to include the false positives
and discount the reported cracking by a variable amount? This type of decision may not be addressed
in the context of system evaluation but it must certainly be considered in a complete crack detection
program.

1.3 TESTING PROCEDURES

This section outlines the several different validation tests we propose for the evaluation of automated
crack detection equipment.

• A “Research level” test has been defined in order to rate the sensor’s resolution and sensitivity
regarding their capability to capture raw pavement data (images or 3D data). The purpose of this
test is to validate the manufacturer’s equipment and to determine what it can and cannot do. The
intent is to run the equipment on a short specific test grid (section of road) with predefined and
pre-cut cracks of various widths, lengths and depths. If the cracks can be seen in the raw data as
a visual image, then the equipment will have passed the equipment validation test.

This “Research level” test is normally carried out by the equipment manufacturer and it is done
for the specific purpose of making sure the sensors are performing to their design and
performance expectations. This validation is designed to reduce, to a minimum, the variability
associated with sources other than the variation of the lasers or cameras and thus, to focus on the
possibilities/performance of the system components. A client could perform this test on new
equipment according to his needs or accept the specifications provided by the manufacturer.

• A “Project level” test has been defined in order to perform a more detailed evaluation of the
equipment. This test would be the result of operating the crack detection equipment on sections
of in-service roads. It could be a single section (several kilometres), or several shorter ones, but
usually having some specific distresses of interest for the road administration. The road sections
would have to be ground truthed in order to perform an evaluation of the equipment. This test
could also be the basis of a monthly quality plan program.

• A “comprehensive network level survey” on 50-100 km of road can also be performed. This test
would need to be prepared according to the agency’s result expectations. This last step is likely
to be the most controversial of any test because the acceptance of the ground truth values will
always come into question.

At a conceptual level, the above referenced testing and evaluation steps seem quite objective and
simple, however, obtaining a reliable and repeatable ground truth, which include all of the particular
cracking patterns and pavement types an agency expects to come across is not simple. Finally, it must
be considered that it may be prudent to have separate tests for the evaluation of the collection and
processing technology. As there are many collection platforms there are many processing platforms as
well. It should be required, as part of the testing process, that crack detection applications are capable
of processing the raw data from other platforms in order to have a more standard evaluation process.
This just may lead to defining a standard format for the output of the raw sensors. Is this appropriate or
would it be considered too restrictive on the manufacturers of such equipment, limiting their ability to
advance technology?

-6-
2. INCREASING THE REPRODUCIBILITY OF THE CRACKS MEASUREMENTS

Excellent pavement distress identification manuals are available in many countries. In general, the
manuals do not cover all particular aspects and conditions that are regularly present when roadways
are assessed. For example, a transversal crack is defined in many manuals as a crack that is generally
perpendicular to the roadway surveyed. Few manuals specify a minimum length and orientation
(angle). There is still no accurate, universal method for measuring the width of cracks. This ambiguity
is quite frequent and exists for most defects measured. Whether automated or not, the analysis of
defects under these conditions may produce considerable variations depending on the operators and
the automated equipment used [1].

In order to properly evaluate automated equipment, we must be very careful not only with the test
procedures, but also, and mostly, with the description of cracks and their geometric characteristics, so
that one can distinguish them. This is the aim of this chapter.

2.1 DELIMITATION OF THE ANALYZED ZONE

With the aim to improve measurement repeatability, it is very important to precisely define the zone
where the systems have to detect and analyse cracks. A common way to do this is the worldwide
approach of strips [2].

The proposed method divides the surveyed lane into five strips. In the direction of travel, strips are
numbered from left to right (see picture below). Strips 2 and 4 match with wheel paths and their width
is 0.75 meter. The width of the central strip, number 3, is 1.00 meter. The width of the outside strips,
number 1 and 5, varies with the width of the lane. A typical lane width is 3.60 meters.

FIGURE 1. PRINCIPLE OF THE WORLDWIDE APPROACH OF STRIPS

-7-
As shown in the figure 1, the surveyed area of the lane is limited by the inside of the lane markings. If
there are no lane markings for a distance ≥ 100 metres, the default lane width to be used is 3.60
metres, positioned at the centreline of the survey vehicle. The suggested widths are just
recommendations. They can be adapted to national or regional needs and requirements.

-8-
2.2 CRACK DESCRIPTION AND TYPE

The table 1 below gives a detailed description of some of the most frequent cracks and distresses
measured automatically.

TABLE 1: DETAILED DESCRIPTION OF CRACKS

TYPE DESCRIPTION SCHEME OR PICTURE

Crack
Minimum length: 0,15 m
definition N/A
Minimum width: 1 mm
Crack with an orientation ≤ 1:3 (1 parallel et 3
Transversal perpendicular to the road axis) and which is Axe de la voie
cracking present on 2 or more longitudinal strips.

Ligne de rive 1 Ligne de centre

Longitudinal Crack with an orientation > 1:3 (1 parallel and 3

cracking perpendicular to the road axis).
Bande # 5 4 3 2 1

Largeur bande 0,75 0,50 0,50 0,75

Voie auscultée

Sens du traffic

Longitudinal crack distant less than 0.25 m from

Edge cracking the edge of the road.

-9-
 Agglomeration of pavement cracks in the form of
a grid, with at least 3 pieces in each direction, and alligator cracking (d ≤ 300 m)
where the diameter of each piece is less than 300
Alligator
mm.
cracking
If the diameter of the pieces is greater than 300
mm, then the cracks are considered as distinct.

distinct cracks (d > 300 mm)

Agglomeration of pavement cracks that run

parallel and that are less than 300 mm apart.
Multiple
If they are more than 300 mm apart, then the
cracks
cracks are considered as distinct.

Hole with an area larger than 50 mm by 50 mm.

The bottom of the hole should be in granular basis
Pothole
(full thickness of the pavement).

- 10 -
2.3 CRACK SEVERITY LEVEL

Cracks can have several appearances. Apart from changes in orientation and in length, width can also
change along a single crack. It also happens quite often that cracks are spalled (loss of material) or
branched out. Of course, this makes their characterization more difficult. When processing crack data,
the simultaneous presence of these phenomena, surface colour, changes of macrotexture or road
deformation, all amplify the problem. In order to improve the repeatability and the reproducibility of
measurements of degrees of severity, some rules are proposed below. These rules can be applied to all
kind of measured cracks, regardless of their orientation and position. They can either be used in
automated analysis algorithms, or in visual identification methods [3].

Each uniform crack segment is evaluated. The minimum segment length considered is 1 meter. The
representative degree of severity for a segment corresponds to the maximum width encountered on the
longest (more than 25%) portion of the segment. Spalled cracks are automatically considered to be of
high severity. Multiple cracks are considered to be of moderate or high severity (see multiple cracks),
and alligator cracks are considered to be of high severity. The thresholds that define the categories of
severity are not covered in this document, and each administration has to determine its own thresholds.

The figure 2 below represents a summary of the suggested method for the measurement of the degree
of severity of a crack.

Homogeneous crack section of

more than 1 m long

Low severity Medium severity High severity

Simple crack - Simple crack having - Alligator cracks

having maximum maximum width whatever the width of
width encountered encountered on more than each crack.
on more than 25 % 25 % of the length between And
of the length less low and high severity - Spalled cracks
than the low thresholds fixed by Road
severity threshold administration.
fixed by road And
administration - Multiple cracks having
maximum width
encountered on more than
25% of the length less than
the high severity threshold

FIGURE 2 : METHOD FOR THE MEASUREMENT OF THE DEGREE OF SEVERITY OF A CRACK.

- 11 -
2.4 CRACK EXTENT

As presented in section 1.2, three tests are proposed:

ƒ Research level validation test;

ƒ Project level validation test;
ƒ Network level validation test.

Each of these tests has its own definition of “crack extent”.

2.4.1 Research level validation test

This phase aims to check precision and bias of automated systems when measuring cracks under
controlled conditions. Tests are made on a track, where cracks are artificially simulated by sawing the
pavement in such a way that the length, orientation and width of the cracks are precisely defined. As
far as transversal cracks are concerned, only their number is counted. For all the other kind of cracks,
the extent is the actual length.

2.4.2 Project level validation test

This phase aims to check the repeatability and reproducibility of automated systems when measuring
cracks on roads at project level. Tests are made on well known sections of in-service roads where all
the defined cracks exist at several degrees of severity.

Keeping in mind the proposed strips, it is furthermore recommended to divide the test area into 1
meter long subsections, as shown in the figure 3. The road surface will thus be divided into cells. The
method consists of recording whether there are cracks in each of the cells or not. When a cell is
“touched” by a crack, then it is an “allocated” cell. This will be the extent of the cracking. This
technique gives a kind of digital picture of the road surface, with pixels “allocated” or not.

Once a cell is defined as “allocated”, a degree of severity can also be added; this will add another
criterion to the analysis. If more than one crack touches a cell, its degree of severity will correspond to
the maximum of the considered cracks.

FIGURE 3. MEASURING THE EXTENT OF CRACKS BY MEANS OF CELLS

- 12 -
2.4.3 Network level validation test

This phase aims to verify whether the automated crack detection system is capable of identifying
lengths on a network that has both high and low levels of cracking. To do this, the surveyed area is
“overlaid” with a square grid; then any grid tile containing a crack is taken into account. The extent of
the crack to be considered is the total area of grid tiles containing a crack within a sub-section of
survey data. The dimensions of the tiles depend on the resolution of the tested device (typically 0,5 m
x 0,5 m). The recommended sub-section length is 50 meters.

3. SETTING THE METHOD TO ASSESS THE AUTOMATED CRACK MEASUREMENT

DEVICES PERFORMANCES

The multiple step method proposed is intended as a suggested method of equipment validation. In
most cases the purpose of the testing is to evaluate specific equipment for use within an agency’s PMS
for their road network. It is up to the agency to determine which steps in the suggested three step
process need to be carried out.

Certain agencies such as the Highways Agency (HA) in the U.K. use a network level approach to
validate proper operation of equipment. This single step approach has been used for several years and
is considered sufficient as the HA is specifically interested in the big picture results for input into the
calculation of their key performance indicators.

As suggested, the three step evaluation methodology is not intended to be a set of mandatory steps but
a suggested process in the complete evaluation of a cracking system.

3.1 RESEARCH LEVEL VALIDATION TEST

The first level of tests consists of evaluating the performance of the system components and
configuration. The intention is to begin by ensuring that the lasers or cameras used have the precision
and repeatability required to perform the desired work. The main objective of this test track is to
provide an objective way to evaluate the reliability of the automated equipment by controlling all the
subjective aspects of the measurements. All the bias and the lack of repeatability can then be attributed
to the equipment.

This test consists of evaluating the bias and repeatability of the systems for measuring certain targeted
crack characteristics. This stage would be carried out under what are reported as controlled conditions:
on a marked track with intentionally created cracks, using a statistical sampling plan that makes it
possible to isolate various targeted characteristics of the cracks, such as their width, position
(longitudinal strip), depth, length, and orientation. The cracks can be created using saw cuts. A guide
is presented below, which presents the method of building the track, the survey procedure to be
followed, and the method for calculating bias and repeatability.

Artificially fissured track

This track is defined to take into account characteristics encountered in road networks around the
world and to allow the classification of systems. Road administrations can adapt the track in regard to
their specific needs (strip width, total surveyed lane, etc…).

- 13 -
The proposed artificially cracked tracks are divided transversally, based on the longitudinal strips
defined in section 2.1, and longitudinally every 10 metres. Each defined cell can then be associated
with a specific crack, however, some cells are left empty (without cracks) in order to avoid a
systematic apparition of cracks every ten meters. The artificially cracked track proposed in table 2
would have the following characteristics:

TABLE 2 : CHARACTERISTICS OF THE ARTIFICALLY CRACKED TRACK PROPOSED

Sampling unit Crack

Number of tracks 1
Number of cracks per track 170
Number of longitudinal cracks 150 (1 cell per crack)
Number of transversal cracks 20 (5 cells per crack)
Track length 600m
Available cells 300
Used cells 250
Repetitions for repeatability 5

The proposed track is defined by three crack characteristics:

ƒ Crack severity (4 levels): 2 mm, 3 mm, 5 mm and 8 mm.

ƒ Crack length (4 levels): 3 m, 5 m, 8 m and 3.6 m (transversal).
ƒ Crack depth (2 levels): 1 time and 2 times the macro texture.

Two levels of macro-texture are strongly recommended.

Data and data analysis

A major advantage of the artificially fissured track is the possibility to work with a crack as the
sampling unit. The resulting data file has one line per crack for each repetition, so for 5 repetitions the
data file should have 850 lines (170 cracks x 5 repetitions). As the systems may not recognize some
cracks or create other ones, the recorded number of lines in the data file can be different than 850.

Appendix 1 presents a typical data file resulting from a research validation test as well as a more
detailed explanation on how to manage and analyse the data. By using this methodology, road
administrations can easily identify if the tested equipment has a problem and associate this problem to
a specific road characteristic (type, width, position, type of crack…).

Classification thresholds
The results obtained (repeatability and bias) can now be used to classify equipment in terms of
precision and reliability according with the tables 3 and 4. The classification thresholds used can be
determined on the basis of their effect on a known cracking index (cracking rate or other). The
highway administration can then make its equipment choice based on its management needs.

The speed range of a surveying vehicle on the test site must be recorded. The classification of vehicles
could only be valid for this speed range and will depend on the collecting process principle of the
survey vehicle. The following conditions of the test site could also influence the cracking index
provided by the survey van:

ƒ Colour of the pavement;

ƒ Climatic surface condition (dry, drying up, wet);
ƒ Weather conditions (sunny, cloudy).

- 14 -
These conditions will influence the results, therefore the comparison of the results of different test
sites. This type of track can then be used to evaluate the influence of these factors on the system
reliability if the passes are performed in these different conditions.

TABLE 3 : CRITERIA PROPOSED FOR DETECTION AND BIAS

CLASS CORRECTLY LENGTH (CRACKS WITH SEVERITY (CRACKS

DETECTED CRACKS RELATIVE BIAS < 7.5%) WITH RELATIVE BIAS <
20%)
AAA 100% 100% 100%
AA ≥ 90% ≥ 90% ≥ 85%
A ≥ 80% ≥ 80% ≥ 70%
B ≥ 70% ≥ 70% ≥ 60%
C < 70% and ≥ 50% < 70% and ≥ 50% < 60% and ≥ 50%
Class C is the lower limit for “suitable” devices.

TABLE 4 : CRITERIA PROPOSED FOR REPEATABILITY

CLASS LENGTH (CRACKS SEVERITY (CRACKS

WITH C.V. < 0.1 %) WITH C.V. < 0.5 %)
AAA 100% 100%
AA ≥ 90% ≥ 85%
A ≥ 80% ≥ 70%
B ≥ 70% ≥ 60%
C < 70% and ≥ 50% < 60% and ≥ 50%
Class C is the lower limit for “suitable” devices.

3.2 PROJECT LEVEL VALIDATION TEST

The main objective of this validation set of tracks is to regularly test the quality of the equipment. For
example, a road agency with several systems can define a set of ten tracks according with the table 5
for validation and force each system to go through the validation process every month in order to
ensure that each system always satisfies a given class criteria. This test is carried out on several tracks
containing natural cracks. These tracks should be carefully selected in order to ensure that all of the
characteristics of the cracks to be measured in network mode would be represented.

The measurement protocol is simplified as much as possible. As presented in section 2.1, the surveyed
lane is divided into 5 longitudinal strips. It is then subdivided longitudinally every 1 meter. This gives
a continuous grid pattern that provides clearly-defined transversal and longitudinal positions, which
eliminates localization errors.

TABLE 5 : DESCRIPTION OF THE TRACKS

Sampling unit 1 m section

Number of tracks 10
Number of cracks per track Variable
Number of longitudinal tracks Variable
Number of transversal tracks Variable
Length of the tracks 50 m
Cells per track 250
Total number of cells 2500
Repetitions for bias 3
Repetitions for repeatability 5

- 15 -
The choice of 1 meter sections is used to minimize the probability for a cell to contain more than one
crack. This length typically results in a meaningful and relevant cracking rate. In order to perform the
1 m validation, the systems tested should have some form of automatic triggering device to ensure that
the measurement starts and stop at the same location.

In order to obtain meaningful cracking rates, a minimal crack length of 15 cm is proposed as a

threshold for a cell to be declared fissured.

The use of 10 tracks is chosen to make sure that we obtain several cracking characteristics and to
ensure that the sample is representative of the most important characteristics fund on the road network.

The number and the type of cracks are variable since the validation process is performed on naturally
fissured tracks. A reference measure of the cracking rate must be obtained manually for all ten tracks.

As the survey van reliability could be affected by the speed, it is recommended to measure the speed
influence. Then, it is recommended that the van perform 5 passes on each track per each tested speed.

Data and data analysis

The first proposed verification is to classify equipment solely in terms of its capacity to detect the
presence or absence of cracks in each cell: a kind of cracking rate or success rate for a given segment
length. Then, using a protocol to classify the degree of severity for each case (0: absence; 1: presence
of severity 1; 2: presence of severity 2; etc.), it would be possible to evaluate the capacity of the
equipment to measure the severity of cracks. A third level of evaluation is considered, consisting of
identifying the types of cracks based on the arrangement of the point-matrix or grid obtained by the
equipment. In all cases, repeatability could also be evaluated.

In its simplest form, presence or absence of cracks in each cell, the data to be collected consist of a
series of 0 (cell without crack) and 1 (cell with at least one crack). The data file would then contain
250 lines for each track and only one column (with zeros and ones) in addition to the column
identifying the location of the cells. The cracking rate is then obtained by summing the number of 1s
(ones) and dividing by 250 (the total number of cells).

In addition to the cracking rate, the concordance of the results will also be considered. The
classification is then determined by the proportion of cells with the same conclusion (presence or not
or crack in cells) for both the referenced and the measured cells.

When crack severity is considered, no cracking rate is calculated, only concordance is used for
classification. In order to evaluate the concordance, each cell is given the value of severity associated
with the most severe crack in the cell.

This methodology can be used without considering the crack type, or it can be used for each type of
crack. For example, one can give the value of 1 in each cell as soon as one crack of any type is
present and thus measure a general cracking rate, or one can give the value of one in each cell only
when a transversal crack is present and thus measure a transversal cracking rate, etc.

The repeatability of the measurement is also considered and the equipment classification is based on
the coefficient of variation of the cracking rate and of the proportion of concordant cell conclusions

Classification thresholds
The results obtained can be used to classify equipment in terms of precision and reliability according
with the tables 6 and 7. The classification thresholds used can be determined on the basis of their
effect on a known cracking index (cracking rate or other). The highway administration can then make
its equipment choice based on its management needs. This classification is limited to systems
evaluated on the same set of ten tracks. Since the tracks will vary from country to country (or agency
to agency), the equipment validation classification in different countries (or agencies) can not directly
be compared.

- 16 -
 TABLE 6: CRITERIA PROPOSED FOR DETECTION AND BIAS

CLASS CRACKING RATE CELLS WITH CRACK CRACK SEVERITY

(FROM REFERENCE) CONCORDANCE WITH CONCORDANCE WITH
REFERENCE REFERENCE
AAA ±1% 100% 100%
AA ±2.5% 95% 90%
A ±5% 90% 80%
B ±10% 80% 70%
C > 10% and < 30 % < 80% ≥ 50% < 70% ≥ 50%
Class C is the lower limit for “suitable” devices.

For the cracking rate we compare the reference cracking rate with the measured cracking rate and the
difference determines the classification. For example, if the reference cracking rate is 80% and the
measured cracking rate is 83% then the equipment is class A. As stated earlier these classifications are
associated to each set of ten tracks.

For crack severity, each cell is given the value of severity associated to the most severe crack in the
cell. The classification is then determined by the proportion of cells with the same severity for both the
referenced and the measured results. For example, if a piece of equipment produces the same
conclusion on severity for 86% of the 2500 cells, the equipment would be class A.

TABLE 7: CRITERIA PROPOSED FOR REPEATABILITY

CLASS CRACKING RATE CRACK PRESENCE CRACK SEVERITY

(CELLS CONCORDANCE) (CELLS CONCORDANCE)
AAA C.V. < TBD C.V. < TBD C.V. < TBD
AA C.V. < TBD C.V. < TBD C.V. < TBD
A C.V. < TBD C.V. < TBD C.V. < TBD
B C.V. < TBD C.V. < TBD C.V. < TBD
C C.V. < TBD C.V. < TBD C.V. < TBD
TBD = to be determined according to the data
Note: The classification can be done for all types of cracks and bands or for any combination
of the type of crack and band.

3.3 NETWORK LEVEL VALIDATION TEST

In the network level tests the goal is to asses the capacity of the equipment to measure cracks over all
the types of road that will be found on the network. Therefore, these tests cover sites up to 100 km in
total length and compare the results delivered by the crack measurement device with the true level of
cracking present on the sites, using a simple approach, as described below.

Description of the sites

The sites for the network level tests should be chosen to cover a broad range of crack intensities,
pavement constructions (including fully flexible, rigid and composite), surface types, surface texture,
crack widths and crack orientations, as may be found on the road network to be surveyed by the
equipment. The network may therefore cover a very different range of surfaces and road types for
different countries, however, it is of most importance that a wide range of conditions be presented to
the crack detection system. For checking the performance of equipment in avoiding the reporting of
false positive cracks, it should be ensured that the sites also cover features such as:

ƒ Different surface types or surface texture (there could be many of these on the network);
ƒ Longitudinal and transverse joints;
ƒ Fretting;
ƒ Fatting;
ƒ Road Markings;

- 17 -
 ƒ Patches (sealed and unsealed);
ƒ Ironwork;
ƒ Longitudinal and transverse grooves (on rigid pavements).

It is probable that the test sites will be located in many different places on the network. These sites
should be defined in terms of road sections for the purposes of the tests.

The network tests would cover a wide range of challenges for the equipment. It is a further advantage
that a wide range of environmental conditions will also be covered (e.g. sun angles, surface state),
thereby providing a more comprehensive test. The tests will need to be carried out on many sites
which may have to be visited over a period of a few days by the equipment.

Data and data analysis

Since the length of sites for the network level tests is considerable, it is not practical to obtain
reference measurements by manually surveying the sites during road closures. Therefore an alternative
method is proposed, i.e. the visual inspection of digital images collected using a normal traffic speed
device. The images provided should have a resolution such that the minimum crack sizes to be
measured can be clearly identified. Experience has shown that a pixel size of 2 mm by 2 mm is
suitable for this. The survey width would also have to be adequate, for example 3 m. The images
would be displayed on a computer screen for visual inspection. Reference data would be obtained by
overlaying the images with a square grid and the grid-marked images would be inspected by eye to
identify cracking. Any grid tile (0.5 m X 0.5 m) containing a crack is counted. This method is
therefore able to provide reference crack data over long lengths without the need for road closures.

To test the automatic crack detection system the reference data is reported as the total area of grid tiles
containing crack within 50 m sub sections of survey data. The data is then normalised by calculating
the average area recorded over all of the sub-sections and dividing the area for each sub-section by this
average to obtain the Normalised Reference Area for each of the 50 m sub-sections. (Note: the overall
average value is 1).

For this network level test we need to know that the crack detection system is capable of identifying
lengths on the network that have high and low levels of cracking, and therefore the Normalised
Reference Area in each 50 m subsections is examined to classify the subsections as follows:

ƒ Sub-sections with a Normalised Reference Area greater than 1.75 are defined as sub-sections
containing high levels of cracking;
ƒ Sub-sections with a Normalised Reference Area less than 0.2 will be defined as sub-sections
containing low levels of cracking.

The automatic crack detection equipment surveys the network sites and the crack data provided is
processed in the same way. A grid is placed over the automated crack data and any grid squares in
which the automatic crack detection equipment has reported a crack is filled in. This is used to obtain
the area of cracking over each 50 m sub-section in the automated crack data. Again, the average
automated cracking area recorded over all of the sub-sections is calculated and the Normalised
Automatic Area for each of the 50 m sub-sections is obtained. The Normalised Automatic Area in
each 50 m subsections is examined to classify the lengths as follows.

ƒ Sub-sections with a Normalised Automatic Area greater than 1.75 are defined as sub-sections
containing high levels of cracking.
ƒ Sub-sections with a Normalised Automatic Area less than 0.2 will be defined as sub-sections
containing low levels of cracking.

Classification thresholds
The two datasets (reference and automatic) are then compared to determine if the system is able to
reliably identify cracking at the network level. In the evaluation the 50 m sub-sections containing a
high and low level of cracking in the Reference Data are compared with those reported in the
Automatic Data to contain a high and low level of cracking, and performance is assessed as follows:

- 18 -
If the Automatic Data reports a high level of cracking over at least 75 % of the 50 m sub-sections that
the Reference Data shows to have high level of cracking;

AND

The Automatic Data shows a low level of a cracking over at least 75 % of the 50 m sub-sections that
the Reference Data shows to have a low level of cracking,

THEN

The system is considered suitable for network level assessment of cracking. If the automatic system is
unable to meet this standard then it is likely that the system has a problem working on the range of
different surfaces and conditions encountered on the network. The developer of the automatic crack
detection equipment may need to investigate the reasons, which could include.

ƒ Problems with different surface types.

ƒ Problems with different crack types (width, angle fragmentation, spalling, etc.).
ƒ Problems with different surface textures.
ƒ Problems with surface features (such as road furniture, joints, etc.).

A further option for the network level tests is to assess the reproducibility of different crack
identification systems. In this test we assume that many systems have undertaken surveys of the test
sites and that there are several sets of Automatic Crack data available (one from each system).

A single average Cracking Area is obtained from this data set for each 50 m sub-section from the
Cracking Area provided by each set of survey equipment by averaging the results obtained from each
system over each 50 m length. The above tests are then repeated on the data provided by each system
to determine if they have a satisfactory level of performance when compared with each other. This test
can also include visual comparison of the crack areas provided by each system to identify any
significant local differences. The test also enables authorities to estimate sensitivity factors for
different systems so that the Cracking Areas reported by different types of equipment can be
correlated with each other.

- 19 -
4.0 CONCLUSION AND RECOMMENDATIONS

Cracks type, severity and extent are one of the most important data to select maintenance and repair
actions. These data are in fact, the basic data of a pavement management system. Much progress has
been made by some road administrations, service providers and researchers to develop automated
systems to collect the data. Since these technologies are available, few studies and publications have
been produced by the world scientific community in order to help road administrations to insure the
reliability of the automated crack measurements.

The quality of a manual survey at the network level depends on many factors such as the time of the
day when the analysis is carried out, the fatigue and the understanding and the subjectivity of the
operators. The automated systems are not affected by the same factors. Experience shows that
automated systems are more constant when it comes to surveying cracks than the manual method.
These systems can record and analyse many kilometres with the same good precision, especially if the
road conditions (texture, rut, sun etc…) do not change. The main problem is that at the network level,
the survey conditions as well as many road surface characteristics change at the same time, combined
with the lack of knowledge of the equipments performance, false detections and areas of inadequate
survey results are difficult to localize and to identify. These road characteristics that can affect the
analysis have to be taken into account when defining the validation test. Road administrations should
take time with the constructor or the service provider to establish a list of possible road conditions
affecting the results and try to test the survey vehicle in these conditions. The validation tests are
expected to evolve year after year and will depend on the knowledge gained of the system’s reliability.
These tests should become more precise and less time consuming over time.

Road administrations also have to identify their needs in terms of pavement management. This first
step will help them to identify which validation test to perform and the level of detail needed. It is
generally recognized that the more detailed the crack data are the more the PMS will recommend
precise and specific maintenance and repair interventions. Road managers should also fix the type and
the precision of the required performance curves they have or they will have in advance, which will
affect considerably the mid and long term strategies.

The concepts presented in this technical paper represent a summary of the discussions and the answers
received from a world wide survey of the road surveys and pavement management experts on this
subject. PIARC 4.2 Technical Committee members hope the concepts presented in this technical paper
will be viewed as real progress that aims at increasing the quality level of crack data measurements in
order to help road administrations to better manage their road networks.

- 20 -
5. BIBLIOGRAPHY

[1] PIARC TECHNICAL COMMITTEE C1 ON THE CHARACTERISTICS OF PAVEMENT SURFACES,

WORKING GROUP A, “Automated Pavement Cracking Assessment Equipment: State of the art”, May 2003.

[2] ASSTHO Designation: PP44-01, Standard Practice for Quantifying Cracks in Asphalt Pavement Surface,
AASHTO Provisional Standards, May 2002.

[3] MINISTÈRE DES TRANSPORTS DU QUÉBEC, Guide de mesure et d’identification des dégradations des
chaussées flexibles, 2007.

[4] PETRA OFFREL, LEIF SJÖGREN AND ROLF MAGNUSSON, Repeatability in Crack Data Collection on
Flexible Pavements: Comparison between Surveys Using Video Cameras, Laser Cameras, and a Simplified
Manual Survey, Journal of Transportation Engineering, July 2005

- 21 -
 APPENDICE

Detailed methodology for the research level validation test

- 22 -
Here is an example of a typical data file resulting from a research level validation process:

TABLE A1. TYPICAL DATA FILE FOR THE RESULTS OF A RESEARCH LEVEL VALIDATION PROCESS

TRACK REP LOCATION BAND LENGTH SEVERITY

# # Start Finish # (m) (mm)
SAW 1 0+000 0+010 1
SAW 1 0+000 0+010 2
SAW 1 0+000 0+010 4
SAW 1 0+000 0+010 5
SAW 1 0+010 0+020 9
… … … … … … …
SAW 2 0+000 0+010 1
SAW 2 0+000 0+010 2
SAW 2 0+000 0+010 4
SAW 2 0+000 0+010 5
SAW 2 0+010 0+020 9
… … … … … … …
SAW 5 0+590 0+600 1
SAW 5 0+590 0+600 2
SAW 5 0+590 0+600 4
SAW 5 0+590 0+600 5

Table A1 shows that each line corresponds to a crack (the sampling unit) and, as we can only have one
crack per band per 10m, it is quite easy to make the association from repetition to repetition of the
same crack. Since the transversal cracks affect several bands the analysis criterion is based on the fact
that we can have only one single crack per line, we suggest to record transversal crack as being in
band #9.

At this stage, three parameters are studied: the proportion of detected cracks, the crack length and its
severity. For each of them, the results are presented in a specific table form. The following tables
present some examples for a better comprehension.

- 23 -
 ƒ For the proportion of detected tracks, the table shows for each band and each repetition, the
proportion of cracks correctly identified. One can easily identify if a piece of equipment has a
problem with some specific bands (for example band #5 in table A2).
ƒ For crack length, the table shows, for each crack, the location of each crack, the reference length
value measured manually, the mean of the 5 repetitions, the standard deviation of the 5
repetitions, and the coefficient of variation (calculated from the reference value), the absolute
bias and the relative bias. The analysis can easily point out if the equipment has some
difficulties with some cracks, or with crack with given characteristics (for example in band #1 in
the cell starting at 140m, in table A3).
ƒ For crack severity, the structure of the table is identical as for crack length once again, the
analysis can easily show if the equipment has some difficulties with some cracks, or with crack
with given characteristics (for example in band #1 in the cell starting at 40m, in table A4).

The classification is directly determined from the table results. For example in table A2 (proportion
of detected cracks), the equipment must be classified AA for bands #1, #2, #3, #4 and #9 but
classified B in band #5 for longitudinal cracks. Notice that transverse cracks are recorded as being in
band #9. If a classification is given for all bands, it must be AA since the equipment detects more
than 90% in all bands. As far transversal cracks are concerned, the equipment would be classified as
AAA. For crack length and severity the classification thresholds are determined by counting the
proportion of cracks satisfying the relative bias or coefficient of variation (C.V.) criteria.

TABLE A2: PROPORTION OF DETECTED CRACKS (EXAMPLE)

REPETITION BAND #1 BAND #2 BAND #3 BAND #4 BAND #5 TRANSVERSALS ALL

(N=27) (N=29) (N=29) (N=30) (N=33) (N=20) (N=168)

1 100% 97% 87% 97% 72% 100% 91%

2 96% 100% 93% 90% 72% 100% 91%
3 93% 100% 93% 100% 70% 100% 92%
4 93% 97% 93% 97% 76% 100% 92%
5 89% 97% 93% 90% 70% 100% 89%
All 94% 98% 92% 95% 72% 100% 91%

- 24 -
 TABLE A3: CRACK LENGTH (M) (EXAMPLE)

DIRECTION START BAND REFERENCE MEAN STD. C.V. BIAS RELATIVE

OF 5 DEV. BIAS
REPS.
south 0 1 5 5.04 0.18 0,036 0.04 0.8%
south 20 1 5 4.71 0.12 0,024 -0.30 -5.9%
south 30 1 5 4.66 0.10 0,020 -0.34 -6.8%
south 50 1 3 2.87 0.08 0,027 -0.13 -4.4%
south 70 1 8 7.85 0.13 0,016 -0.15 -1.8%
south 110 1 3 2.85 0.14 0,047 -0.15 -4.9%
south 140 1 5 3.16 2.20 0,440 -1.84 -36.9%
… … … … … … … … …

TABLE A4: CRACK SEVERITY (MM) (EXAMPLE)

DIRECTION START BAND REFERENCE MEAN STD. C.V. BIAS RELATIVE

OF 5 DEV. BIAS
REPS.
north 0 1 3 5.9 1.20 0,400 2.9 96.7%
north 20 1 3 5.5 1.18 0,393 2.5 83.3%
north 40 1 2 4.8 1.95 0,975 2.8 140.0%
north 70 1 8 7.6 2.66 0,333 -0.4 -5.0%
north 110 1 8 7.9 1.20 0,150 -0.1 -1.3%
north 130 1 5 5.3 1.16 0,232 0.3 6.0%
… … … … … … … … …

- 25 -