Recently, the usage of Driver Assistance Systems
(DAS) and Driver Safety Support Systems (DSSS)
has been increased due to the expansion of
complicated road networks. These systems are used
and deployed in vehicles to ease the driving task and
to improve driver safety. Road signs are one
significant source of information for drivers and for
both DASs and DSSSs, but their visibility decreases
in many situations and under different
circumstances. Circumstances that affect road signs
visibility are either temporal because of bad weather
conditions or permanent because of vandalism and
bad postage of signs. Fig.1 shows some road signs
with low visibility.
Low visibility of road signs decreases the
chances of information transfer between drivers and
road signs and thus; DASs could be used to inform
drivers of warnings in such situations. In fact, good
DAS and DSSS should not provide drivers with a
lot of information over roadways since a lot of
information could lead to driver distraction problem
[1].
Fig.1. Examples of low visible road signs in
cluttered environments.
Computer vision techniques can be deployed in
both DASs and DSSSs to estimate the visibility of
road signs and accordingly inform drivers with the
most important warnings of low visible signs. Using
these techniques will increase the driver safety.
Image-based Visibility Estimation of Road Signs in
Cluttered Environment
Jafar J. Abukhait
Communications, Electronics and Computer Engineering Department
Tafila Technical University
Tafila, 66110, Jordan
jafar@ttu.edu.jo
Abstract: - This paper proposes an imaging-based
system to estimate road sign visibility in a cluttered
environment from the driver’s perspective in daytime
using in-vehicle camera images. The proposed system
can be deployed in both Driver Assistance Systems
(DAS) and Driver Safety Support Systems (DSSS) as
a choosing criterion of what information to provide to
drivers. Driver Assistance Systems can only provide
drivers with warnings about road signs with less
visibility and high importance. The proposed system
estimates the visibility by measuring the detect-ability
of road signs. The detect-ability parameter measures
the ability of the driver to locate road object from a
scene and thus; it measures sign postage with respect
to cluttered or complex environment. Road signs
posted in complex environment are harder to be
recognized by drivers and thus have low value for the
detect-ability parameter. This paper proposes a
visibility estimation system of road signs in the United
States and experimental results are used to show its
effectiveness.
Key-Words: - Road Sign Detection, Color
Segmentation, Edge Detector, Driver Assistance
System, Driver Safety Support Systems, Detect-
ability, Visibility Estimation.
I. INTRODUCTION
Volume 2, 2022
ISSN: 2769-2477
33
Road sign visibility estimation systems benefit from
Road Sign Detection (RSD) techniques. The goal of
RSD is locating the road sign object in a scene or
from an in-vehicle image. Road sign detection
techniques are categorized mainly into color based
and shape based. Color thresholding in RGB space
has been used to segment road sign images in [2, 3]
while Hue Saturation Intensity (HSI) space has been
suggested to segment road signs in [4].
Shape based methods were also suggested by
different researchers. In [2], four vectors of
distances from border to bounding box were trained
with SVM to recognize road sign shape. In [5],
distance to border (DtB) vector was deployed to
recognize the shape of road signs. Boosted detector
cascade was trained with dissociated dipoles to
detect ROI while Hough transform and radial
symmetry were used to recognize triangular or
circular shape road signs [6]. Genetic algorithm was
used in [7] while Haar-like features were used in [4]
to detect road sign shape. A set of cascaded
geometric detectors was used to in [8] to detect and
recognize road sign shapes benefiting from their
symmetric property.
Road sign visibility estimation from digital
images has been proposed by several researchers. In
[9], a novel technique has been suggested to
measure road sign retroreflectivity from two images
with different illumination. Support Vector Machine
(SVM) has been used to classify road signs based on
their deterioration levels [10]. In [11], both
detectability and discriminability of traffic signals
have been measured from in-vehicle images. In
[12], five image features were used to estimate the
visibility of specific road sign. In [13], visibility
estimation in foggy conditions has been proposed
using in-vehicle images.
In this paper, we propose an imaging based
system to estimate the visibility of road signs in the
United States in terms of their detect-ability. Detect-
ability is defined as the ability of the driver to locate
and recognize the existence of specific road sign in
a complex or cluttered environment. This proposed
system can be deployed in DSSSs to reduce the
amount of information provided to drivers. In
addition, transportation agencies could benefit of
such system to evaluate their sign postage over road
networks.
In-vehicle Camera Image
Road Sign Detection and Shape
Recognition
Segmentation of Surrounding
Regions
Road Sign Visibility
Estimation
Road Sign Detectability
Parameters Measurement
Fig.2. Flow diagram of the proposed system.
The road sign imaging-based visibility estimation
system, as shown in Figure 2, follows four stages.
1. Road sign detection and shape recognition: in
this stage, color thresholding and a set of
geometric detectors are applied on the in-
vehicle images to extract and recognize road
sign objects.
2. Segmentation of surrounding regions: this stage
segments geometrically the four neighboring
regions of road sign object.
3. Road sign detect-ability parameters
measurement: in this stage, two visibility
parameters (color difference and surrounding
complexity) that describe road sign detect-
ability are established.
4. Road sign visibility estimation: in this stage, the
values of detect-ability parameters are used to
classify the road sign visibility as: low, medium,
or high.
Extracting the road sign region from the input image
is necessary to estimate its visibility. In [8], we have
proposed a method to detect and recognize road sign
shapes, in which color thresholding is applied firstly
to extract possible speed or warning signs regions
II. BACKGROUND AND SIGNIFICANCE
III. THE PROPOSED SYSTEM
A. Detection and Shape Recognition
Volume 2, 2022
ISSN: 2769-2477
34
S
R1
R2
R3
R4 R1
R4
R2
R3
S
a b
Fig.3. The road sign region and its four surrounding
regions. a) rectangular road sign and surrounding
regions. b) diamond road sign and surrounding
regions.
(blobs). Secondly, a set of geometric detectors have
been applied on each blob to keep only the ones that
are possible road sign regions. These geometric
detectors are: area, solidity, and dimensions ratios.
Finally, the relative positions of the object’s vertices
are used to determine the shape whether it is
rectangular or diamond or other symmetric shape.
Visibility of road sign in this proposed work is
defined as the ability of the driver to detect its
region from background regions in an actual scene.
Different background features could distract the
driver from detecting the road sign region.
Measuring the visibility is done by comparing road
sign region against its background regions. Four
neighbouring regions have been extracted from the
input image for both rectangular and diamond sign
shapes as shown in Fig.3.
Segmentation of road sign regions has been
achieved by finding the four vertices of each sign
shape as shown in Fig.4. The four vertices of
rectangular sign shape are: top-left (TL), top-right
(TR), bottom-right (BR), and bottom-left (BL) while
the four vertices of diamond sign shape are: top (T),
right (R), bottom (B), and left (L). These vertices
are used to calculate the four dimensions of each
symmetric shape.
The four regions are cropped from the input
image such that each region has a symmetric shape
and a double area of the sign region. The four
surrounding regions are labelled as: R1, R2, R3,
and R4 while the sign region is labelled as S as
shown previously in Fig.3.
T
R
B
L
L1
L2
L3
L4
TL TR
BL BR
L1
L2
L3
L4
ab
Fig.4. The four vertices of each sign shape along
with its dimensions. a) rectangular sign shape. b)
diamond sign shape.
Color and shape features of surrounding regions are
used to establish detect-ability parameters which
would be used to determine the visibility level of
road sign. Road sign that has surrounding regions
with complex background with similar background
color is difficult to be detected and thus; has a low
visibility value. On contrast, road sign that has a
simple background of surrounding regions with
different background color is easier to detect and
thus; has a high visibility value.
Two detect-ability parameters are proposed to
describe the visibility of road signs: 1) color
difference between sign region and the four
surrounding regions and 2) shape complexity of
surrounding regions.
Color Difference
The average color of the RGB values is calculated
for the sign region and the surrounding regions. The
color difference between sign region and each one
of the four surrounding region is defined as:
where () are the average RGB colors in the
sign region and () are the average RGB
colors in the surrounding region Ri.
B. Segmentation of Surrounding Regions
C. Detect-ability Parameters Measurement
Volume 2, 2022
ISSN: 2769-2477
35
The four difference values can then be averaged
to calculate the color difference value D. Low color
difference decreases the chances of road sign
detection by a driver while high color difference
increases the detection results. This means that the
highly color difference is the better of road sign
visibility.
Shape Complexity
This Parameter measures the amount of details on
the surrounding regions. The edges of all
surrounding regions are extracted and the number of
pixels of these edges is calculated. The ratio
between the number of edges pixels and the total
number of pixels in the surrounding regions is used
to determine the shape complexity of road sign
surroundings as follow:
where NE is the number of pixels of all edges in the
surrounding regions and NT is the total number of
pixels in the surrounding regions.
Simple road sign surrounding environment will
yield in a low value for the complexity parameter
and thus; will increase the visibility level.
Road signs are classified in terms of visibility levels
to: low, medium, or high. Both the color difference
value (D) and the shape complexity value (C)
calculated in the previous stage are used together to
decide the visibility level as shown in Table 1.
Table 1: Visibility estimation using detect-ability
parameters.
Color
Difference
Complexity
Visibility
Level
low
high
low
low
low
medium
high
high
medium
high
low
high
Color difference and shape complexity
parameters are weighted equally in the decision of
road sign visibility level.
R1
R2
R3
R4
R0
a b
Fig.5. An example of the segmentation process of
the four surrounding regions of a warning sign.
The proposed visibility estimation system has been
tested on road signs from the United States. A
sample of in-vehicle images of 28 rectangular
regulatory signs and 34 warning signs has been
chosen to verify the effectiveness of the proposed
system. These in-vehicle images have been captured
using SAMSUNG ST65 camera in addition to
images from VISATTM Mobile Mapping System.
Moreover, the proposed visibility estimation system
has been implemented in MATLAB software
running on 2.4-GHz i3 CPU.
In our proposed system, each road sign should be
classified as high, medium, or low in terms of
visibility level. The decision of visibility level has
been taken according to the values of both color
difference and complexity parameters between road
sign region and the four surrounding regions. Fig.5
shows an example of segmenting the four regions of
a warning sign. The relation between visibility
parameter values and visibility levels is shown in
Table 2. These parameter threshold values have
been chosen based on a set of road sign images
(training set). Color difference threshold values are
different between white and yellow signs because
illumination affects white color (achromatic color)
sharply.
Table 2: Relation between visibility parameter
values and visibility levels.
Color
difference
with white
signs
Color
difference
with yellow
signs
Complexity
High
>100
>120
> 5%
Low
<100
<120
< 5%
D. Visibility Estimation
IV. EXPERIMENTAL RESULTS
Volume 2, 2022
ISSN: 2769-2477
36
a b
Fig.6. Two in-vehicle images of road signs with visibility estimation decision. a) speed sign with visibility level
low by the proposed system and medium by the expert. b) warning sign with visibility level low by both the
proposed system and the expert.
The visibility results have been compared with
decisions from human expert. Table 3 shows the
results of the proposed system against the human
expert results. The comparison shows an agreement
of both decisions on 52 road signs with an accuracy
of 84% while 10 road signs have been decided
differently. These 10 road signs have not decided
extremely different between the proposed system
and the expert.
Table 3: Comparison between the numbers of road
signs decided similarly and differently by the
proposed system and the expert.
Total
number
Number
of signs
decided
similarly
Number
of signs
decided
differently
Rectangular
road signs
(white
color)
28
21
7
Warning
signs
(yellow
color)
34
31
3
Table 3 shows that the proposed system has
worked better with yellow road signs than white
ones. This difference happens because of
illumination factor which is affect white color
(achromatic color) sharply and thus; the color
difference visibility parameter may not describe the
situation accurately.
Finally, it is worth to say that even for cases of
disagreement, the decision between the proposed
system and the expert does not differ extremely. In
eight disagreement cases between the expert and our
system, the visibility decision has one level
difference. Fig.6 shows examples of visibility
estimation of both cases where agreement and
disagreement happens between the expert and the
proposed system.
In this paper, we have proposed an imaging-based
technique to estimate the visibility of road signs in
the United States. We have concentrated on the
detect-ability of road signs by drivers on roadways.
The proposed system can be deployed in Driver
Assistance Systems (DAS) as a choosing criterion
of what to display to drivers. The proposed system
has measured two visibility parameters; color
difference and shape complexity between road sign
and its background, to classify road signs with three
visibility levels: high, medium, and low.
We are working on improvements such as: 1)
measuring other visibility parameters to estimate the
content readability of road signs and to estimate the
effect of partial occlusion on road signs visibility; 2)
deploying the proposed system on larger and varied
set of road signs; 3) comparing the results of the
proposed system with more number of human
expert decisions; and 4) estimating the visibility of
road sign content.
V. CONCLUSION
Volume 2, 2022
ISSN: 2769-2477
37
[1] Doman, K.; Deguchi, D.; Takahashi, T.;
Mekada, Y.; Ide, I.; Murase, H.; Tamatsu, Y.,
"Estimation of traffic sign visibility toward
smart driver assistance," in Intelligent Vehicles
Symposium (IV), 2010 IEEE , vol., no., pp.45-
50, 21-24 June 2010
doi: 10.1109/IVS.2010.5548137.
[2] S. Maldonado-Bascón, S. Lafuente-Arroyo, P.
Gil-Jiménez, H. Gómez-Moreno, and F. López-
Ferreras, “Road-sign detection and recognition
based on support vector machines,” IEEE
Trans. Intell. Transp. Syst.,vol. 8, no. 2, pp.
264–278, Jun. 2007.
[3] A. de la Escalera, L.E. Moreno, M.A. Salichs,
J.M. Armingol, “Road traffic sign detection and
classification,” Industrial Electronics, IEEE
Transactions on , vol.44, no.6, pp.848-859, Dec
1997.
[4] A. de la Escalera*, J.MaArmingol, M. Mata,
“Traffic sign recognition and analysis for
intelligent vehicles,” Image and Vision
Computing 21 (2003) 247–258.
[5] J.F. Khan, S.M.A. Bhuiyan, R.R. Adhami,
“Image Segmentation and Shape Analysis for
Road-Sign Detection,” Intelligent
Transportation Systems, IEEE Transactions on
, vol.12, no.1, pp.83-96, March 2011.
[6] X. Baro, S. Escalera, J. Vitria, O. Pujol and P.
Radeva, “Traffic Sign Recognition Using
Evolutionary Adaboost Detection and Forest-
ECOC Classification,” Intelligent
Transportation Systems, IEEE Transactions on,
vol. 10, pp. 113-126, 2009.
[7] Jialin Jiao, Zhong Zheng, Jungme Park, Y.L.
Murphey, Yun Luo, “A robust multi-class
traffic sign detection and classification system
using asymmetric and symmetric features,”
Systems, Man and Cybernetics, 2009. SMC
2009. IEEE International Conference on , vol.,
no., pp.3421-3427, 11-14 Oct. 2009.
[8] Abukhait, J.; Abdel-Qader, I.; Jun-Seok Oh;
Abudayyeh, O., "Road sign detection and shape
recognition invariant to sign defects,"
Electro/Information Technology (EIT), 2012
IEEE International Conference on , vol., no.,
pp.1,6, 6-8 May 2012.
[9] Balali, V., Sadeghi, M.A., and Golparvar-Fard,
M. (2015). "Image-based Retro-Reflectivity
Measurement of Traffic Signs in Day Time."
Elsevier Journal of Advanced Engineering
Informatics, 29(4), 1028-1040.
[10] P. Siegmann, S. Lafuente-Arroyo, S.
Maldonado-Basc´on, P. Gil-Jim´enez, and H.
G´omez-Moreno, “Automatic evaluation of
traffic sign visibility using SVM recognition
methods,” in Proc. 5th WSEAS Int.Conf. on
Signal Processing, Computational Geometry &
ArtificialVision, pp. 170–175, September 2005.
[11] Kimura, F.; Takahashi, T.; Mekada, Y.; Ide, I.;
Murase, H.; Miyahara, T.; Tamatsu, Y.,
"Measurement of Visibility Conditions toward
Smart Driver Assistance for Traffic Signals," in
Intelligent Vehicles Symposium, 2007 IEEE ,
vol., no., pp.636-641, 13-15 June 2007.
[12] Doman, K.; Deguchi, D.; Takahashi, T.;
Mekada, Y.; Ide, I.; Murase, H.; Tamatsu, Y.,
"Estimation of traffic sign visibility considering
temporal environmental changes for smart
driver assistance," in Intelligent Vehicles
Symposium (IV), 2011 IEEE , vol., no.,
pp.667-672, 5-9 June 2011.
[13] K. Mori, T. Kato, T. Takahashi, I. Ide, H.
Murase, T. Miyahara and Y. Tamatsu,
“Visibility Estimation in Foggy Conditions by
In-vehicle Camera and Radar,” Proc.
International Conference on Innovative
Computing, Information and Control, Vol. 2,
pp. 548-551, Aug. 2006.
REFERENCES
Volume 2, 2022
ISSN: 2769-2477
38
