2022 Electrical Power, Electronics, Communications, Controls and Informatics Seminar (EECCIS)

FPGA Displays Real-Time Video Camera on VGA

Monitor
Effendi Dodi A. Agus Hendra Wahyudi Kurdianto
Electrical Department Antenna Department Electrical Department
Research Center for Rocket Technology Research Center for Aeronautic Research Center for Remote Sensing
National Research and Innovation National Research and Innovation National Research and Innovation
Agency (BRIN) Agency (BRIN) Agency (BRIN)
Bogor, Indonesia Bogor, Indonesia East Jakarta, Indonesia
2022 11th Electrical Power, Electronics, Communications, Controls and Informatics Seminar (EECCIS) | 978-1-6654-0648-2/22/$31.00 ©2022 IEEE | DOI: 10.1109/EECCIS54468.2022.9902940

[email protected] [email protected] [email protected]

Nugroho Widi Jatmiko Faisa Lailiyul M. Wahyu Widada

Electronic Department Electrical Department Electrical Department
Research Center for Remote Sensing Research Center for Rocket Technology Research Center for Rocket Technology
National Research and Innovation National Research and Innovation National Research and Innovation
Agency (BRIN) Agency (BRIN) Agency (BRIN)
East Jakarta, Indonesia Bogor, Indonesia Bogor, Indonesia
[email protected] [email protected] [email protected]

Abstract—The OV7670 camera module is 8 bits data image understanding which type and language of the FPGA that
and it has a specification image of 30 frames per second. There will be used in the experiment due to many kinds of FPGA
are many applications of research that have been done about chip and language. The FPGA has digital input/output,
how to access this camera module and display it to the monitor. whereas VGA has Analog input for data image and some
This paper presents the interface between this camera module digital inputs for synchronization. According to that, one of
OV7670 and FPGA DE0-Nano-Soc and then displays the video the applications of the FPGA is monitor interfacing via the
to the monitor via a VGA port. VGA port has a minimum of 5 VGA port using VGA driver such as simple as R-to-R DAC
pins; 3 pins for RGB data images, 1 pin for horizontal or another.
synchronous, and 1 pin for vertical synchronization. Due to the
FPGA DE0-Nano-Soc not having the VGA driver, so the VGA The VGA port of a monitor has 15 pins, but not all pins
driver ADV7133 24 bits is used in this research. The result must be connected. There are 5 pins that must be connected
showed that this research can work well enough. The system to the other device if will display an image or video. The 5
successfully to take the image from the camera and displays it pins are ground; 3 pins for RGB, 1 pin for vertical
as video in real-time. synchronous, and 1 pin for horizontal synchronous. The data
input for VGA monitor is analog signal which is represented
Keywords— camera OV7670, FPGA, real-time, VGA driver with RGB (Red-Green-Blue). The analog signal can be
driven from DAC module or simple DAC with combination
I. INTRODUCTION of some resistors.
The OV7670 camera module has its advantages. It is the The OV7670 camera module Interfacing is not easy to
CMOS camera with a relatively cheap price compared to implement, but it become easier by using The FPGA board
other similar cameras. Furthermore, this camera has a good which has a VGA port such as DE-115. The OV7670 camera
enough image performance. Not only has good image module interfacing using the FPGA implementation
performance, but also has 8 bits interface data image, and presented in the cited paper [3]. According that paper, this
some controller such as clock, vertical synchronous, and research is developed by using DE0-Nano-SoC FPGA board
horizontal synchronous. There are many application of this which has no VGA port. Thus, this experiment needs another
camera such as video surveillance car [1], image acquisition module to replace the needed of VGA port which has 24 bits
and display system based on STM32 and OV7670 camera VGA + audio by using the IC ADV7123 chip.
module [2]. Due to this camera is digital, so it can be
connected to many kinds of the microcontroller or FPGA [3]. This experimental result showed that the system design
Some experiments have done with this camera and FPGA worked very well. The FPGA DE0-Nano-Soc board
such as in the application of cleaning garbage on the road successfully interfaced with the OV7670 camera module. It
[4]. worked to take the data image and transfer the data by using
ADV7123 driver interfacing in real-time. The video result
FPGA which abbreviation from Field Programmable was clear enough. However, it is found that some images or
Gate Array is a gate array that can be programmed many videos color is not 100 percent exactly the same as the input.
times such as FPGA Cyclone V. It has an external input This paper is organized on 5 sections; introduction,
clock up to 50MHz. FPGA is a high-performance computing methodology, setup experiment, result, and discussion, the
platform and it has three main contributions which are hard last is the conclusion and future works.
real-time computations, parallelism, and high user I/O pin
count [5]. The FPGA has many fields of application rather
than computing architectures which are communication, II. METHODOLOGY
image processing, control engineering, networks, Implementing of interface between the OV7670 camera
cryptography, mathematics, neuro-computing, processor module with FPGA DE0-Nano-SoC board and monitor via
design, data acquisition, simulation, prototyping, medical, VGA port requires some materials. The input image is from
etc. The first required step to do an experiment with FPGA is the OV7670 camera module and then display it to the

978-1-6654-0648-2/22/$31.00 ©2022 IEEE 129

Authorized licensed use limited to: Texas A M University. Downloaded on November 29,2023 at 13:06:35 UTC from IEEE Xplore. Restrictions apply.
monitor via the VGA port, whereas ADV7123 is used to It can be connected through programmable interconnects.
change the digital signal into the analog signal. Due to this programmable, FPGA is ideal for a large
computation of prototyping, automotive, industrial, medical,
A. The OV7670 Camera Module aerospace and defense etc. In many application that The
The OV7670 camera module is a low voltage CMOS FPGA board DE0-Nao-SoC is used such as in the rocket
device which provides the good functionality of a single-chip technology [10]. Fig. 2 is the module of the DE0-Nano-SoC
image processor in a small package. Another specification of which is used in this experiment and this module has FPGA
Camera OV7670 are low-noise, high integration, wide chip IC Cyclone V 5csema.
dynamic range and excellent low-light performance. The
OV7670 camera module provides an image sensor with
interface data 18 pins which operates at maximum 30 fps [6].
The OV7670 sends the data image in a parallel synchronous
mode. The working of the OV7670 is needed the supply a
clock signal on the XCLK pin, the source of clock can be
from independent clock or from another controller clock
such as FPGA. Due to the FPGA board has internal clock
and the output pin of FPGA also can produce a clock for
OV7670. According data sheet of OV7670 that the clock
must have a frequency between 10 and 48 MHZ. After a
clock signal from the FPGA board has been applied to the Fig. 2. The DE0-Nano-SoC Board Cyclone V [11]
XCLK pin, then OV7670 will start driving its VSYNC,
HREF, and D0-D7 pins. Fig. 1 is the OV7670 camera B. Driver VGA ADV7123
module which is used in this experiment.
VGA (video graphics array) has become standard
Table I is the information about the pin which on the interface in many application such as ATM machines,
camera for the type and function. Therefore there are 18 pins video player, desktop computer or video surveillance. The
will be connected to the FPGA as communication data standard of the VGA was originally developed by IBM in
FPGA Board DE0-Nano-SoC 1987 with resolution 640 x 480 pixels. The VGA standard
The FPGA technology has gained vital importance timings are shown in Table II.
because of its parallel processing which makes it ideal for
video and image processing [7]. Now days, image processing TABLE II. VGA STANDARD TIMING
has become very useful in the field at digital photography,
Horizontal Timing (line) Vertical Timing (frame)
medical image, , video surveillance and IoT application [8].
Scanline Time Time
FPGA is programmable with a group of programmable Part
Pixels
[us]
Frame Part Lines
[ms]
interconnect resources through digital circuits or Visible
Visible area 640 20.31 480 12.67
programmable logic blocks surrounded by input/output Area
blocks [9]. Front porch 24 0.76 Front porch 9 0.23

Sync pulse 40 1.26 Sync pulse 2 0.05

Bach porch 128 4.06 Bach porch 29 0.76

Whole
Whole line 832 26.41 520 13.73
frame

The DE0-Nano-SoC board doesn’t have VGA port, to do

interface between this board and monitor via VGA port, thus
it needs external board of VGA driver. The VGA driver can
be from the simple DAC or using the IC driver, due to the
input of VGA is analog signal. There are two signal
Fig. 1. The OV7670 Camera Module synchronization for VGA interface, those are vertical and
horizontal synchronization.

TABLE I. THE OV7670 CAMERA MODULE PIN OUT

PIN Type Description
VDD Supply Power Supply
GND Supply Ground level
SDIOC Input SCCB clock
SDIOD Input/Output SCCB clock
VSYNC Ouput Vertical synchronization
HREF Output Horizontal synchronization
PCLK Output Pixel clock
XCLK Input System clock
D0-D7 Output Video parallel output
RESET Input Reset (active low) Fig. 3. A Driver VGA Port ADV7123
PWDN Input Power down (active high)

130
Authorized licensed use limited to: Texas A M University. Downloaded on November 29,2023 at 13:06:35 UTC from IEEE Xplore. Restrictions apply.
 This research choosing the driver for VGA from IV. RESULT AND DISCUSSION
ADV7123 board as shown on Fig. 3. The controlling the A general design stage hardware implementation of
synchronization signal from the FPGA pins to the circuit on interface between the OV7670 camera module with the
the ADV7123 board. The ADV7123 has 24 bits to produce FPGA and monitor is illustrated in Fig. 4. The image
the analog data signal; red, green, and blue. The ADV7123 processing is achieved by using a low-cost OV7670 camera
board can support standard VGA specification with module as mentioned before. The camera is connected to the
resolution 640 x 480 pixels at 25 MHz. GPIO-1 of the FPGA DE0-Nano-SoC board using jumper
wires as shown in Fig. 5. Due to the FPGA board doesn’t
III. SETUP EXPERIMENT have a VGD driver as like the FPGA DE1-SoC board or
The experiment setup requires some modules to be another board, thus the VGA driver is connected to the
connected together become a system. The block diagram of FPGA board also using jumper wires then the connection
this system is shown in Fig. 4. The setup includes the DE0- between the VGA driver to monitor using a standard VGA
Nano-Soc FPGA board that has connected camera module cable.
OV7670 and it is also connected to the regular monitor via The working of the OV7670 camera module depends on
driver VGA ADV7123 module. The FPGA board comes some control such as a clock, PWDN, reset, SIOC, and SIOD
without a power adapter and the board doesn’t have a power pins. The output data of the camera is a parallel mode with 8
switch, so the power source must be 5 volt. The connection pins D0-D7 and the data transmission is accomplished only if
between the camera module and the FPGA board is done the Vsync signal is low then Href starts a pixel transfer data
using regular jumper wires, also the connection between the takes cycles.
FPGA module and driver VGA board. The connection
between the VGA driver and the monitor is made through a The following images are the result of real-time video
standard video graphics array. The jumper wires connect the streaming then captured by the phone. The algorithm doesn’t
pins of the camera to the FPGA board one-to-one pay attention to the quality of the image or video streaming.
connections are listed in Table III. The input and output from Fig. 6 show that the input directly
can be displayed to the monitor. The algorithm for taking
The OV7670 The FPGA The VGA Driver images from the OV7670 camera module can be seen on the
Camera Module DE0-Nano-SoC ADV7123 Algorithm 1.

Monitor
VGA Port

Fig. 4. Block Diagram for Setup Experiment

Fig. 5. Setup Module Experiment

a. Video Display

TABLE III. FPGA PIN CONNECTION CAMERA AND VGA DRIVER

FPGA GPIO_0 FPGA GPIO_0

OV7670 Pin OV7670 Pin
Pin Pin
Descrip- Descrip- Descrip-
Pin Pin Pin Pin Description
tion tion tion
1 3v3 29 3v3 2 GND 30 GND

3 SIOC 27 PIN_AE7 4 SIOD 28 PIN_AF6

5 Vsync 25 PIN_T13 6 Href 26 PIN_T11

7 PCLK 23 PIN_AF5 8 XCLK 24 PIN_AE4

9 D7 21 PIN_AF4 10 D6 22 PIN_AG6

11 D5 19 PIN_AH3 12 D4 20 PIN_AH2 b. Input and Ouput

13 D3 17 PIN_AH4 14 D2 18 PIN_AG5
Fig. 6. Input and output Camera
15 D1 15 PIN_AH5 16 D0 16 PIN_AH6

17 Reset 13 PIN_T8 18 PWN 14 PIN_T12

131
Authorized licensed use limited to: Texas A M University. Downloaded on November 29,2023 at 13:06:35 UTC from IEEE Xplore. Restrictions apply.
 Basic knowledge about VHDL programming and digital form arduino, FPGA with included VGA port or not. In this
design as well skills to use the Quartus II are important. project, the goal of implementing a hardware system to show
Therefore, this research experiment is not intended for video with real-time was achieved. In this experiment, we
general people but rather for researcher, educator, and used the FPGA DE0-Nano-SoC module without VGA port.
practitioners in the field of computer vision, FPGA Thus we added the VGA driver ADV73123 to realize the
programming, and digital design. On the internet we can find interfacing with VGA monitor.
a lot of the example of interface camera with microcontroller,
FPGA or another processor. However, a future improvement of the quality of video
result with improving the algorithm or hardware system.
Another improvement that this system can be applied to the
Algorithm 1: Capture Camera
rocket which included to the onboard computer then the
Input : vsync, href. Data camera data can be transmitted to ground station when flying
Output : address, dataOut, We rocket test.
Signal: dataLatch, line, haref, Last, weRTegisrwe, hrefhold, latchVsync
While clk do
If clkRisingHigh then ACKNOWLEDGMENT
Address = address + 1
End
With the fulfillment at this project, we would like to
If HrefHold = 0 and LatchedHref = 1 then thank both of our leader which had supported this research
and the government which had support the funding
End laboratory facility of Research Center for Rocket
HrefHold = LatchedHref Technology, BRIN.
if LatchedHref = 1
DataLatch = DataLacth and LatchedData
End REFERENCES
if LatchedVsyn = 1 then [1] W. Remote and V. Surveillance, “Design of Wireless Remote Video
address = address, others = 0 Surveillance Car,” vol. 35, no. 2, 2017, [Online]. Available: Journal
HrefLast = HrefLast, others = 0 of Jilin University (Information Science Edition).
Line = Line, others = 0 [2] L. Qin and Y. Chen, “Design and implementation of paper count
display system based on STM32,” ACM Int. Conf. Proceeding Ser.,
else no. Icemc, pp. 218–222, 2019.
if HrefLast(2) = 1 then
[3] C. Ababei, S. Duerr, J. Ebel, R. Marineau, M. G. Moghaddam, and T.
if Line(1) = 1 then Sewell, “Open source digital camera on field programmable gate
WeRegister = 1 arrays,” IEEE Int. Conf. Electro Inf. Technol., vol. 2016-Augus, pp.
end 151–155, 2016.
else [4] L. Cui, T. Zhang, X. Yin, and Y. Qi, “Road Garbage Cleaning Device
HreftLast = HrefLast and LachedHref Based on ZigBee Gateway and Image Recognition,” Proc. 31st
end Chinese Control Decis. Conf. CCDC 2019, pp. 5603–5607, 2019.
end [5] K. Vipin and S. A. Fahmy, “FPGA dynamic and partial
if clkFallingEdge then reconfiguration: A survey of architectures, methods, and
applications,” ACM Comput. Surv., vol. 51, no. 4, 2018.
LatchedData = data
[6] C. Vakerlis, “University Of Thessaly Implementation Of Camera
LatchedHref = href
Based Computer Vision Algorithms On A Author : A Thesis
LatchedVsync = vsync submitted for the degree of,” no. July 2015, 2017.
end [7] O. Reiche, M. A. Ozkan, R. Membarth, J. Teich, and F. Hannig,
end “Generating FPGA-based image processing accelerators with Hipacc:
(Invited paper),” IEEE/ACM Int. Conf. Comput. Des. Dig. Tech. Pap.
ICCAD, vol. 2017-November, pp. 1026–1033, 2017.
The pixel’s clock frequency can be calculated as formula [8] A. Rupani, “interfacing of Raspberry-Pi and FPGA using IoT,” pp.
1, where Hactive x Vactive represents the display resolution 350–353, 2017.
such as in this experiment used 640 x 480, thus the HActive [9] J. Wang and A. Huang, “Image recognition of martial arts movements
is 640 and the VActive is 480. The PercentBlanking is the based on FPGA and image processing,” Microprocess. Microsyst.,
blanking period or the percentage of time when active video vol. 80, no. October 2020, pp. 103536, 2021.
is not being displayed [12]. [10] Effendi Dodi Arisandi, “Preliminary Design of Pulse Code
Modulation Encoder for Telemetry Data Rocket,” 2020, p. AIP
PixelClockFreq = HActive * VActive * FrameRatex (1 + Conference Proceedings 2226, 030005 (2020); ht.
PercentBlanking) (1) [11] T. Inc., “De0-nano-soc kit/atlas-soc kit,”
= 640 * 480 * 60 * (1 + 0.36) https://www.terasic.com.tw/cgi-
bin/page/archive.pl?Language=English&No=941 (2013), accessed:
= 25.06 MHz 2021-04-16.
V. CONCLUSION AND FUTURE WORKS [12] I.K. Anyiam, “Finding the right pixel clock frequency and throughput
for an LVDS display resolution,” 2018.
In recent years, interfacing the OV7670 camera module https://e2e.ti.com/blogs_/b/analogwire/posts/finding-the-right-pixel-
has rapidly done in many applications such as robotic, vision, clock-frequency-and-throughput-for-an-lvds-display-resolution.
and automatic control system. The main processor can be

132
Authorized licensed use limited to: Texas A M University. Downloaded on November 29,2023 at 13:06:35 UTC from IEEE Xplore. Restrictions apply.