ISSN (Print) : 0974-6846

Indian Journal of Science and Technology, Vol 10(9), DOI: 10.17485/ijst/2017/v10i9/109060, March 2017 ISSN (Online) : 0974-5645

Design and Development of FPGA Based Image

Acquisition System
Naga Raju Boya1*, Vijay Kumar Jinde1, Bala Venkateswarlu Avvaru2, Sreelekha Kande3 and
Ramanjappa Thogata2
1
Department of Physics, SKUCET, Sri Krishnadevaraya University, Anantapur - 515003, Andhra Pradesh, India;
[email protected], [email protected]
2
Department of Physics, Sri Krishnadevaraya University, Anantapur- 515003, Andhra Pradesh, India;
[email protected], [email protected]
3
Department of Physics, Government Arts and Science College, Anantapur - 515 001, Andhra Pradesh, India;
[email protected]

Abstract
Background/Objective: This article presents an image acquisition system which consists of two important elements,
namely a CMOS sensor and an FPGA. Method: In this article, the domain under observation is incessantly supervised by
capturing the video of the area. This is successfully completed with the help of the CMOS sensor, through which the image
frames are captured. Pixel data interface and CMOS senor are connected to the FPGA through the I2C bus. VGA Controller
module in the FPGA will read the pixel data from one of the ports of DDR SDRAM and it can display the video on LCD
monitor. FPGA used in the present work is Cyclone II, which is manufactured by ALTERA. Quartus II 13.0 suite is used for
software development. FPGA will be programmed with the help of Verilog HDL. Findings: The mode register of the CMOS
sensor is programmed with I2C protocol, so that CMOS settings like resolution, exposure are configured with this protocol.
ALTERA DE2 board consists of SDRAM, which can store the captured image frames. This system consists of the black and
white converter module so that the black and white images only stored in the data. VGA Controller module in the FPGA
always reads the pixel data from one of the ports of DDR SDRAM and it generates the required signals to display the video
on the monitor.

Keywords: CMOS Sensor, FPGA, SDRAM, VGA Controller

1. Introduction cameras for image processing using intelligent embedded

systems have faith on FPGA-based architectures5,6. On
Now a day’s advances in FPGA technology have impres- the end, the most crucial rewards of the FPGA is the facil-
sively increased the use of FPGAs for computer vision ity to effort the in-house parallel existence of many sight
applications1. With the help of advanced versions of algorithms7,8. The persona of FPGA in embedded systems
FPGAs, one can get both software programmable proces- is acquiring importance referable to its increasing poten-
sor and hardware computing resources on the same IC2. tialities and handiness of mighty EDA tools9,10. A number
The hardware is enforced on the sufficient logic blocks of available sources in today’s FPGAs is rather high and
of the chip. To implement the application software, these can practically deal numerous suing operations. FPGA
ICs are also having an additional integrated processor could instantly operate the data coming from the sensor
with the system software3. In the case of signal process- or any acquisition device11.
ing and image processing area, FPGA-based embedded Presently, for the execution of software algorithms,
systems importance is increasing a lot4. In that, smart FPGA technology becomes an alternative. The radically

*Author for correspondence

 Design and Development of FPGA Based Image Acquisition System

distinctive architecture of the FPGA has permitted the used to program the EP2C35F672C6 which is basically a
technology to be used in number of applications like close Cyclone II FPGA.
observation of a person or group (usually by the police)
by capturing video and medical imaging etc. FPGA 2.2 I2C protocol
belongs to a VLSI circuit which can be re-programmed
Inter Integrated Circuit bus can be written in a short form
in a number of times12,13. The word “field programmable”
as I2C. This can also be represented as IIC and I2C, which
concerns the ability to change the performance of the
is a consecutive and co-occurrent bus protocol. How the
device. The word gate array pertains to the basic inter-
data is processed through the I2C bus is clearly charac-
nal structure which allows re-programming14. In general,
terized by the fabricator of the I2C chips. Generally I2C
image processing algorithms are enforced with the help
system can have one master and few slaves, but the mas-
of DSPs and ASICs15. The diligent use of FPGAs in the
ter can produce the clock pulse. The master also specifies
field of image processing and video processing has a great
at what time communication likely to take place. For
regard. This is because of the collimated and superior
bus timing, it is significant that the obtuse slave should,
computational compactness of FPGA as likened to a
however, be capable of following the master’s clock.
general-purpose microprocessor16. Images and videos are
Alternatively, one can say that, the bus is as quick as the
captured by CMOS sensor and it can commute them into
lazy slave.
digital format, procedures and produces gumption of the
information that it accepts in a period of time17. The block
diagram of the FPGA-based image acquisition system is 3. Hardware Specifications
shown in Figure1.

3.1 CMOS Image Sensor

2. Software Specifications The CMOS image sensor used in the present work is
MT9M011, which is one third inch active-pixel digi-
2.1 Quartus II tal image sensor with a resolution of 1280Hx1024V.
Advanced camera operations are integrated on CMOS
Quartus II software allows productivity enhancements
chip, which includes snapshot mode, windowing, column
result­
ing in faster simulation, faster board bring-up, and row skip mode. It will consume very low power and
and faster timing closure. The Quartus II Design Suite with the help of mere two-wire serial interface it can be
is developed by the ALTERA Company. This software is programmed18. The quality of captured pictures by CMOS
being used with different ALTERA family FPGAs/CPLDs is almost achieved CCD picture quality; although assert-
like MAX, Stratix, Apex II, and Flex 10K, Cyclone etc. In ing the built-in size, cost and consolidation advantages of
the present work Quartus II software (Version 13.0) is CMOS19. The user can program the sensor for advanced

Figure 1. Block diagram of FPGA based image acquisition system.

2 Vol 10 (9) | March 2017 | www.indjst.org Indian Journal of Science and Technology
 Naga Raju Boya, Vijay Kumar Jinde, Bala Venkateswarlu Avvaru, Sreelekha Kande and Ramanjappa Thogata

setting, exposure and frame size for different parameters.

13.9 FPS (Frames Per Second) is the default mode out-
puts of an SXGA image. The operating temperature range
of CMOS sensor is from -300C to +700C. It requires low
supply voltage: 2.8V ±0.3V. Its power consumption will
be very low and is ideal for battery operated devices. This
CMOS sensor will produce direct digital output because
its associated circuit is incorporated with ADC. The pho-
tograph of the CMOS sensor with internal ADC circuit is
shown in Figure 2. The construction of CMOS camera is Figure 4. Block diagram of CMOS camera with ADC circuit.
shown in Figure 3. The block diagram of CMOS camera
with ADC circuit is shown in Figure 4.
3.2 SDRAM (Synchronous Dynamic RAM)
IS42S16400
SDRAM is nothing but a DRAM that has a concurrent
interface. Conventionally, for every change in control
inputs DRAM can react as speedily as possible, so that it
has an asynchronous interface. Before reacting to control
inputs, SDRAM can wait for a clock signal and therefore it
is contemporized with the PC’s system bus so that it has a
synchronous interface. In order to pipeline the incoming
instructions, the clock should drive an internal finite state
machine. This permits the chip to have a more complex
pattern of performance than an asynchronous DRAM20.

3.3 FPGA (Field Programmable Gate Array)

The immensely successful low-cost FPGA Cyclone II
broadens the density range to 68,416 LEs and it has 622
Figure 2. CMOS sensor with ADC. usable input/output pins. It is also having 1.1 megabits
of inbuilt memory21. Cyclone II FPGAs are fabricated on
300-mm wafers to assure rapid accessibility and afford-
ability. As Cyclone II devices are using less silicon area,
at a cost of ASICs itself they can endorse composite digi-
tal systems on a single chip22. The affordable price and
optimized characteristic set of Cyclone II FPGAs nomi-
nate them perfect choice for an extensive range of video
processing, automotive supplies, communications, test
& measurement, goods & services, and other commer-
cial applications. The FPGA used in the present work is
Cyclone II EP2C35F672C6.

3.4 Altera DE2 Board

ALTERA DE2 package includes the following impor-
tant features. It consists of an FPGA namely Cyclone
II 2C35F672C6 device with 35,000 LEs, 16MB Serial
Configuration device for AS mode. For programming and
Figure 3. Construction of CMOS camera. user API control it is having in-built USB blaster. It also

Vol 10 (9) | March 2017 | www.indjst.org Indian Journal of Science and Technology 3
 Design and Development of FPGA Based Image Acquisition System

has TV decoder for NTSC/PAL/Multi-format systems, the CMOS sensor is programmed with I2C protocol, so
24-bit CD-quality audio CODEC. It will support two that CMOS settings like resolution, exposure are con-
programming modes namely JTAG and AS. DE2 board figured with this protocol. The serial data is assigned to
also consists VGA DAC with VGA-out connector which the CMOS sensor which is obtained from the available
can support up to 1600x1200 at 100 Hz rate, Ethernet parallel data. The data is ceaselessly captured by the pro-
100/10Mbps, SRAM, 8MB SDRAM, 4MB Flash memory, grammed sensor. CMOS sensor’s I2C interface and Pixel
SD card connector, RS232, 16x2 LCD panel, 4 pushbut- Data interface are connected to the FPGA through GPIO
ton switches, 18 toggle switches etc. The DE2 board also interface of the DE2 Board. FPGA will have I2C Slave
has software endorse for standard input/output ports and controller module that will write the CMOS sensor con-
a control panel feature for accessing several elements. In figurable register values. In order to restrain the stream of
order to supply utmost tractability for the user, all the con- data coming from the sensor which is programmed to the
nections are established through the Cyclone II FPGA23.
memory, the IDFC (Image Data Flow Controller) is used.
The photograph of the Alter DE2 board with CMOS cam-
This IDFC influences the stream of data to the memory.
era is shown in Figure 5.
Whenever the START signal is triggered then only the
data is stored in the memory. It happens continuously
until STOP signal is found. This system consists of the
black and white converter module so that the data is laid-
in in the pattern of black and white images only. ALTERA
DE2 board consists of SDRAM, which can store the cap-
tured image frames. Two frames can lay in the memory
at a time. Therefore, it is split into two and the data is
read from or written into the memory. SDRAM control-
ler is used to supervising these performances. The read
& write signals are rendered by SDRAM controller and it
Figure 5. Altera DE2 board with CMOS camera. can also track which frame is being read or written. The
video can be monitored on a visual display unit such as
3.5 VGA (Video Graphics Array) an LCD monitor. For this purpose, the VGA controller is
used.VGA Controller module in the FPGA always reads
VGA is denoted as an “array” rather than “adapter” since
the pixel data from one of the ports of DDR SDRAM and
it was enforced from the start as a single chip, substitut-
it generates the required signals to display the video on
ing the Motorola 6845 and lots of distinct logic ICs that
the screen.Figure 6 shows real-time video capturing and
occupied the full-length ISA boards of the MDA, CGA,
displaying on the monitor through CMOS camera con-
and EGA. Its single-chip execution also permitted
nected with ALTERA DE2 board.
the VGA to be located instantly on a computer’s CPU
board with a minimum of difficulty (it need only three
things, they are timing crystals, video memory, and an
external RAMDAC).

4. Results and Discussions

In this article, the domain under observation is inces-
santly supervised by capturing the video of the area. With
the help of the CMOS sensor, the capturing of video under
observation is successfully completed. This CMOS sensor
will produce direct digital output because its associated Figure 6. Photograph of the image acquisition system.
circuit is incorporated with ADC. The mode register of

4 Vol 10 (9) | March 2017 | www.indjst.org Indian Journal of Science and Technology
 Naga Raju Boya, Vijay Kumar Jinde, Bala Venkateswarlu Avvaru, Sreelekha Kande and Ramanjappa Thogata

5. Conclusion 9. Valera M, Velastin SA. Intelligent distributed surveillance

systems: a review. In the Proceedings of the Institute of
The CMOS camera interface unit is developed for Electrical and Electronics (IEE) Vision, Image, and Signal
ALTERA Cyclone II EP2C35F672C6 FPGA device which Processing. 2005 Apr;152(2):192–204. Crossref
is presented in the ALTERA DE2 board. This interface 10. Raty TD. Survey on contemporary remote surveillance sys-
unit is having ability to capture update information at tems for public safety. Institute of Electrical and Electronics
the same rate they receive. The design is being oper- Engineers (IEEE) Transactions on Systems, Man, and
Cybernetics. 2010 Sep; 40(5):493–515.
ated with a clock of frequency 27MHz. This design uses
11. Venetianer PL, Deng H. Performance evaluation of an intel-
only 664 out of 33,216 logic elements (2%), 1,128 out of
ligent video surveillance system – a case study. Computer
33,216 combinational functions (3%), 939 out of 33,216 Vision and Image Understanding. 2010 Nov; 114(11):1292–
dedicated logic registers (3%), 247 out of 475 total IO pins 302. Crossref
(52%), 43,545 out of 483,840 memory bits (9%), 0 out of 12. Dee H, Velastin SA. How close are we to solving the
70 embedded multipliers 9-bit elements (0%), 1 out of 4 problem of automated visual surveillance? A review of
PLLs (25%). This synthesis results show that the CMOS real-world surveillance, scientific progress and evaluative
camera interface unit uses only few resources available on mechanisms. Machine Vision and Applications. 2008 Sep;
ALTERA Cyclone II EP2C35F672C6 FPGA. Efforts are in 19(5–6):329–43.
progress to develop FPGA based intruder detection sys- 13. Foresti G, Micheloni C, Snidaro L, Remagnino P, Ellis T.
tem by utilizing remaining resources available on Cyclone Active video-based surveillance system: the low-level
II EP2C35F672C6 FPGA. image and video processing techniques needed for imple-
mentation. Institute of Electrical and Electronics Engineers
(IEEE) Signal Processing Magazine. 2005 Mar; 22(2):25–
6. References 37. Crossref
14. Foresti GL, Mahonen P, Regazzoni CS editors. Multimedia
1. Srinivas K. Implementation of FPGA in the design of
video-based surveillance systems. Kluwer Academic
embedded systems [Electronics and Communication
Publishers, Boston; 2000.
Engineering thesis]. Rourkela, National Institute of
15. Costello CJ, Diehl CP, Banerjee A, Fisher H. Scheduling an
Technology; 2007 May. p. 78.
active camera to observe people. In the Proceedings of the 2nd
2. Monmasson E, Cirstea MN. FPGA design methodology for
industrial control systems—a review. Institute of Electrical Association for Computing Machinery (ACM) International
and Electronics Engineers (IEEE) Transactions on Workshop on Video Surveillance and Sensor Networks, ACM
Industrial Electronics. 2007 Aug; 54(4):1824–42. Crossref Press, New York; 2004 Oct 14. p. 39–45.Crossref
3. Chang TN, Cheng B, Sriwilaijaroen P. Motion control firm- 16. Hampapur L, Brown J, Connell A, Ekin N, Haas M, Lu H,
ware for high-speed robotic systems. Institute of Electrical Merkl S, Pankanti S. Smart video surveillance: exploring
and Electronics Engineers (IEEE) Transactions on the concept of multiscale spatiotemporal tracking. Institute
Industrial Electronics. 2006 Oct; 53(5):1713–22. Crossref of Electrical and Electronics Engineers (IEEE) Signal
4. Ranganathan O, Rasheed AI. Design and development Processing Magazine. 2005 Mar; 22(2):38–51. Crossref
of camera interface controller with video pre-processing 17. Abdelkader MF, Chellappa R, Zheng Q. Integrated motion
modules on FPGA for MAVS. SASTECH Journal. 2013 detection and tracking for visual surveillance‖. In the
Apr; 12(1):9–14. Proceedings of the Fourth Institute of Electrical and
5. Dubey R, Agarwal P, Vasantha MK. Programmable Electronics Engineers (IEEE) International Conference on
logic devices for motion control—A review. Institute of Computer Vision Systems (ICVS); 2006 Jan 4–7. p. 28.
Electrical and Electronics Engineers (IEEE) Transactions 18. Datasheets: 1.3-Megapixel CMOS Active-Pixel Digital
on Industrial Electronics. 2007 Feb; 54(1):559–66. Crossref Image Sensor MT9M413C36STM [Internet]. 2011 [updated
6. Collins R, Lipton A, Fujiyoshi H, Kanade T. Algorithms for 2011 Feb 2; cited 2004]. Available from: Crossref
cooperative multisensor surveillance. In the Proceedings of 19. Pandey JG, Purushottam S, Karmakar A, Shekhar C.
the Institute of Electrical and Electronics Engineers (IEEE). Platform-based extensible hardware-software video
2001 Oct; 89(10):1456–77. Crossref streaming module for a smart camera system. International
7. Zeng H. FPGA based smart NIR camera [Master thesis]. Journal of Modeling and Optimization. 2012 Aug; 2(4):482–
Sweden, Mid Sweden University; 2012. 7. Crossref
8. Kim IS, Choi HS, Moo YK, Young CJ, Kong SG. Intelligent 20. Gabrick M, Nicholson R, Winters F, Young B, Patton J.
visual surveillance — a survey. International Journal of FPGA considerations for automotive applications. SAE
Control, Automation, and Systems. 2010 Oct; 8(5):926–39. Technical Paper; 2006 Apr 3. p. 20. Crossref

Vol 10 (9) | March 2017 | www.indjst.org Indian Journal of Science and Technology 5
 Design and Development of FPGA Based Image Acquisition System

21. ALTERA. Five ways to build flexibility into industrial appli- board. International Journal of Engineering and Advanced
cations with FPGAs. Altera Corporation; 2011. Technology (IJEAT). 2012 Dec; 2(2).
22. Jagdale VB, Vaidya RJ. High definition surveillance system 23. Altera. DE2-70 development and education board user
using motion detection method based on FPGA DE-II 70 manual version 1.08. Terasic Technologies; 2009.

6 Vol 10 (9) | March 2017 | www.indjst.org Indian Journal of Science and Technology