Real Time Video Implementation on FPGA

Abstract

Nowadays, real time video becomes popular in a lot of multimedia equipment, video cameras, tablets, camcorders. The more hardware improved the more application is used. Requesting a faster and cost-effective systems there are triggers a shift to Field Programmable Gate Arrays (FPGAs), where the inherent parallelism results in better performance. The implementation is based on efficient utilization of embedded multipliers and look up table (LUT) of target device to improve speed but also saves the general purpose resources of the target device. This paper proposes new hardware architecture for capture NTSC/PAL video stream. The whole system is implemented on a single low cost FPAG chip, capable of real time procession at frequency 60 MHz. In addition, to increase real-time performance, hardware architecture with streamlined data flow are developed.

Introduction

In recent years, automated video surveillance system is developing and applying massive. That is enable when the progress in technology scaling more robust computationally intensive algorithms. The advantage of surveillance automation over traditional television based on system lies in the fact that it is a self contained system capable of automatic video processing. That is a request for a real-time video system. The implementation system on FPGA gets more advantages than the other hardware. The FPGA and SOC products improved fast. The traditional hardware implementation of image processing uses Digital Signal Processors (DSPs) or Application Specific Integrated Circuits (ASICs). An advanced system can process video stream real time, process image from video frames in time. As a logic capacity of FPGAs increases, they are being increasing used to implement large arithmetic-intensive applications. Since data-path circuits are designed to process multiple-bit-wide data, FPGAs implementing these circuits often have to transport a large amount of multiple-bit-wide signals form one computing element (such as a logic block, a DSP block, or a multi addressable memory cell) to another. With the advent of FPGAs having greater processing capability, it has been regarded as a useful means for implementing algorithms that massive parallelism is required.

In [9], the authors designed system based on Virtex-4 XC4VLX200-10 from Xilinx and two VCC-8350CL cameras there is more complicated in procession data. There also increase computationally in program. In [2], the system also implemented on Xilinx, Virtex-II pro vp30 FPGA. The camera in [2] is Kodak Kac-9648. In this case, the system has to reduce memory usage but they get more noise in results.

In this paper, we present hardware architecture capable of real time video processing with a screen resolution 30 frames per second. The paper is organized as follows. Section 2 discusses the proposed implementation hardware and results. Finally, conclusions are covered in Sect. 3.

Implementation FPGA

The Observation System

We use the system included: camera ST-400CD, kit Altera DE2_115, LCD Monitor (Flatron Wide). There are some detailed descriptions of camera ST-400CD: Standard Camera Sonny 1/3” Super HAD Color CCD Digital Signal Processing, Video Auto Iris Lens.

The board we use in this paper is Altera Kit DE_115 Development. It was implemented with a Altera FPGA chip, EP4CE115F29C7 (Cyclone IV E), that has 11,4480 LEs, 432 M9 K memory blocks, 3,888 embedded memory (Kbits), 4 PLLs, 528 maximum user I/Os, 230 maximum differential channels. The following hardware is provided on the DE2-115 board: Two 64 MB SDRAM, 2 MB SRAM, 8 MB, flash memory, VGA DAC with VGA out connector. The main improvement of our work include: the development of hardware and software components for a flexible, powerful and low-cost video processing engine, and the use of techniques such as run-time reconfiguration. The implementation has been placed and routed using Quartus II v.11.1. The operating frequency of the design on DE2-115 is 50 MHz. Video stream is captured by camera. Video signal transmits to FPGA. Processing video stream on board FPGA, the output signal sends to VGA port. LCD Monitor gets video signal and capture it out screen. There is low cost and high performance system (Fig. 1).

Fig. 1

Real-time video system

Real-time image processing requires high computation power. For example, the NTSC video standard requires 30 frames second, with approximately 0.25 mega pixel per second. PAL video standard has a similar processing load with 25 frames per second, but the frame size is larger. The amount of processing required per pixel depends on the image processing algorithm. In our proposed hardware architecture, we design data flow from captured signal by camera to VGA output signal. In Fig. 2, there is data processing stream from TV decoder chip to VGA chip.

Fig. 2

Data flow in and out FPGA

Block Processing

• I2C Block (I2C_AV Config): I2C block uses to connect FPGA and TV Decoder chip. Two signals I2C_SClk, I2C_SDATA connect to SCLK pin and SDA pin of Decoder chip. Data signal transmit in 3 bytes per frame.

• Locked dectector block: This block check input conditions from VS and HS to know how the chip worked. When chip works, the TD_Stable is high to make Reset_Delay block work.

• ITU-R656 Block: ITU-R BT 656 describes a simple digital video protocol for streaming uncompressed PAL or NTSC Standard Definition TV (525 or 625 lines) signals. The protocol builds upon the 4:2:2 digital video encoding parameters defined in ITU-R Recommendation BT.601, which provides interlaced video data, streaming each filed separately, and uses the YCbCr Color space and a 13.5 MHz sampling frequency for pixels. The standard can be implemented to transmit either 8-bit values (the standard in consumer electronics) or 10-bit values (sometimes used in studio environments).

• SDRAM Frame buffer, MUX, VGA Controller and Control: We used the SDRAM Frame Buffer and a field selection multiplexer (MUX) which is controlled the VGA controller to perform the de-interlacing operation. Internally, the VGA Controller generates data request and odd/even selection signals to the SDRAM Frame Buffer and filed selection multiplexer (MUX).

• YUV 4:2:2 To YUV 4:4:4. The YUV422 to YUV444 block converts the selected YcrCb 4:2:2 (YUV 4:2:2) video data to the YcrCb 4:4:4 (YUV 4:4:4) video data format (Figs. 3, 4, 5).

  Fig. 3

  

  I2C config block connection

  Fig. 4

  

  Locked detector block connection

  Fig. 5

  

  ITU-R656 decoder block connection

Results

From our architecture, we do some experiments to get results. There are some data about our implementation hardware on Cyclone IV E in Table 1.

Table 1 Usage percents on Cyclone IV E

The time download program from PC to board about 20 s. The video displays on LCD response real time when we change the screen. In Fig. 6, when we connect LCD and DE2-115 board, the screen on LCD displays the image about DE2-115. That is original program on DE2-115. In Figs. 6 and 7, when we load the program to FPGA, the camera will capture the environment image, and video will display on LCD.

Fig. 6

Before download program on FPGA

Fig. 7

After download program on FPGA

Conclusions

In this paper, we proposed new implementation hardware to get real video from camera by FPGA implementation. We get real video stream from camera that is very important detail. We apply some image processing application on real time in future.

We use DE2_115 development kit for our study. This board gives more advance condition with video port, VGA port especial consumer chip Cyclone IV E.