Greetings! I am making this project using the DE2 development board (with a CMOS Camera and LCD display attached) which has the following objectives: 1. Capture an image 2. Display the Image on the LCD 3. Store the Image in an SD card 4. Convert the image into a JPEG format. 5. Have a user interface to control these functions. 6. The user interface should be implemented on top of a controller 7. The controller should load the JPEG algorithm bit stream into the FPGA to configure it at runtime from a memory (like the SD card). *very important My project also should have HDL codes to interface to an embedded RAM (where the Image captured will be stored temporarily), CMOS camera, and SD card. Can the embedded RAM, the DE2 board has, generally hold the RAW image from the camera? In reconfiguring the Cyclone II, does the whole chip need to be altered? or can only a portion of it alloted for JPEG compression be changed? I ask this because I plan to implement the controller in the FPGA chip instead of the NIOS II embedded controller. Some of my mentors in school pointed out that the FPGA chip (Cyclone II) might run out of space if I implement the controller and the image processing there. Thank you for your time, some advise, insights, referrals are highly appreciated:)

It looks like your projects needs a lot of software work. The only hardware seems to be the JPEG compression. If you don't have strong real time requirements this also can be done in software. Probably the NIOS allows to better implement all this controller stuff. Hope this helps

--- Quote Start --- 1. Capture an image 2. Display the Image on the LCD 3. Store the Image in an SD card 4. Convert the image into a JPEG format. 5. Have a user interface to control these functions. 6. The user interface should be implemented on top of a controller 7. The controller should load the JPEG algorithm bit stream into the FPGA to configure it at runtime from a memory (like the SD card). --- Quote End --- 1. I don't have a DE2 board with me, so I don't know exactly how to do this. But, you can take a look at the CMOS imager specs and write your HDL based on it. It's basically just handshaking between the FPGA and this CMOS imager, and getting all the data correctly. 2/3. Again, take a look at the LCD's specs. For the SD card, you need to implement the SDIO interface in the FPGA. Think there are a few free implementations on this out there... 4. Haven't done JPEG, sorry... 5. Not sure if the LCD panel is also a touchscreen? If so, you can write HDL that renders the UI graphics to the touchscreen. Just superimpose the UI on top of the image that's currently on the screen. Or, you can play around with alpha-blending. 6. What is this controller for? I am guessing that this controller receives user inputs and processes those inputs. If so, for a simple touchscreen controller, just detect the coordinates of a user "touch". Be sure to add at least a few adjacent pixels to be certain it is the user who "touched" the screen, rather than some bug or dust. 7. I am thinking that you should put the JPEG compression algorithm in the HDL, which means it becomes active as soon as your machine becomes active. But if you really need to load this algorithm everytime you do a JPEG compression, then you can write your own scan chain (see below). --- Quote Start --- In reconfiguring the Cyclone II, does the whole chip need to be altered? or can only a portion of it alloted for JPEG compression be changed? I ask this because I plan to implement the controller in the FPGA chip instead of the NIOS II embedded controller. --- Quote End --- No, you don't need to reconfigure the whole chip. You can try the "partial reconfiguration" feature released by Altera a while ago, though I've never tried it yet. Or, what I'd do is write my own scan chain and have it hooked to an interface to receive data. The interface can be USB/Ethernet/Wi-Fi or whatever. For example, say you've made changes to your JPEG compression algorithm and you want your FPGA to reload the new algorithm, then you can scan in the new algorithm into your FPGA using the said interface + scan chain. The scan chain could be just some shift registers, or controls to internal/external memory. What the scan chain does is just receive data from the interface, and stores the data accordingly (usually in shift registers, but can be other internal/external memory as well). This would update your old algorithm with the new data given by the user. You could also implement your own protocols for storage/memory control or error detection in between the interface controller and the scan chain (if you wish).

--- Quote Start --- What is this controller for? I am guessing that this controller receives user inputs and processes those inputs. --- Quote End --- The controller would be for: 1. Yes, handling commands such as "capture image", "store to sd card", "display on lcd", compress to <compression type>". For the compression type I would implement JPEG but the project leaves room for future developments of other compression types. 2. It would also serve as the configuring device during run time. The idea is when the "compress <>" option is chosen, the corresponding synthesized HDL code of the compression would be loaded to the portion of the FPGA to be reconfigured. The reconfiguring bit stream (compression HDL code) would come from a memory (SD Card for example) and loaded by the controller (NIOS or FPGA implementation) to the FPGA compression portion. --- Quote Start --- 7. I am thinking that you should put the JPEG compression algorithm in the HDL, which means it becomes active as soon as your machine becomes active. --- Quote End --- I will implement the compression in Verilog but I need the capability to load it at run time so that future compression algorithms could be added during run time (as users need them). --- Quote Start --- But if you really need to load this algorithm everytime you do a JPEG compression, then you can write your own scan chain (see below). Or, what I'd do is write my own scan chain and have it hooked to an interface to receive data. you can scan in the new algorithm into your FPGA using the said interface + scan chain. The scan chain could be just some shift registers, or controls to internal/external memory. --- Quote End --- From what I understand about the scan chain, it looks like the concept of using a controller to load the compression algorithm bit stream to reconfigure the fpga. Am I correct?

--- Quote Start --- The controller would be for: 1. Yes, handling commands such as "capture image", "store to sd card", "display on lcd", compress to <compression type>". For the compression type I would implement JPEG but the project leaves room for future developments of other compression types. 2. It would also serve as the configuring device during run time. The idea is when the "compress <>" option is chosen, the corresponding synthesized HDL code of the compression would be loaded to the portion of the FPGA to be reconfigured. The reconfiguring bit stream (compression HDL code) would come from a memory (SD Card for example) and loaded by the controller (NIOS or FPGA implementation) to the FPGA compression portion. --- Quote End --- okay, my guess was something like that too... :) You can write a controller in HDL to do those things. --- Quote Start --- I will implement the compression in Verilog but I need the capability to load it at run time so that future compression algorithms could be added during run time (as users need them). --- Quote End --- you very well need the shift register / scan chain logic then... --- Quote Start --- From what I understand about the scan chain, it looks like the concept of using a controller to load the compression algorithm bit stream to reconfigure the fpga. Am I correct? --- Quote End --- Yes, you're right. It probably would be just some shift registers, or some kind of matrix memory controlled by some logic. You might even like to write your HDL such that it uses the internal RAM (such as M4K, etc.) of the device. Well, you can write your own flavour using your own taste... :)

--- Quote Start --- okay, my guess was something like that too... :) You can write a controller in HDL to do those things. --- Quote End --- So I can implement the controller in a portion of the FPGA chip OR I can use the NIOS right? If I go for the first one then I would have to retain the circuitry of the controller while reconfiguring the other portion for the image processing. If I go for the NIOS then I would have to implement an interface to the FGPA. Which do you suggest?:) --- Quote Start --- you very well need the shift register / scan chain logic then... Yes, you're right. It probably would be just some shift registers, or some kind of matrix memory controlled by some logic. You might even like to write your HDL such that it uses the internal RAM (such as M4K, etc.) of the device. Well, you can write your own flavour using your own taste... :) --- Quote End --- Either choice of controller implementation I would have to use scan chain logic right? What would the memory be used for? I get the concept of scanning in bit stream to configure but I don't get why use a memory/shift register/RAM. Can't I use the controller to read the configuration bits from an external memory like an SD card and just load it to the FPGA? Is the memory you're referring to a memory that is holding the reconfiguration bits of the compression algorithm? Oh yeah! regarding the reconfiguration of the FGPA, my HDL code would be compiled into a bit stream that would configure the chip right? I am in the assumption that I would always need the Quartus software to configure the chip. If that's the case then how can the chip be changed without the software?:)

Image Processing Project on the DE2 board

16 Replies

Altera_Forum
Honored Contributor
15 years ago
--- Quote Start ---
The memory can be for image data, or also your compression algorithm. I'm not sure how you are going to load the algorithm into the FPGA as a bitstream - is it going to be the whole algorithm, or just parameters to the algorithm?
--- Quote End ---

The idea is that I would have a library of compression or image processing algorithms in the SD card then as I choose an algorithm to implement, it will get loaded to the FPGA so that the FPGA (or at least a portion of it) would perform the desired algorithm. So to answer the question, the whole algorithm would be loaded essentially. In this way, do the FGPA still have to access the SD card over and over again? Or can it be just one time loading?

I understand why you say that its better to store it in a RAM first, but if it will be a one time loading to the fpga, can this be ommited for the compression algorithm?

--- Quote Start ---
I would actually write the algorithm in HDL (means this is part of your SOF), and only load the parameters as a bitstream. However, even with this I would load the bitstream into a RAM (best if internal RAM), so the algorithm doesn't need to access the external SD card all the time at every loop. Imagine, if your algorithm is running, and you remove the SD card, your whole system is going to fail. So, I'd always load the bitstream to (internal or external) RAM.
--- Quote End ---

I understand why it should be stored in a RAM (except the parameters part), however, if that's the case then is it not true that the FPGA just have to be changed then it doesn't need to access the algorithm bit stream used to configure it?

--- Quote Start ---
You're almost there. This process will start after you have already configured your FPGA using Quartus. The scan chain bitstream scanning will happen in (b), after your FPGA is already configured by Quartus.
--- Quote End ---

Yes it will start after the first configuration by quartus, but after that the FPGA must be reconfigured without quartus. That's why I didn't understand why you said the scan chain bitstream (compression algorithm) will happen after the FPGA is already configured. What if I change the current algorithm to a bmp compression without the use of quatrus? That's when the scan chain comes in right? The new algorithm is scanned and the controller reconfigures the FPGA.

--- Quote Start ---
(c) is also related to (b), where after scanning in the bitstream, your FPGA automatically already gets configured with the new compression algorithm.
--- Quote End ---

This should happen without the use of quartus.

Then why in the quote below, reconfiguring the FPGA using Quartus? Then the reconfiguration won't be implemented in real time? Furthermore, if the reconfiguration is done by quartus, what is the "another controller to scan in" for?

--- Quote Start ---
Reconfigure the FPGA using the normal way, Quartus programmer via USB JTAG. Your design should contain another controller to scan in the bitstream via another interface (maybe another USB port, or others).
--- Quote End ---
Altera_Forum
Honored Contributor
15 years ago
--- Quote Start ---
The idea is that I would have a library of compression or image processing algorithms in the SD card then as I choose an algorithm to implement, it will get loaded to the FPGA so that the FPGA (or at least a portion of it) would perform the desired algorithm. So to answer the question, the whole algorithm would be loaded essentially. In this way, do the FGPA still have to access the SD card over and over again? Or can it be just one time loading?

I understand why you say that its better to store it in a RAM first, but if it will be a one time loading to the fpga, can this be ommited for the compression algorithm?

--- Quote End ---

Yes, you can do that, provided the FPGA doesn't keep accessing the SD card for any other info after the loading is completed, unless of course it needs to perform another loading.

--- Quote Start ---

Yes it will start after the first configuration by quartus, but after that the FPGA must be reconfigured without quartus. That's why I didn't understand why you said the scan chain bitstream (compression algorithm) will happen after the FPGA is already configured. What if I change the current algorithm to a bmp compression without the use of quatrus? That's when the scan chain comes in right? The new algorithm is scanned and the controller reconfigures the FPGA.

--- Quote End ---

Absolutely right! If you want to change the current algorithm to bmp for example, you just need to scan in the new algorithm bitstream using that interface, and you don't need quartus. Actually, you can even disconnect Quartus totally from your board (plug out USB-Blaster or disconnect the cable) after you've configured your FPGA the first time.

--- Quote Start ---

This should happen without the use of quartus.

--- Quote End ---

Absolutely.
Altera_Forum
Honored Contributor
15 years ago
Alright! Let me see if I get this correctly:)

I attached a block diagram I made to represent the system I want to implement.

The solid lines represent the control signals used for interfacing the devices and the dashed lines are the data paths. The red line would be the data path of the configuration bits (HDL algorithms such as the JPEG compression) used to configure the FPGA portion for the data processing block. The black one are the buses for the image data being manipulated.

For the red line, the scan chain I imagined is like that, that the configuration bit stream would be scanned and loaded by the controller into the FPGA. Can I do this as direct from the SD Card into the FPGA processing block with the controller just commanding the devices? OR am I missing something in this part? (I didn't include the RAM I can use to store the configuration bits anymore)

For the controller block, I divided it into two parts, the interface to the other devices and the scan-chain part. Is this logical?

Furthermore, You mentioned that I have to implement a controller for the reconfiguration when I unplugged the Quartus and the JTAG. Is the controller in the diagram one in the same?
Altera_Forum
Honored Contributor
15 years ago
--- Quote Start ---
Furthermore, You mentioned that I have to implement a controller for the reconfiguration when I unplugged the Quartus and the JTAG. Is the controller in the diagram one in the same?
--- Quote End ---

What I meant was in your design, you already included the controller to do the bitstream scanning (or let's say it JPEG algorithm reconfiguration). Your whole design (which includes this controller), is loaded once into the FPGA by Quartus, after which you can disconnect Quartus from your board. And your JPEG Algorithm Reconfiguration controller should work (with Quartus disconnected) by scanning in a new algorithm from PC to FPGA via some pre-defined interface. So yes, this controller is already included in your block diagram, named as "Controller (HDL or NIOS)". It has always been the same controller we were talking about. :)
Altera_Forum
Honored Contributor
15 years ago
--- Quote Start ---
And your JPEG Algorithm Reconfiguration controller should work (with Quartus disconnected) by scanning in a new algorithm from PC to FPGA via some pre-defined interface.
--- Quote End ---

And from SD Card to FPGA via the interface shown in the diagram can be done as well right?:)

So is the diagram I drew good enough? Does it miss something or is anything wrong with it?
Altera_Forum
Honored Contributor
15 years ago
Thank you very much for your help :)

Forum Discussion

Image Processing Project on the DE2 board

16 Replies

Recent Discussions

AGILEX 5 cvp mode

Arrow AXE5 Eagle Board JTAG issue

PDN file and PCB decoupling

PCIe Example Design for Arrow EAGLE Board

CXL 2.0 support on the NEW Agilex™ 7 FPGA I-Series Development Kit (2x R-Tile and 1x F-Tile)