Buffer Overflow and Underflow in Clocked Video Input/Output

Hi Faisal,

I've had a hell of a time with the VIP suite and sopc builder, designs that worked in 9.0 would not run after I upgraded to 9.1 and the main reason was that I wasn't using TimeQuest to constrain my clocks and important video signals. I'm still learning, and I still have some issues, but I had a similar problem with under/overflows and flickering which were both corrected by timing constraints - have you constrained your design? There are also .sdc files in C:\altera\91sp1\ip\altera\clocked_video_input\lib (and \clocked_video_output) that you should add to the design files dialog in settings too if you have not already done this.

Also, is there a difference in your frame rates on the output and input video?

Hi,

Assuming timing closure is the issue, there is also an .sdc file for the Frame Buffer which might help in your case. You should clean timing errors before you try debugging the issue further.

If you are not using triple buffers (with drop and repeat on) then the input frame rate and the output frame rate must be the same and the PLL that creates the output clock should be driven by your input clock (otherwise you will get glitches when the input and output clocks drift apart from each other).

I believe you have more than enough memory bandwidth but did you do the computation? Is your design working with only one input on? jakobjones posted an excel sheet to do the bandwidth calculation a while ago. You could also try to increase the FIFO depth and burst target parameters of the frame buffers but this is a long shot. Make sure that you did not set the FIFO depth to be equal to the burst target.

Kind regards

--- Quote Start ---

The system for single HD stream consists of CVI, Frame Buffer and CVO block. Three such blocks are cascaded serially.

--- Quote End ---

Are you attempting to do three 1080p frame buffers at the same time? This would require a lot of memory bandwidth.

Assuming standard 1920x1080p60, the total horizontal width is 2200. This means you need to maintain a rate of 1920/2200*148.5=129.6 Megapixels/second in and out of the memory for each frame buffer. If you have three frame buffers, this is a total of 777.6 Megapixels/second.

How many bits per pixel are you doing? How wide is your memory bus? Is anything else using the memory?

Hi all

Thank you very much for your valuable suggestions. I really appreciate your help.

I have attached top level diagram with this message. A picture worths thousand words. Please have a look at attached .JPG file.

bandwidth calculations:

----------------------------------------------------

First about the bandwidth calculation. I have DDR2 memory with 64 bit data line (DQ). Input is 1080p 60fps in RGB 4:4:4 (24 bits or 3 bytes) format. It translates to approximately 148.5 MHz input rate with 24 bit video data.

Since I have three frame buffers in my design, there are total of 6 read and write ports. So

input rate = 148.5MHz * 3 bytes * 6 ports = 2.673 GBytes/sec

There are 64 DDR2 data lines (8 bytes), with data being read/written on both clock edges and assuming 75% availability of DDR2 memory. The DDR2 controller runs at 300 MHz clock. Therefore

memory bandwidth = 300MHz * 8 bytes * 2 * 0.75 = 3.6 GBytes/sec

So I think memory bandwidth is enough to handle three HDMI streams. I could not find the bandwidth calculation sheet. So please send me the link or reply with attachment.

constraints:

------------------------------------------------------

For constraints, I added video_input.sdc and video_output.sdc. I will add video_buffer.sdc and do the synthesis again. For other constraints, I just defined the clocks, derive_pll_clocks command and set a few paths as false paths which I am sure of. I am using Microtronix DDR2 controller which generates its own SDC file which I have included in the design as well.

I haven't set input and output delays on video signals because I am not sure how to calculate them. Please let me know of any other constraints that I should create.

In the top level diagram, "frame_buffer_f0" uses tripple buffering and other frame buffers use double buffering. Should I set them to use tripple buffering as well?

Input and output frame rate are not an issue in my design. I dont want any frame rate adaptation. I tried to use the recovered hdmi clock (hdmi_rx_clk which is approx 148.5MHz) for all three video_input and video_output modules but I did not get any output. So I generated a local 148.5 MHz clock from PLL (hdmi_syn_clk) which is used in video_input and video_output modules as shown in attached diagram.

I hope I did not bored everyone with these details. I'll appreciate your suggestions and help.

Regards

Faisal

The spreadsheet was posted in this thread:

http://www.alteraforum.com/forum/showthread.php?t=6841

What value did you use for the size of the read/write master ports of the frame buffers? With a master data port of 64-bit and R:G:B in parallel, the packing of pixels into memory word is really inefficient with only 2 pixels per 64-bit word (see wasted bits in the spreadsheet).

I am not familiar with the Microtronix memory controller so this could make things worse but if your local Avalon-MM bus is currently 64-bit wide then perhaps you could try increasing it to 128 and either reconfigure the local interface of your controller or see if SOPC Builder can handle the switch fabric? Timing closure could be harder and you could waste logic but this would at least take you down from 25% to 7% waste.

Using double buffers for the second and third paths should be fine in your design. I think they are not even needed in this case so you could consider removing them to test whether memory bandwidth is really the issue.

Hi

The width of read and write masters interfaces in the frame buffer is 128 bits. Yes this wastes some memory but this is not really an issue. Main problem is how to get rid of buffer underflow in video_output_f2 (As shown in the attached) diagram. I previously had buffer overflows in the video_input_f2. Now what I get is buffer underflow in video_output_f2 module. I am not really sure what can be the reason for that.

--- Quote Start ---

Are you attempting to do three 1080p frame buffers at the same time? This would require a lot of memory bandwidth.

Assuming standard 1920x1080p60, the total horizontal width is 2200. This means you need to maintain a rate of 1920/2200*148.5=129.6 Megapixels/second in and out of the memory for each frame buffer. If you have three frame buffers, this is a total of 777.6 Megapixels/second.

How many bits per pixel are you doing? How wide is your memory bus? Is anything else using the memory?

--- Quote End ---

Hi Kevin

Thanks for your answer. I posted a reply showing my bandwidth calculations. Can you please have a look and let me know if I am doing it the right way.

Thanks

Faisal

--- Quote Start ---

input rate = 148.5MHz * 3 bytes * 6 ports = 2.673 GBytes/sec

--- Quote End ---

Due to the horizontal blanking interval, you don't actually need to maintain a rate of 148.5MHz. The average rate for a line is actually 148.5*1920/2200. The average rate for the entire frame is even less due to vertical blanking, but you can't really take advantage of that as the FIFOs are not big enough to average out the demand over an entire frame.

As Vgs was mentioning, there is also some inefficiency in the way pixels get packed into a word. Therefore, simply multiplying the pixel rate by 3 bytes is not accurate.

I think a more accurate calculation would be:

bandwidth = 148.5M * 1920 / 2200 * 16 / 5 * 6 = 2.48 Gbytes/second.

The 16 / 5 factor is assuming 128 bit words. You can fit 5 pixels into each 128 bit (16 byte) word.

So it seems you should have enough bandwidth. Perhaps you need to adjust FIFO sizes and thresholds on the CVI, CVO and framebuffer blocks.

Some other random thoughts:

Do you even need to convert between clocked and avalon streaming video multiple times? Could your custom video IP be adapted to process the avalon stream directly? It would simplify your system a lot if you only had one CVI and one CVO to worry about. Do you even need all the frame buffers? Just based on your block diagram, I see no purpose for the first frame buffer. Unless the custom video IP needs to see the video from different delayed time points, I don't really see what the other frame buffers are for either.

As your video IP is running at 166MHz and your memory at 300MHz, are you doing anything to address potential clock domain crossing issues? The default clock domain crossing logic in SOPC builder can be quite inefficient. You may need to explicitly add a pipeline bridge.

1 - Make sure your timing is properly constrained and that you are meeting timing requirements.

2 - If I understand correctly, you've got a 64-bit DDR2 interface. How wide is the local interface with the Microtronix controller. Make sure all the memory masters on your frame buffers are set to match that width. Also, make sure your burst targets are set to at least 32.

3 - You should get the design working with one video processing path first. Then add to it.

Jake

--- Quote Start ---

Some other random thoughts:

Do you even need to convert between clocked and avalon streaming video multiple times? Could your custom video IP be adapted to process the avalon stream directly? It would simplify your system a lot if you only had one CVI and one CVO to worry about. Do you even need all the frame buffers? Just based on your block diagram, I see no purpose for the first frame buffer. Unless the custom video IP needs to see the video from different delayed time points, I don't really see what the other frame buffers are for either.

As your video IP is running at 166MHz and your memory at 300MHz, are you doing anything to address potential clock domain crossing issues? The default clock domain crossing logic in SOPC builder can be quite inefficient. You may need to explicitly add a pipeline bridge.

--- Quote End ---

Hi Kevin

Thanks for the advice.

You got it right, I am using three frame buffers because the custom video processing block is actually using delayed version the frames. It looks at a particular pixels in the current frame in two previous frames to find out the correlation between the time delayed values of the pixels at the same positions in previous two frames. That is why I am using three frame buffers.

For the clock domain crossing, I inserted pipeline bridges in the previous version but I still had some problems. I will put them back in and see what happens now. As I told that there are six read and write ports in three frame buffers, should I put pipeline bridge for all these six ports? I was assuming that SOPC will automatically see the CDC and put the bridges but I think its much better that I do it myself.

About the buffer sizes in CVI and CVO modules, I am using 4096 pixel buffers each. I get underflow only from the video_out_f2 (the last module) so I increased its FIFO size to 4096 pixels which should be good enough for at least two lines of 1080p video.

Regards

Faisal