Forum Discussion

Altera_Forum's avatar
Altera_Forum
Icon for Honored Contributor rankHonored Contributor
16 years ago

Help needed: clock video input issues early endofpacket

Hi,

We have a design to implement which uses the Altera VIP suit IPs and on quartus 8.0 , on cyclone 3c120 FPGA.

the video pipeline is as follows.

clock video input --> color plane sequencer --> chroma resampler --> color space converter --> (24bit RGB)deinterlacer (as of now BOB, no buffering) --> scaler --> frame buffer --> custom component.

- the input is PAL and frame buffer needs to buffer 640x480 resolution rgb frame.

- the frame buffer has both frame dropping and frame repeating enabled.

- the custom component will read from the frame buffer.

I am using pci board from terasic which has a 64-bit sodimm ddr2 memory.

Currently framebuffer is the only master accessing DDR2 and no other masters are connected to the memory.

I am using DDR2-HP in half rate mode - f = 167MHz

Hence frame buffer read/write master width = 64*4 = 256 bits

frame buffer read/write master clk = 167/2 = 83.5 MHz

If i understand it correctly, even if the frame buffer is not read or not read at rate fast enough, the frame buffer writer component should write the incoming video at rate fast enough and if the written frame is not read by the reader component, it should drop the current frame and write next arriving frame. Basically frame buffer writer component should not depend on the reader component's rate of reading, when frame-dropping and frame-repeating r enabled.

now the problem is, the clocked video input issues an early endofpacket signal. initially for few frames, the endofpacket is generated at the end of full frame, i.e. total pixels = 720 x 288 .. But after a few frames, the clock video input IP starts issuing early end of packet .. somewhere around 9 or 10 pixel lines(720 pixels per line)..

According to the vip user guide , the clock video input will issue early endofpacket only when the downstream IPs don't accept the data at fast enough rate.

Is this the only possible cause of issue of early end ofpacket. ? also, i have tied the vid_locked signal to 1, so there is no early EOP because of loss of vid_locked.

According to the bandwidth calculations - the ddr2 memory should be able to easily meet the requirements. However as observed, even with just writes, I get early frame termination.

I have implemented more complex video pipelines with upto 8 masters accessing the ddr2 on the 3c120 development board.

However i am stuck at resolving this issue.

Is it really the memory problem or something else? The ddr2 controller timing parameters are pretty much generic.

I would really appreciate any help in debbuging this problem.

Thanks

Foram

5 Replies

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    16 years ago

    Hi,

    I believe you may be having an issue because of the deinterlacer. When using bob without buffering, the input rate of the deinterlacer is halved because there are two lines going out for each input lines going in. In practice, this means that the deinterlacer applies back pressure for one line after each line received.

    If the output rate of the clocked video input (number of valid cycles assuming ready remains high) is under 50% then I think you will be able to solve the problem by increasing the buffering in the clocked video input. Otherwise you may be able to increase the clock frequency of your datapath to go below this 50% limit. You may also want to try using bob with double or triple buffering and perhaps even remove the frame buffer in the later case.
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    16 years ago

    You get the early endofpacket when the FIFO in the clocked video input overflows. This is obviously caused by the downstream components not pulling the data out fast enough. You need to calculate how fast to run the video IP to allow the deinterlacer to perform it's line duplication without overflowing the upstream buffer in the clocked video input.

    Jake
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    16 years ago

    Thanks Jake and Vgs for the reply.

    The clocked video input FIFO is set large enough.

    And before trying with no buffering BOB, i was using deinterlacing with triple buffering as in my actual design, i need motion adaptive deinterlacing.

    And thus, with buffering in deinterlacer as well, the same problem persists.

    The board which we are using, doesn't have any reference design for video and this is the first time we have to implement a video pipeline on this board.

    BTW, the clock frequency for the altera VIP IPs is 100 Mhz and DDR2 is configured on half-rate mode. and the frame buffer / deinterlacer masters access the memory at 83.5 MHZ as the DDR2 is run at 167 Mhz, which works fine on the other development boards i have worked on.

    and what i want to know is, if all the following combination doesn't work on this board, how can i find out what could be the actual problem, in the sense how do i debug..

    1. Clock video Input --> Color plane sequencer --> chroma re-sampler --> color space converter --> de-interlacer (triple buffering) --> scaler --> custom component

    2. Clock video Input --> Color plane sequencer --> chroma re-sampler --> color space converter --> de-interlacer (no buffering) --> scaler --> frame buffer (triple buffering) --> custom component

    and there r few more such combinations, which doesnt work and the problem remains there.

    I would need triple buffering in the pipeline as the output frame rate is not same as input frame rate.

    Also, i can get a few full frames, lets say for around a minute, i can see the full frame video but slowly the frame becomes smaller and smaller with early EOP shifting its place, with lesser and lesser pixels appear in the output.

    And i suspect it is not a video pipeline design / clock problem. But i suspect the memory has some issue but i don't know how to debug and catch the problem.

    any suggestions about this would be of great help.

    Thanks.
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    16 years ago

    Considering you have quite a lot of experience building such systems it might not be very useful information but here it goes as this may be useful for others.

    The first step in debugging such a system would be to tap the Avalon-ST connections between the cores. With a trigger on sop high, valid high and data = 0x0F you should be able to catch and analyse the control packets flowing through.

    You have to bear in mind that the last control packet is the one that matters so placing the trigger on the start of the image packet (sop high, valid high and data = 0x00) might give you a better view of what could be happening. AFAIK, both the deinterlacer and the scaler should be adding an extra control packet into the stream so you should expect two control packets after the deinterlacer and three control packets after the scaler.

    In your case, you probably want to know what is coming out of the Clocked Video Input so this could be a first tap point (you may also want to enable the control port to query the core). It might also be worth checking what is happening when eop is sent. It might be difficult to see whether this is an early eop with Signal Tap but you should be able to see how the ready line is behaving at that time.

    You may also tap the control packets coming in and out of the scaler, frame buffer and deinterlacer and check in the manual what is the behaviour of these cores when they receive an early eop. For instance the deinterlacer WITH triple buffering will always pad an image packet to the correct length when an early eop is received. So you should not be seeing an early eop at the end of datapath (1.) unless the control packet received was incorrect.

    If you strongly suspect there is something wrong with the memory then you could focus both on the Avalon-MM interfaces and on the packets sent by the deinterlacer and/or the frame buffer. Control packets are stored and retrieved from the memory and the behaviour of the system could become quite unpredictable if these are altered, especially if their type switch to the type of an image packet.

    vgs
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    16 years ago

    --- Quote Start ---

    Considering you have quite a lot of experience building such systems it might not be very useful information but here it goes as this may be useful for others.

    The first step in debugging such a system would be to tap the Avalon-ST connections between the cores. With a trigger on sop high, valid high and data = 0x0F you should be able to catch and analyse the control packets flowing through.

    You have to bear in mind that the last control packet is the one that matters so placing the trigger on the start of the image packet (sop high, valid high and data = 0x00) might give you a better view of what could be happening. AFAIK, both the deinterlacer and the scaler should be adding an extra control packet into the stream so you should expect two control packets after the deinterlacer and three control packets after the scaler.

    In your case, you probably want to know what is coming out of the Clocked Video Input so this could be a first tap point (you may also want to enable the control port to query the core). It might also be worth checking what is happening when eop is sent. It might be difficult to see whether this is an early eop with Signal Tap but you should be able to see how the ready line is behaving at that time.

    You may also tap the control packets coming in and out of the scaler, frame buffer and deinterlacer and check in the manual what is the behaviour of these cores when they receive an early eop. For instance the deinterlacer WITH triple buffering will always pad an image packet to the correct length when an early eop is received. So you should not be seeing an early eop at the end of datapath (1.) unless the control packet received was incorrect.

    If you strongly suspect there is something wrong with the memory then you could focus both on the Avalon-MM interfaces and on the packets sent by the deinterlacer and/or the frame buffer. Control packets are stored and retrieved from the memory and the behaviour of the system could become quite unpredictable if these are altered, especially if their type switch to the type of an image packet.

    vgs

    --- Quote End ---

    Thanks.

    I suppose these debug techniques would be useful for others. But unfortunately not to me in this case... :(

    anyways, thanks for the reply.