hd_sdi ip problem

You say that you are connecting the tx port to the rx port, so what are you transmitting? In what mode?

You mention "3FFF" "0000" "0000", but these are the start of sync words, only not quite - in HD you need 20 bits, so "FFFFF" and "00000", and in SD you need "3FF" and "000" (10 bits) and the top 10 bits is ignored.

You need to generate the TRS signals correctly, including line number and CRCs in HD, unless you ask the SDI-TX core to generate them for you, but you have to put in the FFFFF and 00000 words yourself and tell it exactly when they are. And it will only recognise the input if you have the TRS signals at the correct spacing.

I suggest you look at the Stratix II GX video reference design, which has a "Pattern Generator" in it for making something suitable to transmit.