Avalon ST custom IP component

I suspect the capture register of the Signaltap module isn't meeting timing either and as a result you don't capture the data correctly. Think of signaltap as a big FIFO that wires up to your logic so if that doesn't meeting timing then the samples will not be stable. Another thing to take a look at are the Quartus warnings from the compilation, it could be an RTL mistake causing the behavior you are seeing as well such as an unconnected wire or synthesis optimizations. You can have Quartus filter on message types which makes it easier to find those, just look for ones that match up with your IP core name.

Yes you can simulate the entire Qsys system. You can have Qsys output simulation models for everything in your system or generate a testbench for you. When it generates a testbench for you it attempts to hook up bus functional models (BFM) to the ports exposed out of your system such as clocks, resets, Avalon, AMBA, etc... Clocks and resets will have their BFMs automatically driven but if you expose Avalon-ST ports for example you'll have to write code to drive those BFMs.