How to simulate serdes in modelsim? Which files must I compile?

If it's the LVDS serdes, I think you just need the "altera_mf.v" libraries

c:\altera\91\quartus\eda\sim_lib

If it's the GX transceivers, you need the library for the corresponding device you are using also located in the above folder.

Jake