Same file name with different content of two custom IP core. is it possible to compile it without error? How to separate compilation library for the files having the same name in QSYS?

You have to name the IP differently.

If the Quartus® Prime project contains more than one IP of the same file name, Qsys Pro retains the first instance and removes all other occurrences of the IP file with the specific name.

Reference: https://www.intel.com/content/www/us/en/programmable/quartushelp/17.0/system/qsys/qsys_com_sync_ip_files.htm