What is difference and relation between gigabit transceiver and PCIe?

"Transceivers" are the general term used to describe the I/O cells. These cells contain high-speed shift-registers called serializers and deserializers, blocks for transmitter pre-emphasis and receiver equalization, and all manner of "magic".

PCIe defines a much broader concept; software and hardware. A PCIe interface uses transceivers, but those same transceivers can be used in other applications, eg., 10Gbps Ethernet using XAUI (4 lanes at 3.125Gbps with 8/10B encoding), or 10.3125Gbps SFP+ (1 lane with 64/66B encoding).

If you look in the transceiver handbook for the device you are interested in, you will see transceivers discussed in generic terms, and then later, in specific terms, for example, how they are used when implementing PCIe.

Cheers,

Dave