--- Quote Start ---
I need to guarantee synchronous conrol (with minimal jitter) of up to 10 devces working together.
--- Quote End ---
So route a reference clock to all 10 devices. It possible to design systems with thousands of FPGAs synchronized. Here's some slides ...
https://www.ovro.caltech.edu/~dwh/correlator/pdf/hawkins_jpl_2014.pdf Slide 16; the system on the left has 600+ FPGAs, the one on the right 1000+. They're all synchronous.
It really comes down to how your system needs to be interconnected. These boards use an external reference clock, and then phase-locked loops to keep all the clocks coherent. The ADCs in the system operate at 1GHz sample rate, sampling signals from 500MHz to 1GHz, and have an ENOB close to optimal for the 8-bit ADCs, so the jitter is sufficiently small (sub-picosecond).
The black cables in the photos carry LVDS data at 125MHz (source synchronous, i.e., a clock and data is carried on the cables, FIFOs are used to cross the phase-locked, but phase offset, clock domains). The data traffic over all the cables is equivalent to 250 million phone calls (the data is noise and is cross-correlated in real time, the output rate is much much lower than the input/sampled data rate).
--- Quote Start ---
To do that we transfer 10-12 control signals at 125 MHz and only 2 of them are used for packet transfering (we have no problems here, just the speed of the "data" channels is slower). We use the tranciever 16-bit data exchange and split it to 16 parallel channels. If we have jitter on a signle receiver, than by the end point of the net we will have much bigger jitter. So, we can't guarantee the synchronous conrol of the devices without solving the problem of jitter.
--- Quote End ---
Generally when you have a synchronous system involving transceivers, when you generate data, eg., at an ADC, you generate a packet with some form of header, eg., a timestamp created from say a 1pps and a count of 125MHz reference clock periods. When that packet travels over the network and arrives at some other FPGA along with packets that have traveled a different route, you buffer the data, read the timestamps in the headers, align the packets (eg., wait until timestamps match, or use the timestamp to load a digital delay line or digital filter).
Try and describe what your system is doing a bit more. If you don't want to describe it on the list, just email me (my forum user name is an email address).
Cheers,
Dave
Honored Contributor
