Forum Discussion
Occasional ContributorCyclone V GT Dev Kit - new units fail to boot from FPP
An update/fix if someone else encounters the same issue.
GordWait an I kept investigating the problem and sharing our findings daily, until we got all our boards working 100% of the time on hundreds of reboots. While we ended up with slightly different solutions the root cause is the same, and I will share the core of what helped me for future reference.The root of the problem is presumably some signal integrity/timing issue across the fpga_config_d/fpga_dclk lines, which seems to be present only on newer boards/chips.
The greatest improvement came from reducing the drive strength of these lines, by adding in the QSF:
set_instance_assignment -name CURRENT_STRENGTH_NEW 7MA -to fpga_dclk
set_instance_assignment -name CURRENT_STRENGTH_NEW 7MA -to fpga_config_d[*]While this solution helped on most of the affected boards it did not fix them all.
To better improve the situation the fpga_dclk can be effectively reduced to 50 MHz, with the easiest way being changing the pfl_x8_inst's "Ratio between input clock and DCLK output" to 2. By doing this the fpga_config_d changes on the fpga_dclk's falling edge, thus improving the hold slack.
To better suit this fix there are some constraints to add in the .sdc.
- fpga_dclk should be a generated one:
create_generated_clock -name {fpga_dclk} -source {clk_config} -divide_by 2 [get_ports {fpga_dclk}]- The data output delay should be based on this clock and respect the Cyclone V datasheet tsu:
set_output_delay -add_delay -max -clock [get_clocks {fpga_dclk}] 5.500 [get_ports {fpga_config_d[*]}]
set_output_delay -add_delay -min -clock [get_clocks {fpga_dclk}] -1.000 [get_ports {fpga_config_d[*]}]- If using the "Ratio" option a multicycle path should actually be defined:
set_multicycle_path -setup -start 1 -from [get_registers {*|alt_pfl_cfg_fpga:*|fpga_data_reg[*]}] -to [get_ports {fpga_config_d[*]}]
set_multicycle_path -hold -start 1 -from [get_registers {*|alt_pfl_cfg_fpga:*|fpga_data_reg[*]}] -to [get_ports {fpga_config_d[*]}]Note that these fixes don't consider the usage of the FPPovJ, for which better constrains should be designed.
Moreover, the FPP boot time is largely limited by the flash access time, so even halving the DCLK won't affect the boot time much. Even with the 50 MHz clock it still boots comfortably within our system's PCIE enumeration window.All of this was tested on a limited number of boards, so I can't assure this works 100% of the time, but until Altera finds a proper solution this can be a good starting point for anyone facing the same problem as we did.
Leonardo
Regular ContributorHi Leonardo,
Thank you for providing the detailed results.
We have identified a related case that is currently under internal review, and we are assessing whether these boards have the same failure mechanism. Since both boards fail with the factory image, AS operation and flash read/write remain functional, and timing changes did not provide a reliable recovery, the symptoms indicate a possible board-level issue or insufficient timing margin in the FPP configuration path.
At this stage, further changes to the MAX V constraints are unlikely to provide a reliable solution. If test equipment is available, it would be helpful to compare a working and affected board by capturing DCLK, nCONFIG, nSTATUS, CONF_DONE, and several FPP data signals during boot, while also checking the configuration power rails and reset sequencing. Please share the waveforms through the support case if possible. However, this debug should not delay warranty verification and RMA or repair evaluation for S/N 5CPCIE00100214 and 5CPCIE00100263.
To proceed with the RMA, please contact the distributor from whom the boards were purchased. The distributor can verify the warranty and submit an Altera Premier Support (APS) RMA case on your behalf.
We apologize for the inconvenience and appreciate your cooperation.
Regards,
Fakhrul
LeonardoGHi Fakhrul,
I have some new information which might be helpful.
I was probing the signals you asked about with an oscilloscope. As I previously observed on the affected boards, nCONFIG and nSTATUS pulse many times over the 6 seconds before timing out. The pulses are also random between power cycles, so the issue is not deterministic. The FPP data and clock look reasonable and consistent with this behavior, with no surprises.
On the other hand, I observed an interesting behavior on the reference working board we have been using. Normally everything works just fine, but if I probe FPGA_DCLK (pin 16 of U33), the FPGA programming through the MAX V starts to fail, requiring many retries and sometimes even timing out at 6 seconds. This was quite repeatable: without the probe connected the FPGA is programmed straight away, while with the probe touching the pin the MAX V struggles to program the FPGA.
To me this suggests a signal integrity / EMI issue on this line specifically. Looking at the schematic, FPGA_DCLK also routes to a few different places spread around the board, which I guess makes it more sensitive to such issues.
For this reason, in the original MAX V firmware I changed the QSF to reduce the drive strength of the FPGA-related lines (fpga_dclk and fpga_config_d[*]). I set the current strength of these lines to 7 mA, and now the previously non-working boards seem to work without any issues.
I will try to investigate this a bit further, but this is the most promising result I have got so far.
Regards,
Leonardo- FakhrulA_altera
Hi Leonardo,
Thank you. This is a very useful finding. The repeatable sensitivity to probing FPGA_DCLK, together with reliable boot after reducing the MAX V drive strength to 7 mA, strongly suggests marginal signal integrity on the FPP path rather than a flash-content issue. Please note that probe capacitance can itself affect DCLK, so we should treat this as strong diagnostic evidence rather than a confirmed root cause.
Could you please validate the 7 mA setting on both affected boards over at least 100 cold power cycles and share:-The exact QSF assignments and Quartus version
-Results for both affected boards and the working board
-Any DCLK/nSTATUS waveforms before and after the change
Thank you again for the detailed investigation.
Regards,
Fakhrul- LeonardoG
Hi Fakhrul
- The QSF assignments that fixed the issue for me on MAX 5 image are:
set_instance_assignment -name CURRENT_STRENGTH_NEW 7MA -to fpga_dclk
set_instance_assignment -name CURRENT_STRENGTH_NEW 7MA -to fpga_config_d[*]
To be honest, in the MAX 5 image I'm currently using I also changed the logic to run fpga_dclk at 50 MHz, but even in the tests I've done at 100 MHz it was working fine (though the testing was not so extensive).- I just tested the 2 affected boards and 1 working board with the new MAX image for around 100 power cycles. They worked 100% of the time. Obviously my sample size is quite limited, and this was only tested at room temperature.
- Regarding the waveforms, you can find the ones regarding dclk and config_d below. On the left is the one obtained with the new MAX image (also with the reduction to 50 MHz), and on the right the original image. While I reckon that the new MAX image is a bit too slow for my liking, the original one has quite a bad overshoot/undershoot.
Below you can find the acquisitions on nConfig and nStatus with the original firmware on a not working board. It is not particularly exciting but you can see the FPGA configuration retrying many times until the 6 seconds timeout. Also note that the time interval between the retries is not repeatable across different power cycles, confirming that the issue has some randomness to it.
Do you have any update on the "related case that is currently under internal review" you mentioned in a previous message? Can you confirm whether you were able to replicate/verify the issue on your side?
Best regards,
Leonardo
