Stratix10 SX Soc Development Kit boot failed randomly

Hi,

We are also having the exact same issue with our production boards. In some cases, our board does not boot.

This occurs on our own design, but we are observing the exact same problems as above. More specifically our device is:

FPGA: 1SX280HU3F50E2VG

We are compiling this design using Quartus Prime Version 19.4.0 Build 64 12/04/2019 SC Pro Edition

This error does not always appear to occur - if you leave it on for long enough, eventually it starts to work. But not always.

Some additional observations:

1) When this occurs, JTAG programming is also likely to fail. We do not know why.

2) The likelyhood of this happening increases if we spray the board with flux remover.

We get the following error messages:

U-Boot SPL 2017.09-00187-g70eb145123 (Apr 06 2020 - 19:11:55)
MPU         1000000 kHz
L3 main     400000 kHz
Main VCO    2000000 kHz
Per VCO     2000000 kHz
EOSC1       125000 kHz
HPS MMC     50000 kHz
UART        100000 kHz
DDR: Initializing Hard Memory Controller
DDR: Triggerring emif_reset
DDR: emif_reset triggered successly
DDR: Triggerring emif_reset
DDR: emif_reset triggered successly
DDR: Triggerring emif_reset
DDR: emif_reset triggered successly
DDR: Error as SDRAM calibration failed
DDR: Initialization failed.
### ERROR ### Please RESET the board ###

Or, we get this:

U-Boot SPL 2017.09-00187-g70eb145123 (Apr 06 2020 - 19:11:55)
MPU         1000000 kHz
L3 main     400000 kHz
Main VCO    2000000 kHz
Per VCO     2000000 kHz
EOSC1       125000 kHz
HPS MMC     50000 kHz
UART        100000 kHz
DDR: Initializing Hard Memory Controller
DDR: Calibration success
SDRAM: Initializing ECC 0x00000000 - 0x80000000
SDRAM-ECC: Initialized success with 1357 ms
DDR: HMC init success
DDR: 2048 MiB
DDR: Running SDRAM size sanity check
DDR: SDRAM size check passed!
QSPI: Reference clock at 400000000 Hz
Trying to boot from MMC1
"Synchronous Abort" handler, esr 0x96000210
ELR:     ffe08fb0
LR:      ffe09034
x 0: 0000000000000001 x 1: 00000000000003e8
x 2: 0000000000000020 x 3: 0000000000000015
x 4: 00000000ffe3d640 x 5: 0000000000000001
x 6: 0000000000000040 x 7: 00000000ffe3db00
x 8: 0000000000000200 x 9: 0000000000000080
x10: 0000000080000010 x11: 0000000080000014
x12: 0000000000000176 x13: 0000000000000454
x14: 00000000ffe3dd6c x15: 00000000ffe12a60
x16: 0000000000030a10 x17: e3fb9eb93f97afdf
x18: 00000000ffe3de90 x19: 000000003fa00800
x20: 00000000ffe3d7a8 x21: 0000000000000010
x22: 00000000ffe3db00 x23: 0000000000000004
x24: 0000000000000400 x25: 000000000003a980
x26: 00000000000008a4 x27: 000000000000be80
x28: 0000000000000010 x29: 00000000ffe3d6a0
 
Resetting CPU ...
 
resetting ...
Mailbox: Issuing mailbox cmd REBOOT_HPS

After which it fails again.

We're very interested in what you recommend, and how to fix this.