I have two Cyc III devices (master-slave, AS and PS respectively) + EPCS16 flash. In both devices I have ROMs (megafunction), one in the master and two of them in the slave. All ROMs are initialized with .mif files. The problem is: the slave ROMs are always broken, i.e. have bit errors, which can be seen in the In-system memory content editor (and also have bad CRC, which is calculated by the external CPU). I can re-write content of ROM with ISMCE, and there are no errors then. If slave is loaded from EPCS - there are errors. If I try to reload slave via JTAG - there are errors. I get no errors from quartus, not when programming EPCS, nor while JTAG operations. ROM megafunction themselves works correctly, i.e. I can read content of ROMs with external CPU with no problems, but content itself is broken. Master is always working correctly - no errors, CRC is ok etc. Connection is as shown in handbook, I also have 2 buffers on the DCLK and DATA[0] lines (74LVC1G125), but no series resistors (they are optional for 3.3V, according to handbook). Trace length from buffer to slave is about 110..120 mm. Appreciate for any help.

I think I've found the problem. There is a trace connected to pin 76 on the master and pin 99 on the slave. Trace is pulled up to 3.3V via 2.2k This **bleep** resistor was dead, and the trace was actually floating. I didn't use this pins before, so I can't reveal the resistor is dead. As I start to use it, all logic became a total mess. Something worked, something not. By elimination I found the culprit. I can't explain how this trace connected with loading errors (when FPGA not even in user mode), but the fact is - replacing resistor solved all problems, ROMs working now (though I don't use them anymore). Thanks to all, problem solved!

Cyclone III slave (PS) embedded memory errors

27 Replies

Vic3Dexe
Occasional Contributor
3 years ago
What worries me most - is it possible to these bit errors happens when the whole slave is being programmed? I mean, can this broke some functionality? AND gates becomes OR for example. All board always loaded correctly for now, but that's not so much code in the slave atm, it's ROM read, address decoding, couple of registers, some bus functionality.

I can do some workaround, for example, move ROMs to the external RAM, and load them from EPCS through ASMI megafunction. But this has no sense if there is a chance slave itself will not load correctly...

Is there some error checks inside Cyclones when they downloads firmware from EPCS? If so, why it not includes content of pre-initialized memory?
hareesh
Frequent Contributor
3 years ago
Hi Vic3Dexe,

From the post, I understand that you are facing issue reading data from ROM through the slave. Commonly this type of issues occur when there is a problem with MIF file. Please regenerate the .MIF file by following the procedure in the below article, it should resolve the issue. Please feel free to write back if you have queries.

Article :
https://www.intel.com/content/www/us/en/support/programmable/articles/000078411.html

Thanks and Regards
Hareesh
- Vic3Dexe
  Occasional Contributor
  3 years ago
  @hareesh wrote:
  
  From the post, I understand that you are facing issue reading data from ROM through the slave.
  
  No, the problem is: the slave is always loads its ROMs with erros (from EPCS or from JTAG). The only way I can load ROMs correctly is through the In-system memory content editor. When I program ECPS I get no errors from quartus (with verification or not). So, EPCS itself is loaded correctly.
  
  MIF files are ok. I did not change them and recompiled the whole project dozens of times.
  
  I've tried to remove buffers and change them to zero Ohm resistors - no changes.
  
  Then I've replaced 0R resistors to 51R ones - no changes.
  
  Then I've put buffers back, cutted traces afters them (which are facing to the slave), and put 51R there. No changes.
  
  I'll repeat myself:
  
  Is there some error checks inside Cyclones when they downloads firmware from EPCS? If so, why it not includes content of pre-initialized memory?
FvM
Super Contributor
3 years ago
Hello,

is the erroneous ROM content reproducible or arbitrarily varying? In the former case, I suspect a bug of the programming file conversion tool.

"If I try to reload slave via JTAG - there are errors." What does this exactly mean? You didn't show the JTAG configuration, are the devices configured in a chain? How should the master/slave AS configuration affect JTAG operation? I presume you have separate .sof files for both devices?

Frank
- Vic3Dexe
  Occasional Contributor
  3 years ago
  @FvM wrote:
  
  is the erroneous ROM content reproducible or arbitrarily varying? In the former case, I suspect a bug of the programming file conversion tool.
  
  It's arbitrarily varying. Sometimes it's only one bit failed, sometimes it's 5-6. Sometimes bits are the same, but most of time they are varying.
  
  The file conversion tool is ok, because, as I said, I can update ROM content through the In-system memory content editor (ISMCE) with the same file used in .sof/.pof, and then the content is ok.
  
  @FvM wrote:
  
  "If I try to reload slave via JTAG - there are errors." What does this exactly mean? You didn't show the JTAG configuration, are the devices configured in a chain? <> I presume you have separate .sof files for both devices?
  
  It means that if I use JTAG to reload .sof file for slave, the ROMs content is still broken. And again, it's different bits are failing.
  
  Yes, ofc, they are in JTAG chain. I have 2 different .sof files, and I can reload one or both devices.
  
  So, here is an example what I'm doing:
  
  1. Power up the board, both devices loads themselfs from EPCS (AS/PS). Master ROM is ok. Slave ROMs are failed.
  
  2. Reloading slave via JTAG - still have ROM errors.
  
  3. Reloading ROMs via ISMCE - ROMs are ok.
  
  @FvM wrote:
  
  How should the master/slave AS configuration affect JTAG operation?
  
  Idk. I assume some ringing somewhere on the board, both DCLK/DATA0 and TCK/TDI/TDO.
  
  But why the slave loads at all? If there is some error check, it should fail, isn't it? Or there is no error check?
  - FvM
    Super Contributor
    3 years ago
    Thanks for detailed explanation. Your conclusions are plausible, sounds like at least the memory initialization part of the configuration bit stream has insufficient consistency check.
    
    Frank
    
    P.S. According to my experience, double edges caused by ringing clock signal is the most popular signal integrity problem related to configuration. Series resistors providing source side termination can be essential to get clean clock.
Vic3Dexe
Occasional Contributor
3 years ago
There is the ROM1 content in the slave after power up:

Then I do "Import data from the file" and use the same .mif file used for ROM init:

Red numbers shows the differences. Then "Write data to in-system memory", read it back again, and "Import data from file" again:

As you can see, all numbers which were red, now are consistent with actual .mif file.
- hareesh
  Frequent Contributor
  3 years ago
  Hi Vic3Dexe,
  
  can you please provide total procedure you are following (bit file generation, bit file loading, loads from EPCS, etc..)
  - Vic3Dexe
    Occasional Contributor
    3 years ago
    1. MIF files were generated only once. I don't change them. They are constant. It's irrelevant how they are maked, because they don't change.
    
    2. I have 2 projects in Quartus (13 sp1 web version) - one for master, and one for slave.
    
    3. In both projects I've created "ROM 1-port" megafunctions (1 for master, and 2 of them for slave) and set according .mif files as init values. Like this:
    
    4. Compiled (or fully recompiled) both projects, so the result is two .sof files.
    
    5. Used convert programming file tool to create .pof file:
    
    6. Used programmer to write EPCS:
    
    Here is the quartus log (one of the million attemp to get it to work). Verification also always passes (if enabled).
    
    Info (209060): Started Programmer operation at Tue Jun 21 17:23:18 2022 Info (209018): Device 1 silicon ID is 0x14 Info (209044): Erasing ASP configuration device(s) Info (209024): Programming device 1 Info (209018): Device 1 silicon ID is 0x14 Info (209011): Successfully performed operation(s) Info (209061): Ended Programmer operation at Tue Jun 21 17:25:47 2022
    
    7. Power up/reset the board. Both devices loads its content from EPCS.
    
    8. Nothing works because of broken ROMs content in the slave (and ONLY in the slave). Look at the previous post, I've described there how to detect those errors.
    
    9. Reload the slave .sof via JTAG. Or both master and slave. No changes, ROM still has errors.
    
    10. Reload ROMs via ISCME. No errors.
    
    I want to empasis the problem - .mif files used for ROM initialization in project don't get in ROMs intact. And this problem exists only for slave.
- hareesh
  Frequent Contributor
  3 years ago
  hi @Vic3Dexe ,
  
  scope shot from oscilloscope is required so we know what happened in the trace DATA and DCLK. can you share that
  
  thanks,
  - Vic3Dexe
    Occasional Contributor
    3 years ago
    No problem as soon as someone share oscilloscope to me.
    
    Where exactly you need traces? AS_DCLK/AS_DATA or SOUTH_DCLK/SOUTH_DATA (after buffers)?
    
    I've cut traces and placed series resistors afters buffers (51R). Should I remove them before measurement?
hareesh
Frequent Contributor
3 years ago
Hi,
Thanks for sharing files, just give me some time, we are working on it.

thanks,
Vic3Dexe
Occasional Contributor
3 years ago
I think I've found the problem.

There is a trace connected to pin 76 on the master and pin 99 on the slave. Trace is pulled up to 3.3V via 2.2k

This **bleep** resistor was dead, and the trace was actually floating.

I didn't use this pins before, so I can't reveal the resistor is dead.

As I start to use it, all logic became a total mess. Something worked, something not. By elimination I found the culprit.

I can't explain how this trace connected with loading errors (when FPGA not even in user mode), but the fact is - replacing resistor solved all problems, ROMs working now (though I don't use them anymore).

Thanks to all, problem solved!