Cyclone V SoC 5CSXC6 Series GXB Utilization and Limitations

Hi Brian,

Apologies for the delay.

I conducted some tests by replacing the PIPE PHY with the PCIe Hard IP, and during the Fitter compilation I encountered the following error:

Error (20039): TX channel <tx_serial_data_t1[0]~CLUSTER~HSSI_TX_CHANNEL_CLUSTER3> and PCIE pll can't share the same duplex channel location.

Just to check with you if you are observing a similar error on your side?

As I search into our internal database, this appears this restriction imposed by the PCIe Hard IP. Specifically, the TX channel within the same transceiver (XCVR) pair associated with a CMU PLL cannot be used.

A potential workaround would be using the PIPE PHY and interfacing it with a third-party PHY MAC/controller in the fabric to build a complete PCIe solution.

Apologies for any inconvenience caused, and thank you for your understanding.

CheepinC_altera​ 

Just to check with you if you are observing a similar error on your side?

From day ONE i am having this atrocious meaningless fail report.

As I search into our internal database, this appears this restriction imposed by the PCIe Hard IP. Specifically, the TX channel within the same transceiver (XCVR) pair associated with a CMU PLL cannot be used.

WHY? WHY PIPE vs hard placement have such huge different?

A potential workaround would be using the PIPE PHY and interfacing it with a third-party PHY MAC/controller in the fabric to build a complete PCIe solution.

I cannot see setting from the IP via soft core implementation like DPHY MEMORY IP etc.

Thanks,

Brian

CheepinC_altera​ 

I am still holding onto the possible way to use hard PCIe under all channel placement.

Thanks

Hi,

Based on the internal reference, the placement restriction is imposed by the Hard IP. In contrast, the PIPE interface operates at the low-level PHY layer and does not include the higher protocol layers integrated within the Hard IP.

For implementations requiring a soft MAC/controller, you would need to use a third-party soft MAC/controller and interface it with the PIPE PHY. 

Sorry for the inconvenience. Thank you.

CheepinC_altera​ 

Did not answer the question WHY?

Hard IP require a full global clock route or any other major reason it is introducing such limitation.

We all now know that it is restricted but WHY?

Please do answer or give out the possible reasons!

Hi Brian,

Thank you for the update, and I apologize for the confusion. I appreciate you bringing this to my attention.

For your reference, this restriction was introduced to address the issue documented in the following Knowledge Base article:

https://community.altera.com/kb/knowledge-base/why-does-the-transceiver-tx-simplex-sends-wrong-data-when-the-cmu-pll-of-pciex1-/342679 

The restriction helps prevent the behavior described in the KDB and was implemented based on the findings from that investigation.

Please let me know if you have any further questions or need any additional clarification. Thank you.

CheepinC_altera​ 

According to the passage, if i use 17.0 or older version i can override this?

If the custom protocol can error corrects or subject to wrong data it is still possible to use all channel while not under version 18.0 or above?

Thanks,

Brian

Hi Brian,

Based on the information provided in the KDB, designs using Quartus versions earlier than 18.0 may be able to bypass this error. However, please note that doing so is generally not recommended, as the restriction was introduced to address a known issue.

If you choose to proceed with bypassing the error, I strongly recommend performing thorough validation and testing to ensure that your custom protocol operates correctly and meets your system requirements under all expected conditions.

We appreciate your understanding. Please let us know if you have any further questions or need additional assistance.

Thank you.

CheepinC_altera​ 

It is possible to override it easily via a force_flag i the file etc under newer version?

It is very tedious  to install old version due to this force error stuff.

Thanks,

Brian

Hi Brian,

Thank you for the update.

My understanding is that it would be difficult to justify reverting this change, as it was introduced to address a known issue. Therefore, we do not anticipate reverting the behavior at this time.

That said, please feel free to let us know if you encounter any issues during the installation or migration process. We will be happy to assist and provide guidance as needed.

Thank you for your understanding.

CheepinC_altera​ 

Need time on my side to under go this testing.

Please hold onto this ticket.

As for your reply do you mean it cannot override this error fail like the PCIe soft reset force setup via qsys or ip file change "xxx=True" ?

Thanks

Hi Brian,

Thank you for the update.

Regarding the Quartus Fitter error, your understanding is correct. The error check and its associated trigger are implemented within the Quartus software itself, rather than within the IP files. As a result, the behavior is determined by the Quartus version being used, independent of the generated IP files.

Please let me know if you have any further questions or if there is anything else I can assist with.

Thank you.

CheepinC_altera​ 

I have one unconfirmed information from:

https://community.altera.com/kb/knowledge-base/why-does-the-transceiver-tx-simplex-sends-wrong-data-when-the-cmu-pll-of-pciex1-/342679

The wrong data is only affected by the  CMU PLL or not affected when fPLL is used?

This require your internal database info.

Thanks

CheepinC_altera​ 

I still need a confirmation on the CMU PLL wrong data situation it is only introduced via CMU PLL or both fPLL case?

Yes 17.1 shows a huge different on fittering

CH4 now can place and fit with the previous debug project.

CheepinC_altera​ 

Final result rolling back the entire project from 18.1 or 20.1 to 17.1 and boot via HPS + RBF shows success result on custom protocol via f-PLL and 4 channel PCIe However due to the inherent bottlenecking the 4 channel performance is not great in the first place.

I enclosed a log for reference.

Hi Brian,

Thank you for the update.

I checked the internal case associated with the KDB and did not find any references to the fPLL. Based on the available information, the internal discussions appear to focus exclusively on the CMU PLL.

Could you please share more details regarding your note with fPLL in previous response ie source of information? This would help me investigate further and clarify.

I look forward to your response.

CheepinC_altera​ 

I am very confused. If there internal original discussion only related to CMU PLL case then why block and forbidden the fittering to place fPLL in the first place?

This make no sense.

For Quartus 17.1 I simply do a very basic placement on fPLL + hard PCIe like what you had done in 20.1 and no fittering issue. So I move onto real design project and simply roll back all IP via tcl script and regenerate necessary rbf and loaded via HPS.

So real system shows no issue on both custom protocol and PCIe via NVMe testing see the log.
However The PCIe RP itself is very slow which bandwidth can only achieve 400MB/s under x4 GEN 1.
Either how it is still working properly under 17.1 via fPLL and shared reconfiguration as we both discussed in this long conversation.

I had also enclosed the placement image that you can see the top fPLL is used along side the with big hard PCIe block.

Thanks,

Brian

Hi Brian,

Thank you for the update.

If I understand your previous note correctly, you have tested the PCIe HIP design using the fPLL, and the design is able to pass Fitter and operate successfully on CV hardware. To ensure we are aligned, would you mind sharing the QAR for the test project? Having a look at the design will help me better understand the configuration and setup used in your test.

I believe the test design I previously provided was using the CMU PLL for the PCIe HIP, so it would be helpful to compare the two implementations.

Thank you, and I look forward to your response.

CheepinC_altera​ 

No the PCIE hard IP cannot modify from user side it is a must on CMU PLL while this limits the use of fPLL placement on remains channels when >= 18.0 Quartus.

The report from the your link only mentioned the TX wrong data and did not clearly mentioned it is on PCIe side or custom channels so this is a bit hard to understand if only CMU PLL is introducing possible issue on remains channel why fPLL would be blocked to use in the first place. Unless the CMU PLL introduce wrong data to hard PCIe as well.

CheepinC_altera​ 

A table maybe clear up things more better:

CheepinC_altera​ 

Meantime, the reconfiguration also successfully tested on our side with modification of Vod and Pre-emphasis first post-tap. Which these two are the most important in most use cases. The recfg shows no issue on register read return status.

Functional-wise also show normal behavior on real hardware.

CheepinC_altera​  - Waiting on your internal investigations result.

So back to the original question, what actually the report really referring to? The CMU PLL creates wrong data along with CH1 / CH4? And why only PCIe x1 use case not x2 x4?
So the report is referring to the channel NOT the PCIe itself but custom or other protocol use case on those remains channel.
However fPLL is complete isolated with the CMU PLL generated clock both // & ser. so there are no possible correlation on wrong data latching between xN x1 x6 clock lanes. How could this even possible to restrict the use and fit for fPLL use cases?

Make zero sense, unless the Mbps and MHz both fPLL and CMU PLL is the same or sub ordering that could introduce "harmonic spurious" or "high jitter" that violates the timing setup hold slacks.

Cannot understand why >= 18.0 ban user to fit channel that is possible to use in first place. 😞😞😞

Thanks,
Brian