Forum Discussion
Altera_Forum
Honored Contributor
Honored ContributorOpenCL on FPGA Not able to generate hardware for matrix_mult.cl
Hello everyone,
I am trying to compile a very basic matrx_mult kernel (also same error occurs for hello_world) for a nallatech 385a board. However, always I am getting the following error:aoc -v -board=p385a_sch_ax115 matrix_mult.cl
aoc: Environment checks are completed successfully.
aoc: If necessary for the compile, your BAK files will be cached here: /var/tmp/aocl/
You are now compiling the full flow!!
aoc: Selected target board p385a_sch_ax115
aoc: Running OpenCL parser....
aoc: OpenCL parser completed successfully.
aoc: Optimizing and doing static analysis of code...
aoc: Linking with IP library ...
Checking if memory usage is larger than 100%
Compiler Warning: Vectorized kernel contains loads/stores that cannot be vectorized. This might reduce performance.
aoc: First stage compilation completed successfully.
Compiling for FPGA. This process may take a long time, please be patient.
Error (23031): Evaluation of Tcl script import_compile.tcl unsuccessful
Error: Quartus Prime Compiler Database Interface was unsuccessful. 1 error, 0 warnings
Error: Compiler Error, not able to generate hardware
nternal Error: Sub-system: DCALC, File: /quartus/ddb/dcalc/dcalc_bcm_modules_cache.cpp, Line: 110
Could not load pdb file - /home/opt/intelFPGA_pro/17.1/quartus/common/devinfo/20nm/ddb_nightfury_cc_dcm_h-ss-1p25-100-hs-n_model aoc -version
Intel(R) FPGA SDK for OpenCL(TM), 64-Bit Offline Compiler
Version 17.1.0 Build 240
Copyright (C) 2017 Intel Corporation
Quartus Prime Analysis & Synthesis
Version 17.1.0 Build 240 10/25/2017 SJ Pro Edition
Copyright (C) 2017 Intel Corporation. All rights reserved.
Quartus Prime Compiler Database Interface
Version 17.1.0 Build 240 10/25/2017 SJ Pro Edition
Copyright (C) 2017 Intel Corporation. All rights reserved.
aocl 17.1.0.240 (Intel(R) FPGA SDK for OpenCL(TM), Version 17.1.0 Build 240, Copyright (C) 2017 Intel Corporation) 21 Replies
Altera_ForumI managed to get a compilation pretty far. It took about an hour and 15 minutes to fail this time for the matrix_mult.cl kernel.
The last error message was:
I am attaching also the generated log. Any suggestions regarding that? I never seen this error message before or I either my compilation went that far. P.S the matrix_mult is directly taken from nallatech example rar for my board. I didnt modify it.Info: Command: quartus_cdb -t import_compile.tcl Info: Using INI file /home/admin/fpga_experiments/nallatech/examples_p385a_sch_ax115/matrix_mult/matrix_mult/bin/matrix_mult/quartus.ini Info: Checking for OpenCL SDK installation, environment should have INTELFPGAOCLSDKROOT defined Info: INTELFPGAOCLSDKROOT=/home/admin/intelFPGA_pro/17.1/hld Info: Successfully completed BAK flow Info: To reduce compile time on future compiles, you can generate a BAK cache by adding the arguments '--bsp-flow regenerate_cache' to aoc to skip BAK Info: Retry strategy set to "retry-flat" Info: Initial preservation set to "final" Info (125061): Changed top-level design entity name to "top" Info (125061): Changed top-level design entity name to "kernel_system" Info (16677): Loading synthesized database Info (16734): Loading "synthesized" snapshot for partition "root_partition". Info (16678): Successfully loaded synthesized database: elapsed time is 00:00:03 Info: Performing a fit attempt Error: Quartus Fitter has failed! Breaking execution...- Altera_Forum
You are getting another random crash:
--- Quote Start --- Internal Error: Sub-system: VPR20KMAIN, File: /quartus/fitter/vpr20k/dap/dap_congestion.cpp, Line: 1525 Internal Error Stack Trace: 0xbb0d4: vpr_qi_jump_to_exit + 0x6f (fitter_vpr20kmain) 0x1b6288: vpr_exit_at_line + 0x83 (fitter_vpr20kmain) 0x562d37: dap_evaluate_move + 0x77 (fitter_vpr20kmain) 0x416556: l_mpp_perform_moves_worker(void*) [clone .isra.671] + 0x30f (fitter_vpr20kmain) 0x416682: l_mpp_worker_thread(void*) + 0x4a (fitter_vpr20kmain) 0xd09d2: l_thread_start_wrapper(void*) + 0x29 (fitter_vpr20kmain) 0x5b4c: thr_final_wrapper + 0xc (ccl_thr) 0x3f21f: msg_thread_wrapper(void* (*)(void*), void*) + 0x62 (ccl_msg) 0xac5c: mem_thread_wrapper(void* (*)(void*), void*) + 0x5c (ccl_mem) 0x8b49: err_thread_wrapper(void* (*)(void*), void*) + 0x27 (ccl_err) 0x5b8f: thr_thread_wrapper + 0x15 (ccl_thr) 0x5e72: thr_thread_begin + 0x46 (ccl_thr) 0x7e25: start_thread + 0xc5 (pthread) 0xf834d: clone + 0x6d (c) End-trace --- Quote End --- Since the crashing point seems to be changing, my guess is that some component in your system must be unstable. This could be the CPU, the memory, the disk, or even the network if you are using a network storage system. Do you have any other machines to use for a compilation test? - Altera_Forum--- Quote Start --- You are getting another random crash: Since the crashing point seems to be changing, my guess is that some component in your system must be unstable. This could be the CPU, the memory, the disk, or even the network if you are using a network storage system. Do you have any other machines to use for a compilation test? --- Quote End --- I can try a different machine. However, the issue is that for the Altera SDK I got floating licence, while for the nallatech BSP the license is binded with the MAC address of that machine. Also, is the only machine around with 64GBs of RAM. I am currently using CentOS 7.4 which is a supported OS for compilation and runtime according to nallatech. Do you think an other OS might be more stable for the current work-flow? Any suggestions? Thanks again for the replies!
- Altera_Forum--- Quote Start --- I can try a different machine. However, the issue is that for the Altera SDK I got floating licence, while for the nallatech BSP the license is binded with the MAC address of that machine. --- Quote End --- Nallatech's BSP license is only for installation of the BSP, it does not bind the BSP to the machine (Altera does not even provide the possibility to do so). After installation, you can copy the BSP folder to any other machine and use it. Though they probably want you to think that you cannot do this. ;) --- Quote Start --- I am currently using CentOS 7.4 which is a supported OS for compilation and runtime according to nallatech. Do you think an other OS might be more stable for the current work-flow? Any suggestions? --- Quote End --- I have used CentOS v7.x for compilation of kernels for Nallatech's 385A board using different BSPs (16.0.2, 16.1.2, 17.1) and never encountered any OS-related problems. Apart from that, as you said, Nallatech's PCI-E driver officially supports CentOS v7.x starting from BSP v17.0, so that should also be fine.
- Altera_Forum
Hello,
Thanks again for the reply. I manage to compile and run successfully on the fpga. I compiled the matrix multiplication kernel and took 2 hours and 25 minutes for completion. However, the problem was very odd. It was due to the network connection SOLUTION: I disconnect the machine from the network ( I took out the Ethernet cable ) and it worked in the first go. Previously, I used only ssh connection to the machine (only through Tmux e.g byobu, screen etc). Is anyone experienced something similar before??? Also, I discover that some of the pre-complied examples given by nallatech for my current board or not compilied for the current BSP that they claim. As many of them failed to run with a CL_INVALID_BINARY, while the rest working out of the box. - Altera_Forum--- Quote Start --- I disconnect the machine from the network ( I took out the Ethernet cable ) and it worked in the first go. Previously, I used only ssh connection to the machine (only through Tmux e.g byobu, screen etc). Is anyone experienced something similar before??? --- Quote End --- That sounds very strange. Honestly I wouldn't be quick to judge in this case; your issue could be caused by a recurring transit problem that just didn't happen this time, and could happen again later. The compilation machines I personally use are all remote machines and I just connect to them via SSH, so the problem is certainly not from remote connections.
- Altera_Forum--- Quote Start --- That sounds very strange. Honestly I wouldn't be quick to judge in this case; your issue could be caused by a recurring transit problem that just didn't happen this time, and could happen again later. The compilation machines I personally use are all remote machines and I just connect to them via SSH, so the problem is certainly not from remote connections. --- Quote End --- Just checked with two more kernels and I can verify that the ethernet causes the random crashes during compilation. As long as the machine is offline, compilation worked fine.
- Altera_Forum--- Quote Start --- That sounds very strange. Honestly I wouldn't be quick to judge in this case; your issue could be caused by a recurring transit problem that just didn't happen this time, and could happen again later. The compilation machines I personally use are all remote machines and I just connect to them via SSH, so the problem is certainly not from remote connections. --- Quote End --- Agreed; there is probably something else about your system that changed. Have you tried rebooting the machine and then building the kernel with and without the ethernet plugged in? If I were you, I would try to write down anything I've done to the machine that may or may not have made the tools work in case whatever change you made is only temporary (and will get undone upon reboot). I asked earlier but do you have a lot of free disk space on the drive you're building on? If you've been running out of room on your hard drive during builds that would explain the seemingly random crashes (b/c the tool can't create files it needs to).
- Altera_Forum--- Quote Start --- Just checked with two more kernels and I can verify that the ethernet causes the random crashes during compilation. As long as the machine is offline, compilation worked fine. --- Quote End --- Fair enough. How do you checkout your floating license without a connection?
- Altera_Forum--- Quote Start --- Agreed; there is probably something else about your system that changed. Have you tried rebooting the machine and then building the kernel with and without the ethernet plugged in? If I were you, I would try to write down anything I've done to the machine that may or may not have made the tools work in case whatever change you made is only temporary (and will get undone upon reboot). I asked earlier but do you have a lot of free disk space on the drive you're building on? If you've been running out of room on your hard drive during builds that would explain the seemingly random crashes (b/c the tool can't create files it needs to). --- Quote End --- I rebooted the system and still works only without the ethernet cable connectesd Currently on my machine I got an SSD drive of 1TB with 600GB free space. I also checked during the successful compilation and my resource utilization is ok for that machine (e.g. about 20GB of RAM out of 64 and CPU usage on all cores)
