How to simulate the FPGA neural network in Altera

Lots of questions here. I will try to answer as best I can.

HDL coder works with, i think, Matlab 2012a and beyond. I use Matlab 2013b. HDL coder and DSP builder both generate only VHDL code. However, as I've said before, since HDL coder is vendor independent - the code it produces is very generic. It would be very difficult to optimize this code by hand, and will depend on the HDL synthesizer's ability to properly allocated resources. DSP builder is more optimal for Altera devices since it was design by Altera for use with their FPGAs. Therefore, they know the architecture of the chip. Either way you need to upgrade you Matlab to a newer version.

When you build with HDL coder or DSP builder you need to set up a base clock. This is because any discrete system needs a time step. It can be 20ns for example, that would require a 50MHz clock which is not a fast speed for FPGAs. You can use Matlab functions, like c2d( ) and tf2sos ( ) to convert your S - domain transfer functions to Z - domain. The system you create using DSP blockset will look a lot like the Simulink design, but you will be using only the parts provided in the DSP Blockset library.

Once you generate VHDL code from DSP builder, you can mix it with Verilog files if you want. The synthesizer does not care, as long as all modules are properly connected. The easiest approach would be to import your VHDL code to Qsys and integrate it with your other modules.

You cannot use Xilinx system generator with Altera FPGAs, only with Xilinx FPGAs. I assume the tool and the amount of work required is the same. Yes, the process of going from continuous to discrete to HDL is tedious, but it is necessary. IT will depend on the complexity of your design.

I do not know enough about FPAAs to comment on them. But I think your design is feasible in an FPGA. But as I said before, you MUST convert to discrete domain; you will not be able to use continuous time functions.

If you want, you can post your Simulink model (the continuous one) and I can take a look at it.