a question in quartus

You are getting on track but there are some issues:

Quartus error seems to do with your operating system. There is a recent thread about working around it.

your state machine isn't there yet in the code.

some syntax issues, clk in sensitivity list but unused. your process :is it combinatorial or sequential?

if(state = state1) then

....

else -- no need for if state = state2

loop from 0 to 127 not 0 to 128

so you are basically switching the inputs to adders and takingoutputs...fair enough. You will need to use memory for your matrix: here you got resource issue. As FvM said you can't use 64k registers in logic. You need to use memory of fpga. Then your access is limited to your address buses.

Can you post the functional requirements of your work(forget about implementation)