VHDL-Code with Avalon Bus Interface-/ and VHDL-Video-System-Interface-Ports

Where do you set the value of dat0 and dat1? or are you just writing some temp code for now that keeps them constant?

Where are you setting x and y? i am surprised what you have compiled because you set them in two different process statements.

Also, the readdata is most likely 1 clock off, as there is a 1 clock delay from when coordinate_rd is set the the correct address value and then another clock till when readdata is set. I would eliminate the coordinate_rd register and just say:

if (avs_read = '1') then
    if (avs_address = '0') then
        coordinate_rd <= std_logic_vector(to_unsigned(dat_0, 32));
    else
        coordinate_rd <= std_logic_vector(to_unsigned(dat_1, 32));
    end if;
end if; 

Also, is there a need to use integers rather than unsigned type? in general it is better to use unsigned because you know the size of the variable.

Besides that, no clue why the change in what you return will change the program from being able to load. I could see the your code not working correctly, but i don't really see why it would fail to load the program into memory. Maybe your *_hw.tcl file is wrong and messes up the bus transactions. You are not trying to load program memory into this hw block i assume, but that would cause a problem.