preloader not loading u-boot image

4GB sounds like you have the whole 32-bit address space mapped in your .axf file. Do you have any custom link sections in your project? For example if you mapped the HPS OCRAM (address 0xFFFF0000) as a link section to put some of your variables there, then your elf file will use addresses in almost all the space. In that case the bin file needs to be 4GB big to cover the whole space.

I had a similar problem, that I solved by stripping away this section from the elf file with the arm-altera-eabi-strip command. The problem with that solution is that the variables in the OCRAM section won't be initialized (or cleared) at startup so you must do this manually in your application if it is required.