The 64-bit executables are fine too, but they will use more memory (every pointer in memory requires 64bit instead of 32bit), so if you only have 4GB and you are only using Quartus for small designs, why waste the little memory you have.
In general, there is no benefit in using the 64bit version unless you need it (again, because your design needs the 64bit address space). But running the 32bit EXE on a 64bit OS is still an improvement as Quartus gets to use the full 4GB (2^32) if needed (instead of the 2GB that win32 limits).
On the side, note that Linux32 does not have the 2GB limitation, and Quartus 32bit exe can use the full 4GB on Linux32.
Hope this answers your question