ModelSim memory use problem / crash

*.vo would be used for a gate-level simulation only, you didn't tell which kind of simulation you're intending. Cause you are also dealing with *.v design files, I guess, you plan a functional simulation. (That would be the first step in design simulation anyway). If so, forget about *.vo and *.sdo first, but supply the necessary libraries. That's somewhat long-winded with Verilog and ModelSim, to my opinion, cause you have to reference anything explicitely, VHDL simulation is basicaly satisfied with library uses in design files.