Quartus ii won't simulate Modelsim from a bdf file

Correct. Modelsim has no concept of schematics.

What you need to do is convert the .bdf to a HDL file. If I recall correctly, there is a button somewhere that does this - basically creates either a Verilog or VHDL file based on the block diagram which you can then use for simulation. You will also have to set the HDL file to be the top level block instead of the .bdf file so that when it compiles for simulation it uses the correct file.