--- Quote Start ---
File name only. It won't read the file if the name doesn't match.
IEEE is brought in by default.
--- Quote End ---
As far as Im aware, it looks for entity names, not just filenames (thats more a verilog thing). By default it will put everything in the work library unless you specify otherwise, but it's not as strict as the standard specifies when looking for entities in libraries. If it cannot find the entity in the work library, it will just look around until it finds it in any other library.
The library section doesnt apply to VHDL. Its more an AHDL throwback. AHDL has .inc files (like .h files in C) that cannot be included in the file list. An AHDL file may use include somefile.inc and Quartus will search through the libraries paths until it finds the .inc files.