How much of a cache's cost could be saved by removing the address and data buses?

The whole point of a cache memory is to provide very low latency (ie, one clock cycle) random access to a small set of locations. Your architecture does not do that. It appears you are describing something more like a FIFO with a write enable input.