I'm not sure how I missed this, but the camera I am interfacing with does have 1k resistors pulling up the sda and scl lines to 3.3 V.
I am having some success reading from the camera if I slow it down far enough, but not consistently. Writing to the board never seems to work though. I looked into the schematic of some generic i2c level shifters online, and they seem to use a pull up resistor on either side of the conversion. Ie, up to 2.5 on the fpga side, and up to 3.3 on the slave. It seems that now I have to figure out how to get an external resistor hooked up to the fpga side. Unfortunately this seems impossible...