You are probably compiling your project into a .sof file and downloading it directly into the FPGA configuration RAM.
It will be active UNTIL you power cycle the board as the RAM configuration does not persist without power.
At power up the configuration EPROM will be read and loaded into the device (the boards are normally setup to do this).
So that is why if you power cycle the board the 'default' configuration stored in the config EPROM reappears.
What you need to do, besides compiling your project to a .sof file, is use the Quartus tool quartus_cpf to generate a config EPROM .jic
which you can then download to the EPROM.
For example I run:
quartus_cpf -c DESIGN.cof
where DESIGN.cof contains:
<?xml version="1.0" encoding="US-ASCII" standalone="yes"?>
<cof>
<eprom_name>EPCS64</eprom_name>
<flash_loader_device>EP4CE115</flash_loader_device>
<output_filename>DESIGN.jic</output_filename>
<n_pages>1</n_pages>
<width>1</width>
<mode>7</mode>
<sof_data>
<user_name>Page_0</user_name>
<page_flags>1</page_flags>
<bit0>
<sof_filename>DESIGN.sof</sof_filename>
</bit0>
</sof_data>
<version>4</version>
<options>
<map_file>1</map_file>
</options>
</cof>
And you will want to add the Altera serial flash loader module into your top level module as well (assuming verilog here):
// Altera Serial Flash Loader module
// Allows external JTAG access to the serial config eeprom for programming
sfl sfl
( .asdo_in (1'b0), // not used
.dclk_in (1'b0), // not used
.ncso_in (1'b0), // not used
.noe_in (1'b0), // always enabled
.asmi_access_granted (1'b1), // external JTAG access always
.asmi_access_request (), // not used
.data0_out () // not used
);