Replying to this post since it does quite well on Google, and I spent a lot of time tracking down the details.
The 0x04 is a version number of the pof header. The PFL Parallel Flash Loader IP inside the configuration CPLD expects to find this, if your bitstream binary header (0xffff, repeating) is found in it's place, the PFL configuration fails. You can inspect generated VHDL code for the PFL flash to determine the format but it is not fun. The .pof headers also include 13-15 bits of length/checksum information, depending on version. Once that many bytes are read, another pof header is expected.
I found this Altera ticket helpful to generate a binary file with PFL headers:
http://www.altera.co.uk/support/kdb/solutions/rd08082012_952.html Briefly:
1) Generate .sof files during Quartus compilation.
2) Use 'Convert Programming File' tool to lay out multiple bitstreams in the same file, in expected locations, and write header bits, giving .pof file.
Page0 - not present
Page 1 @ 0x00040000
Page 2 @ 0x02140000
3) Use quartus_cpf to get a .hexout file. This contains the bitstreams with PFL/.pof packetisation, but is in a 7-bit ascii/Intel HEX .hexout representation.
$ quartus_cpf -c
c:\\workspace\\flash_topboot.pof c:\\workspace\\flash_topboot.hexout 4) use nios2-elf-objcopy to convert .hexout to .rbf
$ nios2-elf-objcopy.exe -I ihex -O binary
c:\\workspace\\flash_topboot.hexout c:\\workspace\\flash.rbf 5) check format with od
$ od -A x -t x4 /cygdrive/c/workspace/flash.rbf
000000 ffffffff 20860040 41862140 ffffffff <- PFL options block (no Page0, defined Page1 and Page2) as documented in PFL docs
000010 ffffffff ffffffff ffffffff ffffffff
*
000080 ffffff03 ffffffff ffffffff ffffffff
000090 ffffffff ffffffff ffffffff ffffffff
*
040000 05fc0401 ffffffff ffffffff ffffffff <- start of first bitstream, POF Version x04 for block 1
040010 ffffffff ffffffff ffffffff ffffffff
*
040080 ffffffff 6a6a6a6a fffff236 fffffa77
040090 fffff1b6 fffff03d fffff7f6 fffff477
0400a0 fffff9b3 fffff8ba fffff35a fffff27a
0400b0 fffff61b fffff759 fffff87a fffff959
0400c0 fffff418 fffff6bc fffff07d fffff698
0400d0 fffff47d fffff63f fffff27a fffff19c
0400e0 fffff118 fffff1fd fffff7fd fffff519
0400f0 fffff059 fffff019 fffff2d9 fffff019
040100 fffff29d fffff15d fffff71c fffff75d
040110 fffff30b fffff75f fffff49a fffff46a
040120 fffff19a fffff96a 551b0000 ffffffff
040130 ffffffff ffffffff ffffffff ffffffff
*
040600 05940401 00000000 00000000 00000000 <- repeat of POF Header for block two, 600 bytes after
040610 00000000 00000000 00000000 00000000
040620 00000000 00000000 00000000 48004800
040630 48004800 00000000 00000000 48004800
040640 00000000 00000000 00000000 00000000
*
040790 00000000 40000000 00000000 00000000
0407a0 00000000 00000000 00000000 00000000
0407b0 40000000 00000000 00000000 00000000
*
0407d0 00000000 00000000 00000000 00000000
*
0408b0 00000000 00000000 00000000 48004800
0408c0 00000000 00000000 00000000 00000000
*
0408e0 00000000 00000000 48004800 00000000
0408f0 00000000 48004800 00000000 48004800
040900 48004800 00000000 00000000 00000000
040910 00000000 00000000 00000000 00000000
*
040940 00000000 00000000 40000000 00000000
040950 40000000 00000000 40000000 00000000
040960 40000000 40000000 00000000 00000000
040970 40000000 40000000 40000000 40000000
040980 00000000 00000000 40000000 00000000
040990 40000000 00000000 40000000 00000000
0409a0 40000000 00000000 00000000 40000001
0409b0 40000001 00000001 40000001 00000001
0409c0 00000001 40000001 00000001 00000001
0409d0 00000001 00000001 40000001 40000001
0409e0 00000001 40000001 00000001 00000001
0409f0 00000001 40000001 01000001 40000000
040a00 00000000 00000000 00000000 00000000
040a10 40000000 00000000 00000000 40000000
040a20 00000000 00000000 00000000 00000000
040a30 00000000 00000000 00000000 41020000
040a40 00000000 00000000 01020000 41020000
040a50 01020000 01020000 00000000 00000000
*
040a70 01020000 00000000 40000000 40000000
040a80 00020000 01020000 00000000 00000000
040a90 00020000 00000000 01020000 00000000
040aa0 00020000 00000000 00020000 00000000
040ab0 00020000 00000000 01000000 00000000
040ac0 00000000 00000000 00000000 00000000
*
040ae0 00000000 00000000 00000000 00020000
040af0 00000000 00000000 00000000 00000001
040b00 00000001 00020001 00000001 41020001
040b10 41020001 41000000 41000001 00000001
040b20 40000001 00000001 40000001 00000001
040b30 40000000 01020000 00000000 01020000
040b40 40000000 00020000 00020000 01020000
040b50 00020000 410000c0 00020000 000000c0
040b60 000000c0 00000000 00020000 00000000
040b70 00020000 41020000 00020000 01000000
040b80 00020000 00000000 00020000 00000000
040b90 40000000 21530000 05940401 00000000 <- repeat of POF Header for block three
040ba0 01020000 00000000 01020000 00000000
040bb0 00000000 000000c0 010200c0 000000c0
040bc0 000000c0 00ec002f 00000000 01020000
040bd0 00000000 01020000 00000000 00000000
040be0 00000000 01020000 00000000 01020000
040bf0 00000000 00000000 00000000 01020000
040c00 00000000 01020000 00ec00ef 00000000
040c10 00000000 00ec002f 01ee002f 00ec00ef
040c20 01ee002f 00000000 00000000 41ee002f
6) If you want to flash specific regions of this file, use $dd to cut out byte ranges and use your flash programming routines to write them.