I'll bite-
1- How can I access to source codes of the GUI file (BoardTestSystem.exe)?
**You need to find who wrote your BoardTestSystem.exe and ask them. Your .exe sounds like a custom written app that could be anything.
2- Is it possible to capture data from FPGA to PC through USB.2 and save it by making a new windows file?
**Yes, it can be done.
3- Can I use bts_ddr2top project to develop FPGA codes and change these codes to match my plan?
**It is impossible to tell from this end. I can not tell which Cyclone III dev board you have, nor can I tell what bts_ddr2top is.
I would suggest trying to generate your own Quartus II project that will run on your dev board. Try blinking the LEDs (if you have them). This should get you started working with your project.