Forum Discussion
Honored ContributorHigh Speed Embedded Memory
Hello guys,
We are in process of designing a new embedded high performance data acquisition platform. Basically, we have let’s say 400 ADC channels which each channel is 16 bit wide and we need to sample these channels at 50Khz frequency which generates almost 40MB of data per second. We need to store this data in order to be downloaded later on. The amount of data which we need to store is about 20 GB of data. We don’t have any problem with buffering this data as we implemented multiple NiosII processors with multiple SGDMAs. However, the only bottleneck of the system is storage memory. We need a device which capable of storing this data. It should be noted that this device should be reliable one and also can operate at extended temperature ranges. Any idea and recommendation on that would be much appreciated. Also, I should mention that this tool is a remotely controlled and battery operated tool. Regards, Aidan.11 Replies
Altera_ForumI should also say that we tried different CF cards. they are ok but not fast enough.
- Altera_Forum
Assuming Ethernet and USB are out of the question (hitting 40MB/s should be possible with both). If connecting over a cable is possible this probably becomes a lot easier (and cheaper too).
But if you must locally store it first how about multiple flash interfaces that each are responsible for 'x' number of ADC channels of storage? Then you can access them concurrently to eliminate the bandwidth problem. I would check to see how much bandwidth you achieve with a single flash channel then scale it up accordingly so that you can try out different solutions easier. What do you have in mind after you are done storing the data? Would you just pull flash cards out and bring them to a host or would you move the entire board and download the contents after? You'll probably require fewer flash interfaces if you just need the later (download from the board) since soldered down devices are usually quicker than cards (unless you use a complicated controller and fast/expensive cards) - Altera_Forum
s-tec (http://www.stec-inc.com/product/mach4cf.php) manufactures industrial Compact Flash cards with sustainable write rates of 50MB/s. The bottleneck could be the Nios CPU though, especially if you need a file system to organize your data.
- Altera_Forum
thanks guys for fantastic comments. they are really helpful and really appreciated. Multiple Compact Flash solution is a good idea which I had it in mind. this is possibly the easiest and the cheapest one. Unfortunately, connection over cable is not possible due to amount a vibration and shock. We tried this several time previously and 80% of the time we end up system failure which cost us a lot. I searched on the internet and found Solid State card such as Intel® Solid-State Drive 310 Series and above. the size of it as small as credit card and power consumption is not that bad and we can cope with it. I just want to know that have you guys ever interfaced Altera FPGAs with SSDs.(SATA) Currently, we are using Cyclone III. I think we may have some difficulty interfacing SSD with cyclone III. it seems they don't support it. any recommendation again greatly appreciated.
Regards, - Altera_Forum
I should also mention that we will download the data via Ethernet Connection. we have a Topside software which designed for this purpose. We currently using 10Mb/s Ethernet for this which is nightmare in term of speed. we will impermanent the Gb Ethernet as well.
Daixiwen, you are absolutely right.Nios II is another bottle neck as well. However, with different techniques, we are coping with it. if we make sure that our storage is responsive enough, we can have multiple Nios for this purpose. - Altera_Forum
I did a study of storage solutions for a rugged system a few years while ago, and indeed one of the reasons why we chose the S-Tec compact flash card was the much easier to implement PATA interface.
There seems to be a sata host core (http://www.altera.com/products/ip/iup/additional_functions_iup/m-intelliprop-sata-host.html) available, apparently you'll either need a FPGA with integrated transceivers, or use an external serdes. I'm sure that if you contact IntelliProp they can give you more information about what is supported. I'm also rather sceptical about the SATA connector under strong vibrations, it looks rather fragile. There could be a rugged version now, but there wasn't at the time I did the research. - Altera_Forum
IDE CF card looks like the best options.
Some more random thoughts: There are SATA cables with clips. But they might not be good enough for the needs. Another option: solder the flash chips directly on the board and use a Flash file system (JFFS2 or YAFFS2). Yet another option: consider more mechanically rugged connection options. For example, Ethernet over AUI uses connectors with screws. - Altera_Forum
Many thanks for fantastic comments. They are really helpful and I learn a lot from them. There is no doubt that CF is the best option. However, we are suffering from performance issue. We have to use FAT32 file system due to customer expectation and by using file system, Nios II processor is not capable of writing data to CF card with that speed. We are currently using Altera CF IP core in True IDE mode. Do you think if we write our own CF IP core, it is going to be faster? Have you guys ever used CF core with that performance? Any comment and help on that greatly appreciated.
- Altera_Forum
I'd check the raw CF write speed to see if you can actually write data fast enough.
Try with writes of multiple sectors (single data xfer for each sector) and look to see if the CF supports multi-sector writes (single data xfer for (say) 4 sectors). IIRC the CF ATA command set also includes an 'erase sectors' command, that may have an effect, and may make writes faster - I didn't try it when I wrote CF support (many years ago). The CF ATA command sequence using PIO is fairly simple so shouldn't be that difficult to test. You will get faster writes using DMA (either using dma mode transfers, or pio mode using dma for the data copy) - but you can probably work out how much that might save by workign out how long the copy itself takes. FAT16 is slightly simpler than FAT32, if you use large clusters the you can get quite big files. Writing FAT16 code that is specific for your application (rather than using the generic FAT code) should let you get filesystem writes near to the physical write speed limit. - Altera_Forum
thanks for the comment. I think we will try to implement out own CF IP which supports UDMA transfer. we will try this. hope, Nios II performance is enough for our application. otherwise we have to do some modifications on FAT32 code to achieve that speed. has anybody have an experience of implementing UDMA? any clue or document to start with is appreciated. I looked at CF specifications and it seems a little bit complicated at this stage. however, I will do my best to implement it. if you know any relevant documents regrading UDMA, can you please kindly let us know. many thanks.
