Forum Discussion

Honored Contributor

13 years ago

Can I upload Nios2 SMP system?

Hi, guys & Altera corp.

I'm now making a Nios2 SMP system for my research purpose. It's still a little bit buggy and slow, but I succeeded to boot Linux kernel and execute bash.


Linux version 2.6.30 (hamada@Messiah2) (gcc version 4.1.2 (Wind River Linux Sour 
cery G++ 4.1-176))# 1915 SMP Tue Sep 4 18:16:32 JST 2012 
console  enabled 
Early printk initialized 
 
 
Linux/Nios II-MMU 
Altera Nios II-MMU support (C) 2004 Wind River Systems. 
init_bootmem_node(?,0x3d0, 0x0, 0x8000) 
free_bootmem(0x3d0000, 0x7c30000) 
reserve_bootmem(0x3d0000, 0x1000) 
Detected 1 available secondary CPU(s) 
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32512 
Kernel command line: kgdboc=ttyS0, 115200 kgdbwait 
NR_IRQS:32 
PID hash table entries: 512 (order: 9, 2048 bytes) 
Console: colour dummy device 80x25 
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes) 
Inode-cache hash table entries: 8192 (order: 3, 32768 bytes) 
We have 32768 pages of RAM 
Memory available: 125824k/3902k RAM, 0k/0k ROM (1457k kernel code, 2445k data) 
Calibrating delay loop... 19.55 BogoMIPS (lpj=97792) 
Mount-cache hash table entries: 512 
CPU1: Booted secondary processor 
Calibrating delay loop... 19.96 BogoMIPS (lpj=99840) 
Brought up 2 CPUs 
SMP: Total of 2 processors activated (39.52 BogoMIPS). 
init_BSP(): registering device resources 
bio: create slab <bio-0> at 0 
msgmni has been set to 246 
io scheduler noop registered 
io scheduler anticipatory registered 
io scheduler deadline registered 
io scheduler cfq registered (default) 
ttyJ0 at MMIO 0xa60a440 (irq = 2) is a Altera JTAG UART 
console handover: boot  -> real  
ttyS0 at MMIO 0x8000060 (irq = 3) is a Altera UART 
ifconfig: socket: Function not implemented 
ifconfig: socket: Function not implemented 
Welcome to 
          ____ _  _ 
         /  __| ||_| 
    _   _| |  | | _ ____  _   _  _  _ 
   | | | | |  | || |  _ \| | | |\ \/ / 
   | |_| | |__| || | | | | |_| |/    \ 
   |  ___\____|_||_|_| |_|\____|\_/\_/ 
   | | 
   |_| 
 
For further information check: 
http://www.uclinux.org/ 
 
Why came here? CPU0, task inetd pte c71f4c40, entry 07a0704b, address 2ab10000 
 
 
BusyBox v1.14.2 (2012-06-26 16:39:29 JST) hush - the humble shell 
Enter 'help' for a list of built-in commands. 
 
/#  ls 
bin   etc   init  mnt   root  sys   usr 
dev   home  lib   proc  sbin  tmp   var 
/#  bash #  ls -lp 
drwxr-xr-x    2 root     root            0 Sep  4  2012 bin/ 
drwxr-xr-x    6 root     root            0 Sep  4  2012 dev/ 
drwxr-xr-x    5 root     root            0 Sep  4  2012 etc/ 
drwxr-xr-x    3 root     root            0 Sep  4  2012 home/ 
lrwxrwxrwx    1 root     root           10 Sep  4  2012 init -> /sbin/init 
drwxr-xr-x    3 root     root            0 Sep  4  2012 lib/ 
drwxr-xr-x    2 root     root            0 Sep  4  2012 mnt/ 
dr-xr-xr-x   34 root     root            0 Nov 30 00:00 proc/ 
drwxr-xr-x    2 root     root            0 Sep  4  2012 root/ 
lrwxrwxrwx    1 root     root            3 Sep  4  2012 sbin -> bin/ 
drwxr-xr-x   11 root     root            0 Nov 30 00:00 sys/ 
drwxr-xr-x    2 root     root            0 Nov 30 00:01 tmp/ 
drwxr-xr-x    5 root     root            0 Sep  4  2012 usr/ 
drwxr-xr-x    7 root     root            0 Nov 30 00:01 var/ #  cat /proc/cpuinfo 
CPU:         NIOS2 MultiCore 
MMU:            ways:16 entries:512 
FPU:            none 
Clocking:       <not supported> 
BogoMips:       19.96 
Calibration:    9984000 loops 
CPU:         NIOS2 MultiCore 
MMU:            ways:16 entries:512 
FPU:            none 
Clocking:       <not supported> 
BogoMips:       19.96 
Calibration:    9984000 loops #  cat /proc/interrupts 
           CPU0 
  0:      13931     NIOS2-INTC  timer 
  2:        133     NIOS2-INTC  JTAGUART 
  3:          0     NIOS2-INTC  UART 
 30:       4875     NIOS2-INTC  IPI 0 
 31:      17375     NIOS2-INTC  IPI 1 #  cat /proc/stat 
cpu  124 0 27259 1939 0 0 2 0 0 
cpu0 54 0 13448 1183 0 0 2 0 0 
cpu1 70 0 13811 756 0 0 0 0 0 
intr 38284 14687 0 163 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 515 
9 18275 
ctxt 33554 
btime 943920000 
processes 674 
procs_running 2 
procs_blocked 0 #

The cpu core is a clone of the genuine Nios2/f core and almost all features are implemented except the details of 1st data cache.

If anyone has interest, I want to upload these to 'Altera Wiki', but it's a problem because the 'cpu' is a clone and Altera corp. has their copyright for Nios2's instruction set and its architecture. If Altera corp. kindly permit me to upload all including hardware's source codes, this is the best way. But if not so, how can we share these result?

Kazu

multiple-attachments.zip23 KB

15 Replies

Altera_Forum
Honored Contributor
13 years ago
How are you getting around the problem that the nios has no locked bus cycles - so you can't implement any mutex or other atomic operations into normal memory?
Typically these need a minimum of a locked 'compare and exchange' instruction - which the avalon bus doesn't support.

If you've re-implemented nios, you might notice that nios is basically a reimplementation of MIPS :-)
Altera_Forum
Honored Contributor
13 years ago
Its not possible to do an SMP Linux system on Altera NIOS (see several discussions on this). Altera would need to implement SMP-save atomic instructions first. (See e.g. ARM's "load locked", "store conditional" instructions for a decent way to do this.)

I understand that you created a NIOS compatible CPU yourself. Of course here you can in fact implement such instructions, but I feel that a NIOS clone (done in Verilog or whatever) will be much slower than an Altera branded thingy that uses low-level optimizations that Verilog and friends don't provide.

I agree that implementing a MIPS clone seems more appropriate than implementing a NIOS clone There are some free 32 Bit CPUs in Verilog code available in the Net.

Of course cache synchronization is a huge task to do.

-Michael
Altera_Forum
Honored Contributor
13 years ago
Hi,

--- Quote Start ---
How are you getting around the problem that the nios has no locked bus cycles - so you can't implement any mutex or other atomic operations into normal memory?
Typically these need a minimum of a locked 'compare and exchange' instruction - which the avalon bus doesn't support.

--- Quote End ---

I implemented an instruction which swaps the values between the register and memory operand in atomic, like

swap ra, imm16
.

Spinlocks are using this. And the 'compare and exchange' instruction is implemented with the combination of spinlocks like

static inline int atomic_cmpxchg(atomic_t *v, int old, int new) { int ret; unsigned long flags; _atomic_spin_lock_irqsave(v, flags); ret = v->counter; if (likely(ret == old)) v->counter = new; _atomic_spin_unlock_irqrestore(v, flags); return ret; }
in the Linux kernel.

--- Quote Start ---

If you've re-implemented nios, you might notice that nios is basically a reimplementation of MIPS :-)
--- Quote End ---

Of course, I know that Nios2 is a copy of ....:D

--- Quote Start ---

I understand that you created a NIOS compatible CPU yourself. Of course here you can in fact implement such instructions, but I feel that a NIOS clone (done in Verilog or whatever) will be much slower than an Altera branded thingy that uses low-level optimizations that Verilog and friends don't provide.

--- Quote End ---

Of course, clone core's fmax is a big problem if you want to achieve better performance than single core case. I compiled the source for my DE2-115 with switches 'less Optimizations' and 'Fast fit', and got the fmax result around 60MHz. By optimizing the details and compilation switches, may be I can get 75~80MHz, but over 100MHz is impossible.

--- Quote Start ---

I agree that implementing a MIPS clone seems more appropriate than implementing a NIOS clone There are some free 32 Bit CPUs in Verilog code available in the Net.

--- Quote End ---

Why Nios2? Because I love Nios2:) and Altera:D.

--- Quote Start ---

Of course cache synchronization is a huge task to do.

--- Quote End ---

To achieve the cache coherency, I implemented the 1st data cache as 'write through' one, and set the cache flush signals which are sent to other cpu's data cache whenever the write operations are done. If the target data has been cached in other caches, it's only flushed from the cache. The new data is filled at the next memory access.

Kazu
Altera_Forum
Honored Contributor
13 years ago
Sounds nice.

It would be great if a way could be found (maybe with Altera's help) to use the original NIOS implementation and add what's necassary.

-Michael
Altera_Forum
Honored Contributor
13 years ago
Hi,

As we discussed it before, to make a clone cpu which has atomic instructions is 'one solution' to build the Nios2 SMP system.

--- Quote Start ---

It would be great if a way could be found (maybe with Altera's help) to use the original NIOS implementation and add what's necassary.

--- Quote End ---

By this research, I understand how to make a SMP system from the view points of hardware and software sides, so I'm now seeking the way to make it by using original Nios2 processors.

Kazu
Altera_Forum
Honored Contributor
13 years ago
If you can do the locked read-write avalon bus cycle you should be able to generate one from a custom instruction - except it would have to use a separate avalon master and so bypass the data cache.

Actually you'll have no cache coherencey either - very grim!
You'd have to use an external data cache.

I've thought that the nios cpu isn't much more than a great heap of mux.
My guess is that RA and RB are always read, pipeline stalls (re-execute) if a write to RA is pending, and for RB if the low two instruction bits differ (NFI why the instructions aren't organised so this is a single bit!). This gives the instrcution three 32bit words to play with, write-back to RB or RC is dependant on the decoded instruction.
That makes me think that the readra/readrb bits of the custom instruction are ignored - but I've not done any experiments.
Altera_Forum
Honored Contributor
13 years ago
Hi,

--- Quote Start ---
If you can do the locked read-write avalon bus cycle you should be able to generate one from a custom instruction - except it would have to use a separate avalon master and so bypass the data cache.

Actually you'll have no cache coherencey either - very grim!
You'd have to use an external data cache.

--- Quote End ---

Of course, we must add some custom instructions to do locked read-write memory operations, but we can't forget the existence of MMU. About the cache coherency, we can remove the original data cache and put special one.

--- Quote Start ---

I've thought that the nios cpu isn't much more than a great heap of mux.
My guess is that RA and RB are always read, pipeline stalls (re-execute) if a write to RA is pending, and for RB if the low two instruction bits differ (NFI why the instructions aren't organised so this is a single bit!). This gives the instrcution three 32bit words to play with, write-back to RB or RC is dependant on the decoded instruction.
That makes me think that the readra/readrb bits of the custom instruction are ignored - but I've not done any experiments.
--- Quote End ---

I think that the readra/readrb bits are used in the forwarding and pipe-lock mechanism.

Kazu
Altera_Forum
Honored Contributor
13 years ago
My point about readra/readrb is that the logic that handles the pipeline stall doesn't really want to look that far into the instruction decode.
Altera_Forum
Honored Contributor
13 years ago
Hi,

--- Quote Start ---
My point about readra/readrb is that the logic that handles the pipeline stall doesn't really want to look that far into the instruction decode.
--- Quote End ---

Maybe, Altera uses embedded memories for the Nios's register files, and sets those in the same clock phase of the 'Decode Stage'. To read the contents of the embedded memories, it takes one clock, so the RA & RB field data must be sent directly from the 'Fetch Stage' to the register file. The contents of the register file are always discharged to the following stage even when those are not used. But to avoid the 'RAW' hazard, the 'Decode Stage' must decide whether the RA and RB are really used or not. I don't know whether Altera uses readra and readrb bits for this purpose or not, because the 'R0' register can be used instead of those.

Kazu

Altera_Forum

Honored Contributor

13 years ago

Hi,

I succeeded to make a Nios2 SMP system by using normal Nios2/f cores. It's still a little bit buggy, but I'm going to upload the sources and documents on Altera wiki in near future.


Linux version 2.6.30-00471-g2e1b9d6-dirty (hamada@Messiah2) (gcc version 4.1.2 (
Wind River Linux Sourcery G++ 4.1-176))# 2077 SMP PREEMPT Sun Oct 21 16:34:00 JS
T 2012
console  enabled
Early printk initialized
 
 
Linux/Nios II-MMU
Altera Nios II-MMU support (C) 2004 Wind River Systems.
init_bootmem_node(?,0x3d9, 0x0, 0x8000)
free_bootmem(0x3d9000, 0x7c27000)
reserve_bootmem(0x3d9000, 0x1000)
Detected 1 available secondary CPU(s)
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32512
Kernel command line: kgdboc=ttyS0, 115200 kgdbwait
NR_IRQS:32
PID hash table entries: 512 (order: 9, 2048 bytes)
Console: colour dummy device 80x25
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
Inode-cache hash table entsucceedries: 8192 (order: 3, 32768 bytes)
We have 32768 pages of RAM
Memory available: 125808k/3938k RAM, 0k/0k ROM (1485k kernel code, 2452k data)
Calibrating delay loop... 24.21 BogoMIPS (lpj=121088)
Mount-cache hash table entries: 512
CPU1: Booted secondary processor
Calibrating delay loop... 24.98 BogoMIPS (lpj=124928)
Brought up 2 CPUs
SMP: Total of 2 processors activated (49.20 BogoMIPS).
init_BSP(): registering device resources
bio: create slab <bio-0> at 0
msgmni has been set to 246
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
ttyJ0 at MMIO 0xa60a500 (irq = 2) is a Altera JTAG UART
console handover: boot  -> real 
ttyS0 at MMIO 0x8000060 (irq = 3) is a Altera UART
init.rc
Welcome to
          ____ _  _
         /  __| ||_|
    _   _| |  | | _ ____  _   _  _  _
   | | | | |  | || |  _ \| | | |\ \/ /
   | |_| | |__| || | | | | |_| |/    
   |  ___\____|_||_|_| |_|\____|\_/\_/
   | |
   |_|
 
For further information check:
http://www.uclinux.org/
 
 
 
BusyBox v1.14.2 (2012-09-17 02:11:26 JST) hush - the humble shell
Enter 'help' for a list of built-in commands.
 
/#  bash#  cat /proc/cpuinfo
CPU:         NIOS2 MultiCore
MMU:            ways:16 entries:512
FPU:            none
Clocking:       <not supported>
BogoMips:       24.98
Calibration:    12492800 loops
CPU:         NIOS2 MultiCore
MMU:            ways:16 entries:512
FPU:            none
Clocking:       <not supported>
BogoMips:       24.98
Calibration:    12492800 loops#  cat /proc/interrupts
           CPU0
  0:      25375     NIOS2-INTC  timer
  2:        130     NIOS2-INTC  JTAGUART
  3:          0     NIOS2-INTC  UART
 30:       1993     NIOS2-INTC  IPI 0
 31:      30282     NIOS2-INTC  IPI 1#  cat /proc/stat
cpu  102 0 41939 10290 0 0 1 0 0
cpu0 67 0 19033 7103 0 0 0 0 0
cpu1 35 0 22906 3187 0 0 1 0 0
intr 59916 26203 0 166 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 220
5 31342
ctxt 15553
btime 943920000
processes 670
procs_running 2
procs_blocked 0#

Kazu

multiple-attachments.zip307 KB

Forum Discussion

Can I upload Nios2 SMP system?

15 Replies

Recent Discussions

Nios-V on Cyclone IV

Nios II IDE File Name too long error

No valid license for Nios processor

Correct way to use mSGDMA with a NIOSV/m processor on a MAX10 FPGA

NIOS SDK SBOM/FOSS info