Forum Discussion

Honored Contributor

20 years ago

kernel init problem

We have developed our own board with an Altera Stratix II and NIOS II Standard CPU. We download the kernel (2.6.16.11) to the SDRAM (32MB) using nios2-download, and nios2-terminal gives us the following output:

uClinux/Nios II

KERNEL -> TEXT=0x04000000-0x04173da8 DATA=0x04173da8-0x04182000 BSS=0x042520a0-0

x042520cf

KERNEL -> MEM=0x4253000-0x6000000 STACK=0x6000000-0x6000000

No Command line passed

Done setup_arch

Built 1 zonelists

Kernel command line:

trap_init reached

init_IRQ done

PID hash table entries: 256 (order: 8, 4096 bytes)

oom-killer: gfp_mask=0xd0, order=0

Stack from 04181e74:<0>

<0> 00000000<0> 04037d8c<0> 00000000<0> 00000042<0> 041765b0<0> 041765b0<0> 00000000<0> 00000000<0>

<0> 000000d0<0> 04173db0<0> 00000000<0> 0403937c<0> 00000000<0> deadbeef<0> deadbeef<0> 04181f04<0>

<0> deadbeef<0> 00000000<0> 00000000<0> 000000d0<0> 00000000<0> 04181f24<0> 040393e0<0> 04159b38<0>

<0> 00000000<0> 04159b38<0> 0406faec<0> 0424713e<0> deadbeef<0> deadbeef<0> 00000000<0> 04159b38<0>

<0> 00000000<0> 04159b30<0> 040706a0<0> 04181f40<0> 00000000<0> deadbeef<0> deadbeef<0> deadbeef<0>

<0> ffffffff<0> 00000000<0> deadbeef<0> 04000654<0> 00000000<0> 00000001<0> 00000000<0> 00000001<0>

Call Trace:<0>

<0> [<04005728>]<0> [<04004c0c>]<0> [<04005740>]<0> [<04000110>]<0>

Mem-info:

DMA per-cpu:

cpu 0 hot: high 6, batch 1 used:0

cpu 0 cold: high 2, batch 1 used:0

DMA32 per-cpu: empty

Normal per-cpu: empty

HighMem per-cpu: empty

Free pages: 0kB (0kB HighMem)

Active:0 inactive:0 dirty:0 writeback:0 unstable:0 free:0 slab:0 mapped:0 pagetables:0

DMA free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:32768kB pages_scanned:0 all_unreclaimable? no

lowmem_reserve[]: 0 0 0 0

DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no

lowmem_reserve[]: 0 0 0 0

Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no

lowmem_reserve[]: 0 0 0 0

HighMem free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no

lowmem_reserve[]: 0 0 0 0

DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB

0*4096kB = 0kB

DMA32: empty

Normal: empty

HighMem: empty

0 pages of RAM

0 free pages

0 reserved pages

0 pages shared

0 pages swap cached

Kernel panic - not syncing: Out of memory and no killable processes...

We have found out that this has something to do with the call to local_irq_enable() in init/main.c (around line 500). If we comment out this line, the execution continues and stops after the "Memory available: " line. Reported memory on this line is correct.

We have a Altera Stratix II with Nios II evaluation board, and the kernel runs fine on this board.

We have successfully run memory tests using the memtest example included in the Nios II 6.0 examples.

Our board also have an SMC91111 ethernet controller.

Anyone has any ideas on how to solve this problem?

Regards

Hein Gustavsen

12 Replies

Altera_Forum
Honored Contributor
20 years ago
yes, it is irq related.

Note, the interrupt vector (ie, exception vector) should be in uClinux program memory, ie sdram. Note in Linux, irq 0 used to be auto-detected and that is not supported in Nios II, so you should not use irq 0 for your devices. Only timer_0 may use irq 0.

so you have to update in sopc builder, change exception vector to sdram.
and assign irq of smc91111 to non-zero value. Then regenerate the new sof.

see,
http://nioswiki.jot.com/wikihome/operating...ms/%c2%b5clinux (http://nioswiki.jot.com/wikihome/operatingsystems/%c2%b5clinux)
Altera_Forum
Honored Contributor
20 years ago
Thank you. This solved our problem.

Hein

--- Quote Start ---
originally posted by hippo@Jul 12 2006, 03:37 PM
yes, it is irq related.

note, the interrupt vector (ie, exception vector) should be in uclinux program memory, ie sdram. note in linux, irq 0 used to be auto-detected and that is not supported in nios ii, so you should not use irq 0 for your devices. only timer_0 may use irq 0.

so you have to update in sopc builder, change exception vector to sdram.
and assign irq of smc91111 to non-zero value. then regenerate the new sof.

see,
http://nioswiki.jot.com/wikihome/operating...ms/%c2%b5clinux (http://nioswiki.jot.com/wikihome/operatingsystems/%c2%b5clinux)
<div align='right'><{post_snapback}> (index.php?act=findpost&pid=16833)
--- quote end ---

--- Quote End ---

Altera_Forum

Honored Contributor

20 years ago

Hi Heing and Hippo,

I need some more understanding on this topic,

If my nios2_system.h has following

<div class='quotetop'>QUOTE </div>

--- Quote Start ---

----------------------------------------------------------------------------

# define na_high_res_timer 0x809209e0# define na_high_res_timer_irq 3

# define na_sys_clk_timer 0x80920940# define na_sys_clk_timer_irq 0

/* Redefining high_res_timer -> timer0 */# undef na_high_res_timer# undef na_high_res_timer_irq

# define na_timer0 ((np_timer*) 0x809209e0)# define na_timer0_irq 3

------------------------------------------------------------------------------------------[/b]

--- Quote End ---

Does this means even na_sys_clk_timer_irq number is 0 (Zero), na_sys_clk_timer_irq is not been used as timer0.

And due to same reason I am facing following crash

Linux version 2.6.11-uc0 (root@rdvd3) (gcc version 3.4.6)# 9 Thu Jul 13 18:29:12
 IST 2006

uClinux/Nios II
Altera Nios II support (C) 2004 Microtronix Datacom Ltd.
setup_arch: No persistant network settings signature at 007F0000
Built 1 zonelists
Kernel command line:
PID hash table entries: 128 (order: 7, 2048 bytes)
Dentry cache hash table entries: 4096 (order: 2, 16384 bytes)
Inode-cache hash table entries: 2048 (order: 1, 8192 bytes)
Memory available: 14296k/16384k RAM, 0k/0k ROM (1408k kernel code, 486k data)
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
scheduling while atomic: swapper/0x04000002/0
Stack from 0116fdf0:<0>
       <0> 04000002<0> 01149fa4<0> 01162438<0> 011620c8<0> 01162434<0> 01160034<
0> 00000000<0> 000000d0<0>
       <0> 00000001<0> 00000000<0> 04000000<0> ffffe000<0> 0116fe28<0> 0114b1a4<
0> 01162438<0> 00000000<0>
       <0> 00000000<0> 0102fae0<0> 00000000<0> 00000000<0> 00000010<0> 00000010<
0> 00000000<0> fffffff4<0>
       <0> 011cd0d0<0> 00000b00<0> ffffe000<0> 01160034<0> 00000b00<0> 0120ad60<
0> 00000001<0> 0102fcec<0>
       <0> 01009568<0> 0121e7e0<0> 00000010<0> 00000001<0> 000000d0<0> ffffffff<
0> 0116ff28<0> 00000000<0>
       <0> deadbeef<0> deadbeef<0> 00000000<0> 00000000<0> 0116ff28<0> 00000001<
0> 00000b00<0> ffffffff<0>
Call Trace:<0>
       <0> <0> <0> <0> <0>
       <0> <0> <0> <0> <0>
       <0> <0> <0> <0> <0>
       <0> <0> <0> <0> <0>
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:       14220kB (0kB HighMem)
Active:0 inactive:0 dirty:0 writeback:0 unstable:0 free:3555 slab:21 mapped:0 pa
getables:0
DMA free:14220kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:16384k
B pages_scanned:0 all_unreclaimable? no
lowmem_reserve: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pag
es_scanned:0 all_unreclaimable? no
lowmem_reserve: 0 0 0
HighMem free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pa
ges_scanned:0 all_unreclaimable? no
lowmem_reserve: 0 0 0
DMA: 2097664*4kB 1*8kB 0*16kB 0*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2
048kB 3*4096kB = 8404872kB
Normal: empty
HighMem: empty
Kernel panic - not syncing: Out of memory and no killable processes...

OR some times I get following bootup crash if I change vmlinux size (add or remove files from my file system, or add-remove any device support)

oom-killer: gfp_mask=0xd0
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:       12148kB (0kB HighMem)
Active:0 inactive:18755125 dirty:0 writeback:0 unstable:0 free:3037 slab:287 map
ped:0 pagetables:0
DMA free:12148kB min:512kB low:640kB high:768kB active:0kB inactive:75020500kB p
resent:16384kB pages_scanned:457899 all_unreclaimable? no
lowmem_reserve: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pag
es_scanned:0 all_unreclaimable? no
lowmem_reserve: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:
0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve: 0 0 0
DMA: 1*4kB 2*8kB 2097664*16kB 1*32kB 1*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 1*2
048kB 2*4096kB = 33574772kB
Normal: empty
HighMem: empty
oom-killer: gfp_mask=0xd0
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:       12148kB (0kB HighMem)
Active:0 inactive:18755125 dirty:0 writeback:0 unstable:0 free:3037 slab:287 map
ped:0 pagetables:0
DMA free:12148kB min:512kB low:640kB high:768kB active:0kB inactive:75020500kB p
resent:16384kB pages_scanned:457899 all_unreclaimable? no
lowmem_reserve: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pag
es_scanned:0 all_unreclaimable? no
lowmem_reserve: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:
0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve: 0 0 0
DMA: 1*4kB 2*8kB 2097664*16kB 1*32kB 1*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 1*2
048kB 2*4096kB = 33574772kB
Normal: empty
HighMem: empty
oom-killer: gfp_mask=0xd0
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
Normal per-cpu: empty
HighMem per-cpu: empty
.
.
..
.
.

Is this crash none other than because of same reason.

waiting for reply,

thanks,

Ankur

Altera_Forum
Honored Contributor
20 years ago
it is OK to use irq0 for timer, as there is no auto irq detection in the driver.
but hwselect use hires timer for system timer, which may be not what you want.
your problem may be the execption vector, it must be at sdram. check it in sopc builder.
Altera_Forum
Honored Contributor
20 years ago
--- Quote Start ---
originally posted by hippo@Jul 14 2006, 01:33 AM
it is ok to use irq0 for timer, as there is no auto irq detection in the driver.
but hwselect use hires timer for system timer, which may be not what you want.
your problem may be the execption vector, it must be at sdram. check it in sopc builder.
<div align='right'><{post_snapback}> (index.php?act=findpost&pid=16887)
--- quote end ---

--- Quote End ---

Exception vector is in sdram at offset 0x0000_0020.
Sdram size is 16 MB, I think its enough memory to run kernel with minimal initramfs support + network, flash, jffs2, ps/2 kybd-mouse, and one my added device driver for fb ( crash comes with / without this driver). No apps till now.

flash and network are enable as per nioswiki.

thanks,
Ankur
Altera_Forum
Honored Contributor
20 years ago
try remove hires timer in sopc builder.
Altera_Forum
Honored Contributor
20 years ago
--- Quote Start ---
originally posted by hippo@Jul 14 2006, 02:49 AM
try remove hires timer in sopc builder.
<div align='right'><{post_snapback}> (index.php?act=findpost&pid=16889)
--- quote end ---

--- Quote End ---

my hires timer is 1ms and sys_clk_timer is 10ms, does it cause any effect,
Is there any way to force use sys_clk_timer as timer0.

thanks,
Ankur
Altera_Forum
Honored Contributor
20 years ago
--- Quote Start ---
originally posted by ankur+jul 14 2006, 03:56 pm--><div class='quotetop'>quote (ankur @ jul 14 2006, 03:56 pm)</div>
--- quote start ---
<!--quotebegin-hippo@Jul 14 2006, 02:49 AM
try remove hires timer in sopc builder.
<div align='right'><{post_snapback}> (index.php?act=findpost&pid=16889)
--- quote end ---

--- Quote End ---

my hires timer is 1ms and sys_clk_timer is 10ms, does it cause any effect,
Is there any way to force use sys_clk_timer as timer0.

thanks,
Ankur
<div align='right'><{post_snapback}> (index.php?act=findpost&pid=16890)</div>
[/b]
--- Quote End ---

the sys clk timer should be a full featured timer.
the kernel will program the timer period.

first try if it works without hires timer.
then you need to modify the perl scripts called from arch/nios2nommu/scripts/hwselect.pl .
Altera_Forum
Honored Contributor
20 years ago
the sys clk timer should be a full featured timer.
the kernel will program the timer period.

first try if it works without hires timer.
then you need to modify the perl scripts called from arch/nios2nommu/scripts/hwselect.pl .
<div align='right'><{post_snapback}> (index.php?act=findpost&pid=16893)</div>

--- Quote End ---

I change sys_clk_timer location in sopc builder (moved up) and now my timer irq is 0 ie same as sys_clk_timer.

I think which ever timer we will specify first in sopc builder will become timer0.

but but but
Still I am facing the crash....
If I reduce size of my filesystem (removing some files or remove devices support) and booting comes upto busybox prompt# but once I do "ls" system hangs.....

I know if further I reduce kernel elf size, it will boot properly as earlier.

Some memory figures
--------------------------
My vmlinux (elf executable) size is coming something around 13 MB
also vmlinux.srec (srec) size is coming around 5 MB

For downloading vmlinux into sdram I am using "nios2-download -g vmlinux" command. Once download completes it gives me 1802 KB download confirmation.
SDRAM is 16MB.

Hippo is it ok with respect to nios memory requirments for running kernel. You are more experienced person among us : )
Heing, how big is your kernel size can you share with us ???

Any idea where to head from here !!!!

Ankur
Altera_Forum
Honored Contributor
20 years ago
first, about the kernel size.
elf is big, because it contains a lot of debug symbols.
you can strip off the debug info, and the striped size will be around 2MB in your case.
the srec is ascii hex, so it will be more than two times of the binary.
the binary size is about 1.8MB in your case.
if you compress the kernel with zImage or (gzip + u-boot) , it will be around 1MB.

it looks like problem of kernel size?
I had kernel of 3.6MB binary, but did not have such problem.

check usr/initramfs_list

my suggestions,
1. always start with minimal hardware, and minimal kernel config .
2. then add more stuff step by step .
I know this takes a long time. but this is what I do when I faced problem like this.

did you enable large memory >1MB allocation in kernel config?

try add some debug messages in init/initramfs.c .

other stuff that might (or might not) be helpful,
1. zImage
2. stand alone init, sash

Forum Discussion

kernel init problem

12 Replies

Recent Discussions

Run-time errors using EMIF in Agilex 5 NIOS-V design

Nios V/g load-writeback hazard

Ashling RISC Free IDE fails to download ELF file

Dual Port Nios V BSP error

set_output_delay explained for dummies