Forum Discussion

Altera_Forum's avatar
Altera_Forum
Icon for Honored Contributor rankHonored Contributor
14 years ago

Interprocess Locking / Atomic Operations Issue

I'm running MMU linux on a Nios2 core. I have a system running many applications accessing a shared memory space.

For synchronization, I'm using sem_open, sem_post, sem_wait calls. However, my applications were crashing because multiple threads/processes somehow all thought they had the lock at the same time....

After a lot of digging and debugging, I found that glibc implementation of these functions relies on atomic operations in userspace. Since there is no architecture support for atomic test-and-set-like operations, it falls back to non-atomic operations implemented in C (glibc/bits/atomic.h).

Has anyone successfully used an interprocess locking mechanism from userspace? I've tried these semaphore operators along with pthread_mutex objects in shared memory space, but none are without flaws.

Is there any fallback that I have? Even Linux's futex() relies on user-space atomic operations on a shared counter...

1 Reply

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    14 years ago

    For any architecture with MMU:

    The non-pthreadlib libc operations (inter-process rather than inter-thread communication) should not do user-space locking in any way, as different processes have different address spaces and this can't work. Only pthreadlib might use atomic operations in user space in combination with the FUTEX system calls.

    NIOS:

    The NIOS Processor does not support any atomic instructions and no cache-sync mechanism, so Hardware-atomicness (e.g. for multiple processors) is not possible, but with using dedicated hardware semaphores (such design is provided by Altera, you need to implement an instance for each semaphore).

    Software-Atomicness in Kernel space can be (and is, AFAIK) achieved by temporarily disabling the interrupt.

    Thread-atomicness in a non-MMU system (e.g. for doing a FUTEX) can be achieved by temporarily disabling the interrupt.

    Thread atomicness in user space with MMU enabled systems (e.g. for doing a FUTEX) can be achieved by implementing an "atomic region" mechanism in the Kernel. But AFAIK, this is not yet done for NIOS. (It is available e.g. for the BlackFin processor and can be done for NIOS in a similar way.)

    AFAIK, the usual Interprocess locking system calls are provided as usual.

    The GLIBC pthreadlib MUTEX function only uses FUTEX (and thus atomic instructions in user space) when the system, libc is compiled for, does support FUTEX. As with the current NIOS implementation, FUTEX is not supported, it would be an error if libc would try to use it. It needs to fall back to using the (in average much slower) Kernel MUTEX calls instead.

    -Michael