Semaphores/shared memory

[Jeff]

I am trying to get the hang of using semaphores for locking share memory.



I am writing a script that runs in many processes, but only one process at a time may access an external resource.  The external resource is identified by a unique string (called the slug (an old newspaper term for a story's unique id)).



I am storing the list of "locked" slugs in a shared page, and trying to control access to that page with semaphores.  I wrote a macro to automate this:

Code Select Expand

(constant 'wait -1 'sig 1 'release 0)

(setq locked (share))
(share locked "")
(setq sid (semaphore))
(semaphore sid sig)

(define-macro (with-locked-slug)
  (letex ((slug (eval (first (args)))) (body (cons 'begin (rest (args)))))
    (let ((current (list slug)) (result nil))
     
      ;; block until slug is out of currently locked slugs list
      (while (member slug current)
        (semaphore sid wait)
        (setq current (parse (share locked)))
        (semaphore sid sig))
     
      ;; lock slug
      (semaphore sid wait)
      (share locked (string (share locked) " " slug))
      (semaphore sid sig)
     
      ;; evaluate body
      (setq result body)
     
      ;; unlock slug
      (semaphore sid wait)
      (setq current (parse (share locked)))
      (pop current (find slug current))
      (share locked (join current " "))
      (semaphore sid sig)
     
      ;; return result of evaluation
      result)))

It seems to hold up (i.e. no race conditions) under a heavy load (3000 requests/100 concurrent clients/4 worker processes).



I am not particularly experienced at low-level ipc stuff.  Does anyone see a problem with this code?  Anyone have any suggestions for making it more efficient?

[lithper]

.. one thing I always do is peek into Perl documentation. Sometimes snippets and more or less appropriate explanations can be dug out in the Camel, sometimes in the Perl Cookbook. Scripts quality may vary, as well as that of the commentaries, but I found something coherent about pre-forking servers, as one example.



The Camel has a section with example code on the use of semaphores. As most of it is in the on-line perl documentation, that is the third place I'd check.



Thanks for an interesting snippet to play with ;))

[Lutz]

A simple example showing a synchronized producer->consumer situation of two concurrent processes is also show here:



http://www.newlisp.org/CodePatterns.html#processes">http://www.newlisp.org/CodePatterns.html#processes



ps: corrected the link

[Jeff]

Lutz,



I've read them.  I just wanted to have someone else look at the code and see if they noticed any deadlocks, race problems, etc., that I had missed.



Lithper,



I tend to look at C documentation over Perl.  You never know exactly how Perl implements something without going through some *really* rough source.  newLISP uses sem.h, and you can get a lot by looking through sample code online for it.



Mainly, I just wanted another pair of eyes to take a look at my code and see if I missed anything big.  I don't have much experience using semaphores to sync state.  My primary language is Python.  Due to the GIL, threads are useful primarily for IO.  I'm not accustomed to doing much locking of critical sections, and I know it can be tricky work to get it right.

[newdep]

Jeff,



looks to me like you run into an OS/Kernel limitation, if it not happens

with i.e. 80 sessions then probably you reached at 100 the memory

reserved for semphores/ipc sessions on your OS. This is configurable btw..



"sysctl -a" shows the defaults addons go into /etc/sysctl.conf



Norman.

[Jeff]

What do you mean?  The semaphores are used as binary semaphores.  The number of processes accessing this function will be limited to a sane number based on the number of cpus on the system.  This is just to make sure they have exclusive access when altering the share.  Also:

Code Select Expand

sysctl -a | grep sem.max
kern.posix.sem.max: 10000

...is taht what you are referring to?  I think that is the largest chunk of semaphores you can allocate at any one time.  Semaphores are generally handed out as an array, rather than one at a time.

[newdep]

I cant think of something else then kernel tuning here...

..because you dont have problems else...

but 100 session over 4 sessions is 25 ..is not much.. Could be a shared

memory issue too? shmem etc..

There was once a bug in linux regarding async timeouts in semaphores

(kernel 2.4) but i realy dont know if that for bsd too..



Im out of recourses.. ;-)

[Jeff]

I'm not having any problems with the code; just wanted a second look to see if anyone else noticed any obvious race or lock issues.

[Lutz]

Discovered these guidelines for sharing resources with semaphores:



http://homepages.uel.ac.uk/u0214248/Harriet2.htm">http://homepages.uel.ac.uk/u0214248/Harriet2.htm



even algorithms for analyzing and avoiding deadlocks/raceconditions (Banker's Algorithm). Many useful links at the end of the article.