Text only | Text with Images

newLISP Fan Club

Forum => newLISP in the real world => Topic started by: Jeff on September 18, 2007, 09:44:00 AM

Title: Max threads
Post by: Jeff on September 18, 2007, 09:44:00 AM
I am throwing an error when starting 72 or more threads on OSX in the latest dev version (9.2 does the same).  It happens regardless of the expression I am forking.  I'm on a pretty powerful machine- there should be no way it's running out of memory.  The default max memory being used is 4 gigs (which is the ram in the machine) and setting the stack size makes no difference (and probably shouldn't).



Any ideas?
Title:
Post by: Jeff on September 18, 2007, 09:49:18 AM
Ok, it's not always 72, but it doesn't go much higher, and it seems like even the simplest operations (like incrementing an integer) die around there.
Title:
Post by: jrh on September 18, 2007, 10:35:52 AM
The default number of procs per user session in OS X is 100.  There is a configuration parameter maxprocperuid in a .conf file over in /etc (i forget which) that you can set to override this limit.
Title:
Post by: Jeff on September 18, 2007, 11:28:19 AM
Thanks!  That fixed it (well, that and a wrapper for WindowServer that lets it run with a higher maxproc as well).
Title:
Post by: jrh on September 18, 2007, 11:50:28 AM
There are a number of subsystems that have configurable settings at startup.  You can run the command sysctl -a to see all of them.



Everything is called from the /etc/rc file and it is usually a good idea to place your custom changes in the appropriate file called by rc such as (in this instance) sysctl.conf rather than editing rc as it is one of those files that may be overwritten by Apple during a software update.
Title:
Post by: Jeff on September 18, 2007, 11:59:00 AM
The problem is that the window server apparently is setting ulimit -u 100 when it starts up, and anything run through the windowserver (including procs run by TextMate, the editor I use) has to obey that.
Title:
Post by: jrh on September 18, 2007, 12:11:01 PM
You can stick the command ulimit -u 256 (or whatever number) in bashrc (or lower) and then all user privilege level shells and child processes will inherit that limit for the lifetime of the process.
Title:
Post by: Jeff on September 18, 2007, 12:20:06 PM
What about processes launched through WindowServer?  If TextMate launches a process, doesn't it inherit its environment from WindowServer's?  Additionally, does WindowServer even run from bash?
Title:
Post by: jrh on September 18, 2007, 12:36:09 PM
Hmmmm..  It looks like WindowServer is started by mach_init way before any global shell stuff is executed.  Just check to make sure your custom wrapper hasn't been nuked after a software upgrade.



So does OS X distribute your forked procs over all the processors?  :-)
Title:
Post by: Jeff on September 18, 2007, 12:56:58 PM
Yep.  For small ops it is completely unhelpful, though, because of the serialization overhead.  But for something like writing a large number of files (the specific purpose I am using it for) it should make a nice difference.
Text only | Text with Images