Important changes to Makefiles for OS X

[itistoday]

NOTE: This post is in reference to newlisp version 10.1.4.



This may not be an issue for some people, but for anyone (e.g. me) using "link.lsp", this is potentially serious, and actually could be serious for everyone if the machine used to generate the newlisp binaries runs on 10.6.



The current Makefiles for OS X compile against the system's libraries, and this means that the binaries they generate will likely not run on older systems. I know that this is definitely true if you've compiled newlisp on 10.6.  A version of newlisp compiled on 10.6 won't run on any older system, you'll get this mysterious error:

Code Select Expand

dyld: unknown required load command 0x80000022

I've modified the 'makefile_darwin_universal_utf8' to make it compile against the 10.5 SDK.  Whether this means it will run on 10.4 I'm not sure though. But it would be easy enough to modify it to get it to run on 10.4.



Here's my modified Makefile (it includes some of the stuff from the existing Makefile):

Code Select Expand

# makefile for newLISP v. 10.x.x on Mac OSX with readline support
#
# for OS X 10.4/Tiger, Tiger comes standard with libreadline
#
# needs readline library and headerfiles installed (X tools)
# use the Mac OS X lipo utitlity to extract one or the other ppc/intel
# example: lipo newlisp-universal -output newlisp-ppc -thin ppc7400
# note: architecture id for ppc on Tiger: ppc on Leopard: ppc7400

OBJS = newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o
nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o nl-utf8.o pcre.o

CFLAGS = -Wall -arch i386 -arch ppc -O2 -c -g -DREADLINE -DMAC_OSX -DSUPPORT_UTF8 -isysroot /Developer/SDKs/MacOSX10.5.sdk
#CFLAGS = -Wall -I/usr/include -c -g -DREADLINE -DMAC_OSX

default: $(OBJS)
$(CC) $(OBJS) -mmacosx-version-min=10.5 -arch i386 -arch ppc -g -lm -lreadline -o newlisp-universal
strip newlisp-universal
# lipo newlisp-universal -output newlisp-intel -thin i386
# lipo newlisp-universal -output newlisp-ppc -thin ppc

.c.o:
$(CC) $(CFLAGS) $<

$(OBJS): primes.h protos.h makefile_darwin_universal_utf8

Two important notes:



1) When compiling the object files, you *must* include:

Code Select Expand

-isysroot /Developer/SDKs/MacOSX10.5.sdk

On my machine changing that to the 10.4u SDK resulted in compile errors, but those might be fixable, or it may not be necessary.



2) When linking all the object files together, you *must* include:

Code Select Expand

-mmacosx-version-min=10.5

Again, that should probably be changed to say '10.4'.

[TedWalther]

Would you care to checkout the git archive, then submit a diff?  I'll apply it to my tree until Lutz can review and incorporate it.



Run the following command:

Code Select Expand


git clone http://reactor-core.org/newlisp/.git


[itistoday]

Sorry, I'm not really too familiar with git (I use bzr, and even then haven't gotten around to learning all of it), otherwise I would. Shouldn't it be simple for you to copy/paste from what I've posted?

[Lutz]

When newLISP is packaged for Intel and PPC, a universal version is made first on a Machine running Tiger 10.4. There are two reasons for this:



1) Leopard 10.5 compiled versions are not backward compatible with 10.4 libraries.



2) libc.dylib in 10.4 still recognizes the "%Y%m%d" format in 'parse-date' as Linux and other BSD flavors do too, but Leopard 10.5 doesn't recognize a format without separators between the year/month/date specs.



So these are the reasons we have a 'makefile_darwin_universal_utf8' used for Tiger 10.4 not 10.5. As it happens on a different machine setup, it is convenient to build a universal version once, then copy it to a Leopard 10.5 environment to make the dmg packages. The packaging scripts then use the 'lipo' utilitiy to extract either the Intel or PPC version from newlisp-universal.

[itistoday]

What I mean is that the current makefile isn't really tuned for any specific system, it's just the "default" for wherever it's compiled. It's possible using the modifications I mentioned above, to change the makefile so that it creates a binary that works on 10.4, but can be compiled on any system after it (provided they have the 10.4 SDK installed).



This way, if you want to upgrade the system that's compiling newlisp you can, and people who use "link.lsp" don't have to be scared of it not working to someone they send it to.

[itistoday]

OK, I seem to have given you a poor reply, and now there is just confusion.



Let me try one more time:

When newLISP is packaged for Intel and PPC, a universal version is made first on a Machine running Tiger 10.4. There are two reasons for this:



1) Leopard 10.5 compiled versions are not backward compatible with 10.4 libraries.

If you build them correctly, they are.

2) libc.dylib in 10.4 still recognizes the "%Y%m%d" format in 'parse-date' as Linux and other BSD flavors do too, but Leopard 10.5 doesn't recognize a format without separators between the year/month/date specs.

I've managed to rewrite the Makefile to get it to compile for 10.4 on 10.6 (turned out to be a bit more effort than getting it to compile for 10.5).  I tested the parse-date thing with your 10.1.5 ppc build that's on the newLISP site:

Code Select Expand

(parse-date "20071007" "%Y%m%d") => 1191715200

My newlisp-universal that I compiled on 10.6 for 10.4 produces the same output, and it runs without a problem on 10.4 ppc.



Here is my updated makefile, it will compile a universal 10.4+ compatible version of newlisp on 10.5 or greater. Note that the 10.4 SDK *must* be installed. Could you include this with newlisp's source? Perhaps rename it "makefile_darwin_universal_uft8_compat" or something like that?

Code Select Expand

# makefile for newLISP v. 10.x.x on Mac OSX with readline support
#
# for OS X 10.4/Tiger, Tiger comes standard with libreadline
#
# needs readline library and headerfiles installed (X tools)
# use the Mac OS X lipo utitlity to extract one or the other ppc/intel
# example: lipo newlisp-universal -output newlisp-ppc -thin ppc7400
# note: architecture id for ppc on Tiger: ppc on Leopard: ppc7400

# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# NOTE: To run this makefile you must be using 10.5
#       or later with the 10.4 SDK installed!
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

# it's possible to make it work on all systems by reading in
# and parsing the value for ProductVersion in this file:
#
#     /System/Library/CoreServices/SystemVersion.plist

# this does not seem necessary (but if there are problems try uncommenting it)
# export MACOSX_DEPLOYMENT_TARGET=10.4

# to run on 10.4 we must use gcc-4.0
CC = /usr/bin/gcc-4.0
SDKROOT = /Developer/SDKs/MacOSX10.4u.sdk

CFLAGS = -Wall -arch i386 -arch ppc -Os -c -g -DREADLINE -DMAC_OSX -DSUPPORT_UTF8 -isysroot $(SDKROOT) -I$(SDKROOT)/usr/lib/gcc/powerpc-apple-darwin8/4.0.1/include -I$(SDKROOT)/usr/lib/gcc/i686-apple-darwin8/4.0.1/include/

OBJS = newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o
nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o nl-utf8.o pcre.o

default: $(OBJS)
$(CC) $(OBJS) -mmacosx-version-min=10.4 -arch i386 -arch ppc -g -lm -lreadline -o newlisp-universal
strip newlisp-universal
# lipo newlisp-universal -output newlisp-intel -thin i386
# lipo newlisp-universal -output newlisp-ppc -thin ppc

.c.o:
$(CC) $(CFLAGS) $<

$(OBJS): primes.h protos.h makefile_darwin_universal_utf8

[Lutz]

Thanks, this one works nicely cross-platform. I have added it as "makefile_darwin_universal_utf8_compat" to the distribution and retired the old "makefile_darwin_universal_utf8". There is now also a new "makefile_darwin_utf8_tigerPPC" based on your's and used for: "make dmg_ppc" in the main "Makefile".



Seems that Snow Leopard fixed the strptime() function broken in Leopard and used in 'parse-date'.

[itistoday]

No problem! BTW I just noticed that part of that CFLAGS line is not necessary:

Code Select Expand

-I$(SDKROOT)/usr/lib/gcc/powerpc-apple-darwin8/4.0.1/include -I$(SDKROOT)/usr/lib/gcc/i686-apple-darwin8/4.0.1/include/

I had included that to get it working before I realized I had to set the CC path manually. Once the CC path is set correctly that's no longer necessary nor should it even be there (as it's implying that headers for both architectures should be included... and they might not have checks in there for the architecture, so it could even be theoretically dangerous).



Also, I like the new forum (it's BIG and BOLD!), but it's slightly disorienting after being used to the old style. :-p