development release version 9.0.14

Lutz · January 04, 2007, 04:58:36 AM

* This is the first version which can be compiled for large pointer LP64 memory model. Many changes where necessary to make this happen. Please handle this version with care, many of lines code have been touched to make the code compilable in either 32-bit or 64-bit memory models.

* The manual has been updated to include the additions and changes made in the 9.0.x development series. The update of the summary index and frame index for the online version will come later today.

* This version is feature complete for 9.1, due on February 15th. A preview summary page of this version can be seen here: http://newlisp.org/index.cgi?page=Preview_91">http://newlisp.org/index.cgi?page=Preview_91

Files and changes notes: http://newlisp.org/downloads/development/">http://newlisp.org/downloads/development/

Lutz

newdep · January 04, 2007, 07:03:38 AM

newlisp has the BEST compression ratio in the whole world..

This is no joke this is a fact!..Although its a fact on my XP Pro machine.

I downloaded newlisp-9014.exe 1.7 MB big..

Here is a screendump of "Installed software" and blow your mind ;-)

http://www.nodep.nl/downloads/newlisp/wow.JPG">

newdep · January 04, 2007, 10:24:24 AM

the linux make is not without any warings anymore...

...Its a minor cosmetic issue below...

$newlisp-9.0.14$ make linux_readline

make -f makefile_linux_readline

make[1]: Entering directory `/newlisp-9.0.14'

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX newlisp.c

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-symbol.c

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-math.c

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-list.c

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-liststr.c

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-string.c

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-filesys.c

nl-filesys.c: In function `p_parseDate':

nl-filesys.c:1580: warning: implicit declaration of function `strptime'

nl-filesys.c:1580: warning: comparison between pointer and integer

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-sock.c

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-import.c

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-xml.c

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-web.c

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-matrix.c

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-debug.c

gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX pcre.c

gcc newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o -g -lm -ldl -lreadline -lncurses -o newlisp

strip newlisp

make[1]: Leaving directory `/newlisp-9.0.14'

newdep · January 04, 2007, 10:39:41 AM

qa-utf8

Nice.. i never saw those befor... and because im Dutch

I need to correct the entry inside qa-utf8 ofcourse ;-)

Just some nagging ;-)

Its say's ->

(set 'Dutch "Ik kan glas eten, het doet m196179 geen kwaad.")

>Ik kan glas eten, het doet mÄ³ geen kwaad.

There are two issues:

(1) Dutch does not need utf8, but if you use it then

"mÄ³ " is incorrect, should be just an "i" and a "j" "ij"

(2) the spelling is wrong and its very odd Dutch ;-)

This is the correct translation from

German -> "Ich kann Glas essen, ohne mir weh zu tun."

German to Dutch -> "Ik kan glas eten zonder mij pijn te doen."

newdep · January 04, 2007, 11:09:17 AM

Hi Lutz,

Just currious but what is your vision on newLisp regarding future releases and

bit structures? Will newLisp always stay on 32bit no matter what or will it eventualy move towards full 64 bits? (I would stick with 32Bits for sure with

releases and do side releases for 64 bits, as it is now).

Aaaaa i almost forgot "BEST WISHES FOR THIS NEW YEAR !"

Regards, Norman.

Lutz · January 04, 2007, 12:52:26 PM

Binary releases will be 32-bit for a longer time until 64-bit OSs are completely mainstream. The source code package will offer both compile flavors 32-bit and 64-bit as it does with the current 9.0.14 for Linux. When 64-bit becomes available for other UNIX flavors it will be added, i.e. the BSDs.

For whatever machine I can get my hands on, I will offer 64-bit binaries of the executable for major releases.

The current Tru64 UNIX version is a 32-bit version running on a 64-bit OS. I am not sure if Pjot will try to offer a full 64-bit build too. If the Tru64 OS supports the LP64 memory model, it should be not too hard.

Currently most 64-bit Linux come with support for both 32-bit and 64-bit binaries. Mac OSX is a 64-bit OS but only comes with 32-bit libraries, and I guess will stay that way for some time.

At least for several years to come 32-bit will be the software application standard even if most hardware will move to 64-bit.

The codebase is now much more portable so it will be easier to adapt to new necessities as they arise.

Lutz

Ps: regarding the strptime() compiler warning you got: on all OSs I compiled, it was declared in /usr/include/time.h. Perhaps you need a newer installation of GCC?

Best wishes for 2007 for you too and everybody else!

newdep · January 04, 2007, 11:58:12 PM

Hi Lutz,

Im using gcc version 3.4.6

I have these in my time.h ->

# ifdef __USE_XOPEN

/* Parse S according to FORMAT and store binary time information in TP.

The return value is a pointer to the first unparsed character in S. */

extern char *strptime (__const char *__restrict __s,

__const char *__restrict __fmt, struct tm *__tp)

__THROW;

# endif

# ifdef __USE_GNU

/* Similar to the two functions above but take the information from

the provided locale and not the global locale. */

# include <xlocale>

extern size_t strftime_l (char *__restrict __s, size_t __maxsize,

__const char *__restrict __format,

__const struct tm *__restrict __tp,

__locale_t __loc) __THROW;

extern char *strptime_l (__const char *__restrict __s,

__const char *__restrict __fmt, struct tm *__tp,

__locale_t __loc) __THROW;

# endif

Norman.

pjot · January 05, 2007, 03:01:21 PM

This version also compiles fine on Tru64Unix, no errors at all.

However, the tests in 'qa-dot' cause a core dump:

~~Quote~~
[peter@olga]# ./qa-dot

Testing built-in functions ...

! != $ % & * + - / < << <= = > >= >> NaN? ^ abs acos add address amb and append Memory fault(coredump)

[peter@olga]#

I will look into it this weekend. Maybe I can remove all TRU64 macros and some compilerflags to have a full 64bit binary. That would be awesome! :-)

Peter

Lutz · January 05, 2007, 04:37:24 PM

Yes, try to compile this 9.0.14 version on Tru64 as a full 64-bit app. In your makefile add -DNEWLISP64, also remove all "#pragma pointer_size short" all other macros can probably stay or they are redundant with the NEWLISP64 define.

This is what the 64-bit compile as of LP64 memory model expects:

char -> 1 byte

int -> 4 byte

short int -> 2 byte

unsigned short int -> 4 byte

long -> 8 byte

unsigned long -> 8 byte (defined as UINT)

long int -> 8 byte

long long int -> 8 byte

double -> 8 byte

float -> 4 byte

double float -> 8 bytes

size_t -> 8 bytes -> unsigned long

ssize_t -> 8 bytes -> long

all pointer types -> 8 byte

all other types like "long double" are of no interest in newLISP currently. You can compile newlisp-x.x.x/util/type.c to find out. If I remember well from one of our conversations earlier, things are fine with your type.c output. So just add -DNEWLISP64 in newlisp.h or your compile flags and things should be fine.

Lutz

PS: -> Norman: I added for Linux: a strptime() declaration to be independent of the: #ifdef __USE_GNU which gives me problems in other compile flavors. That will eliminate the warning you got.

pjot · January 06, 2007, 02:36:24 AM

Well, good news BUT with minor hiccups.

1) First I removed all 'pointer_size' stuff. I still have to use -DTRU64 because of some workarounds for unknown functionnames.

2) Also I ran '/usr/lib/cmplrs/cc/protect_headers_setup.sh -d' again to make sure all 32-bit headers are gone.

3) Adjusted the makefile so it really compiles to 64 bit by adding '-DNEWLISP64' and by removing the '-xtaso' and '-taso' compilerflags.

~~Quote~~
OBJS = newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o

nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o

# CFLAGS = -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -D_POSIX_PII_SOCKET

CFLAGS = -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET

CC = cc

default: $(OBJS)

$(CC) $(OBJS) -lm -lrt -ldb -lbsd -lreadline -ltermcap -o newlisp

strip newlisp

4) Now I run 'make'.

~~Quote~~
[peter@olga]# make

./build

Discovered Tru64 Unix, please read doc/TRU64BUILD

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET newlisp.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-symbol.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-math.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-list.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-liststr.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-string.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-filesys.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-sock.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-import.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-xml.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-web.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-matrix.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-debug.c

cc -I/usr/local/include -ieee -pedantic -c -O3 -DREADLINE -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET pcre.c

cc newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o -lm -lrt -ldb -lbsd -lreadline -ltermcap -o newlisp

strip newlisp

5) Well, that looks good!! Compiling works on Tru64Unix 5.1B (I still have to test 4.0f). Now I run the testfile 'qa-dot', and there is no memory crash.

~~Quote~~

[peter@olga]# ./qa-dot

<snip>

TESTING: newlisp FINISHED WITH ERRORS:

>>>> flt failed nil

>>>> pipe failed string expected in function write-file : [text]

called from user defined function QA:qa

>>>> ~ failed problem in format string in function format : "%llx"

called from user defined function QA:qa

total time: 5042

newLISP v.9.0.14 64-bit on Tru64Unix, execute 'newlisp -h' for more info.

> ring...

> (exit)

So these are the minor hiccups... I thought I already let you know, but to me the overall result looks very promising. Finally, I ran 'types.c'.

~~Quote~~
[peter@olga]# cc -o types types.c

[peter@olga]# ./types

type bytes

---------------

char 1

char * 8

void * 8

short int 2

int 4

long 8

long int 8

long long 8

float 4

double 8

long double 16

wchar_t 4

format input output

------------------------------------------------

%d 0xffffffff -1

%u 0xffffffff 4294967295

%d 0x7fffffff 2147483647

%u 0x7fffffff 2147483647

%d 0x80000000 -2147483648

%u 0x80000000 2147483648

%d 0x7fffffffffffffffLL -1

%u 0x7fffffffffffffffLL 4294967295

%x 0xffffffffffffffffLL ffffffff

%X 0x7fffffffffffffffLL ffffffff

%X 0x8000000000000000LL 0

%llX 0xFFFFFFFFFFFFFFFFLL ffffffff

%llX 0x7FFFFFFFFFFFFFFFLL ffffffff

%llX 0x8000000000000000LL 0

%lld 0xffffffffffffffffLL -1

%llu 0xffffffffffffffffLL 18446744073709551615

%lld 0x7fffffffffffffffLL 9223372036854775807

%llu 0x7fffffffffffffffLL 9223372036854775807

%lld 0x8000000000000000LL -9223372036854775808

%llu 0x8000000000000000LL 9223372036854775808

Probably something is wrong because of the types. I'll continue to troubleshoot these few issues in 'qa-dot' this weekend.

Again it looks very promising!! With a little bit of luck we have a GENUINE 64-bit newlisp during the weekend!!

Peter

pjot · January 06, 2007, 02:59:51 AM

For the '(flt)', I think it is OK.

~~Quote~~
[peter@olga]# newlisp

newLISP v.9.0.14 64-bit on Tru64Unix, execute 'newlisp -h' for more info.

> (flt 1.23)

4394818826404

The test in 'qa-dot' expects 1067282596.

But the function (flt) converts an 8-byte float to a 4-byte float. On a 32-bit system, it converts a 4 byte float to a 2-byte float, and then the result must be 1067282596.

So my bet is that this test also fails on an AMD64.

Peter

pjot · January 06, 2007, 03:08:56 AM

The pipe test fails because the '[text]' symbols are not recognized. This is how it works on Linux:

~~Quote~~
peter[~]$ newlisp

newLISP v.9.0.14 on Linux, execute 'newlisp -h' for more info.

> (println [text]blabla[/text])

blabla

"blabla"

But this is how it looks on Tru64Unix now:

~~Quote~~
[peter@olga]# newlisp

newLISP v.9.0.14 64-bit on Tru64Unix, execute 'newlisp -h' for more info.

> (println [text]blabla[/text])

nilnil

nil

Anything changed in parsing these '[text] - [/text]' symbols?

Peter

Lutz · January 06, 2007, 06:26:09 AM

For speed I coded this for parsing the [text] tag in the function getToken() in newlisp.c, but nothing has changed here for a long time and Tru64 ate it before:

Code Select Expand
else if(*(UINT*)token == *(UINT*)"[tex" && *(INT16*)(token +4) == *(INT16*)"t]")

it probably should be replaced with:

Code Select Expand
else if(strncmp(token, "[text]", 6) == 0)

The speed impact overall on a program is probably very small. Still I wonder why the Tru64 compiler did not eat this correctly, and it should be investigated, because it may lead to other problems in other areas. Casting of address pointer to int's and long's is done frequently throughout the code. Perhaps Tru64 translates string constant addresses in to small pointers? some kind of segmented architecture?

Lutz

Lutz · January 06, 2007, 07:13:52 AM

... oops, now I see what is happening.

Replace

Code Select Expand
if(*(UINT*)token == *(UINT*)"[tex"

with:

Code Select Expand
if(*(unsigned int *)token == *(unsigned int *)"[tex"

UINT is 64-bit is 8 bytes long but should be only 4 bytes for the copmparison. 'int' will be 32-bit in both models.

Lutz

Lutz · January 06, 2007, 07:24:19 AM

~~Quote~~But the function (flt) converts an 8-byte float to a 4-byte float. On a 32-bit system, it converts a 4 byte float to a 2-byte float, and then the result must be 1067282596.

No, in both system 64- and 32-bit (flt x) converts an 8-byte double float ionto a 4-byte float. In both programming models 'double float' is 8 byte and 'float' is 4 byte. The result must aleyas be: (flt 1.23) => 1067282596. Is fine on AMD64.

Lutz

ps: the types.c output (top portion) looks perfect

newLISP Fan Club

News:

development release version 9.0.14

Lutz

newdep

newdep

newdep

newdep

Lutz

newdep

pjot

Lutz

pjot

pjot

pjot

Lutz

Lutz

Lutz