New functions for search and replace in nested lists. A fix for trailing slashes in the directory predicate on Win32.
Lutz
for files and changes notes see: http://newlisp.org/downloads/development/
I think you meant:
http://newlisp.org/downloads/development/
(As always)
;-)
thanks for catching this, corrected in the original post.
Lutz
It's working well here (MacOS X 10.4.11 Intel). Cool new functions too!
Quote from: "cormullion"
I haven't looked into new functions yet, but two my scripts that I use on regular basis (shortcuts to them are even placed to my quick launch bar) are working fine. I do always test this after broken 9.2.6, but, thanks to Lutz, all of 9.2.7 - 9.2.10 was flawless.
... glad 9.2.10 has been fine so far, this development release is feature complete for stable release 9.3, which is due beginning January 2008.
Lutz
This release compiles fine in Tru64 5.x but on 4.x I see the following (minor) issues:
Quote
Discovered Tru64 Unix, please read doc/TRU64BUILD
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET newlisp.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-symbol.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-math.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-list.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-liststr.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-string.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-filesys.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-sock.c
cc: Warning: nl-sock.c, line 407: In this statement, & before array "(dest_sin.sin_zero)" is ignored. (addrarray)
memset(&(dest_sin.sin_zero), ' ', sizeof(dest_sin.sin_zero));
---------------^
cc: Warning: nl-sock.c, line 407: In this statement, & before array "(dest_sin.sin_zero)" is ignored. (addrarray)
memset(&(dest_sin.sin_zero), ' ', sizeof(dest_sin.sin_zero));
---------------^
cc: Warning: nl-sock.c, line 861: In this statement, & before array "(dest_sin.sin_zero)" is ignored. (addrarray)
memset(&(dest_sin.sin_zero), ' ', 8);
-------^
cc: Warning: nl-sock.c, line 861: In this statement, & before array "(dest_sin.sin_zero)" is ignored. (addrarray)
memset(&(dest_sin.sin_zero), ' ', 8);
-------^
cc: Warning: nl-sock.c, line 1191: In this statement, & before array ""
"" is ignored. (addrarray)
if(newLine) write(handle, &LINE_FEED, strlen(LINE_FEED));
--------------------------^
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-import.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-xml.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-web.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-matrix.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-debug.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET pcre.c
cc newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o -lm -lrt -ldb -lbsd -o newlisp
strip newlisp
Also running 'qa-dot' in Tru64Unix 5.x fails for two tests:
Quote
TESTING: newlisp FINISHED WITH ERRORS:
>>>> pretty-print failed nil
>>>> rand failed nil
total time: 6679
I will investigate this later this week.
Peter
The (rand) test fails because of this:
Quote
newLISP v.9.2.10 64-bit on Tru64Unix, execute 'newlisp -h' for more info.
> (set 'sum 0)
0
> (dotimes (x 1000)(inc 'sum (rand 2)))
65385830
> (and (<sum> sum 400)(list? (rand 10 100))))
nil
The 'rand' function does not work:
Quote
> (rand 2)
33878
In the sourcecode of 'nl-math.c' this is going wrong:
rnum = ((scale * random())/(RAND_MAX - 1));
From the manpage of 'random()':
Quote
The random() function uses a nonlinear additive feedback random number gen-
erator employing a default state array size of 31 integers to return suc-
cessive pseudo-random numbers in the range from 0 to (2^31)-1. The period
of this random number generator is approximately 16*((2^31)-1). The size
of the state array determines the period of the random number generator.
Increasing the state array size increases the period.
If I run this for example:
scale * random()
Then the output is 286605828.000000 or something the like. So the 'random()' function already generates a huge number. For some reason, the function bails out here:
rnum = ((scale * random())/(RAND_MAX - 1));
if(params->type == CELL_NIL)
return(stuffInteger((UINT)rnum));
So the params->type == CELL_NIL and the huge number is returned.
Peter
Phew found the cause. The value of RAND_MAX is different in Linux and Tru64Unix.
Linux:
RAND_MAX = 2147483647
Tru64Unix:
RAND_MAX = 32767
Therefore the result of the calculation ((scale * random())/(RAND_MAX - 1)) is much higher. And the test for (rand) fails.
Peter
- prettty-print
the default changed to 80 instead of 64 characters, and this has not been updated in the qa-dot and qa-comma files
- rand
had the last change in 9.2.6, this is fine MAc OS X when compiling as 64-bit version, so I don't know at the moment how to debug this for TRU64. I wonder, If I could have access to your machine?
Pehaps you insert the following code for me to debug:
long testRand; /* for debugging */
testRand = random();
printf("scale=%f testRand=%ld RAND_MAX=%ld scale * testRand=%fn",
scale, testRand, RANDOM_MAX, scale * testRand);
I may have gotten wrong some of the formatting characters for TRU64. The problem problem probably happens in the multiplication and implicit type conversions to float and back to int.
worst case we have to go back to pre 9.2.6 code for rand, let me know if 9.2.5 tests fine for rand on TRU64.
Lutz
ps: would be great to TRU64 working for 9.3 in January
Hi Lutz,
Thanks for the reply. This is the result of your code:
Quote
[peter@olga]# ./newlisp
newLISP v.9.2.10 64-bit on Tru64Unix, execute 'newlisp -h' for more info.
> (rand 2)
scale=2.000000 testRand=2078917053 RAND_MAX=32767 scale * testRand=4157834106.000000
8747
Hope it helps...? Maybe the variable RAND_MAX must be redefined in 'newlisp.h'?
I do not have the sourcepackage for 9.2.5 anymore, but 9.2.0 shows:
Quote
newLISP v.9.2.0 64-bit on Tru64Unix, execute 'newlisp -h' for more info.
> (rand 2)
1
So that looks good.
Quote
ps: would be great to TRU64 working for 9.3 in January
Yes! I hope so! :-)
Peter
Thanks Peter I have redefined RAND_MAX 0x8FFFFFFF in newLISP.h conditionally for the TRU64 flavor.
Lutz
Thanks, please make sure the redefinition occurs AFTER the '#include <stdio>' statement.
Compilation becomes very ugly but it works correct now:
Quote
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET newlisp.c
cc: Warning: newlisp.h, line 478: The redefinition of the macro "RAND_MAX" conflicts with a current definition because the replacement lists differ. The redefinition is now in effect. (macroredef)
#define RAND_MAX 0x8FFFFFFF
-----------------^
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-symbol.c
cc: Warning: newlisp.h, line 478: The redefinition of the macro "RAND_MAX" conflicts with a current definition because the replacement lists differ. The redefinition is now in effect. (macroredef)
#define RAND_MAX 0x8FFFFFFF
-----------------^
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-math.c
cc: Warning: newlisp.h, line 478: The redefinition of the macro "RAND_MAX" conflicts with a current definition because the replacement lists differ. The redefinition is now in effect. (macroredef)
#define RAND_MAX 0x8FFFFFFF
-----------------^
....
[peter@olga]# ./newlisp
newLISP v.9.2.10 64-bit on Tru64Unix, execute 'newlisp -h' for more info.
> (rand 2)
1
>
[/quote]
Cheers
Peter
In the headerfile 'stdlib.h' on Tru64Unix I see this:
#ifdef _BSD
#define RAND_MAX 2147483647 /* max value returned by rand */
#else
#define RAND_MAX 32767 /* max value returned by rand */
#endif
Now, if I add -D_BSD to the makefile, and recompile, I get this:
Quote
[peter@olga]# make
./build
Discovered Tru64 Unix, please read doc/TRU64BUILD
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET -D_BSD newlisp.c
cc: Error: newlisp.c, line 69: This declaration contains a redefinition of "opsys". The previous declaration is at line number 440 in file newlisp.h. (redef)
int opsys = 9;
----^
cc: Error: newlisp.c, line 70: This declaration contains a redefinition of "ostype". The previous declaration is at line number 441 in file newlisp.h. (redef)
char ostype[]="Tru64Unix";
-----^
*** Exit 1
Stop.
*** Exit 1
Stop.
Sorry for all the spam...
In the make file you can use only
I will put the define for MAX_RAND after all other includes in newlisp.h. But I think the real fix would be to fix somehow the error in the TRU64 header files, or live with the repeated warnings that MAX_RAND redefined.
Perhaps there is an option in the TRU64 compiler to suppress the macro-redefinition warning?
Lutz
There are only 3 occurrences of RAND_MAX in the code base and all in nl-math.c. I will code around this introducing MY_RAND_MAX and set it conditionally in newlisp.h.
Lutz
Thanks, this works now! The (rand) function is surviving the test. One remarks though:
1) There is still a redefinition in 'newlisp.h' of RAND_MAX at line 99-101, can you remove it?
Thanks!
Peter
Lutz,
Now I found another issue but I also found the cause.
It seems that your change in (ends-with) crashes newlisp when calling this line from a program:
(ends-with "newlisp" "LISP" nil)
However, running in interactive mode, this line work well. But running from a program with shebang "#!/usr/bin/newlisp", the program creates a coredump.
Hunting down this issue I arrive at this piece of code in 'nl-liststr.c':
1312 /* append $ to the pattern for anchoring at the end */
1313 keydollar = alloca(klen + 4);
1314 *keydollar = '(';
1315 memcpy(keydollar + 1, key, klen);
1316 memcpy(keydollar + 1 + klen, ")$", klen);
1317 *(keydollar + klen + 3) = 0;
1318 klen = klen + 3;
1319 if((pos = searchBufferRegex(string, 0, keydollar, slen, options, &klen)) != -1)
1320 {
1321 if(pos + klen == slen)
1322 return(trueCell);
1323 }
1324 }
The program actually crashes when calling searchBufferRegex. The problem here is the usage of 'alloca'. From the Tru64Unix manpages:
Quote
[Tru64 UNIX] The alloca() function allocates the number of bytes of space
specified by the size parameter in the stack frame of the caller. This
space is automatically freed when the function that called the alloca()
function returns to its caller. The contents of the memory is undefined.
From the Linux manpages:
Quote
BUGS
The alloca() function is machine and compiler dependent. On many systems its implementation
is buggy. Its use is discouraged.
On many systems alloca() cannot be used inside the list of arguments of a function call,
because the stack space reserved by alloca() would appear on the stack in the middle of the
space for the function arguments.
If I change the 'alloca' for a plain 'malloc', everything runs fine and there is no crash. But the memory must be freed afterwards, of course.
Peter
no problem I can change to malloc() for TRU64
Lutz
Maybe you have to also change it for Linux, please check the manpage for 'alloca'...
See my post above, I just changed it.
Thx
Peter
seems to be TRU64 specific. alloca() in 'ends-with' is fine on Windows, Linux BSDs and Mac OS X 64bit.
BTW there are 26 places in the newLISP code base where alloca() is used, but TRU64 seems to behave fine on those. alloca() is allocating memory from the call stack.
There is a new version in your space ;-)
Lutz
Well, those are convincing arguments... :-)
This latest release is running as a dream:
Quote
[peter@olga]# make
./build
Discovered Tru64 Unix, please read doc/TRU64BUILD
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET newlisp.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-symbol.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-math.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-list.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-liststr.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-string.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-filesys.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-sock.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-import.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-xml.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-web.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-matrix.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET nl-debug.c
cc -ieee -pedantic -c -O3 -DSOLARIS -DTRU64 -DNEWLISP64 -D_POSIX_PII_SOCKET pcre.c
cc newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o -lm -lrt -ldb -lbsd -o newlisp
strip newlisp
[peter@olga]#
[peter@olga]# ./qa-dot
Testing built-in functions ...
! != $ % & * + - / : < << <= = > >= >> NaN? ^ abs acos acosh add address amb and append append-file apply args array array-list array? asin asinh assoc atan atan2 atanh atom? base64-dec base64-enc bayes-query bayes-train begin beta betai bind binomial callback case catch ceil change-dir char chop clean close command-line cond cons constant context context? copy-file cos cosh count cpymem crc32 crit-chi2 crit-z current-line curry date date-value debug dec def-new default define define-macro delete delete-file delete-url destroy det device difference directory directory? div do-until do-while doargs dolist dostring dotimes dotree dump dup empty? encrypt ends-with env erf error-event error-number error-text eval eval-string exec exists exit exp expand explode factor fft file-info file? filter find find-all first flat float float? floor flt for for-all fork format fv gammai gammaln gcd get-char get-float get-int get-long get-string get-url global global? if ifft import inc index int integer integer? intersect invert irr join lambda? last legal? length let letex letn list list? load local log lookup lower-case macro? main-args make-dir map mat match max member min mod mul multiply name net-accept net-close net-connect net-error net-eval net-listen net-local net-lookup net-peek net-peer net-ping net-receive net-receive-from net-receive-udp net-select net-send net-send-to net-send-udp net-service net-sessions new nil? normal not now nper npv nth nth-set null? number? open or pack parse parse-date peek pipe pmt pop post-url pow pretty-print primitive? print println prob-chi2 prob-z process protected? push put-url pv quote quote? rand random randomize read-buffer read-char read-file read-key read-line real-path ref ref-all ref-set regex remove-dir rename-file replace replace-assoc reset rest reverse rotate round save search seed seek select semaphore sequence series set set-locale set-nth set-ref set-ref-all setq sgn share signal silent sin sinh sleep slice sort source sqrt starts-with string string? sub swap sym symbol? symbols sys-error sys-info tan tanh throw throw-error time time-of-day timer title-case trace trace-highlight transpose trim true? unify unique unless unpack until upper-case uuid wait-pid when while write-buffer write-char write-file write-line xml-error xml-parse xml-type-tags zero? | ~
Testing contexts as objects and scoping rules ...
ALL FUNCTIONS FINISHED SUCCESSFULL: newlisp
total time: 6588
Thanks for the good work!
Peter
PS The "c:\" with trailing slashes I'll test tomorrow.
_
Quote
You cant simply steal processor power.. you have to declare those..
Ill fax you the paper...Olga is Ours! You have the Lisa and the Pamela ;-)
Damm competitors ..dies days..
but..nice random catch!
Well our company did take over all CB systems, also the Oga ;-)
I have to send you a bill for using newLisp in Tru64Unix ;-)
I would like to dual Tru64 against our new SUN Blade systems,
we must start a newlisp WarRoom in our server park! ;-)