Hi all,
Today I tried to compile version 8.5.7 on Tru64Unix 4.0f (formerly known as Digital Unix). I have adapted the makefile using the makefile for Solaris as a template:
OBJS = newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o
nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o
CFLAGS = -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET
CC = cc
default: $(OBJS)
$(CC) $(OBJS) -lm -lrt -ldb -o newlisp
.c.o:
$(CC) $(CFLAGS) $<
$(OBJS): primes.h protos.h makefile_solaris
Also I made some minor changes in in sourcefiles:
- Put the #define, #elsif, #endif to the first column in the sourcefiles nl-string.c and nl-sock.c
- Removed "RTLD_GLOBAL" from nl-import.c
- Commented the line #define FIONREAD I_NREAD in nl-sock.c since Tru64 knows the FIONREAD
Now there is only 1 thing left which the Compaq C compiler cannot pass:
Quote
pamela_vaspadm> make -f makefile_tru64
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS newlisp.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS nl-symbol.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS nl-math.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS nl-list.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS nl-liststr.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS nl-string.c
cc: Error: nl-string.c, line 499: In this statement, "args" is of type "pointer to signed char", and cannot be converted to "struct declared without a tag". (noconvert)
vasprintf(&ptr, format, args);
------------------------^
*** Exit 1
Stop.
So the 'vasprintf' is known, but it cannot accept the 3rd argument which indeed appears to be a "char*".
If I comment this 'vasprintf' then everything compiles, but the resulting 'newlisp' binary cores.
Any ideas?
Peter
Hm, probably newLisp will not run anyway because of the 64-bit architecture of Tru64Unix.
Still it is nice to know that it *almost* compiles... :-)
First, thanks for doing this work Peter, every new platform for newLISP is always very welcome by somebody.
For the vasprintf just change the line 499 in nl-string.c to
vasprintf(&ptr, format, (va_list)args)
va_list and char* seem to be equivalent on all OS ( hopefully ;-) )
I will do the same in the next version and check that all is OK on the other platforms.
Lutz
Yes, it might not run, but if it compiles/links? why not. I also have a makefile_linux64 for the AMD64 but compile with a special flag -m32, which makes the platform know that it's really a 32bit app.
Lutz
Unfortunately, that doesn't work either, now I receive the following error:
Quote
pamela_vaspadm> make -f makefile_tru64
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET newlisp.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-symbol.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-math.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-list.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-liststr.c
cc -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-string.c
cc: Error: nl-string.c, line 499: In this statement, "va_list" is a struct type, which is not scalar. (needscalartyp)
vasprintf(&ptr, format, (va_list)args);
------------------------^
*** Exit 1
Stop.
I also tried the 'my_vasprintf' in your nl-filesys.c, but also this did not work... :-(
By the way, I compiled the "types.c" file on Tru64 as well, using the Compaq C compiler, maybe you are interested in the result:
Quote
type bytes
---------------
char 1
char * 8
short int 2
int 4
long 8
long long 8
float 4
double 8
wchar_t 4
jmp_buf 672
The types.c file reveals that it wouldn't work because of the size-8 of char*. And that is probably the reason that va_list is not accepted, which seems to be a struct type on Tru64 but is equivalent to a char* scalar on GCC 32bit.
Perhaps there is a switch similar to -m32 on the AMD64, which lets you compile as a 32 bit application? But it still could be necessary to rewrite my_vasprintf() used in 'format', if workiing differently on that platform.
As soon as 64bit goes mainstream with good GCC support over several platforms newLISP will move to 64bit, but this day seems to be still in the future. It basically means changing the structure of the lisp cell to accomodate 64bit pointers.
Lutz
There is such an option: -protect_headers default -xtaso_short
If I compile the types.c with this option, then this is the result:
Quote
type bytes
---------------
char 1
char * 4
short int 2
int 4
long 8
long long 8
float 4
double 8
wchar_t 4
jmp_buf 672
So this seems to be better. Now, if I compile newlisp, again I receive the same result:
Quote
....
cc -pedantic -protect_headers default -xtaso_short -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-math.c
cc -pedantic -protect_headers default -xtaso_short -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-list.c
cc -pedantic -protect_headers default -xtaso_short -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-liststr.c
cc -pedantic -protect_headers default -xtaso_short -c -O2 -DNANOSLEEP -DSOLARIS -D_POSIX_PII_SOCKET nl-string.c
cc: Error: nl-string.c, line 499: In this statement, "va_list" is a struct type, which is not scalar. (needscalartyp)
vasprintf(&ptr, format, (va_list)args);
------------------------^
*** Exit 1
Stop.
So compiling 32bits is possible, but still it stumbles over this vasprintf argument... ?
I would have to study their header files to figure out how "va_list" works on that machine. On the GCC compiler its just the memory addres to the parameters packed one ofter the other in memory.
The types file compile looks good this way, so it should be possible to compile on that machine after the vasprintf obstacle.
Compiling with -DSOLARIS means that the compile is already taking my_vasprintf from nl-filesys.c. You could redefine to take char* on the last argument, but then it might trip over vsnprintf() later in that function and I have no idea how this is defined in your machine'sheader files.
It's tedious to figure out this stuff if you are not on the machine ;)
Lutz
Yes, I understand, if you're not on the machine yourself, it is hard... the Alphaservers belong to my employer, and unfortunately I cannot just take one home and open a line for you... :-(
Thanks for the compile hints again, tomorrow I'll see if I can pass the vasprintf problem.
Peter
You could just out-comment 'format' in primes.h just to see if the whole thing runs after compliling/linking and if it's worth it. Anyway thanks for working on this. The whole vasprintf() and vsnprintf() thing has caused me some head-aches in the past, because it works differently on different compilers. Good luck tomorrow!
Lutz
Well, good news and bad news.
---> Good news: newLisp compiles and runs on Tru64 4.0f and also Tru64 5.1! <---
Bad news: the 'format' statement delivers "Unaligned access" errors and wrong output.
The reason for the unaligned access is caused by the way I created a workaround for the 'vasprintf' trouble. In the file nl-filesys.c I created an extra function called "tru64_vasprintf" which is a copy of the existing 'my_vasprintf':
int tru64_vasprintf(char * * buffer, const char * format, char* argptr)
{
UINT size = MAX_STRING;
int pSize;
while(TRUE)
{
*buffer = allocMemory(size + 2);
pSize = vsnprintf(*buffer, size + 1, format, argptr);
#ifndef MINGW
if(pSize > size)
{
freeMemory(*buffer);
size = pSize;
continue;
}
#else
if(pSize < 0)
{
freeMemory(*buffer);
size = size + size / 2;
continue;
}
#endif
break;
}
return(pSize);
}
The file 'nl-string.c' was changed to point to this function (line 499). Now everything compiles without errors. (Also 'protos.h' was adjusted, adding a prototype.)
However, the 'vsnprinf' takes also a 'va_list' as 3rd argument (why the compiler does not complain now, I don't know). If instead a char* is feeded here, the data alignment error appears.
It might be of your interest to know that the va_list in Tru64 is defined as:
typedef struct {
char **_a0; /* pointer to first homed integer arg */
int _offset; /* byte offset of next param */
} va_list;
Except for the format statement, all other newlisp functions seem to work fine. I will perform some tests today and let you know the results.
Peter
Adjusted the 'tru64_myvasprinf' function to wrap around the va_list:
int tru64_vasprintf(char * * buffer, const char * format, ...)
{
UINT size = MAX_STRING;
int pSize;
va_list ap;
while(TRUE)
{
*buffer = allocMemory(size + 2);
va_start(ap, format);
pSize = vsnprintf(*buffer, size + 1, format, ap);
va_end(ap);
#ifndef MINGW
if(pSize > size)
{
freeMemory(*buffer);
size = pSize;
continue;
}
#else
if(pSize < 0)
{
freeMemory(*buffer);
size = size + size / 2;
continue;
}
#endif
break;
}
return(pSize);
}
Now the alignment errors are gone, but the output is wrong. At every string I run through format, the result is 0 (or "0").
Its the difference between GNU CC and Tru64 CC how arguments are layed out in memory for vsnprintf().
In Gnu CC all args are packes one after the other in a memory area, in Tru64 after each arg piece comes an int describing the offset. I order to make it work p_format() in nl-string.c would have to be rewritten to pack arguments for the format string in a way necessary for Tru64.
The change you did using va_list and va_start was the right thing to do, but would also require to change the function doing the preperation of it's last args ('ap' in your code) parameter
I wonder if you can get GCC for the Tru64 platform? Perhaps that would be easier then trying to work around the peculiarities of the CC found on Tru64?
This is one of those typical cases where porting gets a head-ache. Another problem is how vsnprintf() returns different values in case of failure, as you can see on the MINGW switch, and this is also something that would have to be looked into.
Anyway, good luck in your porting efforts ... perhaps somebody else here on the port as some 'quick' fix hoew to deal with this problem?
Lutz
Indeed I already have been busy to change the code in nl-string.c... the memcpy's create a mem layout which indeed is not compatible with Tru64Unix va_list.
I am able to print the first argument though, like (format "c" 65) -> "A". But more complicated formats deliver problems.
The latest GCC for Tru64Unix I am aware of, is version 2.95, which lacks the '-m32' switch.
So I need to hack the memory layout of the 'va_list' anyway, in order to get the format properly working.
BTW: other things, like lists, signals, read-key and sockets, work. It is *only* this format statement which gives a headache. This weekend I will work on the port.
Peter
The Tru64Unix port is ready.
Peter
newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.
> (setq a 1)
1
> (inc 'a -15)
4294967282
> (setq b -16)
-16
> (inc 'b)
-15
> (inc 'b -17)
8589934560
> (inc 'b -255)
1.28849016e+10
>
Strange, there is an issue with negative numbers:
Quote
newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.
> (set 'a 10)
10
> (inc 'a)
11
> (inc 'a 3)
14
> (inc 'a -1)
4294967309
>
-----------------------
But normal expressions work:
Quote
> (- -1 -5)
4
> (+ 1 -10)
-9
>
-----------------------
I'll look into 'nl-match.c' to see what is going on. Obviously, negative numbers are rounded wrong, though the format works:
Quote
> (format "%x %X" -1 -1)
"ffffffff FFFFFFFF"
As a workaround you can use a float:
Quote
> (set 'b 5)
5
> (inc 'b 0.1)
5.1
> (inc 'b -0.1)
5
> (inc 'b -6.5)
-1.5
Peter
I wonder if true64 formats numbers as IEE 754 double, 8 bytes long. The function doing the 'inc' and 'dec' is called incDec() and is in nl-math.c.
But I would check first if things like:
(sub 10 -1) or (add 10 1)
work. It may be that the function getFloat() in newlisp.c is not working right on true64-unix. Did you run qa_dot, testing all integer and float arithemetik?
Lutz
Hi Lutz,
Well we are in the process of testing, and indeed I was wondering if there was such a thing as a testsuite.
Here the results of the 'div' and 'add':
Quote
newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.
> (sub 10 -1)
-4294967285
> (add 10 1)
11
>
So it seems to be something with negative numbers. A '-1' should be 0xFFFFFFFF right? Probably on Tru64 this is bigger.
If I run the 'qa_dot' program, there is a crash after the 'append':
Quote
pamela_vaspadm> newlisp qa_dot
Testing built-in functions ...
! != $ % &
* + - / <
<< <= = > >=
>> NaN? ^ abs acos
add address amb and append
Segmentation fault (core dumped)
To avoid this, I have commented out the following functions:
-(test-array-list) which causes this crash.
-(test-dotimes , aList) which blocked newLisp
-(test-pmt) which gave a FLOAT error
If I run again, this is the result:
Quote
pamela_vaspadm> newlisp qa_dot
Testing built-in functions ...
! != $ % &
* + - / <
<< <= = > >=
>> NaN? ^ abs acos
add address amb and append
apply args array array-list array?
asin assoc atan atan2 atom?
base64-dec base64-enc begin beta betai
binomial case catch ceil change-dir
char chop close command-line cond
cons constant context context? copy-file
cos count cpymem crit-chi2 crit-z
current-line date date-value debug dec
def-new define define-macro delete delete-file
device difference directory directory? div
do-until do-while dolist dotimes dotree
dump dup empty? encrypt ends-with
env error-event error-number error-text eval
eval-string exec exit exp expand
explode factor fft file-info file?
filter find first flat float
float? floor flt for fork
format fv gammai gammaln get-char
get-float get-int get-integer get-string get-url
global if ifft import inc
index int integer integer? intersect
invert irr join lambda? last
legal? length let letn list
list? load log lookup lower-case
macro? main-args make-dir map match
max member min mod mul
multiply name net-accept net-close net-connect
net-error net-listen net-local net-lookup net-peek
net-peer net-receive net-receive-from net-receive-udp net-select
net-send net-send-to net-send-udp net-service net-sessions
new normal not now nper
npv nth nth-set open or
pack parse pipe pack parse pipe pmt pop
post-url pow pretty-print primitive? print
println prob-chi2 prob-z process println prob-chi2 prob-z process push
put-url pv quote quote? rand
random read-buffer read-char read-file read-key
read-line ref regex remove-dir rename-file
replace replace-assoc reset rest reverse
rotate save search seed seek
select semaphore sequence series set
set! set-locale set-nth setq share
signal silent sin sleep --- sleep duration: 1000 ---
slice
sort source sqrt starts-with string
string? sub swap sym symbol
symbol? symbols sys-error sys-info tan
throw throw-error time time-of-day timer
trace trace-highlight transpose trim unique
unless unpack until upper-case wait-pid
while write-buffer write-char write-file write-line
xml-error xml-parse xml-type-tags | ~
Testing contexts as objects and scoping rules ...
TESTING FINISHED WITH ERRORS:
>>>> / failed nil
>>>> < failed nil
>>>> <= failed nil
>>>> > failed nil
>>>> NaN? failed nil
>>>> abs failed nil
>>>> array-list failed nil
>>>> array? failed nil
>>>> binomial failed nil
>>>> dotimes failed nil
>>>> exec failed nil
>>>> fv failed nil
>>>> int failed nil
>>>> integer failed nil
>>>> net-receive-udp failed nil
>>>> net-send-udp failed nil
>>>> npv failed nil
>>>> pmt failed nil
>>>> semaphore failed nil
>>>> share failed nil
newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.
> ring...
Most of these failed tests use negative numbers or floats.
In this situation:
newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.
> (set 'a 5)
5
> (inc 'a 1)
6
> (inc 'a -1)
4294967301
The function 'CELL * getFloat(CELL * params, double * floatNumber)' determines the '-1' wrongly.
This is caused by the fact that "cell = evaluateExpression(params);" determines the cell->type as a CELL_FLOAT for the number '-1'.
So the cause seems to be in the routine 'evaluateExpression'. Or is the cell->type previously determined by a parser? Continuing...
Looks like for doing this true64 port we need to look in the Compaq 'C' compiler documentation to find out about floating point formats etc. Can you compile types.c again and show the output? Perhaps we missed something there.
Lutz
I found a compiler switch '-ieee' to ensure the IEEE754 compatibility, but the resulting newLisp binary delivers the same results with 'qa_dot'.
Compiled the 'types.c' again:
Quote
pamela_vaspadm> cc -o types -ieee -taso types.c
pamela_vaspadm> ./types
type bytes
---------------
char 1
char * 8
short int 2
int 4
long 8
long long 8
float 4
double 8
wchar_t 4
jmp_buf 672
I have the strong feeling it is an issue with the negative sign '-' on a number...
Hacked the routine 'CELL * incDec(CELL * params, int type)' to print info about variables:
Quote
newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.
> (set 'a 5)
5
> (inc 'a 1)
adjust: 1.000000
type: 1
6
> (inc 'a -1.0)
adjust: -1.000000
type: 1
5
> (inc 'a -1)
adjust: 4294967295.000000
type: 1
4294967300
Are you using INT's or LONG's for normal numbers? Since on the alpha the LONG is twice as big as an INT.
While on X86 both the INT and also the LONG contain 4 bytes.
So a '-1' as a long will be 0xFFFFFFFFFFFFFFFF instead of 0xFFFFFFFF.
If Alpha sees 0xFFFFFFFF it will treat this as a regular number which is 4294967295.
Jep that's it. Any idea what I need to change in the source?
There are no 'long's in newlisp.c but in several other files, all these could be replaced with 'int', because int and long are the same for 32bit GCC. Its mostly in older code from Win16 times, I haven't touched for a longer time. You would have to go through all the files for this. I could do this, but not before the weekend.
But there is a bigger problem with pcre.c there are tons of 'long int' in it and I have no idea if that code was written with 'long int' == 8byte in mind. But you might just try and see ;)
Lutz
Well, you are wrong... :-)
At line 2872 in 'newlisp.c' you cast to long:
Quote
*floatNumber = (long)cell->contents;}
I have changed the (long) to (int), and guess what? The (inc 'a -1) works now.
Quote
*floatNumber = (int)cell->contents;}
Tonight I will test more. Probably all LONGs should be switched to INTs.
Could you also add the '-ieee' compiler switch to 'makefile_tru64'?
That's the one in getFloat() doing the conversion. There are lots of (long) casts in nl-math.c too, you will see. Lets hope that all this streightens out the port to tru64.
Lutz
Well, I can try again to compile with
-protect_headers default -xtaso_short
compile flags, which will force a 32-bit compilation. That denifitively should solve the 8byte/4byte stuff.
But it would ne nice to run in 64-bit mode!
Just ran the 'qa_dot' again, the result is much better:
Quote
Testing contexts as objects and scoping rules ...
TESTING FINISHED WITH ERRORS:
>>>> array-list failed nil
>>>> array? failed nil
>>>> binomial failed nil
>>>> exec failed nil
>>>> fv failed nil
>>>> int failed nil
>>>> integer failed nil
>>>> net-receive-udp failed nil
>>>> net-send-udp failed nil
>>>> npv failed nil
>>>> semaphore failed nil
>>>> share failed nil
newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.
> ring...
So 12 functions left, instead of the 23 yesterday, which means that 11 functions have been solved by changing that cast to (int).
I will run through the other sourcefiles of newLisp tonight to see what is going on.
Peter
Looking through some of the array functions I cannot see anything obvious. The good news is the "ring ..." at the end, which is from testing the timer functions, so signals seem to work.
I want to do a development release on the weekend. If there is anything you want me to merge in just tell me, where I can get it like you did before.
Lutz
Well, I cleaned up and started again to measure the progress.
-makefile_tru64 ---> add the "-ieee" compilerflag so (test-pmt) passes.
makefile_tru64 looks like this now:
Quote
# makefile for newLISP v. 8.x.x on HP Tru64Unix
#
#
OBJS = newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o
nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o tru64.o pcre.o
CFLAGS = -ieee -pedantic -c -O2 -DNANOSLEEP -DSOLARIS -DTRU64 -D_POSIX_PII_SOCKET
CC = cc
default: $(OBJS)
# $(CC) $(OBJS) -taso -lm -lrt -ldb -Lffi-tru64-5.1 -lffi -o newlisp
$(CC) $(OBJS) -taso -lm -lrt -ldb -Lffi-tru64-4.0 -lffi -o newlisp
strip newlisp
.c.o:
$(CC) $(CFLAGS) $<
$(OBJS): primes.h protos.h makefile_tru64
Changed the following:
-'newlisp.c' ---> line2872 cast to (int)
-'nl-math.c' ---> line 123 cast to (int)
-'nl-math.c' ---> line 138 cast to (int)
-'nl-math.c' ---> line 139 cast to (int)
-'nl-math.c' ---> line 140 cast to (int)
-'nl-math.c' ---> line 143 cast to (int)
-'nl-math.c' ---> line 144 cast to (int)
-'nl-math.c' ---> line 145 cast to (int)
-'nl-math.c' ---> line 146 cast to (int)
-'nl-math.c' ---> line 147 cast to (int)
-'nl-math.c' ---> line 148 cast to (int)
-'nl-math.c' ---> line 151 cast to (int)
-'nl-math.c' ---> line 279 cast to (int)
-'nl-math.c' ---> line 573 cast to (int)
-'nl-math.c' ---> line 574 cast to (int)
-'nl-math.c' ---> line 745 cast to (int)
-'nl-math.c' ---> line 1595 cast to (int)
As a result, 12 functions are left to test:
Quote
>>>> array-list failed nil
>>>> array? failed nil
>>>> binomial failed nil
>>>> exec failed nil
>>>> fv failed nil
>>>> int failed nil
>>>> integer failed nil
>>>> net-receive-udp failed nil
>>>> net-send-udp failed nil
>>>> npv failed nil
>>>> semaphore failed nil
>>>> share failed nil
---------------------------------
Started to work on the EXEC statement. I found that the C-function 'fgetc' (which is used in 'nl-filesys.c' lines 407-431 function 'readStreamLine') returns EOF when encountered an error or a signal. Unfortunately, the 'fgetc' function is interrupted by Tru64Unix continuously, thereore generating "Interrupted system call" or the EINTR signal all the time.
This means that sometimes the stream is read, sometimes not.
So, it was necessary to adjust the function 'readStreamLine' in a way, that it will continue when EINTR was received. I did it like this:
char * readStreamLine(STREAM * stream, FILE * inStream)
{
int chr;
openStrStream(stream, MAX_STRING, 1);
do { <--------------------------------------------added
errno = 0; <-------------------------------------added
while((chr = fgetc(inStream)) != EOF)
{
if(chr == 'n') break;
if(chr == 'r')
{
chr = fgetc(inStream);
if(chr == 'n' || chr == EOF) break;
}
writeStreamChar(stream, chr);
}
} while (errno == EINTR); <------------------------added
if(chr == EOF && stream->position == 0) return(NULL);
return(stream->buffer);
}
Ugly, but it works. I hope you have a better idea on how to change this. After a "./newlisp qa_dot", this is the result:
Quote
>>>> array-list failed nil
>>>> array? failed nil
>>>> binomial failed nil
>>>> fv failed nil
>>>> int failed nil
>>>> integer failed nil
>>>> net-receive-udp failed nil
>>>> net-send-udp failed nil
>>>> npv failed nil
>>>> semaphore failed nil
>>>> share failed nil
So the EXEC statement works now. Only 11 functions left... :-)
I will post the adjusted newLisp package with the changes I mention here before the weekend. If you can approve and merge these changes....??
Continuing with the other statements now...
--
Can you test 5.1b ? I like to test it on the DS25 also oh yes and the ES40
aswell... Better then cooking eggs on it ;-)
Thanks! Norman.
(hahahaha..great pjot... :)
Yeah yeah yeah... laugh at me! Now it is a matter of principles... :-)
Anyway, I found the reason why the newLisp INT and INTEGER statements fail. They actually fail on these tests:
(= (integer "10000000000") 2147483647)
(= (integer "-10000000000") -2147483648)
newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.
> (integer "10000000000")
1410065408
In the file 'nl-string.c' the function "CELL * p_integer(CELL * params)" (lines 803-835) concludes with:
return(stuffInteger((UINT)strtol(intString,(char **)0,(int)base)));
As you may suspect, the pain is in the 'strtol' C-function, which converts a string to a long! <sigh>
If I plainly replace the 'strtol' for a 'atoi' then the newLisp statement REPLACE does not work. So I have to think of a workaround for the 'strtol' situation...
Anybody any ideas?
You could do the following, forcing 32bit behaviour:
#ifdef TRU64
long result;
result = strtol(....)
result = result > 2147483647 ? 2147483647 : result;
result = result < -2147483648 ? -2147483648 : result;
return(stuffInteger((int)result));
#endif
Lutz
I found the following trick:
#ifdef TRU64
if (strtol(intString,(char **)0,(int)base) > 2147483647)
return(stuffInteger(atoi(intString)));
if (strtol(intString,(char **)0,(int)base) < -2147483648)
return(stuffInteger(atoi(intString)));
#endif
return(stuffInteger((UINT)strtol(intString,(char **)0,(int)base)));
}
This is at the end of the 'p_integer' function. If I run 'newlisp qa_dot', then this is the result:
Quote
>>>> array-list failed nil
>>>> array? failed nil
>>>> binomial failed nil
>>>> fv failed nil
>>>> net-receive-udp failed nil
>>>> net-send-udp failed nil
>>>> npv failed nil
>>>> semaphore failed nil
>>>> share failed nil
Only 9 left.
Nice exercise by the way... :-)
Hi Lutz, I did not see your posting, I will test your code now...
OK works well!
:-)
Continuing report:
- Changed file 'nl-string.c' lines 836-848 to ensure C integer return value
#ifdef TRU64
/* if (strtol(intString,(char **)0,(int)base) > 2147483647)
return(stuffInteger(atoi(intString)));
if (strtol(intString,(char **)0,(int)base) < -2147483648)
return(stuffInteger(atoi(intString))); */
/* Suggestion by Lutz Mueller */
result = strtol(intString,(char **)0,(int)base);
result = result > 2147483647 ? 2147483647 : result;
result = result < -2147483648 ? -2147483648 : result;
return(stuffInteger((int)result));
#endif
You may choose which solution fits best... :-) the variable 'result' was declared at the top of the function.
Continuing...
Found the cause for the newLisp 'share' statement.
The test for 'share' looks like this:
(if (< opsys 5)
(unix-test-share)
(win32-test-share))
But the 'opsys' for Tru64Unix is 9. According to this test, the code for Windows is executed and this delivers a 'nil'.
Rewrote the test to:
(if (or (< opsys 5)(= opsys 9))
(unix-test-share)
(win32-test-share))
Now after a 'newlisp qa_dot' the 'share' statement is working.
----
The UDP tests do not work because they rely on semaphore. Rewriting the semaphore test for opsys=9 did not solve it. I will hunt it down tomorrow.
Good night, 8 functions left.
Semaphore works now. UDP works.
Fixed the funancial functions 'npv', 'binomial' and 'fv'.
It appeared that the C-function pow() has another prototype on Tru64. The protoype is: pow(double, double).
So I had to cast the second argument of 'pow' to (double).
After running "newlisp qa_dot", this is the result:
Quote
TESTING FINISHED WITH ERRORS:
>>>> array-list failed nil
>>>> array? failed nil
newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.
> ring...
What worries me, is that newLisp exits immediately after the "ring"... also a fork exits newLisp. This is not correct behaviour, the prompt of newLisp should remain.
What happens when a call to 'fork' or 'timer' is invoked in newLisp? Any idea why I arrive at the command prompt?
Peter
------
Congratulations Peter, almost there!
Both, the timer and the fork send a signal. The timer sends a SIGALRM and the forked child process sends a SIGCHLD to the parent when finished.
It looks like the signal handlers defined and initialized at the beginning of newlisp.c do not work on true64.
Lutz
OK thanks! I'll see if there is a trick for those signal handlers.
In the meantime I found the last issue with the ARRAY. It appears to be caused by the fact that newLisp expects pointers to be 4 byte in size. For Tru64 Unix however, pointers are of 8 byte size.
If I compile with -xtaso_short, forcing 4-byte pointers, so compiling as 32bit, all array statements work perfect. But all of a sudden many other statements do not work anymore (like the 'format' statement, but also 'log', the UDP statements etc)
If I compile as 64bit, arrays with large dimensions will cause a coredump.
This was really a pain to find out. But anyway I know the cause now.
------------
So concluded, 2 jobs left: fix the array stuff and look at the signal handlers.
The 'timer' function does work, but after it carried out the task newLisp just exits. The same with 'fork': it works, but will exit newLisp.
The TTYLOCK.LSP demo of Norman, which uses signals extensively, works OK with 64bit newLisp. So it is strange that newLisp exits on 'fork' and 'timer'.
Tomorrow I will try to solve the last issues as far as possible and post the tar.gz with a list of changes.
Peter
Those signals from 'fork' and 'timer' work now, I needed to link with the BSD library (option -lbsd). So after a 'timer' or a 'fork' I remain in the newLisp prompt.
So I am only left with those arrays. Is there some 32bit specific stuff you do with memory and arrays?
Peter
Pretty much everything in the code relies on pointers and int's having both 4 bytes, and all would fall apart immedeately if not. 4 byte pointers are used to address symbols, to link lisp cells and to store string data.
There must be something else going on with arrays and I look into it, when I am back at home tomorrow night.
Lutz
OK thanks! In the meantime I made some observations:
Quote
newLISP v.8.5.8 on Tru64Unix, execute 'newlisp -h' for more info.
> (array 3 3 (sequence 1 6))
((1 2 3) (4 5 6) (1 2 3))
> (array 4 4 (sequence 1 6))
Segmentation fault (core dumped)
So with (larger) multidimensional array's a core is created. Therefore, newLisp passes the '(test-array)' in 'qa_dot' because the array in there has a small size. But the '(test-array-list)' fails because of the larger dimensions of the declared array.
Quote
> (array 100 (sequence 1 5))
(1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2
3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4
5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1
2 3 4 5)
> (set 'a (array 100 (sequence 1 5)))
Segmentation fault (core dumped)
So creating a 1 dimensional array is no problem, all sizes seem to work. But assigning the result to a variable cores newlisp.
Maybe it has to do with those pointer sizes again? I see that a 'allocMemory' returns a void pointer (8 bytes), but in the 'p_array' function (nl-list.c) the allocMemory is casted to UINT, which is 4 bytes.
Still that does not illuminate why 1-dimensional arrays with a large size can be created. Also many other newLisp functions should crash, which is not happening. Indeed it must be something else then.
This works:
(set 'a (array 3))
This crashes:
(set 'a (array 4))
Also I found the Compaq C Programmers Guide online:
http://h30097.www3.hp.com/dtk/Compaq_C_Compiler/doc/lrm/DOCU0040.HTM
Peter
May be memory is segmented, how is it wiht larger strings?
(dup "A" 1000000)
or
(read-file "biggerfile.txt")
That would also force a bigger contiguos memory area to be taken.
Lutz
Yoooo Pjot, some addons...
(dup "a" 10000)
does not show anything..
where (dup "a" 2047) is the maximum that works..
(read-file "core") does work though i did not test it upto the
complete size of 45948928 bytes over telnet ;-)
Norman.
May be the memory is segmented in 2k increments somehow? I will have a look into the Compaq tru64 docs this weekend.
Lutz
Lets check that...
Pjot what does the /etc/sysconfigtab show up with?
and drop a vmstat output..
Norman.
VMstat:
Quote
Virtual Memory Statistics: (pagesize = 8192)
procs memory pages intr cpu
r w u act free wire fault cow zero react pin pout in sy cs us sy id
3 348 32 46K 6641 10K 7M 1M 3M 8812 1M 6152 1 7K 2K 0 1 99
The sysconfigtab shows, next to the possible hardware drivers, many tuning options which I cannot paste here; it's just too much info.
Instead, I will try find out the memory entry's first and paste it later.
It's the variable MAX_STRING in 'newlisp.h' which is bothering us. This variable was put to 2048.
The arrays work now. The newLisp binary survives all tests in 'qa_dot'.
Peter
That doesn't make sense to me at all :)
MAX_STRING is not used in any array code. If that would be a limitation than big arrays or big dup's wouldn't work on Linux/UNIX/Win32. But again it is not used in any array or dup code. Do a grep MAX_STRING and you will see.
Lutz
If MAX_STRING is higher than 2048 then newlisp.c calls 'vasprintf' with variadic args. However, if 'vsnprintf' fails then it returns a '-1' in Tru64. This situation is only defined in your code for MinGW.
Just uploaded the last changes, see the PM.
Peter
Yes, the return value for vsnprintf() has to be checked on any port and even between different GCC's there me be a difference. Now I remember, that it gave me a lot of headache too when moving to MinGW. Returning -1 instead of the used size is the 'older' style of vsprintf(), but still used by MinGW and as you found out in true64.
Lutz
Looks like we have a working true64 port passing all tests! Thanks to Peter for all the hours he has put into this.
Lutz
Yups! many thanks for a running Tru64 Newlisp.. Greeeat enhancement!
Norman.
Some more testing on Tru64Unix 5.1B (so not 5.1A) revealed a couple of minor issues.
1) The following warning during compiletime:
Quote
cc: Warning: nl-sock.c, line 655: In this statement, the referenced type of the pointer value "&remote_sin_len" is "unsigned int", which is not compatible with "unsigned long". (ptrmismatch)
(struct sockaddr *)&remote_sin, &remote_sin_len);
----------------------------------------^
The variable 'remote_sin_len' should be a 'long' instead of an 'int'. I am not sure how serious this is, but it can easily be solved with a macro. It is strange that Tru64Unix 4.0 never complained about this.
So what do you think, change this?
2) Other not-serious compilewarnings:
Quote
cc: Warning: nl-web.c, line 360: In this statement, the referenced type of the pointer value "(char ...)0" is "short pointer to char", which is not compatible with "long pointer to char". (ptrmismatch)
while((size = strtoul(buff, (char**)0, 16)) > 0)
------------------------------------^
There are a couple of these, regarding the (char **)0. But if small or long pointer, since it is a reference to a '0', it looks unimportant to me.
3) The tests (test-exec) and (test-process) fail because the part for Win32 is executed, which assumes that a file is found in the current directory. On my account on the other machines the current path is in my $PATH, but this is not common to Unix. So these tests must be changed for (opsys=9):
(if (and (> opsys 5)(< opsys 9)) (exec "newlisp exectest") (exec "./newlisp exectest"))
...and also...
(if (and (> opsys 5)(< opsys 9)) (process "newlisp processtest") (process "./newlisp processtest"))
Furthermore I will start hosting precompiled Tru64Unix binary's at my newLisp site - that's convenient for me, too. ;-)
Peter
(struct sockaddr *)&remote_sin, &remote_sin_len);
remote_sin_len isn't used afterward so you can just declare it as 'unsigned long' previously in the function, which will not hurt the other compilers.
while((size = strtoul(buff, (char**)0, 16)) > 0)
------------------------------------^
can be changed to:
while((size = strtoul(buff, NULL, 16))
Lutz
Thanks Lutz!
By the way: I tested some obscure functions on Tru64 like '(get-url)' and the UDP multicast, and these ran fine. Good news!
Peter