Additions and changes in both, the core newLISP executable and newLISP-GS.
for files and changes notes see: http://newlisp.org/downloads/development/
Lutz
So far it works fine! (MacOS X Tiger 10.4.11 Intel). And more stuff in the manual too.
Quote
any improvements to the writing in the new chapter "17. Object Oriented Programming in newLISP" are greatly appreciated.
Lutz
Hi Lutz,
Slackware 11.0 (where 9.2.5 compiled fine)>
$make linux_readline
make -f makefile_linux_readline
make[1]: Entering directory `/newlisp-9.2.6'
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX newlisp.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-symbol.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-math.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-list.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-liststr.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-string.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-filesys.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-sock.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-import.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-xml.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-web.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-matrix.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX nl-debug.c
gcc -Wall -pedantic -Wno-uninitialized -Wno-strict-aliasing -Wno-long-long -c -O2 -g -DREADLINE -DLINUX pcre.c
gcc newlisp.o nl-symbol.o nl-math.o nl-list.o nl-liststr.o nl-string.o nl-filesys.o nl-sock.o nl-import.o nl-xml.o nl-web.o nl-matrix.o nl-debug.o pcre.o -g -lm -ldl -lreadline -lncurses -o newlisp
/usr/lib/gcc/i486-slackware-linux/3.4.6/../../../crt1.o(.text+0xc): In function `_start':
../sysdeps/i386/elf/start.S:109: undefined reference to `__libc_csu_fini'
/usr/lib/gcc/i486-slackware-linux/3.4.6/../../../crt1.o(.text+0x11):../sysdeps/i386/elf/start.S:110: undefined reference to `__libc_csu_init'
collect2: ld returned 1 exit status
make[1]: *** [default] Error 1
make[1]: Leaving directory `/newlisp-9.2.6'
make: *** [linux_readline] Error 2
$
I bet 9.2.5 doesn't compile anymore on your machine too. There is only one algorithmic change for Unix/Linux in nl-math.c/p_rand(), and which doesn't involve any different library calls. All other changes only involve Win32 and Java in newLISP-GS.
Lutz
I've got some problems with special characters (in my language) in the version 9.2.6 !
The characters with an accent (e.g. "é" or "è") or other special character ("ç") count for two characters.
Example :
NewLISP v.2.9.5 :
(set 'str "Stéphanie")
(println (str 0) (str 3))
=> Sp
With NewLISP v.9.2.6 :
I must do
(println (str 0) (str 4))
to obtain
=> Sp
because of the "é" , at index #2 (and #3 !)
Sorry, if I'm not clear.
I come back to v.9.2.5 for the moment.
P.S.: I'm on Windows XP ...
You already knew i was stupid ;-)
..The problem isnt newlisp..
Im working on my 2nd linux machine which has an nfs mount.
The P4 wont compile, the AMD64 does.. machines are copy's
so thats odd... Anyway i need to throw something out of the attic window..
Unfortunately I posted the Win32 version of newLISP 9.2.6 with the newLISP-GS UTF-8 Java GUI, but the newlisp.exe excutable is compiled as
In version 9.2.6 (newLISP-GS) characters are UTF-8 encoded. In UTF-8 all non-ASCII characters are encoded in 2 to 4 bytes, i.e. the accented é is encoded as "195169", a double byte character.
The newLISP executable if compiled as UTF-8 knows about it and when doing indexing as in (str 0) (str 3) etc. can pick out variable-byte-wide characters correctly.
In the future (as already happening on Mac OS X) all binary packages will be shipped full UTF-8. The world is growing together and it makes a lot of sense to use a character set like UTF-8 able to display any character in the world. On the web we have most pages already encoded in UTF-8 and web browsers know how to handle this.
You will then be able to use no-ascii multibyte characters for names, strings and programming text in newLISP, and these programs and their output will look identical on all newLISP-GS platforms, no matter if the computer is running in France, Russia, Turkey or China.
When running the UTF-8 version of newLISP be aware that most (but not all) string functions work on character and not byte boundaries. This chapter: http://newlisp.org/downloads/newlisp_manual.html#unicode_utf8 contains a list of those functions.
Lutz
Just reposted newlisp-9206-win-gs-107.exe as consistent UTF-8 for the GUI part
see: http://newlisp.org/downloads/development/
Lutz
The new UTF compiled DLL broke my turtls-demo in the neobook demo pub.
No idea what's the problem is.
Have to futher investigate.
9.2.6 also broke my prodcut configurator (from the contest) which runs fine with 9.2.5!
Still searching for the difference.
When I load my turtle.lsp with this line:
(set 'direction -425.6857918)
I get a '12 symbol expected in function set : .6857918'
Seems it is now starting with a different default locale then before.
When I use (set-locale "C") the turtle demo is running again.
Quote
The UTF-8 version of the newlisp executable switches to the local locale on startup. The non-UTF-8 version enables the 'C' locale on startup.
Quote
the switch to the local (German) locale on the UTF-8 version enables the comma as a decimal separator, so the: -425.6857918 now gets parsed as -425 and .6857918 making the 'set' statement fail. The "C" locale assumes the dot "." as the decimal separator.
Lutz
Still having troubles with my product configurator.
What is the benefit of having UTF enabled by default on WIN EXE and DLL?
17 KB plus (DLL).
Is there a performance difference?
My others troubles seems to come from (char ...).
I used it to made a autolisp-compatibel simple ASCII-shift encoding.
And now I get different results:
Quote
newLISP v.9.2.5 on Win32.
> (char 153)
"153"
newLISP v.9.2.6 on Win32 UTF-8.
> (char 153)
"?"
What are my options now?
A non-UTF native version of (char ..) in the UTF enabled newlisp?
A own (asciichar ..) function written in newlisp?
Stick with the NON-UTF compile on WIN-DLL?
BTW: Do I have to update the compile-toolchain to MinGW 5.1.3 with gcc v.3.4.5 (as announced in change-history)
Sorry but
When I double-clic on a NewLISP script I get only a black DOS-console ... or nothing.
When I double-clic on a NewLISP-GS start-icon I get also only the splash popup.
New version is completely broken for Windows 98! (yes, I know, there is a few Windows 98 users around today, but I am one of them). It does no file I/O. I mean no I/O at all! The interpreter even cannot run scripts! And I have removed 9.2.5 release before I have noted this... No, no, don't worry, I, as a responsible citizen, have a backup, 9.2.5 is reinstalled already!
Lutz, is it possible to compile non-unicode version of newlisp executable and put in the development directory (marking clearly which is which)?
All Win32 users:
I posted http://newlisp.org/downloads/development/newlisp-9206-win-gs-107.exe again.
Thanks for all of your patience, to get this UTF-8 stuff on Win32 right, may take a while. Please make sure to report with each problem the version you are running and the country or locale of your Windows installation. I believe that France and Germany use both comma and Great Britain uses the point for decimals, but I am not sure about any other country.
For this version the newlisp.exe runtime is still UTF-8 enabled but newlisp-edit.lsp and all demo programs do a (set-locale "C") in the beginning because newlisp-edit.lsp and most demo programs use the decimal point when specifying color floats.
Worst case I can conditionally go back to non-UTF8 in both newLISP-GS (the Java part) and ship the non-UTF-8 newlisp.exe executable with it (on Win32).
I would hate to so this though, and we should at least try to get it going with all-UTF-8 :-). Java is Unicode through and through and it would be sad not to take advantage of it.
Also most newLISP users live in countries with non-ASCII character sets and would appreciate the flexibility UTF-8 brings to the table.
HPW:
I am not familiar with Autocad and do not fully understand the problem you are having, but here is a function which does more or less what you are trying to do:
(letn (len (length str) lst (unpack (dup "b" len) str))
(format (dup {%03d} (length lst)) lst)))
> (println (encode "ABC"))
41 42 43
"\041\042\043"
>
or this:
(letn (len (length str) lst (unpack (dup "b" len) str))
(format (dup {x%2X} (length lst)) lst)))
> (println (encode "ABC"))
x41x42x43
"\x41\x42\x43"
>
unpack works always on byte boundaries. Is there perhaps a way to switch Autocad to UTF-8? IT seems to use one of the 256byte ISO character set, where special characters still only use one byte?
Lutz
Quote from: "Lutz"
Alas still doesn't working with Windows 98. It just cannot open any file (including script files intended to run). Russian Windows 98, Russian locale.
I believe there are not many Windows 98 users around here (maybe I am the only one), and I don't need GS. In fact GS just doesn't run on my system (I have not reported this, because I believe it is Windows 98 specific problem, and nobody aside me uses it, and I don't need GS at all -- I am a console fan).
Maybe it is possible to put non-utf8 newlisp.exe on the site somewhere? Neither non-utf newlisp.dll nor non-utf newlisp-edit.lsp are needed by me.
For my decode problem I solved it with this:
(if utf8
(setq pchar (pack "c"(-((unpack "b"(substr cstr dcount))0)shiftval)))
(setq pchar (char(-(char(substr cstr dcount))shiftval)))
)
But still struggling with the other influenced commands:
Quote
newLISP v.9.2.6 on Win32, execute 'newlisp -h' for more info.
> (trim(trim "tinterlübket" "t")" ")
"interl129bke"
>
newLISP v.9.2.6 on Win32 UTF-8, execute 'newlisp -h' for more info.
> (trim(trim "tinterlübket" "t")" ")
"interl-übke"
The Umlauts readed from ASCII-Text files and processed in UTF gets different when I write them back to disk.
I understand your wish to move to UTF and as long you provide the makefile for NON-UTF, I could switch myself to the needed flavour.
I have set up MinGW to the version mentioned in the history and can now compile myself again.
But what happens when I do not know which flavour is installed on the end-user mashine?
Not sure if it is possible, but how about a dual-mode version which runs per default in UTF8-mode.
Then it can be set to ASCII-mode per command:
(set-encode "ASCII")
(set-encode "UTF8")
(set-encode)
>UTF8
Would add some size to the core, but everyone could use what he needs.
Quote
Yes, they get written back in UTF-8 by newLISP-GS. Your other applications probably generate files where
This is, what we will try for version 9.2.7:
(1) there will be only one version of the newLISP executable on all platforms and the utf-8 mode is switchable by a function 'set-utf8' 'true/nil'
(2) Win32 versions are by default (set-utf8 nil) while Mac OS X/Unix/Linux versions are by default (set-utf8 true) on startup. Or should Windows also be UTF8 by default ???
(3) The newLISP-GS guiserver has a gs:set-utf8 to switch the mode on an of too and will set itself to 'nil' when starting up on Windows and to 'true' when starting up on any other platform.
It seems that on Windows, although it can support UTF-8, most applications are written to use the ISO/IEC-8859 one-byte-length character set, where the accented, umlaut and other special characters are encoded in one-byte values > 128. I believe even Russian language or Hebrew language Windows uses ISO 88xx one-byte-length character sets. Perhaps some Russian and Israeli users can confirm this.
Comments from users familiar with UTF-8 issues, and specially those using multi-byte character sets (Asia, Middle East, Africa) are greatly appreciated.
Lutz
I dont understand why UTF-8 should be on by default?
From the history of computing UTF-8 is an extention to the OS
which is enabled by the user but default its off.
I would at least make unix versions default off.
Please leave it on for Mac users ... :-) I don't want to go backwards...!
Quote
(1) there will be only one version of the newLISP executable on all platforms and the utf-8 mode is switchable by a function 'set-utf8' 'true/nil'
Great! Seems the most flexibel solution for me.
(What the default will be is not so important)
actualy..what the default is IS very important..
Setting everything off by default informs the user that its not utf8. (= normal) no extra script settings needed.
Setting it to (set-utf8 true) informs the user that its about a special char case.
If UTF8 will be default in Unix please let the user select a seperate makefile to compile without UTF8 for local use.
Norman.
It would be nice to separate UTF-8 and decimal point/comma settings.
This way we could choose from 4 combinations - nonUTF-8/UTF-8 and point/comma decimal.
Fanda
You can do (set-locale "C") in the UTF-8 version and force the decimal point this way, to be different from your local locale. This is what I did for newlisp-edit.lsp and all demo files in the last newLISP-GS for Win32 posted.
But I will probably go back to non-UTF-8 for the Win32 distribition, but post a newlisp.exe with UTF-8 enabled as optional download. The guiserver will adapt automatically to whatever mode newLISP is running.
On Mac OS X all will stay the same with everything UTF-8.
Lutz
Lutz,
This means unix versions will have utf8 included and on by default as from 9.2.7?
Norman.
The newlisp core will stay as it is with different makefiles for utf8 and non-utf-8. utf-8 cannot be included in all versions, because some platforms don't offer the support for the wchar data type (unicode wide characters).
So for Linux and BSD and other UNIX things basically will stay the same. Win32 will be shipped as non-utf-8 as before, and Mac OS X will be shipped utf-8 as before.
The only change will be that the guiserver will switch to utf-8 if newLISP is running utf-8, and that it can be overwritten using gs:set-utf8.
For Win32 utf-8 compiled newlisp.exe and newlisp.dll will be put in a separate directory.
Lutz
NewLISP 9.2.6 now works on my machine, but I must just change all my accented characters ... and:
Quote from: "newBert"
When I double-clic on a NewLISP script I get only a black DOS-console ... or nothing.
... because NewLISP v.9.2.6 doesn't like accented characters in file names. So I must modify all my file names too.
But:
Quote from: "Lutz"
(...)
For Win32 utf-8 compiled newlisp.exe and newlisp.dll will be put in a separate directory.
Lutz
So I'll wait a moment before modifying anything at all :-)
Using UTF-8 on Windows may only make sense if all your other text processing tools use UTF-8 too, especially any editors you are using.
So for Windows it seems to be better to stay without UTF-8, but you always can enable it by just replacing the newlisp.exe. Guisierver will adapt automatically to it.
Lutz
Quote from: "Lutz"
It seems that on Windows, although it can support UTF-8, most applications are written to use the ISO/IEC-8859 one-byte-length character set, where the accented, umlaut and other special characters are encoded in one-byte values > 128. I believe even Russian language or Hebrew language Windows uses ISO 88xx one-byte-length character sets. Perhaps some Russian and Israeli users can confirm this.
Rissian Windows 9x/2k/XP uses one byte Windows-1251 (mostly for windows apps) mixed with one byte CP866 (file system names and console apps) mixed with two byte utf-8 (Registry and some internals in Win2k/XP).
ISO8859-5 isn't wide spreaded in Russia, but there are a few...
Compatibility with Win98 (for console scripting only, GS and newlisp-edit aren't needed) and compatibility with one-byte encodings is much sufficient for business apps (the enterprise automatisation is arhaic sometimes ;-)
Support for one-byte encodings in Unix is too mandatory for the business, because many applications are related to interacting with windows and many applications are related to interacting with commercially supported transaction systems, that are based on one byte encodings.
In the other hand - to have a switchable utf-8/single byte behavior would be nice.
But what about simple proper locale supporting? I.e., usage of OS libraries for all locale-related stuff?
The question is: what for the support for utf-8 in a system which haven't a locale libs at all?