newlisp readline problem?

[frontera000]

I think the 8.9.3 newlisp.exe binary  built with GNU readline library might be causing some trouble.  Normally I use Linux and never noticed any problems before but when running windows newlisp, I noticed that sending a long expression can cause trouble.  For example, this code from clock.lsp

(set 'digits '((" ### " "  #  " "#####" "#####" "#   #" "#####" "#    " "#####" "#####" " ### "

"     ")("#   #" " ##  " "    #" "    #" "#   #" "#    " "#    " "    #" "#   #" "#   #" "  #  "

)("#   #" "  #  " "#####" " ### " "#####" "#####" "#####" "   ##" "#####" " ####" "     ")("#  

#" "  #  " "#    " "    #" "    #" "    #" "#   #" "   # " "#   #" "   # " "  #  ")(" ### " "  #

  " "#####" "#####" "    #" "#####" "#####" "   # " "#####" "  #  " "     ")))

is OK when loaded from file or pasted into newlisp-tk and evaluated. But if you paste into the newlisp.exe cmd shell it will not work.



It is only one example. I have had different problems with inputting into newlisp.exe by sending it a lot of text.



Another thing I noticed is that when running newlisp.exe inside emacs on Windows the prompt does not appear.  It seems to have something to do with GNU readline.  This is no problem on Linux by the way.



Is this is a bug report?

[Lutz]

You have to put [cmd], [/cmd] tags before and after multiline text when pasting into the console. Do not put backslashes.



Lutz

[frontera000]

Hi



thanks for your reply.



I was not using backslashes. It was just that I was pasting from my emacs screen.  The whole expression was one long line without newlines or backslashes.

(set 'digits '( (" ### " "  #  " "#####" "#####" "#   #" "#####" "#    " "#####" "#####" " ### " "     ") ("#   #" " ##  " "    #" "    #" "#   #" "#    " "#    " "    #" "#   #" "#   #" "  #  ") ("#   #" "  #  " "#####" " ### " "#####" "#####" "#####" "   ##" "#####" " ####" "     ") ("#   #" "  #  " "#    " "    #" "    #" "    #" "#   #" "   # " "#   #" "   # " "  #  ") (" ### " "  #  " "#####" "#####" "    #" "#####" "#####" "   # " "#####" "  #  " "     ")))

if you copy above (it is just one long line) and to into newlisp.exe prompt running inside windows cmd shell, you will get

string token too long " "

error.



This won't happen if you paste the same string into newlisp-tk and evaluate.

[Lutz]

There is a line length limit of 255 characters in the native newLISP console mode when not using [cmd],[/cmd] tags. The newlisp-tk frontend wraps bigger buffers into [cmd], [/cmd] tags each tag on its own line, before sending them to newLISP. newLISP then goes into a streaming mode on the receiving end to allow for unlimeted buffer length.



Lutz

[frontera000]

On linux, newlisp 8.9.3 (the same version) does not have the problem.



I understand newlisp-tk will send the string over tcp/ip and the newlisp side will read from the buffer intead of stdin.  I figured that that is why the code does not have problems.



However, the problem with cmd console windows version of newlisp.exe seems real.  



If nobody else has this problem then I guess it's just my demented mind. :-)



By the way, 255 character limit will break the newlisp.el editing mode.  The way it does communication with newlisp is by removing newlines and sending a line of code which can be the whole buffer!



Just trying to be useful by reporting weird behaviors as i find them.

[Lutz]

Win32 MinGW is not compiled with the readline library but uses the commandline editing facilities built into the Win32 OS and the buffer length is limited to 255 characters when not using [cmd], [/cmd] tags. In this non-readline compile the C function fgets() is used on a max 255 char buffer. fgets() is passed on to Win32, which supplies backspace capabilities etc..



When compiled with the libreadine on Linux and other UNIX then the libreadline function radline() is used on a C pointer, and readline() takes care of streaming and memory allocation and can eat bigger buffers. This is why you don't see the 255 char limit on Linux/UNIX compiled with libreadline.



I recommend to do the [cmd] tags wrapping in your Emacs frontend, so you are fine for all cases (Win32 and UNIX not compiled for libreadline). Wrap like this: [cmd]n...buffer...n[/cmd] effectively having [cmd] and [/cmd] on their own private lines.



Perhaps you can publish the entire improved newlisp.el or tell Tim Johnson about it?



Lutz

[frontera000]

Ah... thanks. That makes more sense.



I started reading newlisp.c.



Starting at line 699:

#ifndef READLINE

                if(isTTY || IOchannel != stdin)

                        varPrintf(OUT_CONSOLE, prompt());

#endif

This is where prompt gets printed (in Windows version).



When running in CMD shell, isTTY will be set and IOchannel will be stdin. Inside Emacs, isTTY is zero and IOchannel is the same. No prompt is printed inside Emacs shell.



Anyway, it might be helpful to allow a command line option to force the prompt when isTTY is zero and IOchannel == stdin. That will make the prompt to appear when running in "inferior mode" inside emacs.



I hope this is useful.

[Lutz]

We could have a -C flag (upper-case C) to force the prompt even when in pipe I/O mode.



Lutz

[Lutz]

Just addeed the -C option for version 8.9.4, online later today.



This also fixes a problem with the MinGW MSYS shell, which on some installations would not show the prompt. Starting newlisp.exe with the -C option fixes this problem.



Lutz

[frontera000]

great! thank you! One of the reasons I love newLISP is due to this kind of quick response and turn-around by the author.



keep up the good work!