`string' generates [text] [/text] markup instead of quotes

[lotabout]

Normally, functions string will wrap the contents with quotes, just like:

Code Select Expand


(string '(a b c))
=> "(a b c)"


but when the text becomes too long, then it will wrap it with [text][/text] markup instead of quotes("). Is there a way to keep using quotes?

Code Select Expand

(string (symbols))
=> "(...)"
instead of

Code Select Expand


(string (symbols))
=> [text]( ... )[/text]

PS: I'm trying to integrate slimv with newlisp, and has to send the text in the quoted format, but the behavior of `string' differs.

[Lutz]

The difference of " or [text] tags is only for display and code parsing. [text] tags are used for strings with more than 2047 characters and they also allow to pass special characters without escaping them. But the contents of the strings itself stays the same, it contains neither quotes " or [text] tags:

Code Select Expand

(set 'str (dup "*" 3000))

When you enter str on the command line you will see the [text] tags but when you pass str to another function built-in or imported, you will pass only the "*" characters. The data in str contains neither quotes " or [text] tags.



Perhaps what you have to do is:

Code Select Expand

(set 'quoted-str (append {"} str {"})
; or
(set 'quoted-str (append """ str """)

now the the quotes are part of the data in quoted-str. The {,} and  chracters are not part of the string, they only serve the parser in newLISP.



The function string does not add quotes or tags, it only transforms the data format from a type other than string to string.



See also here: http://www.newlisp.org/downloads/newlisp_manual.html#type_ids">http://www.newlisp.org/downloads/newlis ... l#type_ids">http://www.newlisp.org/downloads/newlisp_manual.html#type_ids

[Lutz]

Just found out slimv is some kind of utility for the Vim editor, so what I was explaining in my last post may not be relevant to your problem.



But if you look here: http://www.newlisp.org/code/newlisp.vim">http://www.newlisp.org/code/newlisp.vim



you see that [text] tags are defined to be recognized as string delimiters by Vim. Perhaps something similar can be done for slimv.

[lotabout]

I just found that it directly captures the ouput of newlisp from stdout, I will search for some other ways to solve.



Anyway, I encountered another problem, maybe a bug?

it's that read-expr and eval-string will report

ERR: string token too long : ...(some string)...

if the length token is greater than about 2052.



Is this behavior expected? (nothing mentioned in the manual)



Example: some junk text.

Code Select Expand

(eval-string "(string "abc asldfjazpxucvbzixuvbyzociuvbzoicuvbzoixuyvcozixucvyjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjsssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss")")
Will cause this error.

[HPW]

The error is thrown by the string command for the same reasons Lutz explained.

Code Select Expand


(string {abc asldfjazpxucvbzixuvbyzociuvbzoicuvbzoixuyvcozixucvyjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjsssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss})

I use eval-string in the DLL flavour of newlisp and pass very big sources to it without any problems.

[lotabout]

I've tested again using other source strings, and the error occurs again.

It seems that this error is not caused by the `string' command, but the representation of long string with quotes.



That is, we have to represent long string in the format [text] long string [/text] or {long string} instead of "long string"(with quotes), I've changed this representation and it works all right.



Something I've tested:

Code Select Expand

(eval-string {(setq str [text]...(long string)...[/text])})
or (eval-string [text] (string {...(long string)...} ) [/text])

both works fine.



So there must be something strange in the newlisp scanner which might not recognize long string with quotes?



And sorry for just noticed this feature has been described in the manual.

Is there any consideration in choosing this representation? I just felt a little inconvenient with the inconsistency.

[Lutz]

Is there any consideration in choosing this representation?

The main reason of the length limitation for " quoted strings is for loading speed. newLISP's parser is very fast. This is important when loading contexts used as hashes or other cases where data is represented by Lisp expressions. These files can be many megabytes long. Code which permits unlimited length " quoted strings, would run much slower.



Also, If you load some random file or faulty source, the chance of unbalanced " quotes is very high and would cause newLISP to allocate memory again and again for a string which never terminates, instead of just finishing with an error message. Unbalanced  [text] tags are much less likely to happen by accident.