development release newLISP 9.3.1

[Lutz]

• bug fixes

• new 'read' hooks into internal source reader



files and changes notes: http://newlisp.org/downloads/development/">http://newlisp.org/downloads/development/



no binary installers for this development release



Lutz



ps: I am out of the country for 2 weeks and will not be able to be on the internet on a daily schedule to read email, give support etc.

[Jeff]

Lutz,



You are a legend.  Thanks!



Jeff

[cormullion]

Looks fascinating... Will this function allow me to convert newLISP code into a sequence of tokens (eg like my attempt at tokenizing, tokenizer.lsp)? Eg for formatting source code...?

[Jeff]

No, it can either validate the syntactic integrity of a string or create an unevaluated expression.  But it will make tokenizing a string easier.

[Cyril]

Looks fascinating... Will this function allow me to convert newLISP code into a sequence of tokens (eg like my attempt at tokenizing, tokenizer.lsp)? Eg for formatting source code...?

Call me naive, but does parse without the str-break argument not do this?

[Jeff]

No, parse is more like split-by-pattern.

[Cyril]

No, parse is more like split-by-pattern.

If pattern is given. If pattern is not given, it splits by newlisp lexical rules.

Code Select Expand

> (parse "(sin (+ 1 1.5))")
("(" "sin" "(" "+" "1" "1.5" ")" ")")

To be scrupulous, parse doesn't treat [text]blablabla[/text] as a lexeme (probably it is not a lexeme from interpreter point of view), but all others are working well.

[cormullion]

One of the problems with parse is that it doesn't preserve comments. So for pretty-printing or formatting it can be considered a destructive function... :)



There was an issue with colons I seem to remember:

Code Select Expand


(parse {(define (fred:jim) (println fred jim))})


And parse turns quotes into braces, too.

[Lutz]

If you are good with regular expressions, you could use the second syntax of 'parse' or the first syntax of 'find-all' with regular expressions, to tokenize, or a combination of both. Only the parse, without any options behaves as Cormullion describes. But 'find-all' is probably better suited, because there the regular expression describes the token itself and not the space in-between, as 'parse' does.



Lutz

[Jeff]

I haven't had any luck with recursive regexing in newLISP.  What syntax is supported?  Standard PCRE with recursive references does not appear to work (at least, for me).