• bug fixes
• new 'read' hooks into internal source reader
files and changes notes: http://newlisp.org/downloads/development/
no binary installers for this development release
Lutz
ps: I am out of the country for 2 weeks and will not be able to be on the internet on a daily schedule to read email, give support etc.
Lutz,
You are a legend. Thanks!
Jeff
Looks fascinating... Will this function allow me to convert newLISP code into a sequence of tokens (eg like my attempt at tokenizing, tokenizer.lsp)? Eg for formatting source code...?
No, it can either validate the syntactic integrity of a string or create an unevaluated expression. But it will make tokenizing a string easier.
Quote from: "cormullion"
Looks fascinating... Will this function allow me to convert newLISP code into a sequence of tokens (eg like my attempt at tokenizing, tokenizer.lsp)? Eg for formatting source code...?
Call me naive, but does parse without the str-break argument not do this?
No, parse is more like split-by-pattern.
Quote from: "Jeff"
No, parse is more like split-by-pattern.
If pattern is given. If pattern is not given, it splits by newlisp lexical rules.
> (parse "(sin (+ 1 1.5))")
("(" "sin" "(" "+" "1" "1.5" ")" ")")
To be scrupulous, parse doesn't treat [text]blablabla[/text] as a lexeme (probably it is not a lexeme from interpreter point of view), but all others are working well.
One of the problems with parse is that it doesn't preserve comments. So for pretty-printing or formatting it can be considered a destructive function... :)
There was an issue with colons I seem to remember:
(parse {(define (fred:jim) (println fred jim))})
And parse turns quotes into braces, too.
If you are good with regular expressions, you could use the second syntax of 'parse' or the first syntax of 'find-all' with regular expressions, to tokenize, or a combination of both. Only the parse, without any options behaves as Cormullion describes. But 'find-all' is probably better suited, because there the regular expression describes the token itself and not the space in-between, as 'parse' does.
Lutz
I haven't had any luck with recursive regexing in newLISP. What syntax is supported? Standard PCRE with recursive references does not appear to work (at least, for me).