newLISP Fan Club

Forum => newLISP newS => Topic started by: Lutz on February 08, 2008, 04:32:57 PM

Title: development release newLISP 9.3.1
Post by: Lutz on February 08, 2008, 04:32:57 PM
• bug fixes

• new 'read' hooks into internal source reader



files and changes notes: http://newlisp.org/downloads/development/



no binary installers for this development release



Lutz



ps: I am out of the country for 2 weeks and will not be able to be on the internet on a daily schedule to read email, give support etc.
Title:
Post by: Jeff on February 08, 2008, 05:47:46 PM
Lutz,



You are a legend.  Thanks!



Jeff
Title:
Post by: cormullion on February 09, 2008, 12:28:28 AM
Looks fascinating... Will this function allow me to convert newLISP code into a sequence of tokens (eg like my attempt at tokenizing, tokenizer.lsp)? Eg for formatting source code...?
Title:
Post by: Jeff on February 09, 2008, 06:04:17 AM
No, it can either validate the syntactic integrity of a string or create an unevaluated expression.  But it will make tokenizing a string easier.
Title:
Post by: Cyril on February 10, 2008, 05:43:41 PM
Quote from: "cormullion"Looks fascinating... Will this function allow me to convert newLISP code into a sequence of tokens (eg like my attempt at tokenizing, tokenizer.lsp)? Eg for formatting source code...?


Call me naive, but does parse without the str-break argument not do this?
Title:
Post by: Jeff on February 10, 2008, 06:17:28 PM
No, parse is more like split-by-pattern.
Title:
Post by: Cyril on February 10, 2008, 08:33:18 PM
Quote from: "Jeff"No, parse is more like split-by-pattern.


If pattern is given. If pattern is not given, it splits by newlisp lexical rules.


> (parse "(sin (+ 1 1.5))")
("(" "sin" "(" "+" "1" "1.5" ")" ")")


To be scrupulous, parse doesn't treat [text]blablabla[/text] as a lexeme (probably it is not a lexeme from interpreter point of view), but all others are working well.
Title:
Post by: cormullion on February 11, 2008, 08:56:54 AM
One of the problems with parse is that it doesn't preserve comments. So for pretty-printing or formatting it can be considered a destructive function... :)



There was an issue with colons I seem to remember:



(parse {(define (fred:jim) (println fred jim))})


And parse turns quotes into braces, too.
Title:
Post by: Lutz on February 11, 2008, 12:18:37 PM
If you are good with regular expressions, you could use the second syntax of 'parse' or the first syntax of 'find-all' with regular expressions, to tokenize, or a combination of both. Only the parse, without any options behaves as Cormullion describes. But 'find-all' is probably better suited, because there the regular expression describes the token itself and not the space in-between, as 'parse' does.



Lutz
Title:
Post by: Jeff on February 11, 2008, 01:14:44 PM
I haven't had any luck with recursive regexing in newLISP.  What syntax is supported?  Standard PCRE with recursive references does not appear to work (at least, for me).