wp in newLISP

[m i c h a e l]

Dear Club,



I came across this http://ptrace.fefe.de/wp/">listing of a word count program implemented in various languages (see http://ptrace.fefe.de/wp/README.txt">README.txt for more info). As you can probably guess, there was no newLISP version!



So I wrote the following, which seems to work:

Code Select Expand

#!/usr/bin/newlisp

(new Tree 'counts)

(while (read-line)
(dolist (word (parse (current-line) " "))
(counts word (inc (counts word)))
)
)

(dolist (each (sort (counts)))
(println (each 0) " " (each 1))
)

(exit)


Does anyone see room for improvement? If not, I'll submit this to Felix in a few days' time.



m i c h a e l

[Lutz]

Here is a variation with some explanations:

Code Select Expand

(while (read-line)
    (bayes-train (parse (current-line) "\s+" 0) 'Counts))


(dolist (each (Counts))
    (println (each 0) " " (each 1 0)))

; try it out

newlisp counter.lsp < sometext.txt


the program uses the same hash-tree data structure as yours, but uses newLISP's built-in 'bayes-train' function to do the counting. Just like the hash function it will prepend an underscore to the symbol, but hide it when getting the word list using (Counts).



The example programs in the link mostly split by white-space, which would be "\s+", but I prefer "\W+" which does a better job rejecting funny characters, i.e. when parsing from HTML text.



The print functions is almost the same as yours. Sorting is really not necessary, because the list produced by (Count) is already sorted. I need and extra index 0, because 'bayes-train' parenthesizes the counts (there could be more counts in a list). As a bonus you have the toal of all words in Counts:total.

[cormullion]

What's this then?

Code Select Expand

(new Tree 'counts)

[Lutz]

see here:

http://www.newlisp.org/downloads/newlisp_manual.html#system_symbols">http://www.newlisp.org/downloads/newlis ... em_symbols">http://www.newlisp.org/downloads/newlisp_manual.html#system_symbols



and here:

http://www.newlisp.org/downloads/newlisp_manual.html#hash">http://www.newlisp.org/downloads/newlis ... .html#hash">http://www.newlisp.org/downloads/newlisp_manual.html#hash

[cormullion]

Ah, OK.  The same as (define counts:counts).  Must have gone to sleep when reading that page...! :)

[xytroxon]

Ah, OK.  The same as (define counts:counts).  Must have gone to sleep when reading that page...! :)

Don't feel bad...

(counts word (inc (counts word)))

Gave me (and all Lispers), pause... ;)



Should we be using .nl instead of .lsp file extentions for our "newLISP" programs?



-- xytroxon

[cormullion]

Yes, it looks pretty idiomatic...



Perhaps with capital letter for the hash thingy would look slightly better:

Code Select Expand

(Counts word (inc (Counts word)))

The slightly out of step thing at first glance is the double-access (Counts word) X 2.



Interesting suggestion for ".nl" as the suffix!

[xytroxon]

Interesting suggestion for ".nl" as the suffix!

I have to use .nl if I want to use it on my personal (Abyss) server with another Lisp installed. The server uses the extension to know which interpreter to run.



And also .nl can then be expanded...



.nl runs on all machines

.nlm runs on Mac only

.nlx runs on Linux only

.nlw runs on Windows only

etc...



-- xytroxon

[newdep]

what runs on all 64 bits only machines with utf-8 enabled? ;-)

[cormullion]

.nl appears to be used only by "Norton Disk Library" something... It could be a good idea to have an extension that doesn't clash with other Lisps... I think I'll try it out.

[Cyril]

I came across this http://ptrace.fefe.de/wp/">listing of a word count program implemented in various languages (see http://ptrace.fefe.de/wp/README.txt">README.txt for more info). As you can probably guess, there was no newLISP version!

That's fine! I have the similar listing of my own long ago, you can find it http://slobin.pp.ru/bench/">here. And, of course, I do have newLISP! ;-) I have also some languages not found in the listing found by you. And I solve a bit different task: I sort output in alphabetical order of words, not by count. But there is a caveat: all the readme-like text is in Russian language. But you can read source texts nevertheless.

[m i c h a e l]

That's fine! I have the similar listing of my own long ago, you can find it here.

Yes, I did miss your page! Google's Language Tools did a fair job of translating it, so I was able to read it, no problem. Your newLISP implementation matches Lutz's above recommended changes, too.



With Lutz's changes (and a switch from counts to Words), we get:

Code Select Expand

#!/usr/bin/newlisp

(new Tree 'Words)

(while (read-line)
(bayes-train (parse (current-line) {s+} 0) Words)
)

(dolist (each (Words))
(println (each 0) " " (each 1 0))  ; implicit indexing
)

(exit)


m i c h a e l