Text only | Text with Images

newLISP Fan Club

Forum => newLISP in the real world => Topic started by: m i c h a e l on December 09, 2008, 07:35:14 AM

Title: wp in newLISP
Post by: m i c h a e l on December 09, 2008, 07:35:14 AM
Dear Club,



I came across this listing (//http) of a word count program implemented in various languages (see README.txt (//http) for more info). As you can probably guess, there was no newLISP version!



So I wrote the following, which seems to work:


Code Select
#!/usr/bin/newlisp

(new Tree 'counts)

(while (read-line)
(dolist (word (parse (current-line) " "))
(counts word (inc (counts word)))
)
)

(dolist (each (sort (counts)))
(println (each 0) " " (each 1))
)

(exit)


Does anyone see room for improvement? If not, I'll submit this to Felix in a few days' time.



m i c h a e l
Title:
Post by: Lutz on December 09, 2008, 08:07:53 AM
Here is a variation with some explanations:


Code Select
(while (read-line)
    (bayes-train (parse (current-line) "\s+" 0) 'Counts))


(dolist (each (Counts))
    (println (each 0) " " (each 1 0)))

; try it out

newlisp counter.lsp < sometext.txt


the program uses the same hash-tree data structure as yours, but uses newLISP's built-in 'bayes-train' function to do the counting. Just like the hash function it will prepend an underscore to the symbol, but hide it when getting the word list using (Counts).



The example programs in the link mostly split by white-space, which would be "\s+", but I prefer "\W+" which does a better job rejecting funny characters, i.e. when parsing from HTML text.



The print functions is almost the same as yours. Sorting is really not necessary, because the list produced by (Count) is already sorted. I need and extra index 0, because 'bayes-train' parenthesizes the counts (there could be more counts in a list). As a bonus you have the toal of all words in Counts:total.
Title:
Post by: cormullion on December 09, 2008, 09:24:13 AM
What's this then?


Code Select
(new Tree 'counts)
Title:
Post by: Lutz on December 09, 2008, 09:40:46 AM
see here:

http://www.newlisp.org/downloads/newlisp_manual.html#system_symbols



and here:

http://www.newlisp.org/downloads/newlisp_manual.html#hash
Title:
Post by: cormullion on December 09, 2008, 10:59:26 AM
Ah, OK.  The same as (define counts:counts).  Must have gone to sleep when reading that page...! :)
Title:
Post by: xytroxon on December 09, 2008, 03:53:03 PM
Quote from: "cormullion"Ah, OK.  The same as (define counts:counts).  Must have gone to sleep when reading that page...! :)


Don't feel bad...


Quote
(counts word (inc (counts word)))


Gave me (and all Lispers), pause... ;)



Should we be using .nl instead of .lsp file extentions for our "newLISP" programs?



-- xytroxon
Title:
Post by: cormullion on December 10, 2008, 10:07:43 AM
Yes, it looks pretty idiomatic...



Perhaps with capital letter for the hash thingy would look slightly better:


Code Select
(Counts word (inc (Counts word)))

The slightly out of step thing at first glance is the double-access (Counts word) X 2.



Interesting suggestion for ".nl" as the suffix!
Title:
Post by: xytroxon on December 10, 2008, 12:15:02 PM
Quote from: "cormullion"
Interesting suggestion for ".nl" as the suffix!


I have to use .nl if I want to use it on my personal (Abyss) server with another Lisp installed. The server uses the extension to know which interpreter to run.



And also .nl can then be expanded...



.nl runs on all machines

.nlm runs on Mac only

.nlx runs on Linux only

.nlw runs on Windows only

etc...



-- xytroxon
Title:
Post by: newdep on December 10, 2008, 12:46:52 PM
what runs on all 64 bits only machines with utf-8 enabled? ;-)
Title:
Post by: cormullion on December 10, 2008, 01:23:56 PM
.nl appears to be used only by "Norton Disk Library" something... It could be a good idea to have an extension that doesn't clash with other Lisps... I think I'll try it out.
Title: Re: wp in newLISP
Post by: Cyril on December 16, 2008, 07:51:11 AM
Quote from: "m i c h a e l"I came across this listing (//http) of a word count program implemented in various languages (see README.txt (//http) for more info). As you can probably guess, there was no newLISP version!


That's fine! I have the similar listing of my own long ago, you can find it here (//http). And, of course, I do have newLISP! ;-) I have also some languages not found in the listing found by you. And I solve a bit different task: I sort output in alphabetical order of words, not by count. But there is a caveat: all the readme-like text is in Russian language. But you can read source texts nevertheless.
Title:
Post by: m i c h a e l on December 30, 2008, 05:49:28 PM
Quote from: "Cyril"That's fine! I have the similar listing of my own long ago, you can find it here.


Yes, I did miss your page! Google's Language Tools did a fair job of translating it, so I was able to read it, no problem. Your newLISP implementation matches Lutz's above recommended changes, too.



With Lutz's changes (and a switch from counts to Words), we get:


Code Select
#!/usr/bin/newlisp

(new Tree 'Words)

(while (read-line)
(bayes-train (parse (current-line) {s+} 0) Words)
)

(dolist (each (Words))
(println (each 0) " " (each 1 0))  ; implicit indexing
)

(exit)


m i c h a e l
Text only | Text with Images