Program to count lines of code (with filtering)

[itistoday]

Hey, I wrote a program to count lines of code.. it'll go down an entire directory tree.  Move it to /usr/local/bin/numlines or wherever your preferred PATH is and run it like this:



$ cd path/to/project/dir

$ numlines



If you want to filter out someone else's code (say you have code from the Growl framework) you run it like this:



$ numlines Growl



If "Growl" is in the filename (case-sensitive) then it won't be counted as part of the total.

Code Select Expand

#!/usr/bin/newlisp

; numlines

(constant 'S_IFLNK 40960)
(constant 'SIGINT 2)
(constant 'extensions '(".h" ".cpp" ".c" ".cc" ".m" ".java" ".lsp" ".lisp" ".sh" ".py" ".rb"))

(set 'total 0)

(define (string-contains L str)
(!= '() (filter (fn (x) (find x str)) L))
)

(define (string-ends-with L str)
(!= '() (filter (fn (x) (ends-with str x)) L))
)

(define (ctrlC-handler)
(println "Total lines so far: " total)
(exit 1)
)

(define (num-lines-in-file file , (num 0) fd)
(if (string-ends-with extensions file)
(if (string-contains (2 (main-args)) file)
(if (> (length (2 (main-args))) 0)
(println "skipping file: " file)
)
(begin
(set 'fd (open file "r"))
(if fd
(begin
(while (read-line fd) (inc 'num))
(close fd)
(println num "ttlines in: " file)
)
(println "*** couldn't open: " file)
)
)
)
)
(inc 'total num)
)

(define (num-lines-in-dir dir)
(change-dir dir)
(dolist (e (directory))
(if (!= "." (e 0))
(if (directory? e)
(if (= 0 (& S_IFLNK (file-info e 1)))
(num-lines-in-dir e)
(println "skipping symbolic link: " e)
)
(num-lines-in-file e)
)
)
)
(change-dir "..")
)

; begin program

(signal SIGINT 'ctrlC-handler)
(num-lines-in-dir ".")
(println "Total: " total " lines")
(exit)

[cormullion]

Nice work! I think I'll steal the code for detecting symlinks... :) (It doesn't follow MacOS aliases but they're not used much in this context.)



Only ~83,000 lines of newLISP here. Must work harder.

[itistoday]

Nice work! I think I'll steal the code for detecting symlinks... :) (It doesn't follow MacOS aliases but they're not used much in this context.)



Only ~83,000 lines of newLISP here. Must work harder.

Wow, that's a lot of newLISP code!  Whatcha workin' on? :)

[cormullion]

Wow, that's a lot of newLISP code!  Whatcha workin' on? :)

:) Sadly nothing big or amazing. Just stuff I've started/finished/abandoned; OpenGL tutorials, maze makers and solvers, sources for blog posts, tokenizers, ASCII-art fonts (see Jack the Glypher), astronomical calculations, irc clients, two blogging apps, lots of half-finished guiserver projects. Perhaps the largest single file is the Markdown port (662 lines).



I tweaked your code a bit to produce a sorted list -  then I thought that it would be even better to count the number of parentheses as well...! After all, there's nothing significant about a 'line' in newLISP source - perhaps a paren count indicates code structure (function density) more precisely? Also, it mostly ignores comments and fancy formatting.

[Kazimir Majorinc]

The program is really cute. Guys, if you are on the job, can you add counting of the tokens - parentheses included, one pair counted as two tokens? I'd like to have it as well, but improving that program is over my Newlisp head.



Cormullion, 83 000 of lines is very impressive, especially in Newlisp.

[Kazimir Majorinc]

On the second thought maybe it would be good to count parentheses, quotes (') and other tokens independently.

[Kazimir Majorinc]

On the second thought maybe it could be good to count parentheses, quotes (') and other tokens independently.

[itistoday]

Wow, that's a lot of newLISP code!  Whatcha workin' on? :)

:) Sadly nothing big or amazing. Just stuff I've started/finished/abandoned; OpenGL tutorials, maze makers and solvers, sources for blog posts, tokenizers, ASCII-art fonts (see Jack the Glypher), astronomical calculations, irc clients, two blogging apps, lots of half-finished guiserver projects. Perhaps the largest single file is the Markdown port (662 lines).



I tweaked your code a bit to produce a sorted list -  then I thought that it would be even better to count the number of parentheses as well...! After all, there's nothing significant about a 'line' in newLISP source - perhaps a paren count indicates code structure (function density) more precisely? Also, it mostly ignores comments and fancy formatting.

Would you like to post your version here? I'd be interested in see it. :-)

[cormullion]

Code Select Expand

#!/usr/bin/env newlisp

; numlines

(constant 'S_IFLNK 40960)
(constant 'SIGINT 2)
(constant 'extensions '( ".lsp" ))

(define (string-contains L str)
  (!= '() (filter (fn (x) (find x str)) L))
)

(define (string-ends-with L str)
  (!= '() (filter (fn (x) (ends-with str x)) L))
)

(define (ctrlC-handler)
   (exit 1)
)

(define (num-lines-in-file file , (num 0) (num-parens 0) fd)
  (if (string-ends-with extensions file)
   (if (string-contains (2 (main-args)) file)
     (if (> (length (2 (main-args))) 0)
      (print "skipping file: " file)
     )
     (begin
      (set 'fd (open file "r"))
      (if fd
        (begin
         (while (read-line fd)
           (inc 'num)
           (replace {(|)} (current-line) $1 0)
           (inc 'num-parens $0)
           )
         (close fd)
         (push (list num num-parens file) results)
        )
        (println "*** couldn't open: " file)
      )
     )
   )
  )
)

(define (num-lines-in-dir dir)
  (change-dir dir)
  (dolist (e (directory "./" {^[^.]}))
     (if (directory? e)
      (if (= 0 (& S_IFLNK (file-info e 1)))
        (num-lines-in-dir e)
        (println "skipping symbolic link: " e)
      )
      (num-lines-in-file e)
     )
  )
  (change-dir "..")
)

; begin program

(signal SIGINT 'ctrlC-handler)
(num-lines-in-dir (string (env "HOME") "/lisp"))
(println (sort results (fn (x y) (< (x 1) (y 1)))))
(println "total lines " (apply add (map (fn (x) (x 0)) results)))
(println "total parentheses " (apply add (map (fn (x) (x 1)) results)))
(exit)



I just inserted a 'push' instead of a println.



Kasimir - your blog posts are excellent! I doubt whether these scripts are really very challenging...!

[cormullion]

- of course that script is Unix-specific, I think, with the references to "..". Perhaps there should be a cross-platform version...

[itistoday]

I just inserted a 'push' instead of a println.

Cool. :-)

Kasimir - your blog posts are excellent! I doubt whether these scripts are really very challenging...!

I agree, I especially enjoyed reading this one: http://kazimirmajorinc.blogspot.com/2008/06/does-newlisp-function-really-evaluate.html">Does the Function Really Evaluate to Itself?

[cormullion]

I wonder whether this take avoids a few of the cross-platform issues... Don't know, really...

Code Select Expand

(constant 'extensions '(".lsp" "txt"))

(define (string-ends-with L str)
  (exists (fn (x) (ends-with str x)) L))

(define (count-in-file pn)
 (let (contents ""  returns 0 parens 0)
   (when  (string-ends-with extensions pn)
     (set 'contents (read-file pn))
     (replace "n" contents (inc 'returns) 0)
     (replace {(|)} contents (inc 'parens) 0)
     (list returns parens pn))))
 
(define (count-in-tree dir)
   (dolist (nde (directory dir "^[a-z]"))  ; -----------?
     (set 'item (append dir "/" nde))
     (if (directory? item)
       (count-in-tree item)
       (if (set 'r (count-in-file item)) (push r results)))))

(count-in-tree (string (env "HOME") "/lisp"))

(when results
  (sort results (fn (x y) (< (x 1) (y 1))))
  (println "total lines " (apply add (map (fn (x) (x 0)) results)))
  (println "total parentheses " (apply add (map (fn (x) (x 1)) results))))

(exit)

[itistoday]

Hmm.. that's a nice way of doing it! Although I see two problems offhand with that script: first the directory separator for the system needs to be obtained somehow (I'm not sure if newLISP has a function to do that), because on windows it's "" not "/", and secondly, if you're going to use the (directory) function in that way, you should probably regex on {^[^.]} or something like that since folders can start with numbers, capital letters, etc.

[cormullion]

yes - you're right - A directory wildcard could possibly be cross-platform...



(I've never used Windows for scripting, so I wouldn't know.)



But - I seem to recall vaguely that Windows can use either "" or "/"... ?



(Odd that two Mac users are fretting about Windows conventions.... :))

[itistoday]

Hmm, yeah I don't know if windows can do "/" as well...

(Odd that two Mac users are fretting about Windows conventions.... :))

Haha. :-)