(dolist) enhancement

[Fanda]

Hello!

I like the (dolist) in the newLISP - it is similar to TCL's 'foreach', but 'foreach' has more freedom how to iterate through the list(s).



Here is the example from TCL manual:

Code Select Expand

foreach - Iterate over all elements in one or more lists

foreach varname list body
foreach varlist1 list1 ?varlist2 list2 ...? body

1) The following loop uses i and j as loop variables to iterate over pairs of elements of a single list.

set x {}
foreach {i j} {a b c d e f} {
    lappend x $j $i
}
# The value of x is "b a d c f e"
# There are 3 iterations of the loop.

2) The next loop uses i and j to iterate over two lists in parallel.

set x {}
foreach i {a b c} j {d e f g} {
    lappend x $i $j
}
# The value of x is "a d b e c f {} g"
# There are 4 iterations of the loop.

3) The two forms are combined in the following example.

set x {}
foreach i {a b c} {j k} {d e f g} {
    lappend x $i $j $k
}
# The value of x is "a d e b f g c {} {}"
# There are 3 iterations of the loop.


It would add lots of power to newLISP, if we could write something like:

Code Select Expand

(dolist ((x y) '(x1 y1 x2 y2)) (use x y))
(dolist ((name age) '("John" 32 "Richard" 41 "Lucy" 37)) (use name age))

(dolist (a '(a1 a2 a3) b '(b1 b2 b3)) (use a b))

(dolist (p '((1 2) (3 4) (5 6)))
  (dolist ((x y) p)
    (use x y)))


It is VERY READABLE and EASY to do. I AM FOR IT!



Fanda ;-)

[Lutz]

An intriguing idea, let me investigate this



Lutz

[nigelbrown]

I think most of the functionality you're suggesting for dolist is in map and apply, particularly if more of a list of lists approach is taken with data rather than a flat list.

Some egs to consider from your examples:

I've defined use as

> (define (use x y) (string "Params:" x "," y))

(lambda (x y) (string "Params:" x "," y))

> (use x y)

"Params:nil,nil"

>



yours: (dolist ((x y) '(x1 y1 x2 y2)) (use x y))

if x y are really a pair treat them that way by storing as a list of pairs then map apply over that

eg:

> (map (lambda (z) (apply use z)) '((x1 y1) (x2 y2)))

("Params:x1,y1" "Params:x2,y2")

>



yours: (dolist ((name age) '("John" 32 "Richard" 41 "Lucy" 37)) (use name age))

similarly

> (map (lambda (x) (apply use x)) '(("John" 32) ("Richard" 41) ("Lucy" 37)))

("Params:John,32" "Params:Richard,41" "Params:Lucy,37")

>



yours: (dolist (a '(a1 a2 a3) b '(b1 b2 b3)) (use a b))

using map alone

> (map use '(a1 a2 a3) '(b1 b2 b3))

("Params:a1,b1" "Params:a2,b2" "Params:a3,b3")

>



yours:

(dolist (p '((1 2) (3 4) (5 6)))

  (dolist ((x y) p)

    (use x y)))



can be:

> (map (lambda (x) (apply use x)) '((1 2) (3 4) (5 6)))

("Params:1,2" "Params:3,4" "Params:5,6")

>

The success of the above may depend on the details of use as apply doesn't evaluate the args before applying eg

> (define (use x y) (+ x y))

(lambda (x y) (+ x y))

> (use 2 3)

5

> (setq x1 2 x2 3)

3

> (use x1 x2)

5

> (apply use '(x1 x2))



value expected in function + : x



> (apply use '(2 3))

5

>





I find an advantage of lisp is 'map'ing rather 'do'ing (very FORTRAN77).

however, map is more about collecting the results into a list of results while dolist is applying the function for its side effects and only returning the last value.



Nigel



PS not knocking FORTRAN, just used as a canonical style example.



PPS always wanted to find an opportunity to use 'canonical' so took the opportunity to slip it in - hope I used it properly.

[Fanda]

Few comments:

- Thanks for rewriting it to LISP ;-) I am new to LISP and I know about map and apply, just haven't used to them much.



- Yes, you are right that it is more imperative style of programming than functional. I just don't know, how to deal with flat lists - you have to make the pairs first and only after that you can use map or apply. (If you create your own list, you can do it in pairs.)



- I tried all your examples. Apply doesn't evaluate the args before applying, so we can do it by ourselves:

(apply use (map eval '(x1 x2)))



There are reasons to use enhanced dolist:

1) flat lists - need to be converted to list with pairs

2) when (use x y) isn't a simple function, but more complicated code - as it is usually with dolist. (Actually I used (use x y) only as a symbolic example.)



Lutz has the last word :-)



Fanda

[Fanda]

OK, I will add something more :-)



Right now, I am learning how to do conversions between different lists. For example - I want to iterate on points, when I have a list of lines:



point P = (x y)

line  L = (P1 P2)

list    = (L1 L2 ...)

Code Select Expand

(setq N '(((1 2) (3 4)) ((5 6) (7 8)) ((9 10) (11 12))))


I tried different ways:

1)

Code Select Expand

(setq r '())
(map (lambda (X) (map (lambda (Y) (push Y r -1)) X)) N)
(println "map:    " r)
; => map:    ((1 2) (3 4) (5 6) (7 8) (9 10) (11 12))


2)

Code Select Expand

(setq s '())
(dolist (x N)
  (dolist (y x)
    (push y s -1)))
(println "dolist: " s)
; => dolist: ((1 2) (3 4) (5 6) (7 8) (9 10) (11 12))

Now I can use map or dolist again to iterate on points.



Version 1 is hard to read (at least for me). I can do this, it helps reading a little bit:

Code Select Expand

(setq r '())
(map (lambda (X)
  (map (lambda (Y)
    (push Y r -1))
  X))
N)


Version 2 is easier to read and also 2x faster. If you go one level down to 'x' and 'y' it is 3-4x faster.



Fanda

[Fanda]

Just adding:



With new 'dolist' we could do

Code Select Expand

(dolist ((x y) (flat N)) (use x y))
OR

Code Select Expand

(dolist (l N)
  (dolist ((p1 p2) l)
    (use p1 p2)))


If you know the other way, please let me know.



Thank you, Fanda

[Dmi]

Possible, good function for grouping and disposing of lists will be good support.

I.e., something like:

Code Select Expand

(set 'n '(1 2 3 4 5 6 7))
(group 3 n) => ((1 2 3) (4 5 6) (7))
(set 'm '(7 6 5 4 3 2 1))
(dispose m n) => ((7 1) (6 2) (5 3) (4 4)....)

so we could write then:

Code Select Expand

(dolist (a (group 3 n)) (use (a 0) (a 1) (a 2)))
"Tips and tricks" section has an example of pairing function trough conversion to an array and using integer divisoin of an index.

Anybody knows, is it efficient?



PS.

BTW possibility to assign a list of values to list of symbols can be useful (imho).

I.e., something like (not exact of course ;-)

Code Select Expand

(set '(a b c) (1 2 3))
will result: a=1 b=2 c=3 - and similar in dolist etc.

[nigelbrown]

How about mapping slice across an index list generated by sequence

as

> (slice '(1 2 3 4 5 6) 0 2)

(1 2)

eg

> (setq p '(1 2 3 4 5 6))

(1 2 3 4 5 6)

the definition:

> (define (dispose m n) (map (lambda (i) (slice m i n)) (sequence 0 (sub (length m) 1) n)))

(lambda (m n) (map (lambda (i) (slice m i n)) (sequence 0 (sub (length

     m) 1) n)))

test it:

> (dispose p 2)

((1 2) (3 4) (5 6))

> (dispose p 3)

((1 2 3) (4 5 6))

>

It doesn't check for correct length (but could) so

> (dispose p 4)

((1 2 3 4) (5 6))

>



Nigel

[Lutz]

Here is a function from http://newlisp.org/index.cgi?page=Code_Snippets">http://newlisp.org/index.cgi?page=Code_Snippets to 'group' elements in a flat list:

Code Select Expand


;; pair - put members of a list into pairs
;;
;; (pair '(a b c d e f)) => ((a b) (c d) (e f))
;;
(define (pair lst)
  (array-list (array (/ (length lst) 2) 2 lst)))


The function can easily be rewritten for numbers other than 2 with an additional parameter and  you have the 'group' function suggested by DMI.



Also, use:

Code Select Expand


(map set '(a b c) '(1 2 3))


For the list set suggested.



Lutz

[nigelbrown]

Lutz,



Much cleaner using your array approach.



Nigel

[Dmi]

Cool! I Like it :-)

[Fanda]

Dmi:

(dolist (a (group 3 n)) (use (a 0) (a 1) (a 2)))

This is why I would like to have an enhanced 'dolist'. Then you could just do:

Code Select Expand

(dolist ((x y z) n) (use x y z))
Grouping and indexing is done inside the 'dolist'.





Nigel: Your (dispose m n) is in fact a grouping function. (It works great, I like it! :-). Dmi meant by (dispose) the function that merges two lists together.





Lutz: (pair) function needs to have a good length of the list, otherwise it looses a part of the list (good or bad?):

> (pair '(1 2 3 4 5))

((1 2) (3 4))





Yes, I am learning, I am learning :-)



Fanda



ps: Just let me know, if I bug you ;-)

[nigelbrown]

Yes my 'dispose' is group - I made the classic mistake of not reading the question properly.



Here is a dispose that is just a wrapper around the builtin newLISP function transpose.

using args lets you accept any number of lists

viz

 

> (define (dispose) (transpose (args)))

(lambda () (transpose (args)))

> (dispose '(1 2 3) '(4 5 6))

((1 4) (2 5) (3 6))

> (dispose '(1 2 3) '(4 5 6) '(7 8 9))

((1 4 7) (2 5 8) (3 6 9))

> (setq m '(1 2 3 4 5 6 7))

(1 2 3 4 5 6 7)

> (setq n '(7 6 5 4 3 2 1))

(7 6 5 4 3 2 1)

> (dispose m n)

((1 7) (2 6) (3 5) (4 4) (5 3) (6 2) (7 1))

>



Nigel

[Fanda]

Nigel: Thanks for showing me that function.

I was doing:

> (setq L '((1 2) (3 4) (5 6)))

((1 2) (3 4) (5 6))

> (map (lambda (x) (nth 0 x)) L)

(1 3 5)



And now I can also do:

((transpose L) 0)

(1 3 5)



[ 'dolist' version: ]

(setq r '())

(dolist (x L)

  (push (x 0) r -1))



For 1000000 times the function 'time' returns:

map/transpose/dolist  -  2047,672,1266



Transpose is the winner, but 'dolist' is still 1.6x faster than 'map'. That's one of the reasons why use it.



Lutz: If I can ask - what is your comment on the enhanced 'dolist'? Have you done any tests with it? I think it could be useful (e.g. recursive functions), more readable (many times nested lists) and faster. Sorry, if I bother you too much.



Thank you all, Fanda

[nigelbrown]

The apparent slowness for map is in the function you chose to map from my timings.

Instead of (lambda (x) (nth 0 x)) just use 'first'

eg for

> L

((1 2) (3 4) (5 6))

>

we have

> (map first L)

(1 3 5)



Here are my timings:



> (time (dotimes (i 100000) (map first L)))

331

> (time (dotimes (i 100000) (map (lambda (x) (nth 0 x)) L)))

843

> (time (dotimes (i 100000) (setq r '())(dolist (x L)(push (x 0) r -1))))

519



dolisting first without collecting results is the same as map:

> (time (dotimes (i 100000) (dolist (x L) (first x))))

334

>

Which suggests the preference for either map or dolist doesn't hinge on time performance - unless you do need to collect results in which case map is more efficient.



Nigel

PS let me know if I've overlooked something in my timings