Contexts as Objects

[m i c h a e l]

This is a topic of serious interest to me. I've been mulling over the ways in which the concepts of OOD/OOP can be mapped onto the functional paradigm. No aha moment yet. Considering the number of programmers who have absorbed the OO mentality, a helpful guide from 'there' to 'here' would be of considerable worth.



What I miss most from OOD are the high-level diagrams of objects and their relationships. The struggle for an individual programmer is with complexity (the problem OO was supposed to resolve). Whereas abstraction is the usual way to handle complexity in OO, I find newLISP -- a realist -- laughs at all that unnecessary structure. The more structure you add on, the more you have to learn and remember in order to use it properly. For an individual programmer, it makes sense to use a language that encourages straightforward and simple programming. But when those files begin to fill up with functions or I'd like to start with a high-level design, I start wishing for a diagrammatic way of representing all that complexity.



Having spent over 15 years in OOL (Object-Oriented Land), I'm finding my stay in the Land of Functions mildly perplexing. Maybe the answer is in the Tao-like mantra, "Be the object." Now, if I can just learn to be the function ;-)



m i c h a e l

[Dmi]

:-))



Yes. But in the other hand:



OO is only a way to develop the complex applications. All other applications of OO are not means.



But really, OO isn't mandatory. If you can do a good design, simply procedural or functional ways will fit yor needs. Unfortunately, most procedural languages have restrictions (like laks of _good_ lambdas, macro-functions, untyped symbols etc.). I belive that OO was introduced over procedural languages only to compensate that restrictions - otherwise it has never can got popularity.



Functional languages initially have no restrictions of procedural languages, so, I belive, the good developing is able completely without OO. In common, functional methods are mostly "natural-language" - oriented. And this really works fine - my newlisp code has unbeliveable not many mistakes against my similar code on procedural languages.



But I looking for a complete "functional developing style" guide somewhere...

...like there are many explanations over the internet and book stores about how OO must be properly used.

[rickyboy]

I wrote a post at http://kinghajj.blogspot.com/2007/09/contexts-as-objects-in-newlisp.html">//http://kinghajj.blogspot.com/2007/09/contexts-as-objects-in-newlisp.html detailing how to use contexts to implement classes and objects in newLISP.

Nice entry -- short and sweet.  BTW, there is an error (really, a typo causing an error) in your code:

Code Select Expand

(context 'Point)
(define (Point:Point _x _y)
  (setq x _x)
  (setq y _y))

(define (distance-from p)
  (sqrt
    (add
      (pow (sub p:x y) 2)
      (pow (sub p:y y) 2))))
(context 'MAIN)

Do you see it?  :-)

[Jeff]

http://www.cs.cmu.edu/~dst/LispBook/">http://www.cs.cmu.edu/~dst/LispBook/



That is where I learned the fundamentals of lisp.  Lisp isn't strictly functional.  Lisp had map before the concept of "functional" programming was invented.  In fact, when the book was written (sometime in the 80s, although it is entirely still applicable), the technique was called applicative programming.  The other main lisp programming style was recursive.



Lisp has other means of abstraction.  OO abstracts the noun and then uses procedural techniques to manipulate that abstraction.



When I write a large lisp program, I typically think about how I want to use it.  I write a sample of how I intend to use the code.  Then I begin to make each part of the sample work.  I will often try to abstract out every single contiguous procedure into its own function.



This results in a large number of functions, but (for me) this is a direct result of the ease of s-expressions.  By making *all* operators prefix operators, there is no syntactic difference between them.  This means that if I use a function in a specific way or for a specific purpose, I can make the rest of my code incredibly easy to read and use by abstracting that single command into its own named function.  When I do this enough, the resulting code that uses this becomes much easier to read, and when you update something, you typically only need to update things in one or two places (in the specific functions handling a particular structure, for example), rather than in the entire program.

[kinghajj]

I wrote a post at http://kinghajj.blogspot.com/2007/09/contexts-as-objects-in-newlisp.html">//http://kinghajj.blogspot.com/2007/09/contexts-as-objects-in-newlisp.html detailing how to use contexts to implement classes and objects in newLISP.

Nice entry -- short and sweet.  BTW, there is an error (really, a typo causing an error) in your code:

Code Select Expand

(context 'Point)
(define (Point:Point _x _y)
  (setq x _x)
  (setq y _y))

(define (distance-from p)
  (sqrt
    (add
      (pow (sub p:x y) 2)
      (pow (sub p:y y) 2))))
(context 'MAIN)

Do you see it?  :-)

Sorry, I don't see the typo :( What is it?

[cormullion]

(sub p:x y): your code only gives the right answer when x = y. Unluckily, your example happened to give the correct answer since x = y = 1.

[Dmi]

http://www.cs.cmu.edu/~dst/LispBook/">http://www.cs.cmu.edu/~dst/LispBook/

Very nice! Thanks Jeff.

/me gone read :-)

[cormullion]

I've been working on kinghajj's object ideas a bit, so that i can add a bit about object-oriented programming to my introduction to newlisp. I don't know anything about the subject, so I've got a few questions... I know that newLISP isn't designed for object-based programming, but it seems that people want to know more...



1: The idea of using points seems a good example. But if you create hundreds of point objects and have a reasonable number of methods in the object/class, aren't you duplicating the code every time you create a new object? Or is newlisp smart enough to not duplicate the methods...?



2: The basic way of creating points:

Code Select Expand

(new Point 'point1)
(point1 1 1)


is then made easier with the new-object macro:


 

Code Select Expand

(new-object Point point1 1 1)

If I want to create a list of point objects from code, I can do it like this for the first way:


 

Code Select Expand

(dotimes (n 20)
      (set 'p (new Point (context (sym (string {pt} n)))))
      (p (rand 100) (rand 100))
      (push p MAIN:points-list -1))
 

but how can I do it with the second, macro way?



3: context changing



In the above loop, the context keeps changing - it starts at MAIN but finishes in pt19, hence the use of MAIN. Is this the only way to use 'new'?



To sort the points list into distance from origin order, I'm  applying the distance-from to the context, then storing the distance in the points list. This is fine, but it seems to be not very object-oriented any more. Is there a better way?

Code Select Expand


 (new-object Point origin 0 0)
  (dolist (c points-list)
    (set 'dist (apply (sym 'distance-from c) (list origin)))
    (nth-set $idx points-list (list c dist)))
  (sort points-list (fn (x y) (< (last x) (last y))))


[Lutz]

Code Select Expand

aren't you duplicating the code every time you create a new object

Yes, every new copy of the template will also copy the functions (methods in OO lingo). This is why contexts as objects make only sense when the data content of the object is big compared to it's method overhead.



In one of his posts Kinghajj suggested a 'def-struct' macro. Inspired by this here is a 'def-class' macro which uses the context packaging mechanism only for the object maker, initialization, data setter and getter methods, but leaves the object data itself in a simple Lisp list.



This then creates a functional way of doing OO programming:

Code Select Expand


(define-macro (def-class)
  (let (ctx (context (args 0 0)))
    (set (default ctx) (lambda () (args)))
    (dolist (item (rest (args 0)))
      (set (sym item ctx)
      (expand '(lambda-macro (_str _val)
          (if (set '_val (eval _val))
          (nth-set ((eval _str) $idx) _val)    
          ((eval _str) $idx))) '$idx)))
ctx))


The expression '(default ctx)' is the same as '(sym (name ctx) ctx)' but much shorter and faster. The macro creates the default functor of the class/context as the object constructor and also creates the setter and getter functions for the object data members. Now we can create create functional definitions of objects:

Code Select Expand

(def-class (point x y))

(set 'point1 (point 1 2)) => '(1 2) ; constructor with init values

(point:x point1) => 1  ; get x
(point:y point1) => 2  ; get y

(point:y point1 5) ; set y to 5

point1 => (1 5)


It is easy to add other functions to the point class:

Code Select Expand

(define (point:distance p1 p2)
  (sqrt (add (pow (sub (p2 0) (p1 0)) 2)
             (pow (sub (p2 1) (p1 1)) 2)))
)

(set 'point1 (point 1 1))
(set 'point2 (point 2 2))

(point:distance point1 point2) => 1.414213562


Objects defined this way are easily nested:

Code Select Expand

(def-class (line p1 p2))
(define (line:length l)
  (apply point:distance l))

(set 'myline (line (point 1 2) (point 3 4))) => ((1 2) (3 4))

(line:length myline) => 2.828427125


Note how the point definition is nested in the line definition when constructing a new line object. When defining

the 'line:length' function the 'point:difference function can be used to define it.



Perhaps this should be the new way to promote OO programming in newLISP?



Lutz

ps: note that this is not true classic OO where the object "knows" of the class it belongs to. In the above example methods and data reside in separate memory objects. But from a code reading/writing point of view it is very OO like. Using 'new' and 'define-new' on classes created by 'def-class' even inheritance and sub-classing could be simulated.

[Jeff]

I think this is misleading.  OO is a paradigm that newLISP does not follow.  Prototyping objects is not the same as object oriented programming (and I am not arguing *for* OO in newLISP).



The reason for def-struct is not as a shorthand for creating objects; it is a way of creating records and custom types.  The purpose of this is the same as with creating functions that use other functions- abstraction, and the separation of implementation and interface, both of which make future updates and changes to the code easier.



For example, what if in the future your program needs to track a point's z-axis as well?  If you add a point-z to a type and give it a default value of 0, previous data that the program generated should not be affected (depending on the language, changes, etc).



OO in newLISP is incredibly limited by the fact that each context is a child of MAIN.  There can be no introspection and there can be no hierarchies, or objects with other objects as properties.



This is not a bad thing, of course.  OOP is not that wonderful, and it makes for very wordy programs.



What I would rather see is match and unify integrated into function definitions, so that I can bind local variables in a function based on the pattern of arguments passed, as well as break a list into its car/cdr.  Here is an example that I wrote to facilitate this:

Code Select Expand

(define (unbound? var)
  (and (symbol? var) (not (quote? var)) (nil? (eval var))
       (starts-with (string var) {[A-Z]} 0)))

(define (pattern-to-match-expr pattern)
  (cond
    ((null? pattern) '()) ; base case
    ((and (not (list? pattern)) (unbound? pattern)) '(?)) ; tautology
    ((list? (first pattern)) ; nested list
     (cons (pattern-to-match-expr (first pattern))
           (pattern-to-match-expr (rest pattern))))
    ((match '(? | ?) pattern) ; cons expression
     (unless (for-all unbound? (cons (first pattern) (last pattern)))
             (throw-error (string "Invalid pattern segment: " pattern))
             '(? *)))
    ((unbound? (first pattern)) ;  unbound variable
     (cons '? (pattern-to-match-expr (rest pattern))))
    (true ; other atom
     (cons (first pattern) (pattern-to-match-expr (rest pattern))))))

(define (pattern-to-unify-expr pattern)
  (if (and (not (list? pattern)) (unbound? pattern))
      pattern
      (clean (fn (x) (= '| x)) pattern)))

(define (match-unify)
  (let ((match-expr (pattern-to-match-expr (args 0)))
        (unify-expr (pattern-to-unify-expr (args 0)))
        (lst (args 1)))
       (unify unify-expr (match match-expr lst))))

(define-macro (match-cond)
  (letn ((var (args 0))
         (pattern (args 1 0))
         (guards (if (= 'when (args 1 1 0)) (rest (args 1 1)) '()))
         (expr (if guards (2 (args 1)) (1 (args 1))))
         (binding (match-unify pattern (eval var)))
         (next (2 (args))))
        (cond
          ((and binding (for-all true? (map (fn (g) (eval (expand g binding))) guards)))
           (map eval (expand expr binding)))
          ((null? next)
           (throw-error (string "Match failed against " (eval var))))
          (true (eval (cons 'match-cond (cons var next)))))))


match-cond is like cond, but works like this:

Code Select Expand

(setq obj '(some pattern (of lists and stuff)))
(match-cond obj
  ((A B C) (map println '(A B C))))

...with | acting to break a list into the first and rest of its contents.  This kind of thing gives newLISP a real edge and makes programs much more concise - as well as giving a function the ability to handle much more complexity without increasing its own complexity much beyond a cond statement.  It's like method overloading in functional terms.



This idea would make newLISP functions not just first class objects, but functional powerhouses that replace the entire idea of simulating OOP.  Rather than have an object with methods that perform actions on that object, you can simply pass a list around from function to function, with each function knowing how to handle the list depending on its shape.



PS Sorry for the lengthy rant.  I just got finished with a rather large Python project that was severely hindered by the fact that Python needs classes to simulate objects of any complexity.  Having something like this would have made things much, much more simple for me.

[cormullion]

Thanks, Lutz. This will repay some further study...

[m i c h a e l]

Hi cormullion!



Your post sent me back to my dusty OOP newLISP experiments. I discovered that the hash table part of the OO equation I had come up with was actually kind of useful and fun to play with on the command line:

Code Select Expand

> (load "/lisp/hash.lsp")
MAIN
> (new HASH 'h1)
h1
> (h1 '(a 1 b '(1 1) c "three"))
(("a" 1) ("b" '(1 1)) ("c" "three"))
> (h1 'b)
'(1 1)
> (h1 'a 99)
(("a" 99) ("b" '(1 1)) ("c" "three"))
> (h1 'strings (fn () (map (fn (ea) (string (ea 0) " = " (ea 1))) (h1))))
(("a" 99) ("b" '(1 1)) ("c" "three") ("strings" (lambda () (map (lambda
   (ea)
   (string (ea 0) " = " (ea 1)))
  (h1)))))
> ((h1 'strings))
("a = 99" "b = '(1 1)" "c = three"
  "strings = (lambda () (map (lambda (ea) (string (ea 0) " = " (ea 1))) (h1)))")
> (new HASH 'h2)
h2
> (h2 '(a 9 b '(88 77) c "two one"))
(("a" 9) ("b" '(88 77)) ("c" "two one"))
> (h2 'other h1)
(("a" 9) ("b" '(88 77)) ("c" "two one") ("other" h1))
> (h2 'string (fn (e) (string ((h2 'other) e) " " (h2 e))))
(("a" 9) ("b" '(88 77)) ("c" "two one") ("other" h1) ("string"
   (lambda (e) (string ((h2 'other) e) " " (h2 e)))))
> (map (h2 'string) '(a b c))
("99 9" "'(1 1) '(88 77)" "three two one")
> (h1 'string (fn () (join (h1 'strings) " ")) 'print (fn () (println ((h1 'string)) "")))
(("a" 99) ("b" '(1 1)) ("c" "three") ("strings" (lambda () (map (lambda
   (ea)
   (string (ea 0) " = " (ea 1)))
  (h1))))
 ("string" (lambda () (join (h1 'strings) " ")))
 ("print" (lambda () (println ((h1 'string)) ""))))
> (new h1 'h3)
h3
> (h3)
(("a" 99) ("b" '(1 1)) ("c" "three") ("strings" (lambda () (map (lambda
    (ea)
    (string (ea 0) " = " (ea 1)))
   (h1))))
 ("string" (lambda () (join (h1 'strings) " ")))
 ("print" (lambda () (println ((h1 'string)) ""))))
> (= (h1) (h3))
true
> (h3 'a 999)
(("a" 999) ("b" '(1 1)) ("c" "three") ("strings" (lambda () (map
   (lambda (ea) (string (ea 0) " = " (ea 1)))
   (h1))))
 ("string" (lambda () (join (h1 'strings) " ")))
 ("print" (lambda () (println ((h1 'string)) ""))))
> (= (h1) (h3))
nil
> _


You can use numbers as keys:

Code Select Expand

> (h2 42 "meaning of life")
(("a" 9) ("b" '(88 77)) ("c" "two one") ("other" h1) ("string" (lambda
   (e)
   (string ((h2 'other) e) " " (h2 e))))
 (42 "meaning of life"))
> (h2 42)
"meaning of life"
> _


You can even use functions as keys:

Code Select Expand

> (h2 + "plus")
(("a" 9) ("b" '(88 77)) ("c" "two one") ("other" h1) ("string" (lambda
   (e)
   (string ((h2 'other) e) " " (h2 e))))
 (42 "meaning of life")
 (+ <E680> "plus"))
> (h2 +)
"plus"
>


As you can see, these little critters are close to being objects (instance variables and methods), but without things like inheritance or constructors. Every time I try to implement the more complex aspects of OO using newLISP, I give up, wondering if all the object-oriented overhead and complexity is really worth it.



Oh, and before I forget, here's the code for hash.lsp:

Code Select Expand

;; @module HASH
;; This module turns humble contexts into haughty hash tables (dictionaries).
;;

(context 'HASH)

;; @syntax HASH:slots
;; Holds the hash table's key/value pairs.

(set 'slots '())

;; @syntax (HASH:HASH)
;; @return The key/value pairs.
;;
;; @syntax (HASH:HASH key)
;; @param <key> The key of the value to return.
;; @return The value of the key, otherwise nil.
;;
;; @syntax (HASH:HASH key value [key value ...])
;; @param <key> The key (can be any expression) of the value to add or replace.
;; @param <value> The value (can be any expression).
;; @return The key/value pairs.
;;
;; This default context function enables contexts to easily act as hash tables.
;;
;; @example
;; > (new HASH 'h1)
;; h1
;; > ;-o no key/value pairs yet
;; > (h1)
;; ()
;; > ;-o initializing h1's key/value pairs with a list
;; > (h1 '(a 1 b 5 c 7))
;; (("a" 1) ("b" 5) ("c" 7))
;; > ;-o getting the value for key "b"
;; > (h1 "b")
;; 5
;; > ;-o setting "b" to the new value 11
;; > ;-o we can use symbols for the keys (they are converted to strings)
;; > (h1 'b 11)
;; (("a" 1) ("b" 11) ("c" 7))
;; > ;-o adding a new key/value pair
;; > (h1 'd "d's")
;; (("a" 1) ("b" 11) ("c" 7) ("d" "d's"))
;; > ;-o adding multiple key/value pairs
;; > (h1 'string (fn () (join (map string (h1)) " ")) 'print (fn () (println ((h1 'string)) "")))
;; (("a" 99) ("b" '(1 1)) ("c" "three") ("string" (lambda () (join
;;   (map name (h1)) " ")))
;;  ("print" (lambda () (println ((h1 'string)) ""))))
;; > ;-o calling a hash function
;; > ((h1 'print))
;; ("a" 99) ("b" '(1 1)) ("c" "three") ("string" (lambda () (join (map string (h1)) " ")))
;;  ("print" (lambda () (println ((h1 'string)) "")))
;; ""
;; > _

(define (HASH:HASH)
  (set
  'ags (args)
'symbol-safe
(fn (maybe-sym)
(if (symbol? maybe-sym) (name maybe-sym) maybe-sym)
)
'arg1 (symbol-safe (ags 0))
'key-part
(fn (one-pair)
(symbol-safe (one-pair 0))
)
'replace/add
(fn (key/value)
(set
'key (key-part key/value)
'replacement (list key (key/value 1))
)
(unless (replace-assoc key slots replacement)
(push replacement slots -1)
)
slots
)
)
  (case (length ags)
    (0 slots)
    (1
(if (list? arg1)
        (set 'slots
(map
(fn (each-pair)
(list (key-part each-pair) (each-pair 1))
)
(explode arg1 2)
)
)
        (lookup arg1 slots)
)
)
    (2 (replace/add (list arg1 (ags 1))))
(true
(map replace/add (explode ags 2))
slots
)
)
)

(context MAIN)


m i c h a e l

[m i c h a e l]

Hi Lutz!



I've been test-driving your 'def-class' macro a bit and so far it feels pretty natural to use. The only problem is the name: 'def-class' looks too much like Common Lisp! I tried 'class', then switched to 'object'. Maybe 'type', 'kind', or even 'adt' for abstract data type? I think I would even prefer 'define-class' just so it would not look like CL ;-)



m i c h a e l

[rickyboy]

This idea would make newLISP functions not just first class objects, but functional powerhouses that replace the entire idea of simulating OOP. Rather than have an object with methods that perform actions on that object, you can simply pass a list around from function to function, with each function knowing how to handle the list depending on its shape.

Yes, well, this idea is common in functional languages.  Your implementation is akin to what Erlang does with pattern matching in function definitions:



http://erlang.org/course/sequential_programming.html#funcsyntax">//http://erlang.org/course/sequential_programming.html#funcsyntax



And I agree, something like this would be very helpful as an intrinsic capability in newlisp.

[Jeff]

Erlang, ML (OCaml and NJ/ML), et al, all use unification binding.  Erlang does it both in function parameter arguments (lambda lists) as well as match/receive syntax.  OCaml has a shortcut to allow both methods:

Code Select Expand

let regular_function foo bar = (* function with arguments foo and bar *);;

let regular_function = function
  | foo, bar -> (* what to do if we get foo and bar as arguments *)
  | foo -> (* what to do if we just get foo *)
;;