Lutz: get-cell function patch

[ryuo]

I have prepared a brief patch for adding a way to construct a newLISP cell from a pointer to an existing cell. I would prefer to make it a reference to the original cell instead of it getting garbage collected when the symbol's scope ends, but I did not see any easy way to do this. Instead, I wrote a function which takes a cell's pointer from the output of (first (dump foo)) and creates a new duplicate cell.

Code Select Expand


diff -ur a/nl-string.c b/nl-string.c
--- a/nl-string.c 2014-04-08 09:54:24.000000000 -0500
+++ b/nl-string.c 2014-06-14 11:07:48.972430323 -0500
@@ -1359,6 +1359,11 @@
 return(stuffString((char *)getAddress(params)));
 }

+CELL * p_getCELL(CELL *params)
+{
+return(copyCell((CELL *)getAddress(params)));
+}
+
 CELL * p_getInteger(CELL * params)
 {
 return(stuffInteger(*(int *)getAddress(params)));
diff -ur a/primes.h b/primes.h
--- a/primes.h 2014-04-08 09:54:24.000000000 -0500
+++ b/primes.h 2014-06-14 10:49:06.035008299 -0500
@@ -245,6 +245,7 @@
  {"bits", p_bits, 0},
  {"get-float", p_getFloat, 0},
  {"get-string", p_getString, 0},
+ {"get-cell",    p_getCELL,      0},
  {"get-int", p_getInteger, 0},
  {"get-long", p_getLong, 0},
  {"get-char", p_getChar, 0},
diff -ur a/protos.h b/protos.h
--- a/protos.h 2014-04-08 09:54:24.000000000 -0500
+++ b/protos.h 2014-06-14 10:56:08.637362480 -0500
@@ -252,6 +252,7 @@
 CELL * p_getInteger(CELL * params);
 CELL * p_getLong(CELL * params);
 CELL * p_getString(CELL * params);
+CELL * p_getCELL(CELL *params);
 CELL * p_getUrl(CELL * params);
 CELL * p_getenv(CELL * params);
 CELL * p_global(CELL * params);


Where is this useful? I came up with the idea for allowing the FOOP API to also wrap function callbacks for C libraries. Many times these include a "user data" void pointer. I figured a good usage of this would be to allow the passing of a persistent cell's pointer as the "user data" pointer so it could be passed as an argument to a regular lisp function. The missing part is this builtin function to convert the pointer back to a CELL so it can be passed to registered user callbacks as an object rather than as an opaque pointer like a regular C library callback.



Lutz, if this function is acceptable, then I would also be willing to write the documentation. But I'm not going to do that unless I can get this approved somehow. I do not know newLISP as well as you do obviously, but I would really like some way to access a FOOP object (or other cells) from the registered C API callback. Being able to convert a pointer back to a usable cell, either as a reference or a copy, is something I think would be very useful for allowing FOOP to better act as a wrapper for C APIs. I have briefly tested it and it appears to work, but I don't know the newLISP innards very well.



Thanks for reading.

[Lutz]

The implementation looks correct. We could put it as a second call pattern of the copy function using an additional true flag:



(copy <expr>) copies an expression

(copy <address> true) copies an expression from memory address



But I am not convinced yet about the practicality. Can you show us a real world example using this?

[ryuo]

I can't give much of a "real world" example as the functionality doesn't exist yet. But I can give a theoretical example similar to what I had in mind. It will not be for a real C API, as it is just an illustration of the syntax I wanted to make possible.

Code Select Expand


; void *fooCreate();
; void fooAddCallback(void *self, void (*callback) (void*,void*), void *user_data);
; void fooToggleActive(void*self);
; void fooDestroy(void*self);
; Assume these were imported already under context Foo.

(context 'WidgetFoo)

(new Class 'MAIN:WidgetFoo)

; Data == object reference. Should never be called by user directly.
(define (FooCallback Ptr Data)
(let (Cell (get-cell Data))
; All user callbacks called with object as first argument
; and their data as second argument.
(dolist (x (cell 2))
(eval (Data 0) (Cell 1) (Data 1))
)
)

(define (WidgetFoo)
; First real member is the C pointer.
; Second is a list of callbacks.
; Also register the object's private callback during creation.
(let (Ptr (Foo:fooCreate))
(Foo:addCallback Ptr (callback FooCallback "void" "void*" "void*") (first (dump (self))))
(list 'MAIN:WidgetFoo Ptr '())
)
)

(define (AddCallback Function Data)
; Push the new user callback into our list.
; This assumes it is not adapted with the
; callback function.
(push (list Function Data) (self 2))
(self)
)

(define (ToggleActive)
(Foo:fooToggleActive (self 1))
(self)
)

(define (Destroy)
(Foo:fooDestroy (self 1))
; Set the object's members to invalid objects.
; Hopefully provide some detection of reuse after free.
(setf (self 1) 0)
(setf (self 2) nil)
(self)
)

(context 'MAIN)

(set 'widget (FooWidget))

; Widget is a copy of the FOOP object.
; Because it is a copy and not a reference, you should only
; call methods that work with the opaque C object pointer.
; Data should be a reference to something in the user's data.
(define (MyCallback Widget Data)
(:ToggleActive Widget)
)

(:AddCallback widget MyCallback nil)

(:Destroy widget)


It is not intended to be usable, as the API I need for it to work doesn't exist yet. It is mainly an example of how it could be used with FOOP objects that are mainly wrappers around C object APIs, and contain only information for bridging the gap between the object layers of these two different languages.



I have a few reasons for wanting this. It allows newLISP to work like how callbacks work for other languages that bind C libraries. They usually pass an object as the first argument or a hidden argument to a function within their own language. It doesn't expose any low level details from C, such as pointers. This also helps my other reason of wanting to encapsulate most if not all the low level C details so people less knowledgeable than myself can probably still use the object's class. And I feel like this approach allows you to create a more intuitive API for users. If you were new to this language, you'd probably expect your function callbacks to have access to the object they are connected to, not to be passed a "random integer" (which may be what you think it to be before you know what pointers are) instead.



It has only one major drawback I can see. The FOOP object passed to the user callback cannot be modified in a meaningful way, because it's just a copy. But it works fine if the object contains only read-only information itself. Which is likely to be the case for a FOOP class that is only used as a convenience layer between itself and the C source code. Mainly because all the meaningful changes happen outside of newLISP. And none of that needs to be stored in the objects. Only the pointer and some other private details such as the list of registered callbacks.



Thanks for reading.



TLDR: It provides a way for passing the FOOP object as the first argument to a user callback. This allows you to use the FOOP object methods from within the user callback instead of the C functions to manipulate it.

[Lutz]

See the v.10.6.1 changes notes:



http://www.newlisp.org/downloads/development/inprogress/CHANGES-10.6.1.txt">http://www.newlisp.org/downloads/develo ... 10.6.1.txt">http://www.newlisp.org/downloads/development/inprogress/CHANGES-10.6.1.txt