I/O multiplexing on pipes

[jef]

Had to perform some I/O multiplexing on pipes (read multiple pipes in parallel

until they're closed), and realized that newlisp does not provide a function like

"select" in C to monitor file descriptors.



I've looked in the documentation but didn't find any solution. I've ended doing a C

implementation to get the desired result, something like that (select_readable is a

wrapper calling my c implementation, taking a list of file descriptors and returning

the first one that is ready to read):

Code Select Expand


(import "./mylib.so" "c_select_readable")
...
; fd1 and fd2, two filedescriptors I'd like to read
(setq fds (list fd1 fd2))
(while (and fds (setq fd (select_readable fds)))
  (if (nil? (read-line fd))
    (replace fd fds)
    (println current-line)))


If you have any 100% newlisp solution, I would be interested. The peek function

might have been a candidate, but is not enough, as I need to know in a non

blocking way when a pipe has been closed.



PS: as a side note, while coding my C extension, I realized I wasn't able to get a signed

integer values as a return value, even if I mention "int" as the str-return-type during import:

Code Select Expand


$ cat returnneg.c
int returnneg()
{
        return -1;
}

$ gcc -Wall -fPIC returnneg.c -shared -o returnneg.so

> (import "./returnneg.so" "returnneg" "int")
returnneg@7F1E4B1D2645
> (returnneg)
4294967295


Any idea of what I'm doing wrong? Thanks in advance.

[Lutz]

What OS and newLISP are you using? Is your newLISP 32bit or 64bit and does it say "ffilib" in the banner? If it does not say "ffilib" in the banner, it is only compiled for the simple FFI which ignores type specs in the import statement. Only with "ffilib" in the banner it is compiled for the extended FFI described in the reference manual.



There are testcases for int sign extension in newlisp-x.x.x/util/ffitest.c and newlisp-x.x.x/qa-specific-tests/qa-libffi which do work on 32bit and 64bit Unix and 32bit Windows.



For non-blocking pipes try to create named pipes using open with the "non-block" option (only on Unix), then use peek, which will return nil on closed pipe handles.

[jef]

Thanks for the advice about named pipes and peek.



Code has been tested on  Debian/Squeeze and Debian/Jessie, on amd64.

Newlisp is compiled with ffilib.

Code Select Expand


$ gcc -m64 -fPIC ffitest.c -shared -o ffitest.dylib
$ newlisp
newLISP v.10.6.0 64-bit on Linux IPv4/6 UTF-8 libffi, options: newlisp -h

> (sys-info)
(436 576460752303423488 408 1 0 2048 0 4355 10600 1409)
> (import "./ffitest.dylib" "ret_sint32" "int")
ret_sint32@7F3325FA3C1B
> (ret_sint32 -1)
4294967295


[Lutz]

Please try the following:

Code Select Expand

(import "./ffitest.dylib" "ret_sint32" "int" "int")

(ret_sint32 -1)  

(ret_sint32 4294967295)

Note, that the import statement has two times "int", the first for the return type of the function and the second "int" for the argument to the function. Both calls to ret_sint32 should yield -1.

[jef]

Yes your code works for the ret_sint32 function, which takes an int as an argument and return an int as a result. My problems comes when I want to call a function taking no argument and returning a signed int:

Code Select Expand


$ cat returnneg.c
int returnneg()
{
        return -1;
}

$ gcc -Wall -fPIC returnneg.c -shared -o returnneg.so

$ newlisp
newLISP v.10.6.0 64-bit on Linux IPv4/6 UTF-8 libffi, options: newlisp -h

> (import "./returnneg.so" "returnneg" "int")
returnneg@7F1E4B1D2645
> (returnneg)
4294967295


After having a look at newlisp source code in nl-import.c, I realize that

libffi is not used if str-param-type is empty (nl-import.c at lines

148-149 for Unix platforms). So, I have to cheat a bit and make my call

this way to get the expected result:

Code Select Expand


> (import "./returnneg.so" "returnneg" "int" "void")
returnneg@7F1E4B1D2645
> (returnneg)
-1


[Astrobe]

int returnneg(void) is a valid prototype.It's actually "more correct" than int returnneg(): the former tells that the function takes no arguments, while the latter can take any number of arguments.



source: http://www.drdobbs.com/c-and-c-case-studies-in-compatibility/184401562">http://www.drdobbs.com/c-and-c-case-stu ... /184401562">http://www.drdobbs.com/c-and-c-case-studies-in-compatibility/184401562

[rrq]

How about net-select?

http://www.newlisp.org/downloads/newlisp_manual.html#net-select">//http://www.newlisp.org/downloads/newlisp_manual.html#net-select

It's documented to deal with sockets, but at least on linux this means file descriptors in general (implemented at nl-sock.c#p_netSelect).

Code Select Expand

(net-select (list fd1 fd2) "read" -1)
This doesn't beat the pleasure of attaching C code of your own, but it does seem to offer the functionality.

[jef]

How about net-select?.

Yes Ralph, you're totally right, it's exactly what I was looking for.

Works perfectly.