net-select

[frontera000]

Is there a way to wait "forever" in net-select?



Unix select() when passed NULL timeout value will do that. Looking at the manual and source code for p_netSelect() there is no way to do that in newlisp?



perhaps make the timeout value of -1 to signify that NULL will be passed to select() call?

[Lutz]

NULL for the timout evalue in 'net-select' is not implemented, but is easy to do, look for this addition in the next development version.



Meanwhile use:

Code Select Expand

(net-select socket mode 0x7FFFFFFF)

that will give you about 35 minutes of waiting time.



Lutz

[frontera000]

Great! Thanks for adding the -1 feature!



Now, I feel bad for keeping asking stuff but is it possible to have UNIX shutdown() call available in newLISP?  For UDP it doesn't matter, but for TCP shutdown() before close() on a socket does different things.  TCP close statemachine is simplex.   That is one can close only WRITE side and continue to read. This is accomplished using shutdown() indicating which side (READ or WRITE or both) to start to close.  It is then followed by close().  Without this, it is not possible to have a "half-alive" connection.  I know it is rare and most people don't think of using it but it has its uses.



I think it can be accommodated by adding extra options to net-close or just introducing net-shutdown.



What do you think?

[Lutz]

This is accomplished using shutdown() indicating which side (READ or WRITE or both) to start to close. It is then followed by close().

this is, what is done in the code right now using:  shutdown(sock, SHUT_RDWR);  But perhaps you want a flag not to do the shutdown, but wait for close() to close gracefully.



As I understand the GCC documentation, shutdown() is forcing it, while close() alone tries to wait for traffic to finish first before closing.



We could have an optional flag in 'net-close' not to do the shutdown first. On UNIX you could just call the normal newLISP 'close' for files, which would also work on sockets under UNIX and doesn't do the shutdown().



Lutz

[frontera000]

I was afraid that was what was happening. net-close seemed to do shutdown before close.  The data does not get transferred sometimes because shtudown of both side (READ and WRITE) means immediate closure of the socket connection, with no guarantee to do a "graceful close" after delivering all the data.



It will help a program I was testing to have a capability to avoid the automatic shutdown call each time net-close is called.



And perhaps also add net-shutdown?  Just to make sure that it is available for users in the future -- in case someone really wants to shutdown immediately.



Thanks.

[Lutz]

The shutdown is already there by default when you do a 'net-close', so there is not need for an extra 'net-shutdown'.



What we could have is a flag to supress the shutdown in 'net-close' in this case 'net-close' would do a close() only.



But I wonder if the problems you are observing are of different nature: When one side sends a Tcp/Ip buffer, lets say:

Code Select Expand

(net-send sock 'buff 1024)

then there is no guarantee in the Tcp/Ip protocol that that buffer arrives in 1 piece. Win32 is notorious of sometimes sending only 4 bytes in a 1st packet and then the rest few hundred in a different packet. What I mean is, you may need two (receive sock 'buff 1024). The first may receive only 10 bytes and the next the rest 1014, or it even may come in three stages. This is all valid Tcp/Ip behaviour.



I suggest that you implement some kind of protocol, either sending the number of bytes of the buffer first, waiting for some token announcing the end of transmission, or observing some 'quite-time' on the receiving side before closing the socket.



Another often used method is, to wait for the socket close of the sending client (this is how a classic HTTP transaction works, when there is now Content-length: field specified in the senders header), the receiving (net-receive ...) will then exit.



Lutz

[frontera000]

Right.  I do read multiple times to fill the receiving buffer.  Long ago, I wrote a TCP/IP implementation so I am aware of this, I hope. :-)



I agree that it is useful to have an option to not shutdown when doing net-close.



It might be also useful to allow calling shutdown with Read, Write, or Both separately, to retain competibility with original BSD socket API.  It's up to you obviously.



The P2P thingy I hacked up yesterday does exactly what you say. Sender waits for a byte back from the reader at the end. To confirm before closing. Incidentally, this extra feature breaks competibility with other existing Mole-Ster protocol implementations. :-)



Yet, TCP is designed to allow sending and doing a "user close" and still have all the data in the pipe drain properly to the receiver before the FIN is done.  That is, unless the shutdown is done first!



Anyway, it's not a big deal...

[Lutz]

yes, here is a simple way to create your own shutdown in newLISP:

Code Select Expand

(import "/usr/lib/libc.so" "shutdown")

; or on MacOS X
(import "libc.dylib" "shutdown")


now you could just supply the socket handle and use it.

Code Select Expand

(shutdown sock)

I use this for a lot of UNIC libc built-in functions like: getpid(), setuid(), kill() etc.. The return values will be the same as when using in C.



Lutz



ps: the p2p, you wrote is a nice piece, now waiting for the bitorrent program ;)