P2P

[nigelbrown]

Given newLISP's networking capabilities and its small footprint I thought someone may find this interesting - a very simple intro but helpful to me.



"Generic P2P Architecture, Tutorial and Example"



http://www.codeproject.com/useritems/Generic_P2P_Architecture.asp">http://www.codeproject.com/useritems/Ge ... ecture.asp">http://www.codeproject.com/useritems/Generic_P2P_Architecture.asp



Maybe someone would like to do a newLISP example?

Really just an expansion of the tk/newlisp connection.

[nigelbrown]

PS another thing I'd be interested to hear about is whether anyone has used newLISP as multiple cooperative tasks that could distribute across an OpenMOSIX cluster.



Nigel

[Lutz]

Haven't read the p2p tutorial yet, but your last post about MOSIX clusters also caugth my attention.



The remote mode with the commandline options -p and -d where implemented with some sort of cluster capabilites in mind. Everything in newLISP can be serialized using 'string' and/or 'source' so it should be possible to develop some kind cooperative scheme where different newLISPs on different computers work on a task together.



And something else fits into this picture: Rovingeye (on this forum) suggested to implement a 'net-select' which could work on lists of sockets. This could be used for a coordinating newLISP process, which is 'listening' to multiple peers and is serving responses.



Lutz

[newdep]

Im not sure if its the same but ,

The net-select on lists of sockets, Im actualy working on that, if its the same, on a small project of mine. Because for multi server sessions you need to be able to track down the clients connected, thats why i asked for a (net-session) update, which is now working currectly because Lutz fixed it.



The multi-servering listener im building uses 'LISTS as the protocol (string 'list) and extract the lists again from the data. Actualy i used it a lot in Rebol, its easy and quick also in newlisp this is possible. Also very usefull

for multi-programming or remote-programming/execution of newlisp data.



The P2P document i have not read( there are too different way of using P2p so im always building my own protocol), but its not be difficult to build. Im already using a P2P for 3 years now build in TclTk.. was very easy..

Im currenlty working on P2P under newlisp, newlisp has all the ingredients...you only need lists and sockets :) But perhpas an event handler, threading or an FileEvent (like under TCL) will smooth the IO, because currently its all done within a loop..



OpenMOSIX, never heard of it but ill have a look....



Norman.

[Rovingeye]

my first network code was to build a simple socket based lock server, after lots of playing with the file locking stuff I found when I hammerd on it hard it didn't lock properly.



My lock server is a simple newlisp process that handles incoming connections, and a very simple protocol of 'l' for give me a lock, and 'u' for release my lock. The server returns a '1' when you are given a lock, and '0' when its unlocked. I even handled connection being droped with a lock would release the lock to another waiting process. I tired a number of alternative, but it always seemd the lock process was consuming too much cpu resources, either by using the timeout in the select, or a simple sleep. If I increased the timeout/sleep too far the process became unresponsive. (anyone wants the code, let me know)



I would like to experiment with p2p, and clustering, but I'm worried about performance with the current net-select system.



Care should be taken with passing experssions across the network, this is the obvious way to go, but is a very big security risk, unless there is a method in place to check their authenticity.

[Lutz]

Hi Darren,



I am playing with 'net-select' right now and realize that even a short check with 10 microseconds timeout takes about 20milliseconds.



The typical overhead of a newLISP statement is no more than 50 to 100 nano seconds at most on my 1.5Ghz. That means that these 20 milliseconds are spent in the select() call. The measurements are in the same ball park on Win32 and FreeBSD. A net-select call with a non-existing socket return sets net-error after about 17 milli seconds.



The code in newLISP seems to be straight forward and nothing wrong with it (have a look into nl-sock.c in function p_netSelect() ).



I also just finished a net-select working on lists,  that should improve performance, but again the pure overhaed of a select() call inside the OS socket stack seems to be in the 10th of milliseconds area.



Doing the net-select work I also realized that the current one is not clearing net-error which will be fixed.



Lutz

[Rovingeye]

When I talked about performance, I didn't mean the time taken in the call, just that I want to have a single machine running a number of servers for different things, smtp, chat ... and using the current net-select means I am wasting cycles polling the sockets rather than the OS handling things.

[Lutz]

Just finished testing the new list based net-select and it seems to be much faster, I wonder if it will behave better in your environment. Look for a new version 7.5.14 Friday or Saturday.



Listening on 10 ports and pollling in a tight empty loop:



(while (not (net-select '(3 4 5 6 7 8 9 10 11 12) "r" 1)))



keeps the CPU cycles in 'top' on Linux still at 0%. In comparision a:



(while true)    ;; does 99.0%



So the behaviour has definetly changed.



Lutz

[newdep]

list based net-select , good news lutz !!! thanks for that, it makes networking

much more flexible ;-)



Norman.