Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - Jeff

Pages1 2 3 ... 41
#1
newLISP newS / Re: libevent2
January 05, 2013, 02:57:44 PM
Here is an example of how a simple echo server might look:


Code Select
(load "libevent2.lsp")

(setf listen (net-listen 8000))

(unless listen
  (throw-error (net-error)))

(libevent:init)
(libevent:watch listen libevent:READ
  (lambda (fd e id , client)
    ; accept a new connection
    (setf client (net-accept listen))

    ; watch for requests
    (libevent:watch client libevent:READ
      (lambda (fd e id , req)
        ; read request
        (setf req (read-line fd))

        ; On a real server, here is where you would process the request,
        ; most likely in a secondary process. In that case, you would execute
        ; the code below when it finished, rather than immediately after
        ; the request comes in.

        ; wait for client to be ready to accept a response
        (libevent:watch-once client libevent:WRITE
          (lambda (fd e id)
            ; send response
            (write fd req)))))))

(libevent:run)

A real server would have to watch for timeouts, disconnects, etc., as well, but this gives you the basic idea.
#2
newLISP newS / Re: libevent2
January 05, 2013, 02:02:21 PM
Actually, that is not exactly true. That is more how signals work, although not in libevent. In libevent, the application is controlled by the loop. Before starting the loop, you register at least one socket or timer to run in it. It is an error to run a loop with no events registered.



For example, to implement a server, you would register the listening socket. When a remote client connects, your registered callback is triggered. You accept the new connection, creating a new socket, and register an event handler for that socket as well. When it sends a request, your handler is triggered with a read event. Your app determines its response, which it sends on the next write event.



At no point does execution yield. If your server takes a long time to build a response to a request, the entire application will block. This is where multiple processes might be used, allowing the server process to continue to service clients while child processes do the work of interpreting requests and generating responses.
#3
newLISP newS / libevent2
January 04, 2013, 12:01:31 PM
Howdy folks. Haven't been around in a while. Wanted to let you know that I put together a wrapper for libevent2 using the new FFI features. Made it very easy. You can find it on github here - https://github.com/jsober/nl-event">https://github.com/jsober/nl-event.
#4
newLISP newS / Re: libmemcached api
December 15, 2009, 03:51:03 AM
Without checking the code again, it looks like you are doing things right. Unfortunately, I do not have a lot of time for personal projects at the moment. If you can find the bug and fix it, I would be happy to update the sources on my site and google projects.
#5
newLISP in the real world /
October 13, 2009, 07:43:49 AM
Le pido perdon. Hablo solamente ingles y espanol. Y lisp ;)
#6
newLISP in the real world /
October 13, 2009, 07:41:51 AM
Was he trying to append "-Schwanz" or "-Schwanz-Schwanz-Schwanz"?
#7
Whither newLISP? /
October 13, 2009, 07:30:30 AM
In fact, I think *I* may have written that :)
#8
newLISP in the real world /
October 13, 2009, 07:21:22 AM
You need to pass map a function. You can do that with curry:


Code Select
(map (curry add-tail (dup "-Schwanz" 3 true)) '("Hund" "Katze" "Pinguin"))
#9
Whither newLISP? /
October 13, 2009, 06:39:18 AM
Thank you, Lutz. This is exactly the hook we need to extend the language. This is a good thing for the language.



Will reader-event be hooked into run-time compilation as well? That is, when I call read-expr, will the expression be first run through reader-event? I believe it should, so that translations that occur in one place happen everywhere. At the very least, it should be default. If it does not, it creates an inconsistency that must be accounted for in every program using read-expr and reader-event. It adds a workaround that must be memorized by new users, too.



It should not need to be hooked into load. Anything written to file should be in the expanded form in any case (at least, any code expressions written to file).
#10
Whither newLISP? /
October 13, 2009, 05:25:13 AM
Lutz, one other thing. Shouldn't this be reader-event, not read-event?


Code Select
(define (resume)
  (read-event rewrite))

Neither is documented in the manual.
#11
Whither newLISP? /
October 13, 2009, 05:11:17 AM
That's not what I meant. I meant that it would be nice to have the type of compile-time expansion you built using read-expr but in the context of a non-defining expression. More like letex than define, in other words. That way we could use them as part of  a regular newlisp macro. Like the backtick in cl.
#12
Whither newLISP? /
October 12, 2009, 06:08:54 PM
You know, Lutz, it would be nice if there were an operator that expanded at compile-time, rather than the entire function definition. That way, it could be evaluated inside a macro and provide the full power of both f-expressions and compile-time syntax transformations.
#13
Whither newLISP? /
October 12, 2009, 12:17:26 PM
newlisp's new send/receive functions to communicate between processes are very nice. They let you write coroutines between processes.



Also check out the news forum. I just wrote a module wrapping eventlib, which adds event-based processing for sockets.
#14
Whither newLISP? /
October 12, 2009, 11:00:19 AM
This nice thing about these macros is that you are actually performing a compile-time transformation on the code. You can use them to easily define destructive setters on OOP classes, for example, because they would translate into the actual code that returns a reference. Perhaps something like this, using an assoc list to store your member vars in an instance variable:


Code Select
(macro (slot-value inst slot val)
  (if (nil? val) val (setf (assoc slot inst) val)))
#15
Whither newLISP? / Asynchronous I/O with libevent
October 12, 2009, 07:18:22 AM
Hey folks,



I am in the process of polishing up the interface to a wrapper module around libevent, an asynchronous I/O library for Windows, Linux, Unix, and OSX.



What is asynchronous I/O?



Threading and multiprocessing are expensive ways to take advantage of multi-core systems. They add complexity to any application that wishes to use them safely.



In most applications, the single largest bottleneck is input and output - reading/writing a file, downloading a URL, or talking back and forth with a server over a socket. In many cases, the application can do other things while waiting for a request to be sent or a response to arrive. Newlisp provides access to the common denominator of async I/O with net-select.



Most systems provide faster polling mechanisms, and libevent provides a unified API to many of them. It allows you to write your software as a series of responses to particular events - much like GUI programming with the Newlisp GUI server.



Central to this module is the Reactor context. The Reactor can be though of as a scheduler; it does the work of managing registered events and responses, and running callback functions as needed.



A (relatively) simple example



Take a simple downloader. In synchronous Newlisp:


Code Select
(get-url "http://www.artfulcode.net")

Now, the user has to wait for that command to finish before anything else can happen. Using event-based methods and the Reactor module:


Code Select
; The data buffer
(setf data nil)

; This is called when the socket is read to read and reads into the global buffer, data.
(define (reader socket event reactor)
 (if (zero? (net-peek socket))
  (begin
   (setf data (Reactor:get-event-storage event))
   (Reactor:kill-event event)
   (net-close socket))
  (local (buf)
   (net-receive socket buf (net-peek socket))
   (Reactor:write-event-buffer event buf))))

; Write the request to the socket. Then, remove the write event and add a read event.
(define (writer socket event reactor)
 (net-send socket "GET / HTTP/1.1rn")
 (net-send socket "Host: www.artfulcode.netrn")
 (net-send socket "Connection: closernrn")
 (Reactor:kill-event event)
 (Reactor:make-event reactor socket reader nil nil))

; Open a connection to the server
(setf socket (net-connect "www.artfulcode.net" 80))

; Create a reactor
(setf reactor (Reactor:make-reactor))

; Create an event handler that calls 'writer' when the socket is ready to be written to
(setf event (Reactor:make-event reactor socket nil writer nil))

; Run the reactor
(Reactor:run-reactor reactor)

; The reactor runs until it is told to stop in one of the callbacks, and then this code is run.
(println "Received " (length data) " bytes")

This looks a lot more complicated. However, for complex networking programs managing multiple connections, this technique is much faster, more efficient, and with some experience, just as easy to follow.



It works like this. The reactor is created. The initial event is created for the first open socket. The reactor and relevant event are passed to each callback, allowing the callbacks to kill events, create new events, and stop the reactor (effectively moving past the "run-reactor" function call).



Think about this - what if the writer callback not only scheduled a reader for the same socket, but also added a new writer event for another url in a list. Instead of killing the reactor, the reader would do something with the complete download (like write it to a file or store it in a database) and processing would continue. The urls would be downloaded concurrently, and only the work at the end would cause newlisp to do any real work.



Using this technique, I have written applications that can process hundreds of URLs every minute.



Where to get it



You can see the docs here: http://static.artfulcode.net/events/index.html">//http://static.artfulcode.net/events/index.html. You can download the module here: http://static.artfulcode.net/events/reactor.lsp">//http://static.artfulcode.net/events/reactor.lsp. You can get libevent (the required C library) here: http://www.monkey.org/~provos/libevent/">//http://www.monkey.org/~provos/libevent/



This module is new and has not been extensively tested. Please try it out and see what bugs you can find!
Pages1 2 3 ... 41