Request and Response modules for newLISP

[Jeff]

As promised, I've finished the request and response modules for cgi programming.  They are fairly comprehensive (for their purposes), but do not contain the equivalent of "put-page" from the newLISP cgi module and are not compatible with the newLISP cgi module.



One of my next projects is a template module that will replace that functionality.



Here are the links:



Article: http://www.artfulcode.net/articles/request-and-response-modules-newlisp/">http://www.artfulcode.net/articles/requ ... s-newlisp/">http://www.artfulcode.net/articles/request-and-response-modules-newlisp/

Files: http://www.artfulcode.net/projects/newlisp-request-and-response-module/">http://www.artfulcode.net/projects/newl ... se-module/">http://www.artfulcode.net/projects/newlisp-request-and-response-module/

[cormullion]

Cool! I'm going to try this out in the next few days. You may hear from me again... :)

[cormullion]

Hi again. Tell me, if I replace (CGI:get {string}) with (Request:get {string}), should that work?



I tried it and half of the time it worked, half of the time it didn't. More precisely, half of the methods always worked, and the others never did. I wonder whether this is because I've misused the Get and Post methods in my own script and the problem is only just appearing now.

[Jeff]

It would depend on your code.  (Request:get "foo") should work, because that data is always available through (env "QUERY_STRING").  Post, however, should only work if no other script has read the input buffer first.  So, if the official newLISP cgi module is loaded first, it will have the post data, making it inaccessible to the request module.



Also, I have set an arbitrary limit on the size of post data (for safety reasons).  You can alter it at the top of the module.



What does your code look like?  Is it running from apache on nfshost?

[cormullion]

I swapped out cgi.lsp and changed the max value first, so I don't think it's those . But I was running it locally using newlisp's webserver mode - newlisp -httpd - so perhaps that makes a difference.



A quick way of  seeing the code is http://code.google.com/p/lambdapress/source/browse/trunk/index.cgi">//http://code.google.com/p/lambdapress/source/browse/trunk/index.cgi but please remember that I'm not a programmer ... :)

[Jeff]

I don't know.  I have not messed with the built-in http server.  I tested this on apache running newlisp as a cgi.



That code is using the cgi.lsp module.  I'm assuming that is pre-swapping.  You can see all get and post params entered by printing (Request:get) and (Request:post) (without any arguments).  Without a key, they output the entire assoc list.

[cormullion]

Thanks Jeff - I'll try that. Obviously the code on googlecode isn't the same as the version I tested with Request instead of CGI - I like to try stuff out locally out before posting... Trouble is, trying things locally with newlisp server is much easier than setting up Apache (although that's easy enough too).

[Jeff]

Corm,



I just tested it out.  I've updated the template module to include a test page using the request and response modules.  Here is the link:



http://www.artfulcode.net/media/releases/2008/02/26/template-0.2.tar.bz2">http://www.artfulcode.net/media/release ... .2.tar.bz2">http://www.artfulcode.net/media/releases/2008/02/26/template-0.2.tar.bz2



It has a simple form that sends both post and get data to the server, and I tested it against 9.32 in -httpd mode.

[itistoday]

Sorry, what was the reason for this?  Advantages/disadvantages between this and newLISP's CGI module?

[cormullion]

I think there's something here: http://www.alh.net/newlisp/phpbb/viewtopic.php?t=2131">//http://www.alh.net/newlisp/phpbb/viewtopic.php?t=2131

[itistoday]

I think there's something here: http://www.alh.net/newlisp/phpbb/viewtopic.php?t=2131">//http://www.alh.net/newlisp/phpbb/viewtopic.php?t=2131

Scanned over that... still don't really get it... (I don't do much web development), is it that there's something bad about CGI's GET/POST retrieval/differentiation...?

[Jeff]

CGI is dangerous enough without knowing where a parameter got set.  With the default cgi module, there is no way to distinguish whether a parameter was set via post or get, nor is there a way to determine, once loaded, if there was any post data at all.  A common idiom is to check if post data exists.  If it does, continue with validation.  If not, return a 404 error code.  This protects against automatic attacks.



Additionally, the default module does not do any sanity checking on the size of the data coming in from post, which could potentially hang the application completely.



My modules also provide a means to return various types of HTTP headers without having to print them yourself (i.e. 404 not found, 500 error, etc.)

[Lutz]

When using the Apache web-server the request method and content length can be retrieved by checking the environment variables set by Apache. The check can be done before loading cgi.lsp:

Code Select Expand


(env "REQUEST_METHOD") => the string "GET" or "POST"

(int (env "CONTENT_LENGTH")) => the number of bytes in the post request


In both cases 'env' returns a string, so 'int' should be used to convert the content length. CONTENT_LENGTH is only present when when the REQUEST_METHOD is POST. In case of the GET method the environment variable QUERY_STRING can be inspected.

[Jeff]

Yes, but how can you verify that parameter "foo" came from POST data?