HTTP POST processing

[Kirill]

Hi there,



http://www.newlisp.org/code/modules/cgi.lsp.src.html">cgi.lsp processes POST parameters using read-line:

Code Select Expand


; get stdin POST method parameters if present
;
(set 'inline (read-line))
(if inline
(set 'params (get-vars inline)))
 
(if (not params) (set 'params '()))


Today I tried this when CGI ran under http://www.mathopd.org/">Mathopd and it did not work (or actually I was using http://static.artfulcode.net/newlisp/web.lsp.html">web.lsp, but the issue is similar with cgi.lsp): when some data was sent using POST, the script was simply hanging waiting for something. CGI environment provides the length of POST-ed data en CONTENT_LENGTH. So, a (read) should rather be used to collect POST-ed data. For web.lsp, I have a http://km.krot.org/code/web.lsp.diff">patch. For cgi.lsp, the code might look like this (replacing the code above):

Code Select Expand


; get stdin POST method parameters if present
;
(when (= (env "HTTP_METHOD") "POST")
  (read (device) post-data (integer (env "CONTENT_LENGTH")))
  (set 'params (get-vars post-data)))

(if (not params) (set 'params '()))


I have not actually tried it, as I'm not using cgi.lsp, but it seems correct.



Thanks.



EDIT: After I submitted the post, I went through the nl-web.c and it seems to me that my changes will not work with newLISP's built in web server's CGI handler, as it does not set the CONTENT_LENGTH variable. Actually it only sets a subset of variables, at least compared to what CGI/1.1 spec says. It might be worth mentioning in the docs that CGI environment variables apart from the mentioned HTTP_HOST, HTTP_USER_AGENT and HTTP_COOKIE (and some other) are not being set.

[Lutz]

In version 10.3.0, to be released in February, newLISP HTTP server will also set the CONTENT_LENGTH environment variable when present in the incoming client request.



You can see a preview of 10.3.0 here:



http://www.newlisp.org/downloads/development/inprogress/">http://www.newlisp.org/downloads/develo ... nprogress/">http://www.newlisp.org/downloads/development/inprogress/

[Kirill]

Rather than trusting client's headers, would not it be better to let newLISP calculate contect length itself? It's just I don't want to let the client decide how much data I'll need to read().



How about other CGI environment variables, Lutz? In particular, I'm thinking of HTTP_AUTHORIZATION, which becomes set if the client does HTTP AUTH. Doing so, it should be easy to write password-protected somethings using newLISP's built-in web server. Like this:

Code Select Expand


(define (www-not-auth)
  (println "Status: 401 Authorization required")
  (println "WWW-Authenticate: Basic realm="Protected"")
  (println "Content-type: text/plain")
  (println "")
  (println "Please provide a valid username and password")
  (exit))

(unless (and (env "HTTP_AUTHORIZATION")
             (starts-with (env "HTTP_AUTHORIZATION") "basic " 1))
  (www-not-auth))

(setq login-data (parse (base64-dec ((parse (env "HTTP_AUTHORIZATION")) 1)) ":"))

; verify username available as (login-data 0) and password available as (login-data 1)
; and call (www-not-auth) if authentication fails

(env "REMOTE_USER" (login-data 0))
(env "HTTP_AUTHORIZATION" "")


I (load) this in CGI scripts which deal with several users.

[Kirill]

Lutz - will you also update cgi.lsp accordingly, so it uses CONTENT_LENGTH correctly?

[Lutz]

Yes, but not yet. cgi.lsp will be updated eventually, but I will wait a few versions, so new code will still run on older versions of newLISP server.

[Kirill]

cgi.lsp can check for the presence of CONTENT_LENGTH, no?

[Lutz]

Yes, that is what it's doing now in the preview:



http://www.newlisp.org/downloads/development/inprogress/">http://www.newlisp.org/downloads/develo ... nprogress/">http://www.newlisp.org/downloads/development/inprogress/



ps: final 10.3.0 has been released February 2nd