A new web-based library for newLISP

[Jeff]

I've written a replacement CGI library, which can optionally work side-by-side with the official CGI module, with some caveats (see the docs for details.)



It has extensive support for URL-encoding, entities, and sessions, as well as all of the functionality of the CGI module.



http://www.artfulcode.net/articles/a-better-newlisp-web-library/">http://www.artfulcode.net/articles/a-be ... b-library/">http://www.artfulcode.net/articles/a-better-newlisp-web-library/

[cormullion]

Looks great, Jeff, thanks... I'll be looking to try this out soon.



I was expecting "#!/usr/bin/env newlisp" - is it more portable to use "#!/usr/bin/newlisp" ?



Is it just for newlisp v 10? And all versions of v 10.0.* ?

[Jeff]

I rarely use /usr/bin/env. In fact, it really doesn't need a preamble because it is not a standalone app. I had that in there because I was testing it as a cgi.



As to the nl version, it should probably be 10, since it uses setf internally.

[Jeff]

I also added a tutorial on how to write your own session storage module:



http://www.artfulcode.net/articles/custom-session-storage-with-newlisp-web/">http://www.artfulcode.net/articles/cust ... wlisp-web/">http://www.artfulcode.net/articles/custom-session-storage-with-newlisp-web/

[TedWalther]

I bet this is related to your recent re-release of the JSON module.  You released your module at just the right time, I'm migrating a legacy PHP codebase to newLISP.



I'm wondering, when you parse the POST data, do you check the content type?  There are two or three content types that should be url-decoded, but anything else should be left as is, and treated as a stream of raw data.



Does your module handle this?



If the POST data is an XML stream, for instance, what do I do when using your module?



Ted

[TedWalther]

I also added a tutorial on how to write your own session storage module:



http://www.artfulcode.net/articles/custom-session-storage-with-newlisp-web/">http://www.artfulcode.net/articles/cust ... wlisp-web/">http://www.artfulcode.net/articles/custom-session-storage-with-newlisp-web/

I was just about to post that you were wrong about MySQL being best supported in newLISP, then realized I haven't added in the support for auto-converting query return types (int, float, etc).  Until that is done, I'll hold my peace.  But apart from that, I think the Postgres support in newLISP is pretty damn good. :)

[TedWalther]

Why do you use if-not instead of unless?

[TedWalther]

Regarding the MIME type of the POST data, I dug up some old code of mine to see what is happening.  Here are the results:



If CONTENT_TYPE === application/x-www-form-urlencoded, then we url-decode.



If CONTENT_TYPE === multipart/form-data; then we have to do some mime decoding.



Once the MIME decoding is done, I don't know if we have to url-decode, presumably if one  of the mime parts has a content type of x-www-form-urlencoded, then we decode it.



At all other times, the POST should be nil I guess, or at least read into a variable of some sort.



Ted

[TedWalther]

In the cookies function, there should also be a secure-only parameter after the http-only parameter.  The secure option makes the cookies only be returned by the browser over an SSL connection.

[cormullion]

Why do you use if-not instead of unless?

They're slightly different. unless has no else part. From what I've seen, he's used mostly if-not with else clauses, but only once without one.

[Jeff]

Ted -



Sorry for taking so long to respond. I was in meetings much of yesterday and am running a bit behind.



I will work on adding the secure cookie functionality and MIME parsing of the POST body, but it may be a little while. Alas, they make me write actual code for my paycheck, like I don't have other things to attend to :)



In the meantime, if you have a patch or some ready-made code for parsing a MIME-encoded POST body, I would love to take a look at it.

[Jeff]

In trying to parse the MIME content, I'm having some difficulty reading the entirety of the POST data. Or perhaps outputting the entirety of the binary POST content. The problem appears to be that the binary may contain null values, and that write-buffer and read-buffer both halt when encountering nulls. Any ideas on how to work around this?

[TedWalther]

Hi Jeff, I've been coding for a paycheck too, sorry I haven't had time to help out with this.  I can definitely see with your improvements, I'd like to see your cgi module be part of the base distro.



As for the 0 value bytes in the stream, we'll have to ask Lutz about this one.  I know newlisp strings can have 0 value bytes, and there probably is a read- function that will allow it.



Just reading the description of read-buffer, it looks like a BUG in read-buffer, and it also seems like an unecessary limitation that the arg str-wait can't contain a 0 byte either.



To get it working right now, I suppose (while (setq c (read-char 0)) ...) could be used.

[Lutz]

'read-buffer' and 'write-buffer' both handle 0 bytes and so do almost all other functions handing string buffers. Here is a demo and source for transferring files with HTTP POST:



http://www.newlisp.org/index.cgi?page=File_Upload_Script">http://www.newlisp.org/index.cgi?page=F ... oad_Script">http://www.newlisp.org/index.cgi?page=File_Upload_Script



There are few functions when used on binary content which will stop at zeros, i.e. 'string',  'print' and 'println'.



I have also used HTTP PUT to transfer binary content.



ps: there is a limitation only when using wait-string syntax for the wait-string itself

[TedWalther]

In trying to parse the MIME content, I'm having some difficulty reading the entirety of the POST data. Or perhaps outputting the entirety of the binary POST content. The problem appears to be that the binary may contain null values, and that write-buffer and read-buffer both halt when encountering nulls. Any ideas on how to work around this?

Lutz says read-buffer and write-buffer are working, can you give us a test case and localize it to a line of code?



Ted