Newlisp-Apache-CGI Issues Again

[dukester]

@ralph.ronnquist

You bet that I have an Hex editor!!  I don't see any garbage - which puzzles me. I've included a screen capture of the output. Now I'm wondering what Apache is choking on!! Pain in the Pa-toot!! LOL

[Lutz]

Check how NEWLISPDIR and PROGRAMFILES are defined.



See here for an explanation of these directories: http://www.newlisp.org/downloads/newlisp_manual.html#startup">http://www.newlisp.org/downloads/newlis ... ml#startup">http://www.newlisp.org/downloads/newlisp_manual.html#startup



You could use the fowlowing script to check:

(println (env "PROGRAMFILES"))   ;  only defined on Windows

(println (env "NEWLISPDIR"))

(exit)

[dukester]

Both are set.  I've installed newlisp in c:binnewlisp to make sure that I did not have future issues with spaces in directory names like "Program Files" etc. So  NEWLISPDIR  points to c:binnewlisp.



PROGRAMFILES is set to "Program Files".  I changed it to c:binnewlisp.



The change did not fix my problem. BTW what did you do in 2015 to fix this very same issue?  You modified something in newlisp.exe and asked me to try your modifications. One of them worked!!! :-)

[rickyboy]

duke,



Hex dump seems ok to me. I'm assuming you did something like:

Code Select Expand

C:> newlisp-cgi-script > newlisp-cgi.out
Then, looked at newlisp-cgi.out in the hex editor.  Maybe try also something like:

Code Select Expand

C:> perl-cgi-script > perl-cgi.out
and maybe the same with the Ruby, CLISP scripts, etc. (i.e., all the CGI scripts that Apache doesn't choke on), and see if you can spot any difference between each output from those and newlisp-cgi.out.

[Lutz]

Try the following:



Create a subdirectory "newlisp" in "Program Files" and put the newlisp.exe executable in there. If there are any modules you are going to use, create a "modules" directory inside the "newlisp" directory.

Code Select Expand

(env "PROGRAMFILES") => C:Program Files (x86)
(env "NEWLISPDIR") => C:Program Files (x86)newlisp

Path of newlisp.exe is now: C:Program Files (x86)newlispnewlisp.exe



So both environment variables point to directories and the path in NEWLISPDIR must be in the executable path.



May be Windows finds the newlisp executable when using the command line, but not when called from Apache.



The original fix for Apache in 2015 was putting defaults (the above) for both environment variables.

[dukester]

@Lutz Thanks for all the trouble you went through!!  Unfortunately, after doing all your recommendations, nothing changed!  No Cigars!!  I still get the same response from Apache:

[Fri Mar 29 09:35:53.040187 2019] [http:error] [pid 10048:tid 1076] [client 127.0.0.1:51447] AH02429: Response header name 'Environment variable TMP not set, assuming /tmp .Content-Type' contains invalid characters, aborting request

I keep wondering if it is not an Apache issue, but that question quickly fades away because I never get that response from any other interpreter!  So it must be something that "print" or "println" is emitting?  Maybe UTF8?

[rickyboy]

I keep wondering if it is not an Apache issue, but that question quickly fades away because I never get that response from any other interpreter!  So it must be something that "print" or "println" is emitting?  Maybe UTF8?

That's not what your hex dump indicates.



It may be due to a configuration/interface issue — I believe that's the trail Lutz was going down and a good one to pursue.



It would be nice to see what text stream Apache is seeing. It may be something very unlike what we are imagining in our heads. Any way to do that? IOW, does Apache have a better way to diagnose these problems (like a debug level) because that exception message that you get tells us hardly anything at all and it may even be "lying" to us.



Also, if you don't know already, you may want to research into how Apache is calling out to / running your script. There may be some devil in the details there. If they didn't doc it well, you might have to go code spelunking. Ugh.

[dukester]

@rickyboy  You know - I'm thinking that Lutz has already wasted enough time looking into this issue. If there were hundreds of folks using newlisp on Apache, it might then be a worthwhile investment of time.  I'm sure that the folks at Apache would pretty much feel the same way!  How many people are they aware of that use Apache to run newlisp scripts.  I'm sure they couldn't be bothered.



How's the song go? "You got to know when to hold 'em, know when to fold 'em, Know when to walk away and know when to run. "  Maybe that time is come.  I'll pick at it now and then, but let's not waste anymore time.  I thought the solution might be simple, and already solved. But there I went thinking again!  LOL   Thanks anyway!  Everybody ....

[rickyboy]

duke, I wasn't saying for Lutz to look at it, but for *you* to look at it.  lol!



No worries, I get it.  I've been there many times.  We do indeed have to make that determination: whether to spelunk or not.  :) Sometimes, it's not worth it, especially when we have time constraints.  I hope you get some time here and there to mess around with it ... and succeed!



All the best! rickyboy

[dukester]

Yeah!  I realized that you meant me!   I was just trying to be civil and say "No $%%^^* way!!"  LOL  And Lutz does have better things to do! Thanks - I will fiddle with it as and when the mood hits me.  Good luck yourself!

[rrq]

Hey!

Response header name 'Environment variable TMP not set, assuming /tmp .Content-Type' contains invalid characters, aborting request

That error message shows that newlisp has an initial "complaint" about "TMP", which means what it says, and that complaint precedes the script output. It has nothing to do with your script.



So the quest should rather be to get rid of that piece of output. That amounts to making sure that TMP is set before newlisp is invoked, or, you get it compiled without that complaint.

[dukester]

I've got TMP/tmp/temp set in so many places that all hell wont have it. BTW I never get this bleeding tmp issue with any other script.

[rrq]

RIght. And yet it's missing for newlisp.



Apparently Apache has "SetEnv" and "PassEnv" directives that possibly would service your use case. Or else, I suppose you can wrap the script with a shell script for this purpose. Or if you like living on the edge, you could apply a binary editor to your newlisp executable and put a 00 at the beginning of the boring output string ;)

[rickyboy]

Good catch, Ralph. I read that message wrong (but yet there is the response header name, right there between the two single quotes).  Didn't help that duke's OP didn't have the entire message there to begin with. Kinda faked me out. dukester! lol ;)

[dukester]

Problem solved!



I configured Apache to run as a reverse proxy on a sub-directory to my cgi-bin containing newlisp scripts. I run newlisp as -http -d 8080 -w <fullpathtocgi-binnewLISP>



In Apache httpd.conf, simply load:

mod_proxy.so

mod_proxy_http.so

mod_proxy_http2.so



and insert the following directive in the same file:

Code Select Expand


<Location "/cgi-bin/newLISP/">
      ProxyPass  "http://127.0.0.1:8080/"
</Location>

Cigars and joy all around!!!  :-)