newLISP Fan Club

Forum => newLISP in the real world => Topic started by: methodic on August 12, 2005, 12:08:17 PM

Title: Timeout for get-url?
Post by: methodic on August 12, 2005, 12:08:17 PM
What is the default timeout for get-url if it doesn't recieve any data, and is there a way to specify a timeout?



Thanks.
Title:
Post by: Lutz on August 12, 2005, 12:14:57 PM
It depends on the socket stack of your OS. Perhaps some kernel setting?



Lutz
Title:
Post by: methodic on August 12, 2005, 12:30:59 PM
Is there a way to set a time-out, as you would on a regular socket? Or would I have to actually roll my own URL grabbing code?
Title:
Post by: Lutz on August 12, 2005, 12:43:47 PM
In any case you would have to change kernel settings to control this timeout. The system waits in a socket library connect() call. I am not aware of a method to control a timeout of that function call from 'C'. This is different from already connected sockets where you can specify a timeout using select(). So rolling your 'own' get-url won't help you.





Lutz
Title:
Post by: Lutz on August 12, 2005, 12:49:13 PM
On BSD/LINUX you could do a:



sysctl -a


to find out if a timeout settings is available, and then try to change the setting.



Lutz
Title:
Post by: pjot on August 12, 2005, 03:46:54 PM
This is an annoying problem indeed, I suffer from this with my RSS reader as well. Hmm... maybe we can work around it as mentioned previously in another discussion:



(set 'result (exec "ping -c 1 www.newlisp.org "))
(if (> (length result) 0)
  (get-url "www.newlisp.org")
  (println "website not available"))




The '-c 1' option pings only once.



-Peter
Title:
Post by: methodic on August 12, 2005, 03:55:39 PM
That's definitely one way to get around it, although what if the box is panic'd and it still replies to pings, but the kernel itself is hung? I've seen that happen before.
Title:
Post by: pjot on August 12, 2005, 04:13:57 PM
Well, that cannot be solved within newLisp. So you need external tools for this.



Suppose a 'ping' delivers a result, then you must know if port 80 is alive (assuming the webserver is running on this port). The only way I see is using 'nmap' for that:



nmap -p 80 <remote-host>


For example:


Quote
peter@Solarstriker:~/programming/newlisp$ nmap -p 80 www.newlisp.org



Starting nmap 3.50 ( http://www.insecure.org/nmap/ ) at 2005-08-13 01:19 CEST

Interesting ports on ds209-72.ipowerweb.com (66.235.209.72):

PORT   STATE SERVICE

80/tcp open  http



Nmap run completed -- 1 IP address (1 host up) scanned in 0.957 seconds

The result can be parsed in newLisp, same as with the 'ping', so you can find out if the port is really alive.



Even then the remote webserver may be malfunctioning, but there is nothing more you can do - at least, I don't know any other tricks here. :-(





-Peter
Title:
Post by: methodic on August 12, 2005, 04:20:58 PM
nmap was a tool that came to mind, unfortunately i am running this code off an appliance of sorts, where we dont have the room to put nmap on it.
Title:
Post by: pjot on August 12, 2005, 04:24:00 PM
Our network guru Norman has created a portscanner with newLisp:



http://www.nodep.nl/downloads/newlisp/portscan.lsp



I am not sure how well it works, but you can give it a try.



-Peter