Text only | Text with Images

newLISP Fan Club

Forum => newLISP and the O.S. => Topic started by: zxcel on December 11, 2009, 04:23:13 PM

Title: get-url with redirection?
Post by: zxcel on December 11, 2009, 04:23:13 PM
I want to download the file, but get-url can't do it. That's because the url that i have, redirects me to another url. All browsers, Opera,  wget - all of them understand that redirection. How can I do it?
Title: Re: get-url with redirection?
Post by: hilti on December 12, 2009, 01:10:35 AM
Try to use CURL instead


Code Select
(exec (string "curl -u "http://www.google.com" --progress-bar"))

CURL handles HTTPS, too.



Cheers

Hilti
Title: Re: get-url with redirection?
Post by: hilti on December 12, 2009, 01:12:04 AM
Sorry.



That's the correct snippet:


Code Select
(exec (string "curl 'http://www.google.com' --progress-bar"))
Title: Re: get-url with redirection?
Post by: Fritz on March 07, 2010, 11:00:58 AM
You can also look into the text "get-url" returns to you. I believe, inside you will find a correct address of the file.



First get-url:
Code Select
(get-url "http://lenta.ru/info")

Result:
Code Select
"<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">n<html><head>n<title>302 Found</title>n</head><body>n<h1>Found</h1>n<p>The document has moved <a href="http://lenta.ru/info/">here</a>.</p>n<hr>n<address>Apache Server at lenta.ru Port 80</address>n</body></html>n"

Parsing correct address:
Code Select
(first (regex {(?<=href=")[^"]+} (get-url "http://lenta.ru/info")))

Result:
Code Select
"http://lenta.ru/info/"

Final get-url:
Code Select
(get-url (first (regex {(?<=href=")[^"]+} (get-url "http://lenta.ru/info"))))
Title: Re: get-url with redirection?
Post by: Lutz on March 08, 2010, 07:08:30 AM
In the past 'get-url' only handled re-  "Location:" headers together with error codes 301 and 303. I have now added error code 302 for v.10.2.0 to force a second request also on this error code, and it handles the http://lenta.ru/info case correctly.
Text only | Text with Images