XML parsing problems

[hilti]

Hi!



I'm trying to implement a simple RSS and ATOM reading function, but I got some problems in parsing this kind of elements



Example from NYTimes.com RSS feed

Code Select Expand

<dc:creator>By ROBERT D. McFADDEN</dc:creator>

This one doesn't work

Code Select Expand


; TODO: Problems in parsing media:content - ask the forum
;(lookup '(media:content @ url) item) "<br/>"


You can check out my current efforts on this page: http://www.rundragonfly.com/dragonfly_feeds">//http://www.rundragonfly.com/dragonfly_feeds



Cheers!

Hilti

[Lutz]

the xml you posted is parsed like this:

Code Select Expand

> (set 'xml "<dc:creator>By ROBERT D. McFADDEN</dc:creator>")
"<dc:creator>By ROBERT D. McFADDEN</dc:creator>"

> (xml-type-tags nil nil nil nil)
(nil nil nil nil)

> (set 'sxml (xml-parse xml 31))
((dc:creator "By ROBERT D. McFADDEN"))

> (lookup (sym "dc:creator") sxml)   ;  <- the correct way to do it
"By ROBERT D. McFADDEN"
>


you cannot say:

Code Select Expand

(lookup 'dc:creator sxml) => nil ; <- wrong way


In the 'lookip' statement It takes 'dc' as a namespace qualifier because of the colon. Although 'xml-parse' created a symbol "dc:creator" with an illegal colon in it. Using (sym "dc:creator") you can still create that symbol and do the lookup.



you also could this:

Code Select Expand

> (set 'creator (sym "dc:creator"))
dc:creator

> (lookup creator sxml)
"By ROBERT D. McFADDEN"
>


[hilti]

Thanks Lutz! Parsing for elements like

Code Select Expand

<dc:creator>By ALISSA J. RUBIN</dc:creator>

is working great. See it here: http://www.rundragonfly.com/dragonfly_feeds">//http://www.rundragonfly.com/dragonfly_feeds



But one thing I can't get working - the media elements, e.g. NYTimes.com RSS feed

Code Select Expand

<media:content url="http://graphics8.nytimes.com/images/2009/11/08/world/08basra_CA0/thumbStandard.jpg" medium="image" height="75" width="75"/>


I don't know how to access the "url" and "medium" elements.



Thanks for help!

Hilti (slowly getting a RSS parsing expert ;-)

[cormullion]

If you can get as far as this:

Code Select Expand

(set 'mc '(media:content
    (@
      (url "http://graphics8.nytimes.com/images/2009/11/08/world/08peru_CA0/thumbStandard.jpg")  
      (medium "image")  
      (height "75")  
      (width "75"))))

then

Code Select Expand

(lookup 'url (rest (first (rest mc))))

will get



http://graphics8.nytimes.com/images/2009/11/08/world/08peru_CA0/thumbStandard.jpg">http://graphics8.nytimes.com/images/200 ... andard.jpg">http://graphics8.nytimes.com/images/2009/11/08/world/08peru_CA0/thumbStandard.jpg

[Lutz]

Another method uses the 'ref' function:

Code Select Expand

(set 'mc '(media:content
    (@
      (url "http://graphics8.nytimes.com/images/.../thumbStandard.jpg")
      (medium "image")
      (height "75")
      (width "75"))))

(last (mc (ref '(url *) mc match)))

;=> "http://graphics8.nytimes.com/.../thumbStandard.jpg"


The advantage here is, that 'ref' reaches into all nesting levels of a list. The above solution wouldn't need to change if the structure of the XML changes. As long as there is an (url *) elelement in it, 'ref' will find it.



'ref-all' is a version of 'ref' which returns all index vectors found, not only the first one:

Code Select Expand

(set 'mc '(( x y z (url "A.com")) (q ( z (url "B.com"))) (url "C.com")))

(set 'vectors (ref-all '(url *) mc match)) ;=> ((0 3) (1 1 1) (2))

(map last (map 'mc vectors)) ;=> ("A.com" "B.com" "C.com")


What few people know is, that indexing a list also can be mapped by mapping the data structure like a function on to the list of indexes. In this case 'mc' is an complex nested data structure with all urls on a different level, but all are found. Note that 'mc' must be quoted in the mapping expression.

[itistoday]

Code Select Expand

(map last (map 'mc vectors)) ;=> ("A.com" "B.com" "C.com")
What few people know is, that indexing a list also can be mapped by mapping the data structure like a function on to the list of indexes. In this case 'mc' is an complex nested data structure with all urls on a different level, but all are found. Note that 'mc' must be quoted in the mapping expression.

That is nifty! :-)