newLisp Document Search Site

[Joe]

Hi everyone,



I put together a site that allows you to search all of the newLisp documentation, including tutorials, related sites, etc.



http://newlispsearch.nfshost.com/">http://newlispsearch.nfshost.com/



Please let me know what you think. Right now it runs with PHP, but hopefully a future version will be newLisp (patience please, I'm still working though "Hello world").



Please note the slow speed is the fault of the server and not the script.



Enjoy, and please provide feedback.



Joe

[ale870]

Good starting point!

But I think it is too "similar" to a standard google search.

I think you could supply some specific features that I cannot get using a standard search engine.



For example: I think one big missing thing in newLisp shell is an online help. So, why don't you realize a newlisp module to check your site via shell? For example, I could type, in the console, something like:



(help:find "println")



And that module (I called it "help") could send a search query to your engine, showing the results directly in my console!

[cormullion]

I'm impressed - it's pretty cool. I will definitely use it - the sluggishness is occasionally noticeable. That's presumably nfshost? (Or php? :-) ) Or are you loading tons of stuff as well?



I always have comments and suggestions... :)



In the results page, you have headers in the form

"apply.html (1)" or "sys-sym-const.html "... An improvement would be to tell the user what the matching source documents were, rather than the precise name of an html page which (presumably) they need to know nothing about. It could be done as a hover/mouse text, perhaps.



Will you be able to easily update the content when Lutz updates things (which he does often...)?



The links to the Introduction are less than ideal, I know. Ideally the Introduction would be in lots of small sections (for indexing) yet easily combined to make a continuous whole (for printing/downloading) - and I haven't the patience to find out how to do it right. Anyway, the docuwiki's days might be numbered ...



In the CSS: "Sans-Serrif" can't be right, can it?



And it's definitely "newLISP"... :)



Great work - can't wait to see what else you get up to!

[Joe]

... the sluggishness is occasionally noticeable. That's presumably nfshost? (Or php? :-) ) Or are you loading tons of stuff as well?

There is some pretty substantial processing in the PHP code, but when I had it on my test server (also remote), it was very fast, so I think nfhost plays a big role. On the other hand, you can't beat the price.

Will you be able to easily update the content when Lutz updates things (which he does often...)?

No, and that is the real downside. Maybe in 2.0. The problem, at least from my limited understanding is getting the user to specific parts of a long document. Even google doesn't help you there.

The links to the Introduction are less than ideal...

I agree. On the other hand, that is most dynamic content because it always links to the live document. It was just the best way to get it done.

Great work - can't wait to see what else you get up to!

How about a newLISP code repository (think CPAN)?



BTW, this is a quick & dirty (somewhat experimental) version just to test the waters. I know there is a lot more that can be done, and appreciate all suggestions.



Joe

[Joe]

For example: I think one big missing thing in newLisp shell is an online help.

Let me see how this goes. I do appreciate the suggestion, but that is probably a little advanced for my limited newLISP experience. But I will put it on the ToDo list.



Joe

[ale870]

Ok!

About "official documentation" or sites created by us (newLisp fan and supporters!) we can insert, in the html code, some special tags to get: keywords, functions usage, description, "what's new" (documentation updates), etc...

I think in this way we could simplify your parsing process. These tags could be inserted in html comments.

And I think we could make a central code repository and modules installer like "Ruby gems" (I used it, it's great! It's similar to Ubuntu package manager).

I think we only need to define a standard approved by Lutz, then we could work, all together, to implement it.



Joe, if you use php, why don't you look for a faster, free hosting site? There are many providers free and fast.

I think nfshosting is good for newLisp cgi, but if you work in php you are a "world" of opportunities.

[cormullion]

What happened to the Code Repository post? I was going to ask some questions (about approvals, review, updates, longevity, etc) but the post has gone today...

[Joe]

What happened to the Code Repository post? I was going to ask some questions (about approvals, review, updates, longevity, etc) but the post has gone today...

I injured my wrist yesterday and won't be able to work on it for awhile. Since there had been no replies, just pulled the post and the site...for now.



Besides, I had completely overlooked (ADD kicked in again) that newlisp.org already has a code collections page.



We can still discuss it if you like, perhaps open a thread to determine if it is really needed?



Off to find a wrist brace that still lets me use a mouse & keyboard.



Joe

[Lutz]

Actually parsing is currently pretty simple. Here is the function I am  using in nls (in newlisp-x.x.x/util/nls):

Code Select Expand

(define (help func-name)
  (if (find func-name "|+*-") (push "\" func-name))
  (set 'html-text (join (find-all (format {(syntax: (%s.*?))} func-name)
    (read-file "/usr/share/doc/newlisp/newlisp_manual.html")) "n"))
  (replace "<.*?>" html-text "" 0)
  (replace "<" html-text "<")
  (replace ">" html-text ">")
  (replace "&" html-text "&")
  (println html-text)
  "")


This displays only the syntax line(s). E.g. when running the nls shell on Unix you see this:

Code Select Expand

MAIN:/Users/lutz> help pri
syntax: (primitive? exp)
syntax: (print exp-1 [exp-2 ... ])
syntax: (println exp-1 [exp-2 ... ])
MAIN:/Users/lutz>


There is another function help in newlisp.x.x.x/examples in the file init.lsp.example:

Code Select Expand

(define-macro (help func)
   (if (primitive? (eval func))
       (let (func-name (name func))
            (if (ends-with func-name "?") (replace "?" func-name "p"))
       (!  (format "lynx /usr/share/doc/newlisp/newlisp_manual.html#%s" func-name)))
   (format "%s is not a built-in function" (name func))))


This display a whole page, but you need the lynx text web browser installed.



Basically paresing from one {<h4>(syntax: (%s.*?)</h4>} to the next {<h4>(syntax: (%s.*?)</h4>} will cut you out the reference for one function.

[cormullion]

I injured my wrist yesterday and won't be able to work on it for awhile.

No problem - take the time to get better! Some things are more important than coding... :)

[Joe]

Major Update



Filled with newLISPy goodness and new sites added.



http://newlispsearch.nfshost.com/">http://newlispsearch.nfshost.com/



Go ahead, show me some love.



Joe

[HPW]

Schouldn't 'newlisp.og' not read 'newlisp.org' in the site-list?

[Joe]

Schouldn't 'newlisp.og' not read 'newlisp.org' in the site-list?

I would certainly think so.



I will find the person responsible for the error and take appropriate action.



Thanks!

[cormullion]

It looks good! I'll use it in the next few weeks and report back with my experiences...

[Joe]

It looks good! I'll use it in the next few weeks and report back with my experiences...

Thanks. BTW, this version is fully dynamic with the search and results coming directly from specific sites.



Another feature to note, is that all results are categorized by site.



I hope it works well for everyone.