How do I retain a list in memory in Dragonfly between pages?

[Jeremy Reimer]

Hi folks,



Say I have a large list (say, 40,000 records) that takes about five seconds to load with the load("40klist.lsp") command.  I'd like to keep this list in memory when the user navigates to another web page on the same site, so that each page that needs to reference the list won't have the same five second delay.  Is this at all possible?  



Thanks!

[cormullion]

I can see that as a problem - if you quickly step through all pages of a site there'll be a lot of unnecessary loading... You could try loading the list after the page (or most of it) has been displayed - typically people will be reading the text on the page and not notice that some loading is going on behind the scenes...

[Jeremy Reimer]

Well, that isn't really possible in this case, as the text the user would most want to see comes from the list itself.  Also, without using AJAX it isn't possible to delay loading of any part of the page until the list has loaded.



Doesn't Dragonfly itself have to keep all of its context in memory when it runs?

[cormullion]

I think Dragonfly is basically a CGI that runs and exits each time a page is required. On my site, I load up the database each time a page is requested, but it isn't as big as yours... :)

[itistoday]

Normally large amounts of data is stored in some sort of a database, not loaded all at once. Dragonfly has excellent support for sqlite3 (see the docs for DF.DB). You can also use something like memcache.

[Jeremy Reimer]

Just a quick revisit to this topic:



I put the giant list in a SQLite database for the project I was working on, and it works great.  SQLite is just as awesome as everyone says it is, and it works really really well with newLISP.  Also, it has a feature where it can store the database in-memory if really fast response times are required.