(save) is SLOW

[kanen]

Using (save) is very, very slow for some reason. (load) on the other hand, is quite fast.



I understand (save) is doing slightly more work in the following example, but the speed difference is staggering. And, for large lists... the speed difference makes me want to use (write-file) and do conversions, because (save) is unbearable.



Take the following example:

Code Select Expand

(println (date) "-Creating sequence with 1000000 entries")
(set 'seqx (sequence 1 1000000))

(println (date) "-Pushing sequence to list")
(dolist (x seqx)
   (push x foo -1)
)

(println (date) "-Saving list as foo.lst")
(save "foo.lst" 'foo)

(println (date) "-Writing the list as foo.txt")
(write-file "foo.txt" (string foo) )

(println (date) "-Finished")
When run, it produces this output:

Sun Apr 25 15:07:15 2010-Creating sequence with 1000000 entries

Sun Apr 25 15:07:15 2010-Pushing sequence to list

Sun Apr 25 15:07:16 2010-Saving list as foo.lst

Sun Apr 25 15:07:43 2010-Writing the list as foo.txt

Sun Apr 25 15:07:46 2010-Finished

• Creating the sequence - 1 seconds
  
  Pushing the sequence to a list - 1 seconds
  
  Saving the list (save) - 27 seconds
  
  Writing the list (write-file) - 3 seconds

And, this gets exponentially longer the greater the size and depth of the list I am saving.

[Lutz]

The function 'string' writes to memory, while 'save' opens a file then does file I/O piece by piece.



I have now changed 'save' to convert everything to memory first, then write to a file in one swoop. This makes 'save' now actually slightly faster than (write <file> (string <symbol>))   :-)