Text only | Text with Images

newLISP Fan Club

Forum => So, what can you actually DO with newLISP? => Topic started by: newdep on November 24, 2005, 04:11:16 AM

Title: Compression
Post by: newdep on November 24, 2005, 04:11:16 AM
Hi Lutz,



Its on my wishlist for some time now but how do you think about a small and

simple compression/decompression implementation in newlisp? (like a small lha implementation) someehing like: (compress str) and (decompress str).



Regards, Norman.
Title:
Post by: HPW on November 24, 2005, 04:33:56 AM
There was a thread about importing gzip:



http://www.alh.net/newlisp/phpbb/viewtopic.php?t=664&highlight=zlib



And for windows (hpwNLUtility):



http://www.alh.net/newlisp/phpbb/viewtopic.php?t=149&highlight=zlib
Title:
Post by: newdep on November 24, 2005, 05:12:11 AM
Your right about the thread ;-) Though I never came to the Zlib because

it simple too big for a simple compression. I also had a look at miniLha lib

but that one still is 16 Kb..

(I dont care what compression as long as it compresses :-)



How about a simple LZW imlant?



Though Im hoping of a C implemented fast simple compression routine

inside newlisp. the CRC was also done by Lutz and its small and fast.



Perhpas Lutz can do some X-mass features ;-)



Regards, Norman
Title:
Post by: pjot on November 24, 2005, 11:59:07 AM
Do you want compression for text files? Or also binary files?



If only textfiles must be compressed, you could write a compression function yourself in newLisp. This should be fast enough, even for large ASCII files.



Peter
Title:
Post by: newdep on November 24, 2005, 12:38:33 PM
compression is compression so that includes it all ;-)

No bitstuffing or mapping on ascii layers, but a real compression algorithm..



But I wont spoil the fun, if you have a newlisp version for ascii drop it in ;-)



Regards, Norman.
Title:
Post by: newdep on November 28, 2005, 10:18:34 AM
So its finaly Done, we have compression, done by Pjot!



You can download it from his site at -> http://www.turtle.dds.nl/newlisp/lzw.lsp



PS: its not the worldrecord on speed, thought it is compession! ;-)
Title:
Post by: pjot on November 28, 2005, 03:13:50 PM
Speed is a problem, that's true. But there may be some improvements on the code which also may improve speed.



During this week I'll finalize the program. And let you know (of course).



Peter



(PS you can also buy a faster computer.... ;-) )
Title:
Post by: pjot on November 30, 2005, 01:17:59 PM
The LZW compression context has been optimized and is ready for consumption. Compared to the first implementation, the compression speed has improved with 25%, while the decompression speed has improved dramatically with about 80%.



LZW is a technique which normally is used to compress GIF and TIFF files. It is very suitable to compress text. Files up to 500k compress reasonably fast with this context.



http://www.turtle.dds.nl/newlisp/lzw.lsp



If somebody has suggestions to increase the compression speed even more, please let me know. :-)



Enjoy,



Peter
Title:
Post by: pjot on March 13, 2006, 04:02:32 AM
The LZW compression has been optimized even more. The average compression speed compared to the former situation is 5x faster.



On my P4 2Ghz an e-text of 500kB can be compressed in 4 seconds now. If your machine is faster then it will take even less time.



The latest version is here:



http://www.turtle.dds.nl/newlisp/lzw.lsp





To test, just copy some file into the same directory as 'LZW.lsp', with the name "dump.txt", and run 'LZW.lsp'. The compression also works with binary files.



Regards

Peter
Title:
Post by: newdep on March 13, 2006, 04:54:35 AM
On an AMD 64 3200+



filesize: 4614809  (pure text, dictionary file)

Starting compression, please wait... Time spent (msecs): 17020

Starting decompression, please wait... Time spent (msecs): 30283



1/4 of 4614809 (1153703 bytes or the same file) ->

Starting compression, please wait... Time spent (msecs): 4282

Starting decompression, please wait... Time spent (msecs): 7589



They both come pretty close to a 4:1 ...



Norman.
Title:
Post by: pjot on March 17, 2006, 02:13:05 PM
Self-extracting possibilities were added.



http://www.turtle.dds.nl/newlisp/lzw.lsp



Just call


Quote
(LZW:lzx "myfile.ext")

 


and it will create a self-extracting ".lzx" file. If this ".lzx" file is executed by newlisp, it will be decompressed. In Unix/Linux it is also possible to run it directly, like "./myfile.ext.lzx". In Windows you need to execute "newlisp myfile.ext.lzx" to decompress.



Peter
Title:
Post by: tom on March 17, 2006, 03:21:09 PM
this needs to go in a backup/restore script.  I've been using back2cd.tcsh, a script that a guy in my lug wrote, but I think that newlisp could do at least as well, especially with lzw.lsp around...
Title:
Post by: tom on March 19, 2006, 03:16:49 AM
hey peter, stupid question:



how do you give it the file to compress?  



$./lzw.lsp



gives your error,



ERROR: File not found! Exiting...



I did say up front this was a stupid question :-)



thanks.
Title:
Post by: pjot on March 19, 2006, 04:24:33 AM
Hi Tom,



In the MAIN context of "LZW.lsp" the demonstration is looking for a file called "dump.txt". Just create a file with this name in the same directory and run. Currently the LZW creates a self extracting . lzx file, for which you need newLisp to decompress.



Peter
Text only | Text with Images