purpose of the STREAM api in newlisp?

[TedWalther]

Going through the newLISP source code I notice a very complete (and often used) API that passes around STREAM structures.  These STREAM structures can be a file or a string.



I'm wondering, Lutz, what is the reason for this extensive API?  It doesn't seem to be needed for cross-platform stuff.  It does allow the evaluation and compilation functions to operate on strings in memory as well as files.  But then, couldn't one read the file into memory?



Also, the macro MAX_STRING set to 2048 bytes, looks like perhaps the STREAM api was (or is?) intended for a low memory situation and reading source code in chunks.  Perhaps related to the net-eval code?

[Lutz]

Many low-level I/O functions in Windows are not stream based or stream based functions are very slow. Using the self-made stream API sped up file I/O several times on Windows and made newLISP more portable to small compilers and C-libraries. Using streams for strings allows reading, parsing, evaluating large files using very little memory. From early on newLISP was used in embedded systems and small handhelds.



Limiting the MAX_STRING constant to 2048 is also a run C-stack issue. Often strings are allocated on the run-time stack for speed reasons. Of course string length is not limited to 2048 characters, only for " " quote and { } delimited strings the limit is 2048. Strings limited by [text][/text] tags are unlimited in length.

[rickyboy]

Many low-level I/O functions in Windows are not stream based or stream based functions are very slow.

Ah, Windoze!  What a surprise.



Now, I'd like to take this opportunity to say thank you, Lutz, for keeping a Windoze version of newLISP going, as many of us are unwillingly forced to use this piece of ... er ... this "OS" at work.  Thanks to your efforts, we can still be productive and have fun hacking, even on this hampered platform.