Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - John_Small

Pages1
#1
newLISP in the real world / Adding another 30 minutes
June 26, 2007, 02:19:26 PM
I think I should warm up on newLISP again

especially on contexts and def-new and the

new unify etc.



I'll try to then write about these additional subjects



1. lexically scoped programming in newLISP

codified with examples (and perhaps macros)

to make it as easily as rolling off a log.



2. a closer look at macros (newLISP macros

have real potential for exploitation I think

and needs very carefully study to see exactly

how potent it can be.)



3. explore the unify and see if we can fashion

continuations through CPS (continuation passing style)

perhaps using macros in order to facilitate non deterministic

programming.  This is where we have to have

lexical scoping or it will be hopeless.   If we can do

something useful in the way of non determinism (easily)

then newLISP starts to become compelling.  Otherwise

unify is just a nice way to get around "let" and maybe

cracking and mapping structures (which is worthy discussing

anyway).  If we can get non determinism working than

folding this back onto macros can become extremely

interesting as we build non deterministic JIT macros.



I also want to rewrite the flatreplace.lsp or read

what Rick Hanson sent me as soon as I can get

a free minute which might be 2 weeks from now.

I want to test this memory problem again.  Even

though my code was buggy the memory problem

should not have been a problem since I was not

recursively calling anything.  I suspect the problem

is in Windows memory allocation since a port to

PHP of flatreplace also died several hours into

the batch job.
#2
newLISP in the real world / Thanks Ric
June 25, 2007, 03:17:42 PM
I forgot about dynamic scoping.



I wrote this when late at night when

I was half asleep.



Thanks!
#3
newLISP in the real world / I'm not sure where the extra bytes are going
June 25, 2007, 02:12:36 PM
I'm not sure I understand what you are

asking.



Unless I'm unwittingly recursively calling

something and growing the stack what else

could be causing this memory leak?  Is

it my code or the newLISP garbage collector?



TBTW - hanks for taking the time to run it.
#4
newLISP in the real world / I'll send the code to whomever would like to test/use it.
June 25, 2007, 11:22:36 AM
Do you have an email that I can send you the

code?



It is basically a generic filter that anyone can use.

The configuration file customizes it for handling

your particular flat files.



You simply specify the match column regular expressions

and the replace column RE's and replacement text.



So this could be a useful utility if I can eliminate the out

of memory crash.
#5
newLISP in the real world / Out of memory message
June 25, 2007, 10:33:43 AM
I'm using newLISP for filtering large number of

huge ascii text files (one line equals one record).



The filter is making simple changes to the input

text line (record) and printing it to a tmp file

and then when finished replacing the original

file (after it was closed of course).



After running for an hour or so it finally

crashes with a message to the effect it is

out of memory in function such and such.



I'm not using any recursive algorithm so

I don't think it is the stack running out of

space.  I am running a huge number of

huge files through the filter (using "for"

loops to read the archive directory files).

The filter is making changes to these

archive files and this is why it can run

for hours.



I didn't save the exact error message but

I can start the job again and get the message

verbatim if you need it.
#6
newLISP newS / slight correction
June 08, 2007, 05:40:38 PM
I delete the output file if I fail to write anything.



(define (file-map fn-line-map filename out-filename out-temp)

  (setq out-file (open out-filename "write"))

  (if out-file

    (begin

      (setq in-file (open filename "read"))

      (setq out-written nil)

      (if in-file

        (begin

          (while (read-line in-file)

            (write-line (fn-line-map (current-line)) out-file))

          (setq out-written true)

          (close in-file)

        )

      )

      (close out-file)

      (if out-written

        (if out-temp

          (begin

            (delete-file filename)

            (rename-file out-filename filename)

            filename

          )

          out-filename

        )

        (begin

          (delete-file out-filename) ;; since nothing was written

          nil

        )

      ) 

    )

    nil

  )

)
#7
newLISP newS / implementation of file-map
June 08, 2007, 05:34:07 PM
This was done in a hurry and briefly tested.



It allows you to map the lines from the input to

the output file.  The optional 4th argument

indicates the out-file is temporary and that the output

file replaces the input file when finished.  If the function

succeeds it returns the final file name otherwise it returns

nil.  I didn't look to see if newLISP has a temp file

name generator so the function prototype could

be changed to something more appropriate.

I'm using newLISP to write file stream combinators

and filters so I'll be using this all the time.  Seems

like it would be easy to include something similar

in functionality with the standard library.  The reason

I didn't use read-file and write-file is because

the files I'm filtering are gigantic and can't fit into

memory. 







(define (file-map fn-line-map filename out-filename out-temp)

  (setq out-file (open out-filename "write"))

  (if out-file

    (begin

      (setq in-file (open filename "read"))

      (setq out-written nil)

      (if in-file

        (begin

          (while (read-line in-file)

            (write-line (fn-line-map (current-line)) out-file))

          (setq out-written true)

          (close in-file)

        )

      )

      (close out-file)

      (if out-written

        (if out-temp

          (begin

            (delete-file filename)

            (rename-file out-filename filename)

            filename

          )

          out-filename

        )

        nil

      ) 

    )

    nil

  )

)
#8
newLISP newS / exists is fine
June 07, 2007, 05:13:35 PM
my eye sight isn't the greatest any more.



I didn't see the hyperlink to exists in the manual

at the bottom of the example.
#9
newLISP newS / You're right exists does what I was asking for in for-any
June 07, 2007, 04:50:11 PM
I didn't see that before.
#10
newLISP newS / read-file and write-file
June 07, 2007, 04:38:29 PM
Suppose read-file allowed an optional 2nd argument



   (read-file filename fnc)



This fnc is called once for each line in the file.

But instead of returning the entire file in a str-buffer

it returns a continuation function that can be repeatedly

called until returning EOF.  If this is too confusing

suppose it were called read-file-line or something.

The idea here is to dispense with having to test for

EOF and closing the file which is all hidden then just

like it is with read-file.  The difference it larger

files don't have to be read entirely into memory

all at once.



Likewise if  write-file could differentiate between

a str-buffer and an continuation argument then

it could repeatedly call the continuation (i.e. a function

taking no arguments).



Cutting to the chase instead of changing read-file and write-file

suppose instead (or in addition) that copy-file took

an optional 3rd argument that was a current-line mapping

function.  If copy-file returned the name of the destination

file (or nil) we could write something like



    (rename-file src (copy-file src dst map-fnc))



or better yet



   (map-file src map-fnc)
#11
newLISP newS / (for-all cond list1 [... listn])
June 07, 2007, 03:55:26 PM
It would be nice if (for-all ..) allowed for additional,

optional list arguments.



    (for-all (fn (item) ...)  list1 ... listn)



It may not always be efficient to use append to

concatenate the lists since the for-all may be in

a loop itself.



It would also be nice to have a disjunction complement to

for-all called for-any



    (for-any cond  list1 ... listn)
#12
newLISP and the O.S. / write-line outputs CRCRLF on my XP box
May 31, 2007, 05:03:03 PM
Hi,



I'm using version 9.1.1 on windows XP.



  (write-line "")



and



   (println "")



both output a three character sequence



      CRCRLF



instead of just



     CRLF



Is there something I'm doing wrong?
Pages1