newLISP Fan Club

Forum => newLISP in the real world => Topic started by: SHX on November 01, 2007, 06:02:41 PM

Title: Some help needed coding
Post by: SHX on November 01, 2007, 06:02:41 PM
Hi



Here is what I am trying to do.



I have a file that looks like the following
"11","abc","some value"
"12","aaa","another value"
"13","bbb","next value"


What I have to do is

1- read each line

2- check the number in quotes against a list of numbers that should not be used

3- If it does exist in that list, replace it with a number from a list of available numbers

4- rewrite the updated line





I am able to do step 1 and 2 and I get an available number from an available number list.



But how would I rewite the line and replace the number in quotes.



Thanks for your help



Steven
Title:
Post by: HPW on November 02, 2007, 12:11:27 AM
Depends on your first 2 steps.



When you used 'parse' with delimiter ',' then you have the number as a separate element.

Then you can concatanate the new line with 'string' to whatever you need.



When you only as the whole line a replace with a regex should do the job.
Title:
Post by: cormullion on November 02, 2007, 01:59:26 AM
Yes - many ways to do it. Reading from stdin, file, etc....


(set 'file (open {test-file.txt} "read"))

(define (check n-str)
  (let (n (int n-str 0 0))
    (cond
     ((= n 11)   21)
     ((= n 12)   22)
     (true           n))))
       
(while (read-line file)
  (replace {"(d+)(".*$)} (current-line) (println {"} (check $1) $2) 0))


As HPW says, in simple cases, a regex (cursed things) is a good way. For more complicated situations, perhaps parse is better.
Title:
Post by: rickyboy on November 02, 2007, 01:27:16 PM
Steven,



In case your question is more fundamental than I originally perceived and has to do with file I/O ... (if not, please ignore the following)



I usually read the source file and write the changes to another (different, new) file.  If I want to then later overwrite the input file, then I do a mv (rename) of the output file to the input file.  Sometimes I have a multi-step process that reads the input from the previous process and writes an output file for a subsequent process.  I used to think this extra step was lame, but later, as I had to debug some of the trashy code I wrote :-), I found it useful to have both the original file and any intermediate files out there to inspect.



In your case you have a one step process (for now), but the idea is the same.  Hope that helps.



--Rick
Title:
Post by: SHX on November 12, 2007, 05:40:36 AM
Thanks all for your help. I was a little sidelined so am finally getting back to this.



Here is how I did it.
; this will clean out the numbers that should not be used when loading a table

;---- List of Unavailable Numbers
(setq not-available-list
(map int
(parse ( read-file "test4.txt" ) {rn} 0)
)
)
(println "not available " not-available-list)


;---- List of All Numbers in a range
(setq the-number-list
(sequence 1 100)
)
(setq is-available-list
(difference the-number-list not-available-list)
)
(println "is available "is-available-list )


; open the file and read the number
(set 'outFile (open "tempexpo.txt" "write"))
(set 'inFile (open "tempexp.txt" "read"))

(while (read-line inFile)
  (setq the-line   ; create a list from a line in the file
  (parse (current-line) {,} 0)
  )
  (setq thepgmcode (int (eval-string (the-line 0)))) ; get the first member of the list which is the pgm code
  (setq repl-num ; set repl-num to the pgm code or an avail number
  (if (find thepgmcode not-available-list)
  (pop is-available-list )
  thepgmcode )
  )
  (write-line (join (set-nth 0 the-line (string repl-num )) {,} ) outFile)
 )


Any comments. I am new at this and appreciate the feedback.



Also, what would be the best way for me to code a check that there is a number in the position I am expecting and if not to skip that record?



Thanks



Steven
Title:
Post by: rickyboy on November 12, 2007, 09:29:25 AM
Hi Steven!



Your code looks fine.  I wouldn't change anything about it from a functional viewpoint.  As advice or comment which you can take or leave, I would just re-organize the expressions a bit.



These are just what I am comfortable with and somewhat personal, not universal truths.  :-)



(1) I'd move some items which kind of control the processing into a group of defines at the top of the source.  This makes it easier to change the control items, instead of searching throughout the code for them.



(2) I'd change the "list" references to "set" references, e.g. not-available-set vice not-available-list, since the abstract notion we're working with is a set, but the implementation of the set is the list.  BTW, nice use of the difference function -- it's nice to see set theoretic functions used in programs.



(3) I'd wrap the rest of the processing (the "punchline"), starting from the opening of the principal files, into a let statement (see the code below).  I love let statements, they tell the reader "these variables are only meant to be used here."  It doesn't really matter so much in your program since it is relatively small, but in larger programs, this usage of the lets is a God send, when I have to go back and read my own code after 6 weeks.  :-)


;;;; This will clean out the numbers that should not be used when
;;;; loading a table.

;;;-------------------
;;; Control variables

(define code-domain (sequence 1 100))
(define unavail-filename "test4.txt")
(define in-filename "tempexp.txt")
(define out-filename "tempexpo.txt")

;;;-------------
;;; Consequents

;; The set of unavailable codes are contained in the file
;; 'unavail-filename'.
(setq not-available-set
      (map int
           (parse (read-file unavail-filename) {rn} 0)))
(println "not available " not-available-set)

;; Codes that are available to be dynamically alotted are from
;; 'code-domain', but which are also *not* in 'not-available-set'.
(setq is-available-set
      (difference code-domain not-available-set))
(println "is available " is-available-set)

;;;-----------------------------------
;;; Now, let the rubber meet the road.

(let ((inFile (open in-filename "read"))
      (outFile (open out-filename "write")))
  (while (read-line inFile)
    (letn ((the-line (parse (current-line) {,} 0))
           (thepgmcode (int (eval-string (the-line 0))))
           (repl-num (if (find thepgmcode not-available-set)
                         (pop is-available-set)
                       thepgmcode)))
      (write-line (join (set-nth 0 the-line (string repl-num )) {,})
                  outFile))))
Title:
Post by: rickyboy on November 12, 2007, 10:24:08 AM
Quote from: "SHX"Also, what would be the best way for me to code a check that there is a number in the position I am expecting and if not to skip that record?

Don't know what you mean by this.  Can you elaborate?
Title:
Post by: SHX on November 13, 2007, 09:49:55 AM
rickyboy, thanks for getting back to me.



You wrote
(define code-domain (sequence 1 100))
I wrote
(setq the-number-list  (sequence 1 100) )

Why is "define" more appropriate than "setq"



Steven
Title:
Post by: cormullion on November 13, 2007, 10:13:36 AM
I'm interested in the answer to that too - I have wondered. The manual says:


QuoteThe second version of define works like the set function.



example:



(define x 123)  →   123

;; is equivalent to

(set 'x 123)    →   123


so perhaps you can use the one that matches the situation better. Eg consistency with other expressions in the same part of the code, or emphasizing creation rather than changing value? But I'm just guessing...
Title:
Post by: Lutz on November 13, 2007, 10:28:37 AM
Quote from: "Cormullion" Eg consistency with other expressions in the same part of the code, or emphasizing creation rather than changing value?


Yes



Internally there is absolute no difference between:
(define x 123) → 123
;; is equivalent to
(set 'x 123) → 123


the second version of define works/is just like a set and was created for compatibility with other LISPs. I suggest using it at the top of a file when introducing variables which are used frequently in a module.



It is also a good custom using it when defining a new namespace/context and variable at the same time, i.e:


(define Config:path "/usr/data/")

The same could be done with a set, but using the 'define' signals to the code reader, that an important frequently used variable is introduced.



Lutz
Title:
Post by: SHX on November 13, 2007, 01:19:05 PM
Thanks Lutz,



rickyboy
QuoteDon't know what you mean by this. Can you elaborate?


What I mean is do I have to worry about the fact that if I am processing a blank line and I do the parse and find that there is no number in the position where I expected the number to be or if there was a mistake and there weas charachters instead of a number that it wouldn't cause a problem.


QuoteI'd wrap the rest of the processing (the "punchline"), starting from the opening of the principal files, into a let statement (see the code below).


I understand and really appreciate the tip.





Steven
Title:
Post by: rickyboy on November 13, 2007, 02:07:04 PM
Quote from: "SHX"
QuoteDon't know what you mean by this. Can you elaborate?

What I mean is do I have to worry about the fact that if I am processing a blank line and I do the parse and find that there is no number in the position where I expected the number to be or if there was a mistake and there weas charachters instead of a number that it wouldn't cause a problem.


Got it!  OK, well here's one way (but as I said before, not THE way): put an escaping catch/throw combination in the code.
(define allocate-from pop)

;;;-----------------------------------
;;; Now, let the rubber meet the road.

(let ((inFile (open in-filename "read"))
      (outFile (open out-filename "write")))
  (while (read-line inFile)
    (catch
      (letn ((the-line (parse (current-line) {,} 0))
             (thepgmcode (int (eval-string (the-line 0))))
             (repl-num (cond ((null? thepgmcode) (throw 'escape))
                             ((find thepgmcode not-available-set)
                              (allocate-from is-available-set))
                             thepgmcode)))
        (write-line (join (set-nth 0 the-line (string repl-num )) {,})
                    outFile)))))

Unrelated to your question, I changed the pop call to an allocate-from call.  What's allocate-from, you may ask?  Just pop.  :-)  It's my pedantic way of trying to live up to my abstract data type ideals.
Title:
Post by: SHX on November 13, 2007, 07:31:34 PM
Thanks rickyboy.
Title:
Post by: SHX on November 14, 2007, 04:56:55 AM
Rickyboy

Could you have coded it like this?
(catch
      (letn ((the-line (parse (current-line) {,} 0))
             (thepgmcode (int (eval-string (the-line 0))))
             (cond ((null? thepgmcode) (throw 'escape)) )
             (repl-num
                             ((find thepgmcode not-available-set)
                              (allocate-from is-available-set))
                             thepgmcode)))
        (write-line (join (set-nth 0 the-line (string repl-num )) {,})
                    outFile))

Where  (cond ((null? thepgmcode) (throw 'escape))  comes earlier.



It would seem to make more sense.



Steven
Title:
Post by: SHX on November 15, 2007, 07:44:17 AM
Thanks Rickyboy for your help. It really opened my eyes to how to code in Newlisp



I unserstood from you the value of define and used it in definning a condition that I am checking for.







Here is how I did it at the end. I did not wanted to exit the loop but rather just skip blank lines.


(define (the-line-is-empty) (null? (current-line) ) )

;;;-----------------------------------
;;; Now, let the rubber meet the road.

(let ((inFile (open in-filename "read"))
      (outFile (open out-filename "write")))
  (while (read-line inFile)
    (unless (the-line-is-empty) ;... if blank skip
    (letn ((the-line (parse (current-line) {,} 0))
           (thepgmcode (int (eval-string (the-line 0))))
           (repl-num (if (find thepgmcode not-available-set)
                         (allocate-from is-available-set)
                          thepgmcode)))
      (write-line
      (join
      (set-nth 0 the-line (string repl-num )) {,})
      outFile)))))
Title:
Post by: rickyboy on November 15, 2007, 11:33:51 AM
Quote from: "SHX"Here is how I did it at the end. I did not wanted to exit the loop but rather just skip blank lines.

Very good.  I'd say that your choice was most appropriate given your needs.



BTW, I hope that the catch/throw I used didn't exit the loop -- it wasn't supposed to anyway.  And I should have warned you that I didn't test my code.  Sorry.  However, I perceived that you were only really after understanding and not free code.  Clearly I was right, because you accomplished it very well all by yourself.



Now, it's your turn to stick around and help us, now that we know you are capable.  That is, you just "outed" yourself, my friend.  :-)
Title:
Post by: SHX on November 15, 2007, 12:47:23 PM
:-)

just learning. thanks for your help and direction.