Some help needed coding

[SHX]

Hi



Here is what I am trying to do.



I have a file that looks like the following

Code Select Expand

"11","abc","some value"
"12","aaa","another value"
"13","bbb","next value"

What I have to do is

1- read each line

2- check the number in quotes against a list of numbers that should not be used

3- If it does exist in that list, replace it with a number from a list of available numbers

4- rewrite the updated line





I am able to do step 1 and 2 and I get an available number from an available number list.



But how would I rewite the line and replace the number in quotes.



Thanks for your help



Steven

[HPW]

Depends on your first 2 steps.



When you used 'parse' with delimiter ',' then you have the number as a separate element.

Then you can concatanate the new line with 'string' to whatever you need.



When you only as the whole line a replace with a regex should do the job.

[cormullion]

Yes - many ways to do it. Reading from stdin, file, etc....

Code Select Expand

(set 'file (open {test-file.txt} "read"))

(define (check n-str)
  (let (n (int n-str 0 0))
    (cond
     ((= n 11)   21)
     ((= n 12)   22)
     (true           n))))
       
(while (read-line file)
  (replace {"(d+)(".*$)} (current-line) (println {"} (check $1) $2) 0))


As HPW says, in simple cases, a regex (cursed things) is a good way. For more complicated situations, perhaps parse is better.

[rickyboy]

Steven,



In case your question is more fundamental than I originally perceived and has to do with file I/O ... (if not, please ignore the following)



I usually read the source file and write the changes to another (different, new) file.  If I want to then later overwrite the input file, then I do a mv (rename) of the output file to the input file.  Sometimes I have a multi-step process that reads the input from the previous process and writes an output file for a subsequent process.  I used to think this extra step was lame, but later, as I had to debug some of the trashy code I wrote :-), I found it useful to have both the original file and any intermediate files out there to inspect.



In your case you have a one step process (for now), but the idea is the same.  Hope that helps.



--Rick

[SHX]

Thanks all for your help. I was a little sidelined so am finally getting back to this.



Here is how I did it.

Code Select Expand

; this will clean out the numbers that should not be used when loading a table

;---- List of Unavailable Numbers
(setq not-available-list
(map int
(parse ( read-file "test4.txt" ) {rn} 0)
)
)
(println "not available " not-available-list)


;---- List of All Numbers in a range
(setq the-number-list
(sequence 1 100)
)
(setq is-available-list
(difference the-number-list not-available-list)
)
(println "is available "is-available-list )


; open the file and read the number
(set 'outFile (open "tempexpo.txt" "write"))
(set 'inFile (open "tempexp.txt" "read"))

(while (read-line inFile)
  (setq the-line   ; create a list from a line in the file
  (parse (current-line) {,} 0)
  )
  (setq thepgmcode (int (eval-string (the-line 0)))) ; get the first member of the list which is the pgm code
  (setq repl-num ; set repl-num to the pgm code or an avail number
  (if (find thepgmcode not-available-list)
  (pop is-available-list )
  thepgmcode )
  )
  (write-line (join (set-nth 0 the-line (string repl-num )) {,} ) outFile)
 )


Any comments. I am new at this and appreciate the feedback.



Also, what would be the best way for me to code a check that there is a number in the position I am expecting and if not to skip that record?



Thanks



Steven

[rickyboy]

Hi Steven!



Your code looks fine.  I wouldn't change anything about it from a functional viewpoint.  As advice or comment which you can take or leave, I would just re-organize the expressions a bit.



These are just what I am comfortable with and somewhat personal, not universal truths.  :-)



(1) I'd move some items which kind of control the processing into a group of defines at the top of the source.  This makes it easier to change the control items, instead of searching throughout the code for them.



(2) I'd change the "list" references to "set" references, e.g. not-available-set vice not-available-list, since the abstract notion we're working with is a set, but the implementation of the set is the list.  BTW, nice use of the difference function -- it's nice to see set theoretic functions used in programs.



(3) I'd wrap the rest of the processing (the "punchline"), starting from the opening of the principal files, into a let statement (see the code below).  I love let statements, they tell the reader "these variables are only meant to be used here."  It doesn't really matter so much in your program since it is relatively small, but in larger programs, this usage of the lets is a God send, when I have to go back and read my own code after 6 weeks.  :-)

Code Select Expand

;;;; This will clean out the numbers that should not be used when
;;;; loading a table.

;;;-------------------
;;; Control variables

(define code-domain (sequence 1 100))
(define unavail-filename "test4.txt")
(define in-filename "tempexp.txt")
(define out-filename "tempexpo.txt")

;;;-------------
;;; Consequents

;; The set of unavailable codes are contained in the file
;; 'unavail-filename'.
(setq not-available-set
      (map int
           (parse (read-file unavail-filename) {rn} 0)))
(println "not available " not-available-set)

;; Codes that are available to be dynamically alotted are from
;; 'code-domain', but which are also *not* in 'not-available-set'.
(setq is-available-set
      (difference code-domain not-available-set))
(println "is available " is-available-set)

;;;-----------------------------------
;;; Now, let the rubber meet the road.

(let ((inFile (open in-filename "read"))
      (outFile (open out-filename "write")))
  (while (read-line inFile)
    (letn ((the-line (parse (current-line) {,} 0))
           (thepgmcode (int (eval-string (the-line 0))))
           (repl-num (if (find thepgmcode not-available-set)
                         (pop is-available-set)
                       thepgmcode)))
      (write-line (join (set-nth 0 the-line (string repl-num )) {,})
                  outFile))))


[rickyboy]

Also, what would be the best way for me to code a check that there is a number in the position I am expecting and if not to skip that record?

Don't know what you mean by this.  Can you elaborate?

[SHX]

rickyboy, thanks for getting back to me.



You wrote

Code Select Expand

(define code-domain (sequence 1 100))
I wrote

Code Select Expand

(setq the-number-list  (sequence 1 100) )

Why is "define" more appropriate than "setq"



Steven

[cormullion]

I'm interested in the answer to that too - I have wondered. The manual says:

The second version of define works like the set function.



example:



(define x 123)  →   123

;; is equivalent to

(set 'x 123)    →   123

so perhaps you can use the one that matches the situation better. Eg consistency with other expressions in the same part of the code, or emphasizing creation rather than changing value? But I'm just guessing...

[Lutz]

Eg consistency with other expressions in the same part of the code, or emphasizing creation rather than changing value?

Yes



Internally there is absolute no difference between:

Code Select Expand

(define x 123) → 123
;; is equivalent to
(set 'x 123) → 123


the second version of define works/is just like a set and was created for compatibility with other LISPs. I suggest using it at the top of a file when introducing variables which are used frequently in a module.



It is also a good custom using it when defining a new namespace/context and variable at the same time, i.e:

Code Select Expand

(define Config:path "/usr/data/")

The same could be done with a set, but using the 'define' signals to the code reader, that an important frequently used variable is introduced.



Lutz

[SHX]

Thanks Lutz,



rickyboy

Don't know what you mean by this. Can you elaborate?

What I mean is do I have to worry about the fact that if I am processing a blank line and I do the parse and find that there is no number in the position where I expected the number to be or if there was a mistake and there weas charachters instead of a number that it wouldn't cause a problem.

I'd wrap the rest of the processing (the "punchline"), starting from the opening of the principal files, into a let statement (see the code below).

I understand and really appreciate the tip.





Steven

[rickyboy]

Don't know what you mean by this. Can you elaborate?

What I mean is do I have to worry about the fact that if I am processing a blank line and I do the parse and find that there is no number in the position where I expected the number to be or if there was a mistake and there weas charachters instead of a number that it wouldn't cause a problem.

Got it!  OK, well here's one way (but as I said before, not THE way): put an escaping catch/throw combination in the code.

Code Select Expand

(define allocate-from pop)

;;;-----------------------------------
;;; Now, let the rubber meet the road.

(let ((inFile (open in-filename "read"))
      (outFile (open out-filename "write")))
  (while (read-line inFile)
    (catch
      (letn ((the-line (parse (current-line) {,} 0))
             (thepgmcode (int (eval-string (the-line 0))))
             (repl-num (cond ((null? thepgmcode) (throw 'escape))
                             ((find thepgmcode not-available-set)
                              (allocate-from is-available-set))
                             thepgmcode)))
        (write-line (join (set-nth 0 the-line (string repl-num )) {,})
                    outFile)))))

Unrelated to your question, I changed the pop call to an allocate-from call.  What's allocate-from, you may ask?  Just pop.  :-)  It's my pedantic way of trying to live up to my abstract data type ideals.

[SHX]

Thanks rickyboy.

[SHX]

Rickyboy

Could you have coded it like this?

Code Select Expand

(catch
      (letn ((the-line (parse (current-line) {,} 0))
             (thepgmcode (int (eval-string (the-line 0))))
             (cond ((null? thepgmcode) (throw 'escape)) )
             (repl-num
                             ((find thepgmcode not-available-set)
                              (allocate-from is-available-set))
                             thepgmcode)))
        (write-line (join (set-nth 0 the-line (string repl-num )) {,})
                    outFile))
Where  (cond ((null? thepgmcode) (throw 'escape))  comes earlier.



It would seem to make more sense.



Steven

[SHX]

Thanks Rickyboy for your help. It really opened my eyes to how to code in Newlisp



I unserstood from you the value of define and used it in definning a condition that I am checking for.







Here is how I did it at the end. I did not wanted to exit the loop but rather just skip blank lines.

Code Select Expand

(define (the-line-is-empty) (null? (current-line) ) )

;;;-----------------------------------
;;; Now, let the rubber meet the road.

(let ((inFile (open in-filename "read"))
      (outFile (open out-filename "write")))
  (while (read-line inFile)
    (unless (the-line-is-empty) ;... if blank skip
    (letn ((the-line (parse (current-line) {,} 0))
           (thepgmcode (int (eval-string (the-line 0))))
           (repl-num (if (find thepgmcode not-available-set)
                         (allocate-from is-available-set)
                          thepgmcode)))
      (write-line
      (join
      (set-nth 0 the-line (string repl-num )) {,})
      outFile)))))