I'm missing my foreach keys %hash... :(

[_ex_]

Hi there, I want to use hash tables in newLisp,

Take for example this code (Python challenge ;)

Code Select Expand


;* http://www.pythonchallenge.com/pc/def/map.html                                          
(setq input
(string
"g fmnc wms bgblr rpylqjyrc gr zw fylb. rfyrq ufyr amknsrcpq ypc dmp.n"
"bmgle gr gl zw fylb gq glcddgagclr ylb rfyr'q ufw rfgq rcvr gq qm jmle.n"
"sqgle qrpgle.kyicrpylq() gq pcamkkclbcb. lmu ynnjw ml rfc spj."))

; Create table for decryption
(for (k 0 25)
(setq key (char (+ k (char "a"))))
(setq value (char (+ (% (+ k 2) 26) (char "a"))))
(context 'dic key value)
;;(println key " - " value)
)

(define (decrypt input table)
(setq len (length input))
(setq output "")
(for (k 0 (- len 1))
(setq key (char (char input k)))
(if (context? table key)
(setq output (string output (context table key)))
(setq output (string output key))
)
)
(setq return output)
)

(println (decrypt input dic))
(println (decrypt "map" dic))


This way I decoded the string, using a hash table for lookup, we have "context?" for checking existence, that's nice, but what about length or foreach operations?, I guess I'm missing something... :(

I have read the docs about the "context" thing, by the way.



Is not this a *weird* way to add hashes to the language? I read some posts than says earlier implementations of newLisp didn't have context before.

But I want to believe that I'm still trying to program "the perl, ruby, python way". Please tell me how is the newLip way to do this kind of simple things. Please.



(ex)

[cormullion]

Hi _ex_. I don't know what "foreach keys %hash..." is, so I can't help you with contexts and hashes much. There are probably others who know Python here.



But you asked how do others do tasks like this, and this would be my approach:

Code Select Expand

(map (fn (c)
(if   (and  (LT c (char "y"))
               (>= c (char "a")))
              (print (char (+ 2 c)))
         (>= c (char "y"))
              (print (char (- c 24)))
         (LT c (char "a"))
              (print (char c))))
(map char (explode input)))


(for some reason I couldn't get the angle brackets to work - replace LT with an opening one)



Actually at first i just did a map to see what was what:

Code Select Expand


(map (fn (f) (print (char (+ 2 (char f))))) (explode input))


but that revealed that the puntuation hadn't been encrypted, so more logic was needed.



If you want more cryptic puzzles to solve, there's still the Da Vinci Code newLISP puzzle http://newlisper.blogspot.com">//http://newlisper.blogspot.com  ;-(((((

[Lutz]

There are two ways to iterate through a hash, which are better called dictionaries or namespaces in newLISP. Even scripting languages using hashes tend to talk about them as dictionaries lately ;)

Code Select Expand

; make a dictionary
(context 'foo "sdfg" 111)
(context 'foo "yete" 222)
(context 'foo "asdf" 333)

; display all key value pairs in alphabetical order

(dotree (s foo)
    (println s "->" (eval s)))

; or an alternative method

(dolist (s (symbols foo))
    (println s "->" (eval s)))

; generates the output

foo:asdf->333
foo:sdfg->111
foo:yete->222


Instead of 'context' you also can use 'sym' for creating/testing/modifying symbols. Sometimes its more convenient, it depends on the situation.



Lutz



ps: the docoument  http://newlisp.org/DesignPatterns.html">http://newlisp.org/DesignPatterns.html has been updated with the previous explanations in chapter 11

[_ex_]

Thank you guys.

I'm learning new things with newLisp, what is good, and I'm havng a nice time in the way :). map seems an interesting function and so dotree and dolist.

cormullion:

the idea was to use a dictionary for this problem, but your function helpme to undestand the *map* concept, I refined a little using regex, I loved perl just by its regex support...

Code Select Expand


(define (decrypt input , output)
(setq output "")
(map
(fn (c)
(if (regex "[a-z]+" (char c))
(print (char (+ (% (+ (- c (char "a")) 2) 26) (char "a"))))
(print (char c))
)
)
(map char (explode input))
)
(setq return output)
)


Lutz: nice to read that docu :)

How about sorting a dictionary? I can't find an example on how to use sort to sort contexts, something like this in perl:

Code Select Expand


foreach $value (sort {$coins{$a} cmp $coins{$b} } keys %coins)
{
     print "$value $coins{$value}<br>";
}

[/code]

[cormullion]

How about sorting a dictionary? I can't find an example on how to use sort to sort contexts

:-) I remember asking that question too (http://www.alh.net/newlisp/phpbb/viewtopic.php?t=892&highlight=sort+hash">//http://www.alh.net/newlisp/phpbb/viewtopic.php?t=892&highlight=sort+hash and I don't think I've got the hang of this area of newLISP yet enough. Somehow I feel I should be able to reverse-sort a dictionary, but I've managed to cope without one so far. But...



As an example, is there a better way to do this:

Code Select Expand


(context 'english-german "please" "bitte")
(context 'english-german "thanks" "danke schone")
(context 'english-german "excuse me" "entschuldigen sie")
(context 'english-german "beer" "bier")
(context 'english-german "coffee" "kaffee")

(println "English-German dictionary")
(dotree (s  english-german)
    (println " " s "->" (context english-german (name s))))

; reverse 'sort' to create German-English dictionary...
(dotree (s english-german)
(context 'german-english (context english-german (name s)) s))
(println "German-English dictionary")
(dotree (s german-english)
    (println " " s "->" (eval s)))


Which creates a new context with the entries reversed:

Code Select Expand


English-German dictionary
 english-german:beer->bier
 english-german:coffee->kaffee
 english-german:excuse me->entschuldigen sie
 english-german:please->bitte
 english-german:thanks->danke schone
German-English dictionary
 german-english:bier->english-german:beer
 german-english:bitte->english-german:please
 german-english:danke schone->english-german:thanks
 german-english:entschuldigen sie->english-german:excuse me
 german-english:kaffee->english-german:coffee


- as you see, it went a bit wrong here.



I've noticed that sometimes newLISP requires us to 'unlearn' things as well as 'learn' things...

[Lutz]

your code for inverting the dictionary is fine, just use 'name' once more when assigning the value:

Code Select Expand

(dotree (s english-german)
    (context 'german-english (context english-german (name s)) (name s)))

(dotree (s german-english)
    (println " " s "->" (eval s)))
=>
 german-english:bier->beer
 german-english:bitte->please
 german-english:danke schone->thanks
 german-english:entschuldigen sie->excuse me
 german-english:kaffee->coffee


Lutz



Ps: 'name' supresses the context part of a symbol when converting to a string.

[_ex_]

ouch! this hurts...

All that I was trying to do was sort my map in order to decrypt a message, this map has the form:

map frequency { "char" => freq }

where freq is the number of repetitions of the character "char" in one string:

But the dotree function sorts them (seems) as they were strings.. :(



Consider the next problem: you got an *encrytpted* message and the frequency info of characters in that language in one string from more frequent to less frequent. *decrypt* that message...

This was very easy to do in ruby (and I'm learning it at the side of newLisp):

Code Select Expand


# Decrypt
text = "Uid nx, aex jcdjipx iu wzux zp, ta wxtpa jtdaws, ai etkx vis."

freqLang = "TEOIARNSHLMYUCWDGPFBVKJ"

len = text.length
frequency = Hash.new

for k in 0..( len - 1 )
c = text[ k, 1 ].upcase
if c =~ /[A-Z]/
if frequency.has_key?( c )
frequency[ c ] = frequency[ c ] + 1;
else
frequency[ c ] = 1;
end
end
end

dic = Hash.new
freqText = ""
index = 0

frequency.sort{ |a,b| b[1]<=>a[1] }.collect{ |a|
#puts( a[0] + " - " + a[1].to_s )
freqText += a[0]
dic[ a[0].upcase ] = freqLang[ index, 1 ]
index += 1
}

puts( freqLang )
puts( freqText )

decrypted = "";
len = text.length;

for k in 0..( len - 1 )
uper = false
c = text[ k, 1 ]

if c =~ /[A-Z]/ then uper = true
else c = c.upcase end

if dic.has_key?( c )
if uper then decrypted += dic[ c ]
else decrypted += dic[ c ].downcase end
else decrypted += c end
end

puts( decrypted )

I don't want to past the perl code here but is equally suscint.

Now when I tryed the newLips version I got stuck...

Code Select Expand


(setq text
(string "Uid nx, aex jcdjipx iu wzux zp, ta wxtpa jtdaws, ai etkx vis."))

(setq freqLang "TEOIARNSHLMYUCWDGPFBVKJ")

(context 'frequency) ;; Here we are ~creating~ a hash...
(context 'MAIN)  ;; THIS IS UGLY!!! but sadly necessary.. guess why?
(context 'inverse)
(context 'MAIN)

(for (k 0 (- (length text) 1))
(setq c (upper-case (char (char text k))))
(if (regex "[A-Z]" c)
(if (context? frequency c)
(context 'frequency c (+ 1 (context 'frequency c)))
(context 'frequency c 1)
)
)
)

(dotree (s 'frequency)
(println (name s) " - " (eval s))
(context 'inverse (string (eval s)) (name s))
)

(dotree (s 'inverse)
(println s)
)


Lutz you are a kind person, and newLips is REALLY cool but...

newLisp is using context for two totally different kind of things

- context as a perl package

- context as a perl hash

I think seriously in the second form only as a hack (and not of the nice ones)

My friends are going to laugh if I tell them this...

We live in the *regex* world and here perl is KING, but so much Paul Graham is affecting my brain, some neurons on my head are  spiking "hey maybe it's true" but when I see CLisp I don't know where to start, Where are my regex?, Where are my arrays? Where are my hashes? How can I open/write files?  That all that I can think, excuse my rudeness. Sure thing they must be there or maybe a superior version of these, but I don't have the time (years I suspect) to master it.

Now newLisp is cool because it can be run from the console, has regex, has arrays, open/write files, has hashes, errr... no it doesn't have hashes.

And you know dictionaries are hell useful.

Any ~modern~ language has them,  the faty snake python has them, the prima-donna ruby has them, even the minuscule lua has them...

So I hope newlisp.



(ex)

[Lutz]

A namespace/context for implementing dictionaries is a very efficient way to implement a dictionary. newLISP namespaces/contexts are implemented using red-black binary trees, a very speedy and memory efficient method to implement dictionaries and more scalable than implementations using hashing techniques, which need allocate overflow spaces and take care of ties produced by the hashing algorithms.



Here is a short little module to implement dictionaries which feel more like what you are used in Ruby or Python:

Code Select Expand

;; module hash.lsp
(context 'HASH)

(define (HASH:HASH this)
    (new 'MAIN:HASH this))

(define (put key value)
        (context this key value))

(define (get key)
        (context this key))

(define (keys)
    (map name (difference (symbols this)
                (append '(this put get key keys value)
                         (list (sym (name this) this))))))
(context MAIN)


and its called HASH to make everybody feel more familiar ;). This is how to use it:

Code Select Expand

(load "hash.lsp")

; create a new HASH
(HASH 'myhash)                            

; put some values
(myhash:put "abc" 123)                
(myhash:put "xyz" 456)

; retrieve values
(myhash:get "abc")     => 123      
(myhash:get "xyz")     => 456

; retrieve all keys
(myhash:keys)   => ("abc" "xyz")  


Myself I prefer to use the form (context ctx key value) directly because it is directer and faster. But this HASH module gives you a very readable syntax familiar in other scripting languages.



Lutz

[_ex_]

Lutz my friend, seems I still have lots to walk in the newLisp way :)

In your example:

Code Select Expand

; create a new HASH
(HASH 'myhash)

; put some values
(myhash:put "a" 123)
(myhash:put "x" 456)
(myhash:put "d" 5)


How do you sort the dictionary by value?, I mean I wan't to get

Code Select Expand

(println myhash:sort-by-value-descend)
=> 456 "x"
=> 123 "a"
=> 5 "d"

This is the hard part for me.

Regards.

(ex)

[frontera000]

Code Select Expand


(sort (map (lambda (x) (list (myhash:get x) x)) (myhash:keys))  (lambda (x y) (< y x)))


[_ex_]

Thank you very much frontera000!

Code Select Expand

(println (sort (map (lambda (x) (list (myhash:get x) x)) (myhash:keys))  (lambda (x y) (< y x))))
(println (sort (map (lambda (x) (list (myhash:get x) x)) (myhash:keys))  (lambda (x y) (> y x))))


[_ex_]

Finally... but you must to agree:

It takes more code (and more neurons) :)

Code Select Expand


(context 'DIC)

(define (DIC:DIC this)
(new 'MAIN:DIC this))

(define (put key value)
(context this key value))

(define (get key)
(context this key))

(define (has_key key)
(context? this key))

(define (echo)
(map (fn (key) (println key " - " (DIC:get key))) (DIC:keys)))

(define (vsort func)
(sort (map (fn (key) (list (DIC:get key) key)) (DIC:keys)) func))

(define (keys)
(map name (difference (symbols this)
(append '(this put get key keys value has_key echo vsort func)
(list (sym (name this) this))))))

(context MAIN)

; Decrypt
(setq text (string
"Uid nx, aex jcdjipx iu wzux zp, ta wxtpa jtdaws, ai etkx vis.n"
"Scfzezdi Ntapcniai. (hhh.tdaznt.qin/zyak/dcos)"))

(setq freqLang "TEOIARNSHLMYUCWDGPFBVKJ")

(DIC 'frequency)

(for (k 0 (- (length text) 1))
(setq c (upper-case (char (char text k))))
(if (regex "[A-Z]" c)
(if (frequency:has_key c)
(frequency:put c (+ 1 (frequency:get c)))
(frequency:put c 1))))

(setq freqText "")
(DIC 'dic)
(setq ind 0)

(map
(fn (x)
;;(println (x 1) " - " (x 0))
(setq freqText (string freqText (x 1)))
(dic:put (upper-case (string (x 1))) (char (char freqLang ind)))
(inc 'ind))
(frequency:vsort (fn (x y) (< y x))))

(println freqLang)
(println freqText)

(setq decrypted "")

(for (k 0 (- (length text) 1))
(setq uper 0)
(setq c (char (char text k)))
(if (regex "[A-Z]" c)
(setq uper 1)
(setq c (upper-case c)))
(if (dic:has_key c)
(if (= uper 1)
(setq decrypted (string decrypted (dic:get c)))
(setq decrypted (string decrypted (lower-case (dic:get c)))))
(setq decrypted (string decrypted c))))

(println decrypted)


If you feel something is wrong or can be improved, please bring it on! (pedantic-mode-encorauged)

[HPW]

A little minor thing from pedantic-mode. ;-)

Since (setq ..) can take multiple arguments you can make one of them here.

In big loops it could be a bit faster.

Code Select Expand

(for (k 0 (- (length text) 1))
   (setq uper 0
         c (char (char text k)))
   (if (regex "[A-Z]" c)
      (setq uper 1)
      (setq c (upper-case c)))
   (if (dic:has_key c)
      (if (= uper   1)
         (setq decrypted (string decrypted (dic:get c)))
         (setq decrypted (string decrypted (lower-case (dic:get c)))))
      (setq decrypted (string decrypted c))))


[_ex_]

nice to know :)

[cormullion]

Hi _ex_! Good code! I hope you continue with newLISP, despite your contextual reservations!



I was interested in your first Ruby script for decryption, and tried to translate it to a newLISP version, sticking to my own programming style :-)

Code Select Expand


(set
  'text "Uid nx, aex jcdjipx iu wzux zp, ta wxtpa jtdaws, ai etkx vis. Scfzezdi Ntapcniai. (hhh.tdaznt.qin/zyak/dcos"
  'freq-lang "TEOIARNSHLMYUCWDGPFBVKJ"
  'freq-table '())

(map (fn (ch)
  (if (regex "[A-Z]" ch)
    (begin
      (set 'found (lookup ch freq-table))
      (if found
          (replace-assoc ch freq-table (list ch (+ 1 found)))
        (push (list ch 1) freq-table)))))
  (map upper-case (explode text)))

; sort
(sort freq-table (fn (x y) (> (last x) (last y))))

; build dictionary
(map (fn (d e)
  (push (list (first d) (last d) e) dict))  
  freq-table (explode freq-lang))

; decode text
(map (fn (c)
  (set 'a (lookup (upper-case c) dict))
  (set 'upper-case? (= c (upper-case c)))
  (if a
    (if upper-case? (print a) (print (lower-case a)))
    (if upper-case? (print c) (print (lower-case c)))))
  (explode text))


As you see, I tend to not use hash/dictionary/contexts automatically, but stick with my favourite list-based structures. To be honest I'm still not confident in my use of contexts other than for building modules. And with only the alphabet to store...



I was puzzled for a while as to why none of these scripts actually generates the right answer...;-) But it must be because there aren't enough letters in the text for the statistics to work. Also, the answers of my script and yours differ: again, I think this is because the letters with similar frequencies are sorted differently, and assigned differently as a result - not enough coded text to get some differences.



Or I have I misread your code?



Anyway, good stuff.