Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - dexterlagan

Pages1
#1
newLISP in the real world / Re: Writing to a C array of floats
August 21, 2018, 02:04:26 PM
Hi there,



  Yes I use pack / unpack in my code, and it seems to be working, but I must be missing something, as the genann procedure does not seem to notice that I'm writing to its array.



Dex


Quote from: "TedWalther"Dexter, have you looked at the pack and unpack functions?  I think they'll do what you want.
#2
newLISP in the real world / Writing to a C array of floats
August 21, 2018, 06:55:11 AM
Hi There!



  I'm attempting to implement the second example for the GENANN neural network library in newLISP. I made a module which works fine for the first example, however the second example requires writing random floats to an array. A pointer to the array is found in the struct sent back from GENANN's init function. Here is the module code for background:


Code Select

; genann.lsp
;
(context 'genann)
(println "GENANN for newLISP by Dexter Santucci v1.0. July 2018 - Support: dexterlagan@gmail.com")

(setq is-64-bit (= 0x100 (& 0x100 (sys-info -1))))
(setq has-ffi (= 1024 (& 1024 (sys-info -1))))

(if has-ffi
    (println "FFI detected and fully supported. Initializing library...")
    (println "Warning: this library may require newLISP to be compiled with FFI."))

;;; constants

(define RAND_MAX 32767)

;;; structures

; Define a genann structure
; IMPORTANT NOTE: all pointers shall be described at 'void*'
(struct 'network "int"     ; number of inputs
                 "int"     ; number of hidden_layers
                 "int"     ; number of hidden_neurons
                 "int"     ; number of outputs
                 "void*"   ; activation_hidden_func
                 "void*"   ; activation_output_func
                 "int"     ; total_weights
                 "int"     ; total_neurons
                 "void*"  ; weight - pointer to array of weights (array size: total_weights)
                 "void*"  ; pointer to array of input/output (array size: total_neurons)
                 "void*") ; pointer to array of delta of each hidden and output neuron
                           ; (array size: total_neurons - inputs)

;;; public functions

; Creates and returns a new ann.
; genann *genann_init(int inputs, int hidden_layers, int hidden, int outputs)
(setq init (import "genann.dll" "genann_init" "void*" "int" "int" "int" "int"))

; Creates ANN from file saved with genann_write.
; genann *genann_read(FILE *in)
(setq read-ann (import "genann.dll" "genann_read" "void*" "void*"))

; Sets weights randomly. Called by init.
; void genann_randomize(genann *ann)
(setq randomize-ann (import "genann.dll" "genann_randomize" "void" "void*"))

; Returns a new copy of ann.
; genann *genann_copy(genann const *ann)
(setq copy-ann (import "genann.dll" "genann_copy" "void*" "void*"))

; Frees the memory used by an ann.
; void genann_free(genann *ann)
(setq free (import "genann.dll" "genann_free" "void" "void*"))

; Runs the feedforward algorithm to calculate the ann's output.
; double const *genann_run(genann const *ann, double const *inputs)
(setq run (import "genann.dll" "genann_run" "void*" "void*" "void*"))

; Does a single backprop update.
; void genann_train(genann const *ann, double const *inputs,
;                   double const *desired_outputs, double learning_rate)
(setq train (import "genann.dll" "genann_train" "void" "void*" "void*" "void*" "double"))

; Saves the ann.
; void genann_write(genann const *ann, FILE *out)
(setq write-ann (import "genann.dll" "genann_write" "void" "void*" "void*"))

;;; internal functions

; void genann_init_sigmoid_lookup(const genann *ann)
(setq init_sigmoid_lookup (import "genann.dll" "genann_init_sigmoid_lookup" "void" "void*"))

; double genann_act_sigmoid(const genann *ann, double a)
(setq act_sigmoid (import "genann.dll" "genann_act_sigmoid" "double" "void*" "double"))

; double genann_act_sigmoid_cached(const genann *ann, double a)
(setq act_sigmoid_cached (import "genann.dll" "genann_act_sigmoid_cached" "double" "void*" "double"))

; double genann_act_threshold(const genann *ann, double a)
(setq act_threshold (import "genann.dll" "genann_act_threshold" "double" "void*" "double"))

; double genann_act_linear(const genann *ann, double a)
(setq act_linear (import "genann.dll" "genann_act_linear" "double" "void*" "double"))

(println "Library initialized successfully.")

; EOF

Here is the original example2 C source (for background) :


Code Select

#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include "genann.h"

int main(int argc, char *argv[])
{
    printf("GENANN example 2.n");
    printf("Train a small ANN to the XOR function using random search.n");

    /* Input and expected out data for the XOR function. */
    const double input[4][2] = {{0, 0}, {0, 1}, {1, 0}, {1, 1}};
    const double output[4] = {0, 1, 1, 0};
    int i;

    /* New network with 2 inputs,
     * 1 hidden layer of 2 neurons,
     * and 1 output. */
    genann *ann = genann_init(2, 1, 2, 1);

    double err;
    double last_err = 1000;
    int count = 0;

    do {
        ++count;
        if (count % 1000 == 0) {
            /* We're stuck, start over. */
            genann_randomize(ann);
        }

        genann *save = genann_copy(ann);

        /* Take a random guess at the ANN weights. */
        for (i = 0; i < ann->total_weights; ++i) {
            ann->weight[i] += ((double)rand())/RAND_MAX-0.5;
        }

        /* See how we did. */
        err = 0;
        err += pow(*genann_run(ann, input[0]) - output[0], 2.0);
        err += pow(*genann_run(ann, input[1]) - output[1], 2.0);
        err += pow(*genann_run(ann, input[2]) - output[2], 2.0);
        err += pow(*genann_run(ann, input[3]) - output[3], 2.0);

        /* Keep these weights if they're an improvement. */
        if (err < last_err) {
            genann_free(save);
            last_err = err;
        } else {
            genann_free(ann);
            ann = save;
        }

    } while (err > 0.01);

    printf("Finished in %d loops.n", count);

    /* Run the network and see what it predicts. */
    printf("Output for [%1.f, %1.f] is %1.f.n", input[0][0], input[0][1], *genann_run(ann, input[0]));
    printf("Output for [%1.f, %1.f] is %1.f.n", input[1][0], input[1][1], *genann_run(ann, input[1]));
    printf("Output for [%1.f, %1.f] is %1.f.n", input[2][0], input[2][1], *genann_run(ann, input[2]));
    printf("Output for [%1.f, %1.f] is %1.f.n", input[3][0], input[3][1], *genann_run(ann, input[3]));

    genann_free(ann);
    return 0;
}

Finally, here is my attempt at a rewrite of this example2 in newLISP :


Code Select

; Load genann wrapper
(load "genann.lsp")

;;; main

(println "GENANN example 2.")
(println "Train a small ANN to the XOR function using random search.")

(constant 'FLOAT-LENGTH 8)

; Input and expected out data for the XOR function.
(define input  '((0 0) (0 1) (1 0) (1 1)))
(define output '(0 1 1 0))

; Initialize a new neural network with 2 inputs, 1 hidden layer of 2 neurons and 1 output.
(setq ann (genann:init 2 1 2 1))

; Unpack the ann into a struct
(setq ann-contents (unpack genann:network ann))

; Extract the ann structure
(context genann)
(setq inputs         (MAIN:ann-contents 0))
(setq hidden_layers (MAIN:ann-contents 1))
(setq hidden_neurons (MAIN:ann-contents 2))
(setq outputs           (MAIN:ann-contents 3))
(setq activation_hidden_func (MAIN:ann-contents 4))
(setq activation_output_func (MAIN:ann-contents 5))
(setq total_weights (MAIN:ann-contents 6))  
(setq total_neurons (MAIN:ann-contents 7))
(setq weight-ptr (MAIN:ann-contents 8))
(setq output-ptr (MAIN:ann-contents 9))
(setq delta-ptr (MAIN:ann-contents 10))
(context MAIN)

(setq err 0)
(setq last-err 1000)
(setq counter 0)

(do-while (< counter 3)

  (inc counter)
  (if (= (mod counter 1000) 0) ; randomize ANN every 1000 iteration.
    ; We're stuck, start over.
    (genann:randomize-ann ann))

  ; make a backup copy of the ANN for later retrieval in case our error increases.
  (setq backup-network (genann:copy-ann ann))

(println "ann addr   : " (string ann))
(println "weight addr: " (string genann:weight-ptr)) ; weight address is ALWAYS the same!
(println "bak addr   : " (string backup-network))

  ; Take a random guess at the ANN weights.
  (for (i 0 (- genann:total_weights 1)) ; step 1 by default
    (letn (src-addr     (+ genann:weight-ptr (* i FLOAT-LENGTH))
           existing   (get-float src-addr)
    rand-num     (random -0.5 1) ; (sub (div (rand genann:RAND_MAX) genann:RAND_MAX) 0.5) ;
    new-value    (add existing rand-num)
    packed-value (pack "lf" new-value))
      (cpymem packed-value src-addr FLOAT-LENGTH)
      (println "iteration #" i ", src-addr: " src-addr ", existing: " existing ", random number: " rand-num ", new: " new-value)
    ))                           ; using the 64 bits double size = 8 bytes

  ; See how we did
  (setq err 0)
  (for (input-num 0 3)
  (letn (input-pair (input input-num)
packed-input (pack "lf lf" input-pair)
          result (get-float (genann:run ann packed-input))
  delta (sub result (output input-num))
  powered (pow delta 2.0))
 (begin (setq err (add err powered))
            (println "input #" input-num ", input: " input-pair ", output: " result ", delta: " delta ", powered: " powered ", err: " err))) )
 
  (println "Error: " err ". Last Error: " last-err ". ")

  ; Keep these weights if they're an improvement.
  (if (< err last-err) ; if error is lower than previous,
      (begin (println "err < last-err. Freeing backup and saving last error.")
(genann:free backup-network) ; clear backup
             (setq last-err err)) ; save as new last error
      (begin (println "err > last-err. Freeing ann and restoring backup ann.")
(genann:free ann) ; else, clear current ann
(setq ann backup-network))) ; restore backup.             ; (set 'ann (genann:copy-ann backup-network))))

)

; Free memory
(genann:free ann)
(genann:free backup-network)

;(exit)

; EOF

  As you can see I limited the loop in order to examine results. The code works great, apart from the fact that the GENANN run function always returns the same kind of output no matter how I randomize weights. If anybody has experience with C and newLISP, I would really appreciate their insights. If you see anything obvious please let me know.



Dexter
#3
newLISP in the real world / Re: DLL import failing
August 21, 2018, 06:46:40 AM
I figured it out. Lutz pointed out I was using the 32 bits version of MingW, with the 64 bits version of newLISP. All I had to do is :



1) install MingW64

2) compile the genann source with:

    gcc -m64 genann.c -shared -o genann.dll



Cheers,



Dexter
#4
newLISP in the real world / DLL import failing
July 16, 2018, 07:14:14 AM
Hi folks,



  First post ever. I'm attempting to compile the Genann neural network library into a DLL for import into newLISP.



https://github.com/codeplea/genann">https://github.com/codeplea/genann



  I updated the C source and its header to comply with either STDCALL or CDECL conventions (tried both with no success). I tried several methods to output a DLL out of the C source, which all work fine except when importing in newLISP. The methods I tried are described here :



https://www.transmissionzero.co.uk/computing/building-dlls-with-mingw/">https://www.transmissionzero.co.uk/comp ... ith-mingw/">https://www.transmissionzero.co.uk/computing/building-dlls-with-mingw/



  I checked the exports using Dependency Walker, and all functions are exported correctly (at least when using CDECL, somehow the STDCALL one still showed up as a C export.

No matter what I do newLISP returns the usual import error when attempting to import the DLL :



ERR: problem loading library in function import : "genann.dll"



My import line :



(import "genann.dll" "genann_init")     (when stdcall)

or

(import "genann.dll" "genann_init" "cdecl")



  I tried using the absolute path (with slashes) to the DLL, as well as both calling conventions, to no avail.

I tried compiling the DLL both with gcc straight to DLL (-shared etc.) and through mingw's dllwrapper, to the same result. The DLL file I'm getting seems legit, and I have no problem importing it from C.



Has anybody encountered this problem?



Thanks in advance!



Dexter
Pages1