Why is the behavior of "trim" function so strange?

Started by psilwen, August 15, 2017, 05:26:07 AM

Previous topic - Next topic
Print
Go Down Pages1
User actions

psilwen

August 15, 2017, 05:26:07 AM Last Edit: December 31, 1969, 04:00:00 PM by Guest
Code Select
newLISP v.10.7.1 32-bit on Windows IPv4/6 UTF-8 libffi, options: newlisp -h

> (setq str (dup "00" 10))
"00000000000000000000"
> (trim str)
""
> (setq str (dup "00" 31))
"000000000000000000000000000000000000000
00000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 32))
"000000000000000000000000000000000000000
0000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
(reverse \"newlisp\")

varbanov

#1
August 18, 2017, 06:33:52 AM Last Edit: December 31, 1969, 04:00:00 PM by Guest
Hi,



I'm still on a previous version and there's no problem ...
Code Select

newLISP v.10.6.2 32-bit on Win32 IPv4/6 UTF-8 libffi, options: newlisp -h

> (setq str (dup "00" 32))
"0000000000000000000000000000000000000000000000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 52))
"00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
> (trim str)
""
>


Yours,

s.v.

CaveGuy

#2
August 18, 2017, 07:01:54 AM Last Edit: December 31, 1969, 04:00:00 PM by Guest
I am back from the future :) no problem there ?


Code Select
newLISP v.10.7.3 64-bit on Windows IPv4/6 libffi, options: newlisp -h

> (setq str (dup "00" 52))
"00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
> (trim str)
""
>
Bob the Caveguy aka Lord High Fixer.

psilwen

#3
August 18, 2017, 11:03:24 AM Last Edit: December 31, 1969, 04:00:00 PM by Guest
Code Select
newLISP v.10.7.3 32-bit on Windows IPv4/6 libffi, options: newlisp -h

> (setq str (dup "00" 8))
"0000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 1))
"00"
> (trim str)
""
> (setq str (dup "00" 2))
"0000"
> (trim str)
""
> (setq str (dup "00" 3))
"000000"
> (trim str)
""
> (setq str (dup "00" 4))
"00000000"
> (trim str)
""
> (setq str (dup "00" 5))
"0000000000"
> (trim str)
""
> (setq str (dup "00" 6))
"000000000000"
> (trim str)
""
> (setq str (dup "00" 7))
"00000000000000"
> (trim str)
""
> (setq str (dup "00" 8))
"0000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 9))
"000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 10))
"00000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 11))
"0000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 12))
"000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 13))
"00000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 14))
"0000000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 15))
"000000000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 16))
"00000000000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 17))
"0000000000000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 18))
"000000000000000000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 19))
"00000000000000000000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 20))
"000000000000000000000000000000000000000
0"
> (trim str)
""
> (setq str (dup "00" 21))
"000000000000000000000000000000000000000
000"
> (trim str)
""
> (setq str (dup "00" 22))
"000000000000000000000000000000000000000
00000"
> (trim str)
""
> (setq str (dup "00" 23))
"000000000000000000000000000000000000000
0000000"
> (trim str)
""
> (setq str (dup "00" 24))
"000000000000000000000000000000000000000
000000000"
> (trim str)
""
> (setq str (dup "00" 25))
"000000000000000000000000000000000000000
00000000000"
> (trim str)
""
> (setq str (dup "00" 26))
"000000000000000000000000000000000000000
0000000000000"
> (trim str)
""
> (setq str (dup "00" 27))
"000000000000000000000000000000000000000
000000000000000"
> (trim str)
""
> (setq str (dup "00" 28))
"000000000000000000000000000000000000000
00000000000000000"
> (trim str)
""
> (setq str (dup "00" 29))
"000000000000000000000000000000000000000
0000000000000000000"
> (trim str)
""
> (setq str (dup "00" 30))
"000000000000000000000000000000000000000
000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 31))
"000000000000000000000000000000000000000
00000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 32))
"000000000000000000000000000000000000000
0000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 33))
"000000000000000000000000000000000000000
000000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim





Code Select
newLISP v.10.7.3 64-bit on Windows IPv4/6 libffi, options: newlisp -h

> (setq str (dup "00" 8))
"0000000000000000"
> (trim str)
""
> (setq str (dup "00" 1))
"00"
> (trim str)
""
> (setq str (dup "00" 2))
"0000"
> (trim str)
""
> (setq str (dup "00" 3))
"000000"
> (trim str)
""
> (setq str (dup "00" 4))
"00000000"
> (trim str)
""
> (setq str (dup "00" 5))
"0000000000"
> (trim str)
""
> (setq str (dup "00" 6))
"000000000000"
> (trim str)
""
> (setq str (dup "00" 7))
"00000000000000"
> (trim str)
""
> (setq str (dup "00" 8))
"0000000000000000"
> (trim str)
""
> (setq str (dup "00" 9))
"000000000000000000"
> (trim str)
""
> (setq str (dup "00" 10))
"00000000000000000000"
> (trim str)
""
> (setq str (dup "00" 11))
"0000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 12))
"000000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 13))
"00000000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 14))
"0000000000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 15))
"000000000000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 16))
"00000000000000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 17))
"0000000000000000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 18))
"000000000000000000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 19))
"00000000000000000000000000000000000000"
> (trim str)
""
> (setq str (dup "00" 20))
"000000000000000000000000000000000000000
0"
> (trim str)
""
> (setq str (dup "00" 21))
"000000000000000000000000000000000000000
000"
> (trim str)
""
> (setq str (dup "00" 22))
"000000000000000000000000000000000000000
00000"
> (trim str)
""
> (setq str (dup "00" 23))
"000000000000000000000000000000000000000
0000000"
> (trim str)
""
> (setq str (dup "00" 24))
"000000000000000000000000000000000000000
000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 25))
"000000000000000000000000000000000000000
00000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 26))
"000000000000000000000000000000000000000
0000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 27))
"000000000000000000000000000000000000000
000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 28))
"000000000000000000000000000000000000000
00000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 29))
"000000000000000000000000000000000000000
0000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 30))
"000000000000000000000000000000000000000
000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 31))
"000000000000000000000000000000000000000
00000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 32))
"000000000000000000000000000000000000000
0000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 33))
"000000000000000000000000000000000000000
000000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 34))
"000000000000000000000000000000000000000
00000000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 35))
"000000000000000000000000000000000000000
0000000000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 36))
"000000000000000000000000000000000000000
000000000000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 37))
"000000000000000000000000000000000000000
00000000000000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 38))
"000000000000000000000000000000000000000
0000000000000000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 39))
"000000000000000000000000000000000000000
000000000000000000000000000000000000000"
> (trim str)

ERR: not enough memory in function trim
> (setq str (dup "00" 40))
"000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0"
> (trim str)
""
> (setq str (dup "00" 41))
"000000000000000000000000000000000000000
0000000000000000000000000000000000000000
000"
> (trim str)
""
> (setq str (dup "00" 42))
"000000000000000000000000000000000000000
0000000000000000000000000000000000000000
00000"
> (trim str)
""
(reverse \"newlisp\")

psilwen

#4
August 18, 2017, 11:09:01 AM Last Edit: December 31, 1969, 04:00:00 PM by Guest
newLISP v.10.7.3

newLISP v.10.7.1

newLISP v.10.7.0

All have this issues.



Usually starting from 8, 16, 24, 32...., depending on the version.



But newLISP v.10.6.2，I tested from 1 to 65536, everything was fine.
(reverse \"newlisp\")

HPW

#5
August 18, 2017, 11:40:14 PM Last Edit: December 31, 1969, 04:00:00 PM by Guest
Hello,



trim seems not to like the null-character.

So is it valid whitespace character?


Code Select

newLISP v.10.7.2 32-bit on Windows IPv4/6 libffi, options: newlisp -h
> (setq str "t00   t")
"t00   t"
> (trim str)

ERR: not enough memory in function trim


Code Select

> (setq str "tabc00   abct")
"tabc00   abct"
> (trim str)
"abc00   abc"
>

Regards
Hans-Peter

Lutz

#6
August 19, 2017, 09:42:25 AM Last Edit: December 31, 1969, 04:00:00 PM by Guest
'trim' was overrunning memory when all characters in string < 32 (space).



Fixed here:

http://newlisp.nfshost.com/downloads/development/inprogress/">http://newlisp.nfshost.com/downloads/de ... nprogress/">http://newlisp.nfshost.com/downloads/development/inprogress/

HPW

#7
August 19, 2017, 10:41:27 PM Last Edit: December 31, 1969, 04:00:00 PM by Guest
Hello Lutz,



Thanks for the fix.

Will characters <32 now handled as whitespace and get trimmed?

Or do they stay?



Regards
Hans-Peter

Lutz

#8
August 21, 2017, 01:01:19 PM Last Edit: December 31, 1969, 04:00:00 PM by Guest
See last paragraph here:

http://www.newlisp.org/downloads/development/inprogress/CHANGES-10.7.3.txt">http://www.newlisp.org/downloads/develo ... 10.7.3.txt">http://www.newlisp.org/downloads/development/inprogress/CHANGES-10.7.3.txt



So the 10.7.3 now is more forgiving when a string contains zero characters, which it should'nt in perfect ASCII or UTF-8 where the zero character marks the end of the string.

psilwen

#9
September 12, 2017, 10:55:10 PM Last Edit: December 31, 1969, 04:00:00 PM by Guest
(reverse \"newlisp\")

TedWalther

#10
September 13, 2017, 01:20:36 AM Last Edit: December 31, 1969, 04:00:00 PM by Guest
It occurs to me, if I was trimming a string, and it had 00 characters on either the left or the right side, I'd want them GONE.  If it is a binary string where the 00 character belongs, then I wouldn't be running trim on it, since trim is a textual function.
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence.  Nine months later, they left with a baby named newLISP.  The women of the ivory towers wept and wailed.  \"Abomination!\" they cried.

rrq

#11
September 13, 2017, 02:24:32 AM Last Edit: December 31, 1969, 04:00:00 PM by Guest
Technically, since neither "xbbxe1" nor "xcexaaxcaxb2xc3xb4xbbxe1" are valid UTF-8 strings, the trim behaviour is conveniently undefined. It appears the trim function expands "xbb" into a 2-byte code, and "xe1" into a three byte code, making its output valid UTF-8. ("xcexaa", "xcaxb2" and "xc3xb4" are valid UTF-8 characters).
Print
Go Up Pages1
User actions