regex doesn't match utf-8 character using \w pattern

Started by vetelko, January 27, 2017, 12:37:34 PM

Previous topic - Next topic

vetelko

Hello newLISP-ers :)



the following code returns nil for utf-8 character

(regex {w} "č")

and so the following

(regex {w} "č" "u")

while this works

(regex {w} "m")



Is this not implemented or am I doing something wrong?

VT
newLISP v.10.7.6 64-bit on BSD IPv4/6 UTF-8 libffi

rrq

#1
I believe the magic of regex is explained in http://www.newlisp.org/downloads/pcrepattern.html">http://www.newlisp.org/downloads/pcrepattern.html, where you find the two paragraphs:

------

 A "word" character is an underscore or any character less than 256 that is a letter or digit. The definition of letters and digits is controlled by PCRE's low-valued character tables, and may vary if locale-specific matching is taking place (see "Locale support" in the pcreapi page). For example, in the "fr_FR" (French) locale, some character codes greater than 128 are used for accented letters, and these are matched by w.



In UTF-8 mode, characters with values greater than 128 never match d, s, or w, and always match D, S, and W. This is true even when Unicode character property support is available. The use of locales with Unicode is discouraged.

------

vetelko

#2
So there is no way how to match this character as a word character using pattern?
newLISP v.10.7.6 64-bit on BSD IPv4/6 UTF-8 libffi