欢迎各位兄弟 发布技术文章
这里的技术是共享的
Constant CTOOLS_PREG_CLASS_ALNUM in includes/cleanstring.inc contains \x{d800}- which is ill-formed code point. PHP issues this warning:
Warning: preg_match(): Compilation failed: disallowed Unicode code point (>= 0xd800 && <= 0xdfff) at offset 1811 in ctools_cleanstring() (line 157 of /srv/http/XXXX/www/sites/all/modules/contrib/ctools/includes/cleanstring.inc).
Easily reproducable by running this:
define('CTOOLS_PREG_CLASS_ALNUM',
'\x{0}-\x{2f}\x{3a}-\x{40}\x{5b}-\x{60}\x{7b}-\x{bf}\x{d7}\x{f7}\x{2b0}-' .
'\x{385}\x{387}\x{3f6}\x{482}-\x{489}\x{559}-\x{55f}\x{589}-\x{5c7}\x{5f3}-' .
'\x{61f}\x{640}\x{64b}-\x{65e}\x{66a}-\x{66d}\x{670}\x{6d4}\x{6d6}-\x{6ed}' .
'\x{6fd}\x{6fe}\x{700}-\x{70f}\x{711}\x{730}-\x{74a}\x{7a6}-\x{7b0}\x{901}-' .
'\x{903}\x{93c}\x{93e}-\x{94d}\x{951}-\x{954}\x{962}-\x{965}\x{970}\x{981}-' .
'\x{983}\x{9bc}\x{9be}-\x{9cd}\x{9d7}\x{9e2}\x{9e3}\x{9f2}-\x{a03}\x{a3c}-' .
'\x{a4d}\x{a70}\x{a71}\x{a81}-\x{a83}\x{abc}\x{abe}-\x{acd}\x{ae2}\x{ae3}' .
'\x{af1}-\x{b03}\x{b3c}\x{b3e}-\x{b57}\x{b70}\x{b82}\x{bbe}-\x{bd7}\x{bf0}-' .
'\x{c03}\x{c3e}-\x{c56}\x{c82}\x{c83}\x{cbc}\x{cbe}-\x{cd6}\x{d02}\x{d03}' .
'\x{d3e}-\x{d57}\x{d82}\x{d83}\x{dca}-\x{df4}\x{e31}\x{e34}-\x{e3f}\x{e46}-' .
'\x{e4f}\x{e5a}\x{e5b}\x{eb1}\x{eb4}-\x{ebc}\x{ec6}-\x{ecd}\x{f01}-\x{f1f}' .
'\x{f2a}-\x{f3f}\x{f71}-\x{f87}\x{f90}-\x{fd1}\x{102c}-\x{1039}\x{104a}-' .
'\x{104f}\x{1056}-\x{1059}\x{10fb}\x{10fc}\x{135f}-\x{137c}\x{1390}-\x{1399}' .
'\x{166d}\x{166e}\x{1680}\x{169b}\x{169c}\x{16eb}-\x{16f0}\x{1712}-\x{1714}' .
'\x{1732}-\x{1736}\x{1752}\x{1753}\x{1772}\x{1773}\x{17b4}-\x{17db}\x{17dd}' .
'\x{17f0}-\x{180e}\x{1843}\x{18a9}\x{1920}-\x{1945}\x{19b0}-\x{19c0}\x{19c8}' .
'\x{19c9}\x{19de}-\x{19ff}\x{1a17}-\x{1a1f}\x{1d2c}-\x{1d61}\x{1d78}\x{1d9b}-' .
'\x{1dc3}\x{1fbd}\x{1fbf}-\x{1fc1}\x{1fcd}-\x{1fcf}\x{1fdd}-\x{1fdf}\x{1fed}-' .
'\x{1fef}\x{1ffd}-\x{2070}\x{2074}-\x{207e}\x{2080}-\x{2101}\x{2103}-\x{2106}' .
'\x{2108}\x{2109}\x{2114}\x{2116}-\x{2118}\x{211e}-\x{2123}\x{2125}\x{2127}' .
'\x{2129}\x{212e}\x{2132}\x{213a}\x{213b}\x{2140}-\x{2144}\x{214a}-\x{2b13}' .
'\x{2ce5}-\x{2cff}\x{2d6f}\x{2e00}-\x{3005}\x{3007}-\x{303b}\x{303d}-\x{303f}' .
'\x{3099}-\x{309e}\x{30a0}\x{30fb}-\x{30fe}\x{3190}-\x{319f}\x{31c0}-\x{31cf}' .
'\x{3200}-\x{33ff}\x{4dc0}-\x{4dff}\x{a015}\x{a490}-\x{a716}\x{a802}\x{a806}' .
'\x{a80b}\x{a823}-\x{a82b}\x{d800}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}' .
'\x{fdfc}-\x{fe6b}\x{feff}-\x{ff0f}\x{ff1a}-\x{ff20}\x{ff3b}-\x{ff40}\x{ff5b}-' .
'\x{ff65}\x{ff70}\x{ff9e}\x{ff9f}\x{ffe0}-\x{fffd}');
if (preg_match('/^[^' . CTOOLS_PREG_CLASS_ALNUM . ']+$/uD', '-')) {
echo 'Warning issued';
}
Patch to be posted in comment.
Comment | File | Size | Author |
---|---|---|---|
#37 | 2.49 KB | generalconsensus | |
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch ctools-1444006-disallowed-unicode-D6-37.patch. Unable to apply patch. See the log in the details link for more information. View
| |||
#18 | 1.24 KB | gdl | |
PASSED: [[SimpleTest]]: [MySQL] 0 pass(es). View
| |||
#7 | 2.35 KB | kotnik | |
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch ctools-1444006-7-bad-unicode_0.patch. Unable to apply patch. See the log in the details link for more information. View
| |||
#6 | 726 bytes | Marc Angles | |
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch remove-annoying-warnings-about-unicode-D6.patch. Unable to apply patch. See the log in the details link for more information. View
| |||
#1 | 1.21 KB | kotnik | |
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch ctools-1444006-1-removed-bad-unicode.patch. Unable to apply patch. See the log in the details link for more information. View
|
Comments
Comment#1
kotnik CreditAttribution: kotnik commentedPatch with fix attached.
Log in or register to post comments
Comment#2
merlinofchaos CreditAttribution: merlinofchaos commentedNice! And a properly formatted commit message which is rare! Thank you.
Log in or register to post comments
Comment#3
activelink CreditAttribution: activelink commentedThis bug is likely caused by pcre 8.30+, most Linuxes still use older versions, but FreeBSD has 8.30_1 currently. But this version of pcre will be on Linux soon - it's developed on FreeBSD.
It breaks core search too in D6/7.
Ctools has it because it has code in cleanstring.inc that is copied from includes/unicode.inc (D7 core) or modules/search/search.module in D6.
It's examined in some depth here:
Invalid Unicode code range in PREG_CLASS_UNICODE_WORD_BOUNDARY fails with PCRE 8.30
There are patches for D6 and D7, and these work also if adapted for ctools' unicode.inc
These patches are slightly different to the one above. They replace the D800 parameter with E800 and this works. The patch above removes it entirely. I don't understand this code or what it does, but it seems like the patches that I'm referencing are more evolved than the one given on this page (which removes this value rather than replacing it). Maybe someone who understands its function might want to look at it?
Log in or register to post comments
Comment#4
kotnik CreditAttribution: kotnik commentedYes, this is just the same issue as one described in #1446372: Invalid Unicode code range in PREG_CLASS_UNICODE_WORD_BOUNDARY fails with PCRE 8.30 (pcre > 8.30, observable in FreeBSD and Arch, probably Gentoo too).
As a matter of fact, CTOOLS_PREG_CLASS_ALNUM (changed in patch from #1) is copied from search.module's PREG_CLASS_SEARCH_EXCLUDE.
Removing d800 is quite safe, since CTOOLS_PREG_CLASS_ALNUM is used to escape separator if it's not alphanumeric.
Log in or register to post comments
Comment#6
Marc Angles CreditAttribution: Marc Angles commentedHi, I really don't know if this patch is right, but for the moment there is no more messages on my websites (This is D6.25)
Log in or register to post comments
Comment#7
kotnik CreditAttribution: kotnik commentedUpdated to be in sync with #1446372: Invalid Unicode code range in PREG_CLASS_UNICODE_WORD_BOUNDARY fails with PCRE 8.30.
As noted in 1446372's comment #16 it's better to scan Unicode private use instead of surrogate area, then to ignore it.
This code has been merged in Drupal core 8.x and 7.x (patch waiting to be ported to 6.x), and I think ctools should do the same.
PS. @Marc Angles: yes, that's the same fix that'll be included in next 6.x core.
Log in or register to post comments
Comment#8
lucascaro CreditAttribution: lucascaro commentedI've just manually replaced
with
'\x{a80b}\x{a823}-\x{a82b}\x{e000}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
as in #7 and it fixed the error messages in 6.x-1.8+50-dev
Since the first half of the patch seems to be already committed, that last change is all that is needed to remove the message (at least using my configuration).
RTBC, anyone?
Log in or register to post comments
Comment#9
kotnik CreditAttribution: kotnik commentedThe exact same fix as in #7 has already been commited in D6, D7, and D8. Marking it as RTBC.
Log in or register to post comments
Comment#10
lucascaro CreditAttribution: lucascaro commentedare they committed to dev? awesome! also, maybe this should be marked as fixed?
thanks!
Log in or register to post comments
Comment#11
kotnik CreditAttribution: kotnik commented@lucascaro:
No, patch from #7 needs to be commited to ctools, since the exact same fix was applied to Drupal 6, 7 and 8.
Log in or register to post comments
Comment#12
lucascaro CreditAttribution: lucascaro commentedooh, nice. This is starting to trigger error messages in all of my projects, so I'd love it to be committed. How can I help making it happen?
Log in or register to post comments
Comment#13
tim.plunkett CreditAttribution: tim.plunkett commentedFor tonight.
Log in or register to post comments
Comment#14
lucascaro CreditAttribution: lucascaro commentedawesome!
Log in or register to post comments
Comment#15
tim.plunkett CreditAttribution: tim.plunkett commentedOh wow the format-patch threw me off there. This is much more straightforward then I thought.
Also, why'd you unassign me? I almost couldn't find the issue.
http://drupalcode.org/project/ctools.git/commit/a1c5c9b
Log in or register to post comments
Comment#16
lucascaro CreditAttribution: lucascaro commentedog god, sorry about that. I don't know why that happened, I didn't touch the assigned box. :S
Log in or register to post comments
Comment#18
gdl CreditAttribution: gdl commentedSorry to reopen, but it looks to me like this didn't make it into 6.x-1.x-dev.
Attached is a patch for 6.x-1.x-dev made by copying PREG_CLASS_SEARCH_EXCLUDE from search.module from Drupal 6.26. Works for me, but not thoroughly tested.
Best,
-G
Log in or register to post comments
Comment#19
hansfn CreditAttribution: hansfn commentedI agree that this seems to be unfixed still.
@gdl: Why the difference between your patch and the one in #1697538: Unicode compilation fails because of invalid code range in CTOOLS_PREG_CLASS_ALNUM?
Log in or register to post comments
Comment#20
gdl CreditAttribution: gdl commentedIt looks like the patch in #1697538 changes just a single code point, while mine changes the list of code points in CTools to match the code points from the core search module. I guess it's up the the developers to determine which approach to take. I don't have an informed opinion about which is better - I just want to see the issue resolved!
Cheers,
-G
Log in or register to post comments
Comment#21
jhedstrom CreditAttribution: jhedstrom commentedFix in #18 gets rid of the warnings in 6.x-1.x.
Log in or register to post comments
Comment#22
carnau CreditAttribution: carnau commentedThe #18 patch works here, with pcre package version 7.8 and ctools 6.x-1.10+0-dev.
Log in or register to post comments
Comment#23
EugenMayer CreditAttribution: EugenMayer commentedconfirmin, patch from #18 fixes the issues you get when using PHP > 5.3.18 . Applying did not succeed though. Thanks!
*removed since it up there already*
Log in or register to post comments
Comment#24
trotskyicepick CreditAttribution: trotskyicepick commentedJust upgraded to 6.x-1.10, and found the error from the OP was again occurring, #8 again solves the problem.
Why is this not changed in the module core?
Log in or register to post comments
Comment#25
neclimdul CreditAttribution: neclimdul commentedThanks guys. Tested and applied #1697538-2: Unicode compilation fails because of invalid code range in CTOOLS_PREG_CLASS_ALNUM since it brings the regex into a state matching the 7.x-1.x branch it seemed more reasonable. If we need the extra change here can you reroll with an explanation?
Log in or register to post comments
Comment#26
beto_beto CreditAttribution: beto_beto commentedhello
i am not professional in Drupal core settings
and this error appear for me too
warning: preg_match(): Compilation failed: disallowed Unicode code point (>= 0xd800 && <= 0xdfff) at offset 1811 in C:\inetpub\vhosts\xxxxx\httpdocs\sites\all\modules\ctools\includes\cleanstring.inc on line 157.
how can i fix this problem pleas ????
Log in or register to post comments
Comment#27
beto_beto CreditAttribution: beto_beto commented#8 Work for me :D
I've just manually replaced
'\x{a80b}\x{a823}-\x{a82b}\x{d800}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
with
'\x{a80b}\x{a823}-\x{a82b}\x{e000}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
Log in or register to post comments
Comment#28
crutch CreditAttribution: crutch commentedthis is happening more often after I search and search, then post, I find the issue when searching google on drupal. I could not find this issue in Drupal search.
possible duplicate: http://drupal.org/node/1878284
This seems still unfixed in the module.
Log in or register to post comments
Comment#29
aitona CreditAttribution: aitona commentedI solved it doing this. I TRUNCATE the following tables: cache_page, acceslog, cache_menu and cache_filter.
I think that commen #27 don't work for me, but i set the line like beto_beto say:
'\x{a80b}\x{a823}-\x{a82b}\x{e000}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
Log in or register to post comments
Comment#30
gnuget CreditAttribution: gnuget commentedI've the same bug and also the patch from #18 fixed it.
Log in or register to post comments
Comment#31
arsnova CreditAttribution: arsnova commentedI just ran into this problem for this first time today (6.x-1.10).
#8 solved the problem.
Log in or register to post comments
Comment#32
bmcginnis CreditAttribution: bmcginnis commentedI'm on 6.x-1.10, running PHP 5.3.21, and running Drupal 6.28. Anyone know if #18 or #8 is the better route to take?
Log in or register to post comments
Comment#33
bmcginnis CreditAttribution: bmcginnis commentedI've used the approach from #8 and that's solved the problem for me.
Log in or register to post comments
Comment#34
Grabby CreditAttribution: Grabby commentedI was getting
warning: preg_match() [function.preg-match]: Compilation failed: disallowed Unicode code point (>= 0xd800 && <= 0xdfff) at offset 1811 in /var/www/mysite/public_html/sites/all/modules/ctools/includes/cleanstring.inc on line 157
a number of times after upgrading my server from
PHP 5.2.17-0.dotdeb.0
MySQL 5.1.56
to
PHP 5.3.21-1~dotdeb.0
MySQL 5.1.66
on a Drupal 6.28 site running cTools 6.x-1.10, but #8 solved the problem. Not sure why this hasn’t been committed!
Log in or register to post comments
Comment#35
kmare CreditAttribution: kmare commentedSame as Grabby for me, #8 fixed the problem on my drupal6 installations. I really believe it should be commited.
Log in or register to post comments
Comment#36
jacquelynfisher CreditAttribution: jacquelynfisher commentedThanks! I had the same issue as #34: the error appeared after our servers were upgraded to PHP5.3.x. The simple fix in #8 resolved the problem.
Log in or register to post comments
Comment#37
generalconsensus CreditAttribution: generalconsensus commentedThe fix from #8 works but doesn't affect search.module. This patch incorporates #8 with a fix for the search module using the same code.
Works for me on:
FREEBSD
Drupal 6.16
PHP 5.3.19
APACHE 2.2.23
Log in or register to post comments
Comment#38
dadderley CreditAttribution: dadderley commentedAnother vote for solution in #8
Log in or register to post comments
Comment#39
laken CreditAttribution: laken commentedConfirm that patch in #18 worked - CTools 6.x-1.10, Drupal 6.28, PHP 5.3.19, Debian 6.0.6
Log in or register to post comments
Comment#40
jonathanhuot CreditAttribution: jonathanhuot commented#18 worked very well for my installation :
CTools 6.x-1.10
PHP 5.3.23
Drupal 6.28
I updated to "tested by the community" because it's enough trivial to include it into git.
Log in or register to post comments
Comment#42
merlinofchaos CreditAttribution: merlinofchaos commentedAs per #25, this was marked fixed; neclimdul committed the patch. Why are there 17 comments after that asking for this to be committed?
The reason it failed testing is the patch, which is already committed, doesn't apply.
Do people actually read issues before they start in? Because it seems like a giant waste of time to have so many people asking why a fix, which is committed, hasn't been committed. Because it's been committed.
Log in or register to post comments
Comment#43
hansfn CreditAttribution: hansfn commentedThere are 17 comments because, as I said already in comment 19, the fix was only committed to the 7.x-1.x branch, not the 6.x-1.x branch. Hence, the bug is still present on 6.x-1.10... This is clearly frustrating for people using Drupal 6. (The fix is almost one year old.)
OK, merlin, you are right. The fix was applied to the 6.x-1.x branch - ref this commit. So the only problem is that there hasn't been a new release of Ctools since 6.x-1.10.
Log in or register to post comments
Comment#44
hansfn CreditAttribution: hansfn commentedUpdating version info since this isn't fixed for 6.x-1.10, only in 6.x-1.x-dev.
Log in or register to post comments
Comment#45
hansfn CreditAttribution: hansfn commentedSorry about the noise. The bug, which is fixed, was in version 6.x-1.10. Some times I'm just that stupid.
Log in or register to post comments
Comment#46
Anybody CreditAttribution: Anybody commentedThis is a quite important issue. Wouldn't it make sense to release a new stable version 2 month after that? We all have to switch to .dev else.
Log in or register to post comments
Comment#47
KarlShea CreditAttribution: KarlShea commentedI have 6.x-1.10 installed and the commit you reference that fixes the bug has not been applied.
cleanstring.inc line 61 is:
'\x{a80b}\x{a823}-\x{a82b}\x{d800}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
d800 should be e000
Log in or register to post comments
Comment#48
GiorgosK CreditAttribution: GiorgosK commentedHad same problem with 6.x-1.10
#18 patch solves this problem
#37 includes a patch for core search module that should be posted to core queue
Log in or register to post comments
Comment#49
merlinofchaos CreditAttribution: merlinofchaos commentedAs per #42, this was committed. It was committed after 6.x-1.10 was released. Please pay attention when changing issue statuses.
And I'm sorry there hasn't been a new release, but releases for Drupal 6 are not high on my priority list.
Log in or register to post comments
Comment#50
KarlShea CreditAttribution: KarlShea commentedUh, I was paying attention. #45 indicates that it was in 6.x-1.10. Which it isn't.
Log in or register to post comments
Comment#51
ifrik CreditAttribution: ifrik commentedpatch #18 worked for me
Log in or register to post comments
Comment#52
darshanchoudhary CreditAttribution: darshanchoudhary commentedpatch #18 worked for me too.. :)
Log in or register to post comments
Comment#52.0
darshanchoudhary CreditAttribution: darshanchoudhary commented:)
Log in or register to post comments
Comment#53
beto_beto CreditAttribution: beto_beto commented#8 Work for ME
this error was appear to me
Warning: preg_match(): Compilation failed: disallowed Unicode code point (>= 0xd800 && <= 0xdfff) at offset 1811 in ctools_cleanstring() (line 157 of /srv/http/XXXX/www/sites/all/modules/contrib/ctools/includes/cleanstring.inc).
Solution
on
sites/all/modules/contrib/ctools/includes/cleanstring.inc
I've just manually replaced
'\x{a80b}\x{a823}-\x{a82b}\x{d800}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
with
'\x{a80b}\x{a823}-\x{a82b}\x{e000}-\x{f8ff}\x{fb1e}\x{fb29}\x{fd3e}\x{fd3f}'.
Thanks
Log in or register to post comments
Comment#54
Sylvain Lasnier CreditAttribution: Sylvain Lasnier commentedme too
Log in or register to post comments
Comment#55
jasonabc CreditAttribution: jasonabc commentedthanks @gdl (comment #18) for the D6 patch. Works great.
Log in or register to post comments
Comment#56
gpk CreditAttribution: gpk commented@50 and for general clarification : #45 indicates that the bug was in 6.x-1.10 (which was released Nov 14, 2012) and since the fix was committed on Nov 29, 2012 it only appears in subsequent releases, i.e. 6.x-1.11 released Jan 12, 2014 and any later releases that there may be.
Log in or register to post comments
Comment#57
davidwbarratt CreditAttribution: davidwbarratt commentedI'm receiving the same/similar error on 7.x-1.0-rc1:
Any ideas?
Log in or register to post comments
Comment#59
DamienMcKenna CreditAttribution: DamienMcKenna commented@davidwbarratt: Please open a new issue for the D7 problem if it persists in the latest D7 release.
来自 https://www.drupal.org/node/1444006