Bug 9579: fix truncation of facets containing multi-byte characters

We seem to be relying on whatever Zoom::Results->render return, and
Perl doesn't explicitly consider it UNICODE data. That's why CORE::substr
(and probably CORE::length too) cut the bytes wrong.

This patch just decodes the UTF-8 data that render() returns and then
Perl behaves, heh.

It uses Encode::decode_utf8 which is already a dependency for the current
stable Koha releases.

REVISED TEST PLAN
-----------------
1) Import the attached sample records.
2) Rebuild your indexes
3) In OPAC search for يكيمكتبات : قبسي ، كرم
-- There will be ugly diamonds with question marks in the facets
4) apply the patch
5) Search again.
-- The names will be properly truncated.
NOTE: This test assumes FacetLabelTruncationLength = 20.

Sponsored-by: Universidad Nacional de Cordoba

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Works as described, tested with several German, English and
the Arabic test record. Arabic strings now display correctly
and no regression was found.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

I've reviewed it and approve its inclusion in 3.14.x and earlier.  I
will use the patches for bug 11096, once they pass QA, for the master
branch.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This commit is contained in:
Tomas Cohen Arazi 2014-01-15 21:06:04 -03:00 committed by Fridolin Somers
parent 7b68bfd53c
commit 171e2b4746
1 changed files with 3 additions and 2 deletions

View File

@ -36,6 +36,7 @@ use URI::Escape;
use Business::ISBN;
use MARC::Record;
use MARC::Field;
use Encode qw/decode_utf8/;
use utf8;
use vars qw($VERSION @ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $DEBUG);
@ -503,8 +504,8 @@ sub getRecords {
$size > $facets_maxrecs ? $facets_maxrecs : $size;
for my $facet (@$facets) {
for ( my $j = 0 ; $j < $jmax ; $j++ ) {
my $render_record =
$results[ $i - 1 ]->record($j)->render();
my $render_record = decode_utf8(
$results[ $i - 1 ]->record($j)->render());
my @used_datas = ();
foreach my $tag ( @{ $facet->{tags} } ) {