gitea
/
Koha-clms
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Bug 17661: (follow-up) Update regex to support Unicode characters 

Rather than limiting initials to [A-Z] we should test for a broad
range of uppercase letters.

The ES/Zebra changes are slightly different because of Perl vs Java regex
conventions. POerl may support either, but I found 'Uppercase' to be a bit more explicit

More info here:
https://perldoc.perl.org/perlunicode.html

TO test:
Same plan as before but use Ж. as the ending initial
Confirm the period is preserved and other punctuation removed

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
(cherry picked from commit 224ac84aec)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit f297fb819c)

Signed-off-by: Aleisha Amohia <aleishaamohia@hotmail.com>
This commit is contained in:
Nick Clemens 2020-08-27 16:27:09 +00:00 committed by Aleisha Amohia
parent 4506ca63e3
commit fdf5fd0fc2
2 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
C4/Search.pm
View File

@ -711,7 +711,7 @@ sub _get_facets_data_from_record {
next if $field->indicator(1) eq 'z';
my $data = $field->as_string( $subfield_letters, $facet->{ sep } );
$data =~ s/\s*(?<![A-Z])[.\-,;]*\s*$//;
$data =~ s/\s*(?<!\p{Uppercase})[.\-,;]*\s*$//;
unless ( grep { /^\Q$data\E$/ } @used_datas ) {
push @used_datas, $data;
@ -810,7 +810,7 @@ sub _get_facet_from_result_set {
my $facets = {};
foreach my $term ( @terms ) {
my $facet_value = $term->textContent;
$facet_value =~ s/\s*(?<![A-Z])[.\-,;]*\s*$//;
$facet_value =~ s/\s*(?<!\p{Uppercase})[.\-,;]*\s*$//;
$facet_value =~ s/\Q$internal_sep\E/$sep/ if defined $sep;
$facets->{ $facet_value } = ( defined $facets->{$facet_value} ) ? $facets->{ $facet_value } + $term->getAttribute( 'occur' ) : $term->getAttribute( 'occur' );
}

2
admin/searchengine/elasticsearch/index_config.yaml
View File

@ -39,6 +39,6 @@ index:
replacement: ''
facet:
type: pattern_replace
pattern: '\s*(?<![A-Z])[.\-,;]*\s*$'
pattern: '\s*(?<!\p{Lu})[.\-,;]*\s*$'
replacement: ''
index.mapping.total_fields.limit: 10000