9684fcd5c3
When ElasticsearchMARCFormat is set to ARRAY, the marc_data_array
field uses dynamic:true mapping. Elasticsearch auto-detects field
types based on the first document indexed. If a MARC subfield value
looks like a date (e.g. 2024-01-15 in 952$e), ES locks that
subfield to type date. Subsequent records with non-date values in
the same subfield fail to index with a document_parsing_exception.
This patch sets date_detection and numeric_detection to false at
the mapping root level, so ES treats all dynamically mapped fields
as text instead of guessing types.
Test plan:
1. $ ktd --search-engine es8 --name test-es --proxy up -d
$ ktd --name test-es --wait-ready 180
$ ktd --name test-es --shell
2. k$ koha-mysql kohadev -e "UPDATE systempreferences
SET value='ARRAY'
WHERE variable='ElasticsearchMARCFormat';"
k$ echo 'flush_all' | nc -q1 memcached 11211
k$ koha-mysql kohadev -e "DELETE FROM biblio_metadata;
DELETE FROM items; DELETE FROM biblioitems;
DELETE FROM biblio;"
k$ rebuild_elasticsearch.pl -d -r --biblios
3. Add a record with date-like 952$e and index it:
k$ koha-shell kohadev -p -c 'perl -MC4::Biblio
-MMARC::Record -MMARC::Field -MKoha::Item -e "
my \$r = MARC::Record->new();
\$r->append_fields(
MARC::Field->new(q{245},q{1},q{0},a=>q{Record one}),
MARC::Field->new(q{942},q{ },q{ },c=>q{BK}));
my (\$bn) = C4::Biblio::AddBiblio(\$r, q{});
Koha::Item->new({biblionumber=>\$bn,
biblioitemnumber=>\$bn, homebranch=>q{CPL},
holdingbranch=>q{CPL}, itype=>q{BK},
booksellerid=>q{2024-01-15}})->store;
print qq{Added biblio \$bn\n};"'
k$ rebuild_elasticsearch.pl --biblios -bn <biblionumber>
4. Check the mapping for 952$e:
k$ curl -s es:9200/koha_kohadev_biblios/_mapping/field/\
marc_data_array.fields.952.subfields.e
=> FAIL: type is "date" (should be "text")
5. Add a record with non-date 952$e and index it:
k$ koha-shell kohadev -p -c 'perl -MC4::Biblio
-MMARC::Record -MMARC::Field -MKoha::Item -e "
my \$r = MARC::Record->new();
\$r->append_fields(
MARC::Field->new(q{245},q{1},q{0},a=>q{Record two}),
MARC::Field->new(q{942},q{ },q{ },c=>q{BK}));
my (\$bn) = C4::Biblio::AddBiblio(\$r, q{});
Koha::Item->new({biblionumber=>\$bn,
biblioitemnumber=>\$bn, homebranch=>q{CPL},
holdingbranch=>q{CPL}, itype=>q{BK},
booksellerid=>q{My Vendor Name}})->store;
print qq{Added biblio \$bn\n};"'
k$ rebuild_elasticsearch.pl --biblios -bn <biblionumber> -v -v -v
=> FAIL: document_parsing_exception - failed to parse date field
[My Vendor Name]
6. Apply this patch, repeat from step 2
7. Repeat step 4:
=> SUCCESS: type is "text"
8. Repeat step 5:
=> SUCCESS: No indexing errors
9. Sign off :-D
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
|
||
|---|---|---|
| .github/workflows | ||
| C4 | ||
| Koha | ||
| acqui | ||
| admin | ||
| api | ||
| authorities | ||
| basket | ||
| bin | ||
| bookings | ||
| catalogue | ||
| cataloguing | ||
| circ | ||
| clubs | ||
| course_reserves | ||
| debian | ||
| docs | ||
| erm | ||
| errors | ||
| etc | ||
| ill | ||
| installer | ||
| koha-tmpl | ||
| labels | ||
| lib | ||
| members | ||
| misc | ||
| offline_circ | ||
| opac | ||
| patron_lists | ||
| patroncards | ||
| plugins | ||
| pos | ||
| preservation | ||
| recalls | ||
| reports | ||
| reserve | ||
| reviews | ||
| rotating_collections | ||
| serials | ||
| services | ||
| sip2 | ||
| skel | ||
| suggestion | ||
| svc | ||
| t | ||
| tags | ||
| tools | ||
| virtualshelves | ||
| xt | ||
| .codespell-ignore | ||
| .editorconfig | ||
| .git-blame-ignore-revs | ||
| .gitignore | ||
| .htaccess | ||
| .mailmap | ||
| .nvmrc | ||
| .perlcriticrc | ||
| .perltidyrc | ||
| .prettierrc.js | ||
| .proverc.dist | ||
| .stylelintrc.json | ||
| INSTALL | ||
| Koha.pm | ||
| LICENSE | ||
| MANIFEST.SKIP | ||
| Makefile.PL | ||
| README | ||
| README.md | ||
| README.robots | ||
| SECURITY.md | ||
| about.pl | ||
| app.psgi | ||
| build-resources.PL | ||
| changelanguage.pl | ||
| cpanfile | ||
| cypress.config.ts | ||
| eslint.config.mjs | ||
| fix-perl-path.PL | ||
| gulpfile.js | ||
| help.pl | ||
| kohaversion.pl | ||
| mainpage.pl | ||
| package.json | ||
| rewrite-config.PL | ||
| rspack.config.js | ||
| tsconfig.json | ||
| yarn.lock | ||
README.md
Koha is a free software integrated library system (ILS).
Koha is distributed under the GNU GPL version 3 or later.
Note: Koha does not accept pull requests from git hosting sites.
Note: This project has its own bug tracker, to report a bug or submit a patch visit https://bugs.koha-community.org.
For guidelines on submitting patches for Koha please visit https://wiki.koha-community.org/wiki/SubmitingAPatch
The developers handbook can be found at https://wiki.koha-community.org/wiki/Developer_handbook
