Koha-clms/Koha/SearchEngine/Elasticsearch
Nick Clemens 19f442c18f Bug 35086: Add chunk_size option to elasticsearch configuration
Whne performing batch operations we can send a large numebr of records for reindexing at once.
Currently this can create requetss that are too large for Elasticsearch to process. We need
to break these requests into chunks/

This patch adds a chunk_size configuration to the elasticsearch stanza in koha-conf.xml

If blank we default to 5000.

To test:
0 - Have Koha using Elasticsearch
1 - Create and download a report of all barcodes:
    SELECT barcode FROM items
2 - Batch modify these items
3 - Note a single ESindexing job is created
4 - Create and download a report of all authority ids:
    SELECT auth_header.authid FROM auth_header
5 - Setup a marc modification template, and batch modify all the authorities
6 - Again note a single ES backgorund job is created
7 - Apply patch
8 - Repeat the modifications above - you still get a single job
9 - Edit koha-conf.xml and add <chunk_size>250</chunk_size> to elasticsearch stanza
10 - Repeat modifications - you now get several background ES jobs
11 - prove -v t/db_dependent/Koha/SearchEngine/Elasticsearch/Indexer.t

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 9951e230e4)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 636920558c)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2024-02-02 16:25:22 +00:00
..
Browse.pm Bug 24823: Remove unused parameters 2020-05-12 11:43:30 +01:00
Indexer.pm Bug 35086: Add chunk_size option to elasticsearch configuration 2024-02-02 16:25:22 +00:00
QueryBuilder.pm Bug 33406: Filter out some known isolated trouble makers 2023-09-17 22:49:50 -10:00
Search.pm Bug 29561: (follow-up) Don't remove facet '0' 2022-10-11 10:41:52 -03:00