Koha-clms/C4
Nick Clemens 7054818646 Bug 25265: [20.05.x] Prevent double reindex of the same item in batchmod
When batch editing, 2 reindex calls are sent to ES/Zebra.
We can easily avoid that reusing the skip_modzebra_update (renamed skip_record_index)

Additionally we should only send one request for biblio, and we should
only do it if we succeed

As the whole batch mod is in a transaction it is possible to fail in which case
Zebra queue is reset, but ES indexes have already been set

In addition to the skip param this patchset moves Zebra and Elasticsearch calls to
Indexer modules and introduces a generic Koha::SearchEngine::Indexer so that we don't
need to check the engine when calling for index

The new index_records routine takes an array so that we can reduce the calls to
the ES server.

The index_records routine for Zebra loops over ModZebra to avoid affecting current behaviour

Test plan:

General tests, under both search engines:
  1 - Add a biblio and confirm it is searchable
  2 - Edit the biblio and confirm changes are searchable
  3 - Add an item, confirm it is searchable
  4 - Delete an item, confirm it is not searchable
  5 - Delete a biblio, confirm it is not searchable
  6 - Add an authority and confirm it is searchable
  7 - Delete an authority and confirm it is not searchable

Batch mod tests, under both search engines
  1 - Have a bib with several items, none marked 'not for loan'
  2 - Do a staff search that returns this biblio
  3 - Items show as available
  4 - Click on title to go to details page
  5 - Edit->Item in a batch
  6 - Set the not for loan status for all items
  7 - Repeat your search
  8 - Items show as not for loan
  9 - Test batch deleting items
    a - Test with a list of items, not deleting bibs
    b - Test with a list of items, deleting bibs if no items remain where all items are only item on a biblio:
     SELECT MAX(barcode) FROM items GROUP BY biblionumber HAVING COUNT(barcode) IN (1)
    c - Test with a list of items, deleting bibs if no items remain where some items are the only item on a biblio:
     SELECT MAX(barcode) FROM items GROUP BY biblionumber HAVING COUNT(barcode) IN (1,2)
 10 - Confirm records are update/deleted as appropriate

Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Bug 25265: Rename skip_modzebra_update to skip_record_index

Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Bug 25265: Fix copy paste error for parameter

Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Bug 25265: (follow-up) Don't index malformed records

This is analogous to 26522, we shoudl skip record that cannot be retrieved for indexing

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Bug 25265: (QA follow-up) Add shebang to Indexer.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Bug 25265: (QA follow-up) Rename biblionumber in ModZebra, index_records

ModZebra:
The name is very misleading: we can index authid's too here.
And yes, it should not be in C4/Biblio too ;) A first step..

Adding the same change here in Koha/SearchEngine/Zebra/Indexer.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Bug 25265: (QA follow-up) Check server type in Elasticsearch::index_records

Doing the same change as previously (renaming biblionumber), but fixing
at the same the record fetch. If (theoretically) an authority is passed
without a record, it would have fetched a biblio record.

Test plan:
You need Elasticsearch here.
Replaced this line in AddAuthority:
    $indexer->index_records( $authid, "specialUpdate", "authorityserver", $record );
by
    $indexer->index_records( $authid, "specialUpdate", "authorityserver", undef );
And updated an authority record. Check if you can search for the change.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2020-10-16 14:15:22 +00:00
..
AuthoritiesMarc Bug 24217: use Modern::Perl for modules when strict is missing 2020-01-03 10:45:46 +00:00
Barcodes Bug 21395: Fix C4/Barcodes/ValueBuilder.pm 2020-07-13 17:13:16 +00:00
ClassSortRoutine
…
ClassSplitRoutine Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Creators Bug 25852: Improve C4::Creators::Lib reliability under plack 2020-08-13 16:23:59 +00:00
External Bug 22010: Check preferences for ebook integration 2019-04-30 11:42:51 +00:00
Form
…
Heading Bug 24269: Adjust C4::Heading to generate headings from auth tags 2020-02-19 13:33:32 +00:00
ILSDI Bug 24769: (bug 14711 follow-up) Fix typo in parameter name for AddReserve 2020-05-05 11:03:33 +01:00
Installer Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Labels Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Linker Bug 25189: Don't create authority if results found 2020-07-10 15:53:23 +00:00
Members Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
OAI Bug 23084: Replace grep {^$var$} with grep {$_ eq $var} 2020-02-17 10:44:45 +00:00
Output
…
Patroncards Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Reports Bug 22001: execute reports in an eval 2020-03-31 17:48:14 +01:00
SIP Bug 25903: Sending a SIP patron information request with a summary field flag in indexes 6-9 will crash server 2020-09-04 17:59:48 +00:00
Search Bug 24217: use Modern::Perl for modules when strict is missing 2020-01-03 10:45:46 +00:00
Serials Bug 20144: [sql_modes] Add default value for subscription_numberpatterns.description in tests 2018-02-13 13:58:53 -03:00
Utils Bug 25875: Move check for module_bit and code to the JOIN 2020-07-13 16:22:27 +00:00
Accounts.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Acquisition.pm Bug 25750: fix fallback to ecost_tax_included/ecost_tax_excluded 2020-09-03 15:40:42 +00:00
Auth.pm Bug 25360: (follow-up) Remove the https FIXME in Auth.pm 2020-08-31 20:05:23 +00:00
Auth_cas_servers.yaml.sample
…
Auth_with_cas.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Auth_with_ldap.pm Bug 20443: Use search_with_library_limits for attribute types 2020-03-23 13:49:22 +00:00
Auth_with_shibboleth.pm Bug 24199: (bug 23042 follow-up) Prevent t/Auth_with_shibboleth.t to fail randomly 2019-12-10 10:43:20 +00:00
AuthoritiesMarc.pm Bug 25265: [20.05.x] Prevent double reindex of the same item in batchmod 2020-10-16 14:15:22 +00:00
BackgroundJob.pm Bug 24217: use Modern::Perl for modules when strict is missing 2020-01-03 10:45:46 +00:00
Barcodes.pm
…
Biblio.pm Bug 25265: [20.05.x] Prevent double reindex of the same item in batchmod 2020-10-16 14:15:22 +00:00
Boolean.pm
…
Breeding.pm Bug 23542: Fix SRU import encoding 2020-08-17 14:31:09 +00:00
Budgets.pm Bug 24361: (bug 24217 follow-up) Fix several warnings in C4 modules 2020-01-07 13:59:51 +00:00
Calendar.pm Bug 25723: (QA follow-up) Handle holiday and exception on same day 2020-07-27 17:40:25 +00:00
Charset.pm Bug 22721: Remove frameworkcode parameter in GetMarcFromKohaField calls 2019-07-15 11:28:08 +01:00
Circulation.pm Bug 25265: [20.05.x] Prevent double reindex of the same item in batchmod 2020-10-16 14:15:22 +00:00
ClassSortRoutine.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
ClassSource.pm Bug 15836: Remove deprecated subroutines 2018-11-13 18:56:18 +00:00
ClassSplitRoutine.pm Bug 15836: Add the ability to defined custom methods to split call number in labels 2018-11-13 15:18:22 +00:00
Context.pm Bug 25360: Use secure flag for CGISESSID cookie when using HTTPS 2020-08-31 20:05:05 +00:00
Contract.pm Bug 24217: use Modern::Perl for modules when strict is missing 2020-01-03 10:45:46 +00:00
CourseReserves.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Creators.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Debug.pm
…
HTML5Media.pm Bug 20783: Use iframe to embed Youtube videos 2020-06-17 22:37:45 +00:00
Heading.pm Bug 24269: Adjust C4::Heading to generate headings from auth tags 2020-02-19 13:33:32 +00:00
HoldsQueue.pm Bug 25786: Holds Queue building may target the wrong item for item level requests that match holds queue priority 2020-06-18 18:53:42 +02:00
Images.pm Revert "Bug 21987: Do not generate true color thumbnails if not needed" 2019-11-04 13:21:53 +00:00
ImportBatch.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
ImportExportFramework.pm Bug 18143: Silence ods MARC framework export flood 2019-02-19 13:52:15 +00:00
InstallAuth.pm Bug 25360: Use secure flag for CGISESSID cookie when using HTTPS 2020-08-31 20:05:05 +00:00
Installer.pm Bug 24883: Add POD entry and remove unused variable 2020-03-27 12:11:32 +00:00
ItemCirculationAlertPreference.pm
…
Items.pm Bug 25265: [20.05.x] Prevent double reindex of the same item in batchmod 2020-10-16 14:15:22 +00:00
Koha.pm Bug 17845: Remove unused code related to printers 2020-03-10 15:18:27 +00:00
Labels.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Languages.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Letters.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Linker.pm
…
Log.pm Bug 23632: Remove C4::Logs::GetLogs 2020-09-04 14:39:36 +00:00
MarcModificationTemplates.pm Bug 24840: Replace DateTime->now with dt_from_string 2020-04-08 11:54:23 +01:00
Matcher.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Members.pm Bug 20443: Remove SearchIdMatchingAttribute 2020-03-23 13:49:21 +00:00
Message.pm Bug 21206: Replace C4::Items::GetItem 2019-02-26 13:24:07 +00:00
NewsChannels.pm Bug 24002: Incorrect DATE value: '00-00-0000' in NewsChannels.pm 2019-11-12 13:49:58 +00:00
Output.pm Bug 11994: OpenSearch plugins needs to be delivered with Content-Type application/opensearchdescription+xml 2020-07-10 16:03:57 +00:00
Overdues.pm Bug 25127: Use Koha::Number::Price->round for comparison 2020-05-01 15:44:55 +01:00
Patroncards.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Record.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Reports.pm Bug 24217: use Modern::Perl for modules when strict is missing 2020-01-03 10:45:46 +00:00
Reserves.pm Bug 24683: IsAvailableForItemLevelRequest sub description expanded 2020-09-04 17:49:30 +00:00
Ris.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
RotatingCollections.pm Bug 24299: Record 'RotatingCollection' as a transfer trigger 2020-03-20 15:54:55 +00:00
SMS.pm Bug 19134: C4::SMS falils on long driver name 2017-08-25 10:51:24 -03:00
Scheduler.pm Bug 24217: use Modern::Perl for modules when strict is missing 2020-01-03 10:45:46 +00:00
Scrubber.pm
…
Search.pm Bug 17661: Trivial simplification using += operator 2020-09-04 20:45:34 +00:00
Serials.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
Service.pm
…
ShelfBrowser.pm Bug 22445: Custom cover images - opac shelfbrowser 2019-10-22 15:02:48 +01:00
SocialData.pm
…
Stats.pm Bug 23084: Replace grep {^$var$} with grep {$_ eq $var} 2020-02-17 10:44:45 +00:00
Suggestions.pm Bug 22784: Add the ability to archive/unarchive a purchase suggestions 2020-04-14 16:19:02 +01:00
TTParser.pm Bug 25305: Translation process - Open all files specifying the utf8 encoding 2020-05-04 08:50:07 +01:00
Tags.pm Bug 23084: Replace grep {^$var$} with grep {$_ eq $var} 2020-02-17 10:44:45 +00:00
Templates.pm Bug 21395: Make perlcritic happy 2020-07-13 16:44:43 +00:00
TmplToken.pm
…
TmplTokenType.pm Bug 24217: use Modern::Perl for modules when strict is missing 2020-01-03 10:45:46 +00:00
UsageStats.pm Bug 25147: (RM follow-up) Update pref references. 2020-04-17 09:03:47 +01:00
XISBN.pm Bug 26270: Fix ThingISBN after a change in behaviour 2020-09-02 14:47:50 +00:00
XSLT.pm Bug 24052: Rename XSLT_Handler 2020-03-24 10:42:23 +00:00