Koha-clms/t/db_dependent
Jonathan Druart 979bc2e6fa Bug 18966: Do not deal with duplicate issue_id on checkin
Koha suffers of big bugs due to its history: When data are deleted, they are moved to another tables.
For instance issues and old_issues: when a checkin is done, it is moved to the old_issues table.
That leads to a main problem that is described on https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix

However we tried first to fix the problem (for issues/old_issues) at code level on bug 18242.
The goal was to prevent data lost.
Data lost may happens in this case:
Check an item out (issue_id = 1)
Check an item in (issue_id = 1)
Restart MySQL (reset auto increment for issue_id to 1)
Check an item out (issue_id = 1)
Check an item in => BOOM, the issue_id is a PK in old_issues and the move fails.
Before bug 18242 the data were lost, we inserted the value into old_issues, which fails silently (because of RaiseError set to 0 in Koha::Database), then delete the row from issues.
That has been fixed using a transaction.

This patch introduced a regression we tried to fix on bug 18651 comment 0, the patron was charged even if the checkin was rejected.
A good way to fix that would have been to LOCK the tables:
1- Start a transaction
2- LOCK the table to make sure nobody will read id and avoid race conditions
3- Move the content from one table to the other, dealing with ids
4- UNLOCK the table
5- Commit the transaction
But there were problems using LOCK and DBIx::Class (See commit 905572910b - Do no LOCK/UNLOCK the table).

Finally the solution implemented is not acceptable for several reasons:
- batch checkins may fail
- issue_id will always stay out of sync (between issues and old_issues)
See 18651 comment 66.

Since the next stable releases are very soon, and we absolutely need to fix this problem, I am suggesting to:
1- Execute the move in a transaction to avoid data lost and reject the checkin if we face IDs dup
=> It will only reject 1 checkin (max is 1 * MySQL restart), no need to deal with race conditions,
2- Display a warning on the checkin page and link to a solution/explanation
3- Communicate as much as we can on the proper fix: Update auto increment values when the DBMS is restarted - https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix
4- Display a warning on the about page for corrupted data (see bug 18931)
5- Write and make available a maintenance script to fix corrupted data (TODO LATER)

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Mason James <mtj@kohaaloha.com>
2017-07-25 11:51:43 +12:00
..
Acquisition Revert "Bug 17564: Fix t/db_dependent/Acquisition/OrderUsers.t" 2016-12-31 14:43:38 +13:00
Auth
…
Authorities Bug 17913: [16.05.x] Authority merge fix 2017-02-24 09:42:24 +13:00
Biblio
…
Circulation Bug 18966: Do not deal with duplicate issue_id on checkin 2017-07-25 11:51:43 +12:00
Creators Bug 18044: Add a test 2017-02-16 07:23:50 +13:00
Exporter Bug 17644 - t/db_dependent/Exporter/Record.t fails 2016-12-12 14:54:12 +13:00
Holds Bug 17574: Remove itemtype-related warnings from LocalHoldsPriority.t 2016-12-12 14:54:09 +13:00
Items Bug 16607: Remove CPL/MPL from two unit tests 2016-06-24 07:28:45 +02:00
Koha Bug 15451: Add the 2 new modules Koha::CsvProfile[s] 2017-04-30 23:20:00 +12:00
LDAP
…
Labels
…
Letters
…
Members Bug 18009 - IssueSlip.t test fails if launched between 00:00 and 00:59 2017-01-29 00:52:07 +13:00
OAI
…
Patron Bug 17603: Remove itemtype-related Borrower_Discharge.t warnings 2016-12-12 14:54:09 +13:00
Record
…
Reports
…
Reserves
…
SIP Bug 15006: Centralize timeout logic and allow zero client timeout 2016-08-01 11:52:51 +02:00
Search Bug 17323: Column search_history.time cannot be null 2016-12-31 15:19:46 +13:00
Serials Bug 7728: QA fixes 2017-04-24 11:08:33 +12:00
Template/Plugin
…
Utils Bug 18094: Add tests to highlight the problem 2017-05-04 02:52:33 +12:00
XSLT_Handler
…
api/v1 Bug 16271: Allow more filters on /api/v1/holds 2016-08-01 11:11:13 +02:00
data
…
selenium Bug 13691: Add changes for 3.22 2016-08-01 11:41:11 +02:00
www Bug 18773: Fix failing history.t test 2017-06-13 17:23:21 +12:00
00-strict.t Bug 18233 - t/db_dependent/00-strict.t has non-existant resetversion.pl 2017-05-24 14:26:22 +12:00
Accounts.t Bug 16607: Remove CPL/MPL from two unit tests 2016-06-24 07:28:45 +02:00
Acquisition.t Bug 16934: Add test for ModOrder 2016-08-01 13:31:23 +02:00
AdditionalField.t
…
Amazon.t
…
AudioAlerts.t
…
Auth.t Bug 18442: Add a test 2017-05-24 13:45:02 +12:00
Auth_with_cas.t
…
Auth_with_ldap.t Bug 17615 - Fix unit tests 2017-01-25 14:53:46 +13:00
AuthorisedValues.t
…
AuthoritiesMarc.t
…
BackgroundJob.t Bug 17426: Do not set AutoCommit in tests 2016-10-22 15:10:09 +02:00
Barcodes.t Bug 15878 - Updated unit tests for hbyymmincr barcodes 2016-05-23 17:42:10 +00:00
Barcodes_ValueBuilder.t
…
Biblio.t Bug 16431: Use Koha::Cache to cache marc subfield structure 2016-08-01 10:34:57 +02:00
Bookseller.t
…
Branch.t
…
Breeding.t
…
Budgets.t Bug 15009 [QA Followup] - Update number of tests run 2016-04-29 13:01:07 +00:00
Category.t
…
Charset.t
…
Circulation.t Revert "Bug 14598: [QA Follow-up] Small changes" 2016-12-31 14:35:57 +13:00
ClassSource.t Bug 16860: Remove the comment about the warning 2016-08-01 10:50:08 +02:00
ColumnsSettings.t
…
Context.t Bug 16813: Allow empty string for OPACBaseURL 2016-08-01 13:31:21 +02:00
Contract.t
…
CourseReserves.t
…
DecreaseLoanHighHolds.t Bug 17573: Remove itemtype-related warnings from DecreaseLoanHighHolds.t 2016-12-12 14:54:09 +13:00
Filter_MARC_ViewPolicy.t
…
Fines.t
…
FrameworkPlugin.t
…
Heading_MARC21.t
…
Hold.t Bug 16609: Catch warning from Koha::Hold in Hold.t 2016-06-15 09:04:06 +02:00
Holds.t Bug 16717: Improve variable naming 2016-06-24 07:29:30 +02:00
HoldsQueue.t Fix t/db_dependent/HoldsQueue.t tests 2017-05-05 21:13:46 +12:00
Holidays.t Bug 16376: (regression tests) 2017-01-03 14:22:37 +13:00
ILSDI_Services.t Bug 16155: (QA followup) fix small bug in t/db_dependent/ILSDI_Services.t 2016-05-04 13:47:58 +00:00
ImportBatch.t Bug 9259: Use is instead of is_deeply 2016-08-01 10:59:55 +02:00
Installer.t Bug 17234: Need to separate KEY and FOREIGN KEY checks 2017-01-25 13:19:12 +13:00
Items.t Bug 17512: Improve handling dates in C4::Items 2017-02-23 09:36:21 +13:00
KitchenSinkPlugin.kpz Bug 17368: Update KitchenSinkPlugin.kpz 2016-10-22 15:12:40 +02:00
Koha.t
…
Koha_Authority.t
…
Koha_Database.t
…
Koha_Elasticsearch. Bug 17372: Standardize Elasticsearch paths 2016-10-22 13:56:11 +02:00
Koha_Elasticsearch_Indexer.t Bug 17372: Standardize Elasticsearch paths 2016-10-22 13:56:11 +02:00
Koha_Misc_Files.t
…
Koha_SearchEngine_Elasticsearch_Search.t Bug 18068: ES - Fix location and (home|holding)branch facets 2017-02-23 10:26:47 +13:00
Koha_borrower_modifications.t Revert "Bug 17494: Prevent duplicate tokens from getting stored" 2016-12-31 14:44:57 +13:00
Languages.t Bug 16866: Catch warning t/db_dependent/Languages.t 2016-08-01 11:33:45 +02:00
Letters.t Bug 18494: Pass a code, not hash & Fix double commas 2017-05-24 14:26:22 +12:00
Linker_FirstMatch.t Bug 16868: Silence error t/db_dependent/Linker_FirstMatch.t 2016-08-05 09:00:34 +02:00
Log.t
…
MarcModificationTemplates.t
…
Members.t Bug 15702: Add test cases for modified code 2017-05-24 13:45:00 +12:00
MungeMarcPrice.t
…
NewsChannels.t Bug 16550: Clean the tests 2016-05-23 17:37:48 +00:00
Overdues.t
…
Passwordrecovery.t Bug 18025: Fix test 2017-02-23 10:26:47 +13:00
Patron.t Bug 14577 [QA Followup] 2016-04-29 11:54:23 +00:00
PatronLists.t Bug 16452: Remove the warnings raised by PatronLists.t 2016-05-05 20:44:47 +00:00
Patrons.t Bug 17742: Use TestBuilder to create the library and patron category 2017-01-25 12:47:57 +13:00
Plugins.t Bug 16502: Replace a few other ok-calls by is-calls 2016-06-24 07:25:41 +02:00
QueryParser.t
…
Ratings.t
…
Record.t
…
RecordProcessor_EmbedSeeFromHeadings.t
…
Reports.t
…
Reserves.t Bug 14968 - Provides unit test 2016-08-01 10:33:02 +02:00
Review.t
…
RotatingCollections.t
…
Search.t Revert "Bug 17917: [QA Follow-up] Reprove Search.t" 2017-01-27 17:07:56 +13:00
Serials.t Bug 18460: Fix undefined itemtype warning in Serials.t 2017-04-30 23:19:59 +12:00
Serials_2.t
…
Service.t
…
ShelfBrowser.t
…
Sitemapper.t Bug 17920: t/db_dependent/Sitemapper.t fails because of permissions 2017-01-25 15:05:56 +13:00
Stats.t
…
SuggestionEngine_ExplodedTerms.t Bug 16869: Silence and catch warnings in t/db_dependent/SuggestionEngine_ExplodedTerms.t 2016-08-01 10:43:57 +02:00
Suggestions.t Bug 16325: Add a test for SearchSuggestions when searching for STATUS='' 2016-05-05 20:46:01 +00:00
Tags.t
…
Templates.t Bug 18010: Unit test for gettemplate 2017-04-04 03:45:18 +12:00
TestBuilder.t Bug 17726: [QA Follow-up] Add test descriptions 2017-02-22 21:58:09 +13:00
Upload.t Bug 16501: Remove some unneeded warns in Upload.t 2016-05-16 17:46:25 +00:00
UsageStats.t Bug 18066: Hea V2 [squashed] 2017-05-24 13:44:59 +12:00
Virtualshelves.t
…
XISBN.t
…
XSLT_Handler.t
…
check_sysprefs.t Revert "Bug 17917: [AMENDED] Fix path issue in t/db_dependent/check_sysprefs.t" 2017-01-27 17:07:44 +13:00
db_structure.t
…
default_search_class.pl
…
rollingloans.t
…
sysprefs.t
…
zebra_config.pl
…