Koha-clms/t/db_dependent
Jonathan Druart 38f92df4e5 Bug 14722: Refactor the export tool
Why a refactoring was need for this script?
The export tool (tools/export.pl) can be called from the command line
and some parts of code were unnecessarity complicated (just look at the
code, you will understand).

Worse still, the script does not provide the same options for both
interface. For instance you cannot export records given a range of
biblionumbers, authids, callnumbers, etc. from the commandline.

What does this patch?
1/ Important: The script tools/export.pl does not work anymore if called from
the command line (should be in the release notes).
2/ The code used to generated a file (csv, iso2709 or xml) has been moved to a new
module (Koha::Exporter::Record) and tests have been provided.
3/ No change is done on the web interface
4/ Some new options have been added to the commandline script
(misc/export.pl):
    - starting_authid
    - ending_authid
    - authtype
    - starting_biblionumber
    - ending_biblionumber
    - itemtype
    - starting_callnumber
    - ending_callnumber
    - start_accession
    - end_accession
5/ There is a change in the behavior if an error occurs:
Can't call method "as_usmarc" on an undefined value at Koha/Exporter/Record.pm line 114.
record (number 5530) is invalid and therefore not exported because its reopening generates warnings above at Koha/Exporter/Record.pm line 117.

Before this patch, they were not displayed (using the command line).

What does not do this patch?
It does not provide the 'clean', 'timestamp' and 'deleted_barcodes' options to
the web interface (same as before).

What about the perfs?
With a DB with ~800 biblios (MARC21)
Before: perl tools/export.pl 14.79s user 0.83s system 71% cpu 21.905 total
After:  perl misc/export.pl  17.19s user 0.84s system 75% cpu 24.018 total

With a DB with ~6400 biblios (UNIMARC)
Before: perl tools/export.pl 26.55s user 0.76s system 76% cpu 35.498 total
After:  perl misc/export.pl  26.78s user 0.84s system 80% cpu 34.494 total

How to test this patch?
Test plan:
A. Web interface:
1/ On the current master, export some records, biblios and authorities (with
the 3 differents exports) playing with the different filters (item type,
libraries, callnumber, accession date, don't export items, remove
non-local items, don't export fields, etc.).
2/ Apply this patch, export again the same records, and compare the
generated files. They must be identical!
3/ Confirm that the export features on the checkout list
(circ/circulation.pl) works as before this patch.

B. The command line
1/ On the current master, export some records, biblios and authorities (with
the 2 differents exports) playing with the different options (date,
deleted_barcodes, clean).
2/ Apply this patch, export again the same records, and compare the
generated files. They must be identical!

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-10-27 17:01:28 -03:00
..
Acquisition Bug 8417: Fix test 2015-10-21 08:46:20 -03:00
Circulation Bug 14788: Add unit tests for GetTopIssues 2015-10-22 11:35:06 -03:00
Creators Bug 14059: Deleting 'issues' table before processing the tests 2015-05-22 17:02:22 -03:00
Exporter Bug 14722: Refactor the export tool 2015-10-27 17:01:28 -03:00
Holds Bug 9809: Update unit tests 2015-08-26 10:26:29 -03:00
Items Bug 8723: Update the reserves table for item-level holds 2015-09-16 10:39:02 -03:00
Koha Bug 14888: Revert "Bug 14888: (QA followup) remove unused lib/var" 2015-10-05 15:05:44 -03:00
LDAP
…
Labels Bug 14468: (QA followup) remove useless diags 2015-10-27 10:38:15 -03:00
Letters Bug 13215: Fix GetLetterTemplates should return default templates if branchcode is not defined 2014-11-27 11:42:26 -03:00
Members Bug 15029: Make the IssueSlip tests pass even if launched after 23:00 2015-10-20 14:21:05 -03:00
OAI Bug 13940 : (QA followup) Add test for ModOAISet 2015-04-09 11:56:53 -03:00
Record Bug 12404: Allow equal sign '=' in the TT directive 2015-04-27 10:43:23 -03:00
Reports Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Reserves Bug 14702: Unit tests for GetReserveFee and ChargeReservesFee 2015-09-07 12:04:49 -03:00
Search Bug 10933: (followup of the followup) fix the previous patch to work with master 2015-10-27 13:06:01 -03:00
Serials Bug 12671: Guess next serial date when there are several issues per unit. Added support for real month and year lenghts. 2015-05-05 15:22:10 -03:00
Template/Plugin Bug 7380: Add tests for TT plugins 2015-05-19 10:10:31 -03:00
Utils Bug 14782: Add tests 2015-09-16 10:36:40 -03:00
XSLT_Handler Bug 13262 - Add parameters to XSLT Handler transform method 2015-02-05 15:24:29 -03:00
data Bug 12419: (QA followup) regression tests 2014-06-30 15:57:23 -03:00
www Bug 10363: (QA followups) Fix tests 2015-10-21 17:23:38 -03:00
00-strict.t
…
Accounts.t Bug 11229: Make the tests pass even if issues exist in the DB 2015-08-18 13:18:21 -03:00
Acquisition.t Bug 13967: (QA followup) Make DBIx control transactions on tests 2015-06-04 15:12:45 -03:00
AdditionalField.t Bug 10855: Remove additional field values when subscription is removed 2015-10-02 15:10:31 -03:00
Amazon.t
…
Auth.t Bug 14439: Add test - template path should finish by .tt 2015-06-24 10:53:25 -03:00
Auth_with_cas.t Bug 14121: Silence warnings t/db_dependent/Auth_with_cas.t 2015-07-20 10:32:21 -03:00
Auth_with_ldap.t Bug 12831: (RM followup) regression test 2014-11-27 14:45:11 -03:00
AuthorisedValues.t Bug 10363: Use Koha::AuthorisedValue[s] in the admin page 2015-10-19 12:46:27 -03:00
AuthoritiesMarc.t Bug 10985: [UNIMARC] Fix authority summary 2015-04-22 11:54:36 -03:00
BackgroundJob.t Bug 13606: Add tests and fix existing ones 2015-04-23 13:18:49 -03:00
Barcodes.t Bug 13673: (QA followup) Add license 2015-05-15 16:00:28 -03:00
Barcodes_ValueBuilder.t Bug 14778: Make Barcodes_ValueBuilder.t db dependent 2015-10-23 12:01:19 -03:00
Biblio.t Bug 10961: (followup) Make query fields explicit and add regression tests 2015-07-28 10:19:51 -03:00
Bookseller.t Bug 14334: t/db_dependent/Bookseller.t AutoCommit fix 2015-06-07 18:51:21 -03:00
Borrower.t Bug 14778: Make 3 tests pass 2015-10-23 12:01:17 -03:00
Borrower_Debarments.t
…
Borrower_Discharge.t Bug 14368: [Follow-up] Fix workflow 2015-09-28 11:57:45 -03:00
Borrower_Files.t Bug 12417: adding the units tests of the module Koha::Borrower::Files.pm 2014-07-11 16:16:11 -03:00
Borrowers.t Bug 13967: Add a couple of tests for SysPref and Object 2015-06-04 10:39:16 -03:00
Branch.t Bug 9530: Follow-up - Fix t/db_dependent/Branch.t 2014-10-28 09:26:03 -03:00
Breeding.t Bug 12898: (regression test) test terms inside double quotes 2014-09-14 02:02:51 -03:00
Budgets.t Bug 14334: t/db_dependent/Budgets.t AutoCommit fix 2015-06-07 18:51:22 -03:00
Category.t Bug 12604: refactoring Category.t with TestBuilder 2015-04-20 10:12:08 -03:00
Charset.t Bug 8218 : Add a maintenance script to sanitize biblio records 2014-11-11 15:38:36 -03:00
Circulation.t Bug 8236: (QA followup) fix test count error due to bad merge 2015-10-20 10:52:23 -03:00
Circulation_Branch.t Bug 14045: Make GetBranchBorrowerCircRule return maxonsiteissueqty 2015-10-13 11:13:24 -03:00
Circulation_Issuingrule.t Bug 13590: Add ability to charge fines at start of charge period 2015-10-22 14:51:24 -03:00
Circulation_OfflineOperation.t
…
Circulation_dateexpiry.t Bug 14494: Add some unit tests too for CalcDateDue 2015-07-20 10:16:24 -03:00
Circulation_issue.t Bug 14640: Add tests 2015-09-16 11:00:17 -03:00
Circulation_issuingrules.t Bug 14778: Use mock_dbh where it should be used 2015-10-23 12:01:18 -03:00
Circulation_transfers.t Bug 12122: TransferSlip should accept both itemnumber and barcode 2015-03-05 11:47:12 +01:00
ClassSource.t
…
ColumnsSettings.t Bug 10212: Columns configuration for tables - Unit tests 2014-08-26 09:28:18 -03:00
Context.t Bug 14778: Use mock_dbh where it should be used 2015-10-23 12:01:18 -03:00
Contract.t Bug 14693: Contract.t should generate dates based on current date 2015-08-21 10:43:38 -03:00
CourseReserves.t
…
Csv.t
…
External_BakerTaylor.t
…
Fines.t Bug 13590: Add ability to charge fines at start of charge period 2015-10-22 14:51:24 -03:00
FrameworkPlugin.t Bug 10480: New module and unit test for framework plugins 2015-04-16 14:48:35 -03:00
Heading_MARC21.t
…
Hold.t Bug 13853: Add tests for is_waiting 2015-08-11 14:48:44 -03:00
Holds.t Bug 5144: Add unit test 2015-10-06 10:01:45 -03:00
HoldsQueue.t Bug 14297: Unit Tests 2015-09-25 11:59:56 -03:00
Holidays.t Bug 14522: (QA followup) Remove POD and fix tests 2015-10-02 12:08:04 -03:00
ILSDI_Services.t Bug 11944: use CGI( -utf8 ) everywhere 2015-01-13 13:07:21 -03:00
ImportBatch.t Bug 6520: Add unit test for C4::ImportBatch::GetRecordFromImportBiblio 2015-04-16 13:40:59 -03:00
Installer.t
…
Items.t Bug 14778: Make 3 tests pass 2015-10-23 12:01:17 -03:00
Items_DelItem.t Bug 12583: (followup) rename new test file 2014-07-26 10:17:09 -03:00
KitchenSinkPlugin.kpz
…
Koha.t Bug 10937: cleanup and rename DOCTYPECAT to ITEMTYPECAT 2015-10-22 11:06:22 -03:00
Koha_Authority.t Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Koha_Database.t Bug 13645: Cache the DBIx connection 2015-02-20 14:37:08 -03:00
Koha_Misc_Files.t
…
Koha_borrower_modifications.t Bug 12623: SQLHelper replacement - Borrower::Modifications 2014-08-11 11:42:58 -03:00
Koha_template_plugin_Branches.t Bug 10900 - Follow up, since more has been added to master 2015-02-09 17:00:15 -03:00
Koha_template_plugin_KohaDates.t Bug 12669: Centralize the timezone handle into Koha::DateUtils 2015-02-20 10:20:30 -03:00
Languages.t Bug 14030 - Added tests for duplicates. 2015-04-24 09:55:54 -03:00
Letters.t Bug 15028: Make t/db_dependent/Letters.t set the numbering pattern it needs 2015-10-19 11:52:10 -03:00
Linker_FirstMatch.t
…
Log.t Bug 13889: Add unit tests 2015-05-04 11:11:32 -03:00
MarcModificationTemplates.t Bug 11413: Fix field_numbers 2014-11-14 12:05:47 -03:00
Members.t Bug 10020: Remove code related to ethnicity 2015-10-06 10:30:32 -03:00
Members_Attributes.t Bug 12427: Allow execution of UTs on a DB with issues 2014-06-24 09:43:04 -03:00
Members_columns.t
…
MungeMarcPrice.t
…
NewsChannels.t Bug 14248: Regression tests 2015-10-27 16:26:33 -03:00
Overdues.t Bug 9180: All branches should be returned if a default rule exists 2014-08-19 09:29:51 -03:00
PatronLists.t
…
Plugins.t Bug 12412: Add ability for plugins to convert arbitrary files to MARC from record staging tool 2015-05-04 15:33:51 -03:00
QueryParser.t Bug 12738: (regression tests) C4::Context should set keyword search as default for QueryParser 2014-08-11 10:05:59 -03:00
Ratings.t Bug 13998: (RM followup) tests expect data on the database 2015-04-21 13:53:40 -03:00
Record.t Bug 12357: Following whitespace failures on koha qa test tools. 2015-09-07 13:59:07 -03:00
RecordProcessor_EmbedSeeFromHeadings.t Bug 14456: (regression test) Holdings fields should be skipped by EmbedSeeFromHeadings 2015-07-24 13:39:53 -03:00
Reports.t
…
ReportsGuided.t Bug 14778: Use mock_dbh where it should be used 2015-10-23 12:01:18 -03:00
Reports_Guided.t Bug 14778: Make 3 tests pass 2015-10-23 12:01:17 -03:00
Reserves.t Bug 5144: Add test on carp message 2015-10-06 10:02:06 -03:00
Review.t Bug 13969: (QA followup) Delete issues before borrowers 2015-05-14 11:11:46 -03:00
RotatingCollections.t Bug 8836: (RM followup) unit tests should use the new API 2014-11-07 11:09:14 -03:00
SIP_ILS.t Bug 7904: (QA followup) fix tests 2015-02-05 15:50:32 -03:00
Search.t Bug 14778: (QA followup) remove warnings 2015-10-26 11:18:04 -03:00
Serials.t Bug 13423: Fix number of tests in Serials.t 2015-05-14 11:48:17 -03:00
Serials_2.t Bug 5342: Serial claiming improvements: add a counter 2014-10-28 10:07:37 -03:00
Service.t
…
ShelfBrowser.t
…
Sitemapper.t Bug 14778: Example - Replace DBI::Mock with Test::DBIx::Class - Sitemapper.t 2015-10-23 12:01:18 -03:00
Stats.t Bug 11230 - Refactor C4::Stats::UpdateStats and add UT 2014-07-27 11:29:28 -03:00
SuggestionEngine_ExplodedTerms.t Bug 12653: remaining prog reference in t/db_dependent/SuggestionEngine_ExplodedTerms.t 2014-10-29 23:37:54 -03:00
Suggestions.t Bug 13014: (QA followup) have new warnings tested 2015-06-24 14:54:36 -03:00
Tags.t Bug 12916 - Missing Test to demonstrate warnings. 2014-10-30 00:31:26 -03:00
Templates.t Bug 5338: (QA followup) update the tests for the new reports 2015-05-19 13:54:09 -03:00
TestBuilder.t Bug 15052: Add diagnostic for the TestBuilder tests 2015-10-23 09:35:13 -03:00
Upload.t Bug 14321: [QA Follow-up] Additional QA requests 2015-09-25 12:02:52 -03:00
VirtualShelves.t Bug 14383: Fix some typos in comments and documentation 2015-06-22 17:34:46 -03:00
VirtualShelves_Page.t
…
XISBN.t Bug 12782: (followup) skip tests that fail because of provider's error 2014-08-20 12:29:27 -03:00
XSLT_Handler.t Bug 13262: (QA followup) fix duplicate variable definition 2015-02-05 15:25:50 -03:00
check_sysprefs.t Bug 14383: Fix some typos in comments and documentation 2015-06-22 17:34:46 -03:00
default_search_class.pl Bug 12738: (regression tests) C4::Context should set keyword search as default for QueryParser 2014-08-11 10:05:59 -03:00
rollingloans.t Bug 13315 - Add feedback for last item checked out to circulation.pl 2015-04-13 14:52:26 -03:00
sysprefs.t Bug 13967: Add a couple of tests for SysPref and Object 2015-06-04 10:39:16 -03:00
zebra_config.pl Bug 12651: DOM indexing is the default 2014-10-27 12:35:44 -03:00