Commit Graph

3180 Commits

Author SHA1 Message Date
Nick Clemens d7f4a93008
Bug 33418: Add overlay_framework option to connexion scripts
This patch adds the option to specify a framework to be used when overlaying records from webservices/connexion

To test:
1 - vim /etc/koha/sites/kohadev/connexion.cnf
2 - Set content:
    host:
    port: 8888
    koha:http://localhost:8081
    log:/var/log/koha/kohadev/connexion.log
    match:ISBN
    user:kohauser
    password:kohapass
    overlay_action:replace
    nomatch_action:create_new
    item_action:always_add
    import_mode:direct
    framework:BKS
    overlay_framework:
    debug:1
3 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach)
4 - On the command line:
    perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf
5 - In another terminal:
    cat bug_33418.test  | nc -v localhost 8888
6 - It should report success and a biblionumber
7 - In Koha: Cataloguing->Manage staged MARC record
8 - View the most recent batch with file name (webservice)
9 - Confirm it was imported, no match
10 - Click 'View' under the 'Record' column
11 - Confirm record loads correctly and 'MARC framework' detail is 'Books, Booklets, Workbooks'
12 - On the terminal repeat the command:
    cat bug_33418.test  | nc -v localhost 8888
13 - It should succed
14 - View the new batch, confirm the record matched this time
15 - View the record details, confirm framework is now 'default'
16 - On the first terminal hit Ctrl+C to stop the daemon
16 - Edit connexion.cnf and change:
    import_mode:stage
    framework:ACQ
    overlay_framework:IR
17 - Restart daemon:
     perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf
18 - Delete the record created above
19 - On the second terminal repeat the command:
    cat bug_33418.test  | nc -v localhost 8888
20 - Confirm the batch is created, but not imported
21 - In terminal:
     perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS
22 - Confirm batch imported, and record in ACQ framework
23 - In terminal:
     cat bug_33418.test  | nc -v localhost 8888
     perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS
24 - Confirm batch added, record matched, record imported, and record now in Books framework
25 - Stop the deamon, edit connexion.cnf:
     import_mode:direct
26 - Start the daemon, and on other terminal repeat:
     cat bug_33418.test  | nc -v localhost 8888
27 - Confirm record in Binders framework
28 - Set record framework to Books
29 - Stop daemon, edit cnf and remove 'overlay_framework' setting
30 - Start daemon and cat the file again
31 - Confirm the record remains in Books framework

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-05-10 16:45:50 +02:00
Fridolin Somers 18ac387e6a
Bug 36787: staticfines.pl add use Koha::DateUtils::output_pref
misc/cronjobs/staticfines.pl is calling output_pref() but it is missing from module import :
use Koha::DateUtils qw( dt_from_string );

Test but running staticfines.pl

Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-05-07 17:41:20 +02:00
Martin Renvoize 7dd3fc3cef
Bug 35169: (QA follow-up) Fix spelling and preference name
Update the preference name from 'Borrower' to 'Patron' and correct
spelling of receive.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-05-07 15:53:49 +02:00
Matthias Le Gac 0ebc5a78d9
Bug 35169: Resolve the override situations
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-05-07 15:53:47 +02:00
Matthias Le Gac 7c9a7fe7ff
Bug 35169: Make long overdue borrower category options configurable in interface
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-05-07 15:53:45 +02:00
Lucas Gass 5e06fd00ce
Bug 36709: Add confirm flag to update_localuse_from_statistics.pl script
To test:
1. APPLY PATCH and restart all
2. Run the update_localuse_from_statistics.pl script without a confirm flag, nothing happens.
3. Run the script and add the --confirm flag, stuff happens.

Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-05-07 15:53:44 +02:00
Lucas Gass 307acdf96b
Bug 36137: Make update_totalissues cron always skip the holds queue
To test:
1. Run the update_totalissues cron, the holds queue is updated.
2. APPLY PATCH, restart services
3. Run update_totalissues cron again, this time the holds queue should always be skipped.

Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-05-03 18:02:15 +02:00
Julian Maurice 675c8263b7
Bug 36526: Remove circular dependency from Koha::Objects
Koha::Objects depends on Koha::DateUtils, which depends on C4::Context,
which depends on Koha::Config::SysPrefs, which depends on Koha::Objects

Apart from the circular dependency, the dependency on C4::Context alone
is problematic as it loads a bunch of modules that are not needed at all
in Koha::Objects (YAML::XS and ZOOM for instance).
As Koha::Objects is used as a base for a lot of modules, we should take
care to only load the minimum required.

This patch removes uses of Koha::DateUtils from Koha::Objects.
It was only used in Koha::Objects::filter_by_last_update

filter_by_last_update now requires that the 'from' and 'to' parameters
must be DateTime objects. Previously it would also allow date and
datetime strings. This possibility was only used in two places:
* misc/cronjobs/cleanup_database.pl
* tools/cleanborrowers.pl

Now they call dt_from_string first and pass a DateTime object to
filter_by_last_update

Test plan:
1. Run `perl -cw Koha/Objects.pm`. It should only say:
   "Koha/Objects.pm syntax OK" without warnings
2. Run `prove t/db_dependent/Koha/Objects.t`
3. Verify that misc/cronjobs/cleanup_database.pl works as before,
   especially with the options --pseudo-transactions,
   --pseudo-transactions-from and --pseudo-transactions-to
4. Go to Tools » Batch patron deletion and anonymization, check "Verify
   you want to anonymize patron checkout history" and enter a date in
   the text input below. Then click Next and verify that the correct
   count of borrowers is shown. Click on the "Finish" button and verify
   that the circulation history has been correctly anonymized

See also bug 36432

Signed-off-by: Tadeusz Sośnierz <tadeusz@sosnierz.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-05-03 16:02:44 +02:00
David Cook 9dcac6dfe9
Bug 36700: Update svc to use CSRF-TOKEN
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2024-05-01 13:40:19 +01:00
Nick Clemens 4da2e1444b
Bug 32565: (follow-up) Tidy
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-30 14:32:07 +02:00
Nick Clemens 939f1f389b
Bug 32565: Add unallocated option to holds queue
Add an unallocated option to CreateQueue and pass through as needed
Avoid deletion of the tmp_holdsqueue, and only check holds
and items that are not currently matched

A future hold with a higher priority will still fail here - because the
item may already be assigned, but on next change to the biblio it would
be corrected

To test:
1) Apply both patches
2) Enable RealTimeHoldsQueue and set HoldsQueueSkipClosed to "open"
3) Add a holiday to the calendar for all libraries for today, visit:
/cgi-bin/koha/tools/holidays.pl
-- Click today's day on the calendar and pick "Holiday repeated every same day of the week"
-- Click "Copy to all libraries". Hit "Save.
4) Place a biblio-level hold on a biblio record and set the pickup location to a library that has available copies, visit:
-- /cgi-bin/koha/reserve/request.pl?biblionumber=76&borrowernumber=51
-- Click the first "Place hold" button to place the biblio-level hold.
5) Verify that that hold got added to the holds queue, visit:
/cgi-bin/koha/circ/view_holdsqueue.pl?branchlimit=&itemtypeslimit=&ccodeslimit=&locationslimit=&run_report=1
6) Place a biblio-level hold on a biblio record where there are no other holds and copies are available at another location, but not the pickup location, visit:
-- /cgi-bin/koha/reserve/request.pl?biblionumber=437&borrowernumber=51
-- On the "pickup at" dropdown, pick something else other than "Centerville", e.g. "Fairfield".
-- Click the first "Place hold" button to place the biblio-level hold.
7) Check the holds queue again, notice that this 2nd hold was not added to the queue:
/cgi-bin/koha/circ/view_holdsqueue.pl?branchlimit=&itemtypeslimit=&ccodeslimit=&locationslimit=&run_report=1
8) Run the updated cronscript:
perl misc/cronjobs/holds/build_holds_queue.pl --force --unallocated
9) Notice nothing changed in the holds queue, visit:
/cgi-bin/koha/circ/view_holdsqueue.pl?branchlimit=&itemtypeslimit=&ccodeslimit=&locationslimit=&run_report=1
10) Remove the holiday we created previously, visit:
/cgi-bin/koha/tools/holidays.pl
-- Click today's day on the calendar and pick "Delete this holiday"
-- Click "Copy to all libraries". Hit "Save.
11) Run the updated cronscript:
perl misc/cronjobs/holds/build_holds_queue.pl --force --unallocated
12) Confirm the second hold is added to the holds queue, visit:
/cgi-bin/koha/circ/view_holdsqueue.pl?branchlimit=&itemtypeslimit=&ccodeslimit=&locationslimit=&run_report=1

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-30 14:32:05 +02:00
Kyle M Hall 83f9535fab
Bug 31345: Add ability to exit process_message_queue.pl early if any plugin before_send_messages hook fails
Sometimes it would be better for process_message_queue.pl to stop if a plugin hook fails rather than continue processing. It would be nice if that was a command line option.

Test Plan:
1) Install any plugin with a before_send_messages hook
2) Modify the plugin, add a 'die;' statement at the start of the
   before_send_messages method of the plugin.
3) Run process_message_queue.pl as usual
4) Note the exit code is 0
5) Run it again with the new -e setting
6) Note the exit code is 1

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-29 18:53:14 +02:00
Matt Blenkinsop 875809d66d
Bug 35996: Make it clear that --category-code can not be used alone
This patch adds a clarification to writeoff_debts.pl to make it clear that --category-code can't be used as the only filter when running the script. If this is the case, the script just displays the help menu as it is expecting one of the other filters

Test plan:
1) Run perl misc/cronjobs/writeoff_debts.pl --category-code TEST --confirm
2) Observe that the help menu is displayed instead of running the script
3) Check the help menu for the text added in this patch

WNC amended patch - added same warning in the option section

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-29 09:35:37 +02:00
Martin Renvoize f8b5c5029a
Bug 16122: Tidy changed code blocks
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-26 14:15:02 +02:00
Lucas Gass 95ff045c65
Bug 16122: Add maintenance script to update items.localuse from stats
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Andrew Fuerste Henry <andrewfh@dubcolib.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-26 14:15:01 +02:00
Tomas Cohen Arazi 5d8f6a76ec
Bug 13706: Minor tidy/spelling issues
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-26 14:14:54 +02:00
Julian Maurice d9b71ff26d
Bug 13706: New script dedup_authorities.pl
This script allow to deduplicate authorities automatically.

Script is in misc/maintenance/

It works this way:
 1) authorities are fetched from the database. You can limit fetched
 results by authtypecode, or directly by specifying WHERE clause
 2) for each authority:
  2.1) build a Zebra query using the 'search_form' for the heading
  2.2) run the query, retrieve the results
  2.3) among duplicates, choose the one we want to keep (use
  --choose-method option).
  2.5) use C4::Authorities::merge to merge authorities
 3) delete the merged authorities

Use --help for more informations on options

To be done:
1 - Move to module and cover with tests
2 - Add option to only merge unused authorities
3 - Expand 'ppn' option to be 'control-number' option and allow specifying field
4 - More?

1 & 2 I will attempt - 3 & 4 may be future enhancements

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-26 14:14:53 +02:00
Thomas Klausner 61e7aa374e
Bug 35345: Add --where option to rebuild_elasticsearch.pl
Sometimes we need to only re-index a subset of our bibliographic data or authorities. Currently this is only possible by enumerating all id (-bn or -ai), which does not work well when indexing eg 100.000 items of a 2.000.000 DB. Re-indexing everything is also overkill.

This patch adds an `--where` flag to misc/search_tools/rebuild_elasticsearch.pl which can take arbitrary SQL (that of course has to match the respective tables) and adds it as an additional param to the resultset to index

To test, start koha-testing-docker with ElasticSearch enabled, for example via `ktd --es7 up

Before applying the patch, rebuild_elasticsearch will index all data:

Biblios:
$ misc/search_tools/rebuild_elasticsearch.pl -b -v
[12387] Checking state of biblios index
[12387] Indexing biblios
[12387] Committing final records...
[12387] Total 435 records indexed
(there might be a waring regarding a broken biblio, which can be ignored)

Auth:
$ misc/search_tools/rebuild_elasticsearch.pl -a -v
[12546] Checking state of authorities index
[12546] Indexing authorities
[12546] 1000 records processed
[12546] Committing final records...
[12546] Total 1706 records indexed

Now apply the patch

Biblio, limit by range of biblioid:
$ misc/search_tools/rebuild_elasticsearch.pl -b -v --where "biblionumber between 100 and 150"
[12765] Checking state of biblios index
[12765] Indexing biblios
[12765] Committing final records...
[12765] Total 50 records indexed

Note that only 50 records where indexed (instead of the whole set of 435 records)

Auth, limit by authtypecode:
$ misc/search_tools/rebuild_elasticsearch.pl -a -v --where "authtypecode = 'GEOGR_NAME'"
[12848] Checking state of authorities index
[12848] Indexing authorities
[12848] Committing final records...
[12848] Total 142 records indexed

Again, only 142 have been indexed.

Sponsored-by: Steiermärkische Landesbibliothek
Sponsored-by: HKS3 / koha-support.eu

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-22 12:39:26 +02:00
Pedro Amorim ac157b7a75
Bug 35581: Koha::Illrequest -> Koha::ILL::Request
Koha/Illrequests.pm -> Koha/ILL/Requests.pm

Merged:
t/db_dependent/Koha/Illrequests.t
t/db_dependent/Illrequests.t

Into:
t/db_dependent/Koha/ILL/Requests.t

ILL classes file structure is, for the most part, around 7 years old and doesn't follow a strict logic. It's so confusing that some test files exist redundantly.

This housekeeping should help future work in regards to ISO18626 to add Koha as a supplying agency instead of just requesting agency, as is now.
It should also help future housekeeping of moving backend related logic out of the Illrequest.pm into Illbackend.pm (now ILL/Request.pm and ILL/Backend.pm as of this patchset).
It should also help in structuring the addition of a master generic form (see bug 35570)

This patchset will require existing backends to be updated to match the new class names and structure, if they invoke them.

Test plan, k-t-d, run tests:
prove t/db_dependent/api/v1/ill_*
prove t/db_dependent/Koha/ILL/*

Test plan, k-t-d, manual:
1) Install FreeForm, enable ILL module, run:
bash <(curl -s https://raw.githubusercontent.com/ammopt/koha-ill-dev/master/start-ill-dev.sh)
2) You'll have to switch the FreeForm repo to the one compatible with this work, like:
cd /kohadevbox/koha/Koha/Illbackends/FreeForm
git checkout reorganize_ILL
3) Do some generic ILL testing:
3.1) Create a request
3.2) Add a comment to a request
3.3) Edit a request
3.4) Edit a request's item metadata
3.5) Confirm a request
3.6) List requests
3.7) Filter requests list using left side filters
4) Install a metadata enrichment plugin:
https://github.com/PTFS-Europe/koha-plugin-api-pubmed
4.1) Create an ILL batch and insert a pubmedid like 123
4.2) Add the request and finish batch
5) Verify all of the above works as expected

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-22 08:57:45 +02:00
Pedro Amorim 5ea2e83700
Bug 35581: Koha::Illrequest::SupplierUpdate* -> Koha::ILL::Request::SupplierUpdate*
Koha/Illrequest/SupplierUpdate.pm -> Koha/ILL/Request/SupplierUpdate.pm
Koha/Illrequest/SupplierUpdateProcessor.pm -> Koha/ILL/Request/SupplierUpdateProcessor.pm

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-22 08:57:45 +02:00
Pedro Amorim bcc0f17317
Bug 35581: Koha::Illrequest::Config -> Koha::ILL::Request::Config
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-22 08:57:44 +02:00
Tomas Cohen Arazi df1f5da673
Bug 36309: Make create_superlibrarian.pl output more useful
In situations in which you are not familiar with all the Koha settings,
and table structure, the fact this script just fails telling there's a
broken FK is just not practical.

We should capture those exceptions and display a useful message instead.

This script does that. It adds some validations and some exception
handling too. It prints a nice message about the bad value the user
passed, and the valid values too!

To test:
1. Run this on a fresh KTD:
   $ ktd --shell
  k$ perl  misc/devel/create_superlibrarian.pl \
      --userid tcohen \
      --password tomasito \
      --cardnumber 123456789 \
      --categorycode POT \
      --branchcode ATO
=> FAIL: It explodes with a MySQL exception message!
2. Apply this patch
3. Repeat 1
=> SUCCESS: It tells you which value is wrong and what values you can
pick to make the command work
4. Pick a valid value, and repeat
=> SUCCESS: Now the other value is wrong, a nice message is displayed!
5. Fix with a valid value and repeat
=> SUCCESS: Patron created!
6. Sign off :-D

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-22 08:57:35 +02:00
Marcel de Rooy 33e463c108
Bug 36068: (QA follow-up) Add OPTIONS section in POD
Test plan:
Run script with --help.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-19 18:06:31 +02:00
Marcel de Rooy e50582293b
Bug 36068: Add maint script acq_cancel_obsolete_orders.pl
Introducing $orders->filter_by_obsolete and $orders->cancel.

Test plan:
Run t/db_dependent/Koha/Acquisition/Orders.t
Create basket with few orders. Remove some biblio records.
Run acq_cancel_obsolete_orders.pl -c

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-19 18:06:30 +02:00
Kyle M Hall c6641ef24f
Bug 34621: Tidy import_patrons.pl
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-12 14:02:43 +02:00
Kyle M Hall b89d744e86
Bug 34621: implement Patron import option to 'Renew existing patrons' 'from the current membership expiry date'
Test Plan:
1) Test importing a patron with the "from the current membership expiry date" option,
   note it does not work
2) Apply this patch
3) Restart all the things!
4) Re-test, note the expiration was renewed from the patron's current
   expiration date!

Signed-off-by: David Nind <david@davidnind.com>
Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-12 14:02:41 +02:00
Magnus Enger 4848def5e2
Bug 36517: Fix output from install_plugins.pl
When misc/devel/install_plugins.pl does not find any plugins, it prints
the list of pluginsdir, but with a literal \n separating the dirs, and
no newline at the end.

To test:
- Edit koha-conf.xml and add a second entry for <pluginsdir>, so there
  are two entries. The second one could just be a copy of the original.
- Run "perl misc/devel/install_plugins.pl"
- Note the output looks something like this:

No plugins found
pluginsdir contains:
/var/lib/koha/kohadev/plugins\n/var/lib/koha/kohadev/pluginsroot@kohadevbox:koha(master)$

- Apply the patch and run the script again. Output should be:

No plugins found
pluginsdir contains:
/var/lib/koha/kohadev/plugins
/var/lib/koha/kohadev/plugins
root@kohadevbox:koha(master)$

- Sign off

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-12 11:26:01 +02:00
Jonathan Druart 0026ee3e97
Bug 36516: Fix useless warning from translation script
I lost too much time trying to know what the translation script outputs
  sh: 1: Syntax error: end of file unexpected

To recreate:
touch 'koha-tmpl/intranet-tmpl/prog/en/modules/<' (yes, don't ask!)
koha-translate -i de-DE --dev kohadev
or
koha-translate -u de-DE --dev kohadev

And you get the shell error without any ideas what's happening!

With this patch applied you will not get any errors because the quote
will actually copy the file (what is expected).

But without the quote the cp command will fail and the explanation will be
displayed
sh: 1: Syntax error: end of file unexpected
Cannot copy /kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/en/modules/< to /kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/de-DE/modules/< at /kohadevbox/koha/misc/translator/tmpl_process3.pl line 385.

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-05 11:51:19 +02:00
Marcel de Rooy 3feca570d8
Bug 36031: Remove obsoleted script
Since it is not up to date, we should remove it.
The folder structure changed and this would need digging into SELinux to update that side as well.
In other words, remove this until someone finds time do it. But dont let it here in its current outdated state.

No test plan here.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-05 11:51:16 +02:00
Lucas Gass ade75ea561
Bug 23296: Set patron branchcode when preparing non-digest version
To test:
1. APPLY PATCH
2. Set up circ rules to allow for auto renewals.
3. Set AutoRenewalNotices to 'according to message prefs'
4. Make a branch specific version of the the AUTO_RENEWAL notice.
5. Find a patron belonging to that branch and set the message prefs to allow for a AUTO_RENEWAL. But no digest option!
6. Check some things out and make sure they are eligble for auto renewal.
7. Run the cron.
8. Make sure the patron gets the branch specific notice.
9. Delete the branch specific notice.
10. DO 6-7 again.
11. Make sure they now get the all library notice.

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Tidied one line.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-05 11:51:12 +02:00
Fridolin Somers 1ff71cf200
Bug 36323: Move koha_perl_deps.pl to misc/devel
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-04-03 17:05:51 +02:00
Fridolin Somers fd70ee6a94
Bug 31286: Embed see-from headings into bibliographic records export
In misc/export_records.pl add an option to add see-from headings (from authorities 4xx) into bibliographic records export.
Like it is done during record indexing.

Test plan :
1) Choose a biblio record having a field (for example 650) linked to an authority with a see-form.
2) Export this record without see-from headings :
   misc/export_records.pl --starting_biblionumber=X --ending_biblionumber=X --filename /tmp/record_without.xml --format xml
3) Export this record with see-from headings :
   misc/export_records.pl --starting_biblionumber=X --ending_biblionumber=X --filename /tmp/record_with.xml --format xml --embed_see_from_headings
4) Compare /tmp/record_without.xml and /tmp/record_with.xml
=> you should see two 650, one with main heading and one with see-from heading

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-25 09:53:24 +01:00
Emily Lamancusa 7552b73503
Bug 35653: (QA follow-up) Perltidy
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-25 09:53:21 +01:00
Lucas Gass 3198621787
Bug 35653: Log import_patrons.pl info to action logs
To test:
1. APPLY PATCH
2. Create a CSV file with a few lines that contain at least firstname, surname, cardnumber
3. Turn on "CronjobLog".
4. Run the import patron script. I did this: perl /kohadevbox/koha/misc/import_patrons.pl --file 'test1.csv' --matchpoint cardnumber -c
5. Check the action logs for cronjobs.
6. You should see info. about the import patron script.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-25 09:53:21 +01:00
Marcel de Rooy d8e04545b8
Bug 35536: Refine verbose handling in some Koha::Plugins calls
Three routines in Plugins got the verbose parameter on 35507.
We can refine this a bit further.
The idea here is report when you are installing plugins but not
report when just calling plugins (flooding logs).

[1] GetPlugins: Most callers do not expect (or check) results for
    failing plugins. This patch makes GetPlugins only return
    errors when passing the *errors* flag (in 2 cases).
    [a] The misc/devel script prints warnings now using verbose,
        so does not need the errors flag anymore.
    [b] plugins/plugins-home is the only case left. Tiny adjustment
        to keep current behavior. Fixed colspan in template.
        Does not need verbose in favor of 'errors' (passed to
        template).
    [c] For most calls we do not want verbose. New default is 0.

[2] InstallPlugins
    [a] Disabled verbose in plugin-upload. Not really needed.
        Added a FIXME; we need to improve individual install.
    [b] misc/devel: No warnings anymore when calling InstallPlugins
        after GetPlugins.

[3] get_enabled_plugins
    [a] Plugins->call does not need verbose.
    [b] Plugins->feature_enabled does not need it too.

Test plan:

[1] See previous plan. With TestMR data but without patch, run
misc script and go to plugins-home. Do you see load errors on
commandline or form?

[2] Run plugins/plugins-upload (uploading just some file is good
enough); verify that you do not see TestMR lines in logfile.

[3] Run t/db_dependent/Koha/Plugins/Plugins.t for the additional
test on verbose and errors flag.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-22 12:26:40 +01:00
Jonathan Druart 30816a7fc5
Bug 30349: Add missing double dash (--delete) to POD
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-22 10:35:24 +01:00
David Schmidt 06e7553dc9
Bug 30349: More double dashes
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-22 10:35:24 +01:00
David Schmidt c84c614146
Bug 30349: Use double dash for multi character option names
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-22 10:35:23 +01:00
David Schmidt ccf18acb97
Bug 30349: Tidy bulkmarcimport.pl to fix indentations
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-22 10:35:23 +01:00
Nick Clemens a1a7193110
Bug 34611: Confirm run if pseudonymized transactions found and add verbose feedback
Signed-off-by: AFHDubCoLib <andrewfh@dubcolib.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-19 10:25:06 +01:00
Nick Clemens 59368f5c9d
Bug 34611: Add a script to pseudonymize statistics from before a given date
This script takes a date parameter in SQL format and pseudonymizes all statistics
found before this date.

Only values that can be found will be added, i.e. no deleted patron or item info
will be present.

Additionally - the values stored will be the current values from patrons and items, so
some info will be approximate, much as it is when joining from the statistics table for reporting.

To test:
 1 - Perform some issues/returns/renewals/on-site checkouts
 2 - Make sure Pseudonymization is disabled
 3 - perl misc/maintenance/pseudonymize_statistics.pl
 4 - Script ends and reports that preference is not active
 5 - Enable the pref, and choose some borrower and item fields
    NOTE: See bug 28911 if you need a bcrypt key for your koha-conf.xml
 6 - perl misc/maintenance/pseudonymize_statistics.pl
 7 - sudo koha-mysql kohadev
 8 - SELECT * FROM pseudonymized_transactions
 9 - Confirm data is correctly stored
10 - DELETE FROM pseudonymized_transactions;
11 - UPDATE statistics SET datetime = '2023-01-01 00:00:00';
12 - perl misc/maintenance/pseudonymize_statistics.pl -b "2022-12-31 23:59:59";
13 - SELECT * FROM pseudonymized_transactions;
14 - Confirm no entries were made
15 - Select different options in Pseudonmyization prefs, including borrower attributes
     This wil require defining an attribute that can be kept for pseudonymization
16 - Confirm options are correctly pseudonymized

Signed-off-by: AFHDubCoLib <andrewfh@dubcolib.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-19 10:25:05 +01:00
David Gustafsson 6aa7118c13
Bug 25539: Strip items when adding new biblio to preserve previous behaviour of the script
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 18:03:40 +01:00
David Gustafsson e1479daf11
Bug 25539: Remove AddBiblio option "defer_marc_save"
Items are no longer embedded in the MARCXML and because of this the MARC data
does not need to be saved once more after changing record items data.
The "defer_marc_save" is no longer needed since bulkmarcimport.pl was the only
place this option was utilized in order to resave MARC data after possibly changing
items data. There is also a bug bulkmarcimport.pl where the record data is re-saved
without stripping items if duplicate items are found and the dedup barcodes
option is enabled that is resolved by this change.

This change enables MARC Overlay rules to be enabled for bulkmarkcimport.pl as using
the defer_marc option would previously effectively bypass any defined rules.

To test
1) Apply patch 29440 (which this depends on), but no not yet apply this
   patch
2) Remove comments around "bulkmarcimport: _("bulkmarcimport.pl"),"
   in koha-tmpl/intranet-tmpl/prog/en/modules/admin/marc-overlay-rules.tt
   line 463
3) Enable MARC overlay rules and add a rule (with source => "*" or
   'bulkmarcimport') to protect some field.
4) Import a biblio using the bulkmarcimport script. Overlay rules will
   only be applied on updates so a match condition matching the
   record in Koha when bulkmarcimport.pl is run the next time for the same
   record needs to be supplied (for example -match "control-number,001")
5) Edit the imported biblio and change the value of the protected field
6) Run bulkmarcimport with the same parameters as in 4) and verify the
   the field that should have been protected has been overwritten
7) Revert changes for line 463 in marc-overlay-rules.tt (or will result in
   conflict applying the patch), apply the patch and repeat steps 4-5,
   verify that the field now was protected from being overwritten.

Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 18:03:40 +01:00
David Gustafsson 70de25b439
Bug 29440: Fix AddBiblio/ModBiblio options args
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:47 +01:00
David Gustafsson 188ea40db1
Bug 29440: Replace removed syspref BiblioAddsAuthorities with AutoLinkBiblios
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:46 +01:00
David Gustafsson 6b92516214
Bug 29440: Only log items has been inserted if actually have been so, and remove extra out of place logging of record insertion
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:46 +01:00
David Gustafsson fd3efab876
Bug 29440: Rename $modify_biblio_marc_options to $mod_bilbio_options
This variable actually contains ModBibio options and
ModifyBiblioMarc accepts a subset of these

Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:45 +01:00
David Gustafsson 33d274bbb9
Bug 29440: Remove pointless call to _strip_item_fields
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:45 +01:00
David Gustafsson 2d50eb242d
Bug 29440: replace usage of removed GetMarcBiblio
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:44 +01:00
David Gustafsson 08908b5765
Bug 29440: defer biblio autolinking until records are indexed
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:44 +01:00
David Gustafsson 2dddbb4d14
Bug 29440: Fix QA-tool issues
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:43 +01:00
David Gustafsson aa86825c33
Bug 29440: Make authorities import behavior consistent with biblios
The current behavior for when importing authorities, when match
option is enabled, is to insert as a new authority fetching from db
fails. This is both a bit confusing and not consistent with the biblio
case where an error is generated. Also clean up error and log messages
so both messages/errors and when messages/errors are issued are more
consistent for the biblio and authority case.

Signed-off-by: Frank Hansen <frank.hansen@ub.lu.se>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:43 +01:00
David Gustafsson 261baeff85
Bug 29440: Refactor and clean up bulkmarcimport.pl
To test:
1) Import biblios using ./bulkmarcimport.pl -v -b -file biblios.mrc -insert -update -c=MARC21 -match "control-number,001" -n=1 -l "/tmp/import.log"
2) Verify in /tmp/import.log that one biblio has been inserted
3) Import again without the "-n" option, verify that one biblio has been
   updated and the rest inserted
4) In the staff interface search the catalog for some string appearing in
   the imported biblios to verify records have been indexed
5) View/edit some of the biblios the staff interface to very have been
   properly imported
6) Import authorities using ./bulkmarcimport.pl -v -m=MARCXML -a -file authorities.xml -insert -update -c=MARC21 -keepids="024a" -match "heading-main,100a" -l "/tmp/import.log" -yaml="/tmp/ids.yml"
7) Go to the authorities page and find the imported authority by for example search for a
   400 or 100 field value. Verify that the 001 field of the incoming record has been moved to 024a.
   Also check that the value of 024a appear as a key in /tmp/ids.yml.
8) Edit the authorities.xml file and change the value of field 400a
9) Import authorities again using the same options
10) Verify that the 400a field of the authority in Koha has not been
   updated (the import was skippped since revision did not change)
11) Edit authorities.xml, replace the value of field 005 in the file with field 005
    of the authority in Koha incremented by one
12) Import again with the same options
13) Verify that the 400a field of the authority in Koha now have been
    updated

Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-18 11:03:42 +01:00
Kyle M Hall e29e058851
Bug 33898: Implement reaping for database polling
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-15 09:38:37 +01:00
Marcel de Rooy bc42006019
Bug 33898: Alternative approach with receive frame timeout
See bug 33898 comment6.

Test plan:
Based on comment12: Stage MARC import and manage. Look at the same
time to the results of ps aux|grep Z. Verify that the lines with
[background_jobs] <defunct> disappear within 10 seconds.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-15 09:38:36 +01:00
Marcel de Rooy 3b8a3c7b86
Bug 31427: Remove a few refs to CanBookBeRenewed
Actually, the module is not even needed anymore here.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:44 +01:00
Marcel de Rooy c86e6ee82d
Bug 35819: (QA follow-up) Prevent warning on uninitialized retries count
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:39 +01:00
Jonathan Druart 7c1d4716be
Bug 35819: Adjust es_indexer_daemon
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Add forgotten module
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:39 +01:00
Jonathan Druart 30e69ebc56
Bug 35819: Improve logging
Log (warn) if the job will be processed later, but add a debug however.

Have a specific log for bad status

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:38 +01:00
Jonathan Druart 8394092568
Bug 35819: nack and not requeue if frame is invalid
If a frame cannot be correctly processed (most probably because the body
is not valid JSON) then we are not acking or nacking the frame and the
worker is stuck.

In this specific case we should nack without requeuing the frame.

NOTE that requeue must be 'true' or 'false', not 1 or 0, or the default
'true' will be used.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:38 +01:00
Marcel de Rooy 0feb10df7f
Bug 35819: Add simple delay
Here I add 500 ms. In my testing with the 1s sleep from the test
plan, I might see one or two 'not found' lines. Obviously things
depend on the time needed before the txn commits. But it will
reduce a flood of these messages.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:37 +01:00
Tomas Cohen Arazi 1a51c2e973
Bug 35819: Notify NACK and requeue when Job ID not found
This patch makes the worker reject the incoming frame for putting the
message back in the queue, in the event the job id doesn't exist yet.
Which is the case when some actions are being triggered inside a
transaction which hasn't been commited to the DB yet.

To test you will need 3 KTD shells
(a) mysql:
   $ ktd --shell
  k$ sudo koha-mysql kohadev

(b) logs:
   $ ktd --shell
   # for restarting the worker and looking at the logs
  k$ sudo koha-worker --restart kohadev  ; tail -f /var/log/koha/kohadev/worker-*.log
(c) running the test:
   $ ktd --shell

1. Have (a), (b) and (c) terminals ready
2. On (c), run:
   $ perl -MKoha::Database -MKoha::BackgroundJob::BatchUpdateBiblioHoldsQueue -e 'Koha::Database->schema->txn_do( sub { Koha::BackgroundJob::BatchUpdateBiblioHoldsQueue->new->enqueue({ biblio_ids => [ 1 ] }); sleep 1;  } );'
=> FAIL:
   * (b) shows (once) an error about a job not existing
3. On (a) run:
   > SELECT * FROM background_jobs;
=> FAIL: Notice the job ID mentioned on 2 stands as 'new'.
4. Apply this patch
5. Ctrl+c on (b), and re-run to launch the worker with the patch applied
6. Repeat 2
=> SUCCESS (partial): The error about the job not existing is displayed
many times
7. Repeat 3
=> SUCCESS: The job ID mentioned on 6 stands as 'finished'.
8. Sign off :-D

Discussion:

* The `requeue` header I added is correct, but it is the default
  behavior anyway. I prefered to make it explicit, though.

* To avoid that bunch of retries, we should requeue with some delay. I
  didn't manage to make it work (yet) but there's a 'delay' plugin for
  rabbit [1]. We already install the 'stomp' plugin in
  koha-common.postinst. But this plugin requires downloading it. Which
  would require further investigation.

* As Nick and Marcel pointed, we need to revisit the whole architecture,
  the need of a MQ (DB polling wouldn't have this problem), etc. But
  that's for another place.

[1] https://hevodata.com/learn/rabbitmq-delayed-message/#:~:text=To%20delay%20a%20message%2C%20the,to%20queues%20or%20other%20exchanges.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:36 +01:00
Julian Maurice 972efe8b5c
Bug 36009: Document background_jobs_worker.pl --queue elastic_index
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 15:02:49 +01:00
David Cook 0fe82b601b
Bug 36084: Add CSRF token support to svc/authentication
GET svc/authentication will return a CSRF token in a response header

POST svc/authentication requires a CSRF token which can be sourced
from the response header of GET svc/authentication or some other
place like the meta element on a HTML page

Note: misc/migration_tools/koha-svc.pl is a simple script which
can be used to practically evaluate svc/authentication and svc/bib

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2024-03-01 11:00:51 +01:00
Jonathan Druart 632146366a
Bug 34478: op =~ ^cud- in pl/pm - Manual revert changes to recreateIssueStatistics.pl
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2024-03-01 10:57:15 +01:00
Jonathan Druart a1338de383
Bug 34478: op =~ ^cud- in pl/pm - Manual revert changes to sip_cli_emulator.pl
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2024-03-01 10:57:14 +01:00
Jonathan Druart 18e808240f
Bug 34478: op =~ ^cud- in pl/pm
This is the result of
  bash op_must_start_with_cud-perl.sh

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2024-03-01 10:56:11 +01:00
Pedro Amorim d7a633048e
Bug 35479: (QA follow-up): Tidy
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-31 14:44:41 +01:00
Nick Clemens ea30aaff62
Bug 35479: (follow-up) Match formatting for other cronjob logs
To test:
1 - Install the Kitchen Sink plugin
2 - Restart all
3 - Enable 'CronjobLog' system preference
4 - perl misc/cronjobs/plugins_nightly.pl
5 - Note you see on the command line 'Remember to clean the kitchen' - this indicates the plugin cron ran
6 - Tools->log viewer, select 'cronjob' and view
7 - Note you only see 'plugins_nightly.pl' Run and End lines
8 - Apply patches
9 - perl misc/cronjobs/plugins_nightly.pl
10 - View logs agian
11 - Note you now see Run and End lines for 'Koha::Plugin::Com::ByWaterSolutions::KitchenSink'
12 - Confirm they look like the other lines
13 - Edit KitchenSink.pm and add 'die "Kittens";' to the cronjob nightly
14 - perl misc/cronjobs/plugins_nightly.pl
15 - View logs, confirm there is a FAILED error message for the KitchenSink cron

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-31 14:44:41 +01:00
Kyle M Hall c4b777b056
Bug 35479: Log start, end, and any failures for each plugin's nightly cronjob
When running the plugins_nightly.pl cronjob, we should record the plugins that have a nightly method, logging the start and end of each plugins routine

Test Plan:
1) Enable CronjobLog
2) Install a plugin with a nightly cronjob ( e.g.
   https://github.com/bywatersolutions/koha-plugin-book-list-printer )
3) Run plugins_nightly.pl
4) Note new entries in the cronjob viewer for the start and end of the
   plugin's nightly cronjob run
5) Edit the plugin, add a line like "die 'this is a test';" to the
   plugin's nightly cronjob
6) Run plugins_nightly.pl
7) View the action logs, not the log for the error you added!

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-31 14:44:40 +01:00
Caroline Cyr La Rose 633533049a
Bug 35596: Error in writeoff_debts documentation
This patch corrects an error in the documentation of the
writeoff_debts.pl script. The --type parameter accepts debit type codes,
not credit type codes.

To test:
-  Run ./misc/cronjobs/writeoff_debts.pl
-  Make sure the sentence for --type makes sense, grammar and spelling
   are correct

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-29 12:24:54 +01:00
Katrin Fischer 898d96adb9
Bug 35373: Remove false FIXME comment from gather_print_notices.pl
Removes a FIXME line mentioning bug 8000 that implemented
SendAllEmailsTo. But the note was added to the "From" line
and the "To" address is taken from a command line parameter.

To test:
* Review change carefully

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-26 15:14:07 +01:00
Martin Renvoize 78d1093cb5
Bug 34854: (follow-up) Param name change + POD
This patch updates the parameter name to more clearly define what it's
used for and also adds missing documentation.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-17 11:39:05 +01:00
Kyle M Hall efd932ad42
Bug 34854: Add ability to skip Talking Tech Itiva notifications for a patron if a given field matches a given value
Because some notices are mandatory, some libraries would like to allow patrons to opt-in to phone notices. The request is to be able to decide if a patron should get a notice based on the value of a given field ( usually sort1 or sort2 ).

Test Plan:
1) Apply this patch
2) Enable TalkingTechItivaPhoneNotification
3) Set up some overdue items for a patron
4) Enable phone notices for overdues in the notice status triggers
5) Run the outbound script for OVERDUE with -t OVERDUE
6) Note the output the output contains a line for that patron
7) Set the sort1 field for that patron to TEST
8) Run the script again, adding in the option "-i sort1:TEST"
9) Note the ouput no longer contains that patron!

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-17 11:39:04 +01:00
Kyle M Hall b4a9445971
Bug 26831: (follow-up) Tidy cleanup_database.pl
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-17 11:39:03 +01:00
Alex Buckley 428f6d5b17
Bug 26831: Use new PurgeListShareInvitesOlderThan syspref
Enable librarians in the staff client to control when unaccepted list
invites will be removed from the database.

Test plan:
1. Apply patches

2. Update database
sudo koha-shell <instance>
cd installer/data/mysql
./updatedatabase.pl

3. Confirm the PurgeListShareInvitesOlderThan syspref = 14

4. Run cleanup_database.pl with --list-invites parameter defined:
sudo koha-shell <instance>
cd misc/cronjobs
./cleanup_database.pl --list-invites 10 -v

5. Confirm the output contains:
"Purging unaccepted list share invites older than 10 days."

6. Set PurgeListShareInvitesOlderThan syspref = 15

7. Run cleanup_database.pl without the --list-invites parameter:
sudo koha-shell <instance>
cd misc/cronjobs
./cleanup_database.pl -v

8. Confirm the output contains:
"Purging unaccepted list share invites older than 15 days."

9. Empty PurgeListShareInvitesOlderThan syspref

10. Run cleanup_database without the --list-invites parameter:
sudo koha-shell <instance.
cd misc/cronjobs
./cleanup_database.pl

11. Observe you get the following output (along with documentation on all
parameters of the script):
"You did not specify any cleanup work for the script to do."

12. Run the cleanup_database with no days defined for the --list-invites
parameter:
sudo koha-shell <instance>
cd misc/cronjobs
./cleanup_database.pl --list-invites -v

13. Confirm the output contains:
"Purging unaccepted list share invites older than 14 days."

Sponsored-by: Catalyst IT, New Zealand
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-17 11:39:01 +01:00
Nick Clemens 7496a603cd
Bug 35086: Also split chunks when indexing from background job
The es background indexer is designed to combine background jobs when started based on the 'batch_size' option.

While this is helpful for combining individual updates, it can be problematic when there are several large batch modifications, or when worker has stopped and is restarted.

This patch uses the same logic as in the indexer to split the chunks that are sent directly for indexing.

To test:
1 - Follow test plan on previous patch
2 - Confirm items are correctly indexed and jobs marked

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-16 12:06:00 +01:00
Fridolin Somers 289e06edea
Bug 34091: Fix typo log-modules in help for cleanupdatabase.pl
This parameter is actually --log-module.

Test plan :
Run misc/cronjobs/cleanup_database.pl -h
Confirm you see --log-module

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2023-12-11 19:41:27 +01:00
Tomas Cohen Arazi a4a343c543
Add release notes for 23.11.0
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-30 16:17:58 -03:00
Koha translators 986edc2aa9 Bug 35174: Remove .po files from Koha
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2023-11-16 08:56:36 +01:00
Marcel de Rooy 1fd6bc61d7
Bug 15157: (QA follow-up) Some more changes
Renamed some vars. Removed unused $success.
Extended die statement.
Added Modern::Perl.

Added verbose parameter.
Added some logic around the confirm parameter. Making it possible
to run the script in 'dry' mode.
Print totals at the end.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 11:41:30 -03:00
Emmi Takkinen ac0c8cf181
Bug 15157: Add my $amount and fix options
Variable my $amount was missing. Also option
amount has to be declared as integer and
option confirm is no longer string.

Sponsored by: Koha-Suomi Oy

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

[EDIT] Run perltidy to resolve one line.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 11:41:30 -03:00
Martin Renvoize 4eb84b0dd7
Bug 15157: (follow-up) Add ability to set lower limit
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 11:41:29 -03:00
Olli-Antti Kivilahti 9e3f47d6b2
Bug 15157: Cronjob to debar Patrons with unpaid accountlines
If your library wants to debar all Borrower who haven't paid
their fines by the end of the year, this script will do that
trick :)

You can give the message from a file if the cronjob runner
doesn't deal with quotes, or as a command-line parameter.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Bug 15157: Update script

This patch updates debarrBorrowersWithFines.pl script to match
changes made in bug 15156.

To test:
1. Have patron(s) with unpaid fines
2. Run e.g debarrBorrowersWithFines.pl --confirm -m "This is a description of you bad deeds"
(test other options too)
3. Confirm patron(s) with fines has been debarred with the explanation

Sponsored-by: Koha-Suomi Oy
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Bug 15157: Modernise and Update for bug 15156

This patch updates the script to use filter_by_amount_owed, renames
it to debar_patrons_with_fines.pl and moves it to the cronjobs
directory whilst also adding a copyright notice and POD.

We could add a series of options to the script to allow more fine
grained control.

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

[EDIT] Run perltidy to resolve three lines.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 11:41:29 -03:00
Tomas Cohen Arazi f88cb1da4f
Bug 33050: Tidy
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 11:41:14 -03:00
Fridolin Somers 4d604ac2a9
Bug 33050: Allow to specify quote char in runreport.pl
Add to script misc/cronjobs/runreport.pl quote char arg (only for CSV).
See perl doc https://metacpan.org/pod/Text::CSV#new

This patch also adds missing '--separator' in POD.

Test plan :
1 - Write a SQL report
2 - perl misc/cronjobs/runreport.pl --format csv 1 (or correct report number)
3 - Note you get double quotes
4 - Apply patch
5 - Repeat #2 - no change
6 - perl misc/cronjobs/runreport.pl --format csv --quote "'" 1
7 - Now it is single quote delimited
8 - perl misc/cronjobs/runreport.pl --format tsv --separator "'" 1
9 - Error is reported, you cannot set quote unless csv
10 - Try empty string as quote

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 11:41:13 -03:00
Katrin Fischer 4c15ad658d
Bug 34704: Remove regex for adding additional line breaks to print templates
This makes sure that we don't change the line breaking for either
HTML nor plain text print notices.
For plain text, the <pre> is used to keep the line breaks as defined
in the notices template.
For HTML we require on the line breaks added by block elements in the
HTML notice, not adding any additional ones like before.
This gives the full control of the formatting to the notice editor.

To test:
* Make sure your patron doesn't have an email address
* Make sure to check the hold filled notice in messaging preferences
* Place a old on any item
* Checkin the item
* Confirm hold
* Verify notices tab shows the Hold (print) notice
* ./misc/cronjobs/gather_print_notices.pl ./ --html
* Verify the generated file has the HTML output wrapped in pre.
* Reformat the HOLD print notice to be HTML by setting the checkbox.
  Mmake sure to add some empty lines for (they should not print later)
  and some <br>, <p>, <div>  - They should be reflected in the notice later.
* Repeat test.
* All should be well :)

Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 11:41:10 -03:00
Tomas Cohen Arazi 4310e13a64
Bug 30024: Make link_bibs_to_authorities.pl rely on LinkerRelink
This patch makes the trivial change of making the linker script rely on
LinkerRelink instead of CatalogModuleRelink, which is confusing and
incorrect (according to both sysprefs descriptions).

To test:
1. Verify that relinking is now tied to the right syspref

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 14:22:45 -03:00
Katrin Fischer e49abfa6b5
Bug 35196: Remove misc/perlmodule_* scripts
These scripts are outdated and no longer needed in
the codebase.

To test:
* Verify both perlmodule_* scripts have been removed from misc/

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:42 -03:00
Aleisha Amohia 3a1c40a290
Bug 8367: (follow-up) Remove unnecessary code and atomic update
Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 18:01:33 -03:00
Aleisha Amohia 6c0eb32a08
Bug 8367: Add holds_pickup_period circulation rule
So that pickup delay can have a different value per patron category,
   item type or branch.

To test:

1) Update database, restart services
2) Set ReservesMaxPickUpDelay syspref (if not already set)
3) Edit your circulation rules and set a value under 'Holds pickup
period (day) that is DIFFERENT from ReservesMaxPickUpDelay. Set a few
different numbers for different branches as well.
4) Place a hold on a biblio from the staff client.
5) Check in an item from that biblio and confirm the hold as waiting
6) Confirm the expiration date is calculated using the 'Holds pickup
period' value instead of the ReservesMaxPickUpDelay syspref
7) Revert the waiting status and delete the hold
8) Re-place the hold on the biblio on the OPAC. Notice that when you
change the pick up location, the number of days in the pickup message
below the dropdown changes based on the circ rules.
9) Create a holiday with a date that will overlap with the 'Holds pickup
period'
10) Check in an item from that biblio and confirm the hold as waiting
11) Confirm the expiration date is calculated using the 'Holds pickup
period' value AND considers the special holiday
12) Confirm tests pass t/db_dependent/Holds/WaitingReserves.t
13) Test Talking Tech:
13a) Enable TalkingTechItivaPhoneNotification
13b) Go to Tools -> Notices & slips. Add content to the HOLD phone
(itiva) notice.
13c) In your terminal, run perl
/path/to/koha/misc/cronjobs/thirdparty/TalkingTech_itiva_outbound.pl -o
~/itiva.tmp -w 0 --type=RESERVE

Sponsored-by: Catalyst IT

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 18:01:30 -03:00
Victor Grousset/tuxayo 1e53f48efe
Bug 35171: (QA follow-up): tidy up POD
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:17 -03:00
Nick Clemens 4bfd1e21c6
Bug 35171: Add send_empty option to runreport
This patch adds a new 'send_empty' option to runreport.pl

To test:
1 - Create a report in Koha that will not return any results:
    SELECT barcode FROM items WHERE 1=2
2 - perl misc/cronjobs/runreport.pl 1
3 - Output is: NO OUTPUT: 0 results from execute_query
4 - perl misc/cronjobs/runreport.pl 1 --send_empty
5 - Output is: no results were returned for the report
6 - perl misc/cronjobs/runreport.pl 1 --send_empty --email
7 - It will die on an email error unless you have SMTP configured - this si good, it means we sent an email
8 - Bonus points: Test on a system that can correctly send emails, confirm it works :-)

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:16 -03:00
Martin Renvoize 508d55eb7b
Bug 35074: Add support for category_code to writeoff_debts
This patch adds support for passing category_code as a limiter for the
writeoff_debts script.

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:12 -03:00
Nick Clemens 17cf580f19
Bug 34587: (QA follow-up) Tidy
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:32 -03:00
Matt Blenkinsop cd2a642c31
Bug 34587: Fix dry run and end date in the cronjob
The cronjob always logs a dry run even on a live harvest and the end date parameter was not working correctly. This patch addresses both issues

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:22 -03:00
Pedro Amorim ca8ea02fda
Bug 34587: Move everything into a ERM/EUsage subfolder
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:18 -03:00
Pedro Amorim 2951edc697
Bug 34587: SUSHI COUNTER Harvester
This commit is a squash of the following:

SUSHI harvesting process in the data providers class:
* Builds the URL query and requests the SUSHI service endpoint
* Parses the JSON response and builds the csv COUNTER file and adds it to counter_files table

Usage statistics data processing:
* When a counter_files entry is stored, CounterFile.pm will:
* Parse the csv COUNTER file and
* Add a usage_titles entry for each unique title in the COUNTER file
* Add the title's respective erm_usage_mus (monthly usage) entries, repeating for each metric_type
* Add the title's respective erm_usage_yus (yearly usage) entries, repeating for each metric_type

Harvesting cronjob;

'Run now':
* API endpoint to start the harvesting process of a data provider
* Button in the data providers list to run the harvesting process for each data provider upon clicked

ERM SUSHI: Background job

Job progress is updated to total amount of usage titles after retrieving
the response from SUSHI;
Job warning and success messages are added accordingly
Redundant duplicate titles will not be added
Redundant duplicate monthly and yearly usage statistics will not be added
Data provider harvest background job harvests once per report_type

Enqueue one background job for each report_type in the usage data provider

Update the way we measure progress in the background job.
It now uses the COUNTER report body rows instead of SUSHI response results.
We're now incrementing and showing the number of skipped mus, skipped
yus, added mus and added yus
There's a bug in the way we calculate yus
Updates to background job progress bar - Depends on 34468

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:45:58 -03:00
Emily Lamancusa 0680910177
Bug 34924: (QA follow-up) correctly handle digest data compilation for successful renewal
automatic_renewals.pl does some data compilation for digest notices that
depends on the error value returned from attempt_auto_renew. However, on
successful renewal $error is undefined. Add additional checks on value
of $success and definedness of $error to compile data accurately and
avoid warns for undefined variable

Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 11:02:48 -03:00
Nick Clemens 7ecb05f4ec
Bug 34924: Add 'auto_renew_final' and 'auto_unseen_final' return to CanBookBeRenewed
There is a desire for auto_renewals to treat the final renewal differently. We would like to notify the patron of the final renewal - but not again when the next renewal fails. This patch adds the new return value and tests.

To test:
prove -v t/db_dependent/Circulation.t

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 11:02:46 -03:00
Nick Clemens 95f6015d43
Bug 34924: Add Koha::Checkout->attempt_auto_renew
This patch moves the actual renewal out of the auto_renewals cronjob script and into the object and adds tests. The logic for notices is still handled in the script.

To test:
1 - prove -v t/db_dependent/Koha/Checkouts.t
2 - Add a circ rule with auto_renew checked
3 - Checkout an item to a patron and set due date in the past
4 - Checkout an item to a patron and set due date in the future
5 - perl misc/cronjobs/automatic_renewals.pl -v
6 - Confirm one would be renewed and the other is too_soon
7 - perl misc/cronjobs/automatic_renewals.pl -v --confirm
8 - Confirm the expected issue is successfully renewed

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 11:02:46 -03:00
Thibaud Guillot 24ee60cb62
Bug 29811: Add timestamp option on authority record type
According to the timestamp option for bibs record type I added the
timestamp option for authority records.
Timestamp is already present in database on field "modification_time"

Test Plan :
1 - Be sure to have authority record type for easiest test create one
2 - Execute script export_records.pl in your koha/misc directory and
choose a date (example yesterday if you just created an authority
right now).(see export_records.pl -h for help)
3 - Timestamp option has no effect on authority record type
4 - Execute script again but choose the date of tomorrow for example
5 - Same result
6 - Apply this patch
7 - Play again steps 2 and 4
8 - On step 2 you will see only your authorities created today (because
script show you authority has changes since the date you choose in
option) and for step 4 you must see an empty file.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 11:02:40 -03:00
David Cook 2988c54971
Bug 35111: Ignore SIGPIPE in background jobs worker
This change explicitly ignores SIGPIPE signals in the background jobs
worker.

Daemons like Starman ignore SIGPIPE so it makes sense to explicitly set this.
Differences in the inner workings of MySQL vs MariaDB client libraries have yielded
different behaviours in automatic reconnections and potentially SIGPIPE handling,
so this helps to make the overall behaviour more consistent.

Test plan:
0. Apply patch and run "restart_all"
1. Go to http://localhost:8081/cgi-bin/koha/catalogue/detail.pl?biblionumber=29
2. Click "Save" > "MARCXML"
3. Go to http://localhost:8081/cgi-bin/koha/tools/stage-marc-import.pl
4. Click "Choose file", choose the MARCXML file, click "Upload file"
5. Click "Stage for import"
6. Note the job is marked as "100% Finished"

7. In a separate window run "docker restart koha-db-1"
8. Repeat steps 3-5 for uploading file and running stage for import
9. Note that the job is marked as "100% Finished" as you'd expect

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Added comment on the SIG PIPE line.
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-30 09:02:14 -03:00
Jonathan Druart 6757cd5f86
Bug 35043: Use Locale::PO->quote
We have our own "quote_po" sub that is stating that Locale::PO::quote
is buggy because it does not deal with quoting new lines correctly.
However it seems that it is fixed now.

Ideally we could use Locale::PO::quote everywhere, but it does not
escape tab characters:
  $string =~ s/\\(?!t)/\\\\/g;           # \t is a tab

This means the following:
msgid "Tabulation (\\t)"
msgstr "Tabulation (\\t)"
become:
-msgid "Tabulation (\t)"
-msgstr "Tabulation (\t)"

And we are seeing the following on Weblate:
https://snipboard.io/BjQmDC.jpg

Note that Locale::PO has not been updated since 2014...

The real problem behind this is that we have 2 methods to quote strings.
At first glance it seems that Locale::PO::quote was not used before, but
with the introduction of the koha-i18n project we will have scripts that
will use Locale::PO->save_file_fromarray, which uses Locale::PO->quote
=> Those scripts will be used on the translation server for post
processing (security reason, marking potential XSS strings as fuzzy).

Test plan:
0. Do not apply the patch
1. gulp po:update --lang LANG # Replace LANG with your favorite language
   code
2. git commit -a -m"init PO files"
3. Apply this patch
4. Repeate 1.
5. git diff
   => The change is about the "Tabulation" and "New line" strings from
      tools/csv-profiles.tt
6. Translate them (replace the \t and \n with %s) and remove the fuzzy
   flag
7. install the template: cd misc/translator && perl translate install
   LANG
8. Enable the language, use it and go to the "Nouveau profil CSV" view
   => Notice that the \t and \n are correctly displayed.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
2023-10-27 16:45:52 -03:00
Marcel de Rooy 3439185bb0
Bug 35141: (QA follow-up) Increment the bad count
Also move earlier print to warn for consistency.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
2023-10-27 16:44:56 -03:00
Nick Clemens 706641e747
Bug 35141: Prevent link_bibs_to_authorities from dying on invalid metadata
When fetching the record we need to eval in case of exception

To test:
1 - Apply first patch
2 - perl misc/link_bibs_to_authorities.pl
3 - Note it dies on parser error at record 369
4 - Apply this patch
5 - perl misc/link_bibs_to_authorities.pl
6 - The process completes - with warning thrown on record 369

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
2023-10-27 16:44:56 -03:00
Nick Clemens 22c0be31c7
Bug 35141: Catch ES search exceptions in link_bibs_to_authorities, warn, and continue
This patch simplay adds an eval and a warning to link_bibs_to_authorities to catch any errors while searching and continue linking.

To test:
1 - edit /etc/kohia/sites/kohadev/koha-conf.xml and add to elasticsearch stanza
    <request_timeout>0</request_timeout>
2 - perl misc/link_bibs_to_authorities.pl
3 - It dies immediately
4 - Apply patch
5 - perl misc/link_bibs_to_authorities.pl
6 - Now it tries every record, throwing warnings along the way, but completes

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
2023-10-27 16:44:55 -03:00
Kyle M Hall 1508e81505
Bug 35148: before_send_messages plugin hook does not pass the --where option
The before_send_messages plugin hook currently sends all the non-smtp related options with the exception of the --where option. This should be added.

Test Plan:
1) Install the latest version of the kitchen sink plugin (
   https://github.com/bywatersolutions/dev-koha-plugin-kitchen-sink/releases
   )
2) Apply this patch
3) Run process_message_queue.pl with a --where option
4) Run the command: ./misc/cronjobs/process_message_queue.pl --where "letter_code='test'"
5) Note the output looks like:
Plugin hook before_send_message called with the params: $VAR1 = {
          'verbose' => 0,
          'where' => 'letter_code=\'test\'',
          'letter_code' => [],
          'type' => [],
          'limit' => undef
        };

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-25 11:08:03 -03:00
Marcel de Rooy 3c319d988c
Bug 35001: Add some checks to membership_expiry
Test plan:
Toggle pref TrackLastPatronActivityTriggers (clear it).
And try variations of --active, --inactive to see the warnings.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-24 15:44:32 -03:00
Marcel de Rooy a5fc346403
Bug 34983: Force a sort order in JSON output of po2json
This patch does so by default. If you do not want them sorted,
export CANONICAL-0 on the command line.

Test plan:
Copy a staff PO file from misc/translator to test.po
Now run:   time misc/translator/po2json test.po > json1
And do:    time misc/translator/po2json test.po > json2
Run:       diff json1 json2; #They should be the same.
Now:       export CANONICAL=0
And run:   time misc/translator/po2json test.po > json3
And again: time misc/translator/po2json test.po > json4
And run:   diff json3 json4; # Lots of changes
Remove the created cruft. And signoff :)

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-24 10:05:27 -03:00
Marcel de Rooy a939f8da15
Bug 15504: (QA follow-up) Final changes
[1] Comment in Patron->is_active sharpened.
[2] Patron->update_lastseen
      return if $cached && $cached eq $now->ymd;
    Should be
      return $self if $cached && $cached eq $now->ymd;
    Consistent return value.
[3] Typo if there are not TrackLastPatronActivityTriggers enabled
[4] Typo TrackLastPatronActivityTriggers must be contain
[5] Typo patron is does .. option => performs an action
[6] Tidy on dbrev to fix
      The file is less tidy than before (bad/messy lines before: 0, now: 4)

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-24 10:05:23 -03:00
Martin Renvoize b46b614b6c
Bug 15504: Remove PatronLastActivity preference
This patch removes the older PatronLastActivity preference opting to
replace it by the new PatronLastActivityTriggers preference. The feature
can now be disabled by simply not selecting any triggers

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-24 10:05:22 -03:00
Martin Renvoize 01a5d302b9
Bug 15504: (follow-up) Clarify --last_seen option in delete_patrons
This patch adds clarifications to the last_seen option in delete_patrons
to clarify that TrackLastPatronActivity and
TrackLastPatronActivityTriggers system preferences act as a couplet.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-24 10:05:21 -03:00
Laura Escamilla d742f29d77
Bug 34954: Fixed typo to 'dateexpiry'
To test:

1. In KTD use grep -rn --exclude=*.po "datexpiry" *
2. Notice that the four locations below are showing the typo:
    1. git add koha-tmpl/intranet-tmpl/prog/en/modules/tools/import_borrowers.tt
    2. git add misc/release_notes/release_notes_18_05_00.html
    3. git add misc/release_notes/release_notes_18_05_00.md
    4. git add t/db_dependent/Koha/Patrons/Import.t
3. Apply patch. Repeat step 1.
4. Notice that no results show. All instances of the typo have been fixed to dateexpiry.
5. Sign off and have a great day :)

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-20 17:22:02 -03:00
Nick Clemens 5b18fa1fc2
Bug 25393: (QA follow-up) Tidy
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-20 16:31:24 -03:00
Nick Clemens d67ab23373
Bug 25393: (QA follow-up) Report 'too_soon' errors from cron
This patchset made it possible for 'too_soon' to be returned instead of 'auto_too_soon' - we just need to take that into account in the error listing

To test:
1 - Set circ rule to no renewal before 2, no auto renewal before 3
2 - Checkout 3 items with auto renew, due tomorrow, in 2 days, in 3 days
3 - perl misc/cronjobs/automatic_renewals.pl -v
4 - Confirm 1 can be renewed, 1 is too soon, 1 is auto too soon

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-20 16:31:23 -03:00
Marcel de Rooy 709dabb7e8
Bug 32305: Counterpart for es_indexer_daemon.pl
This is a kind of copy-and-paste variant of background worker.
We probably could do better than having two scripts here ;)

Test plan:
See former test plan. Apply it to ES indexing.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Removed queue from query at MQ side. Discussed on IRC.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-20 11:02:56 -03:00
Marcel de Rooy f1edd8fe77
Bug 32305: Worker should ignore wrong status or queue in MQ mode
MQ mode: The worker has seen that MQ is running and consumes messages
instead of polling db only.

Test plan:
Stop worker for long tasks.
Make sure that Rabbit MQ runs.
Stage a file. (This adds a long task.)
Goto staff view of jobs and cancel this job.
Check if job is still in MQ with rabbitmqctl list_queues.
Now start worker for long tasks.
Check if job is gone in MQ with rabbitmqctl list_queues.
And check logfile for the adjusted warning like:
  [WARN] Job 5 not found, or has wrong status/queue main:: /usr/share/koha/misc/workers/background_jobs_worker.pl (134)

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Removed queue from query at MQ side. Discussed on IRC.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-20 11:02:55 -03:00
Jonathan Druart 4b337b107e
Bug 34959: Sort PO files correctly
The PO files are not sorted when we update them which leads to
unnecessary changes that are commited: hard to see differences and
make git index grow superfluously.

Test plan:
0. Do not apply this patch
1. gulp po:update --lang es-ES
2. git commit -a -m"First PO update"
3. Run again the gulp update command
4. git diff
=> You have a lot of changes generated here, the po:update is not
idempotent.

5. Apply this patch
6. Run the gulp update command
7 git commit -a -m"PO update after 34959"
8. Run the gulp update command
9. git diff
=> No changes are generated

Note that this patch will all the entries by files, and per line
numbers.
It fixes a bug in some condition, where we add information/context about
the string. For instance search for "For the first occurrence" in the
file. Prior to this patch this was not correct, we didn't add info about
the first occurrence (but whichever in the list).

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-18 10:25:56 -03:00
Jonathan Druart e4d638e148
Bug 34959: Remove unused sub
Nothing special here, this sub is not used and we can remove it.

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-18 10:25:56 -03:00
Hammat Wele 2f8a24bd58
Bug 32740: Add a new option patron home library to OverdueNoticeFrom
This patch add 'patron home library' option to the OverdueNoticeFrom preference. When it's selected notification messages for overdues will be bundled and sent from the borrowing user's home library

To Test
1. Create 3 libraries B1, B2 and B3
    1.1. Go to Koha administration, click on Libraries
    1.2. Click New library
    1.3. Fill the Library Code (ex. B1), Name(ex. B1) and email(ex. b1@test.com) fields.
2. Create a user belonging to B1
    2.1. Go to patron module, click on New patron and patron
    2.2. Fill the Surname,Card number,Primary email,Card number field and select B1 as Library.
3. Create 2 notices : the first belongs to B2 and the second belong to B3
    3.1. Go to Cataloging module and click on new
    3.2. field the required field (000 subfield 0, 003 subfield 0,  005 subfield 0, 008 subfield 0, 040 subfield c,245 subfield a, and 942 subfield c )
    3.3. Click on save
    3.4. In the item form, Select Home library = B2 and Current library = B2 and type a Barcode.
    3.5. Repeat 3.1 3.2 3.3 3.4 to create a second notice belong to B3
4. Have (or create) a default notice for all branches in Notices and Slips Module
5. Make sure you connected to branch B2 or Switch to branch B2
6. checkout item I2 (belonging to B2) to the user created in step 2 (specifying as due date the day before (1 day overdue))
7. switch to branch B3
8. checkout item I3 (belonging to B3) to the user created in step 2 (specifying as due date the day before (1 day overdue))
9. make sure you have defined 'Overdue notice/status triggers' for that patron category, ie with Delay=1 so that 'Overdue Notice' is selected and 'Email' is checked.
Note: we have a user belong to B1, two item: I2 belong to B2, I3 belong to B3 and the user has borrowed I2 from B2 and I3 from B3.
10. Test
    10.1. Search for OverdueNoticeFrom in System preferences and select item home library
    10.2. from cli run 'perl ./misc/cronjobs/overdue_notices.pl'
        => Two new entries will be generated in message_queue DB table, coming from different branches that have the SAME list of overdue items.
        => The sender address is the item home library address for each entry.
11. Apply the patch
12. Search for OverdueNoticeFrom in System preferences
        => There is now 'patron home library' in the options list
13. Select patron home library
14. from cli run 'perl ./misc/cronjobs/overdue_notices.pl'
        => You will see that only one message had been created. Its content shows both B2 and B3 overdue items
        => The sender address matches B1 address (the patron home library).
15. Search for OverdueNoticeFrom in System preferences and select cron
16. from cli run './misc/cronjobs/overdue_notices.pl --frombranch patron-homebranch'
    => you will get the same result at step 14

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Hinemoea Viault <hinemoea.viault@inlibro.com>

Was failing the qa script, so I perltidied the changed lines.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-11 12:13:04 -03:00
Kyle M Hall fecd7a0a27
Bug 33239: (QA follow-up) Tidy and improve description
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-11 10:31:46 -03:00
Fridolin Somers 5c531ead87
Bug 33239: Add the ability to run borrowers-force-messaging-defaults.pl only on a specified message name
Test plan:
1) Select a patron A
2) Change all the message prefs away from defaults.
3) Run misc/maintenance/borrowers-force-messaging-defaults.pl --doit --message-name Item_due
    Verify that patron A prefs changed only for 'Item due'

Signed-off-by: Sam Lau <samalau@gmail.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-11 10:31:46 -03:00
Jonathan Druart 9e4b0c4c72
Bug 33837: Remove datetime parameter
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Perltidied one line in cleanup_database.pl.

Bug 33837: Perl tidy block in Objects.t

WARN   t/db_dependent/Koha/Objects.t
   WARN   tidiness
                The file is less tidy than before (bad/messy lines before: 319, now: 320)

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-11 10:31:45 -03:00
Kyle M Hall ba298ebdf9
Bug 33204: (QA follow-up) Improve option description
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-10 10:54:06 -03:00
Fridolin Somers 20a3b13516
Bug 33204: Add the ability to filter on patron library for borrowers-force-messaging-defaults.pl
Like filter on category Bug 19454, it would be helpful to filter on a specified library for misc/maintenance/borrowers-force-messaging-defaults.pl

Test plan:
1) Select two patrons A and B in say libraries L1 and L2.
2) Change the msg prefs for A and B away from defaults.
3) Run misc/maintenance/borrowers-force-messaging-defaults.pl -doit -library L1
    Verify that patron A changed and patron B did not.
4) Run with both category and library filters

Signed-off-by: Sam Lau <samalau@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-10 10:54:06 -03:00
Tomas Cohen Arazi 813aaa6969
Bug 34064: Tidy script
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-10 10:54:05 -03:00
David Cook c30dd8d6b0
Bug 34064: Remove no_batch_alters option
This change removes the no_batch_alters option.

When working with real world databases, you really want to use batch
alters to group together changes. Otherwise, the output becomes
too overwhelming/unwieldy.

On real world databases, I'm noticing big discrepancies, although
I think most of the time this is a difference in the collation
at the column level.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-10 10:54:04 -03:00
David Cook 5d3bfc5d5b
Bug 34064: Add an audit script to compare database with kohastructure.sql
This script can take a connected database handle and compare that database
schema against kohastructure.sql to see what changes the database would
need in order to match kohastructure.sql

NOTE: It uses SQL::Translation::Diff, which is installed with DBIx::Class.

WARNING: The diff doesn't seem to compare comments, so that difference
won't appear in the output. If we wanted, we could easily enhance the
audit_database.pl script to also compare comments.

WARNING: The output is a proposed series of SQL commands. While they
are useful to review, they won't always duplicate the changes done
by updatedatabase.pl, so it's important to carefully analyze the output.
The key purpose of this audit script is to just highlight the differences
between the two.

Test plan:
0. Apply patch
1. vi ./installer/data/mysql/kohastructure.sql
2. Comment out some columns, change NULL status, or whatever you like
3. perl misc/maintenance/audit_database.pl \
    --filename /kohadevbox/koha/installer/data/mysql/kohastructure.sql
4. Note that the output includes SQL commands to change the database
to match the new kohastructure.sql

5a. Try using koha-foreach and note that the database name appears above
the database comparison
5b. koha-foreach "perl misc/maintenance/audit_database.pl \
    --filename /kohadevbox/koha/installer/data/mysql/kohastructure.sql"

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-10 10:54:03 -03:00
Martin Renvoize ed6a556df3
Bug 23241: Remove unused koha-index-daemon-ctl.sh
This script was never fully promoted or documented and it requires third
party libraries are installed to use. This patch removes it from the
repository and the Makefile so it is no longer installed.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-10 09:22:37 -03:00
Marcel de Rooy f6119e7503
Bug 28688: (follow-up) Remove redundant options
Test plan:
Run misc/cronjobs/membership_expiry.pl --help
misc/cronjobs/membership_expiry.pl --man

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-10 09:22:32 -03:00
Jonathan Druart bb9f7b5558
Bug 28688: Use pod2usage for incorrect usage
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-10 09:22:31 -03:00
Marcel de Rooy 5588d688fd
Bug 28688: Add renewal to crontab example
Note: I did not add an example to the debian cron daily file, since
we have no commented lines there. And automatic renewal is no default
behavior.

Test plan:
Read new lines in misc/cronjobs/crontab.example. Just comments.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-10 09:22:31 -03:00
Marcel de Rooy 9ab80608f1
Bug 28688: Allow to pass another renewal notice
Just as passing another expiry notice.

Test plan:
Pick a patron to expire.
Create another membership renewal notice, say X.
Run misc/cronjobs/membership_expiry.pl -c -n -v -letter_renew X
Check output on command line.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-10 09:22:30 -03:00
Marcel de Rooy 9022ec16c3
Bug 28688: Add renew option to membership_expiry.pl
Test plan:
[1] Pick an account to expire soon.
[2] Choose MembershipExpiryDaysNotice so that it will be selected.
    (Note: You can extend the selection with -before or -after.)
[3] Run misc/cronjobs/membership_expiry.pl -c -v -renew
[4] Check new expiry date in the notice. Does it correspond with
    enrollment period of category and pref BorrowerRenewalPeriodBase?
[5] If the patron category has an enrollment fee, check account too.
[6] Reset dateexpiry of this patron to original date.
[7] Run misc/cronjobs/membership_expiry.pl -c -v
[8] Check if you got an expiry notice instead and dateexpiry did not
    change.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Removed $fees and $substitute in favor of notice approach

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-10 09:22:29 -03:00
Marcel de Rooy 6296658e86
Bug 33522: (follow-up) Do not rely on fallback
Bug 33245 has been adjusted; the fallback has been removed.
We should check now if the -active or -inactive parameter
actually has been passed.

Test plan:
See earlier test plan.
Run the script without active/inactive.
Run the script with active.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-04 11:07:51 -04:00
Marcel de Rooy d6a2a3c88c
Bug 33522: Add active and inactive parameter to membership_expiry.pl
Test plan:
Enable TrackLastPatronActivity.
Set MembershipExpiryDaysNotice to X days.
Pick a patron, make it expire in X days. Add email address.
Update borrowers.lastseen for this patron to make it active.
Run membership_expiry.pl -c -v -active 6
You should see at least one enqueued notice.
Run membership_expiry.pl -c -v -inactive 6
You should see that it skipped a number of active patrons (>=1).

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Sam Lau <samalau@gmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-04 11:07:50 -04:00
Stefan Berndtsson f2d2a7839e
Bug 29145: use overdues restrict delays when removing overdues restriction upon return
How to test:
1) Run tests in t/db_dependent/Circulation/MarkIssueReturned.t

Sponsored by: Gothenburg University Library

Signed-off-by: Michaela <michaela.sieber@kit.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-03 09:24:37 -04:00
Marcel de Rooy bda428399a
Bug 34728: (QA follow-up) Remove o modifier, bit more compact
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-18 15:31:50 -03:00
Matt Blenkinsop d947ae3c1a
Bug 34728: Make pre tag conditional
This patch makes the pre-formatting of notice templates conditional on the content type.

Test plan:
1) Navigate to tools > Notices and slips
2) Edit the notice with the code of HOLD
3) Under print, paste the following
<p>Please collect within 10 days from the date of this letter and don't forget to bring your library card with you.</p>
<p>Be aware that opening hours vary at different library branches so if in doubt double check the times before you visit.</p>
<p>Should you no longer require this item please contact us so we can make it available for another customer.</p>

<p>Thank you</p>
4) Check the box that says HTML message and save the notice
5) In Patrons, select a patron and in Patron messaging preferences, check the box in the Email column for Hold filled
6) Find a record in the catalogue
7) Copy the barcode for one of the items
8) Place a hold on the item for the patron you selected
9) Navigate to Circulation > Check-in
10) Paste the barcode in the Check-in box and click Check-in
11) In the pop-up box, click Confirm hold
12) In the command line, run perl misc/cronjobs/gather_print_notices.pl --letter_code=HOLD --html /tmp
13) cd /tmp
14) There should be a file in there called 'notices_HOLD-date.html
15) cat this file
16) The html you pasted in the template will be wrapped in a <pre> tag
17) Apply patch and restart_all
18) rm the notices file that was created
19) cd /kohadevbox/koha
20) Select a new record from the catalogue and repeat steps 7-15 for this record, using the same patron
21) This time, the html file should not contain the <pre> tag
22) Sign off!

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-18 15:31:49 -03:00
Matt Blenkinsop 91dd8dd16f
Bug 34743: Fix POD description
Currently --expiration-from-today has the same definition as --update-expiration. The description has now been updated to reflect the fact that it will set the date from today, and not the patron's enrollment date

Test plan:
Look at the patch file and confirm that the POD now mentions today's date rather than the patron's enrollment date

Signed-off-by: Laura Escamilla <laura.escamilla@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-15 11:57:15 -03:00
Martin Renvoize 0e9603bb49
Bug 34764: Set fee_acknowledged to expect a string
For consistency with other options I opted to have the fee acknowledged
parameter to expect a string and as it's an optional parameter I've
dropped the default value of 'N' too.

Test plan
1) Prior to this patch
1a) Attempt a checkout without passing -fa/--fee_acknowledged flag
    `./misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --patron 23529000035676 --item 39999000007756 -m checkout`
    The final field of the SIP request will be '|BON', the default
1b) Attempt a checkout passing -fa/--fee_acknowledged flag
    `./misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --fee-acknowledged --patron 23529000035676 --item 39999000007756 -m checkout`
    The final field of the SIP request will still be '|BON', failure
1c) Attempt a checkout passing a string for fee_acknoewledeged flag
    `./misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --fee-acknowledged Y --patron 23529000035676 --item 39999000007756 -m checkout`
    The final field of the SIP request will still be '|BON', failure
2) Apply the patch
2a) Attempt a checkout without passing -fa/--fee_acknowledged flag
    `./misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --patron 23529000035676 --item 39999000007756 -m checkout`
    The optional `|BO` element should not be present
2b) Attempt a checkout passing -fa/--fee_acknowledged flag
    `./misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --fee-acknowledged N --patron 23529000035676 --item 39999000007756 -m checkout`
    The final field of the SIP request will now be '|BON', success
2c) Attempt a checkout passing a string for fee_acknoewledeged flag
    `./misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --fee-acknowledged Y --patron 23529000035676 --item 39999000007756 -m checkout`
    The final field of the SIP request will now be '|BOY', success

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-15 11:57:10 -03:00
Martin Renvoize 8f7d565df5
Bug 28995: Update aliases
Add an alias for added-before, added-after (Using an alias makes it
backwards compatible) and update the POD.

Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-08 11:54:55 -03:00
Martin Renvoize 0a5f4350bd
Bug 28995: Add `--added_after` filter to writeoff_debts script
This adds, as requested, the `added_after` filter option to
writeoff_debts.

Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-08 11:54:55 -03:00
Tomas Cohen Arazi a707d10d80
Bug 34569: (QA follow-up) Tidy block
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-01 12:08:38 -03:00
Fridolin Somers f4f4feb50b
Bug 34569: Fix misc/cronjobs/holds/holds_reminder.pl trigger arg
misc/cronjobs/holds/holds_reminder.pl help says one can use arge -t for --triggered
But not implemented in code.
Should be like in misc/cronjobs/overdue_notices.pl :
  't|triggered'    => \$triggered,

Test plan :
Play with misc/cronjobs/holds/holds_reminder.pl with arg -t and
--trigger

Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-01 12:08:38 -03:00
Petr Svoboda 7c3a0b9131
Bug 34583: Overdue notice - wrong coding in outlook in czech e-mail in “print” mode
Koha overdue notice “print” email in Outlook is incorrectly encoded in czech language.
Tested in KohaTestingDocker - https://gitlab.com/koha-community/koha-testing-docker
This patch fixes that error.

Test plan:
Overdue notice, set print in UI, https://snipboard.io/FLkIxf.jpg.

Message body:
<table style="padding:0; margin: 0; border-collapse: collapse; font-family: 'Trebuchet MS', Helvetica, sans-serif; width: 100%; "><tr><td style="vertical-align: top; padding: 0"> </td><td style="width: 75mm; vertical-align: top;"><table style="margin-top: 2.5cm; font-size: 11pt;"><tr><td><<borrowers.cardnumber>></td></tr><tr><td><<borrowers.firstname>> <<borrowers.surname>></td></tr><tr><td><<borrowers.address>> <<borrowers.streetnumber>></td></tr><tr><td><<borrowers.city>></td></tr><tr><td><<borrowers.zipcode>></td></tr></table></td></tr><tr><td colspan="2" style="padding: 0;"><table style="width: 100%; margin: 3cm 0 0 0;"><tr><td style="width: 5mm; height: 5mm; border-bottom: 1px dotted #000;"> </td><td style="height: 1mm;"></td><td style="width: 5mm; height: 1mm; border-bottom: 1px dotted #000;"> </td></tr></table></td></tr><tr><td colspan="2"><table><tr><td style="padding: 0.5cm 0.3cm">V Chrudimi dne <<today>></td></tr><tr><td style="padding: 0 0.3cm">Upozorňujeme Vás, že byla již o 30 dnů překročena doba, na kterou jsme Vám půjčili dokumenty (celkem<<count>>). Žádáme Vás o brzké vrácení těchto výpůjček:</td></tr><tr><td style="padding: 0.5cm 0.3cm;"><table><tr><th>Od</th><th>Do</th><th>Název</th><th>Autor</th><th>Částka</th></tr><item><tr><td><<issues.issuedate>></td><td><strong><<issues.date_due>></strong></td><td><<biblio.title>></td><td><<biblio.author>></td><td><<items.fine>> Kč</td></tr></item></table></td></tr><tr><td style="padding: 0 0.3cm">Zpozdné je účtováno dle ceníku knihovny.</td></tr><tr><td style="padding: 0 0.3cm">Dokumenty můžete vrátit na jakémkoli oddělení knihovny a pobočce v jejich provozní době.</td></tr><tr><td style="padding: 0 0.3cm">Nesplní-li čtenář svou povinnost vyrovnat závazky s knihovou, má knihovna právo postupovat podle příslušných ustanovení občanského zákoníku.</td></tr></table></td></tr></table>

Set delay 1 day, https://snipboard.io/boAd3F.jpg.

Checkout any book.

Set in database issues->date_due = today() - 1
for checkouted book, https://snipboard.io/PrylGB.jpg.

Run this scripts:

koha-shell kohadev -c "/kohadevbox/koha/misc/cronjobs/overdue_notices.pl -v -t"

You must have KOHA set up for sending e-mails (SMTP server). Replace xxx@email.cz to your e-mail and run this script:

koha-shell kohadev -c "/kohadevbox/koha/misc/cronjobs/gather_print_notices.pl /var/spool/koha/kohadev/ --email xxx@email.cz"

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-01 12:08:36 -03:00
Marcel de Rooy 4fd847b7ae
Bug 33871: Use Koha object in Sitemapper, add optional filter
Test plan:
Run misc/cronjobs/sitemap.pl -where "biblionumber>X" (replace X
by some clever value)
Verify results by browsing sitemap files.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-09-01 11:23:25 -03:00
Fridolin Somers c26dff61e7
Bug 34505: Patron invalid age in search_for_data_inconsistencies.pl skip expired patrons
Bug 26311 added patron invalid age in search_for_data_inconsistencies.pl
But this is not relevant for expired patrons, especially in child categories.
Check should skip expired patrons

1) Set a non-expired patron with invalid age
2) Run misc/maintenance/search_for_data_inconsistencies.pl
=> Check you see the patron
3) Edit patron to be expired
4) Run misc/maintenance/search_for_data_inconsistencies.pl
=> Check you do not see the patron

Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] If we do not want to test valid age for expired patrons, you should
obviously test expired first. Reversing the order in the test.
Putting it in the search criteria would be nicer but leads to more
complicated code.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-08-29 14:59:08 -03:00
Emily Lamancusa a3c64f62c0
Bug 34470: Initialize random seed after spawning a child worker process
When background_jobs_worker.pl spawns a new child process, it needs to
explicitly reinitialize the random seed - otherwise each child process
will inherit the same random seed from the parent process, and any
randomization will produce identical results each time.

This patch adds a call to srand immediately after the fork to
reinitialize the seed. Note that child processes should not call
srand with no parameter anywhere else, as the Perl documentation
indicates that srand should not be called with no parameter more than
once per process.

To test:
1. Apply the logging patch only
2. Set system preferences:
    a. RealTimeHoldsQueue -> Enable
    b. RandomizeHoldsQueueWeight -> in random order
3. Watch the logs for the staff interface
   in ktd:
   ktd --shell
   koha-intra-err
4. Place a hold. Note that the logs display the branch list before and
   after it is randomized.
5. Place some more holds. Note that the branch order after randomization
   is identical each time.
6. Apply both patches and restart_all
7. Repeat steps 3-5.
   -> Note that the branch order before randomization hasn't changed
   -> Note that the branch order after randomization is now different
      each time.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-08-15 11:25:48 +03:00
Pedro Amorim 39e6b100cb
Bug 34059: Add only issues from the branch that is creating the notice
Test plan, on k-t-d
1) Go to 'my account' on top right user menu
2) On 'Patron messaging preferences', click 'Edit'
3) On the 'Item due' row, check the 'Email' and 'Digests only' checkboxes and save
4) On the top search bar, press 'Check out' and enter '42' (koha user cardnumber)
5) On the checkout input bar, enter 39999000001372 and press checkout
7) Go to 'Set library' on top right user menu and pick a different library
8) Repeat step 4), then, on the checkout input, enter 39999000004571 and press checkout
9) Verify that this user now has 2 items checked out, from 2 different libraries at /cgi-bin/koha/circ/circulation.pl?borrowernumber=51
9) Run the following 2 queries to force the due_date to be equal to 'today's' date for both issues:
NOTE: change the YYYY-MM-DD below to whatever day it is you're running this test plan

UPDATE issues SET date_due = '2023-06-19 23:59:00' where issue_id = 1;
UPDATE issues SET date_due = '2023-06-19 23:59:00' where issue_id = 2;

10) Run the cronjob:
./koha/misc/cronjobs/advance_notices.pl -c --digest-per-branch

11) Verify that two DUEDGEST notices were created, one per each library, but both notices contain both issues:
SELECT letter_code, time_queued, content FROM message_queue ORDER BY message_id DESC LIMIT 2;

12) Apply patch, then do 10) and 11) again
13) Verify that each notice only contains the issue for its respective library

Signed-off-by: Sam Lau <samalau@gmail.com>
Signed-off-by: Stephen Graham <s.graham4@herts.ac.uk>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-08-15 11:25:39 +03:00
Jonathan Druart 2442a4537d
Bug 34213: import_patrons.pl - Fix short version of matchpoint option in POD
44     'c|confirm'                      => \$confirm,

 46     'm|matchpoint=s'                 => \$matchpoint,

The POD is wrong.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-07-21 17:14:40 -03:00
Marcel de Rooy dfe1c36704
Bug 33444: (QA follow-up) Tidy block in automatic_renewals.pl
Resolves:
The file is less tidy than before (bad/messy lines before: 94, now: 101)

Test plan:
Run qa tools.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-07-19 12:06:53 -03:00
Nick Clemens 7afbba200d
Bug 33444: Update AddRenewal to take a hashref of params
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Removed skip_record_index => 1 from automatic_renewals.pl. See BZ.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-07-19 12:06:52 -03:00
Aleisha Amohia 0d2052088e
Bug 33992: Only consider the date when auto-expiring problematic recalls
This patch carries this fix into the misc/cronjobs/recalls/expire_recalls.pl cronjob so that recalls are automatically expired when they have been waiting a problematic number of days, not considering hours, as expected.

To test, follow the test plan from the first patch. This will set you up with a waiting problematic recall.

Run the cronjob manually

perl misc/cronjobs/recalls/expire_recalls.pl

Refresh your 'Recalls awaiting pickup' page. Your problematic recall should be gone/expired.

Expiration dates will apply when expiring any 'unfulfilled' recall i.e. newly requested, overdue to be returned, and awaiting pickup.

Sponsored-by: Auckland University of Technology
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-07-19 12:06:50 -03:00
Michael Hafen 7e1104da83
Bug 18855: Don't create duplicate overdue fines if an issue is returned
This includes Jonathan's followup

If cronjobs/fines.pl is running during circulation hours, then an issue may
be considered for having it's overdue fine updated after the issue has been
returned and it's fine status flipped from 'UNRETURNED' to 'RETURNED'.  In
this case UpdateFine will create a duplicate fine because it can't find the
specific accountline for the (formerly) overdue issue.

This changes cronjobs/fines.pl to double check the issue before updating
the fine.  If the issue has changed between starting the script and updating
the fine, then the script will skip it.

There is a small amount of time between the check and calling UpdateFine
where the issue can be changed and this problem can reoccure.  The chance
of that happening is so small that it's probably fine to leave as is.

It is also possible that the fine won't be updated because the issue was
returned.  In this case the fine payed by the patron will be lower, but that
is better then the patron finding later that there is more to a fine they
thought they had paid all of.

Test plan (by Caroline):
0. Preliminary settings
   0.1. finesMode system preference must be set to Calculate and charge
   0.2. There must be a circulation rule that will charge fines (beware of bug 32271)
   0.3. In Tools > Calendar, today must not be a holiday

1. Make a lot of overdue checkouts - I used the batch checkout feature, but if your system already has a lot of overdue checkouts, you can skip to step 2

   1.1. Enable batch checkouts
        1.1.1. Go to Administration > Global system preferences
        1.1.2. Search for BatchCheckouts
        1.1.3. Set BatchCheckouts to Allow
        1.1.4. Select all categories in BatchCheckoutsValidCategories
        1.1.5. Click "Save all Circulation preferences"

   1.2. Get a list of barcodes
        1.2.1. Go to Reports
        1.2.2. Click "Create from SQL"
        1.2.3. Give the report a name
        1.2.4. For the SQL query, enter

               SELECT barcode FROM items WHERE onloan IS NULL LIMIT 60;

	1.2.5. Click "Save report"
	1.2.6. Click "Run report"
	1.2.7. Click "Download" > "Tab separated text"

   1.3. Go to a patron's file
        1.3.1. Go to Patrons
        1.3.2. Click on "Search"
        1.3.3. Click on a patron's name

   1.4. Do a batch checkout with a due date in the past
        1.4.1. Click on the "Batch check out" tab on the left
        1.4.2. In "Use a file", click "Choose file"
        1.4.3. Choose the file downloaded from the report
        1.4.4. In "Hard due date", choose a date in the past
        1.4.5. Click "Check out"
        1.4.6. Click "Checkout or renew"

2. Find the last issue in the database

   2.1. In the database (or in reports), type the following query

        SELECT issues.*, items.itype as itemtype, items.homebranch, items.barcode, items.itemlost, items.replacementprice, items.biblionumber FROM issues  LEFT JOIN items USING (itemnumber) WHERE date_due < NOW() \G;

   2.2. Copy the barcode from the last entry

3. Set up so that you can run fines.pl and check in the item at the same time (or very close to the same time)

   3.1. In Koha, click the "Check in" option in the search bar at the top of the page
   3.2. Paste the barcode in the search bar BUT DO NOT PRESS ENTER OR THE ARROW RIGHT AWAY
   3.3. In a terminal, enter the fines.pl command

        ./misc/cronjobs/fines.pl

   3.4. Execute the command and immediately click on the arrow in the staff interface to check in the item

4. Check the patron's fines

    4.1. Click on the patron's name in the check in screen
    4.2. Go to the Accounting tab on the left
    4.3. In the search box just above the table, paste in the returned item's barcode

    --> Without the patch, there are two fines, one Fine (Accruing) and one Fine (Returned) for the same item at the same time
    --> With the patch, there is only one fine, Fine (Returned)

Signed-off-by: Sam Lau <samalau@gmail.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-07-06 10:08:44 -03:00
Laura Escamilla edd65cbfaf
Bug 33698: Add fields to the verbose output that show the barcode and title of items that cannot be deleted
Signed-off-by: Sam Lau <samalau@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-06-09 08:50:14 -03:00
Pedro Amorim fac7bb247a
Bug 33900: Fix handling of due_digest
Test plan, on k-t-d
1) Go to 'my account'
2) On 'Patron messaging preferences', click 'Edit'
3) On the 'Item due' row, check the 'Email' and 'Digests only' checkboxes and save
4) On the top search bar, press 'Check out' and enter '42' (koha user cardnumber)
5) On the checkout input bar, enter a barcode e.g. 39999000001372 and press checkout
6) Run the following query to force the due_date to be equal to 'todays' date:
NOTE: change the YYYY-MM-DD below to whatever day it is you're running this test plan

UPDATE issues SET date_due = '2023-06-05 23:59:00' where issue_id = 1;

7) Run the cronjob:
./koha/misc/cronjobs/advance_notices.pl -c --digest-per-branch

8) Notice it hangs, it's stuck on an infinite loop on the 2nd while cycle after if ()
9) Cancel the cronjob script, apply patch
10) Run the cronjob script again (step 7), notice it finishes
11) Access patron's notices:
/cgi-bin/koha/members/notices.pl?borrowernumber=51

Check that the 'Item due reminder' notice is there.

Signed-off-by: Sam Lau <samalau@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-06-08 08:32:39 -03:00
Tomas Cohen Arazi 2eca0fe363
Amend release team
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-31 21:09:08 -03:00
Tomas Cohen Arazi dd0d35be1e
Translation fixes
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-31 16:10:07 -03:00
Koha translators edd378f4d1
Translation updates for Koha 23.05.00
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-31 16:09:54 -03:00
Tomas Cohen Arazi 56c149c31b
Add release notes for 23.05.00
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-31 16:09:50 -03:00
Julian Maurice ac86c2ab6d
Bug 23773: Send membership expiry notices by sms too
Test plan:
1. Verify in "About" page that SMS::Send is installed.
   If not, install it.
2. Set system preferences:
   - SMSSendDriver = 'Test'
   - MembershipExpiryDaysNotice = 30
3. In "Notices and Slips" tool, edit the 'MEMBERSHIP_EXPIRY' letter and
   verify that both email and sms parts are filled. Use a different
   title and body for the sms part
4. Create a new borrower with an email address and an SMS number. Set
   their expiry date to a date next week
5. Run misc/cronjobs/membership_expiry.pl --confirm
6. Verify in the patron "Notices" tab that two notices are pending, one
   for email, one for sms

Sponsored-by: Médiathèque de Montauban
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-18 11:35:14 -03:00
Caroline Cyr La Rose 27a086b91e
Bug 33717: Typo in search_for_data_inconsistencies.pl
This patch corrects a typo in the output of
search_for_data_inconsistencies.pl when a bibliographic record has no
title.

The patch also replaces biblio to bibliographic record in the same
sentence for terminology consistency.

To test:
- Have a bibliographic record without a title
- Run misc/maintenance/search_for_data_inconsistencies.pl
- Read output, make sure the sentence is correct

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-15 08:53:47 -03:00
David Gustafsson ddc2906b77
Bug 31735: Avoid re-fetcing objects from database by passing them directly instead of ids to various subroutines
To test:

1) Run the following test and make sure all pass:
  t/db_dependent/api/v1/biblios.t
  t/db_dependent/api/v1/checkouts.t
  t/db_dependent/api/v1/return_claims.t
  t/db_dependent/Circulation/CalcDateDue.t
  t/db_dependent/Circulation/CheckIfIssuedToPatron.t
  t/db_dependent/Circulation/dateexpiry.t
  t/db_dependent/Circulation/GetPendingOnSiteCheckouts.t
  t/db_dependent/Circulation/GetTopIssues.t
  t/db_dependent/Circulation_holdsqueue.t
  t/db_dependent/Circulation/IsItemIssued.t
  t/db_dependent/Circulation/issue.t
  t/db_dependent/Circulation/MarkIssueReturned.t
  t/db_dependent/Circulation/maxsuspensiondays.t
  t/db_dependent/Circulation/ReturnClaims.t
  t/db_dependent/Circulation/Returns.t
  t/db_dependent/Circulation/SwitchOnSiteCheckouts.t
  t/db_dependent/Circulation.t
  t/db_dependent/Circulation/TooMany.t
  t/db_dependent/Circulation/transferbook.t
  t/db_dependent/DecreaseLoanHighHolds.t
  t/db_dependent/Holds/DisallowHoldIfItemsAvailable.t
  t/db_dependent/HoldsQueue.t
  t/db_dependent/Holds/RevertWaitingStatus.t
  t/db_dependent/Illrequests.t
  t/db_dependent/ILSDI_Services.t
  t/db_dependent/Items.t
  t/db_dependent/Koha/Account/Line.t
  t/db_dependent/Koha/Acquisition/Order.t
  t/db_dependent/Koha/Biblio.t
  t/db_dependent/Koha/Holds.t
  t/db_dependent/Koha/Items.t
  t/db_dependent/Koha/Item.t
  t/db_dependent/Koha/Object.t
  t/db_dependent/Koha/Patrons.t
  t/db_dependent/Koha/Plugins/Circulation_hooks.t
  t/db_dependent/Koha/Pseudonymization.t
  t/db_dependent/Koha/Recalls.t
  t/db_dependent/Koha/Recall.t
  t/db_dependent/Koha/Template/Plugin/CirculationRules.t
  t/db_dependent/Letters/TemplateToolkit.t
  t/db_dependent/Members/GetAllIssues.t
  t/db_dependent/Members/IssueSlip.t
  t/db_dependent/Patron/Borrower_Discharge.t
  t/db_dependent/Patron/Borrower_PrevCheckout.t
  t/db_dependent/Reserves/GetReserveFee.t
  t/db_dependent/Reserves.t
  t/db_dependent/rollingloans.t
  t/db_dependent/selenium/regressions.t
  t/db_dependent/SIP/ILS.t
  t/db_dependent/Holds.t
  t/db_dependent/Holds/LocalHoldsPriority.t
  t/db_dependent/Holds/HoldFulfillmentPolicy.t
  t/db_dependent/Holds/HoldItemtypeLimit.t
  t/db_dependent/Circulation/transferbook.t
2) Performe one or more checkouts for a patron, making sure
  that the circulation rules allows for renewals (for example by
  setting an earlier due-date).
3) Log in as this patron in OPAC and make sure the list of
  checkouts is displayed correctly, and that renewing an issue
  still works.

Sponsored-by: Gothenburg University Library
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-12 12:40:21 -03:00
Jonathan Druart d3995bf08a
Bug 23924: Reset the fields to add
We do not want to copy fields from the previous records!

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-05 12:13:53 -03:00
Jonathan Druart c04accc95c
Bug 23924: Add --date-field to add_date_fields_to_marc_records.pl
The date used in the script will use this parameter, instead of today.

Test plan:
Use the POD of the script to understand how this flag works. Then use
the script to create fields with a date contained in a specific MARC
field.

Signed-off-by: Hugo Agud <hagud@orex.es>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-05 12:13:52 -03:00
Nick Clemens daa92a9371
Bug 33626: Remove invalid params
This patch reformats the params passed to scroll_helper as defined here:
https://metacpan.org/pod/Search::Elasticsearch::Client::7_0::Scroll

To test:
1 - perl misc/maintenance/compare_es_to_db.pl
2 - It dies:
[Param] ** Unknown param (scroll_in_qs) in (search) request. , called from sub Search::Elasticsearch::Client::7_0::Direct::scroll_helper at misc/maintenance/compare_es_to_db.pl line 55.
3 - Apply patch
4 - Repeat
5 - It succeeds!

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-05 10:18:56 -03:00
Magnus Enger 48a2952032
Bug 33649: Fix use of cronlogaction
C4::Log::cronlogaction() takes a hashref as argument, with "info"
and possibly "action" as keys. But there are a couple of places
where it is called with just a string as argument, and that does
not work. Both places need lock_exec to fail to trigger the error.
I have seen this on a production server, but not been able to
reproduce in ktd.

To test:
- Run this on the Koha repo: grep -r "cronlogaction(" *
- Verify that fines.pl and process_message_queue.pl are the only
  scripts that call cronlogaction without a hashref as argument,
  but do it like this: cronlogaction( $message );
- Apply this patch
- Run the grep again and verify that all calls to cronlogaction
  now take a hashref as argument

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-05 10:18:37 -03:00
Nick Clemens a3a58b93c3
Bug 32250: (follow-up) Remove use of AutoCommit=0
This patch moves to using txn_begin and txn_commit

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 32250: (follow-up) Remove one more dbh commit and don't start a new transaction when done

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-05 09:13:23 -03:00
Nick Clemens bfc1e3c226
Bug 32250: (follow-up) Commit background job to the DB
We can't run a background job if it isn't in the database, however,
this script runs with AutoCommit disabled. We need to enable it while
generating the background job, then disable for the updates.

I don't nkow if using a transaction would be preferable.

I tried to solve this independently, but it requires consolidating the
index requests to make this work easily

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-05 09:13:23 -03:00
Nick Clemens cacc16c659
Bug 32250: Only reindex records when commiting
This patch adds an array to catch updated bibs, and defers indexing until a
batch of changes is committed

To test:
1 - Set LinkerModule system preference to either first or last match
    Alternate this between runs of the linker to ensure changes are made
2 - Set SearchEngine to Elasticsearch and reindex (to ensure index is built)
3 - perl  misc/link_bibs_to_authorities.pl -v
4 - Check Admin->Jobs and see that many ES index jobs are queued
5 - Apply patch
6 - perl  misc/link_bibs_to_authorities.pl -v
7 - Check Admin->Jobs and see that 1 index per commit from is enqueued

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-05 09:13:22 -03:00
Nick Clemens d29e5c6e0f
Bug 32250: Linking authorities does not change holdability
If the only change is a linked heading, we don't need to rebuild
the holds queue

To test:
1 - Set preference RealTimeHoldsQueue to enable
2 - Run link_bibs_to_authorities
3 - Note holds queue jobs generated
4 - Apply patch
5 - repeat
6 - No holds queue updates

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-05 09:13:21 -03:00
Fridolin Somers cc1695ba96
Bug 33603: Fix misc/maintenance/search_for_data_inconsistencies.pl when biblio.biblionumber on control field
Since bug 29486, misc/maintenance/search_for_data_inconsistencies.pl search for biblio.biblionumber in MARC record with $record->subfield().
This fails when field is a control field (< 10).
Idem for biblioitems.biblioitemnumber

Test plan :
1.0) On a UNIMARC database (biblio.biblionumber is on 001)
1.1) Run misc/maintenance/search_for_data_inconsistencies.pl
=> Without patch you get error : Control fields (generally, just tags below 010) do not have subfields, use data()
=> With patch no error
2.0) On a MARC21 database (biblio.biblionumber is on 999c)
2.1) Run misc/maintenance/search_for_data_inconsistencies.pl
=> Check you see no error

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-05-05 09:13:15 -03:00
Jonathan Druart 043822ce8f
Bug 32334: Fix spelling
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-12 11:22:04 -03:00
Marcel de Rooy 3b4aab475f
Bug 32334: (follow-up) Improvement for schema parameter
Essentially only adds a warn, some cosmetic changes too.

Test plan:
Copy your kohastructure to xx.sql.
Run sync_db_comments.pl -schema xx.sql. You will see usage.
Run sync_db_comments.pl -schema xyz.sql. You will see a warn and
the usage statement.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-12 11:22:03 -03:00
Marcel de Rooy 6a706f0e6a
Bug 32334: (QA follow-up) Improve finding schema file
Adds a schema parameter to the cmdline script now too.

Test plan:
Run sync_db_comments.pl with -schema file where file does not exist.
(On dev install) rename kohastructure.sql, try with[out] referring
to it using -schema. You could also use the standard path
intranet/cgi-bin/installer/data/mysql.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-12 11:22:01 -03:00
Marcel de Rooy a59c530786
Bug 32334: (QA follow-up) Remove fh and verbose parameter in favor of messages
The module returns messages. The script can print them in verbose
mode. Test script adjusted accordingly.

Test plan:
Run t/db_dependent/Koha/Database/Commenter.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-12 11:22:01 -03:00
Marcel de Rooy 6767b4e240
Bug 32334: (QA follow-up) Remove unused modules
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-12 11:22:00 -03:00
Marcel de Rooy 0fc078b05f
Bug 32334: (QA follow-up) Invert dry run mode to commit parameter
Note: This is only done (and 'needed') for the command line, not
for the module subroutines.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-12 11:21:59 -03:00
Marcel de Rooy 7967dab028
Bug 32334: (QA follow-up) Force mutual exclusiveness of cmdline flags
Test plan:
Run sync_db_comments.pl --clear --renumber

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-12 11:21:58 -03:00
Marcel de Rooy be83a52298
Bug 32334: Add sync_db_comments script
Test plan:
[1] Backup your database, if not done already.
[2] Check output of dry_run when clearing a table:
    misc/maintenance/sync_db_comments.pl -clear -table items -dry
[3] Save output of misc/devel/update_dbix_class_files before changing
    comments in order to compare later. (Commit your changes.)
    You may not have changes after running (at least on a fresh
    database). That's fine.
[4] Clear all comments:
    misc/maintenance/sync_db_comments.pl -clear
[5] Renumber all comments:
    misc/maintenance/sync_db_comments.pl -renum
[6] Reset all comments to schema. Make sure that script finds your
    structure in installer/data/mysql folder.
    misc/maintenance/sync_db_comments.pl -reset
[7] Run update_dbix_class_files again and inspect changes as compared
    to previous run.
    Can you explain them? You should only see changes related to
    column comments. If you did not have changes in step 3, you
    should not have them here too.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-12 11:21:57 -03:00
Matt Blenkinsop 0fe026c6fa
Bug 32013: Autorenewal batch indexing
The automatic_renewals.pl cron script currently loops through items for automatic renewal and calls the indexer for each one individually. skip_record_index has now been added as a parameter to the AddRenewal function to skip the indexing process. The item numbers are now added to an array and then the indexer is called once from within automatic_renewals.pl and passed the array to queue one indexing job instead of multiple jobs.

Test plan:
1) AddRenewal uses Koha::Items->store() to trigger the indexing process. Run prove -vv t/db_dependent/Koha/SearchEngine/Indexer.t and check tests 5,6,29,30. These tests prove whether passing skip_record_index to store() triggers or skips the indexing process. All four tests should pass to show that skip_index_records can prevent the indexing being triggered.
2) Add multiple renewals that are able to be autorenewed and run the automatic_renewals.pl script. There should be multiple items queued in zebraqueue.
3) Apply patch and try again
4) There should now only be one job queued in zebraqueue

Mentored-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-12 09:26:56 -03:00
Nick Clemens 37bf88d252
Bug 32800: Don't embed items for deleted records
To test:
1 - Delete a biblio
2 - perl /kohadevbox/koha/misc/migration_tools/build_oai_sets.pl -v -i -r
3 - Error:
    Can't call method "items" on an undefined value at /kohadevbox/koha/Koha/Biblio/Metadata.pm line 163.
4 - Apply patch
5 - Repeat
6 - Success!

Signed-off-by: Magnus Enger <magnus@libriotech.no>
Works as advertised.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-12 09:26:50 -03:00
Alex Buckley f1ff6c5764
Bug 32686: Specify action of action_logs to purge
Add --log-action parameter to cleanup_database.pl

Test plan:
1. Apply patch
2. Enable cataloguing and borrowers log
3. Make some changes to borrowers, create some borrowers, and edit some
biblio records
4. Change the action_logs.timestamp for all action_logs entries to 367
days ago
5. Run cleanup_database.pl with --logs 365 --log-module=MEMBERS
--log-action=CREATE --confirm
6. Confirm only the borrowers creation action_logs entries are removed
7. Run cleanup_database.pl with --logs 365
8. Confirm all action_logs entries are removed

Sponsored-By: Toi Ohomai Institute of Technology, New Zealand
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-06 10:03:08 -03:00
Kevin Carnes 5fdc436119
Bug 31640: Fuzzy translations of preferences can cause missing sections and inaccurate translations
This patch ignores fuzzy translations for preferences and warns if there are multiple sections with the same translated name.

Test Plan:
1)  Install English United Kingdom translations (./misc/translator/translate install en-GB)
2)  Go to Koha administration in the staff interface
3)  Click Global system preferences
4)  Select I18N/L10N preferences
5)  Enable English United Kingdom in the language preference for staff interface
6)  Save all I18N/L10N preferences
7)  Return to Koha administration
8)  Select English United Kingdom as the language at the bottom of the screen
9)  Click on Global system preferences
10) Select Circulation
11) Observe that there is only SelfCheckInMainUserBlock or StockRotation, but not both
12) Apply the patch
13) Install English United Kingdom translations (./misc/translator/translate install en-GB)
14) Go to Koha administration
15) Select English United Kingdom as the language at the bottom of the screen
16) Click on Global system preferences
17) Select Circulation
18) Observe that SelfCheckInMainUserBlock and StockRotation are both present

Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-06 09:29:27 -03:00
Fridolin Somers 6221538b6d
Bug 33285: (QA follow-up) add POD and fix some code style
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-06 09:29:24 -03:00
Nick Clemens bad0d52245
Bug 33285: Allow specifying the delimeter for runreport.pl
To test:
1 - Write a report in koha
2 - perl misc/cronjobs/runreport.pl --format csv 1 (or correct report number)
3 - Note you get commas
4 - Apply patch
5 - Repeat #2 - no change
6 - perl misc/cronjobs/runreport.pl --format csv --separator "|" 1
7 - Now it is pipe delimited
8 - perl misc/cronjobs/runreport.pl --format tsv --separator "|" 1
9 - Error is reported, you cannot set separator unless csv
10 - Try other separators

Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-04-06 09:29:23 -03:00
Tomas Cohen Arazi 10d12f999f
Bug 33341: Address some perlcritic errors in 5.36
Some old-style code is making our tests fail when run in Debian Testing.

This patch addresses this.

To test:
1. Launch bookworm KTD:
   $ KOHA_IMAGE=master-bookworm ktd up -d
2. Run:
   $ ktd --shell
  k$ prove t/00-testcritic.t
=> FAIL: It fails!
3. Apply the patch
4. Repeat 2
=> SUCCESS: Tests now pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-28 14:50:33 +02:00
Jonathan Druart d8721bbc36
Bug 32594: Mark jobs as started and finished
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-28 14:49:44 +02:00
Nick Clemens 8e00746095
Bug 33108: (follow-up) Don't die, only warn if not using ES
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-20 09:39:47 -03:00
Magnus Enger 6d1890fc8e
Bug 29354: Make overdue_notices.pl send .html
If you have EmailOverduesNoEmail = Send and specify "--html somedir",
overdue_notices.pl will send a file by email that contains partial
HTML, as a file called attachment.txt. This patch fixes that.

To reproduce in koha-testing-docker:
- EmailOverduesNoEmail = Send
- Make sure you have a loan that was due yesterday, by backdating the
  due date
- Set up an overdue action to send "Overdue Notice" (ODUE) to the
  category you made the loan to above, when a loan is one day overdue
- Run this command:
  $ sudo koha-shell -c "perl misc/cronjobs/overdue_notices.pl -v -t -html /tmp/" kohadev
- Look at the file /tmp/notices-<DATE>.html and make sure it is a full
  HTML document, with <html>, <head>, <body> etc.
- Create a report like this:
  SELECT message_id, letter_id, borrowernumber, subject, CONCAT( '<pre>', content, '</pre>' ) AS content,
    metadata, letter_code, message_transport_type, time_queued, updated_on, to_address, content_type, failure_code
  FROM message_queue
  WHERE subject = 'Overdue Notices'
  ORDER BY message_id DESC
- Run the report and verify there is a line like this in the "content"
  of the newest message:
  Content-Type: text/plain; name=attachment.txt
- A part of the "content" will be a block of several lines of gibberish
  (base64) that look something like "RGVhciAga29oYSwNCg0KQWN...". Copy
  this block of text to somewhere like base64decode.org and decode the
  text. You should see a fragment of HTML, without <html>, <head>,
  <body> etc.

To test:
- Apply the patch
- Run overdue_notices.pl again, with the same arguments as above
- Make sure /tmp/notices-<DATE>.html is still a full HTML document
- Re-run the report, and make sure you now have this in the "content":
  Content-Type: text/html; name=attachment.html
- Decode the base64 and make sure it is now a full HTML document, with
  <html>, <head>, <body> etc.
- Re-run overdue_notices.pl as above, but replace "-html /tmp/" with
  "-csv /tmp/test.csv"
- Make sure /tmp/test.csv and the decoded base64 from the report
  contains CSV data
- Re-run overdue_notices.pl as above, but replace "-html /tmp/" with
  "-text /tmp/"
- Make sure /tmp/notices-<DATE>.txt and the decoded base64 from the
  report contains no HTML

Note:
- The actual text from the different messages will be enclosed in
  <pre>-tags
- If you have HTML in your ODUE message template and run with -v, you
  will have warnings saying "The following terms were not matched and
  replaced"
These are due to Bug 14347, and are not adressed by the current patch.

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-14 08:49:31 -03:00
Marcel de Rooy 3bf4addb24 Bug 32558: (QA follow-up) Leave default to 1, remove extra fork
The default of 1 resembles the old behavior: 1 fork for the job.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-06 13:57:53 -03:00
Kyle M Hall 311d68815f Bug 32558: Add ability for background_jobs_worker.pl to process multiple jobs simultaneously up to a limit
Right now background_jobs_worker.pl only processes jobs in serial. It would make sense to handle jobs in parallel up to a user definable limit.

Test Plan:
1) Apply this patch
2) Stop background_jobs_worker.pl
3) Generate some background jobs by editing records, placing holds, etc
4) Watch processes in a new terminal: watch -n 0.1 'ps aux | grep background_jobs_worker.pl'
5) Run background_jobs_worker.pl with parameter -m 3 or some other
   number of max processes
6) Note the multiple forked processes in the ps output

Test notes - also tested the following on KTD:
1. Stop background_jobs_worker.pl
2. Edit /etc/koha/sites/kohadev/koha-conf.xml - set max_processes to 10
3. Generate some background jobs
4. Watch processes in a new terminal: watch -n 0.1 'ps aux | grep background_jobs_worker.pl'
5. Restart all
6. Confirm multiple forked processes in the ps output

Both methods work as expected and generate multiple forked processes
based on the value set for max processes.

Signed-off-by: emlam <emily.lamancusa@montgomerycountymd.gov>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-06 13:57:53 -03:00
Fridolin Somers 45323886ae
Bug 32678: Add new line in authorized values tests in search_for_data_inconsistencies.pl
In search_for_data_inconsistencies.pl, the test for authorized values is a list in one line :
	* The Framework *VR* is using the authorised value's category *LOC*, but the following items.location do not have a value defined ({itemnumber => value }):
 {94 => AV} {95 => AV} {96 => AV} {97 => AV} {98 => AV} {99 => AV} {100 => AV} {101 => AV} {102 => AV} {103 => AV}

It would be more clear with new lines, especially for scripts (grep, awk ...) :
	* The Framework *VR* is using the authorised value's category *LOC*, but the following items.location do not have a value defined ({itemnumber => value }):
 {94 => AV}
 {95 => AV}
 {96 => AV}
 {97 => AV}
 {98 => AV}
 {99 => AV}
 {100 => AV}
 {101 => AV}
 {102 => AV}
 {103 => AV}

Test plan :
1) In koha-testing-docker
2) Delete in authorized values LOC the value AV
3) Run misc/maintenance/search_for_data_inconsistencies.pl
=> You see the new line in result

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-06 09:51:50 -03:00
Victor Grousset/tuxayo e6e587a821
Bug 32518: (QA follow-up) Fix alignment
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-02 14:46:17 -03:00
Nick Clemens 406d61cbb2
Bug 32518: Add reason option to cancel_unfilled_holds cronjob
This adds a reaosn parameter and passes it into the cancellation if supplied

To test:
1 - Place a hold for a patron in your system
2 - Run script with --days 0 -v
3 - verify that it would cancel the reserves (and that you are okay with cancelling the ones it found)
4 - Make sure you have a notice in the holds module with code 'HOLD_CANCELLATION'
5 - Set content of the notice like:
[% IF hold.cancellation_reason=='too_old' %]
Canceled old
[% END %]
6 - Run script with --days 0 -v --reason too_bad -c
7 - Confirm hold cancelled, no notice sent to patron
8 - Place another hold
9 - Run script with --days 0 -v --reason too_old -c
10 - Confirm hold cancelled, notice sent to patron

Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-02 14:46:16 -03:00
Kyle Hall 611a77ca51
Bug 30069: (QA follow-up) Use pure DBIC for select and delete with single query
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-02 12:00:22 -03:00
Matt Blenkinsop 6b90fa3ec4
Bug 30069: (QA follow-up) Rebase and add filter
This patch now checks the status of messages and ignores any message
with a status of 'new'. It is also rebased to account for changes made
to cleanup_database.pl in bug 17350.

Test plan:
1) Ensure you have some EDI orders or even just some dummy messages in the edifact_messages table with a mixture of statuses including 'new'
2) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose (Change the number of days according to the data in your table)
3) The response should show a number of messages that would have been deleted
4) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose --confirm
5) The response should now show the same number of messages have been deleted
6) Check your edifact_messages table to confirm that the data has been deleted
7) Confirm that no messages marked 'new' have been deleted

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-02 12:00:21 -03:00
Katrin Fischer ee91389edd
Bug 30069: (QA follow-up) Fix capitalization for edifact
Changes a few occurrences of edifact in the output messages
to EDIFACT.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-02 12:00:20 -03:00
Matt Blenkinsop 2b08584255
Bug 30069: Add edifact-messages to cleanup_database.pl
This patch allows users to clear out old edifact_messages using the cleanup_database script. The number of days can either be set in the CLI or the default value of 365 can be used.

Test plan:
1) Ensure you have some EDI orders or even just some dummy messages in the edifact_messages table
2) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose (Change the number of days according to the data in your table)
3) The response should show a number of messages that would have been deleted
4) Run perl misc/cronjobs/cleanup_database.pl --edifact-messages 100 --verbose --confirm
5) The response should now show the same number of messages have been deleted
6) Check your edifact_messages table to confirm that the data has been deleted

Sponsored-by: PTFS Europe
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-02 12:00:19 -03:00
Kyle M Hall b3c945dd09
Bug 31453: Add ability to filter messages to process using process_message_queue.pl via a command line parameter
It would be very useful to be able to tell process_message_queue.pl to skip processing some messages. This is particularly useful where a plugin handles sending some message using the before_send_messages hook, but while that plugin is processing, more messages meant for the plugin might be queued. At that point, control moves back to the script and SendQueuedMessages is called, and those messages end up being processed there instead of by the plugin.

Test Plan:
1) Apply this patch
2) Queue two messages, each with a unique word
3) Run process_message_queue --where "content NOT LIKE '%WORD%'"
   where WORD is a unique word in one of the two message
4) Note the message containing "WORD" was not processed
5) prove t/db_dependent/Letters.t

Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-02 09:37:44 -03:00
Tomas Cohen Arazi a3087d76bd
Bug 32594: (QA follow-up) Fix POD
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-03-02 09:37:32 -03:00
Katrin Fischer 41c439bf41
Bug 17350: Add note about how the saved reports data is created in first place
This amends the documentation of the cleanup_database.pl script
to include a hint for how the saved reports data is created.

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-27 13:08:06 -03:00
Andrew Fuerste-Henry fe6a03c43a
Bug 17350: Purge old saved reports via cleanup_database
To test:
apply patch
1 - have or create a report
2 - run your report via the command line with --store-results. Do this twice.
3 - update the saved_reports table to set date_run for one of your two saved results set to a datetime more than 5 days ago
4 - perl misc/cronjobs/cleanup_database.pl --reports 5 --verbose --confirm
5 - Koha tells you its deleting saved reports data from more than 5 days ago
6 - confirm in the database and the staff interface that it's done so

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-27 13:08:05 -03:00
Nick Clemens aba2453ad6
Bug 32594: Add a dedicated Elasticsearch biblio indexing background queue
Currently we generate large numebrs if single record reindex for circulation and other
actions. It can take a long time to process these as we need to load the ES settings for each.

This patch updates the Elasticsearch background jobs to throw records into a new queue
that can be processed by it's own worker and adds a dedicated worker that batches the jobs
every 1 second.

To test:
1 - Apply patches, set SearchEngine system preference to 'Elasticsearch'
2 - perl misc/search_tools/es_indexer_daemon.pl
3 - Leave the running in terminal and perform actions in staff interface:
    - Checking out a bib
    - Returning a bib
    - Editing a single bib
    - Editing a single item
    - Batch editing bibs
    - Batch editing items
4 - Confirm for each action that records are updated in search/search results
5 - Stop the script
6 - set SearchEngine system preference to 'Zebra'
7 - perl misc/search_tools/es_indexer_daemon.pl
8 - Script dies as Elasticsearch not enabled

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>

Bug 32594: (follow-up) Adjust logging per bug 32612

JD amended patch: tidy! There were tabs here...

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-24 17:52:20 -03:00
Nick Clemens 4c51596a4b
Bug 32992: Move background_jobs_worker to misc/workers
On bug 32594 we are adding a new worker, dedicated to Elastic indexing.
We should have a common place for workers, and we agreed on misc/workers

To test:
1 - Apply patch
2 - reset_all in koha testing docker
3 - ps aux | grep background
4 - Confirm the workers are running, and running in the new directory
5 - Perform a batch item modification
6 - Ensure the job is processed by the worker

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-24 17:52:19 -03:00
Nick Clemens d6433ff71e
Bug 32798: Update parameter passed to Koha::Biblio::Metadata->record
To test:
1 - Enable OAI sets, and define a set with mapping 952 y = BK
2 - perl misc/migration_tools/build_oai_sets.pl -v -i -r
3 - The script dies:
    Koha::Biblio::Metadata->record must be called on an instantiated object or like a class method with a record passed in parameter
4 - Apply patch
5 - Repeat
6 - Success!

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Edit: tcohen updated indentation
2023-02-21 10:23:34 -03:00
Magnus Enger 6a0cd4cc5e
Bug 32922: Remove space in shebang
Some of our scripts have a space in the "shebang" (first) line:

  #! /usr/bin/perl

This is not illegal, and it does work, but it is good to be
consistent, so this patch removes the space.

To test:
- Run: grep -rn --include=*.pl '#! /usr/' *
- See the list of files that have a space in the shebang
- Apply the patch
- Run the command again, there should be no output, meaning there
  are no more files with space in the shebang
- Have a look at the patch and check that it only changes the
  shebangs
- Sign off

Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-20 09:44:06 -03:00
Kyle M Hall e8c232fcf7
Bug 30642: (QA follow-up) Do not rely on script names in modules, add unit test
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Edit: Kyle, stop impersonating John Doe
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-10 11:07:59 -03:00
Jonathan Druart aa3444505f
Bug 32356: Install installer translated files on update
If the installer files exist for a given language, the translate script
won't update it.
We should get a confirmation from Bernardo (author of bug 24262), but I
don't understand why it could be needed (side-effects?)

Test plan:
Installer several times the same language, drop the DB and run the
installer+onboarding process.
Check files installed by the installer (yaml for notice templates,
biblio frameworks) have inserted the data properly in DB.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-09 10:21:17 -03:00
Kyle Hall d81bca8270
Bug 29100: (QA follow-up) Rename issue(s) keys to checkout(s)
Adapted test plan:

1) Apply this patch
2) For DUE and PREDUE notices, set the message body to the following:
   Title: [% checkout.title %]
3) For DUEDGST and PREDUEDGST notices, set the message body to the following:
   Titles:
   [% FOREACH c IN checkouts %]
    * [% c.title %][% END %]
4) Generate PREDUE and DUE notices for patrons including digests
5) Verify those notices contain the checkout titles

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Edit: tcohen renamed @issues => @checkouts as well
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-07 15:36:21 -03:00
Kyle Hall fd7a612c6e
Bug 29100: Remove unnecessary loops
Remove loops that only operate one one result only

Signed-off-by: Felicity Brown <Felicity.Brown@montgomerycountymd.gov>
Signed-off-by: George Veranis <gveranis@dataly.gr>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-07 15:36:21 -03:00
Kyle Hall 2114c70d11
Bug 29100: Add checkouts data loop to predue notices script ( advance_notices.pl )
Predue / due notices are limited to using itemscontent to display checkouts data. It would be nice to make all the checkouts data available for those notices so that libraries could format that data more nicely if they wish.

1) Apply this patch
2) For DUE and PREDUE notices, set the message body to the following:
Title: [% issue.title %]
3) For DUEDGST and PREDUEDGST notices, set the message body to the following:
Titles:
[% FOREACH i IN issues %]
* [% i.title %]
[% END %]
4) Generate PREDUE and DUE notices for patrons including digests
5) Verify those notices contain the checkout titles

Signed-off-by: Felicity Brown <Felicity.Brown@montgomerycountymd.gov>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-07 15:36:21 -03:00
Nick Clemens e5f251a709
Bug 15869: Change framework on overlay
This patch allows for selection of framework to use when overlaying
records - by default it is set to keep the initial framework

To test:
 1 - Create some records using one framework
 2 - Export the records
 3 - Edit the records to add fields not in original framework
 4 - Stage records using a rule that will find matches
 5 - Import
 6 - Note records contain new fields on display, but they are lost on edit
 7 - Apply patch
 8 - Stage records again
 9 - Select a framework that contains the new fields on import
10 - Import records
11 - Note records now use selected framework and are displayed/edited
correctly

Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-07 10:32:22 -03:00
Pedro Amorim 4734ee5d00
Bug 32793: import_patrons.pl typo in usage
Typo fix to prevent confusion in CLI usage

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-03 10:30:14 -03:00
Kyle Hall 2570062d25
Bug 32537: (QA follow-up): Tidy code
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-03 10:30:10 -03:00
Kyle Hall b1867f8485
Bug 32537: (QA follow-up) Don't allow invalid no block values
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-03 10:30:09 -03:00
Nick Clemens 0880d80949
Bug 32537: Add no-block option to SIP cli emulator
This patch adds the option to pass a no-block option and use this
in checkout/checkin/renew messages

To test:
1 - perl misc/sip_cli_emulator.pl -a localhost -p 6001 -l CPL -su term1 -sp term1 -m checkin --item 39999000010831
2 - Confirm out like:
    SEND: 09N20221227    20183920221227    201839APCPL|AOCPL|AB39999000010831|ACterm1|BIN|
    (Note the N after 09)
3 - Apply patch
4 - Restart SIP, repeat 1
5 - Confirm still an N
6 - perl misc/sip_cli_emulator.pl -a localhost -p 6001 -l CPL -su term1 -sp term1 -m checkin --item 39999000010831 -n N
7 - Confirm still an N
8 - perl misc/sip_cli_emulator.pl -a localhost -p 6001 -l CPL -su term1 -sp term1 -m checkin --item 39999000010831 -n Y
9 - Confirm the send has a Y after 09

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-03 10:30:08 -03:00
Jonathan Druart 3bf31ae4d4
Bug 32673: Remove misc/load_testing/ scripts
see comment 0 for more info

Signed-off-by: David Cook <dcook@prosentient.com.au>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-03 10:30:07 -03:00
Marcel de Rooy 89ff35b025
Bug 32612: (QA follow-up) Add missing interface for Logger
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-03 10:30:06 -03:00
Nick Clemens 3b1f2c57ea
Bug 32612: Add worker-error to log4perl conf
To test:
1 - Apply patch
2 - vim /etc/koha/sites/kohadev/log4perl.conf, Add lines below:
log4perl.logger.worker = WARN, WORKER
log4perl.appender.WORKER=Log::Log4perl::Appender::Screen
log4perl.appender.WORKER.stderr=1
log4perl.appender.WORKER.mode=append
log4perl.appender.WORKER.layout=PatternLayout
log4perl.appender.WORKER.layout.ConversionPattern=[%d] [%p] %m %l%n
log4perl.appender.WORKER.utf8=1
3 - Restart all
4 - Edit misc/background_jobs_worker.pl
-        my $job = Koha::BackgroundJobs->find($args->{job_id});
+        my $job;# = Koha::BackgroundJobs->find($args->{job_id});
5 - In another terminal: tail -f /var/log/koha/kohadev/koha-worker-error.log
6 - Force enqueue a job (that won't be found because of #4
  perl -e 'use Koha::BackgroundJob::BatchUpdateItem; my $bg = Koha::BackgroundJob::BatchUpdateItem->new(); $bg->enqueue({ record_ids=>['888888']});'
7 - Note error in log like:
  [2023/01/11 19:26:10] [WARN] No job found for id=2983 main:: /kohadevbox/koha/misc/background_jobs_worker.pl (111)

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-02-03 10:30:02 -03:00
Marcel de Rooy 08cc23626c
Bug 32393: (QA follow-up) Add explicit undef response in two catch blocks
Do not implicitly depend on last statement returning nothing.
Make it explicit. We want $args to be null here.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-01-27 15:10:01 -03:00
Jonathan Druart 85c330d8f2
Bug 32393: Deal with the DB fallback part
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-01-27 15:10:01 -03:00
Jonathan Druart 8df96262c1
Bug 32393: Split into 2 try catch blocks
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-01-27 15:10:00 -03:00
Jonathan Druart ca9a598183
Bug 32393: Prevent invalid job to block the job queue
I have faced a problem when testing an incorrect version of bug 32370.
The frame sent to the message broker was not a correct JSON encoded
string, and its decoding was obviously failing, exploding the worker
script.
Additionally, as we don't send a ack for this frame, the next pull will
result in processing the same message, and so in the same explosion.
No more messages can be processed!

This patch is logging the error and ack the message to the broker, in
order to not get stuck.

Test plan:
0. Dont' apply this patch
1. Enqueue a bad message
  a. Apply 32370
  b. Comment the following line in Koha::BackgroundJob::enqueue
    $self->set_encoded_json_field( { data => $job_args,    field => 'data' } );
  c. restart_all
  d. Use the batch item modification tool to enqueue a new job
=> Notice the error in the log
=> Note that the status of the job is "new"
=> Inspect rabbitmq queue:
% rabbitmq-plugins enable rabbitmq_management
% rabbitmqadmin get queue=koha_kohadev-long_tasks
You will notice there is a message in the "long_tasks" queue
2. Enqueue a good message
  a. Remove the change from 1.b
  b. restart_all
  c. Enqueue another job
=> Same error in the log
=> Both jobs are new
=> Inspect rabbitmq, there are 2 messages
3. Apply this patch
4. restart_all
=> Second (good) job is finished
=> rabbitmq long_tasks queue is empty

We cannot mark the first job as done, we have no idea which job it was!

QA: Note that this patch is dealing with another problem, not tested in
this test plan. If an exception is not correctly caught by the ->process
method of the job, we won't crash the worker. The job will be marked as
failed.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-01-27 15:10:00 -03:00
Kyle Hall b3c5bd02e1
Bug 32573: Send ACK message to RabbitMQ before handling the job
Splitting off this functionality from bug 32558. This is the comment that started this discussion: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32558#c15

From the O'Reilly book Mobile and Web Messaging:

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-01-27 15:09:52 -03:00
Kyle Hall 790f00781a
Bug 32656: Script delete_records_via_leader.pl no longer deletes items
Bug 29788 inadvertantly replaced a call to safe_delete() with safe_to_delete()
such that any time the script should delete an item it only checks to see if
the item is delectable, after which deletion of the record fails because the
items were not deleted.

Test Plan:
1) Mark a record with items to be deleted via the record leader
2) Run delete_records_via_leader.pl -i -b -v
3) Note the script says it is deleting the items but then the record
   deletion fails. Note the items remain in the items table of the
   database.
4) Apply this patch
5) Repeat step 2
6) This time the items and record should be deleted!

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2023-01-20 13:53:09 +00:00
David Cook cd89c38321
Bug 32481: Limit prefetch size for background jobs worker
This patch adds a prefetch size of 1 to the background jobs worker,
so that it fetches 1 message at a time. Without this change,
the RabbitMQ connection timeout when too many messages for slow tasks
are fetched at the same time.

To test:
0. Apply patch
1. Run background worker
2. Rapidly enqueue multiple jobs that in total will take longer
than 30 minutes to process

Bug 32481: Use correct prefetch syntax for RabbitMQ

According to https://www.rabbitmq.com/stomp.html the header to
use for managing the prefetch is "prefetch-count".

You can verify the number of delivered and unacknowledged messages
on a channel on a connection by running "rabbitmqctl list_channels"
on the RabbitMQ host. This will tell you how many messages have been
delivered and are awaiting acknowledgement

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-01-11 20:46:01 -03:00
Koha translators 623fb377ad
Translation updates for Koha 22.11.00
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-25 15:20:05 -03:00
Tomas Cohen Arazi 0b6837ea22
Add release notes for Koha 22.11.00
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-25 15:20:02 -03:00
Jonathan Druart 404c88fe87 Bug 31969: Use filter_by_last_update
This script has a pattern to delete rows depending on a given
date/number of days, we should use the filter_by_last_update
Koha::Objects method.
No need for another method and tests, everything is already tested
there.

This patch also suggests to rename the reference to "background" and
"bg" with "jobs", which seems more appropriate and not an abbreviation

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-25 09:40:09 -03:00
Katrin Fischer 356e136b0f
Bug 32093: Make help and code match for bg-days in cleanup_database.pl
There as a mismatch between the parameter name in documentation (bg-days)
and what was checked in the code (bg-jobs). This makes both match.

To test:
* Apply patch
* Make sure you have some background_jobs older than one day
* Verify the help of the script documents the parameter bg-days
* Run the cleanup_database.pl job
  Example:  ./misc/cronjobs/cleanup_database.pl --bg-days  --confirm -v
* Verify there are no errors and the lines have been deleted as expected

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-17 14:55:12 -03:00
Kyle Hall a9e0bcff16
Bug 27920: (QA follow-up) Fix POD in cli tool
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-09 14:37:27 -03:00
Kyle Hall 1afc3b741b
Bug 27920: (QA follow-up) Fix syntax error
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-09 14:37:20 -03:00
Kyle Hall e4e453d5df
Bug 27920: Add ability to update patron expiration dates when importing patrons
Some libraries need to recalculate a patron's expiration date any time they are updated via a patron import from file.

Test Plan:
1) Apply this patch
2) prove t/db_dependent/Koha/Patrons/Import.t

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-09 14:37:07 -03:00
Jonathan Druart 0232ee9cc4
Bug 32030: Remove maintenance file
This script is actually not used.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-09 09:48:21 -03:00
Tomas Cohen Arazi 862de51915
Bug 32030: Wrong permissions on maintenance script
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-08 21:04:10 -03:00
Jonathan Druart de77f3a719
Bug 32030: I18N rewrite
The previous implementation was using vue-i18n inject localized strings
into the Vue app. However it was using json files, and we needed
additional overhead to convert from/to PO files.

I've also tried vue3-gettext that use PO, but the overhead to work with
our workflow was existent as well (see branch joubu/vue3-gettext).

vue-i18n-extract was using for extracting, and a specific misc script
(misc/translate_json.pl) was also used to generate the json file. They
can be removed.

Here we are simply reusing our existing workflow, and we will improve it
(ie. make it more vue-ish) later if we need it.

Signed-off-by: Jonathan Field <jonathan.field@ptfs-europe.com>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-08 09:44:38 -03:00
Jonathan Druart 6be1de94e0
Bug 32030: I18N - Load de-DE, es-ES, fr-FR strings
Signed-off-by: Jonathan Field <jonathan.field@ptfs-europe.com>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-08 09:44:18 -03:00
Jonathan Druart 487470b411
Bug 32030: maintenance holdingsIQ
Signed-off-by: Jonathan Field <jonathan.field@ptfs-europe.com>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-08 09:44:05 -03:00
Jonathan Druart 151f6f3f0c
Bug 32030: add maintenance script for HoldingsIQ
Signed-off-by: Jonathan Field <jonathan.field@ptfs-europe.com>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-08 09:44:03 -03:00
Jonathan Druart 74be73121e
Bug 32030: Update I18N
Note that we are adding an extra space for id and counter, otherwise
they got removed in favor of the "simple" string.
  { Agreement: Agreement }
replaced
  { Agreement: { id: ..., counter: ... } }

Signed-off-by: Jonathan Field <jonathan.field@ptfs-europe.com>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-08 09:44:02 -03:00
Jonathan Druart 0c763b7dc4
Bug 32030: Improve I18N
* Remove ":" from the translation to prevent duplication string
* Add a script to auto-translate the strings in English

echo "{}" > koha-tmpl/intranet-tmpl/prog/js/vue/locales/en.json
npx vue-i18n-extract --vueFiles 'koha-tmpl/intranet-tmpl/prog/js/vue/**/*.?(js|vue)' \
                     --exclude koha-tmpl/intranet-tmpl/prog/js/vue/dist/main.js \
                     --languageFiles 'koha-tmpl/intranet-tmpl/prog/js/vue/locales/*.json' \
                     --add --remove
perl misc/translate_json.pl | sponge koha-tmpl/intranet-tmpl/prog/js/vue/locales/en.json

Signed-off-by: Jonathan Field <jonathan.field@ptfs-europe.com>

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-08 09:44:01 -03:00
Joonas Kylmälä 8a63d78c4b
Bug 27259: Add HomeOrHoldingBranch checks where it was missing from
The TooMany() function and fine calculation functions were incorrectly
hard coded to use homebranch for fetching the circulation rules. Those
ignored completely the syspref HomeOrHoldingBranch where the user
might have set it to holdingbranch and therefore the fines and whether
patron has too many checkouts (TooMany()) were counted using the
unintended branch's rules. This problem only arises in the cases where
there are branch specific circulation rules defined.

Test plan:
1. Make sure following tests pass:
   $ prove t/db_dependent/Circulation/_CalculateAndUpdateFine.t
   $ prove t/db_dependent/Circulation/TooMany.t

Test plan for fines.pl:
1. Add branch specific fine rules for branches A and B. A having a
   fine of 1 per day and B having a fine of 0 per day.
2. Set sysprefs:
   CircControl = the library the items is from
   finesMode = Calculate and charge
   HomeOrHoldingBranch = holdingbranch
3. Create an item with home and holding branch of A
4. Checkout the item with a due date in the past (the past due date can be
   specified by clicking "Checkout settings" in the checkout page) and
   make sure the branch you are checking from is B.
5. Run perl /usr/share/koha/bin/cronjobs/fines.pl
6. Notice that fines have popped up now to the patron incorrectly
7. Apply patch
8. Pay fines, Check-in the item and check it out again
9. Run perl /usr/share/koha/bin/cronjobs/fines.pl
10. Notice that fine is now 0. This means that the branch
    B (holdingbranch of the checked-out item) specific rule is used.

Test plan for staticfines.pl:
1. Add branch specific fine rules for branches A and B. A having a
   fine of 1 per day and B having a fine of 0 per day.
2. Set sysprefs:
   CircControl = the library the items is from
   finesMode = Calculate and charge
   HomeOrHoldingBranch = holdingbranch
3. Create an item with homebranch A and holding branch of A
4. Checkout the item with a due date in the past (the past due date can be
   specified by clicking "Checkout settings" in the checkout page) and
   make sure the branch you are checking from is B.
5. Run perl staticfines.pl --library A --library B --category <PATRONS_CATEGORYCODE>
   and notice that now there is inccorectly fines
6. Apply patch
7. Pay fines, Check-in the item and check it out again
8. Run perl staticfines.pl --library A --library B --category <PATRONS_CATEGORYCODE>
    and notice the fines are now not generated

Rebased-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Petro Vashchuk <stalkernoid@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-04 19:04:18 -03:00
Katrin Fischer 469aff4c16
Bug 31854: (QA follow-up) Terminology: loans > checkouts
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-03 10:04:27 -03:00
Magnus Enger e57ed68a02
Bug 31854: Document --not_borrowed_since and anonymization
To test:
- Run "perldoc misc/cronjobs/delete_patrons.pl"
- Note the absence of information about the potential conflict
  between --not_borrowed_since and anonymization
- Apply this patch
- Re-run the perldoc command and make sure the note about
  anonymization make sense

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-03 10:04:27 -03:00
Katrin Fischer a4f2e6ae85
Bug 31995: (QA follow-up) Add 'system preference' to help text
We often get asked where something can be found in setting,
adding the hint that RealTimeHoldsQueue is a system preference
is hopefully helpful here.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-03 09:54:33 -03:00
Kyle Hall 1088e3fb79
Bug 31995: build_holds_queue.pl should check to see if the RealTimeHoldsQueue syspref is on
The real time hold queue and the build_holds_queue.pl jobs are not 100% compatible in that we should not be running the cron if the real time queue is enabled, this could lead to double server work. It would be good to have a check in build_holds_queue for the RealTimeHoldsQueue syspref and not run the job if the preference is enabled.

There might be times when we'd want to force a run of this job without changing the syspref. To that end we would also want a flag for this job so that system administrators could force the job from the command line if required, overriding this limitation.

Test Plan:
1) Apply this patch
2) Try run misc/cronjobs/holds/build_holds_queue.pl with the -h/--help and -m/--man options
3) Disable RealTimeHoldsQueue
4) Run with no options, should succeed
5) Enable RealTimeHoldsQueue
6) Run with no options, should display a message and not rebuild the
   holds queue
7) Run again with the -f/--force option, should succeed

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-03 09:54:32 -03:00
Fridolin Somers 9aa21276b2
Bug 32012: runreport.pl should use binmode UTF-8
Like Bug 26832 added binmode UTF-8 to script misc/search_tools/export_elasticsearch_mappings.pl, this should be added to misc/cronjobs/runreport.pl.

Test plan :
1) Do not apply patch
2) Create a SQL report with :
   SELECT 'accentué',barcode FROM items limit 3
3) Note the id of this report, for example 1
4) Run : misc/cronjobs/runreport.pl 1 --format csv | tee /tmp/without.csv
=> You see output with unknown character instead of é :
5) Run : file --mime-type /tmp/without.csv
=> You see : /tmp/without.csv: iso-8859-1
6) Apply patch
7) Run : misc/cronjobs/runreport.pl 1 --format csv | tee /tmp/with.csv
=> You see correct output with é
8) Run : file --mime-type /tmp/without.csv
=> You see : /tmp/without.csv: utf-8

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-11-02 19:15:48 -03:00