Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz>
We have our own "quote_po" sub that is stating that Locale::PO::quote
is buggy because it does not deal with quoting new lines correctly.
However it seems that it is fixed now.
Ideally we could use Locale::PO::quote everywhere, but it does not
escape tab characters:
$string =~ s/\\(?!t)/\\\\/g; # \t is a tab
This means the following:
msgid "Tabulation (\\t)"
msgstr "Tabulation (\\t)"
become:
-msgid "Tabulation (\t)"
-msgstr "Tabulation (\t)"
And we are seeing the following on Weblate:
https://snipboard.io/BjQmDC.jpg
Note that Locale::PO has not been updated since 2014...
The real problem behind this is that we have 2 methods to quote strings.
At first glance it seems that Locale::PO::quote was not used before, but
with the introduction of the koha-i18n project we will have scripts that
will use Locale::PO->save_file_fromarray, which uses Locale::PO->quote
=> Those scripts will be used on the translation server for post
processing (security reason, marking potential XSS strings as fuzzy).
Test plan:
0. Do not apply the patch
1. gulp po:update --lang LANG # Replace LANG with your favorite language
code
2. git commit -a -m"init PO files"
3. Apply this patch
4. Repeate 1.
5. git diff
=> The change is about the "Tabulation" and "New line" strings from
tools/csv-profiles.tt
6. Translate them (replace the \t and \n with %s) and remove the fuzzy
flag
7. install the template: cd misc/translator && perl translate install
LANG
8. Enable the language, use it and go to the "Nouveau profil CSV" view
=> Notice that the \t and \n are correctly displayed.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
The PO files are not sorted when we update them which leads to
unnecessary changes that are commited: hard to see differences and
make git index grow superfluously.
Test plan:
0. Do not apply this patch
1. gulp po:update --lang es-ES
2. git commit -a -m"First PO update"
3. Run again the gulp update command
4. git diff
=> You have a lot of changes generated here, the po:update is not
idempotent.
5. Apply this patch
6. Run the gulp update command
7 git commit -a -m"PO update after 34959"
8. Run the gulp update command
9. git diff
=> No changes are generated
Note that this patch will all the entries by files, and per line
numbers.
It fixes a bug in some condition, where we add information/context about
the string. For instance search for "For the first occurrence" in the
file. Prior to this patch this was not correct, we didn't add info about
the first occurrence (but whichever in the list).
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Nothing special here, this sub is not used and we can remove it.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
44 'c|confirm' => \$confirm,
46 'm|matchpoint=s' => \$matchpoint,
The POD is wrong.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 2442a4537d)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 5fcaf0aa09)
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
(cherry picked from commit 54b530e351)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch carries this fix into the misc/cronjobs/recalls/expire_recalls.pl cronjob so that recalls are automatically expired when they have been waiting a problematic number of days, not considering hours, as expected.
To test, follow the test plan from the first patch. This will set you up with a waiting problematic recall.
Run the cronjob manually
perl misc/cronjobs/recalls/expire_recalls.pl
Refresh your 'Recalls awaiting pickup' page. Your problematic recall should be gone/expired.
Expiration dates will apply when expiring any 'unfulfilled' recall i.e. newly requested, overdue to be returned, and awaiting pickup.
Sponsored-by: Auckland University of Technology
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 0d2052088e)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 3a5f4d1701)
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
(cherry picked from commit 4afe2a2b0a)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch corrects a typo in the output of
search_for_data_inconsistencies.pl when a bibliographic record has no
title.
The patch also replaces biblio to bibliographic record in the same
sentence for terminology consistency.
To test:
- Have a bibliographic record without a title
- Run misc/maintenance/search_for_data_inconsistencies.pl
- Read output, make sure the sentence is correct
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 27a086b91e)
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
(cherry picked from commit ed32f9812d7ddebc862360662772505cffbb1aa8)
(cherry picked from commit d54de73bff)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch reformats the params passed to scroll_helper as defined here:
https://metacpan.org/pod/Search::Elasticsearch::Client::7_0::Scroll
To test:
1 - perl misc/maintenance/compare_es_to_db.pl
2 - It dies:
[Param] ** Unknown param (scroll_in_qs) in (search) request. , called from sub Search::Elasticsearch::Client::7_0::Direct::scroll_helper at misc/maintenance/compare_es_to_db.pl line 55.
3 - Apply patch
4 - Repeat
5 - It succeeds!
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit daa92a9371)
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
(cherry picked from commit 38666a8bd7)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Since bug 29486, misc/maintenance/search_for_data_inconsistencies.pl search for biblio.biblionumber in MARC record with $record->subfield().
This fails when field is a control field (< 10).
Idem for biblioitems.biblioitemnumber
Test plan :
1.0) On a UNIMARC database (biblio.biblionumber is on 001)
1.1) Run misc/maintenance/search_for_data_inconsistencies.pl
=> Without patch you get error : Control fields (generally, just tags below 010) do not have subfields, use data()
=> With patch no error
2.0) On a MARC21 database (biblio.biblionumber is on 999c)
2.1) Run misc/maintenance/search_for_data_inconsistencies.pl
=> Check you see no error
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit cc1695ba96)
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
(cherry picked from commit 9971011fb9)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 6221538b6d)
Signed-off-by: Jacob O'Mara <jacobomara901@gmail.com>
(cherry picked from commit 4a1c87fe36)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Some old-style code is making our tests fail when run in Debian Testing.
This patch addresses this.
To test:
1. Launch bookworm KTD:
$ KOHA_IMAGE=master-bookworm ktd up -d
2. Run:
$ ktd --shell
k$ prove t/00-testcritic.t
=> FAIL: It fails!
3. Apply the patch
4. Repeat 2
=> SUCCESS: Tests now pass!
5. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 10d12f999f)
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
(cherry picked from commit c1c5420afa)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The default of 1 resembles the old behavior: 1 fork for the job.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 3bf4addb24)
Signed-off-by: Jacob O'Mara <jacob.omara@ptfs-europe.com>
(cherry picked from commit 13023b2fe8)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Right now background_jobs_worker.pl only processes jobs in serial. It would make sense to handle jobs in parallel up to a user definable limit.
Test Plan:
1) Apply this patch
2) Stop background_jobs_worker.pl
3) Generate some background jobs by editing records, placing holds, etc
4) Watch processes in a new terminal: watch -n 0.1 'ps aux | grep background_jobs_worker.pl'
5) Run background_jobs_worker.pl with parameter -m 3 or some other
number of max processes
6) Note the multiple forked processes in the ps output
Test notes - also tested the following on KTD:
1. Stop background_jobs_worker.pl
2. Edit /etc/koha/sites/kohadev/koha-conf.xml - set max_processes to 10
3. Generate some background jobs
4. Watch processes in a new terminal: watch -n 0.1 'ps aux | grep background_jobs_worker.pl'
5. Restart all
6. Confirm multiple forked processes in the ps output
Both methods work as expected and generate multiple forked processes
based on the value set for max processes.
Signed-off-by: emlam <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 311d68815f)
Signed-off-by: Jacob O'Mara <jacob.omara@ptfs-europe.com>
(cherry picked from commit 95a5d612f3)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
In search_for_data_inconsistencies.pl, the test for authorized values is a list in one line :
* The Framework *VR* is using the authorised value's category *LOC*, but the following items.location do not have a value defined ({itemnumber => value }):
{94 => AV} {95 => AV} {96 => AV} {97 => AV} {98 => AV} {99 => AV} {100 => AV} {101 => AV} {102 => AV} {103 => AV}
It would be more clear with new lines, especially for scripts (grep, awk ...) :
* The Framework *VR* is using the authorised value's category *LOC*, but the following items.location do not have a value defined ({itemnumber => value }):
{94 => AV}
{95 => AV}
{96 => AV}
{97 => AV}
{98 => AV}
{99 => AV}
{100 => AV}
{101 => AV}
{102 => AV}
{103 => AV}
Test plan :
1) In koha-testing-docker
2) Delete in authorized values LOC the value AV
3) Run misc/maintenance/search_for_data_inconsistencies.pl
=> You see the new line in result
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 45323886ae)
Signed-off-by: Jacob O'Mara <jacob.omara@ptfs-europe.com>
(cherry picked from commit 1453472e93)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
If the installer files exist for a given language, the translate script
won't update it.
We should get a confirmation from Bernardo (author of bug 24262), but I
don't understand why it could be needed (side-effects?)
Test plan:
Installer several times the same language, drop the DB and run the
installer+onboarding process.
Check files installed by the installer (yaml for notice templates,
biblio frameworks) have inserted the data properly in DB.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit aa3444505f)
Signed-off-by: Jacob O'Mara <jacob.omara@ptfs-europe.com>
(cherry picked from commit 27d128c3ab)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 2570062d25)
Signed-off-by: Jacob O'Mara <jacob.omara@ptfs-europe.com>
(cherry picked from commit 2b7d647553)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit b1867f8485)
Signed-off-by: Jacob O'Mara <jacob.omara@ptfs-europe.com>
(cherry picked from commit 3593ed50fe)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch adds the option to pass a no-block option and use this
in checkout/checkin/renew messages
To test:
1 - perl misc/sip_cli_emulator.pl -a localhost -p 6001 -l CPL -su term1 -sp term1 -m checkin --item 39999000010831
2 - Confirm out like:
SEND: 09N20221227 20183920221227 201839APCPL|AOCPL|AB39999000010831|ACterm1|BIN|
(Note the N after 09)
3 - Apply patch
4 - Restart SIP, repeat 1
5 - Confirm still an N
6 - perl misc/sip_cli_emulator.pl -a localhost -p 6001 -l CPL -su term1 -sp term1 -m checkin --item 39999000010831 -n N
7 - Confirm still an N
8 - perl misc/sip_cli_emulator.pl -a localhost -p 6001 -l CPL -su term1 -sp term1 -m checkin --item 39999000010831 -n Y
9 - Confirm the send has a Y after 09
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 0880d80949)
Signed-off-by: Jacob O'Mara <jacob.omara@ptfs-europe.com>
(cherry picked from commit 61c4d9d374)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 89ff35b025)
Signed-off-by: Jacob O'Mara <jacob.omara@ptfs-europe.com>
(cherry picked from commit cd3e4c196c)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
To test:
1 - Apply patch
2 - vim /etc/koha/sites/kohadev/log4perl.conf, Add lines below:
log4perl.logger.worker = WARN, WORKER
log4perl.appender.WORKER=Log::Log4perl::Appender::Screen
log4perl.appender.WORKER.stderr=1
log4perl.appender.WORKER.mode=append
log4perl.appender.WORKER.layout=PatternLayout
log4perl.appender.WORKER.layout.ConversionPattern=[%d] [%p] %m %l%n
log4perl.appender.WORKER.utf8=1
3 - Restart all
4 - Edit misc/background_jobs_worker.pl
- my $job = Koha::BackgroundJobs->find($args->{job_id});
+ my $job;# = Koha::BackgroundJobs->find($args->{job_id});
5 - In another terminal: tail -f /var/log/koha/kohadev/koha-worker-error.log
6 - Force enqueue a job (that won't be found because of #4
perl -e 'use Koha::BackgroundJob::BatchUpdateItem; my $bg = Koha::BackgroundJob::BatchUpdateItem->new(); $bg->enqueue({ record_ids=>['888888']});'
7 - Note error in log like:
[2023/01/11 19:26:10] [WARN] No job found for id=2983 main:: /kohadevbox/koha/misc/background_jobs_worker.pl (111)
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 3b1f2c57ea)
Signed-off-by: Jacob O'Mara <jacob.omara@ptfs-europe.com>
(cherry picked from commit 872d03963e)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Do not implicitly depend on last statement returning nothing.
Make it explicit. We want $args to be null here.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 08cc23626c)
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
(cherry picked from commit 223583d3b5)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 85c330d8f2)
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
(cherry picked from commit 54e87225ea)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 8df96262c1)
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
(cherry picked from commit 5fc37e5706)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
I have faced a problem when testing an incorrect version of bug 32370.
The frame sent to the message broker was not a correct JSON encoded
string, and its decoding was obviously failing, exploding the worker
script.
Additionally, as we don't send a ack for this frame, the next pull will
result in processing the same message, and so in the same explosion.
No more messages can be processed!
This patch is logging the error and ack the message to the broker, in
order to not get stuck.
Test plan:
0. Dont' apply this patch
1. Enqueue a bad message
a. Apply 32370
b. Comment the following line in Koha::BackgroundJob::enqueue
$self->set_encoded_json_field( { data => $job_args, field => 'data' } );
c. restart_all
d. Use the batch item modification tool to enqueue a new job
=> Notice the error in the log
=> Note that the status of the job is "new"
=> Inspect rabbitmq queue:
% rabbitmq-plugins enable rabbitmq_management
% rabbitmqadmin get queue=koha_kohadev-long_tasks
You will notice there is a message in the "long_tasks" queue
2. Enqueue a good message
a. Remove the change from 1.b
b. restart_all
c. Enqueue another job
=> Same error in the log
=> Both jobs are new
=> Inspect rabbitmq, there are 2 messages
3. Apply this patch
4. restart_all
=> Second (good) job is finished
=> rabbitmq long_tasks queue is empty
We cannot mark the first job as done, we have no idea which job it was!
QA: Note that this patch is dealing with another problem, not tested in
this test plan. If an exception is not correctly caught by the ->process
method of the job, we won't crash the worker. The job will be marked as
failed.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit ca9a598183)
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Splitting off this functionality from bug 32558. This is the comment that started this discussion: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32558#c15
From the O'Reilly book Mobile and Web Messaging:
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit b3c5bd02e1)
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
(cherry picked from commit 0f7b8008f7)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Bug 29788 inadvertantly replaced a call to safe_delete() with safe_to_delete()
such that any time the script should delete an item it only checks to see if
the item is delectable, after which deletion of the record fails because the
items were not deleted.
Test Plan:
1) Mark a record with items to be deleted via the record leader
2) Run delete_records_via_leader.pl -i -b -v
3) Note the script says it is deleting the items but then the record
deletion fails. Note the items remain in the items table of the
database.
4) Apply this patch
5) Repeat step 2
6) This time the items and record should be deleted!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
(cherry picked from commit 790f00781a)
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
(cherry picked from commit 03a9b13958)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Wainui Witika-Park
Aleisha Amohia
Fridolin Somers
Lucas Gass
Koha translators
Jonathan Druart
Aleisha Amohia
Tomas Cohen Arazi
Caroline Cyr La Rose
Nick Clemens
Marcel de Rooy
Kyle M Hall
