add_field/maybe_add need $server to apply the hide_fields and
allow_fields account options; without it the AJ field is always sent.
- Pass $server at the FID_TITLE_ID call sites in the checkout, checkin,
item information, item status update and renew responses
Assisted-by: Fable 5 (Anthropic)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch adds a custom aj_field_template to the sip accounts
This allows libraries to specify a template for the title (AJ) field in sip
messages
The template allows using item and biblio fields (item provides access to biblio. but we pass
both in order to makes the templates simpler
To test:
1 - Install patches
2 - updatedatabase, yarn build, restart_all
3 - Setup a sip account, KTD already has a sip patron, term1, you can use that
don't add the aj_field_template yet
4 - Find or create a biblio with 245a and 245b like 245$aTitle : $bSubtitle and add item with TESTBARCODE
5 - From the command line:
perl misc/sip_cli_emulator.pl -su term1 -sp term1 -l CPL -t CR -a localhost -p 6001 -m item_information --item TESTBARCODE
6 - Confirm AJ field shows "Title :" as expected
7 - Edit sip account, set aj_field_template to:
[% biblio.title %] [% biblio.subtitle %] [% item.barcode %]
8 - Repeat the sip test
9 - Confirm you get the response like:
Title : Subtitle TESTBARCODE
10 - Sign off!
Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
After the previous patch, $flags->{WAITING} from
C4::Members::patronflags is no longer read anywhere in Koha.
Test Plan:
1) Apply this patch
2) prove t/db_dependent/SIP/ t/db_dependent/Members.t
3) Note all tests pass!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Aleisha Amohia <aleisha@catalyst.net.nz>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch fetches the patron's holds once into a list of
Koha::Hold objects and derives both hold_items and unavail_holds
from that single fetch.
Test Plan:
1) Apply this patch
2) prove t/db_dependent/SIP/
3) Note all tests pass!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Aleisha Amohia <aleisha@catalyst.net.nz>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Some 3rd party SIP clients calculate the number of holds ready
for pickup as:
holds ready for pickup = hold_items - unavail_holds
Right now this can produce a negative number since hold_items
contains only waiting holds.
This patch makes hold_items contain all of the patron's holds,
so hold_items - unavail_holds correctly gives the count of holds
ready to be picked up.
Test Plan:
1) Apply the first patch only
2) prove t/db_dependent/SIP/Patron.t
3) Note the new "hold_items counts all holds" subtest fails
4) Apply this patch
5) prove t/db_dependent/SIP/Patron.t
6) Note the tests pass!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Aleisha Amohia <aleisha@catalyst.net.nz>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
login_core only cleared leftover session state on the failed-auth
branch. The earlier 'SIP access not authorized' return (unknown
terminal id) left any account/ils/institution/policy/credentials from
a prior connection on a reused prefork worker in place.
Move the reset to the top of login_core so every non-success exit path
leaves clean state, and only a successful api_auth re-establishes the
session. This keeps the no-authenticated-state-until-auth invariant in
a single place rather than duplicating the key list per branch.
Also extend the unit test to cover the unknown-terminal path on a
reused worker.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
After a SIP terminal logs in successfully, a later connection handled by the
same worker could bypass authentication entirely. login_core() stores the account
in the session before calling api_auth(), and left it in place when authentication
failed. The check in raw_transport() that is meant to drop unauthenticated connections
only looks at $self->{account}->{id}, which is now populated straight from the SIP config,
so the connection is treated as logged in.
Test Plan:
1) Create a SIP user with the username and password "term1"
2) Restart the SIP server
3) Apply the first patch
4) prove t/db_dependent/SIP/MsgType.t
5) Note the new test fails
6) Telnet to the SIP server port ( 6001 for ktd )
5) Send a successful login: 9300CNterm1|COterm1
6) Note you get a 941 back
7) Disconnect, then telnet to localhost 6001 again
8) Send a bad login: 9300CNterm1|COterm2
9) Note you get a 940 back, but the connection stays open
10) Send a patron information request, using a real patron cardnumber:
6300120260615 110310 AOCPL|AA23529001000463
11) Note you get a 64 patron information response with the patron's data back,
even though the login failed!
12) Apply the second patch
13) prove t/db_dependent/SIP/MsgType.t
14) Note the subtest now passes!
15) Restart all the things!
16) Repeat steps 4 to 10
17) Note the the patron info request returns no patron data and the connection is dropped!
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When an item is checked in via SIP2 at the correct target branch after a
transfer, the system was incorrectly triggering an alert flag. This occurred
because the TransferArrived message was not being filtered out for accounts
with checked_in_ok="1", causing the alert logic to treat successful transfer
completion as a failure condition.
The TransferArrived message indicates successful transfer completion and is
purely informational - it should not trigger SIP alerts. This fix ensures
that TransferArrived messages are filtered out before alert evaluation,
preventing false positive alerts on successful operations.
Test Plan:
1. Set up a SIP2 account with checked_in_ok="1" in SIPconfig.xml
2. Via staff client, check out an item to a patron
3. Return the item to a different branch to trigger a transfer
4. Use misc/sip_cli_emulator.pl to check in the item at the target branch
5. Verify SIP response shows alert flag 'N' (no alert) instead of 'Y'
6. Test wrong branch checkin still shows alert flag 'Y' with CV04 type
Before: Transfer completion at correct branch incorrectly shows alert='Y'
After: Transfer completion at correct branch correctly shows alert='N'
Sponsored-by: OpenFifth <https://openfifth.co.uk>
Signed-off-by: Jackie Usher <jackie.usher@westsussex.gov.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch adds the ability to set an item type as magentic for SIP
response purposes.
Test plan:
1) Run included database updates
2) Navigate to admin > itemtypes
3) Edit/Add an itemtype and scroll to the 'Sip media types' input
4) A new 'Sip magnetic' checkbox should be available
5) Confirm setting/unsetting it works as expected
6) The SIP magnetic flag bit should respect the setting
7) To test SIP functionality:
a) Start the SIP server (e.g., koha-sip service)
b) Create an item with an itemtype that has sip_magnetic checked
c) Use misc/sip_cli_emulator.pl to send a checkout request:
perl misc/sip_cli_emulator.pl -a localhost -p 6001 -su <sip_username> -sp <sip_password> -l <location_code> --patron <patron_cardnumber> --password <patron_password> --item <item_barcode> -m checkout
d) Check the SIP response to ensure it handles magnetic media correctly (e.g., no desensitization flag)
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Marcus Michel <marcus.michel@rbkc.gov.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The `magnetic_media` flag was never set, it can be set from
`sip_media_type` if it's present. `sip_media_type` is set via the
Authorized Values system.
Checkins will not be sorted correctly by Biblioteca without this flag.
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Marcus Michel <marcus.michel@rbkc.gov.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
SIP2 accounts/institutions are stored in the database and the SIP server is supposed to reload its configuration when they change.
In practice, after adding or editing a SIP2 account, most SIP worker processes keep serving the old configuration (e.g. new accounts get "SIP access not authorized") until the SIP server is restarted ( I tested multiple times over 30 minutes ).
There seem to be two issues:
* L1 cache flushed too late. process_request() calls _config_up_to_date() before Koha::Caches->flush_L1_caches(). The SIP parent reads sip2_resource_last_modified into its own L1 cache at startup and never flushes it. Every worker inherits that stale L1, so a worker's first connection compares against the startup-era value instead of the current shared value.
* sip2_config_read_timestamp is stored in the shared (memcached) cache, but the configuration it tracks is per-process. When one worker reloads the config from the database, it bumps the shared timestamp, so every other worker then evaluates as "up to date" and never reloads its own stale in-memory config.
Test Plan:
1) Configure the SIP server with several worker processes (e.g. set
min_servers and max_servers to 5) and confirm an existing SIP2
account works
2) Add a new SIP2 account in the staff interface
3) Without restarting the SIP server, test the new account with misc/sip_cli_emulator.pl
4) Note the login fails
5) Apply this patch
6) Restart all the things!
7) Add another new SIP2 account
8) Without restarting the SIP server test this new account with sip_cli_emulator.pl
9) Note that the new account works!
10) prove t/db_dependent/SIP/SIPServer.t
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
In Koha, the sip2_resource_last_modified key is updated any time the SIP config is edited from the SIP2 editor in the staff interface ( institutions, accounts, system preference overrides ).
If sip2_resource_last_modified is missing from memcached, Koha SIP emits "Couldn't find sip2_resource_last_modified, considering configuration not up to date" in the logs, and fetches the entire configuration from the database, *for each request*.
This causes SIP performance to drop significantly. In my production testing with 83 institutions and 171 accounts the login message response time dropped from 0.283 seconds to 2.899 seconds.
I think the solution is pretty simple. If there is a cache miss on sip2_resource_last_modified, SIP should load the config from database as it does already, but then it should also set sip2_resource_last_modified!
Test Plan:
1) Apply this patch
2) Prove t/db_dependent/Koha/SIP2/SIP2ModuleMigration.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch extends the SIP checkout handler to honour the new
three-valued AllowItemsOnHoldCheckoutSIP preference:
* 0 - Don't allow checkout of any held item
* 1 - Allow checkout when the hold is still pending (not attached)
* 2 - Also allow checkout when the hold is waiting, in transit or
being processed for another patron
Previously the preference was a simple YesNo toggle that, when
enabled, allowed the checkout regardless of whether the hold was
pending or already attached to an item on the hold shelf. Libraries
could not block the more disruptive case (checking out a waiting
hold destined for another patron) without also blocking pending
holds.
The existing post-confirmation filter is reworked into a single
branch per confirmation type that consults the preference value.
Test plan:
1. prove t/db_dependent/SIP/Transaction.t
2. With AllowItemsOnHoldCheckoutSIP=0, place a pending hold for
patron A on an item, attempt SIP checkout to patron B: refused.
3. Repeat with the hold marked waiting: refused.
4. Set AllowItemsOnHoldCheckoutSIP=1, pending hold: allowed;
waiting hold: refused.
5. Set AllowItemsOnHoldCheckoutSIP=2, waiting hold: allowed.
Signed-off-by: Kristi Krueger <kkrueger@cuyahogalibrary.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Standardizes error code spelling across the codebase:
- Changes 'too_much_oweing' to 'too_much_owing'
- Changes 'auto_too_much_oweing' to 'auto_too_much_owing'
The word "owing" is the correct spelling in English.
Files updated:
- C4/Circulation.pm
- C4/SIP/Sip/MsgType.pm
- circ/renew.pl
- koha-tmpl/intranet-tmpl/prog/en/modules/circ/renew.tt
- koha-tmpl/intranet-tmpl/prog/js/checkouts.js
- koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-reserve.tt
- koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-user.tt
- misc/cronjobs/automatic_renewals.pl
- opac/opac-reserve.pl
- opac/opac-user.pl
- t/db_dependent/Circulation.t
All tests should pass successfully.
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
As Jan Kissig correctly mentioned on the BZ report, the various
summary info on charged items etc. are included.
This is done via the summary_info sub.
Note that the TODOs about
BZ FID_HOLD_ITEMS_LMT
CA FID_OVERDUE_ITEMS_LMT
CB FID_CHARGED_ITEMS_LMT
are still valid. This is not included yet.
Test plan.
Since we are just removing comments, read the code.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This bug adds Koha::Patron::Message::message_without_newlines, which has been added to C4::SIP::ILS::Patron in lieu of Koha::Patron::Message::message. The aim of this change is to prevent SIP2 units from crashing out when they encounter a patron message with newlines (\n or \r) in it, as units generally use a newline character to denote the end of a SIP2 transaction.
To test:
a) go to /cgi-bin/koha/sip2/institutions
*) create an institution, name CPL, timeout 300, the rest default
b) go to /cgi-bin/koha/sip2/accounts
*) create an account, userid koha, institution CPL, terminator crlf, the rest default
c) go to /cgi-bin/koha/members/moremember.pl?borrowernumber=51
*) click add message in toolbar
*) enter some multi-line message e.g. test[RETURN]1[RETURN]2[RETURN]3[RETURN][RETURN][RETURN]
d) run tail -f /var/log/koha/kohadev/sip-output.log, keep this window open
e) run ./misc/sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su koha -sp koha -l CPL --patron 42 -m patron_information
f) look at the tail log, notice how the MSG 64 line has a newline before the final pipe (|) character
== APPLY PATCH ==
g) restart_all
h) repeat steps e-f
i) notice how in the tail log, the MSG 64 line now has the final pipe character contiguously placed with the successive characters
== SIGN OFF ==
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
If a message ends in a non-alphanumeric character, that character is stripped from the message.
For example:
6300120260210 182641 AOMPL|AA23529000035676(|
64 00120260210 182831000000000000000000000000AOCPL|AA23529000035676|AEHenry Acevedo|BLY|BHUSD|BV0|CC5|BD4345 Library Rd. Springfield, MA 44224|BF(212) 555-1212|PB19580530|PCS|PIY|AFGreetings from Koha. |
versus:
6300120260210 182641 AOMPL|AA23529000035676(|ACkoha|
64YYYY 00120260210 182835000000000000000000000000AOCPL|AA23529000035676(|AE|BLN|AFInvalid cardnumber|
The former should be returning the same "Invalid cardnumber" message the latter does.
Test Plan ( assumes you are using KTD ):
1) Install telnet: sudo apt install -y telnet
2) telnet 127.0.0.1 6001
3) Log in with: 9300CNkoha|COkoha|CPMPL|
4) Send the patron information request: 6300120260210 183346 AOMPL|AA23529000035676(|
5) Note you get a patron back
6) Apply this patch
7) Restart SIP!
8) Repeat steps 2-5
9) Noe you doe not get a patron back in the response!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
When starting SIP, Koha passes an incorrect value to the
Net::Server "group" parameter, which can generate an
error in the logs":
Couldn't become gid "<uid>": Operation not permitted
This change ensures that a valid numeric GID is passed instead.
Test plan:
1. Enter KTD shell.
2. To reproduce the error: sudo groupmod -g 8001 kohadev-koha
3. Restart SIP: sudo koha-sip --restart kohadev
4. Check SIP logs:
/var/log/koha/kohadev/sip-output.log
5. Observe the error:
Couldn't become gid "<uid>": Operation not permitted
6. Apply this patch.
7. Repeat steps 2 to 4.
8. Verify that the "Couldn't become gid" error is no longer logged.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
For SIP based payments, no branchcode is recorded for the corresponding
credit. Koha should be recording the institution id (i.e. branchcode)
for SIP payments.
Test plan:
1) Apply this patch
2) prove t/db_dependent/SIP/Transaction.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This fixes an error in SIP2 patron information response (64) when no currency is set, which can be the case on newly created instances.
Test plan for KTD:
a) Delete currency and related tables
- run 'koha-mysql kohadev' to start SQL client. Then run the following queries:
- delete from aqbasket;
delete from aqbooksellers;
delete from currency;
b) exit
c) run 'perl misc/sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su term1 -sp term1 -l CPL -m patron_information --patron 42'
Attempting socket connection to 127.0.0.1:6001...connected!
SEND: 9300CNterm1|COterm1|CPCPL|
READ: 941
Trying 'patron_information'
SEND: 6300120251202 115652 AOCPL|AA42|ACterm1|
READ: undef
d) recognize the last response: READ: undef
e) apply patch and restart sip2 server via 'sudo koha-sip --restart kohadev'
e) rerun 'perl misc/sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su term1 -sp term1 -l CPL -m patron_information --patron 42'
Attempting socket connection to 127.0.0.1:6001...connected!
SEND: 9300CNterm1|COterm1|CPCPL|
READ: 941
Trying 'patron_information'
SEND: 6300120251202 122545 AOCPL|AA42|ACterm1|
READ: 64 00120251202 122546000000000000000000000000AOCPL|AA42|AE koha|BLY|BV0|CC5|PCS|PIY|AFGreetings from Koha. |
f) check the last response is now a patron information response (64)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
So it is done before the other use statements.
Also uniformize the pattern, must now be:
```
use Modern::Perl;
use base 'Exporter';
BEGIN {
our @EXPORT_OK = qw(subroutine list);
}
```
Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Bug38936 exported redirect_if_opac_suppressed
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This was a bad workaround.
It was ugly and is no longer relevant now that pod_coverage.t no longer attempts to launch a SIP server.
prove xt/author/pod_coverage.t
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Because the file is now being properly checked for pod_coverage, it fails (because a lot of methods dont have pod coverage. Add here
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
By using 'unless caller' we ensure the class is not executed when Pod::Coverage->new does 'require C4/SIP/SIPServer.pm
Ensure no regressions are added:
prove t/db_dependent/SIP/*
koha-sip --restart kohadev
koha-sip --status kohadev
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
To test:
prove -v /kohadevbox/koha/xt/author/pod_coverage.t
The changes to Schema files are fixes that were overlooked originally.
The 'die' line in C4/SIP/Sip/Configuration.pm is to force the script to die if SIP configuration does not exist, which only happens when running pod_coverage.t
The original code (prior to 37893) did not consider the possibiliy of a config_file being invalid or undef, parser->XMLin(config_file) died silently, but because pod_coverage.t skips over files that die, it didn't fail. See bug 41271.
Now, with the changes from bug 37893, if a config_file doesnt exist the code falls back to whats in the cache, so it doesn't try to XMLin load an invalid config_file anymore, and fails verbosely.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This follow-up removes the security vulnerability of storing plaintext
passwords in the sip_accounts table. The SIP2 system now relies entirely
on Koha's secure password authentication.
Changes:
- Remove plaintext password comparison from login_core() and telnet_transport()
- Remove login_password column from sip_accounts table
- Update database schema files and atomic update
- Remove password field from API definitions and Vue form
- Update all tests to remove login_password references
Authentication now works in two stages:
1. Access control: Check if userid exists in sip_accounts (allowlist)
2. Authentication: Validate password against hash in borrowers table
This maintains SIP access control while eliminating plaintext password
storage. Staff users authenticate with their Koha password (securely hashed).
Test plan:
1. Run updated tests to verify they pass
2. Verify SIP2 authentication works with staff user credentials
3. Confirm no plaintext passwords are stored
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The classes that use Koha::SIP2::Object as base update the
config_timestamp server param whenever an object of said classes is
inserted, updated or deleted.
This patch considers that, and performs a check to see if the
config_timestamp is more recent than the one which the SIP server was
started with. If it is, config is fetched from the database for up to
date configurations.
Note: The classes currently using Koha::SIP2::Object as base are as
follows:
Koha::SIP2::Account
Koha::SIP2::Institution
Koha::SIP2::SystemPreferenceOverride
The 2 classes that are left out are:
Koha::SIP2::Listener
Koha::SIP2::ServerParam
A UI message is shown when CRUDing any of the 2 above informing that a
SIP server restart is required for the changes to take effect.
The reason these 2 are left out is because they're loaded and only
utilized when the SIP server is launched.
Co-authored-by: Mark Hofstetter <mark@hofstetter.at>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
To test:
perl ./C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch adds debug logging to C4::SIP::Sip::MsgType::_initialize() to help
identify when critical fields (patron_id, item_id) are parsed as empty values.
Empty critical fields in SIP messages can later cause DBIx::Class constraint
errors when passed to Koha::Objects->find() operations, generating confusing
'Odd number of elements in anonymous hash' errors in SIP server logs.
The enhanced logging helps administrators identify the root cause by logging
when empty patron_id or item_id fields are detected during message parsing.
To test:
1. Apply regression tests patch first
2. Run:
$ ktd --shell
k$ prove t/db_dependent/SIP/MsgType.t
=> FAIL: Debug logging tests fail (no enhanced logging present)
3. Apply this patch
4. Run:
$ ktd --shell
k$ prove t/db_dependent/SIP/MsgType.t
=> SUCCESS: All tests pass, debug logging works correctly
5. Sign off :-D
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Patron messages containing carriage returns, line feeds, or CRLF
combinations can break SIP protocol communication since CR is used
as the SIP message terminator. This causes message parsing failures
and protocol errors in self-check systems.
The previous regex only handled single carriage returns (\r),
missing Windows CRLF (\r\n) and Unix LF (\n) sequences.
This patch normalizes all line ending variations to spaces to
prevent SIP message corruption while preserving message content.
Test plan:
1. Go to a patron record and click "Add message"
2. Select "Add message for OPAC..." from dropdown
3. In a text editor (e.g., Word), create a message with line breaks:
"This is line one[Enter]This is line two[Enter]This is line three"
4. Copy and paste this multi-line text into the message box and save
5. Run the SIP emulator: misc/sip_cli_emulator.pl
6. Execute a patron information request for this patron
7. Before patch: Observe message breaks at line endings, corrupting
the SIP response
8. Apply patch and restart SIP server
9. Repeat steps 5-6
10. After patch: Verify the message appears as single line with
spaces replacing line breaks: "This is line one This is line two
This is line three"
11. Test with different line ending types (Windows, Mac, Unix) to
ensure all variations are handled correctly
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Adjusted the comment in Sip.pm
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
To test:
1. In circ rules set a Default waiting hold cancellation policy to allow hold cancellation requests.
2. Add a hold for a patron and create a hold cancellation request.
3. Check the item in via SIP, I used the SIP emulator:
kohadevbox/koha/misc/sip_cli_emulator.pl -a localhost -p 6001 -l MPL -su term1 -sp term1 -m checkin --patron 23529000035676 --item 39999000001310
4. The hold remains.
5. APPLY PATCH, restart_all
6. Try 2 - 4 again, the hold should be cancelled
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
An alternative approach to the original submission for bug 9762. Instead
of recording a distinct action log line for a set of override messages
in addition to the standard issue record, this patch updates the
existing issue log to include any confirmation or impossible responses
from the preceeding CanBookBeIssued call.
We continue to record these as a JSON structure as the prior patch did,
but instead of human messages we use the predefined codes as returned
from the existing CanBookBeIssued response.
To test:
After each step, a single action log message should be displayed under
`Tools > Log viewer`. The message itself will now consist of a JSON
string with an array of 'confirmations' and 'forced' strings
representing the overriden feedback from CanBookBeIssued.
Circulation -- Checking out
1) Check out an item that is age-restricted (AgeRestrictionOverride syspref)
2) Check out an item that has a “not for loan” status (AllowNotForLoanOverride
syspref)
3) Check out an item that has a “lost” status (IssueLostItem syspref)
4) Check out an item to a patron who has reached the checkout limit
(AllowTooManyOverride syspref)
5) Check out an item to a patron who is not allowed to borrow this item type
6) Check out an item to a patron who has unpaid fines (AllFinesNeedOverride
and/or AllowFineOverride + noissuescharge sysprefs)
7) Check out an item on hold for someone else
8) Check out an item on hold ready for pickup by someone else
9) Check out an item already checked out to someone else
(AutoReturnCheckedOutItems syspref)
10) Check out to a patron who has restrictions
Circulation -- Checking in
11) Ignore a hold upon check in
12) Ignore a hold and transfer upon check in
Circulation – Renewing
13) Renew an item on hold for someone else
(AllowRenewalOnHoldOverride syspref)
14) Renew an item that has reached the maximum number of renewals
(AllowRenewalLimitOverride syspref)
Holds
15) Place a hold for a patron who is not allowed to place this item
type on hold (AllowHoldPolicyOverride syspref)
16) Place a hold for a patron who has reached the maximum number of
holds
17) Place an item-level hold when rules dictate that this is not
allowed
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The SIP server generates warnings when logging account information
for failed logins:
Use of uninitialized value $args[0] in sprintf at /kohadevbox/koha/C4/SIP/Sip.pm line 314
Use of uninitialized value $args[1] in sprintf at /kohadevbox/koha/C4/SIP/Sip.pm line 314
This occurs when $self->{account}->{id} or $self->{account}->{institution}
are undefined during failed authentication attempts.
This patch fixes the issue by:
1. Adding a safer sprintf implementation in siplog that handles undefined values
2. Providing default values for undefined account information in logging calls
To test:
1. Start SIP server
2. Attempt to connect with invalid credentials using sip_cli_emulator.pl:
./misc/sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su invaliduser -sp wrongpass -l kohalibrary2 --patron koha -m patron_information
3. Check /var/log/koha/kohadev/sip-output.log
=> FAIL: 'Use of uninitialized value...' warns
4. Apply this patch
5. Repeat 2
=> SUCCESS: No "Use of uninitialized value" warnings appear
=> SUCCESS: Logging shows 'undef/undef' for failed logins instead of empty strings
6. Try a valid login
=> SUCCESS: Still works correctly for valid logins
7. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
**Problem:**
When using SIP checkout messages with the "no block" flag, the specified due date
was being ignored. Instead of honoring the provided due date, the system would:
1. Use the due date as the transaction timestamp (incorrect behavior)
2. Calculate a new due date based on circulation rules
3. Return an empty due date field (AH) in the SIP response
This affected SIP2 integrations where external systems needed to specify exact
due dates for checkouts, particularly for offline circulation scenarios.
**Root Cause:**
The `ProcessOfflineIssue` function in C4::Circulation was only returning a
success/failure message, but the SIP checkout transaction needed access to the
resulting checkout object to properly set the due date. The SIP Transaction::Checkout
module was calling `ProcessOfflineIssue` but couldn't retrieve the checkout details
to populate the transaction's due date field.
**Solution:**
1. Modified `ProcessOfflineIssue` to return both the status message AND the checkout
object: `return ( "Success.", $checkout );`
2. Updated `ProcessOfflineOperation` to handle the new return format by capturing
only the message: `( $report ) = ProcessOfflineIssue( $operation );`
3. Modified SIP Transaction::Checkout to capture both return values and use the
checkout object to set the proper due date via `duedatefromissue`
4. Fixed the timestamp parameter to use `dt_from_string` instead of the due date
**Test Plan:**
1. **Unit Tests:**
- Run `prove t/db_dependent/Circulation/OfflineCirculation.t` to verify
ProcessOfflineIssue returns checkout object and respects due_date parameter
- Run `prove t/db_dependent/SIP/Transaction.t` to verify SIP checkout honors
no_block_due_date parameter
2. **Manual SIP Testing:**
- Connect to SIP server via telnet on port 6001
- Send checkout message with no block flag and specific due date:
`111YN20250115 18000120241201 180001AP|AO|AC|AD|AB<barcode>|AA<cardnumber>|`
- Verify response contains correct due date in AH field
- Verify checkout record in database has the specified due date
3. **Regression Testing:**
- Verify normal SIP checkouts (without no block) still work correctly
- Verify offline circulation operations continue to function
- Verify ProcessOfflineOperation still processes queued operations correctly
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Following the original test plan you notice, `THIS IS A TEST` is printed
twice:
* In the logs.
* In STDERR for the SIP server.
The latter is redundant, and also problematic for
`xt/author/pod_coverage.t` which fails the `no warnings` tests because
of that.
To test:
1. Run:
$ ktd --shell
k$ prove xt/author/pod_coverage.t
=> FAIL: Scary errors, lots of them
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests pass!
4. Follow the original bug test plan
=> SUCCESS: There's `THIS IS A TEST` in the logs, which was the author's
goal in the first place.
5. Sign off :-D
6. Visit bug 40144 :-P
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
At some point our SIP server stopped logging fatal errors to either logs or standard error. We should restore this vital functionality.
Test Plan:
1) Apply the first patch which will cause the patron information request
to fail
2) Stop the SIP server
3) Start it again with: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
4) Trigger the failure with the following command:
./misc/sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su koha -sp koha -l MPL --patron koha -m patron_information
5) Note "THIS IS A TEST" is not found in logs or stderr
6) Apply the second patch
7) Repeat steps 2-4
8) Note the error message is now visible from stderr at the command line
and also in the SIP log
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch simply adds the holdingbranch to the SIP item as current_location
To test:
1 - Find an item and check it in at a different location than its homebranch
2 - perl misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL -t CR -m item_information --item 39999000004151
3 - Note no AP field is returned
4 - Apply patch, restart all
5 - perl misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL -t CR -m item_information --item 39999000004151
6 - You see the current holdingbranch in the AP field
7 - Return the item to its homebranch
8 - perl misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL -t CR -m item_information --item 39999000004151
9 - Confirm home and holding branches are the same now
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Package C4::SIP::ILS::Transaction::Checkout was missing block to
handle new "CURRENTISSUE" message. This patch adds this block
so tests should no longer fail.
To test prove t/db_dependent/SIP/Message.t
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Paul Derscheid
Nick Clemens
Kyle M Hall
Martin Renvoize
Colin Campbell
Marcel de Rooy
Jake Deery
Slava Shishkin
Jan Kissig
Jonathan Druart
Pedro Amorim
Pedro Amorim
Tomás Cohen Arazi
Bernard Scaife
Lucas Gass
Lari Strand
Emmi Takkinen