This follow-up removes the security vulnerability of storing plaintext
passwords in the sip_accounts table. The SIP2 system now relies entirely
on Koha's secure password authentication.
Changes:
- Remove plaintext password comparison from login_core() and telnet_transport()
- Remove login_password column from sip_accounts table
- Update database schema files and atomic update
- Remove password field from API definitions and Vue form
- Update all tests to remove login_password references
Authentication now works in two stages:
1. Access control: Check if userid exists in sip_accounts (allowlist)
2. Authentication: Validate password against hash in borrowers table
This maintains SIP access control while eliminating plaintext password
storage. Staff users authenticate with their Koha password (securely hashed).
Test plan:
1. Run updated tests to verify they pass
2. Verify SIP2 authentication works with staff user credentials
3. Confirm no plaintext passwords are stored
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
In order to prevent permission errors to user without the 'parameters'
permission we are introducing a separate endpoint for the ERM module to
retrieve additional fields for this specific module.
* Adds a new endpoint /erm/extended_attribute_types
* Adjusts Cypress and REST API tests
* Adds REST API swagger specs
I wanted to keep the code as simple as possible and decided to create a
wrapper for the additional fields API client. The magic is in
koha-tmpl/intranet-tmpl/prog/js/fetch/additional-fields-api-client.js
Ideally we should provide a test for this but I failed to write them.
If required by QA I would suggest to deal with them on a separate bug to
not delay this bugfix. The code has been kept designed to be used in
other contexts but the api client is not used outside of Vue and
outside of the ERM module.
Test plan:
Confirm that user without the 'parameters' permission can use the ERM
module with additional fields.
Sponsored-by: Karlsruhe Institute of Technology (KIT)
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
prove t/db_dependent/api/v1/patrons_hold_groups.t
Signed-off-by: Anneli Österman <anneli.osterman@koha-suomi.fi>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch makes the API representation of the identity providers
domains match the new metadata.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
prove t/db_dependent/api/v1/holds.t
Signed-off-by: Phillip Berg <phillip.berg@mainlib.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Enforce no hold group when creating hold object
When a hold is created by build_object, it's auto-assigned a hold_group_id (as expected).
When a hold is ->fill, it's hold_group_id is cleared by cleanup_hold_group (as expected).
The problem is then this causes the object before ->fill to be out of sync with the up to date object returned by the API (i.e. the latter has an empty hold_group_id, as expected, but the former has the auto-assigned hold_group_id).
Using ->discard_changes or ->get_from_storage doesnt work because the became a after ->fill.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
list and add actions:
/patrons/{patron_id}/hold_groups
New test file:
prove t/db_dependent/api/v1/patrons_hold_groups.t
Signed-off-by: Anneli <anneli.osterman@koha-suomi.fi>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch makes the Public Biblio endpoint respect the OpacSuppression
system preference and the biblio.opac_suppressed field introduced in
Bug 38330.
When OpacSuppression is enabled and a biblio has opac_suppressed set to 1,
the endpoint now returns a 404 (Not Found) response instead of exposing
the bibliographic record.
Test plan:
1. Apply patch
2. Run: prove t/db_dependent/api/v1/biblios.t
3. Verify all tests pass, including the new OpacSuppression subtest
4. Test manually:
- Enable OpacSuppression syspref
- Set a biblio's opac_suppressed field to 1
- Access /api/v1/public/biblios/{biblio_id}
- Confirm it returns 404
- Set opac_suppressed to 0
- Confirm it returns 200 with the record
- Disable OpacSuppression syspref
- Confirm record is visible regardless of opac_suppressed value
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch adds unit tests to verify the hold override logging
functionality implemented in the previous patches. The tests cover the
logging of hold overrides in `C4/Reserves.pm` and the `Koha/REST/V1/Holds.pm`
API endpoint.
To test:
1. Run the following tests and confirm they pass:
- prove -lrv t/db_dependent/Circulation.t
- prove -lrv t/db_dependent/Reserves.t
- prove -lrv t/db_dependent/api/v1/holds.t
2. All tests should pass.
Changes:
t/db_dependent/Circulation.t:
- Added tests for hold override scenarios in circulation.
t/db_dependent/Reserves.t:
- Added tests to verify that hold overrides are correctly logged when
a hold is placed.
t/db_dependent/api/v1/holds.t:
- Added tests to ensure that the holds API correctly logs overrides
when a hold is created with the `x-koha-override` header.
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch:
* Removes unused logger object instantiation
* Adds missing Test::NoWarnings to the API tests
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch renames the entire API structure from SFTP-specific to generic
file transport terminology to reflect the polymorphic nature of the system.
API Changes:
- Endpoints: /config/sftp_servers -> /config/file_transports
- Parameters: sftp_server_id -> file_transport_id
- Definitions: sftp_server -> file_transport
- Controller: Config::SFTP::Servers -> Config::File::Transports
File Changes:
- api/v1/swagger/definitions/sftp_server.yaml -> file_transport.yaml
- api/v1/swagger/paths/config_sftp_servers.yaml -> config_file_transports.yaml
- Koha/REST/V1/Config/SFTP/Servers.pm -> Koha/REST/V1/Config/File/Transports.pm
- t/db_dependent/api/v1/sftp_servers.t -> file_transports.t
Updated References:
- swagger.yaml: paths, definitions, parameters, tags
- Template JavaScript: API URLs and variable names
- Controller: package name, variable names, error messages
- Tests: endpoints, parameters, descriptions
- API mapping: to_api_mapping field name correction
To test:
1. Apply this patch
2. Run: yarn api:bundle (to regenerate swagger bundle)
3. Run: prove t/db_dependent/api/v1/file_transports.t
4. All API tests should pass
5. Verify admin interface uses new API endpoints
6. Test CRUD operations work correctly
The API now consistently uses 'file_transports' terminology matching
the polymorphic file transport system that supports FTP, SFTP, and
future transport types.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch adds a central FTP/SFTP management UI, with an associated
database object and class, plus the beginnings of an API. We can use
this code as a building block for adding FTP/SFTP connections to
other parts of the system, as well as to improve the usability of the
EDI transports. It is also now possible to test these connections
from within the Koha UI.
To test:-
You will need access to an FTP or SFTP server
Contact me if you need a disposable FTP/SFTP server
a) Go to Koha Administration
b) Notice no FTP/SFTP referenced on this page
c) APPLY PATCH, reset_all
d) Create new user, let's say sftpuser
e) Set permissions for new user to include:
1) catalogue
2) parameters -> manage_sftp_servers
f) Log in with this new user, go back to Koha Administration
1) Notice the FTP/SFTP servers option
g) Undo step e2) for the sftpuser via the koha admin user
1) Refresh the Koha Administration page as the sftpuser
2) Notice how FTP/SFTP servers has disappeared
h) Back to koha admin, go to Koha Administration -> FTP/SFTP Servers -> New
i) Complete form depending on the FTP or SFTP server you have access to, Submit
1) Notice you are prompted to confirm before saving
j) Select Edit, observe your details are the same, and Submit again
k) Repeat step J one additional time
l) Select Test, observe all tests passing
m) Select Delete, confirm deletion, and observe server disappear
n) Optionally, run prove on new tests
1) t/Koha/Auth/Permissions.t
2) t/db_dependent/Koha/SFTP/Server.t
3) t/db_dependent/api/v1/sftp_servers.t
o) SIGN-OFF
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch introduces tests for the feature enabling patron's `lastseen`
attribute be updated when a successful password validation happens
through the API.
To test:
1. Apply this patch
2. Run:
$ ktd --shell
k$ prove t/db_dependent/api/v1/password_validation.t
=> FAIL: Tests fail! The feature is not implemented!
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Also add tests to searching for specific language code
prove t/db_dependent/api/v1/additional_contents.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
prove t/db_dependent/api/v1/additional_contents.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
01:18:38 koha_1 | # Failed test 'no warnings'
01:18:38 koha_1 | # at /usr/share/perl/5.38/Test/Builder.pm line 193.
01:18:38 koha_1 | # There were 6 warning(s)
01:18:38 koha_1 | # Previous test 4 '401 Unauthorized'
01:18:38 koha_1 | # Use of uninitialized value $aMask[1079] in bitwise xor (^) at /usr/share/perl5/GD/Barcode/QRcode.pm line 217.
01:18:38 koha_1 | # at /usr/share/perl5/GD/Barcode/QRcode.pm line 217.
jammy (22.04LTS) (perl): 1.15-7
noble (24.04LTS) (perl): 2.00-1
I guess it's fixed by
85e7911e65
But it is not released yet: version 2 is from September 2023 and commit from March 2024.
Test plan:
Install version 2 of GD::Barcode:
`cpanm GD::Barcode@2.0.0`
Run the tests
Without this patch the "no warnings" test fails
With this patch applied the tests pass
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch updates the /auth/password/validation endpoint to use the new
api_validate_password permission instead of the generic borrowers
permission.
This provides more granular control over who can use the password
validation API endpoint.
To test:
1. Apply this patches
2. Rebuild the API spec
$ ktd --shell
k$ yarn api:bundle
3. Run:
$ ktd --shell
k$ prove t/db_dependent/api/v1/password_validation.t
=> SUCCESS: Tests pass!
4. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@openfifth.co.uk>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
t/db_dependent/api/v1/holds.t .. 1/16 No reserves HOLD_CANCELLATION letter transported by email at /kohadevbox/koha/C4/Letters.pm line 602.
We do not test the notification here, we can simply ignore the letters
code
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
02:28:36 koha_1 | WARNING: The key derivation method "opensslv1" is deprecated. Using -pbkdf=>'pbkdf2' would be better.
02:28:36 koha_1 | Pass -nodeprecate=>1 to inhibit this message.
02:28:36 koha_1 | at /kohadevbox/koha/Koha/Patron.pm line 3216.
We should deal with that on a separate bug report. For now we can remove
the warnings from the log/output.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch makes the tests related to `can_place_holds` cover more use
cases, and highlight an underlying issue in the controller.
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Test plan:
1) Apply this test patch. Run. Verify it fails
2) Apply other patch. Run. Verify it passes.
prove t/db_dependent/api/v1/patrons.t
Verify no other regressions are introduced, run:
prove t/db_dependent/Koha/Patron/Attribute.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch does:
* Mock `CanItemBeReserved` and `CanBookBeReserved` to better control the
test scenario and make it agnostic from existing data.
* Adds tests for the item-level group holds use case
* Adds tests for the warnings printed when placing the hold should not
be possible
To test:
1. Apply this patch
2. Run:
$ ktd --shell
k$ prove t/db_dependent/api/v1/clubs_holds.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Tadeusz „tadzik” Sośnierz <tadeusz@sosnierz.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
prove -v t/db_dependent/api/v1/clubs_holds.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
We don't need to whole list of baskets and subscriptions
(cherry picked from commit 391b4d3b32b57c97bfb7e66ab394020ae7eed0e5)
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit cae75569e6e8f2bfe5a2e6d3acc111485f81cc1e)
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch splits the controller in two:
* A controller for staff users use
* An unprivileged access controller
A new endpoint is added for publi access:
* `/public/patrons/:patron_id/ill/requests`
The public one makes use of the state-of-the-art in Koha for public
endpoints, using the `$c->auth->public` validation helper and leaving
the (newly added) public path spec with no special permissions
requirements.
The staff-oriented endpoint gets the right permissions added (at least
those used in the main `/ill/requests` endpoint.
To test:
1. Apply this patch
2. Run:
$ ktd --shell
k$ yarn api:bundle
k$ prove t/db_dependent/api/v1/ill_requests.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Tadeusz „tadzik” Sośnierz <tadeusz@sosnierz.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Sponsored-by: Wiko <https://www.wiko-berlin.de/>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Martin Renvoize
Pedro Amorim
Jonathan Druart
Tomas Cohen Arazi
Lucas Gass
Martin Renvoize
Jake Deery
David Cook
Pedro Amorim
Nick Clemens
Tadeusz „tadzik” Sośnierz