We currently have to set options, explanation and type in sysprefs.sql and the atomic/dbrev files.
However we (almost) never use them, we rely on the values in the yml files.
We could remove them and don't require them when when add new sysprefs.
It will avoid the mess we currently have (mismatch between new and updated installations, see bug 41682 and bug 41800).
Instead of fixing the discrepancies (again, see bug 41800) we should
NULL the 3 columns in sysprefs.sql and so at the DB level to rely only
on the yaml files.
This patch does several things:
1. Remove the 3 columns from sysprefs.sql to use the default NULL
2. Atomic update:
* Set to NULL for existing installations as well
* Rename UseICUStyleQUotes => UseICUStyleQuotes
* Delete OpacMoreSearches and OPACMySummaryNote, old prefs that have been moved
to additional contents
3. Fix some errors in yaml files (mostly empty entries)
4. Move parsing of yaml files to a dedicated Koha::Devel::Syspref module
5. Retrieve all yaml content using new methods added to Koha::Config::SysPrefs
(that will be reused in follow-up bugs)
Test plan:
1. Run updatedatabase and confirm that now all systempreferences rows in DB
have options, explanation and type set to NULL (except local-use prefs)
2. `prove t/db_dependent/check_sysprefs.t` should return green
3. Confirm that the UI is still working correctly
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Bug 41494 renamed "Koha administration" to "Administration" in the
staff interface. Update the Batches_spec.ts Cypress test to use the
new label.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
The Handler::run change to skip disabled plugins broke the
RecordsFromMarcPlugin subtest in ImportBatch.t. The MarcFieldValues
plugin was installed but never enabled, so Handler::run returned
early. Add $plugin->enable() before calling RecordsFromMarcPlugin.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Apply just the 2 cypress tests patches, run:
cypress run --spec t/cypress/integration/ILL/Batches_spec.ts
Notice it fails.
Apply the fix patch. Repeat the command above. Tests must pass.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Add a way to access and manipulate koha-conf.xml (we need to ensure plugins_restricted is 0 for the actual tests)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
It appears that edi_cron.pl will run disabled EDI plugins. It uses Koha::Plugins::Handler::run that does not check if the plugins is gets are disabled.
Koha::Plugins::Handler::run needs to be fixed to not run disabled plugins.
Test Plan:
1) Apply this patch
2) prove t/db_dependent/Koha/Plugins/Plugins.t
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
The intercept is done too late, it must be done before we load the view,
and then we should wait for it.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
16:40:49 <failure message="The following error originated from your application code, not from Cypress. It was caused by an unhandled promise rejection.
16:40:49
16:40:49 > response.filter is not a function
16:40:49
16:40:49 When Cypress detects uncaught errors originating from your application it will automatically fail the current test.
16:40:49
16:40:49 This behavior is configurable, and you can choose to turn this off by listening to the `uncaught:exception` event.
16:40:49
16:40:49 https://on.cypress.io/uncaught-exception-from-application" type="TypeError"><![CDATA[TypeError: The following error originated from your application code, not from Cypress. It was caused by an unhandled promise rejection.
16:40:49
16:40:49 > response.filter is not a function
16:40:49
16:41:13 When Cypress detects uncaught errors originating from your application it will automatically fail the current test.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Test plan:
0. Apply the patch
1. Run the following:
prove t/db_dependent/FrameworkPlugin.t \
t/db_dependent/Koha/Plugins/Valuebuilder_hooks.t \
t/db_dependent/Koha/UI/Form/Builder/Biblio.t \
t/db_dependent/Koha/UI/Form/Builder/Item.t \
t/db_dependent/Serials.t
Squashed version of the following patches:
Bug 38365: Fix Koha/UI/Form/Builder/Biblio.t
# Failed test at t/db_dependent/Koha/UI/Form/Builder/Biblio.t line 216.
# '<script nonce="">
# $(document).ready(function () {
#
# });
# </script>
# '
# doesn't match '(?^us:<script>.*</script>)'
# Looks like you failed 1 test of 4.
Bug 38365: Fix Koha/UI/Form/Builder/Item.t
# There were 16 warning(s)
# Previous test 0 ''
# Use of uninitialized value $csp_nonce in concatenation (.) or string at /kohadevbox/koha/Koha/FrameworkPlugin.pm line 381.
# at /kohadevbox/koha/Koha/FrameworkPlugin.pm line 381.
Bug 38365: Fix Serials.t
Bug 38365: Fix FrameworkPlugin.t
We never do that in our tests, and KOHA_HOME is not defined for
the kohadev-koha user in ktd.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This file now tidies after the patch
"Make tidy.pl compatible with script elements with attributes"
To test:
1. prove t/db_dependent/Koha/Middleware/ContentSecurityPolicy.t
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This initialises the CSP nonce in the unit test so that
t/lib/plugins/Koha/Plugin/TestValuebuilder/test_valuebuilder_popup.tt
will get its script element rewritten correctly.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Schema type [array,object] is not valid in OpenAPI version 2.
There are two different CSP reporting directives.
One is the now deprecated "report-uri" directive. Browsers typically send a single
JSON object following the CSP2 specification.
"report-uri" is deprecated in CSP3 in favor of the newer "report-to" directive. It
is expected to contain an array of report objects.
There are browser that do not yet support the "report-to" directive, so both
directives should be supported.
There are also variations among the "report-uri" format. See
https://www.tollmanz.com/content-security-policy-report-samples/
Because of these variations, it becomes quite complex to handle the most common
cases in a single API endpoint.
We could build two separate endpoints for both directives.
Or we can accept any type of input and handle the variations in a single endpoint.
This patch attempts to do that.
Before this patch input parameters were validated by JSON::Validator. That is the
normal and recommended way to build OpenAPI endpoints in Koha. After this patch we
remove all input parameter restrictions from the OpenAPI spec, and simply accept
any type of body.
This means the controller will now always return a HTTP 204 regardless of the type
of data passed to the controller. It instead truncates too long input before
writing them into log files.
As this is not an endpoint that developers will build new Koha clients for, we
do not have to offer a very in-depth input parameter spec for this endpoint.
To test:
1. t/db_dependent/api/v1/public/csp_reports.t
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
It seems the test outside of db_dependent required presence of database after all
To test:
1. prove t/db_dependent/Koha/Middleware/ContentSecurityPolicy.t
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
1. prove t/Koha/ContentSecurityPolicy.t
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
1. prove t/db_dependent/Koha/Template/Plugin/Koha.t
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
By defining a default policy in the Perl module, we can easily update
it over time. Using koha-conf.xml will just be for overrides, so people
can manage their own there.
Signed-off-by: Lari Taskula <lari.taskula@hypernova.fi>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
1. prove t/db_dependent/api/v1/public/csp_reports.t
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch fixes failing tests from the previous patch and adds more tests that
cover multiple violation reports in one POST request.
To test:
1. prove t/db_dependent/api/v1/public/csp_reports.t
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch attempts to set some sane input limits to the CSP violation report.
To test:
1. Rebuild API
redocly bundle --ext json api/v1/swagger/swagger.yaml --output api/v1/swagger/swagger_bundle.json
2. t/db_dependent/api/v1/public/csp_reports.t
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
report-uri is deprecated, replaced by report-to and Reporting-Endpoints.
To test:
1. prove t/db_dependent/Koha/Middleware/ContentSecurityPolicy.t
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds the default configuration to log4perl.conf.
Note we are passing a non-standard interface 'csp' to Koha::Logger. This is to
avoid log entries from propagating up to api-error.log.
To test:
1. prove t/db_dependent/api/v1/public/csp_reports.t
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Lari Taskula <lari.taskula@hypernova.fi>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Add /api/v1/public/csp-reports endpoint to receive and log
Content-Security-Policy violation reports from browsers.
When CSP is configured with a report-uri directive pointing to this
endpoint, browsers will POST violation reports whenever a CSP rule
is violated. These reports are logged using Koha's logging system
for administrator review.
Features:
- Public endpoint (no authentication required, as browsers send
these reports automatically)
- Logs violations at WARN level with key details (violated directive,
blocked URI, document URI, source location)
- Logs full report at DEBUG level for detailed analysis
- Accepts both application/csp-report and application/json content types
Configuration example (in koha-conf.xml csp_header_value):
...; report-uri /api/v1/public/csp-reports
Log4perl configuration to capture CSP violations:
log4perl.logger.api.csp = WARN, CSP
log4perl.appender.CSP = Log::Log4perl::Appender::File
log4perl.appender.CSP.filename = /var/log/koha/csp-violations.log
log4perl.appender.CSP.layout = PatternLayout
log4perl.appender.CSP.layout.ConversionPattern = [%d] %m%n
Also updates koha-conf.xml documentation with CSP configuration guidance.
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Lari Taskula <lari.taskula@hypernova.fi>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
OWASP recommends CSP nonces have at least 128 bits of entropy.
The previous 10-character alphanumeric nonce provided only ~59 bits
(log2(62^10) ≈ 59.5).
Increase to 22 characters which provides ~130 bits of entropy
(log2(62^22) ≈ 130.5), exceeding the OWASP recommendation.
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Lari Taskula <lari.taskula@hypernova.fi>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds a new Template plugin method, Koha.CSPNonce, that can be used to
attach the Content Security Policy nonce into our templates.
To test:
1. prove t/Koha/Middleware/ContentSecurityPolicy.t
2. prove t/db_dependent/Koha/Template/Plugin/Koha.t
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds a new Perl package, Koha::Middleware::ContentSecurityPolicy.
It is called for each Plack requests, and its job is to add the
Content-Security-Policy[-Report-Only] header with the configured value.
To test:
1. prove t/Koha/Middleware/ContentSecurityPolicy.t
2. prove t/db_dependent/Koha/Middleware/ContentSecurityPolicy.t
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds a new Perl package, Koha::ContentSecurityPolicy.
It provides the following methods:
$csp->header_name
Returns the name of the CSP header, based on koha-conf configuration.
$csp->header_value
Returns the value of the CSP header, based on koha-conf configuration.
$csp->is_enabled
Returns true or false, based on koha-conf configuration.
$csp->nonce
Generates or sets, and returns the nonce. A CSP nonce is a random token that is
used both in the inline scripts and the Content-Security-Policy[-Report-Only]
response header.
To test:
1. prove t/Koha/ContentSecurityPolicy.t
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
I think this happens because we select an option that didn't exist in
the dropdown, and "Sorry, there is no matching options" from select2 is
covering the discount input that is right under "Tax rates"
09:45:36 <testcase name="Vendor CRUD operations should add a vendor" time="0.000" classname="should add a vendor">
09:45:36 <failure message="Timed out retrying after 10000ms: `cy.type()` failed because this element:
09:45:36
09:45:36 `<input data-v-2a08b27a="" id="discount" inputmode="numeric" placeholder="Discount (%)" size="6">`
09:45:36
09:45:36 is being covered by another element:
09:45:36
09:45:36 `<li class="vs__no-options">Sorry, ...</li>`
09:45:36
09:45:36 Fix this problem, or use {force: true} to disable error checking.
09:45:36
09:45:36 https://on.cypress.io/element-cannot-be-interacted-with" type="CypressError"><
David Cook
Jonathan Druart
Martin Renvoize
Pedro Amorim
Kyle M Hall
Lari Taskula
Baptiste Wojtkowski
Nick Clemens
phette23
Laura_Escamilla