This patch is the Koha part of the Hea v2 project.
You can find the (testing) code for the server at
hea-ws - https://github.com/joubu/hea-ws/commits/v2
hea-app - https://github.com/joubu/hea-app/commits/v2
They contain the different pull requests made over the last 6 months.
More information on Hea at https://wiki.koha-community.org/wiki/KohaUsageStat_RFC
The goal of this commit message is to provide an overview of what could
be a new version of Hea.
Prior to these changes, the Hea database was filled with 1 line per Koha
installation. System preferences were filled by the libraries and a
cronjob (share_usage_with_koha_community.pl) collected these values to send
them to a webservice (hea-ws/upload.pl).
With the need to collect more data we would want to collect data at the library
level (branch) and not at the installation level.
For instance the geolocation, the url or the country can be different from one
library to another, even if managed from the same Koha installation.
The Hea DB has been upgraded to reflect that change (see hea-app/sql/schema.sql).
The hidden goal of this patch is to make Hea sexier and explain
better to libraries how it can be useful to share their information
with the Koha community. I guess the main problem is the lack of
communication and explanations about what we are doing we these data.
To fill this gap I'd like to (TODO)
1. Communicate on the ML about this new version of Hea (once it got
pushed and backported)
2. Link the Privacy_Policy.md from the Hea interface
3. Get help from a native English speaker to add
popup/help/info/whatever on "Home › Administration › Usage statistics",
to clearly explain what happens (and what will not happen!) when an option or
another is set.
You can find screenshot of this whole enhancement on bug 18066, comment 2.
What this patch does:
- Create a new branches.geolocation DB field
- Add 3 new sysprefs:
* UsageStatsGeolocation
* UsageStatsLibrariesInfo
* UsageStatsPublicID
- Integrate the Leaflet JS library to get a fancy map to pick
geolocations
How does it works:
On the new administration page where statistics to share are configured,
there are several new things. It is now possible to share information either
per Koha installation or libraries. If UsageStatsLibrariesInfo is set,
the info at library level (url, name, country, geolocation) will be
sent to the Hea webservice. If it is not set, you can decide to fill
UsageStatsLibraryUrl, UsageStatsLibraryName, UsageStatsCountry,
UsageStatsGeolocation to share these information. Note that even if the
data are retrieved at installation level, it's better to fill the prefs
as well: On the Hea website the different libraries defined for a given
Koha installation could be displayed on the same page.
This page is a public page which will be attributed to every
installation (with the pref UsageStatsPublicID). On this page all the
info available publicly will be displayed.
TODO later:
- Add a button on the administration page to delete the info shared
publicly. It will be easy to show that the info are no longer displayed
on the public page.
- Add an icon per Koha installation to get a better "public page"
- Any suggestions?
Test plan:
We will need to test hea-ws, hea-app and the Koha-side code to test the
whole enhancement.
1/ To start, clone the hea-ws and hea-app project and checkout the
'master' branch (*not* 'v2')
2/ Create the hea database and user
CREATE DATABASE hea
CREATE USER 'hea'@'localhost' IDENTIFIED BY 'hea';
GRANT ALL PRIVILEGES ON hea.* TO 'hea'@'localhost';
FLUSH PRIVILEGES;
3/ Fill the DB with some data
mysql hea < hea-app/sql/schema.sql
mysql hea < hea-app/sql/sql/mock-data.sql
4/ Checkout the 'v2' branch for both hea-ws and hea-app
5/ Execute the upgrade DB script
% cd hea-app
% perl -p -i -e 's/REPLACE_ME/hea/' sql/upgrade.pl # Fill the DB info
% perl sql/upgrade.pl
Now the DB is using the v2 structure. That means we have 1 installation
row per library previously defined. 1 library row has also been created.
5/ Configure hea-ws
% echo '192.168.50.1 hea.koha-community.org' >> /etc/hosts
<VirtualHost *:80>
DocumentRoot "/path/to/hea-ws"
ServerName "hea.koha-community.org"
<Directory "/">
Options +ExecCGI
Require all granted
AddHandler cgi-script .pl
</Directory>
</VirtualHost>
And enable it with a2ensite, then restart apache.
The copy the database.yml.sample to database.yml and edit it to fill the
DB info.
6/ Launch the hea-app
% cd hea-app
% edit README.md # to install the missing modules
% cp environments/config.yml environments/development.yml
% edit environments/development.yml # to fill the DB info
% perl bin/app.pl
Then hit localhost:3000
You should see a local version of Hea with sample data
7/ Back to Koha side
A. We will test that the webservice still works with previous version of Koha (without v2)
a. Do not configure Hea
% perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
Then hit localhost:3000
=> Nothing added
b. Configure Hea on admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> New library added
c. Modify the Hea configuration
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Info are modified
B. Not we will test that it works with the new version (much more fun ;))
% git checkout hea-v2 # koha
a. Configure Hea using /admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Check the result on localhost:3000
b. Share libraries's info
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
c. Continue to play a bit and share the info.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Mason James <mtj@kohaaloha.com>
The update_dbix_class_files.pl script generates ActionLog.pm instead, which is
already on the source tree.
To test:
- Apply the patch
=> SUCCESS: Koha/Schema/Result/ActionLogs.pm is removed
- Run:
$ mysql -uroot
> CREATE DATABASE dbic; \q
$ mysql -uroot dbic < kohaclone/installer/data/mysql/kohastructure.sql
$ misc/devel/update_dbix_class_files.pl --db_name dbic --db_user root
=> SUCCESS: Koha/Schema/Result/ActionLogs.pm is not re-generated
- Run:
$ git grep ActionLogs
=> SUCCESS: There's no code using it
- Sign off
Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit b8c950a4c1)
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
This allows creation of special baskets that include standing orders.
These orders do not have a known quantity (and may not have a known
price in advance). Upon receipt, the received items are split into a new
completed order.
Test plan:
1) Run updatedatabase.pl.
2) Run prove t/db_dependent/Acquisition/StandingOrders.t . (and the
other Acquisition tests).
3) Create a new basket, mark it as a standing order basket.
4) Add an order to this basket, and notice that the quantity field is
missing (and thus not required).
5) Receive items for this order, and notice that the original order is
unchanged. The new child order line should have the correct price
and quantity information.
(Note: the QA tools output what seems to be a spurious spelling error
for Test::More's "isnt" in StandingOrders.t.)
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Some libraries would like the ability to select the itemtype to request
when placing holds. For example, if a record has 3 copies of BookA and 3
copies of BookA in large print, this feature would allow a person to
place a hold on the record, but still be able to target only the Large
Print edition so that the first Large Print copy that becomes available
is targeted, rather than forcing the patron to select a particular copy
to hold.
Test Plan:
1) Apply this patch
2) Run updatedatabase.pl
3) Create a record with items of two or more itemtypes
4) Place a record level hold on the record while choosing one particular
itemtype
5) Check in an item from the record that is not of that itemtype
6) Notee it is not trapped for the hold
7) Check in an item from the record that does match the selected itemtype
8) Note the item is trapped for the hold
Signed-off-by: Andreas Hedström Mace <andreas.hedstrom.mace@sub.su.se>
Signed-off-by: Benjamin Rokseth <benjamin.rokseth@kul.oslo.kommune.no>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
As requested by QA, the sub should be moved.
Test plan:
See previous patches.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
The three permissions in tools/upload-file.pl are moved to a custom
routine in DBIx's UploadedFile.pm. An additional granular permission
upload_general_files is added. (The dbrev patch will contain it too.)
At some point in time this could be moved to a Koha::Object class.
The routine is tested in Upload.t.
Test plan:
[1] Run t/db_dependent/Upload.t
[2] If you only apply this patch, you can also test uploading with a
user that has edit_catalogue but does not have tools or circulate.
Upload status should say: Denied.
Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
This will create the add_to_* accessors for us.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
These files have been generated with the
misc/devel/update_dbix_class_files.pl script.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
This patch applies several changes to the DB structure introduced by
the previous ES patch set:
1/ Add a search_field.label column
=> It will be easier to understand what the search_field does.
Indeed, some are not user friendly: acqdate, an, pl, ff8-23, ln, etc.
TODO later: Update the labels with correct values (at the moment
label=name)
2/ Update the foreign key search_marc_to_field.search_marc_map_id with
on delete cascase. This will permit to automatically remove the entries
in search_marc_to_field when they are deleted in search_marc_map.
3/ Remove the index_name_2 unique constraint on the search_marc_map table.
I don't understand how this could be useful, it was defined like (index_name,
marc_type, marc_field), so it means a mapping cannot be defined twice with
different values for facet, suggestible and sort.
This limitation does not seem to make sense.
Robin, please correct me if I am wrong :)
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
This allows sorting to be configured within a field. For example, while
many values are included for search on author, sorting should only be
done on the main entry values. This permits that by have a sort value,
which can be true, false, or null. true and null are pretty much the
same, but false means that a field isn't available for sorting on. By
default (null), fields can be sorted on.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
This makes the mapping schema a proper many-to-many relational
structure, which is a little more fiddly to work with, but will make
things like editors for it easier to write.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
(Not fetched yet though.)
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
msg_invoice is not used merged into patch in error
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Koha's EDIFACT module works great for many European vendors,
but does not work will for US vendors, which have a much different
interpretation of 'standard'. In fact, each vendor may require
different arrangements of values in EDIFACT messages. It would be
impossible to encompass all these requirements within Koha's EDIFACT
module itself. Instead, we should allow the module to be pluggable, so
versions of the module can be developed for vendors that require EDIFACT
messages that don't conform to the standard set by Koha's EDIFACT
module.
Test Plan:
1) Apply this patch
2) Run updatedatabase
3) Enable Koha plugins
4) Install the Edifact stub plugin available at
https://github.com/bywatersolutions/koha-plugin-edifact-stub
5) Edit the EDI Vendor account, assign the plugin to a Vendor EDI account
6) Test EDI functionality ( ORDER, INVOICE ), there should be no errors
or changes to the EDIFACT message input or output
Signed-off-by: Jason DeShaw <JDeShaw@cityoffargo.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Add support for processing incoming Edifact Quotes, Invoices
and order responses and generating and transmission of
Edifact Orders.
Basic workflow is that an incoming quote generates an aquisition
basket in Koha, with each line corresponding to an order record
The user can then generate an edifact order from this (or another)
basket, which is transferred to the vendor's site
The supplier generates an invoice on despatch and this will
result in corresponding invoices being generated in Koha
The orderlines on the invoice are receipted automatically.
We also support order response messages. This may include
simple order acknowledgements, supplier reports/amendments
on availability. Cancellation messages cause the koha order
to be cancelled, other messages are recorded against the order
Which messages are to be supported/processed is specifiable on a
vendor by vendor basis via the admin screens
You can also specify auto order i.e. to generate orders from quotes
without user intervention - This reflects existing
workflows where most work is done on the suppliers website
then generating a dummy quote
Received messages are stored in the edifact_messages table
and the original can be viewed via the online
Database changes are in installer/data/mysql/atomicchanges/edifact.sql
Note new perl dependencies:
Net::SFTP:Foreign
Text::Unidecode
Signed-off-by: Paul Johnson <p.johnson@staffs.ac.uk>
Signed-off-by: Sally Healey <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Otherwise some tests won't pass
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
This ensures that the DBIx structure matches the SQL structure changes.
See comment #48 as to how this was generated.
Also, I had to modify misc/devel/update_dbix_class_files.pl
to have an "overwrite_modifications => 1" option to get it to generate.
This should correct issues raised in comment #45.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch includes:
[1] Adds primary key borrowernumber to new table.
[2] Fixes collation.
[3] Removes manual PK in DBIx schema file.
[4] Fixes typo CompletePasswordRevovery.
[5] Removes use strict from opac-password-recovery; Modern::Perl is used.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
This is a collection of changes taken from different comments (but mostly comment 21 and comment 122).
Passes qa and prove, on my machine at least.
There's also a new test file, Passwordrecovery.t, which covers every method of C4::Passwordrecovery.
To test:
All normal checks plus :
1/ Receive the email
2/ Click on the link
3/ Change the pwd
4/ Click again on the link
5/ You should immediately get an error message
Problems with Math/Random/Secure.pm, is solved in following patch, signing off
Signed-off-by: Marc Veron <veron@veron.ch>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
I've addressed a lot of Liz Rea's points.
1. I have moved the code from updatedatabase.pl and kohastructure.sql to a file in the atomicupdates directory.
1a. The feature is now off by default when the atomicupdate is run.
2. The password reset link is now visible on the home page, in the modal box and on opac-user.pl .
3. The password recovery pages now use bootstrap markup.
4. I am unsure here. I see "New Password:" and "Confirm new password:".
5. This should still work :).
6. I could not reproduce.
7. I have added the userid field.
You can now reset the password by submitting either your useid or email address.
Both fields can be filled, but the email address must be one of the borrower's (email, emailpro or b_email).
When entering only the email address and two borrowers use that same address, the system tells the user to try with another address or to specify his userid.
8. The text is in the atomicupdate file. Have at it, anyone.
Concerning the email. It is inconvenient for the use to have to wait X minutes for the message queue the be processed.
Maybe we could add a sub in Letters.pm that:
Takes the same argments as EnqueueLetter
Sends the letter.
Saves the letter in the message queue with a 'sent' status.
TEST PLAN:
Setup)
1) apply the patch
2) go to system preferences OPAC>>Privacy and set 'OpacResetPassword' to ON.
2b) make sure that OpacPasswordChange is also ON.
A)
1) refresh front page, click on 'Forgot your password' and enter a VALID address
1b) Also try an INVALID address (valid yet not in your koha db). An error message will show up.
2) An email should be received at that address with a link.
3) Follow the link in the mail to fill the new password.
Until a satisfactory new password is entered, the old password is not reset.
4) Go to main page try the new password.
B)
1) Repeat the password reset, this time use the userid (username) field.
2) Try to reset the password using a userid and an email not linked to the account. An error appears.
3) Make sure the borrower has many available email addresses.
4) For each email, reset the password using both the userid and the email. The link should be sent to the specified address
C)
1) Make sure two borrowers use the same email.
2) Repeat the reset procedure in test case A). An error message appears
http://bugs.koha-community.org/show_bug.cgi?id=13068
Author: Maxime Beaulieu <maxime.beaulieu@inlibro.com>
Followed test plan. Works as described.
Signed-off-by: Marc Veron <veron@veron.ch>
New sign-off after testing all patches together
Signed-off-by: Marc Veron <veron@veron.ch>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
When regenerating the db schema on master, the script will error out
with a checksum mismatch on Borrower.pm, and also DeletedBorrower.pm
Test Plan:
1) Check out master
2) Run ./misc/devel/update_dbix_class_files.pl
3) Note the error
4) Apply this patch
5) Re-reun ./misc/devel/update_dbix_class_files.pl
6) Note there is no error
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
When regenerating the db schema on master, the script will error out
with a checksum mismatch on Borrower.pm, and also DeletedBorrower.pm
Test Plan:
1) Check out master
2) Run ./misc/devel/update_dbix_class_files.pl
3) Note the error
4) Apply this patch
5) Re-reun ./misc/devel/update_dbix_class_files.pl
6) Note there is no error
Currently if the AnonymousPatron system preference is in use, all patron
data is anonymized. Some libraries would like to be able to see the last
patron who returned out an item ( in case of damage ) but still keep all
other patrons anonymized.
* Add the table items_last_borrower ( id, itemnumber, borrowernumber )
* Add new system preference StoreLastBorrower
* If StoreLastBorrower is enabled, upon checkin have Koha insert into
this table the patron who last returned this item. Replace existing
row based on itemnumber if exists.
* If table has a row for a given item, link to the patron from the item
details page.
Test plan:
1) Apply patch
2) Run updatedatabase.pl
3) Enable StoreLastBorrower
4) Issue an item to a patron and return said item
5) Issue the same item to a second patron, do not return it.
6) View moredetail.pl for the given bib, find the given item. There
should be a new field in the history list 'Last returned by' with a link
to the last patron to return the item.
Optionally, you can also verify this works even if patron issuing
history has been set to anonymize issues upon return.
Signed-off-by: Nick Clemens <nick@quecheelibrary.org>
Signed-off-by: Jen DeMuth <JDeMuth@roseville.ca.us>
Signed-off-by: Tom Misilo <misilot@fit.edu>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
Signed-off-by: Cindy Murdock Ames <cmurdock@ccfls.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Katrin Fischer
Jonathan Druart
Mason James
Kyle M Hall
Tomas Cohen Arazi
Jesse Weaver
Marcel de Rooy
Robin Sheat
Colin Campbell
Brendan A Gallagher
Mark Tompsett
Charles Farmer
Maxime Beaulieu