gitea
/
Koha-clms
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Koha-clms/Koha
History
David Gustafsson cbd56aa5fc Bug 24807: [20.05.x] Add "year" type to improve sorting behaviour 
Add a "year" search field type. Fields with this type will only
retain values that looks like years, so invalid values such as
whitespace or word characters will not be indexed.
This for instance improves the behaviour when sorting by
"date-of-publication". If all values are indexed, records with
junk data instead of valid years will appear first among the search
results, drowning out more relevant hits. If assigning this field
the "year" type these records will instead always appear last,
regarless of sort order.

To test:

1) Have at least two biblios, one with a valid year in 008 (pos 7-10)
and another with an invalid one ("uuuu" for example)
2) Perform a wildcard search (*) and sort results by publication date.
3) The record with invalid year of pulication in 008 should appear first
4) Apply patch and run database updates
5) Reindex ElasticSearch
6) Perform the same search as in 2)
7) The record with the invalid year should now appear last

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Bug 24807: Add database update script

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Bug 24807: Update tests

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Bug 24807: Add suppport for uncertain fields and ranges

To test:
1 - Have some records with uncertain dates in the 008
    19uu, 195u, etc.
2 - Index them in Elasticsearch
3 - Do a search that will return them
4 - Sort results by publication/copyright date
5 - Note odd results
6 - Apply patch
7 - Reindex
8 - Sorting should be improved

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Bug 24807: Refactor using tokenize_callbacks

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Bug 24807: Simplify with new and imporved value_callbacks

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Bug 24807: (follow-up) Fix spelling

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Bug 24807: (follow-up) Add support for spaces as unknown characters

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Bug 24807: (QA follow-up) Remove uneccessary tests

These tests fail now, the code expects a real response from ES in Indexer.pm
but these tests mock 'bulk' and so don't have the necessary fields.

We are testing the same code above and can just add the _id == biblionumber test

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
 		2020-10-16 14:27:07 +00:00
..
Account Bug 25998: (QA follow-up) Add DBIC level relation 2020-07-28 14:55:05 +00:00
Acquisition Bug 24161: (follow-up) Fix typo in POD 2020-05-04 12:48:17 +01:00
ArticleRequest Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Authority Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Biblio Bug 24827: Standardise on 'UTF-8' as the encoding name 2020-03-16 14:45:39 +00:00
BiblioUtils Bug 24827: Specify UTF-8 as encoding in new_from_xml() 2020-03-16 14:45:47 +00:00
Cache Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Cash Bug 23185: (follow-up) K:C:Register->make_default should use no_triggers 2020-05-08 13:14:59 +02:00
Charges Bug 21443: Remove the dependency on finesCalendar 2020-03-24 11:09:16 +00:00
Checkouts Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Club Bug 23820: Add checkout to default to patron's home branch on club hold 2020-07-27 16:40:08 +00:00
Config Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Course Bug 23727: Editing course reserve items is broken 2020-04-17 13:45:56 +01:00
Edifact Bug 24840: Replace DateTime->now with dt_from_string 2020-04-08 11:54:23 +01:00
Exceptions Bug 25423: Add Koha::Exceptions::Object::NotInstantiated 2020-05-08 15:42:37 +02:00
Exporter
…
ExternalContent Bug 25527: Add logger to Koha::ExternalContent 2020-05-18 15:25:03 +01:00
Filter/MARC Bug 25019: Make the ViewPolicy filter initialization standard 2020-04-08 11:45:47 +01:00
Illrequest Bug 23173: (follow-up) Fix QA script errors 2020-04-06 11:05:14 +01:00
Indexer Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Item Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Library Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
List
…
MetadataRecord Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Middleware Bug 25950: Remove <client> in X-Forwarded-For from proxy tests 2020-07-24 21:22:17 +00:00
Misc
…
Notice Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Number
…
OAI/Server Bug 24052: Rename XSLT_Handler 2020-03-24 10:42:23 +00:00
Object Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Objects Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Old Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Patron Bug 24840: Replace DateTime->now with dt_from_string 2020-04-08 11:54:23 +01:00
Patrons Bug 24379: Fix failing tests 2020-07-24 22:45:38 +00:00
Plugins Bug 25099: Sending a LANG variable to plug-in template 2020-05-11 09:49:07 +01:00
REST Bug 26322: Permissions not checked correctly for plugins 2020-09-22 23:12:43 +00:00
RecordProcessor
…
SMS Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Schema Bug 23797: DBIC Schema Changes 2020-07-31 20:03:37 +00:00
SearchEngine Bug 24807: [20.05.x] Add "year" type to improve sorting behaviour 2020-10-16 14:27:07 +00:00
Serial Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Sitemapper Bug 24840: Replace DateTime->now with dt_from_string 2020-04-08 11:54:23 +01:00
Subscription Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
SuggestionEngine
…
Tags Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Template/Plugin Bug 25765: (QA follow-up) Save a DB hit 2020-07-30 17:30:41 +00:00
Util Bug 26127: local_referer should not compare with OPACBaseURL case insensitive 2020-09-03 15:23:13 +00:00
XSLT Bug 23290: (RM follow-up) Correction to copyright statement 2020-02-25 14:35:39 +00:00
Z3950Responder Bug 25277: Don't specify a field in query if none passed 2020-05-18 15:27:11 +01:00
pdfformat Bug 25507: Fix table in basket group PDF German 2-pages 2020-06-17 22:23:33 +00:00
Account.pm Bug 24474: (follow-up) Add LOST_FOUND logic to Koha::Account::pay 2020-04-03 14:26:32 +01:00
ActionLog.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
ActionLogs.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
AdditionalField.pm
…
AdditionalFieldValue.pm
…
AdditionalFieldValues.pm
…
AdditionalFields.pm
…
AdvancedEditorMacro.pm Bug 17268: (follow-up) Fix translatability 2020-05-04 08:26:22 +01:00
AdvancedEditorMacros.pm Bug 17268: (follow-up) Fix translatability 2020-05-04 08:26:22 +01:00
ApiKey.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
ApiKeys.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
ArticleRequest.pm Bug 23384: Fix use statement order for ArticleRequest::Status 2020-03-06 15:00:25 +00:00
ArticleRequests.pm Bug 23384: Fix use statement order for ArticleRequest::Status 2020-03-06 15:00:25 +00:00
AudioAlert.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
AudioAlerts.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
AuthUtils.pm Bug 20370: Misleading comment for bcrypt - #encrypt it; Instead it should be #hash it 2020-04-14 08:35:24 +01:00
AuthorisedValue.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
AuthorisedValueCategories.pm Bug 17355: (QA follow-up) Fix used package name 2020-05-07 19:10:29 -03:00
AuthorisedValueCategory.pm Bug 17355: (QA follow-up) Fix used package name 2020-05-07 19:10:29 -03:00
AuthorisedValues.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Authorities.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Authority.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Biblio.pm Bug 25421: Make ->pickup_locations only return an arrayref 2020-05-12 10:59:58 +01:00
BiblioFramework.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
BiblioFrameworks.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
BiblioUtils.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Biblioitem.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Biblioitems.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Biblios.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Cache.pm Bug 13193: Make Memcached usage fork safe 2020-02-07 09:58:47 +00:00
Caches.pm
…
Calendar.pm Bug 25723: (QA follow-up) Handle holiday and exception on same day 2020-07-27 17:40:25 +00:00
Checkout.pm Bug 24840: Replace DateTime->now with dt_from_string 2020-04-08 11:54:23 +01:00
Checkouts.pm Bug 24840: Replace DateTime->now with dt_from_string 2020-04-08 11:54:23 +01:00
CirculationRule.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
CirculationRules.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Cities.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
City.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
ClassSortRule.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
ClassSortRules.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
ClassSource.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
ClassSources.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
ClassSplitRule.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
ClassSplitRules.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Club.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Clubs.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Config.pm
…
Course.pm Bug 23727: Add Koha Object(s) related to course reserves 2020-04-17 13:45:51 +01:00
Courses.pm Bug 23727: Add Koha Object(s) related to course reserves 2020-04-17 13:45:51 +01:00
CsvProfile.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
CsvProfiles.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Database.pm Bug 22001: Set unsafe only if RaiseError is not set 2020-03-31 15:17:23 +01:00
DateUtils.pm Bug 25617: (bug 25133 follow-up) 12 PM is not 24 but 0 2020-05-29 11:30:02 +02:00
Desk.pm Bug 13881: Add desk management 2020-03-25 09:19:15 +00:00
Desks.pm Bug 13881: Add desk management 2020-03-25 09:19:15 +00:00
EDI.pm Bug 26082: Call store on new items to update itemnumber 2020-09-04 16:34:36 +00:00
Edifact.pm
…
Email.pm Bug 24826: Remove warnings from Sendmail 2020-04-08 11:47:19 +01:00
Exceptions.pm Bug 24080: Add 'payout' method to Koha::Account::Line 2020-01-03 16:35:38 +00:00
ExternalContent.pm Bug 25527: Add logger to Koha::ExternalContent 2020-05-18 15:25:03 +01:00
FrameworkPlugin.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Hold.pm Bug 25535: Hold API mapping maps cancellationdate to cancelation_date, but it should be cancellation_date 2020-05-19 08:30:08 +01:00
Holds.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
I18N.pm Bug 25517: Look in all possible places for MO files 2020-05-23 08:31:21 +01:00
Illcomment.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Illcomments.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Illrequest.pm Bug 24840: (follow-up) Catch some further cases of DateTime->now 2020-04-08 11:54:36 +01:00
Illrequestattribute.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Illrequestattributes.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Illrequests.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Item.pm Bug 25265: [20.05.x] Prevent double reindex of the same item in batchmod 2020-10-16 14:15:22 +00:00
ItemType.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
ItemTypes.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Items.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
KeyboardShortcut.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
KeyboardShortcuts.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Libraries.pm Bug 24368: Remove Koha::Libraries->pickup_locations 2020-05-13 10:30:41 +01:00
Library.pm Bug 22823: return undef if KohaAdminEmailAddress not exist 2020-03-18 15:44:27 +00:00
Linktracker.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Localization.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Localizations.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Logger.pm Bug 25172: Remove _recheck_logfile 2020-04-22 13:43:58 +01:00
Manual.pm Bug 25576: Add link to the manual for the ILL module 2020-06-17 23:04:02 +00:00
MarcSubfieldStructure.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
MarcSubfieldStructures.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
MetaSearcher.pm Bug 24173: Advanced Editor: Show subtitle & published date on the search page 2020-01-03 16:21:29 +00:00
MetadataIterator.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
MetadataRecord.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
MoreUtils.pm
…
News.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
NewsItem.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
OAuth.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
OAuthAccessToken.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
OAuthAccessTokens.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Object.pm Bug 25638: Update TODO 2020-06-16 21:48:28 +00:00
Objects.pm Bug 25423: Force call to update and empty with an instanciated object 2020-05-08 15:42:37 +02:00
Patron.pm Bug 23634: Add is_superlibrarian method to Koha::Patron 2020-08-31 20:00:06 +00:00
Patrons.pm Bug 25683: (follow-up) Use COALESCE instead of IFNULL 2020-08-14 21:13:18 +00:00
Plugins.pm
…
ProblemReport.pm Bug 4461: Update license statements 2020-04-06 11:18:39 +01:00
ProblemReports.pm Bug 4461: Update license statements 2020-04-06 11:18:39 +01:00
RDF.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Rating.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Ratings.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
RecordProcessor.pm Bug 25008: Overload Koha::RecordProcessor->options to update filters 2020-04-14 08:31:37 +01:00
RefundLostItemFeeRules.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Report.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Reports.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Review.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Reviews.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Schema.pm
…
Script.pm Bug 23571: (QA follow-up) Remove perlcritic warning 2020-04-14 17:24:35 +01:00
SearchEngine.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
SearchField.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
SearchFields.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
SearchMarcMap.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
SearchMarcMaps.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Serial.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Serials.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
SharedContent.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
SimpleMARC.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Sitemapper.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Statistic.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Statistics.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
StockRotationItem.pm Bug 24840: (follow-up) Catch some further cases of DateTime->now 2020-04-08 11:54:36 +01:00
StockRotationItems.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
StockRotationRota.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
StockRotationRotas.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
StockRotationStage.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
StockRotationStages.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Subscription.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Subscriptions.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Suggestion.pm Bug 23591: Add some useful methods to Koha::Suggestions 2020-04-14 16:36:13 +01:00
SuggestionEngine.pm
…
Suggestions.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Tag.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Tags.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Token.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
UploadedFile.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
UploadedFiles.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Uploader.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Virtualshelf.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Virtualshelfcontent.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Virtualshelfcontents.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Virtualshelfshare.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Virtualshelfshares.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Virtualshelves.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Z3950Responder.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Z3950Server.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00
Z3950Servers.pm Bug 24545: Fix license statements 2020-02-24 13:31:26 +00:00