# Release Notes for Spinque Desk ## June 8, 2025 Release 4.15 ### Editor * Multiple attributes in block configuration * Blocks for vector similarity * Blocks for using CIFF-indices ### Datasets * Pipeline and dataset version cleanup * Multiple iterations of post-processing actions * Faster JSON processing * Various improvements in ArchiveCache operations ### API * Optimizations in cache building * Entity views * Use of parameters from JWT tokens (experimental) ### Misc * Various other small improvements and fixes ## April 18, 2025 Release 4.14 ### UI * New design with various improvements in Strategy editor, Data mapping, API endpoint tester and API status * Dark mode * Ability to add new blocks directly from block output connector * Open in new tabs (settings) ### Datasets * Update dataset actions when creating snapshot * JSON pipeline default options have changed * Experimental new feature: CIFF exports ### API * API update process can be cancelled ## February 5, 2025 Release 4.13 ### Datasets * Post-processing actions for normal datasets are production-ready. With an action you add new attributes or relations to the graph, or remove existing attributes, relation or items. * The UI for the dataset build report is improved. * In the data mapping interface the problems are shown in a new "console" panel at the bottom of the screen. The new panal also contains the output of the debug statements. ### API * Intermediate cache items are rebuilt only when strictly necessary. * API settings, e.g. authentication and cors settings, can be applied directly. You don't need to disable/enable the API anymore. * Improve API status notification and actions. * Avoid emitting errors on requests that could succeed upon retry. * All tabs in an API page are accessible when the API is down. ### Strategies * Blocks that are dependent on strategy parameter are visually marked. * Fix documentation icon not visible on some strategy blocks. ## October 22, 2024 Release 4.12 ### Datasets - New graph tab layout - Improved dataset build report - Actions (post-processing) are available on standard datasets, not just virtual datasets. (Experimental and not yet available on production/managed APIs). - More types of actions: Add relations and attributes, remove relations, attributes and nodes, and generate new nodes - Progress bar for file uploads. - New pipeline for Common Index File Format (CIFF). Developed within https://openwebsearch.eu/. ### API - Manage domains that have access to an API (CORS). Add/remove domains in the settings tab. - Improve error messages ### Building blocks - New blocks for string concatenation ### Core - Automatic clean up of unused data in the datastore ### Other - New dashboard screen, providing quick access to datasets, strategies and apis. ## May 13, 2024 Release 4.10 ### Datasets - Better caching of external contents - Improved support for multiline headers in XLSX files - overhauled layout of build report ### Strategies - improved documentation of the configuration settings of building blocks ### API - CORS-domains can now be set per API (Not yet in the UI). ### Generic - overhaul of front-end application (faster loading) - various bug / performance fixes ## March 4, 2024 Release 4.9 ### Datasets - Improved content in build report and UI. ### Strategies - Strategy editor: optional inputs on building blocks - New block: "match by (related) object" ### API - Allow API role to view API via WebInterface ## December 15, 2023 Release 4.8 ### Datasets - Overhaul of error handling in data mapping and loading. - Probabilities can now be inspected on a per-statement basis in the data mapping editor. - Mapping editor preferences (auto-debug, heatmap on/off, etc) are now saved between editing sessions. - Multiple URLs can be added to a pipeline at once. - OAI: Caches can be fully reloaded if caches are not correctly updated. - JSON: Allow specifying a root node to start processing (in line with XML pipeline). - Allow downloading of large files. - Reorder actions in virtual datasets. ### Strategies - Strategy parameters can now be inputted directly in the result inspector, instead of first having to be in a testset. - Strategy history has a new look. - Adding tags to strategies has become easier. - Enhanced strategy list viewing and sorting. - SpinQL compiler updated to 4.1 ### APIs - Endpoints can now have a description, which you can use for your own documentation. ## September 8, 2023 Release 4.7 ### Datasets - Revamped the way tasks (such as loading datasets) are scheduled and executed. - Improved dataset change detection (backend + frontend). - Improved performance of PDF-processing. Previously seen data is now processed more efficiently. - Added a pipeline that takes RDF as input and feeds the triples to a mapping. This allows for adding, modifying and/or filtering triples while ingesting. ### Strategies - The strategies now use spinql 4.0 sbs/spinql#208. ## May 5, 2023 Release 4.6 ### Desk - Communicate new release of Spinque Desk. Notifies a user when a new release of Spinque Desk is about to be activated and provides a link to the release notes when a new release is used for the first time. ### Datasets - Communicate dataset changes. On the dataset page it is indicated when a pipeline, a mapping or the uploaded data has changed. This functionality provides the user a cue when to rebuild the graph. In future releases this functionality will also be used to improve automatic rebuilding of the graph. - Errors in loading and mapping data. More details are provided when debugging mappings, making it easier to identify issues. - Data download. Manually uploaded data can now also be downloaded afterwards. - Error handling SPARQL pipeline. A 'timeout'-parameter has been added for the SPARQL Pipeline, to avoid waiting for a SPARQL endpoint indefinitely. ### Strategies - Performance improvements edit-distance. For short strings (such as names), the edit-distance comparison has been improved. It is now safe to do edit-distance comparisons of up to k=2 with sets to 1M records on both sides. With k=1 comparisons of up to 10M on each side can be safely executed. WARNING: results may still explode in size. example: a self-join on exactly the same string 10k times will end up with 100m results. This hasn’t changed. Long strings, or strings with fairly long common-prefixes (such as URLs) currently don’t benefit yet from the performance improvements. We aim to address these in later releases. ### Misc ... and of course, like always, lots of things behind the scenes to improve the service. ## March 16th, 2023 Release 4.5 ### Datasets - Revamp of Dataset page. Most notable in the interface is the new layout of the dataset page. The content is now split into multiple tabs. - Working with samples in the data mapping. Copy a document to the samples and be able to edit the document and test your mapping on it. - ArchiveCache (use caching for OAI-PMH endpoints). Under the hood, all OAI-PMH connections are now cached. Reloads of such datasets run much more swiftly. - Preparation for better support in data mapping. - Output statements (IDs, types, attributes, relations) can now be traced back to the line in the data mapping that’s responsible for it. - All data-fragments are now traceable (it is now known from what source they came from). This allows for more details in the mapping-editor, as well as in the dataset loading report. - Under the hood all errors in a pipeline are now logged into a data loading report. This allows for more fine-grained analysis of the quality of a pipeline/dataset. This work is not visible in the UI yet, but in future releases more elaborate problem-reports will be unleashed. ### Strategies - Improvements on collaborative strategy editing - The history view of a strategy better shows the previous snapshots and outstanding changes. - Help the user understand when a strategy has changes compared to the version used in an API. A notification is shown when a user opens a changed strategy in the editor.. - Layout of the toolbar in the result inspector (right-side panel) is improved