# Change Log All notable changes to GDS will be documented in this file. ## 2024.254 ### Important THIS IS A MAJOR UPGRADE. MANUAL SETUP IS REQUIRED. Please read the following ChangeLog and the instructions for [Database Migrations after 2017.269]( https://docs.gempa.de/gds/current/base/installation.html#database-migrations-after-2017-269) carefully. Experienced users may run ```sh cd $SEISCOMP_ROOT/share/gds/web python3 manage.py showmigrations python3 manage.py migrate ``` ### Fixed - PostgreSQL DB Date format parsing for SeisComP >= 6.4 - The Django PostgreSQL DB backend maps DateTimeFields to `timestamp with time zone` by default. Time zone conversion never was supported by the GDS backend using the SeisComP date time classes. Prior to SeisComP 6.4 the timezone information was silently dropped leading to inaccurate date time comparisons if the systems time was not set to UTC. Since SeisComP 6.4 an error is thrown which surfaces if the GDS compares the origin time of a previous dissemination to the current one in an attempt to detect a significant change. - This migration removes the timezone information from timestamp columns of the log tables. Time stamps are converted to UTC if a locale time zone was used. ## 2024.240 ### Fixed - Fix attachment path in TOAST filter ## 2024.236 ### Updated - Send Journal to create TsunamiLogs when resolving template failed in filter_toast.py - Adjust documentation regarding NGINX integration on newer OS ## 2024.214 ### Updated - Adapt database query in TOAST multi-user client library to changed tsunami data model ## 2024.176 ### Fixed - DB Date format parsing for SeisComP >= 6.4 ## 2024.165 ### Changed - Use `/usr/share/nginx/html/gds` as default directory for static files - Update documentation ## 2024.098 ### Fixed - Duplicated dissemination of OnlyOnce or OnSignificantChange subscriptions in case of HA failover. - DB queries for SQLite backends ### Changed - Add callback function in filter_toast.py allowing to skip templates in derived classes ## 2024.068 ### Changed - Set default reverse proxy location to /gds - Set default MySQL/MariaDB CHARACTER SET to `utf8mb4` - Simplify installation instructions by removing Apache and uWSGI options ## 2024.018 ### Important THIS IS A MAJOR UPGRADE. MANUAL SETUP IS REQUIRED. Please read the following ChangeLog and the instructions for [Database Migrations after 2017.269]( https://docs.gempa.de/gds/current/base/installation.html#database-migrations-after-2017-269) carefully. ### Changed - Speed up DB cleanup - add composite index to gds_log_service using service and timestamp column - add composite index to gds_log_service using service, content and timestamp column - change bulletin clean up query to filter for rows with content ## 2024.016 ### Fixed - Prevent modification of extra tab configuration through user defined partial templates. ### Changed - Add configuration variable `addTargetPath` to send_archive spooler.py. ## 2023.348 ### Important THIS IS A MAJOR UPGRADE. MANUAL SETUP IS REQUIRED. Please read the following ChangeLog and the instructions for [Database Migrations after 2017.269]( https://docs.gempa.de/gds/current/base/installation.html#database-migrations-after-2017-269) carefully. ### Fixed - Adjust size of target log field to 180 characters - Increase max commline length to 500 ### Changed - Remove GDS_DEBUG setting. Same functionallity is available through default Python logger configuration. - Replace `sys.stderr` statements with call to logging facility. - Implement pylint recommendations. ## 2023.320 ### Added - Allow configuration of GDS command interface request limits - Support test mode in filter_toast.py ### Changed - Remove SeisComP 3 support from tools library and server backend ## 2023.289 ### Added - Evaluation of file name, content type and encoding in filter_toast.py ## 2023.226 ### Fixed - Parsing of preferred nodal plain in TOAST multi-user client library ## 2023.226 ### Added - Query for incident variables in TOAST multi-user client library ## 2023.215 ### Added - TOAST multi-user client library to - Request incident id for earthquake parameters - Load configured templates from TOAST database - Resolve clearsilver templates - Synchronize bulletin revision and bulletin context variables - Display bulletin attachment CID in GDS front end ## 2023.122 ### Changed - Call autosigma in filter_email.py overriding output.exportScript with ''. The change prevents that the output gets lost when autosigma is configured with output.exportScript and files are removed by the script. ## 2023.114 ### Changed - Do not warn about disabled services if the service is used in automatic queues only and the current user has right to access automatic queues ## 2023.011 ### Changed - Change favicon, add SVG and PNG variants ## 2022.339 ### Important THIS IS A MAJOR UPGRADE. MANUAL SETUP IS REQUIRED. Please read the following ChangeLog and the instructions for [Database Migrations after 2017.269]( https://docs.gempa.de/gds/current/base/installation.html#database-migrations-after-2017-269) carefully. ### Changed - Add indexes to service log, subscription and subscriber tables to speedup DB queries. ## 2022.329 ### Changed - Speedup DB query for event logs of specific day. ## 2022.301 ### Fixed - Prevent simultaneous execution of deferred and manual disseminations resulting in MySQL error 2014 (Command out of sync). - Deployment note in Gunicorn example configuration. ### Added - Nearest city information to XML filter example. ## 2022.285 ### Important MANUAL SETUP IS REQUIRED. Please copy configuration parameters `GDS_SSL` and `GDS_CAFILE` from `$SEISCOMP_ROOT/share/gds/web/gds/example-settings.py` to your `settings.py` in the same folder. ### Added - Support for backend SSL sockets. ### Fixed - Character encoding in preview of MIME type text ## 2022.252 ### Important THIS IS A MAJOR UPGRADE. MANUAL SETUP IS REQUIRED. Please read the following ChangeLog and the instructions for [Database Migrations after 2017.269]( https://docs.gempa.de/gds/current/base/installation.html#database-migrations-after-2017-269) carefully. ### Changed - Decrease target field of *Subscription* from 400 to 180 characters. Version 2022.231 increased the field length from 100 to 400 characters. With 400 x 4 bytes = 1600 bytes, the upper index limit of 767 bytes for older MySQL/MariaDB versions was exceeded and the migration step failed. In case you already specified targets longer than 180 characters the new migration will fail, e.g., ``` MySQLdb.DataError: (1265, "Data truncated for column 'service_target' ... ``` and you have to login to your database and manually reduce the target string, e.g., ``` UPDATE gds_subscription SET service_target = LEFT(service_target, 180); ``` ## 2022.244 ### Fixed - Database migration 0012 broken in version 2022.231 ## 2022.231 ### Important THIS IS A MAJOR UPGRADE. MANUAL SETUP IS REQUIRED. Please read the following ChangeLog and the instructions for [Database Migrations after 2017.269]( https://docs.gempa.de/gds/current/base/installation.html#database-migrations-after-2017-269) carefully. ### Changed - Distinguish magnitude and depth availability from delta values in significant change properties of *Subscriber* - From now on delta values are only evaluated if they are available in the current and previous dissemination - The new *depth/magnitude added/removed* flags may be used to define a significant change based on the availability of these fields - The availability fields are disabled by default. However, they are enabled by the database migration routine for all subscribers which define a corresponding delta value in order to preserve the old rule set. - Increase target field of *Subscription* from 100 to 400 characters - Require fix django-simple-history version `3.0.*` since later versions dropped Python 3.6 support and introduced datamodel changes. ### Fixed - Typos in log messages ## 2022.221 ### Changed - Prioritize dictionary entries from service specific tab over entries of 'extra' tabs ## 2022.126 ### Added - sink.py in tools/lib for SeisComP object serialization compatible with all SeisComP and Python versions ### Changed - GDSConfigParser renamed to ConfigParser and moved to tools/lib/compat.py - tools/lib/scml.py code move to tools/lib/seismo.py ## 2022.112 ### Fixed - Typos in NGINX/Gunicorn installation instructions - Handle NaN values in tools/lib/cities.py ### Added - Allow deactivation of initialization checks in SMS spooler - Systemd service example for Gunicorn Version < 20 ## 2022.055 ### Fixed - Spooler timeout (`-t`) parameter broken since 2020.282 ### Changed - Add FAX spooler using HylaFAX service on local machine - Add NetGSM spooler using HWg-SMS-GW3 SMS gateway on network - Load text attachments as text content for proper display in web interface - Documentation: - Correct typos - Highlight configuration of production settings ## 2022.021 ### Changed - Improved documentation ### Added - FTP spooler - `toBytes` method in attachment class of `bulletin.py` ## 2021.355 ### Fixed - Use full available height in bulletin preview and log perspective - Extra tab type filter configured in settings.py was not evaluated correctly ### Added - Archive spooler - `saveToFile` method in bulletin attachment class - NGINX and Gunicorn installation instructions ### Changed - Refine NGINX and uWSGI installation instructions ## 2021.335 ### Fixed - Attachment URL loading in `bulletin.py` for requests with POST data under Python 3 ### Added - Documentation on NGINX web server integration ## 2021.322 ### Fixed - Database migration 0012 introduced by 2020.282 cleared the content column of the `gds_log_service` table - Attachment parsing in GDS bulletin library for data produced before 2019.207 ## 2021.321 ### Fixed - Show list of recipients in GDS service log perspective if messages content is not requested - Define sort order for paginator in service and content log perspective - Remove reference to map file from gds.css ### Changed - When filtering waveforms using SeisComP Butterworth filters, the upper corner frequency must be below the Nyqusit frequency of the signal. Since BH? streams are often sampled at 20 Hz, the default filter in attachTracesImageServer() of `share/gds/tools/filter_email.py` we set to BW(3,1.0,8). Scripts derived from this template may require adjustment, too. - Improve `send_sms.py` - Send CUR command as early as possible to disable periodic messages send automatically by some devices - Support more modems ## 2021.305 ### Fixed - Blue background representing selected state in tabs of log perspective - Page reporting insufficient permissions ### Added - Show icon and name of current perspective in page header ## 2021.301 ### Fixed - django import in `share/gds/web/gds/wsgi.py - `loadFromCommand` in `share/gds/web/gds/lib/bullein.py` when used with stdin data under Python3 - `tools/lib/send_sms.py` for Python3 ## 2021.294 ### Important THIS IS A MAJOR UPGRADE. MANUAL SETUP IS REQUIRED. Please read the following ChangeLog and the [Update Instructions (2021.294)](https://docs.gempa.de/gds/current/base/installation.html#migration-to-2021-294-django-3-2-simple-history) carefully. ### Added - Support for GeoJSON polygon *Region* definition - History support for the data model allowing to review all changes to a particular configuration object and to revert back to a previous revision. ### Changed - Django update from `3.1` to `3.2.*` (LTS version) - Improve front-end support for binary attachments, e.g. PDF files - Display decoded attachment size - Log levels - Decrease log level for missing magnitude from warning to info - Increase log level for configuration summary from debug to info - Reduce width of criterion inlines - Remove columns: Min/max magnitude count, RMS - Reduce with of columns - Limit hight of multiple select fields ### Fixed - Link to specific data model object in the admin log entry list. ## 2021.253 ### Changed - Update jquery (3.6.0) and fancybox (3.5.7) java script libraries - Remove tinymce HTML editor and display editable text area with raw HTML instead. Motivation - The editor was rarely used - The output of the editor diverted significantly from the HTML produced by the content filter, e.g., the HTML header was stripped. - Improve HTML bulletin preview and log view by rendering the HTML without the GDS style sheets. - Default to MIME type `application/xml` if no filter is configured ## 2021.189 ### Changed - Increase default command interface client connection limit from 2 to 200 - `connections` configuration parameter ## 2021.175 ### Added - QuakeLink `keepAliveInterval` configuration option ## 2021.166 ### Fixed - `tools/lib` - Read string configuration parameters under Python 2 as byte string - Stop using of deprecated `SafeConfigParser` ### Added - Python 3 support for send_sms spooler ## 2021.148 ### Fixed - `tools/lib/bulletin.py` - Use `kill()` instead of `terminate()` to ensure subprocess stops if timeout is exceeded in `Attachment.loadFromCommand()` - Use `psutil` to recursively send kill signal to all children of subprocess ### Added - `tools/lib/aoi.py` for simple area of interest definitions in filter scripts - Turn `Save and add another` button of *Queues* and *Subscriptions* into a `Save as new` button allowing to clone the object including all references. In case of the *Queue* all *Criteria* and all *Subscriptions* will be duplicated. ## 2021.130 ### Fixed - Last dissemination time in disseminate perspective ## 2021.025 ### Fixed - MIME type parsing in DISSEMINATE command ## 2021.015 ### Fixed - iSMS spooler: Use GET instead of POST, remove unicode characters ### Added - Improve documentation on installation an update instructions - Automatically read GDS bulletin in spooler base class if requested by specific spooler implementation ## 2020.344 ### Fixed - Test for a maximum of 4 images per tweet in Twitter spooler - Attachment loading in html2bulletin.py ## 2020.339 ### Added - Image and video upload feature to Twitter filter and spooler ## 2020.300 ### Added - URL encode attachment's file name during Content-ID generation (RFC2392) ## 2020.296 ### Fixed - Errors in the GDS Python library introduced in 2020.282 (Python 3 support) - Correct log level check - SMS spooler errors related to Python 3 unicode and integer division handling ## 2020.295 ### Fixed - During manual dissemination uploaded files where not included while they appeared in bulletin preview ## 2020.282 ### Important THIS IS A MAJOR UPGRADE. MANUAL SETUP IS REQUIRED. Please read the following ChangeLog and the [Update Instructions (2020.282)](https://docs.gempa.de/gds/current/base/installation.html#migration-to-2020-282-django-3-1) carefully. - The front-end was upgraded to Django version 3.1 and requires a minimum Python version of 3.6. Please make sure Python 3.6 is available before proceeding with the update. - The back-end was updated to support Python 3 but continues to support Python 2 starting from version 2.7 in order to support SeisComP3 Jakarta 2018.327 and older. - Descriptions in the data model changed. Please run `python3 manage.py migrate` to complete the installation. - Python library - seiscomp-python - The GDS Python library (`$SEISCOMP_ROOT/share/gds/tools`) uses the seiscomp-python shebang to determine the Python version the SeisComP Python bindings are compiled for. For SeisComP versions <= 2018.327 an alias pointing to either python2 or python3 needs to be created. When the GDS is installed via GSM this will done automatically. - renamed `lib/xml.py` to `lib/scml.py` to avoid naming conflicts - Logging facility - new `messageLimit` configuration parameter - try to read log directory from SeisComP Environment, use `~/.seiscomp/log` as fallback - Mandatory MIME types for content preview in GDS front-end - In contrast to previous GDS versions the MIME type of a filter is no longer guessed but must be specified in the GDS back-end configuration. Common MIME types are - `text/gds` - GDS bulletin format used by most of the sample filter scripts shipped with GDS - `text/x-python` - Python dictionary displayed as key-value pairs - `image/[subtype]` - Images of any type - `text/[subtype]` - Generic text ## 2020.098 ### Added - Support for QuakeLink API=2 and SUMMARY/2 format providing author information and complete evaluation status and evaluation mode support - SHOW_AUTHOR_INFORMATION configuration variable adding origin author information during manual event dissemination ## 2020.042 ### Added - Optional parameter `--automatic` to the `DISSEMINATE` command of the back-end API allowing to trigger only automatic *Queues*. ## 2020.021 ### Important The data model changed. Please run `python manage.py migrate` to complete the installation. ### Added - Magnitude type to the `significant change` properties of the *Subscriber* ## 2020.009 ### Fixed - Use timeout parameter for HTTP POST requests of bulletin library ## 2020.006 ### Fixed - Add missing migration script ## 2019.354 ### Important - The data model changed. Please run `python manage.py migrate` to complete the installation. - If you use an account without super user status for manual dissemination you might want to grant the permission `gds | queue | Can view and trigger automatic queues` to be able to trigger automatic *Queues*. ### Added - User permission controlling access to automatic *Queues* ## 2019.340 ### Added - Support binary bulletin types, e.g., images or PDF documents - Support gempa bulletins without content, e.g., attachments only - Support review of non image gempa bulletin attachments, e.g. PDF documents ## 2019.315 ### Fixed - Missing char in SMS spooler for multipart GSM (7bit) encoded messages ## 2019.305 ### Added - Keep current selected tab in GDS front-end on `Load bulletin` action - Switch to toggle all *Services* at once ## 2019.291 ### Fixed - Exception handling in GDS front-end ## 2019.287 ### Fixed - Minimum *Subscriber* delay value was not evaluated for *Subscription* pre-selection in manual dissemination via GDS web interface. This bug was introduced in version 2017.257. ## 2019.267 ### Fixed - Default HA timeout was set to 10,000 instead of 10 seconds ## 2019.267 ### Added - Support text attachments in send_email.py spooler ## 2019.254 ### Added - Maintain scroll position on *Subscription* list toggle ## 2019.207 ### Changed - Web interface redesign - *Queue* and *Subscription* selection - Review *Queue* evaluation result (hover mouse over *Queue* name) - Show/hide automatic *Queues* - Overall toggle for all *Subscriptions* and *Queues* - Dissemination - Ask for confirmation (optionally) - Exclusion of entire *Services* prior to dissemination - Option to skip event selection. Dissemination may be manually triggered without event information using user input or other data sources. - Bulletin preview and modification - *Service* tabs instead of side menu - Synchronize only those fields which actually changed - Allow deselection of attachments and header fields - Fix attachment cid rendering, e.g., for images embedded in HTML email - Show stderr output of filter application as info message - Customizable bulletin forms trough partials (share/gds/web/gds/templates/gds/partials), e.g., define select boxes for input - Navigation - Step menu in dissemination and log perspective - Store selected date in session - Store event or recent items selection in session - Direct access to *Queue* configuration - Open Django admin perspective in new tab - Web configuration (settings.py) - Add RECENT_DEFAULT option to prefer recent items perspective over calendar - Add CONFIRM_DISSEMINATE option to ask for confirmation on dissemination - Add LOAD_BULLETINS option to automatically active bulletin review - Add DISS_EXTRA_TABS option for extra user input used by multiple bulletins, e.g, define a 'Main' tab with general parameters used in all bulletins - Add SHOW_AUTOMATIC_QUEUES option to show/hide automatic *Queues* in dissemination perspective - Correct MESSAGE_LEVEL parameter controlling verbosity of messages shown in web interface - GDS back-end - Add API version check to ensure GDS web interface and back-end match - Change GDS command protocol to report *Queue* evaluation result - support evaluation of match state in filter scripts through '@MATCH@' command line parameter, e.g., service.email.filter.primary.cmd = "@DATADIR@/gds/tools/filter_email.py --match @MATCH@" - Option to stop automatic dissemination after first manual dissemination - Log all event parameters if dissemination is triggered with pre-formatted content - Cache primary filter results for identical filter commands - Filter and Spooler scripts (the `share/gds/tools` folder) - Filter library - Add --match parameter allowing match-dependent content generation - Add --input parameter to read input from file instead of stdin - Bulletin library - Use requests library improving support for multipart GIS requests - Fix UTF-8 encoding - Add support for text attachments - New cities library - Calculation of direction and distance of N nearest cities optionally filtered by population - Calculation of direction and distance to specific city - Read data from SeisComP cities.xml or CSV file - Filter cities.xml into CSV based on region and population for faster lookup - New database library - query event specific dissemination serial number - Email filter - add example to attach shake map generated by SIGMA - add GIS multipart example rendering overlay image and layer - Email spooler - Allow specification of To email address in case of BCC mails - Use `Undisclosed Recipients ` as default where EMAIL is the configured From email address - Data model - Add *Service* long name and sort order - Change *Queue* `manual` field to `automatic` adding option to stop automatic dissemination once a manual dissemination was performed - Improved documentation - Update of external components - django 1.11.20 - django_extensions 2.1.6 - fancybox 3.5.1 - tinymce 4.9.2 ## 2018.318 ### Fixed - Segfault when ran in spooler only mode ## 2018.269 ### Fixed - Logging of partial success ## 2018.222 ### Added - UNSET option to evaluation mode, evaluation status and event type parameter allowing to explicitly check for undefined values ## 2018.176 ### Added - Log event revision ## 2018.134 ### Added - Sort *Queues* by name in dissemination perspective (3/3) of web interface ## 2018.122 ### Added - Multiple choice event type parameter to *Criterion* - Event region name regular expression filter to *Region* ### Fixed - In automatic mode do not include once matched *Subscriptions* of manual *Queues* ## 2018.107 ### Added - Significant change flags to *Subscriber* allowing to resend if evaluation mode/status, region, event type was changed or focal mechanism was added/removed ## 2018.106 ### Fixed - Resend mode 'Once Matched & Significant Change': Dissemination was triggered regardless of significant change status if current refinement did not match but previous dissemination existed ## 2017.319 ### Added - Permission granting read only access to dissemination perspective ## 2017.286 ### Fixed - Content download in web interface ## 2017.269 ### Important This purpose of this GDS update is to upgrade the Django framework to 1.11.7. Please follow upgrade instructions under ### Fixed - Unicode error when changing description field of *Subscription* ## 2017.257 ### Added - Use a timer for *Subscriptions* not matching the `minDelay` criterion. This ensures that the dissemination is triggered even if no further event update is received from the QuakeLink server. - Reverse event revision order in publish step 2/3 to make sure that the latest revision is displayed on the first page ## 2017.081 ### Fixed - Resend option ## 2017.079 ### Important This is a major update introducing many new features. **Note:** The database model changed significantly. Migration scripts are provided under `share/gds/web/gds/migrations` and need to be executed in sequential order. Also the bitfield Django plugin needs to be installed. Please check the documentation for an installation description. ### Added - Support for circular and polygon *Region*. *Region* definitions becomes optional for a *Criterion*. - *External Criteria* may now be defined per *Criterion* instead of per *Queue* - Allow definition of significant change parameters per *Subscriber*. New recent modes: `On significant change`, `Onces matched and significant change` - Allow *Criteria*, *Subscribers* and *Subscriptions* to be disabled - Add `Evaluation Status` *Criterion* parameter - Log `stderr` output of *External Criterion* scripts - Improved log output for *Queue* match/mismatch - Support list of recent events in log and dissemination pages in addition to calendar perspective - Show precise values when moving mouse over table cell - Update HTML editor and attachment viewer ## 2017.047 ### Fixed - Log correct MIME type if no second filter stage is configured. This bug was introduced in version 2015.320. ## 2016.333 ### Fixed - Repeated dissemination if gds.id is not specified ## 2016.297 ### Added - Add high availability (HA) mode allowing multiple GDS instances sharing one database to work as one. This feature requires a special GDS HA license. ## 2016.287 ### Added - Support spooling of external *Services* not managed by GDS, add `spoolOnly` mode - Set `logID` to `None` in spooler lib if extraction from file name failed ## 2016.173 ### Fixed - Use socket timeout only if keep alive messages have been requested ## 2018.176 ### Added - Log event revision ### Fixed - Shutdown ### Added - Support cross-referenced magnitudes (preferred magnitude child of non-preferred origin) ## 2016.160 ### Added - Support PostgreSQL database server ## 2016.109 ### Added - If enabled, reconnect if no alive message is received within 60s ## 2016.051 ### Fixed - Query for *Subscriber* delay ## 2015.320 ### Fixed - Apply 2nd filter to bulletins in SEND command - Format preview if MIME type is set to application/gds ## 2015.239 ### Added - Allow NATIVE (non filtered) QuakeLink query ## 2015.266 ### Fixed - QuakeLink reconnect problem on some operating systems ## 2015.216 ### Added - Python tools: keep track about execution time in spooler and filter library - Added timeout command line parameter (-t, --timeout) - Use timeout in filter_email.py to abort image generation