# Change Log All notable changes to CAPS will be documented in this file. ## 2022-06-10 ### Fixed - Improve bad chunk detection in corrup files. Although CAPS is pretty stable when it comes to corrupted files other tools might not. This improvement will trigger a file repair if a bad chunk has been detected. ## 2022-06-07 ### Fixed - Infinite loop if segments with resolution >= 1 were requested ## 2022-05-30 ### Added - Add "info server" request to query internal server state. ## 2022-05-18 ### Fixed - Fix posssible bug in combination with websocket requests. The issue exhibits as such as the connection does not respond anymore. Closing and reopening the connection would work. ## 2022-05-09 ### Added - Add gap/segment query ## 2022-04-26 ### Important - With this release we have split the server and the tools - riffdump - riffsniff - rifftest - capstool into separate packages. We did this because for some use cases it make sense to install only these tools. The new package is called `caps-tools` and activated for all CAPS customers. ## 2022-03-28 ### Changed - Update command-line help for capstool ## 2022-03-03 ### Added - Log plugin IP and port on accept - Log plugin IP and port on package store error ## 2021-12-20 ### Added - Explain record sorting in capstool documentation ## 2021-11-09 ### Fixed - Fixed helicorder request in combination with filtering. The issue caused wrong helicorder min/max samples to be returned. ## 2021-10-26 ### Fixed - Fixed data extraction for the first record if it does not intersect with the requested time window. ## 2021-10-19 ### Changed - Update print-access help page entry - Print help page in case of unrecognized command line options ### Fixed - Do not print archive stats when the help page or version information is requested ## 2021-09-20 ### Fixed - Fixed crash if an FDSNWS request with an empty compiled channel list has been made ## 2021-09-17 ### Added - New config option `AS.filebase.purge.referenceTime` defining which reference time should be used while purge run. Available are: - EndTime: The purge run uses the end time per stream as reference time. - Now: The purge run uses the current time as reference time. By default the purge operation uses the stream end time as reference time. To switch to **Now** add the following entry to the caps configuration. ```config AS.filebase.purge.referenceTime = Now ``` ## 2021-05-03 ### Changed - Log login and logout attempts as well as blocked stream requests to request log - Allow whitespaces in passwords ## 2021-04-15 ### Fixed - Rework CAPS access rule evaluation ### Changed - Comprehensive rework of CAPS authentication feature documentation ## 2021-03-11 ### Important - Reworked data file format. An high performance index has been added to the data files which require an conversion of the data files. See CAPS documentation about upgrading. The conversion is done transparently in the background but could affect performance while the conversion is in progress. ## 2020-10-12 ### Added - Provide documentation of the yet2caps plugin. ## 2020-09-04 ### Fixed - Fixed gaps in helicorder request ## 2020-07-01 ### Fixed - Don't modify stream start time if the assoicated data file couldn't deleted while purge run. This approach makes sure that stream start time and the data files are kept in sync. ## 2020-02-24 ### Added - Extended purge log. The extended purge log can be enabled with the configuration parameter `AS.logPurge`. This feature is not enabled by default. ### Changed - Log maximum number of days to keep data per stream at start ## 2020-01-27 ### Fixed - Typo in command line output ## 2019-11-26 ### Added - Added new command line option `configtest that runs a configuration file syntax check. It parses the configuration files and either reports Syntax OK or detailed information about the particular syntax error. - Added Websocket interface which accepts HTTP connections (e.g. from a web browser) and provides the CAPS protocol via Websockets. An additional configuration will be necessary: ```config AS.WS.port = 18006 # Provides the Websocket interface via secure sockets layer. # The certificate and key used will be read from # AS.SSL.certificate and AS.SSL.key. AS.WS.SSL.port = 18007 ``` ### Changed - Simplified the authorization configuration. Instead of using one login file for each CAPS interface we read the authentication information from a shadow file. The file contains one line per user where each line is of format "username:encrypted_pwd". To encrypt a password mkpasswd can be used. It is recommended to apply a strong algorithm such as sha-256 or sha-512. The command "user=sysop pw=`mkpasswd -m sha-512` && echo $user:$pw" would generate a line for e.g. user "sysop". The shadow file can be configured with the config option `AS.users.shadow`. Example: ```config # The username is equal to the password test:$6$jHt4SqxUerU$pFTb6Q9wDsEKN5yHisPN4g2PPlZlYnVjqKFl5aIR14lryuODLUgVdt6aJ.2NqaphlEz3ZXS/HD3NL8f2vdlmm0 user1:$6$mZM8gpmKdF9D$wqJo1HgGInLr1Tmk6kDrCCt1dY06Xr/luyQrlH0sXbXzSIVd63wglJqzX4nxHRTt/I6y9BjM5X4JJ.Tb7XY.d0 user2:$6$zE77VXo7CRLev9ly$F8kg.MC8eLz.DHR2IWREGrSwPyLaxObyfUgwpeJdQfasD8L/pBTgJhyGYtMjUR6IONL6E6lQN.2QLqZ5O5atO/ ``` In addition to user authentication user access control properties are defined in a passwd file. It can be configured with the config option `AS.users.passwd`. Each line of the file contains a user name or a group id and a list of properties in format "username:prop1,prop2,prop3". Those properties are used to grant access to certain functionalities. Currently the following properties are supported by CAPS: read, write.: "read and write.". By default a anonymous user with read and write permissions exists. Groups use the prefix **%** so that they are clearly different from users. Example: ```config user1: read,write %test: read ``` The group file maps users to different groups. Each line of the file maps a group id to a list of user names. It can be configured with the config option `AS.users.group`. Example: ```config test: user2 ``` With the reserved keyword **ALL** a rule will be applied to all users. Example: ```config STATIONS.DENY = all STATIONS.AM.ALLOW = user1 ``` - We no longer watch the status of the inventory and the access file with Inotify because it could be dangerous in case of an incomplete saved configuration. A reload of the configuration can be triggered by sending a SIGUSR1 signal to the CAPS process. Example: ```bash kill -SIGUSR1 ``` CAPS reloads the following files, if necessary: - shadow - passwd - access list - inventory ## 2019-10-15 ### Changed - Run archive clean up after start and every day at midnight(UTC). ## 2019-10-01 ### Changed - Increase shutdown timeout to 60s ## 2019-05-08 ### Fixed - Fixed potential deadlock in combination with inventory updates ## 2019-04-23 ### Fixed - Improved plugin data scheduling which could have caused increased delays of data if one plugin transmits big amounts of data through a low latency network connection, e.g. localhost ## 2019-04-08 ### Added - Added new config option `AS.filebase.purge.initIdleTime` that allows to postpone the initial purge process up to n seconds. Normally after a start the server tries to catch up all data which might be an IO intensive operation. In case of a huge archive the purge operation slow downs the read/write performace of the system too. To reduce the load at start it is a good idea to postpone this operation. ## 2019-03-29 ### Added - Added index file check during archive scan and rebuild them if corrupt. The lack of a check sometimes caused CAPS to freeze while starting up. ## 2018-12-11 ### Added - Added support for SC3 schema 0.11 ## 2018-10-18 ### Fixed - Spin up threads correctly in case of erroneous configuration during life reconfiguration ## 2018-10-17 ### Fixed - Reinitalize server ports correctly after reloading the access list. This was not a functional bug, only a small memory leak. ## 2018-09-14 ### Fixed - High IO usage while data storage purge. In worst case the purge operation could slow down the complete system so that incoming packets could not be handled anymore. ## 2018-09-05 ### Added - Access rule changes do not require a restart of the server anymore ## 2018-08-29 ### Changed - Assigned human readable descriptions to threads. Process information tools like top or htop can display this information. ## 2018-08-08 ### Changed - Reduced server load for real-time client connections ## 2018-05-30 ### Fixed - Fixed unexpected closed SSL connections ## 2018-05-25 ### Fixed - Fixed high load if many clients request many streams in real-time ## 2018-05-18 ### Added - Add option to log anonymous IP addresses ## 2018-04-17 ### Fixed - Improved handling of incoming packets to prevent packet loss to subscribed sessions in case of heavy load. ## 2018-03-08 ### Fixed - Fixed access list evaluator. Rather than replacing general rules with concrete rules they are now merged hierarchically. ## 2018-02-13 ### Added - Restrict plugin stream codes to [A-Z][a-z][0-9][-_] ## 2018-01-31 ### Changed - CAPS archive log will be removed at startup and written at shutdown. With this approach we want to force a rescan of the complete archive in case of an unexpected server crash. ## 2018-01-30 ### Fixed - Fixed parameter name if HTTP SSL port, which should be `AS.http.SSL.port` but was `AS.SSL.http.port` ## 2018-01-29 ### Fixed - Fixed caps protocol real time handler bug which caused gaps on client-side when retrieving real time data ## 2018-01-26 ### Changed - Log requests per CAPS server instance ### Fixed - Improved data scheduler to hopefully prevent clients from stalling the plugin input connections ## 2018-01-02 ### Fixed - Fixed bug in combination with SSL connections that caused CAPS to not accept any incoming connections after some time ## 2017-11-15 ### Added - Added option `AS.inventory` which lets CAPS read an SC3 inventory XML file to be used together with WWS requests to populate channel geo locations which will enable e.g. the map feature in Swarm. ## 2017-11-14 ### Fixed - Data store start time calculation in case of the first record start time is greater than the requested one ## 2017-11-08 ### Fixed - WWS Heli request now returns correct timestamps for data with gaps ## 2017-10-13 ### Fixed - FDSN request did not return the first record requested ## 2017-08-30 ### Fixed - Segmentation fault caused by invalid FDSN request - Timing bug in the CAPS WWS protocol implementation ## 2017-06-15 ### Added - Add `AS.minDelay which delays time window requests for the specified number of seconds. This parameter is only effective with FDSNWS and WWS. ## 2017-05-30 ### Feature - Add experimental Winston Wave Server(WWS) support. This feature is disabled by default. ## 2017-05-09 ### Feature - Add FDSNWS dataselect support for archives miniSEED records. This support is implicitely enabled if HTTP is activated ## 2017-05-03 ### Feature - Support for SSL and authentication in AS, client and HTTP transport ## 2017-03-24 ### Fixed - MSEED support ## 2017-03-09 ### Changed - Moved log output that the index was reset and that an incoming record has not ignored to debug channel ## 2016-06-14 ### Added - Added option `AS.clientBufferSize` to configure the buffer size for each client connection. The higher the buffer size the better the request performance ## 2016-06-09 ### Added - Added out-of-order requests for clients. The rsas plugin with version >= 0.6.0 supports requesting out-of-order packets with parameter `ooo`, e.g. `caps://localhost?ooo` - Improved record insertion speed with out-of-order records ## 2016-03-09 ### Fixed - Low packet upload rate