Change Log

All notable changes to CAPS will be documented in this file.

2022-06-10

Fixed

  • Improve bad chunk detection in corrup files. Although CAPS is pretty stable when it comes to corrupted files other tools might not. This improvement will trigger a file repair if a bad chunk has been detected.

2022-06-07

Fixed

  • Infinite loop if segments with resolution >= 1 were requested

2022-05-30

Added

  • Add “info server” request to query internal server state.

2022-05-18

Fixed

  • Fix posssible bug in combination with websocket requests. The issue exhibits as such as the connection does not respond anymore. Closing and reopening the connection would work.

2022-05-09

Added

  • Add gap/segment query

2022-04-26

Important

  • With this release we have split the server and the tools

    • riffdump

    • riffsniff

    • rifftest

    • capstool

    into separate packages. We did this because for some use cases it make sense to install only these tools. The new package is called caps-tools and activated for all CAPS customers.

2022-03-28

Changed

  • Update command-line help for capstool

2022-03-03

Added

  • Log plugin IP and port on accept

  • Log plugin IP and port on package store error

2021-12-20

Added

  • Explain record sorting in capstool documentation

2021-11-09

Fixed

  • Fixed helicorder request in combination with filtering. The issue caused wrong helicorder min/max samples to be returned.

2021-10-26

Fixed

  • Fixed data extraction for the first record if it does not intersect with the requested time window.

2021-10-19

Changed

  • Update print-access help page entry

  • Print help page in case of unrecognized command line options

Fixed

  • Do not print archive stats when the help page or version information is requested

2021-09-20

Fixed

  • Fixed crash if an FDSNWS request with an empty compiled channel list has been made

2021-09-17

Added

  • New config option AS.filebase.purge.referenceTime defining which reference time should be used while purge run. Available are:

    • EndTime: The purge run uses the end time per stream as reference time.

    • Now: The purge run uses the current time as reference time.

    By default the purge operation uses the stream end time as reference time. To switch to Now add the following entry to the caps configuration.

    AS.filebase.purge.referenceTime = Now

2021-05-03

Changed

  • Log login and logout attempts as well as blocked stream requests to request log

  • Allow whitespaces in passwords

2021-04-15

Fixed

  • Rework CAPS access rule evaluation

Changed

  • Comprehensive rework of CAPS authentication feature documentation

2021-03-11

Important

  • Reworked data file format. An high performance index has been added to the data files which require an conversion of the data files. See CAPS documentation about upgrading. The conversion is done transparently in the background but could affect performance while the conversion is in progress.

2020-10-12

Added

  • Provide documentation of the yet2caps plugin.

2020-09-04

Fixed

  • Fixed gaps in helicorder request

2020-07-01

Fixed

  • Don’t modify stream start time if the assoicated data file couldn’t deleted while purge run. This approach makes sure that stream start time and the data files are kept in sync.

2020-02-24

Added

  • Extended purge log. The extended purge log can be enabled with the configuration parameter AS.logPurge. This feature is not enabled by default.

Changed

  • Log maximum number of days to keep data per stream at start

2020-01-27

Fixed

  • Typo in command line output

2019-11-26

Added

  • Added new command line option `configtest that runs a configuration file syntax check. It parses the configuration files and either reports Syntax OK or detailed information about the particular syntax error.

  • Added Websocket interface which accepts HTTP connections (e.g. from a web browser) and provides the CAPS protocol via Websockets. An additional configuration will be necessary:

    AS.WS.port = 18006
# Provides the Websocket interface via secure sockets layer.
# The certificate and key used will be read from
# AS.SSL.certificate and AS.SSL.key.
AS.WS.SSL.port = 18007

Changed

  • Simplified the authorization configuration. Instead of using one login file for each CAPS interface we read the authentication information from a shadow file. The file contains one line per user where each line is of format "username:encrypted_pwd". To encrypt a password mkpasswd can be used. It is recommended to apply a strong algorithm such as sha-256 or sha-512. The command "user=sysop pw=mkpasswd -m sha-512 && echo $user:$pw" would generate a line for e.g. user "sysop". The shadow file can be configured with the config option AS.users.shadow.

    Example:

    # The username is equal to the password
test:$6$jHt4SqxUerU$pFTb6Q9wDsEKN5yHisPN4g2PPlZlYnVjqKFl5aIR14lryuODLUgVdt6aJ.2NqaphlEz3ZXS/HD3NL8f2vdlmm0
user1:$6$mZM8gpmKdF9D$wqJo1HgGInLr1Tmk6kDrCCt1dY06Xr/luyQrlH0sXbXzSIVd63wglJqzX4nxHRTt/I6y9BjM5X4JJ.Tb7XY.d0
user2:$6$zE77VXo7CRLev9ly$F8kg.MC8eLz.DHR2IWREGrSwPyLaxObyfUgwpeJdQfasD8L/pBTgJhyGYtMjUR6IONL6E6lQN.2QLqZ5O5atO/

    In addition to user authentication user access control properties are defined in a passwd file. It can be configured with the config option AS.users.passwd. Each line of the file contains a user name or a group id and a list of properties in format "username:prop1,prop2,prop3". Those properties are used to grant access to certain functionalities. Currently the following properties are supported by CAPS: read, write.: "read and write.". By default a anonymous user with read and write permissions exists. Groups use the prefix % so that they are clearly different from users.

    Example:

    user1: read,write
%test: read

    The group file maps users to different groups. Each line of the file maps a group id to a list of user names. It can be configured with the config option AS.users.group.

    Example:

    test: user2

    With the reserved keyword ALL a rule will be applied to all users.

    Example:

    STATIONS.DENY = all
STATIONS.AM.ALLOW = user1

  • We no longer watch the status of the inventory and the access file with Inotify because it could be dangerous in case of an incomplete saved configuration. A reload of the configuration can be triggered by sending a SIGUSR1 signal to the CAPS process. Example:

    kill -SIGUSR1 <pid>

    CAPS reloads the following files, if necessary:

    • shadow

    • passwd

    • access list

    • inventory

2019-10-15

Changed

  • Run archive clean up after start and every day at midnight(UTC).

2019-10-01

Changed

  • Increase shutdown timeout to 60s

2019-05-08

Fixed

  • Fixed potential deadlock in combination with inventory updates

2019-04-23

Fixed

  • Improved plugin data scheduling which could have caused increased delays of data if one plugin transmits big amounts of data through a low latency network connection, e.g. localhost

2019-04-08

Added

  • Added new config option AS.filebase.purge.initIdleTime that allows to postpone the initial purge process up to n seconds. Normally after a start the server tries to catch up all data which might be an IO intensive operation. In case of a huge archive the purge operation slow downs the read/write performace of the system too. To reduce the load at start it is a good idea to postpone this operation.

2019-03-29

Added

  • Added index file check during archive scan and rebuild them if corrupt. The lack of a check sometimes caused CAPS to freeze while starting up.

2018-12-11

Added

  • Added support for SC3 schema 0.11

2018-10-18

Fixed

  • Spin up threads correctly in case of erroneous configuration during life reconfiguration

2018-10-17

Fixed

  • Reinitalize server ports correctly after reloading the access list. This was not a functional bug, only a small memory leak.

2018-09-14

Fixed

  • High IO usage while data storage purge. In worst case the purge operation could slow down the complete system so that incoming packets could not be handled anymore.

2018-09-05

Added

  • Access rule changes do not require a restart of the server anymore

2018-08-29

Changed

  • Assigned human readable descriptions to threads. Process information tools like top or htop can display this information.

2018-08-08

Changed

  • Reduced server load for real-time client connections

2018-05-30

Fixed

  • Fixed unexpected closed SSL connections

2018-05-25

Fixed

  • Fixed high load if many clients request many streams in real-time

2018-05-18

Added

  • Add option to log anonymous IP addresses

2018-04-17

Fixed

  • Improved handling of incoming packets to prevent packet loss to subscribed sessions in case of heavy load.

2018-03-08

Fixed

  • Fixed access list evaluator. Rather than replacing general rules with concrete rules they are now merged hierarchically.

2018-02-13

Added

  • Restrict plugin stream codes to [A-Z][a-z][0-9][-_]

2018-01-31

Changed

  • CAPS archive log will be removed at startup and written at shutdown. With this approach we want to force a rescan of the complete archive in case of an unexpected server crash.

2018-01-30

Fixed

  • Fixed parameter name if HTTP SSL port, which should be AS.http.SSL.port but was AS.SSL.http.port

2018-01-29

Fixed

  • Fixed caps protocol real time handler bug which caused gaps on client-side when retrieving real time data

2018-01-26

Changed

  • Log requests per CAPS server instance

Fixed

  • Improved data scheduler to hopefully prevent clients from stalling the plugin input connections

2018-01-02

Fixed

  • Fixed bug in combination with SSL connections that caused CAPS to not accept any incoming connections after some time

2017-11-15

Added

  • Added option AS.inventory which lets CAPS read an SC3 inventory XML file to be used together with WWS requests to populate channel geo locations which will enable e.g. the map feature in Swarm.

2017-11-14

Fixed

  • Data store start time calculation in case of the first record start time is greater than the requested one

2017-11-08

Fixed

  • WWS Heli request now returns correct timestamps for data with gaps

2017-10-13

Fixed

  • FDSN request did not return the first record requested

2017-08-30

Fixed

  • Segmentation fault caused by invalid FDSN request

  • Timing bug in the CAPS WWS protocol implementation

2017-06-15

Added

  • Add `AS.minDelay which delays time window requests for the specified number of seconds. This parameter is only effective with FDSNWS and WWS.

2017-05-30

Feature

  • Add experimental Winston Wave Server(WWS) support. This feature is disabled by default.

2017-05-09

Feature

  • Add FDSNWS dataselect support for archives miniSEED records. This support is implicitely enabled if HTTP is activated

2017-05-03

Feature

  • Support for SSL and authentication in AS, client and HTTP transport

2017-03-24

Fixed

  • MSEED support

2017-03-09

Changed

  • Moved log output that the index was reset and that an incoming record has not ignored to debug channel

2016-06-14

Added

  • Added option AS.clientBufferSize to configure the buffer size for each client connection. The higher the buffer size the better the request performance

2016-06-09

Added

  • Added out-of-order requests for clients. The rsas plugin with version >= 0.6.0 supports requesting out-of-order packets with parameter ooo, e.g. caps://localhost?ooo

  • Improved record insertion speed with out-of-order records

2016-03-09

Fixed

  • Low packet upload rate