Execution and Automatic Startup

CAPS uses the SeisComP infrastructure for startup, configuration and logging. Please refer to the SeisComP documentation for a comprehensive description of SeisComP.

Figure 9 shows a screen shot of scconfig, which is the central SeisComP GUI allowing to configure, start and monitor the CAPS server.

scconfig: SeisComP utility allowing to configure, start and monitor CAPS.

On the command line the following sequence may be used to enable, start and monitor the CAPS:

seiscomp enable caps
seoscomp start caps
seiscomp check caps

Dependent on the configured log level CAPS will log to ~/.seiscomp/log/caps. For debugging purposes it is a good practice to stop the CAPS background process and run it in the foreground using the --debug switch:

seiscomp stop caps
seiscomp exec caps --debug

File System Tuning

Depending on the number of streams a CAPS server handles a number of settings can improve the I/O throughput and overall performance. Since channel data are organized in an archive structure where each stream is written into a dedicated file, CAPS needs to open and close a lot of files if thousands of streams are fed into it. In the default configuration CAPS caches up to 250 open files for later reuse. An open file here is not only the data file for the CAPS stream but might also include the index file if records have been received out-of-order. So in the default configuration CAPS need to open 500 file at the same time.

Operating systems control the maximum number of open file descriptors a process might hold. Often a default value is 1024. If the maximum open files in CAPS should be increased to 2000 (assuming CAPS manages 2000 streams) then the limit for the user who runs CAPS should be increased to at least 4000. In many Linux distributions ulimit can be used for that.

Furthermore CAPS requires file descriptors for incoming connections. Each active connection holds a socket descriptor for network communication and a file descriptor (or two if index files are present) for reading data.

Depending on the number of concurrent connections one is expecting, it would be safe to add this number times three to the user limit in the operating system.

Example for 2000 streams:

# The maximum number of open files managed by CAPS.
# 2000 + margin
AS.filebase.cache.openFileLimit = 2100

# Set ulimit to 7500 files: 2100 * 2 + 1000 * 3 (network)
$ ulimit -n 7200

Security and Access Control

Access control

CAPS provides access control on the service and stream level. On the service level access can be granted by client IP, on the stream level by client IP or user/group name obtained during authentication. In addition read and write permission may be granted for individual users and groups. The configuration is described in the following sections.

Service level access

Service level access is defined in the main caps configuration file, e.g. @SYSTEMCONFIGDIR@/caps.cfg

The following services are availble:

• Plugin - Incoming data send by CAPS plugins, configuration prefix: AS.plugin

• Client - Default CAPS client protocol, e.g. used by the CAPS recordstream or by the capstool, configuration prefix: AS

• HTTP - Administrative web interface and FDSNWS dataselect service, configuration prefix: AS.http

• WWS - Built-in Winston waveform server, configuration prefix: AS.WWS

For each sevice access can be granted on IP level through allow and deny rule sets. By default no restrictions are in place. If an allow rule is present access is only granted to matching IPs. Deny rules may be used to override a subset of the IP range defined in the allow set.

The formal definition of a rule is:

IP_MASK[, IP_MASK[, ...]]

where IP_MASK may be a single address or a subnet described by a network mask.

Using the HTTP service as an example the configuration options are AS.http.allow and AS.http.deny.

Example:

AS.http.allow = 192.168.1.0/24
AS.http.deny = 192.168.1.42

These rules provide access to the HTTP service for all clients of the 192.168.1.0/24 subnet except for the IP 192.168.1.42.

Stream level access

Stream level access is controlled by an access file defined by AS.auth.basic.access-list.

Each line of the file consists of a ALLOW or DENY rule. The formal definition of one rule is:

STREAMID.ALLOW|DENY= IP_MASK|USER|%GROUP[, IP_MASK|USER|%GROUP[, ...]]

where

• STREAMID is defined as: [NET[.STA[.LOC[.CHA]]]]. Regular expressions are not supported.

• USER is a user account defined in the shadow file or the special id all.

• GROUP is a user group definition from the group file. A % must be placed before the group name to distinguish it from a user.

Note

For access control, two cases must be distinguished:

1. Client access without username and password

   All client sessions have guest permissions when no login credentials are provided. By default data can be read and written. The guest account can be restricted by IP rules only. Please have in mind that for instance the rule DENY=all does not have any effect here.

2. Client access with username and password

   In this case user rules will be evaluated only and IP restrictions have no effect. In addition user rules does not apply to the guest user. This leads to that DENY=all prohibits access for all users except the guest user. If the access should be denied for all users the following rule must be used: DENY=all, 0.0.0.0/0.

This leads to that the rule DENY = all prohibits data access for all users but anonymous logins can still access data. If guest access should also be prohibited the rule must be extended by an IP address.

By default access is unrestricted. If a stream ID is not matched by any access rule then access will be granted. This behavior is different from the service level access where an allow rule will implicitly revoke access to any non matching IP.

To restrict access by default you may add a global DENY rule which references no stream id and which matches all IP addresses and all users using the special user id all:

DENY = 0.0.0.0/0, all

The rules in the access file are evaluated independent of the order in which they are defined. A rule with more stream id components overrules a more generic line. E.g., considering a request from the local machine the following rule set would

• grant access to all networks except for AM

• grant access to station AM.R0000 except for the stream 00.ENN stream

AM.DENY = 127.0.0.1
AM.R0000.ALLOW = 127.0.0.1
AM.R0000.00.ENN.DENY = 127.0.0.1

The client IP is only evaluated in the absence of user authentication. E.g., the following rule would block access to any anonymous user but still grant access to any authenticated user:

DENY = 0.0.0.0/0

Please refer to Service-specific users for a definition of service specific users.

The following example shows how anonymous access by IP and access by user name may be combined:

AM.DENY = 0.0.0.0/0, all
AM.ALLOW = 127.0.0.1, %group1, user1
AM.R0000.ALLOW = user2
AM.R0000.DENY = user1

The example above

• grants access to anybody except for the AM network

• grants access to the AM network for

  • anonymous users on the same machine

  • users belonging to the group1 group

  • the user user1

• in addition grants access to the station AM.R0000 to the user user2 while local anonymous users and authenticated users of the group1 would still have access

• explicitly denies access to station AM.R0000 for user1

The stream level access can be tested and debugged on the command line by specifying a stream and (optionally) an IP to test for:

$ caps -v --print-access AM.R0000.00.ENN 1.2.3.4

Authentication by user name and password (shadow file)

Authentication can be used, e.g. together with the capss RecordStream or capstool. It is performed against a shadow file defined by AS.auth.basic.users.shadow. It contains the user name and password information for the user accounts. Each line consist of a user name and password hash separated by a colon (:). The formal definition of one line is:

USER:PWD_HASH.

To encrypt a password mkpasswd can be used. It is recommended to apply a strong algorithm such as sha-256 or sha-512. The command

$ user=sysop pw=`mkpasswd -m sha-512` && echo $user:$pw

generates a password hash for user sysop. An empty password is represented by an asterisk (*).

Example:

# The user name is equal to the password
user1:$6$mZM8gpmKdF9D$wqJo1HgGInLr1Tmk6kDrCCt1dY06Xr/luyQrlH0sXbXzSIVd63wglJqzX4nxHRTt/I6y9BjM5X4JJ.Tb7XY.d0
user2:$6$zE77VXo7CRLev9ly$F8kg.MC8eLz.DHR2IWREGrSwPyLaxObyfUgwpeJdQfasD8L/pBTgJhyGYtMjUR6IONL6E6lQN.2QLqZ5O5atO/
FDSNWS:*

Guest user

The CAPS server ships with a pre-configured anonymous user identified by guest. It may be used during login at the web interface in which case access is authorized against the client IP.

The guest user may be assigned to a user group and its access properties may be defined.

Anonymous access may be disabled through IP-based DENY rules in the access control list file.

Service-specific users

For some services it might be desirable to disable the authentication entirely. This can be archived by adding one of the special service specific users to the shadow file followed by an asterisk indicating an empty password. Optionally stream specific access can be granted or revoked to this user as well. The flowing users are available for the individual services:

• HTTP - Access to the web interface

• FDSNWS - Access to Built-in FDSNWS dataselect service served through the HTTP protocol (/fdsnws/dataselect/1/query)

• WWS - Access to the Built-in Winston waveform server Protocol

Groups

A group file, defined by AS.auth.basic.users.group, allows to assign users to groups. Each line of the file consists of a group name followed by a user list. The formal definition of one rule is:

GROUP: USER[, USER[, ...]]

where

• GROUP is the name of the new group definition

• USER is a user account defined in the shadow file or the special id guest

Example:

group1: user1, user2

A group may by referenced by the access control or Passwd: user access properties file. In both cases a % prefix is required to distinguish it from a user name.

Passwd: user access properties

In addition to authentication by user name and password, user access control properties can be set in a passwd file defined by AS.auth.basic.users.passwd. The formal definition of a line is

USER|%GROUP:PROP[, PROP[, ...]]

where

• USER is a user account defined in the shadow file or one of the special ids all or guest.

• GROUP is a user group definition from the group file. A % must be placed before the group name to distinguish it from a user.

• PROP is a property granted to the user or group. The following properties are currently supported:

  • read - Grants permission to request data from the server

  • write - Grants permission to store data into the server

  • admin - Grants permission to request server statistics and the view server statistics on the server website.

By default read and write permissions are granted to the guest user and all authenticated users not listed in this file.

The following example changes this and revokes read and write permissions per default. Read access is provided to anonymous and users belonging to the group1 while write access is only granted to user1.

all:
guest: read
%group1: read
user1: read,write

Secure sockets layer (SSL)

The Secure Sockets Layer (SSL) is a standard for establishing a secured communication between applications using insecure networks. Neither client requests nor server responses are readable by communication hubs in between. SSL is based on a public-key infrastructure (PKI) to establish trust about the identity of the communication counterpart. The concept of a PKI is based on public certificates and private keys.

The following example illustrates how to generate a self-signed certificate using the OpenSSL library:

$ openssl req -new -x509 -sha512 -newkey rsa:4096 -out caps.crt -keyout caps.key -nodes

The last parameter -nodes disables the password protection of the private key. If omitted, a password must be defined which will be requested when accessing the private key. CAPS will request the password on the command line during startup.

To enable SSL in CAPS the AS.SSL.port as well as the location of the AS.SSL.certificate and AS.SSL.key file must be specified. Optionally the unencrypted AS.port may be deactivated by setting a value of -1.

Module Configuration

etc/defaults/global.cfg

etc/defaults/caps.cfg

etc/global.cfg

etc/caps.cfg

~/.seiscomp/global.cfg

~/.seiscomp/caps.cfg

caps inherits global options.

Note

Modules/plugins may require a license file. The default path to license files is @DATADIR@/licenses/ which can be overridden by global configuration of the parameter gempa.licensePath. Example:

gempa.licensePath = @CONFIGDIR@/licenses

Note

AS.* CAPS server control parameters

AS.filebase

Default: @ROOTDIR@/var/lib/caps/archive

Type: string

Defines the path to the archive directory.

AS.port

Default: 18002

Type: int

Defines the server port for client requests.

AS.clientBufferSize

Default: 16384

Unit: B

Type: int

Size of the client buffer in bytes. In case the client fails to read the buffered data in time (buffer overflow) the connection falls back to archive requests.

AS.minDelay

Default: -1

Unit: s

Type: int

Limits the retrieval of real-time data. The value specifies the maximum relative end time of the time range to be requested. The maximum absolute end time is now - minDelay. This is only valid for FDSNWS and WWS.

AS.inventory

Type: path

The path to an optional inventory XML file with SeisComP3 schema. This inventory information is used by WWS to populate the channel coordinates. In future possibly more endpoints will make use of it.

AS.logRequests

Default: false

Type: boolean

Whether to maintain a request log file or not. Each request will be logged and partly traced.

AS.logAnonymousIP

Default: false

Type: boolean

Log only parts of the IP to respect users privacy.

AS.logPurge

Default: false

Type: boolean

Whether to maintain a purge log file or not. Each purge operation will be logged.

AS.allow

Type: list:string

List of IPs which are allowed to access the caps(s) port. By default access is unrestricted.

AS.deny

Type: list:string

List of IPs which are not allowed to access the caps(s) port. By default access is unrestricted.

Note

AS.filebase.* File buffer control parameters

AS.filebase.logFile

Type: path

The path to the archive log file which contains the stream start and end times. By default it is written to $filebase/archive.log.

AS.filebase.keep

Default: *.*.*.*:-1

Type: list:string

Number of days to keep data per stream ID before "AS.filebase.purge.referenceTime". For stream-specific configuration create a list of pairs consisting of stream ID : days. Separate pairs by comma. The first occurrence in the list takes priority.

Example keeping all streams but AM.* and GR.* for 14 days:

GR.*:-1, AM.*.*.*:365, *.*.*.*:14

Default (empty parameter) or -1: keep all data forever.

AS.filebase.preallocationSize

Default: 65535

Unit: B

Type: int

Preallocation size of data files in bytes. Some file system allow to reserve disk space for files in advance. Especially on spinning disks the read performance will be improved if data can be read sequentially. The speed is traded for disk space consumed by the file since its size will be a multiple of the specified value. Set the value to 0 to disable this feature.

Note

AS.filebase.cache.* CAPS does not keep all files of all streams open. It tries to keep open the most frequently used files and closes all others. The more files CAPS can keep open the faster the population of the archive. The limit of open files depends on the security settings of the user under which CAPS is running.

AS.filebase.cache.openFileLimit

Default: 250

Type: int

The maximum number of open files. Because a stream file can have an associated index file this value is half of the physically opened files in worst case.

AS.filebase.cache.unusedFileLimit

Default: 1000

Type: int

Limit of cached files in total. This value affects also files that are actually explicitly closed by the application. CAPS will keep them open (respecting the openFileLimit parameter) as long as possible and preserve a file handle to speed up reopening the file later.

AS.filebase.params.writeMetaOnClose

Default: false

Type: boolean

This is an optimization to write the datafile meta record only on file close and not every time a new record has been added to a file. To save IO bandwidth when handling many channels, this could be helpful.

AS.filebase.params.alignIndexPages

Default: false

Type: boolean

This forces to align index pages in the file at 4k boundaries. In order to achieve that, NULL chunks must be inserted to allow padding. This will lead to less device page updates but slightly larger data files.

AS.filebase.params.priority

Default: 0

Type: int

A value greater than 0 will raise the write thread priority to the given value. This value is in accordance to the pthread_setschedparam function.

AS.filebase.params.q

Default: 1000

Type: int

The real-time notification queue size.

Note

AS.filebase.purge.* Parameters controlling IO resources occupied by the purge operation. The deletion of many data files at once may have a significant impact on the server performance. E.g. if the server did not run for a while or the keep parameter was reduced significantly, the purge operation may slow down the processing of real-time data.

AS.filebase.purge.referenceTime

Default: EndTime

Type: string

Values: EndTime,Now

The reference time defining the end of the time window to keep the data. The window length is set by "AS.filebase.keep". Data outside the window will be purged. Available values:

EndTime: The reference time is the end time per stream. This keeps older data if no more recent data arrive.

Now: The reference time is current time. This deletes old data even if no recent data arrive.

AS.filebase.purge.idleTime

Default: 5

Unit: s

Type: double

Idle time between two purge runs.

AS.filebase.purge.initIdleTime

Default: 0

Unit: s

Type: double

Idle time before the first purge run starts. Normally after a start the server tries to catch up all data which might be an IO intensive operation. In case of a huge archive the purge operation slow downs the read/write performace of the system too. To reduce the load at start it is a good idea to postpone this operation.

AS.filebase.purge.maxProcessTime

Default: 1

Unit: s

Type: double

Maximum processing time for one purge run. If exceeded the purge task will pause for AS.filebase.purge.idleTime seconds freeing IO resources.

AS.filebase.purge.startTime

Default: 00:30

Type: string

Time of the day when to run the daily purge run. Time is in UTC.

Note

AS.SSL.* Parameters for SSL-based data requests

AS.SSL.port

Type: int

Defines the SSL server port for client requests. By default SSL requests are disabled.

AS.SSL.certificate

Type: string

Defines the path to the SSL certificate to use.

AS.SSL.key

Type: string

Defines the path to the private SSL key to use. This key is not shared with clients.

Note

AS.auth.* Parameters controlling the authentication system for data requests based on user ID, IP addresses, access roles and access control lists.

AS.auth.backend

Default: basic

Type: string

The server provides an authentication plug-in interface. An authentication plugin implements access control checks. It is free where it gets the access information from e.g from a local database/file or a remote server. The option sets which authentication plugin should be used for authentication. Don’t forget to load the plugin in the plugin section. The basic plugin is built-in.

Note

AS.auth.basic.* Basic authentication parameters. The configuration can be reloaded without restarting the server. Use “seiscomp reload caps`” to reload the authentication parameters without a restart.

AS.auth.basic.access-list

Default: @SYSTEMCONFIGDIR@/caps/access.cfg

Type: file

Path to the access control list controlling access based on rules. By default access is unrestricted. Allow rules are evaluated first.

AM.DENY = 127.0.0.1

AM.ALLOW = 127.0.0.1

This example rule set prohibits all AM network stations for localhost because the DENY rule is evaluated after the ALLOW rule.

IP restrictions apply to the guest user only. In addition to IPs the access can be also restricted by user or group. In the latter case the "%" must be placed in front of the group name. Here an example:

AM.ALLOW = %users

AM.R44F5.ALLOW = sysop

Rules are evaluated on the basis of one another. This can lead to misunderstandings. Here an example:

AM.ALLOW = sysop

This rule will allow the AM network for sysop only. But

DENY = %users AM.ALLOW = sysop

will allow the access to the AM network for all users except those are member of the group users.

AS.auth.basic.users.shadow

Default: @SYSTEMCONFIGDIR@/caps/shadow.cfg

Type: file

Location of the users authentication file. For each user one line of the following format must exist:

username:encrypted_pwd

To encrypt the password mkpasswd can be used. It is recommended to apply a strong algorithm such as sha-256 or sha-512. The command

u=sysop pw=`mkpasswd -m sha-512` && echo $u:$pw

generates one line for user "sysop". Add the line to the authentication file.

AS.auth.basic.users.passwd

Default: @SYSTEMCONFIGDIR@/caps/passwd.cfg

Type: file

Location of the users access control file. Each line starts with a user ID (uid) or a group ID (gid) and a list of access properties in the form:

uid:prop1,prop2

or

%gid:prop1,prop2

"%" indicates a gid instead of a uid. The properties grant access to certain CAPS features. Supported access property values are: read, write, admin.

AS.auth.basic.users.group

Default: @SYSTEMCONFIGDIR@/caps/group.cfg

Type: file

Location of the optional group file. Each line maps a group id to a list of users in format

gid:user1,user2,user3

AS.plugins.port

Default: 18003

Type: int

Defines the server port to use for plugin connections.

AS.plugins.allow

Type: list:string

List of IPs which are allowed to access the plugin port. By default access is unrestricted.

AS.plugins.deny

Type: list:string

List of IPs which are not allowed to access the plugin port. By default access is unrestricted.

AS.plugins.SSL.port

Type: int

Defines the SSL server port to use for plugin SSL connections. The SSL port is disabled by default.

AS.plugins.SSL.certificate

Type: string

Defines the path to the SSL certificate to use.

AS.plugins.SSL.key

Type: string

Defines the path to the private SSL key to use. This key is not shared with clients.

Note

AS.http.* Web interface control parameters

AS.http.port

Type: int

Defines the server port for HTTP connections. By default the Web interface is disabled.

Typical value: 18081

AS.http.allow

Type: list:string

List of IPs which are allowed to access the http(s) port. By default access is unrestricted.

AS.http.deny

Type: list:string

List of IPs which are not allowed to access the http(s) port. By default access is unrestricted.

AS.http.resolveProxyClient

Default: false

Type: boolean

Sets if the X-Forwarded-For HTTP header is evaluated to retrieve the real client IP address from a proxy server. This is important if the web frontend is behind a proxy, e.g. Apache. Since data access is configured per IP, the real IP is required to grant access to requested channels. Enabling this opens a possible security hole as clients can then easily spoof their IP if the proxy does not correctly maintain this header or if CAPS does not run behind a proxy.

AS.http.disableBasicAuthorization

Default: false

Type: boolean

Controls whether basic authorization is enabled or not. In case you are running CAPS behind a proxy which already configures basic authorization then enable this flag. If basic authorization is disabled then the default HTTP user should have access without a password.

AS.http.fdsnws

Type: string

Sets the optional relative FDSNWS path which is being used by the CAPS frontend client. Do not append "fdsnws/dataselect/1/query" as this is done automatically. Set it to "/" if the CAPS frontend is running with a relative path behind e.g. Nginx.

Note

AS.http.SSL.* Use https instead of http when setting the following parameters

AS.http.SSL.port

Type: int

Defines the server port for HTTPS connections. By default the SSL Web interface is disabled.

AS.http.SSL.certificate

Type: string

Defines the path to the SSL certificate to use.

AS.http.SSL.key

Type: string

Defines the path to the private SSL key to use. This key is not shared with clients.

Note

AS.FDSNWS.* FDSNWS control parameters for dataselect. The FDSNWS service is provided through the “AS.http.port”.

AS.FDSNWS.maxTimeWindow

Default: 0

Unit: s

Type: int

Maximum length of time window per request. A value greater than zero limits the maximum request time window including all data. 0 disables the limit.

AS.FDSNWS.maxRequests

Default: 1000

Type: int

Maximum number of requests per post. A value greater than or equal to zero limits the number of request lines per POST request.

Note

AS.WWS.* Winston waveform server (WWS) control parameters. When set, CAPS will also serve WWS.

AS.WWS.port

Type: int

Server port for WWS connections. Please note that inventory information (see AS.inventory) is required to fully support WWS requests otherwise empty values for the channel location and unit will be returned.

Default (no value): The WWS interface is disabled.

AS.WWS.maxTimeWindow

Default: 90000

Unit: s

Type: int

Maximum length of time window in seconds per request. A value greater than zero limits the maximum request time window including all data. 0 disables the limit.

AS.WWS.maxRequests

Default: 100

Type: int

A value greater than or equal to zero limits the number of request lines per POST request.

AS.WWS.allow

Type: list:string

List of IPs which are allowed to access the WWS port. By default access is unrestricted.

AS.WWS.deny

Type: list:string

List of IPs which are not allowed to access the WWS port. By default access is unrestricted.

Command-Line Options

caps [options]

Generic

-h, --help

Show help message.

-V, --version

Show version information.

--config-file arg

Use alternative configuration file. When this option is used the loading of all stages is disabled. Only the given configuration file is parsed and used. To use another name for the configuration create a symbolic link of the application or copy it. Example: scautopick -> scautopick2.

--plugins arg

Load given plugins.

-D, --daemon

Run as daemon. This means the application will fork itself and doesn’t need to be started with &.

Verbosity

--verbosity arg

Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info, 4:debug.

-v, --v

Increase verbosity level (may be repeated, eg. -vv).

-q, --quiet

Quiet mode: no logging output.

--print-component arg

For each log entry print the component right after the log level. By default the component output is enabled for file output but disabled for console output.

--component arg

Limit the logging to a certain component. This option can be given more than once.

-s, --syslog

Use syslog logging backend. The output usually goes to /var/lib/messages.

-l, --lockfile arg

Path to lock file.

--console arg

Send log output to stdout.

--debug

Execute in debug mode. Equivalent to --verbosity=4 --console=1 .

--trace

Execute in trace mode. Equivalent to --verbosity=4 --console=1 --print-component=1 --print-context=1 .

--log-file arg

Use alternative log file.

Server

-p, --server-port int

Overrides configuration parameter AS.port.

--server-ssl-port int

Overrides configuration parameter AS.SSL.port.

-P, --plugin-port int

Overrides configuration parameter AS.plugins.port.

--http-port int

Overrides configuration parameter AS.http.port.

--read-only

Do not store any packets.

Test

--configtest

Run a configuration file syntax test. It parses the configuration files and either reports Syntax Ok or detailed information about the particular syntax error.

--print-access

Print access information for one or more channels from a given IP and a user with password, format: NET.STA.LOC.CHA, e.g.,

IP check

caps --print-access GE.*.*.* 127.0.0.1

IP and user:password check

caps --print-access GE.APE.*.* --user gempa:gempa 127.0.0.1

The stream ID filter supports wildcards. Use option -v to enable the trace mode to get detailed information about the rule evaluation.

-u, --user

Server user and password. Format: user:password .