sceewlog

Logs EEW magnitude messages via file, email, ActiveMQ and FCM.

Description

Part of the EEW package (used by VS and FinDer).

sceewlog logs the VS and FinDer magnitude messages received from scvsmag and scfinder (configurable), broadcasts these alerts, and triggers scripts, generates and disseminates reports.

sceewlog is released under the GNU Affero General Public License (Free Software Foundation, version 3 or later). It requires the Python 3 and the package dateutil [2] to be installed. sceewlog replaces scvslog that was originally part of the SeisComP implementation of the Virtual Seismologist [1] (VS) Earthquake Early Warning algorithm.

Once an event has timed out, the modules generates report files. These report files are saved to disk and can also be sent via email.

It also implements ActiveMQ [3] and Firebase Cloud Messaging [8] interfaces which can send alert messages in real-time. It supports regionalized filter profiles (based on polygons and EQ parameter threshold values). Currently, messages can be sent in four different formats (sc3ml, qml1.2-rt, shakealertml the UserDisplay format, CAP1.2). Optionally, a comment with ID EEW storing the EEW update number, can be added to each magnitude that passes all configured filters and scoring.

The SED-ETHZ team provides a client that can display qml1.2-rt messages sent via ActiveMQ, the Earthquake Early Warning Display (EEWD) [4] an OpenSource user interface developed within the European REAKT project and based on early versions of the ShakeAlert UserDisplay [5].

To receive alerts with the EEWD set the format to qml1.2-rt. There are currently no clients which can digest sc3ml and the shakealertml is not supported. Using pipelines alerts can be sent out in more than one format.

The real-time ActiveMQ interface requires the Python packages stompy [6] and lxml [7] to be installed.

It is beyond the scope of this documentation to explain the complete setup of an ActiveMQ broker. However, since sceewlog uses the STOMP protocol to send messages to the broker it is essential to add the following line to configuration of the ActiveMQ broker.

<connector>
<serverTransport uri="stomp://your-server-name:your-port"/>
</connector>

Please refer to ActiveMQ [3] for setting up an ActiveMQ broker.

Firebase Cloud Messaging

EEW messages are controlled by the parameters Magnitude Association and Regionalized Profiles. Notifications are sent using the HTTP v1 [9] when topic notification option is enabled and it requires the following python libraries:

google
requests
google-auth-oauthlib
firebase-admin

Further information on the Firebase Cloud Messaging interface can be found in Firebase Cloud Messaging [8]. This interface is activated in the module configuration file with the option:

FCM.activate = true

When this is enable then two options are possible:

1. For sending push notification to a topic:

FCM.topicnotification = true

In this case the configuration file defined in FCM.dataFile must be provided and containing the firebase configuration. Additionally, the python libraries mentioned above must be installed. The configuration file includes the path to a service JSON (see the Firebase project console Service JSON File [10]), the broadcasted topic (notification topic), the project ID string (see Project ID [11]), and it controls (enable or disable) notification to Android, iOS and a legacy notification format. Below is an example how this file, referenced in the FCM.dataFile, looks like:

[TOPICS]
topic=eqAlerts
[SERVICEFILE]
servicefile= /opt/fcm/credentials/projectServiceFile.json
[PROJECTID]
projectid = myappid
[ENABLED_OS]
android = true
ios = true
[SUPPORT_OLD_FORMAT]
oldformat = true

2. For creating an eew message dictionary:

FCM.eewmessagecomment = true

In this case this message contains the EQ origin information. It will be added as a comment type: eewmessage and text will be the message dictionary. In addition to this, the comment will be sent to the messaging system so that any other module can listen to this comment.

Reports

Below is an example of the first few lines of a report file:

                                                                   |#St.   |
Tdiff |Type|Mag.|Lat.  |Lon.   |Depth |origin time (UTC)      |Lik.|Or.|Ma.|Str.|Len. |Author   |Creation t.            |Tdiff(current o.)
------------------------------------------------------------------------------------------------------------------------------------------
  5.24| MVS|2.40| 46.05|   6.89| 20.53|2020-06-23T06:25:38.55Z|0.40|  4|  2|    |     |scvsmag2@|2020-06-23T06:25:45.99Z|  7.44
  6.24| MVS|3.69| 46.05|   6.89| 20.53|2020-06-23T06:25:38.55Z|0.40|  4|  4|    |     |scvsmag2@|2020-06-23T06:25:46.99Z|  8.45
  6.79| MVS|3.71| 46.05|   6.89| 20.53|2020-06-23T06:25:38.55Z|0.40|  4|  3|    |     |scvsmag@s|2020-06-23T06:25:47.54Z|  8.99
  7.24| MVS|3.65| 46.05|   6.89| 22.30|2020-06-23T06:25:38.33Z|0.99|  6|  5|    |     |scvsmag2@|2020-06-23T06:25:48.00Z|  9.67
  7.79| MVS|3.53| 46.05|   6.89| 22.30|2020-06-23T06:25:38.33Z|0.99|  6|  5|    |     |scvsmag@s|2020-06-23T06:25:48.54Z| 10.21
  8.24| MVS|3.61| 46.05|   6.89| 22.30|2020-06-23T06:25:38.33Z|0.99|  6|  5|    |     |scvsmag2@|2020-06-23T06:25:48.99Z| 10.66
  8.62| Mfd|4.00| 46.04|   6.88|  5.00|2020-06-23T06:25:41.93Z|0.88|  0|   |  80| 0.28|scfdalpin|2020-06-23T06:25:49.37Z|  7.44
  8.62| Mfd|3.90| 46.04|   6.88| 12.00|2020-06-23T06:25:40.29Z|0.85|  0|   | 140| 0.38|scfdforel|2020-06-23T06:25:49.37Z|  9.07

Creation time is the time the VS magnitude message was generated, tdiff is the time difference between creation time and last origin time in seconds, lik. is the likelihood that this event is a real event (see documentation of scvsmag), #St.(Or.) is the number of stations that contributed to the origin and #St.(Ma.) the number of envelope streams that contributed to the magnitude. Str. and Len. are the strike and length of the fault line provided by scfinder.

Regionalized Filters

To filter alerts to be sent out through ActiveMQ or FCM, it is necessary to set profiles on RegFilters section. Since this is using regions defined as closed polygons, then the first step is to provide a BNA file that contains the polygons. If the user does not provide a BNA file, then the other profile parameters will be evaluated globally.

RegFilters.bnaFile = /opt/seiscomp/share/sceewlog/closedpolygons.bna

Then profile names have to be set. Two profile examples are provided below.

RegFilters.profiles = global, America

The global profile is not configured with polygon since this spans on the entire world. The America profile uses the “America” closed polygon defined in activeMQ.bnaFile.

RegFilters.profile.global.bnaPolygonName = none
RegFilters.profile.America.bnaPolygonName = America

The magnitude and likelihood threshold values might be:

RegFilters.profile.global.magThresh = 6.0
RegFilters.profile.global.likelihoodThresh = 0.5
RegFilters.profile.America.magThresh = 5.0
RegFilters.profile.America.likelihoodThresh = 0.3

There might also be a depth filter for each profile. The following parameters might be used to configure the global profile with shallow events, and the America profile with events from 0 to 100 km deep.

RegFilters.profile.global.minDepth = 0
RegFilters.profile.global.maxDepth = 33
RegFilters.profile.America.minDepth = 0
RegFilters.profile.America.maxDepth = 100

Finally, to avoid sending alerts for events outside of the network of interest for EEW applications, a maxTime can be set. The maxTime is the maximum delay in seconds between the magnitude creation time since the origin time. For the examples, on the global profile this parameter might be “-1” in order to skip this filter, whereas it could be set to 60 seconds for America. However, each of the Virtual Seismologist and Finite-Fault Rupture Detector algorithms have their own default thresholds superseding maxTime defined in sceewlog.

RegFilters.profile.global.maxTime = -1
RegFilters.profile.America.maxTime = 60

Magnitude Association

The magnitude association is similar than the scevent preferred-origin selection. The magnitude association evaluation is activated with the next configuration key-value pair:

magAssociation.activate = true

The following priorities are available:

1. typeThresh

2. likelihood

3. Authors

4. StationMagNumber

and they can be listed on the next key-value:

magAssociation.priority = magThresh,likelihood,authors,stationMagNumber

The priority decreases in the order of the parameters.

The magThresh prority is a list of minimal magnitude to be allowed for each type of magnitude:

magAssociation.typeThresh = Mfd:6.0,MVS:3.5,Mlv:2.5

For one specific update of an event, its magnitude type-value must be equal or higher than the listed ones to continue the evaluation. Otherwise, the evaluation ends and the evalution does not continue.

The authors can be also used and their priority depends on the position on which they are listed. For example:

#if magAssociation.priority contains authors then
#the next parameter must contain valid magnitude authors' names
magAssociation.authors = scvsmag@@@hostname@, \
scvsmag0@@@hostname@, \
scfd85sym@@@hostname@, \
scfd20asym@@@hostname@, \
scfdcrust@@@hostname@

In this list of authors the highest value is for scvsmag and for this particular example this author has a value of 5. The author value reduces after each comma separator. For the same example scvsmag0 is 4, scfd85sym is 3, and so. The author is evaluated for every event update. If the author for a particular update has a higher o equal priority than an already evaluated and sent alert, then the evaluation continues, otherwise, whether the author is listed or not, the evaluation ends.

If likelihood is listed on priorities then this is evaluated against with a previous update that has been sent out as alert. Only when there is a first event update the likelihood is not evaluated. In case that an update has been sent out as an alert, if the current likelihood is higher or equal than the previous one, then it goes to the next priority evaluation, otherwise, the update is discarted.

For the stationMagNumber evaluation is necessary to have a similar list of Magtype and min number of stations used to quantify the magnitude value. For example:

magAssociation.stationMagNumber = Mfd:7,MVS:4

In this example for the mag type Mfd is necessary to at least have 7 stations to pass this evaluation, otherwise, the evaluation ends. For MVS must be at least 4 stations that have been used to qunatify the magnitude, otherwise, the valution ends.

Headline Change for CAP1.2 XML alerts

The converted CAP1.2 xml alert messages contain a headline. The default headline is:

@AGENCY@ Magnitude X.X Date and Time (UTC): YYYY-MM-dd HH:mm:s.sssZ.

An alternative headline format might be preferred. The following alternative format can be selected:

@AGENCY@/Earthquake Magnitude X.X, XX km NNW of SOMECITY, SOMECOUNTRY

The aternative format supports both spanish and english languages. The spanish version is:

@AGENCY@/Sismo Magnitud X.X, XX km al SSO de SOMECITY, SOMECOUNTRY

The alternative format can be enable as follows:

CAPheadlineChange = true

The alternative format requires to specify the language and the corresponding file listing the world cities CitiesFile. The language can be selected as follows:

#Uncomment the next line to select English
AlertTextLanguage = en-US
#Uncomment the next line to select Spanish
#AlertTextLanguage = es-US

The file listing the world cities CitiesFile must have the following format:

city,country,lon,lat
Tokyo,Japan,139.6922,35.6897
Jakarta,Indonesia,106.8451,-6.2146
Delhi,India,77.23,28.66
Mumbai,India,72.8333,18.9667
Manila,Philippines,120.9833,14.6
Shanghai,China,121.4667,31.1667
Sao Paulo,Brazil,-46.6339,-23.5504

Both an english and a spanish version are provided in “@DATADIR@/sceewlog/world_cities_english.csv” and “@DATADIR@/sceewlog/world_cities_spanish.csv”.

References

[1]

http://www.seismo.ethz.ch/en/knowledge/earthquake-data-and-analysis-tools/EEW/Virtual-Seismologist/index.html

[2]

https://pypi.python.org/pypi/python-dateutil/

[3] (1,2)

http://activemq.apache.org/

[4]

http://www.seismo.ethz.ch/en/knowledge/earthquake-data-and-analysis-tools/EEW/earthquake-early-warning-display-eewd/

[5]

http://www.eew.caltech.edu/research/userdisplay.html

[6]

https://pypi.python.org/pypi/stompy/

[7]

http://lxml.de/

[8] (1,2)

https://firebase.google.com/docs/cloud-messaging

[9]

https://firebase.google.com/docs/cloud-messaging/migrate-v1?hl=es-419

[10]

https://console.firebase.google.com/u/0/project/_/settings/serviceaccounts/adminsdk

[11]

https://console.firebase.google.com/u/0/project/_/settings/general/

Configuration

etc/defaults/global.cfg

etc/defaults/sceewlog.cfg

etc/global.cfg

etc/sceewlog.cfg

~/.seiscomp/global.cfg

~/.seiscomp/sceewlog.cfg

sceewlog inherits global options.

magTypes

Type: list:string

The list of magnitude types handled by sceewlog Default is MVS, Mfd.

generateReportTimeout

Type: int

Timeout after which a report is generated. The timer is started on magnitude reception and restarted whenerver a new magnitude is received for the same event. Default is 5.

CitiesFile

Type: string

CSV file that contains the location of cities. It is mandatory when changeHeadline or FCM is true. Default is @ROOTDIR@/share/sceewlog/world_cities_english.csv.

EEW.comment

Type: boolean

Save a comment for each EEW alert sent out through ActiveMQ or FCM. The comment is part of the Magnitude object and stored on the Comment table of the SC-DB. Default is false.

EEW.script

Type: string

Executes an external script for each EEW alert that are sent out. The script must be defined with full file path, and it must be executable. It receives all EEW parameters in arguments as follows: magID=.. type=M.. author=.. magnitude=.. lat=.. lon=.. depth=.. nstorg=.. nstmag=.. ot=.. When the module is started, the script is tested with argument test=1. An example of a script preparing a data file for posting on social media is provided with $SEISCOMP_ROOT/bin/sceewlog2file.

CAPheadlineChange

Type: boolean

Change the headline in the cap1.2 XML alert. Default is false.

AlertTextLanguage

Type: string

Language for the string text on headline (cap1.2 XML) or FCM near location. Default is en-US.

email.activate

Type: boolean

Send email notifications. Default is true.

email.smtpserver

Type: string

URL of the smtp server to send the report files to.

email.port

Type: int

Port where the SMTP server accepts connections. Default is 25.

email.usetls

Type: boolean

Whether to use TLS when connecting to the smtp server. Default is false.

email.usessl

Type: boolean

Whether to use SSL when connecting to the smtp server. Note, only ‘email.usetls’ or ‘user.ssl’ can be true. Default is false.

email.authenticate

Type: boolean

Whether the smtp server requires authentication (username + password). Default is false.

email.credentials

Type: file

If the smtp server requires authentication you have to specify a file that contains username and password in the format:

username=your-username

password=your-password

Make sure that you set the file permission as restrictive as possible.

email.senderaddress

Type: string

Email address that will appear as sender in the report email.

email.subject

Type: string

Any string that should be prepended to the email’s subject string.

email.host

Type: string

Host as it is supposed to appear in the email’s subject string.

email.recipients

Type: list:string

A list of email addresses that receive the report emails.

email.magThresh

Type: double

Only send a notification email if the magnitude threshold is exceeded. Default is 0.0.

email.sendForAlertOnly

Type: boolean

Only send a notification email if an alert was sent (activeMQ, FCM, or script). Default is false.

report.activate

Type: boolean

Save reports to disk. Default is true.

report.eventbuffer

Type: int

Unit: s

Time in seconds that events and the related objects are buffered. Default is 600.

report.directory

Type: dir

Directory to save reports to. Default is ~/.seiscomp/log/EEW_reports.

ActiveMQ.activate

Type: boolean

Send Alerts through ActiveMQ msg broker. Default is true.

ActiveMQ.hostname

Type: string

Server name that runs the ActiveMQ broker. Default is localhost.

ActiveMQ.port

Type: int

Port on which ActiveMQ broker accepts stomp messages. Default is 61618.

ActiveMQ.username

Type: string

Username with which messages are sent. Default is your-ActiveMQ-username.

ActiveMQ.password

Type: string

Password with which messages are sent. Default is your-ActiveMQ-password.

ActiveMQ.topic

Type: string

ActiveMQ broker topic to send alerts to. Default is /topic/your-ActiveMQ-topic.

ActiveMQ.hbtopic

Type: string

ActiveMQ broker topic to send heart beats to. Default is /topic/your-ActiveMQ-topic.

ActiveMQ.messageFormat

Type: string

ActiveMQ message format; can be either ‘cap1.2’, ‘qml1.2-rt’, ‘shakealert’, or ‘sc3ml’. Default is qml1.2-rt.

RegFilters.profiles

Type: string

Profile names for regionalized alert filter. Use comma for more than one profile. Default is global.

RegFilters.bnaFile

Type: string

Path of BNA file that contains closed polygons which is used for regionalized filters. Default is none.

Note

RegFilters.profile.* Profiles containing the profile-specific alert filtering parameters.

Note

RegFilters.profile.$name.* $name is a placeholder for the name to be used and needs to be added to RegFilters.profiles to become active.

RegFilters.profiles = a,b
RegFilters.profile.a.value1 = ...
RegFilters.profile.b.value1 = ...
# c is not active because it has not been added
# to the list of RegFilters.profiles
RegFilters.profile.c.value1 = ...

RegFilters.profile.$name.magThresh

Type: double

Magnitude threshold; can be any float value. Default is 0.0.

RegFilters.profile.$name.likelihoodThresh

Type: double

Likelihood threshold; can be between 0.0 to 1.0. Default is 0.0.

RegFilters.profile.$name.minDepth

Type: double

Minimum depth value; can be any float value but less than maxDepth. Default is 0.0.

RegFilters.profile.$name.maxDepth

Type: double

Maximum depth value; can be any float value but greater than minDepth. Default is 800.0.

RegFilters.profile.$name.bnaPolygonName

Type: string

The polygon name must be one of the closed polygons that contains the ActiveMQ.bnaFile. If this is set to none, then the origin location will not be checked against any polygon. Default is none.

RegFilters.profile.$name.maxTime

Type: double

Time difference between origin time and magnitude creation time (in seconds). If this parameter is setup to -1, the time difference is not checked. Default is -1.

magAssociation.activate

Type: boolean

Magnitude association and scoring. The score for each EQ update is the product of magnitude, likelihood, author and num. of arrivals. Default is false.

magAssociation.priority

Type: string

The priorities can be set as: magThresh, likelihood or authors. It can be all together or just one or two. Listed comma separated. Default is magThresh,likelihood.

magAssociation.typeThresh

Type: string

Set the minimum magnitude threshold values for each magnitude type configured in magTypes. Default is Mfd:6,MVS:3.5.

magAssociation.authors

Type: string

Set the magnitude authors in a list comma separated. This should be generally the app name followed by the hostname. Default is scvsmag@@@hostname@,scfinder@@@hostname@.

magAssociation.stationMagNumber

Type: string

Set the minimum number of stations on which magnitude was quantified for every magnitude type. Default is Mfd:3,MVS:1.

FCM.activate

Type: boolean

Enable or disable FCM for using the eews2fcm interface. Default is false.

FCM.topicnofication

Type: boolean

Enable or disable FCM for topic notifications. It this case it is mandatory to provide the dataFile. Default is false.

FCM.eewmessagecomment

Type: boolean

Enable or disable this option to create a message dictionary that will be notify to the messaging system and persist it in the DB. Default is false.

FCM.dataFile

Type: string

The datafile must contain the authorization key and topic to send out a message preformated on the eews2fcm interface. Default is /home/sysop/seiscomp/share/sceewlog/.fcmdatafile.

Command Line

Generic

Verbosity

Messaging

Reports

--savedir path

Directory to save reports to.

Mode

--playback

Disable database connection.