.. _deepc-applications: Application and Setup ===================== The tasks of |deepc| can be applied along with phase picking in `scautopick`, `screpick` and `scolv` or the corresponding module aliases. To make them available to a module add the plugin in the global configuration (:file:`global.cfg` or the specific module configuration, e.g. of scautopick, :file:`scautpick.cfg`) like: .. code-block:: properties plugins = ${plugins}, deepc Depending on the module, a module configuration may be required to define the specific tasks of deepc. However, the specific configuration of |deepc| is done by global bindings parameters residing under ``picker.deepc`` and ``spicker.deepc``. .. note:: * By adding a line like ``module.trunk.global.`` in a module configuration file, each global binding parameter ** is set globally for all stations. This kind of parameters overrides any configured station binding and the configuration is not supported by :cite:t:`scconfig`. * In the following, many of the example configurations will for simplicity be presented as globally set in a module configuration file. In production, you should configure your global binding profiles instead. Example: scautopick ------------------- The main module using |deepc| is scautopick. As shown in :numref:`fig-deepc-workflow-scautopick`, `scautopick` forwards *detections* to |deepc|, which extracts waveforms in a window around the initial trigger. |deepc| then feeds the data to the loaded DL model and generates the output according to the model type and configuration. The *DeepC* package ships with 4 kinds of sub-pickers (tasks): 3 primary pickers *deepcec*, *deepcpc*, *deepcpr* and a secondary picker *deepcsr*. Each sub-picker has its own configurable global binding parameter group. See :ref:`this table ` for an overview. To start scautopick with |deepc|, define one of the sub-pickers to be applied to P and S phases by adjusting the module parameters ``picker`` and ``spicker``, respectively. Example (:file:`scautopick.cfg`): .. code-block:: properties plugins = ${plugins}, deepc picker = deepcec # picker = deepcpc # picker = deepcpr spicker = deepcsr In regular operation you should no activate the scautopick parameter ``sendDetections`` which suppresses emitting initial triggers in addition to |deepc|'s picks. However, activating this parameter can be really helpful for tuning and debugging. Add a global or scautopick binding or edit an existing one for configuring a sub-picker. In `scconfig`, navigate to ``Bindings``, choose a global or scautopick binding, search (Ctrl-F) for "deepc". Set the :confval:`picker.deepc.modelsPath` and :confval:`spicker.deepc.modelsPath` when re-picking P and S picks, respectively, and consider the remaining parameters in the sub-picker's section. There are also other common parameters which are less important. Their application will be discussed later. Further, you need to specify the model to use in each case. You find a collection of available models :ref:`here`. For an offline playback, run scautopick with your miniSEED data file writing the output picks to a well-formed :term:`SCML` file. .. code-block:: sh scautopick -I data.mseed --ep --playback --debug > picks.xml Task 1: Event classification ---------------------------- Currently no event classification model is provided by gempa. Once you have access to an appropriate model, follow these steps to use it. scautopick ^^^^^^^^^^ Add to your module configuration file (:file:`scautopick.cfg`): .. _deepc-config-example-scautopick: .. code-block:: properties [...] picker = "deepc-ec" [...] The next step is to choose a model that suits your data (currently none is available) and edit other parameters if needed. .. See section :ref:`deepc-available-models` for further details. * :confval:`picker.deepc.deepcEC.modelWeights` - this is the name of the chosen model ("weights") * :confval:`picker.deepc.deepcEC.picker` - edit if you want another, conventional picker like *AIC* as intermediate picker; you could keep your existing settings as they are by setting your preferred picker here instead of its usual configuration (:numref:`deepc-config-example-scautopick`, line 3); |deepc| will then classify the wrapped picker's pick instead of the STA/LTA trigger * :confval:`picker.deepc.deepcEC.threshold` - edit if you want to use a custom confidence threshold that labels picks not surpassing it as 'Noise'. A comment will be added with the proposed signal classification. screpick ^^^^^^^^ Use screpick to classify the signal type of picks. In this case, you can skip creating bindings and add the model to use globally directly in the screpick configuration (:file:`screpick.cfg`): .. _deepc-config-example-screpick: .. code-block:: properties plugins = ${plugins}, deepc picker = deepc-ec anyPhase = true module.trunk.global.picker.deepc.deepcEC.modelWeights = Now, just provide the file with already calculated picks and apply event classification using your preferred model: .. code-block:: sh screpick --debug -I data.mseed --ep picks.xml -f > classified-picks.xml .. note:: Ensure screpick's parameter `anyPhase` is set. If unsure, add `-A` to the command. The output file `classified-picks.xml` will contain the same picks provided with a comment about the event classification per pick. Output ^^^^^^ Example section of the output XML: .. code-block:: xml [...] earthquake deepc:eventTypeHint [...] To a add comment containing the confidence value of the classification, activate parameter :confval:`picker.deepc.addComment`. Task 2: Phase classification ---------------------------- scautopick ^^^^^^^^^^ Add to your module configuration (:file:`scautopick.cfg`): .. _deepcPC-config-example-scautopick: .. code-block:: properties picker = "deepc-pc" Choose a model that suits your data (currently only one is available) and edit the other parameters as needed. - :confval:`picker.deepc.deepcPC.modelWeights` - this is the name of the chosen model ("weights") - :confval:`picker.deepc.deepcPC.picker` - edit if you want another, conventional picker like *AIC* as intermediate picker; you could keep your existing settings as they are by setting your preferred picker here instead of its usual configuration (:numref:`deepcPC-config-example-scautopick`, line 3); |deepc| will then classify the wrapped picker's pick instead of the STA/LTA trigger - :confval:`picker.deepc.deepcPC.threshold` - edit if you want to use a custom confidence threshold that labels picks not surpassing it as 'Noise'. The phase hint of the pick will change if the model re-classifies it. In case of noise, no pick will be emitted unless you turn on :confval:`picker.deepc.sendNoise`. screpick ^^^^^^^^ If you have picks whose phases you like to (re)classify, you can skip bindings and, for convenience, add the model to use globally in the screpick configuration (:file:`screpick.cfg`): .. _deepc-config-example-screpick: .. code-block:: properties plugins = ${plugins}, deepc picker = deepc-pc anyPhase = true module.trunk.global.picker.deepc.deepcPC.modelWeights = Now, add your file with already calculated picks and screpick will apply phase classification using your preferred model: .. code-block:: sh screpick --debug -I data.mseed --ep picks.xml -f > classified-picks.xml .. note:: Activate the screpick configuration parameter `anyPhase`. If unsure, add :option:`-A` to the command. Output ^^^^^^ If the pick is not classified as noise, the phaseHint will be reset to the new phase. If :confval:`picker.deepc.addComment` is active, a comment will be added containing the old and the new phase hint as well as the confidence, e.g (:file:`classified-picks.xml`): .. code-block:: xml [...] PC: P -> S: 95.53% deepc [...] Task 3: Phase classification and pick refinement ------------------------------------------------ scautopick ^^^^^^^^^^ Set scautopick parameter `picker` to |deepcp| .. code-block:: properties picker = "deepc-pr" Choose a model that suits your data and configure the other parameters as needed. * :confval:`picker.deepc.deepcPR.modelWeights` - this is the name of the chosen model ("weights") * :confval:`picker.deepc.deepcPR.P.signalBegin`, :confval:`picker.deepc.deepcPR.P.signalEnd`, :confval:`picker.deepc.deepcPR.S.signalBegin`, :confval:`picker.deepc.deepcPR.S.signalEnd` - always adjust these values not only to your data, but also in compliance with the input length of the chosen model; e.g., for a model with only 5s input, reasonable restriction intervals could be :math:`[-1, 1]` for P and :math:`[0.25, 2.5]` for S. |deepcp| will only look for pick inside this range. See section :ref:`deepc-range-restriction` for details. * :confval:`picker.deepc.deepcPR.threshold` - if you want to overwrite the model's recommended confidence thresholds; reasonable values ranges from 0.1 to 0.9; in most cases, 0.3 is good start unless the model has already a recommended value which you can find out by leaving these parameters at -1; 0.1 would be the default if nothing was set or found in the models config file. Note that you can set P and S threshold separately. Look up the parameter for details on the configuration syntax. The pick time will be changed to the refined time and the phase hint of the pick will change in case the model re-classifies it. In case of noise, no pick will be emitted unless you turn on :confval:`picker.deepc.sendNoise`. .. _treating-s-picks: S picks """"""" |deepc| is not restricted to being the *primary picker*. It can also be used to create S picks which will reference the P pick produced by a *primary picker*. This can be achieved by using the special sub-picker |deepcs| as *secondary picker*: .. _deepc-config-example-scautopick-s: .. code-block:: properties spicker = "deepc-sr" As secondary picker, |deepcs| takes the same trigger as |deepcp|, but cuts a window slightly to the advantage of a potential S pick, applies DL inference and returns the first S pick after the trigger. .. note:: You can conveniently use |deepcp| and |deepcs| in the same run as `picker` and `spicker`, respectively. But in case |deepcp| reclassifies a P pick as S, there is a chance that you get two S picks when |deepcs| sees and emits the same S pick, this is because both instances work independently from each other. .. .. warning:: .. .. Regarding potential reclassification, |deepcs| uses the trigger provided by scautopick which .. is "meant" to be a P pick. However, the plugin knows that it should look for an S onset in the trace. .. The correct way of using |deepcs| is with parameter :confval:`picker.deepc.deepcSR.reclassify` unset. .. Thhis way, itwill just output its best guess for an S pick or nothing if the best candidate doesn't .. surpass the confidence threshold for S picks. .. Turning on this parameter would focus on the trigger and assume that it was meant to be an S pick .. but this is not the case for the trigger provided by scautopick. .. Therefore, it is recommended to unset this parameter. .. In the future, the decision tree might be reworked and reclassification might make sense. In the resulting pick object, the phase hint will be changed. Additionally, the phase classification can be found with the model's **confidence**, in a comment when :confval:`picker.deepc.addComment` is active. screpick ^^^^^^^^ :cite:t:`screpick` focuses on P picks and you **cannot** use |deepcs| as secondary picker. Instead, |deepcp| sufficiently generates S picks in reclassification mode although it may prefer P picks over S picks. Here is an example configuration (:file:`screpick.cfg`) using PhaseNet trained on STEAD ("pn-st"): .. _deepc-config-example-screpick: .. code-block:: properties plugins = ${plugins}, deepc picker = deepc anyPhase = true module.trunk.global.picker.deepcPR.reclassify = true module.trunk.global.picker.deepcPR.modelWeights = pn-st Note that `anyPhase` should be enabled otherwise S picks won't be processed. .. code-block:: sh screpick --debug -I data.mseed --ep picks.xml -f > repicks.xml .. warning:: If you turn off :confval:`picker.deepc.deepcPR.reclassify` but leave screpick's parameter ``anyPhase`` enabled, |deepcp| may be also fed with S picks, but then an S pick might be "corrected" to the time of a potential P pick, because |deepcp| can't see the original phase hint. So it guesses that this pick was way off from the intended P pick and it will change the pick time accordingly, although this pick was supposed to mark an S onset. (This restriction might change in the future) Here is a comprehension of use cases: .. csv-table:: :header: Purpose, anyPhase, reclassify, comment :widths: 40, 20, 20, 20 Repick P phases only, false, false, Repick S phases only, - , - , not possible Repick P and S phases, true, true, **Output**: The phase hint of the pick will change if the model re-classifies it. In case of noise, no pick will be emitted unless you turn on :confval:`picker.deepc.sendNoise`. The confidence can be added in a comment via :confval:`picker.deepc.addComment`. scolv ^^^^^ You can use |deepc| as interactive re-picker inside the scolv waveform picker window. Since most picker models need three components, it is therefore necessary to load them in scolv by pressing 'T' (followed by 'E' and 'N' for horizontal components to show up), or use the scolv parameter :confval:`picker.loadAllComponents`. Remember to unset the filter before repicking due to most models being trained on raw waveform data or having their own filter preprocessor. It is advisable to only repick filtered data when the model couldn't find a pick in the raw data. You can pick both, P and S onsets, but currently the only available sub-picker for both is |deepcp|, which has a preference for P picks. To explicitly tell |deepcp| to only pick P onsets, turn **off** :confval:`picker.deepc.deepcPR.reclassify`. Turn **on** the same parameter if you want to allow S picks. Detailed information about the classification is currently only provided in the terminal. You will find a message whether a pick was found and whether it was re-classified to S. .. note:: In reclassification mode, |deepcp| will always assume that you wanted to refine a P pick. Have a look into the :ref:`decision tree ` for comprehension. In consequence, |deepcp| may return a P pick if the confidence for an S pick does not exceed the threshold in the picking range that was specifically allowed for S picks. See next section of how to define this range. Starting scolv """""""""""""" Scolv needs to know how much data it should send to |deepc|. Therefore two scolv module parameters have to be set: `picker.repickerStart` and `--picker.repickerEnd`. They define the input window for the model relative to the time at the cursor position. This information can be extracted from the *model's config file*. Here is an example for PhaseNet(ETHZ): .. code-block:: sh :caption: *File /home/data/dlmodels/dlmodels-pick/pn-et.json* [...] 39 "time_before_pick_sec": 15.005, 40 "time_after_pick_sec": 15.005, [...] `time_before_pick_sec` and `time_after_pick_sec` are used by |deepc| to cut the window around the trigger. Therefore, adjust the scolv parameters `picker.repickerStart` and `picker.repickerEnd` accordingly while keeping in mind that scolv won't call |deepc| unless there is enough data. In general these values do not vary a lot, default values for PhaseNet are -15.005 and 15.005, for EqTransformer models -30 and 30. You don't need to be this exact, in fact it is not a bad idea to enlarge the window by a second or two since this helps with the data pre-processing steps. **Example call**: .. code-block:: sh scolv -I data.mseed -i picks.xml --debug Turn on the debug mode to see your model's decisions in the terminal while you are repicking with mouse and keyboard. Confidence output inside scolv GUI """""""""""""""""""""""""""""""""" By default, the confidence is shown in the *SNR* field of the marked trace. If the confidence was too low for the pick, it will be shown as **negative number**. To prevent this behavior and get the actual SNR value instead, deactivate :confval:`picker.deepc.deepcPR.publishConfidenceAsSnr`. .. _deepc-range-restriction: Restricting the picking range ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In most scenarios it is of advantage to restrict the range of allowed picks in a much narrower window around the cursor in contrast to the full length of the time window fed into |deepcp| and |deepcs|. The input window might be so wide that it includes other onsets that you are not interested in. |deepcp| always returns the pick with the highest confidence in the entire window. This means a pick with a high SNR is very likely to be returned as the most confident pick, but may not be of interest for you. You can force |deepcp| to only pick inside `P.signalBegin` and `P.signalEnd` (`S.signalBegin` and `S.signalEnd` respectively) around the cursor in scolv / around the trigger, in general. Example using scolv repicker 1 """""""""""""""""""""""""""""" .. code-block:: properties :emphasize-lines: 2,3 module.trunk.global.picker.deepc.deepcPR.publishConfidenceAsSnr = true module.trunk.global.picker.deepc.deepcPR.P.signalBegin = -5 module.trunk.global.picker.deepc.deepcPR.P.signalEnd = 4 module.trunk.global.picker.deepc.deepcPR.reclassify = false module.trunk.global.picker.deepc.deepcPR.threshold = 0.4 These settings would allow |deepcp| to return a pick only if it was found between five seconds before the mouse cursor's position and 4 seconds after it. If there is no confidence peak above the given threshold ``0.4``, |deepcp| will tell you it's noise (in the terminal) even though there might already be A reasonable pick outside of this interval and inside of the waveform data |deepcp| was fed with (blue-ish zone). The SNR field shows the confidence of the model about it's noise classification but as negative value. ``[SCREENSHOT]`` Example using scolv repicker 2 """""""""""""""""""""""""""""" .. code-block:: sh :emphasize-lines: 4,5,6,7 module.trunk.global.picker.deepc.deepcPR.publishConfidenceAsSnr = true module.trunk.global.picker.deepc.deepcPR.P.signalBegin = -5 module.trunk.global.picker.deepc.deepcPR.P.signalEnd = 4 module.trunk.global.picker.deepc.deepcPR.reclassify = true module.trunk.global.picker.deepc.deepcPR.S.signalBegin = 0.5 module.trunk.global.picker.deepc.deepcPR.S.signalEnd = 10 module.trunk.global.picker.deepc.deepcPR.threshold = 0.4,S:0.3 As the first example, these settings would allow |deepcp| to return a pick only if it was found between five seconds before the mouse cursor's position and 4 seconds after it. If there is no confidence peak above the given threshold ``0.4``, |deepcp| will check if a potential S pick would be in the range of 0.5 to 10 s after the trigger, and whether its confidence exceeds the threshold of ``0.3``. In Figure .. this is the case, the mouse cursor is moved to position of the S pick and the SNR field shows the confidence of the classification. ``[SCREENSHOT]`` Filtering waveforms before input -------------------------------- Most models are not trained on filtered waveform data and require raw data since filters are already encoded within the models parameters by training. However, some data might benefit from filtering. Use these parameters carefully. - :confval:`picker.deepc.deepcEC.filter` - :confval:`picker.deepc.deepcPC.filter` - :confval:`picker.deepc.deepcPR.filter` - :confval:`spicker.deepc.deepcSR.filter` The value must comply with the usual |scname| filter syntax. ``[LINK?]`` Treating problematic data ------------------------- The following descriptions are valid not only for parameter group `picker.deepc`, but also for `spicker.deepc.deepcSR`. Ranges of zeros in waveforms can be a problem for DL models as their predictions are not reliable. You can define the length of leading zeros that falsifies a pick in the parameter group :confval:`picker.deepc.pickFalsification`. This check is performed for the pick (or trigger) that |deepc| receives on start, such that, when the check fails, it won't process the waveforms and won't send the initial pick. To send the initial pick regardless of the zero-check, set parameter :confval:`picker.deepc.sendRejected`. Set `picker.deepc.pickFalsification.signalBegin` to define the window start relative to the trigger. Set `picker.deepc.pickFalsification.zeros` to define the number of adjacent zeros to look for in this window.