Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

...

Warning

Be aware that when new scripts are placed in the /usr/share/indeni-knowledge/overwrite/ind/parsers/ folder they only overwrite scripts that have the same "command name" in the META section of the script. The scripts they overwrite are the ones /usr/share/indeni-knowledge/stable/ind/parsers/src/.

This can cause issues when renaming *.ind script files, or, in general, when using the overwrite directory. You may end up with two different *.ind scripts (different file names), both with the same command name (in the META section). In this case, the script in /stable/ind/parsers/src/ folder and the script in the /overwrite/ind/parsers/ folder would both execute and write data for the same metric – generally not what you want.

Checking

...

metric value in the time series database

Code Block
curl -G -k -u "admin:admin123!" "https://localhost:9009/api/v1/metrics" --data-urlencode "query=(im.name==config-unsaved and device-id=='f8dccd39-fc7f-4e41-aa03-81965c9c9fde')" | python -m json.tool

You got an alert, but it doesn't make sense? How did it happen? What was the metrics' values when it was generated?
You can access the database and pull double metrics yourself. SSH into the
indeni Indeni server and use this: (the last part of the line is using sed to get each device on a separate line for improved readability)

Notes:

  • Currently, these queries only work for double metrics; there's no way of querying complex metrics, unless they are tagged as live-config (in which case the last value of the three will appear in the device information).
  • Replace

    • config-unsaved with the double-metric you'd like to fetch.

    • device-id with the device ID. To find your device id, run the following command first (it'll dump the list of devices from the database):

      Code Block
      psql -c "select id,name,ip_address from device;"

      The output of a metric query looks like this:

      Code Block
      [
        {
      	"type": "ts",
          "tags": {
            "im.dstype": "gauge",                  # i.m. --> "Indeni Metric" Data Storage Type
            "im.dstype.displaytype": "boolean",
            "im.name": "config-unsaved",           # Indeni Metric Name
            "im.step": "300",                      # Monitoring interval (from script META) in seconds
            "device-id": "a158058a-afa4-48f7-bbbe-a789ddc82ed7",
            "live-config": "true",
            "display-name": "Configuration Unsaved?",
          },
          "points": [null,null,null,null,null,null,null,null,null,null,null,1.0,null,null,null,null,0.0,null,null,null,null,0.0,null,null,null,null,null,0.0,null,null,null,null,0.0,null,null,null,null,0.0,null,null,null,null,0.0,null,null,null,null,0.0,null,null,null,null,0.0,null,null,null,null,0.0,null,null,null],
          "start": 1484481240000,
          "step": 60000
        }
      ]

      Which shows you the metric's tags, as well as the points, in a "time series", themselves. Ignore the nulls. The series shows data collected in 1 minute intervals, according to the "step": 60000 (60,000 miliseconds) field. The script monitoring interval is shown in the "im.step": "300" field, in seconds (so 5 mins).
      That's why you see a non-null entry every ~5 nulls (since the script started running -- to start with, the entire series is null).
      In the above, you'll notice that earlier in the series, the metric was 1.0, then later 0.0, and stayed 0.0 for the rest of the period.
      The points are sorted by oldest-first, so the first entry in the list is the oldest one. If you've used the command above, it means the oldest is roughly an hour old.

Clearing metrics from the database

Some of the metrics are written to an in-memory DB; in this case you can just restart the indeni-server and the metrics will be deleted. To restart the server, from your Indeni server CLI, you can run imanage 3.

...

Deleting sub-directories here (organized by device UUID) will delete any metrics stored here.

Deleting alerts from a device

When an alert is acknowledged it will not re-appear even if the issue is ongoing.
The issue needs to get resolved and then re-appear for the alert to trigger again.

If you would like to re-trigger an alert, you can delete the alert (or all alerts) from a device, and start your test over again.

To delete a device's alerts, you need to use the command-line on the indeni server and connect to the Postgres database there:

Code Block
indeni@ind-local:~$ psql indeni        <-- start the posgres shell against the indeni db
indeni=> select id, name, ip_address from device;     <-- list the devices connected to the indeni server (to find your device id)
                  id                  |           name           |   ip_address   
--------------------------------------+--------------------------+----------------
 9b5f66e5-3e10-4363-9973-c3dc0478bf9c | chkp-lab-CP-MGMT1-2      | 192.168.194.31
 c25acab5-9d15-4e6c-9cab-a33049534a72 | chkp-lab-CP-GW1          | 192.168.194.36
...
indeni=> select id from device where name = 'chkp-lab-CP-GW1';   <-- another command to find the device id by device name
                  id                  
--------------------------------------
 c25acab5-9d15-4e6c-9cab-a33049534a72

indeni=> delete from alert where device_id = 'c25acab5-9d15-4e6c-9cab-a33049534a72';   <-- delete ALL alerts for this device (using YOUR device id, not this one :)
indeni=> select alert_id, headline from alert where device_id = 'c25acab5-9d15-4e6c-9cab-a33049534a72';   <-- list alerts for a given device
 alert_id |                headline                
----------+----------------------------------------
      207 | Clock set incorrectly
      228 | Device not responding
...
indeni=> delete from alert where alert_id = 207;   <-- delete an alert with a specific id
indeni=> \q    <-- quit the psql shell

# A few other useful commands:
indeni=> \dt+                          <-- list the tables in the db
                                      List of relations
 Schema |                  Name                  | Type  | Owner  |    Size    | Description 
--------+----------------------------------------+-------+--------+------------+-------------
 public | alert                                  | table | indeni | 72 kB      | 
...
 public | device                                 | table | indeni | 16 kB      | 
...
indeni=> \d+ device      <-- list the columns in the "device" table

...