AWK Helper Functions
In indeni.hawk, a series of functions are defined. They are simply helper functions that can be used in .ind scripts.
String manipulation functions
trim(string)
Remove whitespace from string. Returns modified value.ltrim(string)
Remove whitespace at the beginning of string. Returns modified value.rtrim(string)
Remove whitespace at the end of string. Returns modified value.cleanJsonValue(string)
Remove enclosing quotes and comma around string. Returns modified value.
Array manipulation functions
arraylen(array)
Returns the size of an arrayjoin(array, separator)
Appends all the elements of array by putting the separator between each two elements. Returns a string of joined array elements.
Date manipulation functions
date(year, month, day)
Returns the number of seconds since epoch (01/01/1970) of the date described by the year, month (1-12) and day provided. All timestamp representations in the collector are in number of seconds since epoch.datetime(year, month, day, hour, minute, second)
Returns the number of seconds since epoch (01/01/1970) of the date-time described by the year, month (1-12),day,hour, minute and second provided. All timestamp representations in the collector are in number of seconds since epoch.now()
Returns the number of seconds since epoch (01/01/1970) for "now". This is according to the indeni Collector server's clock, not the device we are connecting to.parseMonthThreeLetter(mmm)
Rarses mmm as jan/feb/mar/apr/etc. and returns month number (1, 2, 3... 12). Not case sensitive.
Table parsing functions
Let's say you have an input table that looks like this:
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 1449 root 4 76 0 214M 45968K select 0 269:40 99.17% flowd_octeon 1471 root 1 76 0 12460K 5004K select 0 1:04 0.00% license-check 1438 root 1 76 0 28088K 9720K select 0 0:38 7.10% mib2d 1440 root 1 76 0 20300K 7808K select 0 0:36 0.00% l2ald
How do you parse this easily? With this code:
/(PID|USERNAME|COMMAND)/ {
# Parse the line into a column array. The second parameter is the separator between column names.
# The last parameter is the array to save the data into.
getColumns(trim($0), "[ \t]+", columns)
}
/root.*%/ {
# Use getColData to parse out the data for the specific column from the current line. The current line will be
# split according to the same separator we've passed in the getColumns function (it's stored in the "columns" variable).
# If the column cannot be found, the result of getColData is null (not "null").
cpu = getColData(trim($0), columns, "WCPU")
memory = getColData(trim($0), columns, "RES")
pid = getColData(trim($0), columns, "PID")
processname = getColData(trim($0), columns, "COMMAND")
if (getColData(trim($0), columns, "FOOBAR") == null) {
# Handle column not found in original list of columns
}
}
Collector output functions
See Metrics for more information about the metric types. This will help you when choosing type for new metrics.
writeTag(name, value)
write a tag to be associated with a device. Tags can only be written in an interrogation script. These tags are used by monitoring scripts (as well as some interrogation scripts) in the "requires" section at the top of the script (see /wiki/spaces/IKP/pages/54198315).
writeDoubleMetric(imName, tags, dsType, value, isLiveConfig, [displayName], [displayType], [identityTags])
writes a DOUBLE metric with the following information:
imName - the name of the metric
tags - tags to add to the metric. You can just write "null" (without quotes) to keep this empty. If you want to include tags (and read the "Tags" session in metrics VERY carefully), you need to initialize a tag array prior to making the call to writeDoubleMetric, like so:
For this input: Water level tank #1: 1.7 L Water level tank #2: 2.8 L This script: /Water level/ { id = $4 gsub(/#/,"", id) waterLevelTags["id"] = id writeDoubleMetric("water-level", waterLevelTags, "gauge", 60, $(NF-1)) }TIP: All variables in awk are global. DO NOT reuse the same variable names for tags for different metrics. In the example above, if we were to use waterLevelTags with another metric it would copy the "id" element with it. Only re-use the tag variables if indeed the tags must be the same.
dsType - Either "counter" or "gauge". Most often, it's gauge. A gauge metric just has a value: it can be any double value. Often it's a boolean, either 0.0 or 1.0. But, it could be, e.g., current CPU or memory usage: any sort of 'point-in-time' value that can go 'up or down'. "counter" is for a value that only increases: use it for for metrics which count how many times something has happened (for example, RX packets on a network interface). counter can never go down in value (unless it reaches its maximum value and wraps back to 0).
- value - a number (like 17.1 or 155513532423.1015) for the metric
- isLiveConfig – a string that should be either "true" or "false", denoting whether or not you want the metric to go to liveConfig. The "live config" (AKA "actual config") represents the most current data Indeni has pulled from a device (e.g., current memory usage, currently configured DNS servers). This data gets displayed in the Indeni GUI. Note that displayName is a mandatory argument when isLiveConfig is "true"
displayName - A string to categorize the metric. Shown to the user in the live config. Optional when isLiveConfig is "false" and mandatory when isLiveConfig is "true"
displayType - A string identifying the units to display, as documented in Reserved Metric/Tag Names. E.g., "percentage", "kilobytes", or "duration". May add units symbols to the live config display (e.g., "%")
identityTags - A string which corresponds to a key in the metric tag(s) array (see example below). If you pass no metric tags (i.e., the "tags" parameter is "null"), this string should be empty (""). You can pass multiple identityTags in the same string parameter, each separated by "|" (see notes below).
Example
memoryTag["name"] = "Free Memory" writeDoubleMetric("memory-free-kbytes", memoryTag, "gauge", free, "true", "Memory Usage", "kilobytes", "name") memoryTag["name"] = "Total Memory" writeDoubleMetric("memory-total-kbytes", memoryTag, "gauge", total, "true", "Memory Usage", "kilobytes", "name") memoryTag["name"] = "Used Memory" writeDoubleMetric("memory-used-kbytes", memoryTag, "gauge", used, "true", "Memory Usage", "kilobytes", "name") memoryTag["name"] = "Percent Memory Usage" writeDoubleMetric("memory-usage", memoryTag, "gauge", percentMemoryUsage, "true", "Memory Usage", "percentage", "name")The above code results in this live config:
In this way, you can create a "category" ("Memory Usage" above) and group related metrics. A few notes:- If you don't use any metricTag or identityTag, the live config data will display under the "Overview" category.
- Some rules require a metricTag/identityTag pair. They essentially require that you add live config for display. E.g., in the rule "/templatebased/crossvendor/cross_vendor_high_memory_usage.scala", there is an applicableMetricTag = "name" passed to the constructor. This gives you a clue that you need to include a name metricTag/identityTag (exactly as per the example above); if you don't, you won't get an alert!
- You can add any data to the live config – you don't need a rule or any existing, known metric. E.g.,metricTag["mylabel"] = "First Label" writeDoubleMetric("foo-metric", metricTag, "gauge", "77", "true", "Special Category", "percentage", "mylabel")This will display just fine and only in the live config.
- You can also pass multiple identityTags in the identityTags string parameter, each separated by "|": E.g.,pstags["name"] = pid pstags["process-name"] = processname writeDoubleMetric("process-cpu", pstags, "gauge", cpuUsage, "true", "Processes (CPU)", "percentage", "name|process-name")This will display like: "name-<PID> process-name-<some process name> <cpuUsage>"
writeComplexMetricString(imName, tags, value, isLiveConfig, [displayName])
similar to writeDoubleMetric, except that the value is a string. For example, for the "vendor" metric described in the list of reserved metrics, you would use this function:
isLiveConfig – A string that should be either "true" or "false"
writeComplexMetricString("vendor", null, "acme", "false")
displayName – Optional if isLiveConfig is "false" and mandatory if isLiveConfig is "true". See displayName in example for doubleMetric live config above.writeComplexMetricObjectArray(imName, tags, arrayValue, isLiveConfig, [displayName])
similar to writeComplexMetricString, except that the value is a json array of objects.isLiveConfig – A string that should be either "true" or "false"
displayName – Optional if isLiveConfig is "false" and mandatory if isLiveConfig is "true". See displayName in example for doubleMetric live config above.leverages AWK's capabilities of building a key-value map to make it easier to describe a complex object array. Here is an example of how to use it:
For this input: User Uid Gid Home Dir. Shell Real Name admin 0 0 /home/admin /etc/cli.sh n/a indeni 0 100 /home/indeni /bin/bash Indeni monitor 102 100 /home/monitor /etc/cli.sh Monitor We can use this ind script: /home/ { iuser++ users[iuser, "username"] = $1 users[iuser, "uid"] = $2 users[iuser, "gid"] = $3 } END { # We normally dump the metric itself at the end of the script, because we spend the script collecting the items # and then we write it once we have it all collected. writeComplexMetricObjectArray("users", null, users, "false") }writeDebug(string)
debug(string)
Writes the string to the debug output for the AWK parser. The debug output is only visible when the command runner, or the indeni-collector (in a live system) is run under debug mode. For example, you added this line to an ind script:
debug("I am a debug string")Then this would result in nothing:
bash command-runner.sh full-command --ssh indeni,indeni test.ind 10.3.3.72 | grep "I am"
Whereas this (using the --verbose switch) would would print the debug string:
bash command-runner.sh full-command --verbose --ssh indeni,indeni test.ind 10.3.3.72 | grep "I am" 2016-10-10 07:22:58,309 26823 DEBUG - I am a debug string
Flow related statements
Statements that can be used to control the execution flow of your script.
- next
The next statement forces awk to immediately stop processing the current record and go on to the next record. This means that no further rules are executed for the current record, and the rest of the current rule’s action isn’t executed. Note that with the 'next' statement the execution flow is altered. You simply stop the process and go immediately to the processing of the next available line. If you are processing the last line and you use the 'next' statement then you will force tha awk-engine to go directly to the 'END' block.
A nice use case of the "next" statement is to filter your input. You can use a filtering criteria at the beginning of your script, and if the input line is not compliant with your rule, you can easily ignore the line and go asap to the next line using the 'next' statement. For example, let's assume that each line that you want to process has only 3 fields, you can do something like this.
NF != 3 {
#This line doesn't have 3 fields. There is no reason to process it. Ignore and start-over with the next line.
next
}
# .. your code continues here.
# Only the lines with more than 3 fields will reach that point.
- getline
The getline statement is very similar to the "next" statement. The difference is that "getline" will read the next record immediately but it will not alter the flow of control in any way.
Meaning that the rest of the current actions executes with the new input record. (The "next" statement will start the process from the beginning)
Assuming that you have the following input:
My Data ------------- my-username my-password a useless line.. my-email@email.com
and you script have to print only the username and the password lines.
So the logic is to read the 'username' after the line '--------------' and then continue the process.
For example:
/^---------/ {
getline
user = $0
getline
pass = $0
getline
getline
email = $0
}
END {
writeDebug("user is " user)
writeDebug("pass is " pass)
writeDebug("email is " email)
}
Note that the The getline command returns 1 if it finds a record and 0 if it encounters the end of the file. If there is some error in getting a record, such as a file that cannot be opened, then getline returns -1.
- break
The break statement jumps out of the innermost for, while, or do loop that encloses it. Some times you don't want to continue the processing in
Multi-Step functions
- writeDynamicVariable(dynamicVariableName, sourceVariable)
Create a "dynamic" variable to pass from one step to the next in a /wiki/spaces/IKP/pages/75890692. E.g., writeDynamicVariable("nics", niclist). - dynamic(dynamicVariableName)
Use the dynamic varibale name in your code. This function will actually replace return as result the variable value. E.g., dynamic("nics")