Indeni Workflow Tutorial

Owned by Ido Radai (Unlicensed)
Last updated: Jun 11, 2020 by Tomer Vainshtok (Unlicensed)
17 min read

Introduction

Using Indeni-Workflow, the IKE will have the power to:

  • Write workflow in an easy way

  • Debug workflow

  • Debug workflow block

  • Run workflow on a real device

  • Run workflow on mock data

  • View workflow in a graphic way

Prepare your environment 

1. install indeni-parser

indeni-workflow uses indeni-parser methods to parse raw data into objects (parse_data_as_list etc..)

use this link to install indeni-parser

2. Install indeni-workflow

Using the terminal, install the latest version of indeni-parser

pip3 install "indeni_workflow==0.0.0.*" --extra-index-url https://indeni.jfrog.io/indeni/api/pypi/indeni-pypi-develop/simple

In Ubuntu systems version 18.X (other Debian based systems could be affected) above command don’t work due the blackout in supporting TLS 1.0 and 1.1 in pip which happened 1-2 years ago. In this cases pip must be updated before to install indeni-parser

curl https://bootstrap.pypa.io/get-pip.py | sudo python3

in order to uninstall indeni-workflow use 

pip3 uninstall indeni-workflow

3. Install PyCharm

use this link

4. Define Project Interpreter

use this link

Write your workflow

Files Architecture

Every workflow directory should contain below files, under automation/workflows/<workflow_name>/<vendor_name>/

  • <workflow_name> should be the rulename

  • vendor_name

    • checkpoint

    • paloaltonetworks

Note: See cross_vendor_next_hop_router_inaccessible in https://bitbucket.org/indeni/indeni-knowledge/src/IKP-3685-ansible-to-slimsible-ate-migration/

File

Description

Details

Example

File

Description

Details

Example

<workflow_name>_workflow.yaml
Mandatory

Contains the workflow structure

The workflow contains the following fields

  • id: textual identification string
    convention: <workflow_name>_<vendor_name>

  • start_block: the id of the block that starts the workflow

  • blocks: list of all available blocks in the workflow

 

Each block is responsible to route to the next block.

The following fields are mandatory in each block:

  • type: define the block type

  • name: block friendly name (will be shown in the UI)

According to the type of the block, more fields are required.

 

<workflow_name>_parser.py
Mandatory

Contains supporting parser methods for <workflow_name>_workflow.yaml

 

 

<workflow_name>_mock_data.py
Mandatory

Contains mappings of parser input to expected output

Eg. GET_CPU_UTILIZATION_DATA_1 = ('cpu: usage 25%', 25.0)

The name should be uppercase, beginning with the name of the parser it tests. Followed by TUPLE_n where n is the appropriate number (1 for first, etc.) See the file at the bottom of this page for more examples, and an example how how to write multi-line strings in Python

 

<workflow_name>_parser_test.py
Mandatory

Contains:

  • Parsing tests

  • Block unit tests

  • Workflow unit tests

This file should have one function named test_<parser_function_name> per parser function. Each test may have multiple asserts within it. An example is available towards the bottom of this page.

 

<workflow_name>_workflow_test.yaml

Contains information required to run WIT (workflow integration tool) for a particular workflow.

The file should define the workflow name, the tags of the device to simulate, and a list of “flows”. Each flow is an execution path through the workflow that terminates in a particular conclusion.

*.textfsm
Optional

Template files that workflow_name.py require

 

 

__init__.py
Optional

Pointing to a textfsm file from py file will be in a relative way

 

 

 

workflow_catalog.yaml

 

Block types

Type

Description

Required fields

Example

Type

Description

Required fields

Example

1

device_tags

get the device_tags data as dict[str,str]

  • register_to: name of the result parameter

  • go_to: name of next block_id to go

 

2

device_task

run ssh/https command on the device and parse it

  • runner:

    • type: remote type (SSH/HTTPS)

    • command: remote command (dynamic)

  • parser:

    • args: more args to pass to the parsing name

    • method: the parsing method name (located in workflow_name.py

  • register_to: name of the result parameter

  • go_to: name of next block_id to go

 

3

if

go to next block based on condition

  • condition: the boolean condition to check. the condition can use any python statement with the collected args. unlike dynamic strings, the condition arg does not need to be wrapped in {{}}.

  • then_go_to: name of next block_id to go in case the condition is True

  • else_go_to: name of next block_id to go in case the condition is False

 

4

logic

run a python logic on args and return result

  • args: more args to pass to the parsing name

  • method: the parsing method name (located in workflow_name.py

  • register_to: name of the result parameter

  • go_to: name of next block_id to go

 

5

generate_panos_key

generate a panos api-key

  • register_to: name of the result parameter

  • go_to: name of next block_id to go

 

6

conclusion

the conclusion. after the conclusion block, the workflow should end

  • triage_conclusion: the conclusion text (dynamic)

  • triage_remediation_steps: the remediation steps (dynamic)

 

7

issue_items

Get the issue items as a list of strings

  • register_to: name of the result parameter

  • go_to: name of next block_id to go

 

8

foreach

Go over each issue item

  • issue_items_arg: argument to pass to each loop step

  • register_item_to: name of the result parameter

  • start_block: name of next block_id to go

  • blocks : list of the internal blocks

9

ping (not yet merged)

Ping a target server, returns bool (True/False) for success/failure.

Note: ping is from the Indeni server to the target server.

  • server: IP or address of server to ping

  • register_to: name of the result parameter

  • go_to: name of next block_id to go to

 

10

port_check (not yet merged)

Probe a server’s port and get the status. Returns one of 'open', 'closed', 'filtered', 'unfiltered', 'open|filtered', 'closed|filtered'

Note: probe is from the Indeni server to the target server.

  • server: IP or address of server to probe

  • port: Port of server to check

  • register_to: name of the result parameter

  • go_to: name of next block_id to go to

 

11

loop

Perform actions in a loop for each item in an iterable

  • start_block: The initial block that starts the iteration

  • go_to: Name of next block_id to go to after all iterations are over

  • blocks: The list of blocks that that performs actions within the loop

  • iterable: An iterable variable, which the loop will iterate over

  • register_iterable_to: Name of the iterable-item

  • register_loop_results_to: The map(dictionary) containing the results.

12

end_loop

Marks the end of a loop block iteration and registers the final result

  • iteration_result: A python code that will be evaluated and saved as the iteration's result

Args And Dynamic Text

Currently, args must have same name into workflow as is defined as argument of the method defined (parser). Example:

Block definition:

Parser definition:

 

There are string fields in the yaml, where the args can be passed in {{}}, and the workflow will know to replace the expression with the value.

The current dynamic fields are:

  • device_task.runner.command

  • conclusion.triage_conclusion

  • conclusion.triage_remediation_steps

examples:

  • The device has {{len(num_processors)}} cores

  • This device was rebooted due a operation identified on logs: {{restart_entries}}

Testing your workflow

WorkflowTestTool have 2 main methods to run workflows:

  • run_workflow - run a full workflow:
    Arguments:

    • workflow_path- the workflow path

    • device_data - the device IP and credentials

    • issue_items - a set of the issue item names. (Optional)

  • run_workflow_block

    • run a specific workflow block:

    Arguments:

    • workflow_path - the workflow path

    • block_id - the block to run

    • device_data - the device IP and credentials

    • scope - the current arg scope

    • run_single_block - should the workflow stop after finishing running the block

    • issue_item - a set of the issue item names. (Optional)

 

Using WorkflowTestTool you can run several tests:

 

Real device

Mock data

 

Real device

Mock data

Workflow testing

  1. Create DeviceCredentials (Can be SshCredentials/HttpsCredentials)

  2. Create DeviceData and pass IP, device_tags and the created DeviceCredentials

  3. Run the relevant workflow using WorkflowTestTool.run_workflow

  4. The method returned a list of block records that you can print.

for example:

 

  1. Create MockCredentials while passing:

    1. command_to_response: a dict[str,str] of command to respond

  2. Create DeviceData and pass IP, device_tags and the created DeviceCredentials

  3. Run the relevant workflow using WorkflowTestTool.run_workflow

  4. The method returned a list of block records that you can print.

  5.  

Block testing

Create DeviceCredentials (Can be SshCredentials/HttpsCredentials)

  1. Create DeviceData and pass IP, device_tags and the created DeviceCredentials

  2. Create a scope with the current args as dict[str,str]

  3. Run the relevant workflow using WorkflowTestTool.run_workflow_block.

  4. The last argument to pass to the method is run_single_block. should the workflow run only the single block or continue to the next one?

  5. The method returned a list of block records that you can print.

For example:

 

For example:

Enabled debug logs

By default, the WorkflowTestTool shows only Info logs. in case debug logs are wanted, run

Draw workflow

Using the dot language, we can draw a graphical view of the workflow. This should be used as part of the PR, to show that the workflow matches the requirements:

Call WorkflowTestTool.draw_workflow (requires internet connection). Example:

 

Guidelines for PR submission

  1. Required files:

    1. <workflow_name>_workflow.yaml

    2. <workflow_name>_parser.py

    3. <workflow_name>_mock_data.py

    4. <workflow_name>_parser_test.py

    5. <workflow_name>_workflow_test.py

  2. ATE folder name should be the same as the rule name

  3. Add a link to the server Knowledge Explorer workflow

  4. Capitals in first letter of name

  5. Capitals used with acronyms (e.g DNS and not dns)

  6. Fields that are being used by the UI must give meaningful info and should be super descriptive (short as possible, but descriptive). Some comments

    1. If the conclusion is the the problem is now resolved, leave the triage_remediation_stepsempty in the following manner:

triage_remediation_steps: ””

If the field is empty without a empty string defined in previous example the WorkflowTestTool crash when run

6. Do not state “N/A” in triage_remediation_steps or triage_conclusion

UI fields are:

  1. triage_remediation_steps

  2. triage_conclusion

  3. name

  4. Tests

 

An example of a <workflow_name>_parser_test.py

An partial example of <workflow_name>_mock_data.py