The docker platform, is used in order to create “virtual devices” on top of the OS. Each of the virtual devices (also known as Mock device), is officially named: container inside the docker. Potentially, we can have complete OS as a Mock device. The current usage for indeni, is that each container holds pre-defined commands and their outputs, used for interrogation, and simulating an actual device with it's tags. The pre-defined data is being collected by a recording process, of an actual live device. After this recording is saved to a file, we can use it, in order to create a container inside the docker platform. Once we have this container, we can create the device on the indeni server, as if it was the actual live server it was recorded from.
Obviously, the container or Mock device, is static, and will not change it’s state dynamically as a live machine. However, we have the option to “inject” some new commands and their outputs, for multiple tests.
The purpose of this document, is to have a step by step guide to create a Mock device, and basic operations over it.
Detailed steps:
Machine
Action
Steps
Examples and screenshots:
Machine
Action
Steps
Examples and screenshots:
1
Indeni server
Record device interrogation
Pause all devices:
Login to Indeni server
In Devices page, select all devices
Pause all devices
Start recording: Wait about 2 minutes after all you suspend all devices, before you start recording, in order to record minimum devices. Then run: curl -k -X POST https://localhost:9000/recording/start -H "Content-type: application/json" --data '{"duration":300000000000,"force_interrogation":true,"devices":[],"commands":[],"relative_path":"some_folder_name"}' Note: when recording a new device, do not use the same: “relative_path”.
Add device in UI (interrogate + save)
Once interrogation is completed, wait for a couple of minutes, and then run, in order to stop the recording: curl -k -X POST https://localhost:9000/recording/stop -H "Content-type: application/json"
The recording file will be stored in /usr/share/indeni-collector/logs/recording/<some_folder_name> (from the end of step 1). Important note: make sure the file size of the record is enough! It is suggested to monitor the size of the file increase, until it becomes steady, this way, we can be more certain that all the needed metrics and interrogation were collected. You can use this command for example: # watch -d -n1 "ls -lah /usr/share/indeni-collector/logs/recording/record_test/"
Notes about the files inside this folder: commands-recording-report.yaml- Log file for failed / recording. successful record might not show logs. recording-commands-<device_id>.log - The actual record file. The number in the middle is the device_id. In order to correlate which recording we would like, we can retrieve the device_id, using iktool, as follow: “iktool op device_info_show <device IP we interrogate>.
Delete device from the UI (optional).
Pause all devices:
Start recording: indeni@indeni-server:~$ curl -k -X POST https://localhost:9000/recording/start -H "Content-type: application/json" --data '{"duration":300000000000,"force_interrogation":true,"devices":[],"commands":[],"relative_path":"record_test"}' The request has been accepted for processing, but the processing has not been completed
To stop the record: indeni@indeni-server:~$ curl -k -X POST https://localhost:9000/recording/stop -H "Content-type: application/json" OK If the interrogation stopped or finished already, you may accept the following output: Recoding is not running
Folder contents: indeni@indeni-server:~$ ls -lah /usr/share/indeni-collector/logs/recording/record_test/ total 528K drwxr-xr-x 2 indeni indeni 4.0K May 12 17:15 . drwxr-xr-x 3 indeni indeni 4.0K May 12 17:11 .. rw-r--r-- 1 indeni indeni 212 May 12 17:15 commands-recording-report.yaml rw-r--r-- 1 indeni indeni 454K May 12 17:15 recording-commands-01647531-4c4a-47e9-b583-7d64234b4c9e.log rw-r--r-- 1 indeni indeni 58K May 12 17:11 recording-commands-36c3c43f-0434-4365-af22-4f5eb6876eab.log
Each new Mock device, requires a unique IP. Otherwise, when creating the new device on the indeni server, we will receive the error: {"type":"error","message":"IndeniDataException: Unique key violation"}.
Need to edit the file: /etc/netplan/50-cloud-init.yaml, and add the new IP (using vi / nano). Once done, apply the changes, so the alias IP will be applied: sudo netplan apply
Add new address to networks section, in the UI, with set of credentials. Make sure you have connectivity with Ping to the new address from external server.
Content file of /etc/netplan/50-cloud-init.yaml. Add the new address under the “addresses” section (example from 10.10.10.137): network: {config: disabled}
Copy the relevant record file from Indeni server to the mock server, under the /home/indeni/recording-commandsfolder.
Add new Mock device to the docker platform: sudo docker run -d -e "TZ=Asia/Jerusalem" -p <IP of Mock device>:<SSH external port>:<SSH internal port> -p <IP of Mock device>:<ssl external port>:<ssl internal port> -v /home/indeni/recording-commands:/home/indeni/mock_device/records --name=<mock_device_name> indeni-docker-private.jfrog.io/mock-device:<Mock image version> --records="/home/indeni/mock_device/records/<record file name>" --mode=<dynamic/static> --ssh-port=<ssh internal port> --ssl-port=<ssl internal port> SSL/ SSH external ports - these are the ports which the docker platform listen to on the OS, and communicate over the network, forwarding these services to the container. The default ports are: 8022 for SSH, 8443 for HTTPS - NO NEED TO CHANGE THESE. SSL/SSH internal ports - these are the ports which the docker use in order to communicate internally with each container. Thus, each mock device (=container) requires unique values.
Some notes about the above: Mock image version - could be 2.1.2 or the latest 2.1.3 mode=<dynamic/static> - Dynamic mode will let us add only one command to the device, Static mode will allow us to add multiple outputs to the same command.
Copy the file from indeni server: root@automation:~# scp indeni@10.11.80.23:/usr/share/indeni-collector/logs/recording/record_test/recording-commands-01647531-4c4a-47e9-b583-7d64234b4c9e.log /home/indeni/recording-commands/record.log
Create the new device on the indeni server via API call, from the MOCK server: curl -k -X POST "https://<indeni-server-ip>/api/v1/devices/bulk" -H "accept: application/json" -H "x-access-token: <token>" -H "Content-Type: application/json" -d "[ { \"ip_address\": \"<MOCK_device_ip>\", \"name\": \"<new_device_name>\", \"parameters\": { \"SSH_Port\": \"<ssh_port>\", \"HTTPS_Port\": \"<HTTPS_PORT>\" }, \"labels\": [], \"tags\": {} }]"
Notes about the parameters:
ip_address: IP of the Mock device. name: Name for the device in the UI. SSH_Port: Will be the number corresponding to the “ssh-port” from the previous step (we configured: --ssh-port=2001, which represent the container listening port. On the beginning of the command, we embedded the corresponding port for the server: 8023. So, in our case, we need to put in the ssh_port: 8023. HTTPS_Port: Will be the number corresponding to the “ssh-port” from the previous step (we configured: --ssl-port=3001, which represent the container listening port. On the beginning of the command, we embedded the corresponding port for the server: 8445. So, in our case, we need to put in the HTTPS_port: 8445. <token> - Token value taken from the “Network” tab using “developer tab” on browsers (check the example). Note: the token id is valid for about 12 hours.
How to collect the token id: Get into the indeni server web page, and enter the “developer tab”. Inside, click the “Network” tab. Click on one of the http requests, and search for the parameter: “x-access-token”. Example:
There are 2 options to add / edit a command, using CURL, and using scp. As a best practice, use scp command below. Currently, all our Mock devices are running with Mock version 2.1.3.
Using CURL command:
To add SSH command:curl -k -X POST https://<ip:ssl external port>/mock/management -d '{"command": "<command>","action": "ADD","base64EncodeOutput": "d2hhdCBkbyB5b3Ugd2FudD8gOi0o","protocol": "ssh"}' -H "Accept: application/json"
To add HTTPS command: curl -k -X POST https://10.10.10.137:8446/mock/management -d '{"command":{Full API command}&key={API-Key}","action": "ADD","base64EncodeOutput": "","protocol": "https"}' -H "Accept: application/json"
Using SCP command:
For a long output, which is common case for HTTPS, we will need to use the following option, ONLY using mock image 2.1.3 and above: scp -P <external SSH port> <file name which include the output in base64 format> indeni@<Mock IP>:'{"command": "<API command>","action": "ADD","protocol": "https", "contentType":"application/xml"}'
You can use scp in order to add SSH command as well, just change the “protocol”, to SSH: scp -P <external SSH port> <file name which include the output in base64 format> indeni@<Mock IP>:'{"command": "<SSH command>","action": "ADD","protocol": "ssh"}'
Example for HTTPS command: curl -k -X POST https://10.10.10.137:8446/mock/management -d '{"command": "/api/?type=op&cmd=<show><interface>all</interface></show>&key=LUFRPT0yMWgrSGRMa09MNWVnc091NXplS2FpWTZkeVk9czdNTk5SeENtK0N3VFF4SEloWFFzT2RBdXcydFB3SXh2dzQvbkRHZ29tU3FhdTlwdGo4aWxJdlZZYjhPenhGKw==","action": "ADD","base64EncodeOutput": "PHJlc3BvbnNlIHN0YXR1cz0ic3VjY2VzcyI+Cgk8cmVzdWx0PgoJCTxpZm5ldD4KCQkJPGVudHJ5PgoJCQkJPG5hbWU+ZXRoZXJuZXQxLzM8L25hbWU+CgkJCQk8em9uZSAvPgoJCQkJPGZ3ZD5oYTwvZndkPgoJCQkJPHZzeXM+MTwvdnN5cz4KCQkJCTxkeW4tYWRkciAvPgoJCQkJPGFkZHI2IC8+CgkJCQk8dGFnPjA8L3RhZz4KCQkJCTxpcD4xMC4yNTUuMjU1LjIvMzA8L2lwPgoJCQkJPGlkPjE4PC9pZD4KCQkJCTxhZGRyIC8+CgkJCTwvZW50cnk+CgkJPC9pZm5ldD4KCQk8aHc+CgkJCTxlbnRyeT4KCQkJCTxuYW1lPmV0aGVybmV0MS8zPC9uYW1lPgoJCQkJPGR1cGxleD5mdWxsPC9kdXBsZXg+CgkJCQk8dHlwZT4wPC90eXBlPgoJCQkJPHN0YXRlPnVwPC9zdGF0ZT4KCQkJCTxzdD4xMDAwL2Z1bGwvdXA8L3N0PgoJCQkJPG1hYz4wMDo1MDo1NjphYzo4ZjpmNjwvbWFjPgoJCQkJPG1vZGU+KGF1dG9uZWcpPC9tb2RlPgoJCQkJPHNwZWVkPjEwMDA8L3NwZWVkPgoJCQkJPGlkPjE4PC9pZD4KCQkJPC9lbnRyeT4KCQk8L2h3PgoJPC9yZXN1bHQ+CjwvcmVzcG9uc2U+","protocol": "https"}' -H "Accept: application/json"
Example for long output adding command: scp -P 8024 test.txt indeni@10.10.10.137:'{"command": "/api/test","action": "ADD","protocol": "https", "contentType":"application/xml"}'
When running a command-runner against a Mock device, you need to specify the relevant external port being used. For SSH, this is done using the flag: “--ssh-port” in the command-runner syntax. For HTTPS, you will need to use --http-port and --basic-authentication --api-key flags.
On the indeni server, record a device interrogation: - Suspend all devices on the UI. - Run on the CLI: curl -k -X POST https://localhost:9000/recording/start -H "Content-type: application/json" --data '{"duration":300000000000,"force_interrogation":true,"devices":[],"commands":[],"relative_path":"some_folder_name"}' - Start interrogation with the desired device, on the UI. - Once done, stop the recording, with the CLI command: curl -k -X POST https://localhost:9000/recording/stop -H "Content-type: application/json"
On the MOCK server: - Copy the record file to the folder: /home/indeni/recording-commands/ - Create a new container using the recorded file: sudo docker run -d -e "TZ=Asia/Jerusalem" -p 10.10.10.138:<SSH external port>:<SSH internal port> -p 10.10.10.138:<ssl external port>:<ssl internal port> -v /home/indeni/recording-commands:/home/indeni/mock_device/records --name=<mock_device_name> indeni-docker-private.jfrog.io/mock-device:2.1.2 --records="/home/indeni/mock_device/records/<record file name>" --mode=dynamic --ssh-port=<ssh internal port> --ssl-port=<ssl internal port> - Create the new device on the Indeni server, using API command from the Mock server: curl -k -X POST "https://<indeni-server-ip>/api/v1/devices/bulk" -H "accept: application/json" -H "x-access-token: <token>" -H "Content-Type: application/json" -d "[ { \"ip_address\": \"<MOCK_device_ip>\", \"name\": \"<new_device_name>\", \"parameters\": { \"SSH_Port\": \"<ssh external port>\", \"HTTPS_Port\": \"<https external port>\" }, \"labels\": [], \"tags\": {} }]"
List of current Mock devices for knowledge team:
Name
IP
API command to build on the Indeni server
Name
IP
API command to build on the Indeni server
The actual Mock server, which holds all the Mock devices
