Zabbix

Configure Zabbix Docker

Configure Zabbix Server to point to zabbix-agent container

Figure 1

607-370-max.png

Figure 2

961-558-max.png

Configure Zabbix Server Timezone

Edit /opt/zabbix-docker/.env_web uncomment the following line:

#PHP_TZ=Europe/Riga

Navigate to http://php.net/manual/en/timezones.php locate your local timezone and set PHP_TZ to it, for example for American Eastern you would set it to the following:

PHP_TZ=America/New_York

Configure Zabbix Server Server Name

Edit /opt/zabbix-docker/.env_web and set the following line:

ZBX_SERVER_NAME=Composed installation

to a server name that you like (Example: ZBX_*SERVER_*NAME=Widgets, Inc)

Configure Zabbix Auto Discover

Figure 3

405-160-max.png

Figure 4

548-364-max.png

Enable Zabbix Notifications

Figure 5

447-175-max.png

Figure 6

1024-230.png

Enable Zabbix FreeNAS Monitoring

Figure 7

1024-532.png

Figure 8

598-365-max.png

Figure 9

680-527-max.png
/usr/local/share/snmp/mibs/FREENAS-MIB.txt

/opt/zabbix-docker/zbx_env/var/lib/zabbix/mibs

cd /opt/zabbix-docker && docker-compose down
cd /opt/zabbix-docker && docker-compose up -d

Figure 10

636-384-max.png

Figure 11

1024-190.png
SNMP Intefaces discovery

https://share.zabbix.com/official-templates/snmp-devices/snmp-interfaces-discovery

SNMP Interfaces discovery

https://share.zabbix.com/official-templates/snmp-devices/snmp-processors-discovery

SNMP Generic

https://share.zabbix.com/official-templates/snmp-devices/snmp-generic

FreeNAS 11 SNMP

https://share.zabbix.com/storage-devices/freenas/freenas-11

Figure 12

736-582-max.png

Figure 13

982-698-max.png

Figure 14

773-325-max.png

Figure 15

1024-28.png

Enable Zabbix VMware Monitoring

Original Guide URL:

https://bestmonitoringtools.com/vmware-monitoring-with-zabbix-esxi-vcenter-vm-vsphere/

ZBX_STARTVMWARECOLLECTORS=3
ZBX_VMWAREFREQUENCY=60
ZBX_VMWAREPERFFREQUENCY=60
ZBX_VMWARECACHESIZE=128M
ZBX_VMWARETIMEOUT=120

Restart the Zabbix docker stack:

cd /opt/zabbix-docker && docker-compose down
cd /opt/zabbix-docker && docker-compose up -d

Figure 16

981-543-max.png

Figure 17

782-265-max.png
{$VMWARE.PASSWORD}
{$VMWARE.URL}
{$VMWARE.USERNAME}

Figure 18

881-643-max.png

Figure 19

1024-472.png

Figure 20

963-84-max.png

Figure 21

1024-435.png

Figure 22

1024-504.png

Figure 23

1024-191.png
Free space is less than 5% on datastore "{#DATASTORE}"

{Template VM VMware:vmware.datastore.size[{$VMWARE.URL},{#DATASTORE},pfree].max(15m)}<5

Figure 24

783-690-max.png

 

 

Install Zabbix Agent Ubuntu

Ubuntu 22.04

 wget https://repo.zabbix.com/zabbix/5.0/ubuntu/pool/main/z/zabbix-release/zabbix-release_5.0-2+ubuntu22.04_all.deb
dpkg -i zabbix-release_5.0-2+ubuntu22.04_all.deb
apt-get update
apt-get install zabbix-agent

Ubuntu 20.04

wget https://repo.zabbix.com/zabbix/5.0/ubuntu/pool/main/z/zabbix-release/zabbix-release_5.0-1%2Bfocal_all.deb
dpkg -i zabbix-release_5.0-1+focal_all.deb
apt-get update
apt-get install zabbix-agent

Ubuntu 18.04

wget https://repo.zabbix.com/zabbix/5.0/ubuntu/pool/main/z/zabbix-release/zabbix-release_5.0-1%2Bbionic_all.deb
dpkg -i zabbix-release_5.0-1+bionic_all.deb
sudo apt update
apt-get install zabbix-agent

Configure Zabbix Agent for Passive Checks

Edit the zabbix configuration file:

sudo vi /etc/zabbix/zabbix_agentd.conf

Set the following parameter according to your environment:

Server=192.168.100.51

save the file and close

Restart and enable zabbix agent service

sudo systemctl restart zabbix-agent
sudo systemctl enable zabbix-agent

Start/stop/check status of zabbix agent service:

sudo systemctl start zabbix-agent
sudo systemctl stop zabbix-agent 
sudo systemctl status zabbix-agent

Configure Zabbix Agent for Active Checks

Edit the zabbix configuration file:

sudo vi /etc/zabbix/zabbix_agentd.conf

Set the following parameters according to your environment:

ServerActive=192.168.100.51
Hostname=zabbix.fosstechnix.local

save the file and close

Restart and enable zabbix agent service

sudo systemctl restart zabbix-agent
sudo systemctl enable zabbix-agent

Start/stop/check status of zabbix agent service:

sudo systemctl start zabbix-agent
sudo systemctl stop zabbix-agent 
sudo systemctl status zabbix-agent

Configure TLS with PSK

Edit the zabbix configuration file:

sudo nano /etc/zabbix/zabbix_agentd.conf

Set the following parameters where unique_psk_identity is some unique value for the agent you are configuring. I can be anything as long as it's unique for this host:

TLSConnect=psk
TLSAccept=psk
TLSPSKIdentity=unique_psk_identity
TLSPSKFile=/etc/zabbix/zabbix_agentd.psk

save the file and close

Run the following command to generate a psk file using the path and name specified in the TLSPSKFile parameter from above:

openssl rand -hex 32 | sudo tee /etc/zabbix/zabbix_agentd.psk

Note the generated random string as you will need it to configure the agent on the Zabbix server under the Configuration --> Hosts --> Host -->Encryption tab.

Start/stop/check status of zabbix agent service:

sudo systemctl start zabbix-agent
sudo systemctl stop zabbix-agent 
sudo systemctl status zabbix-agent

 

Zammad Monitoring with Zabbix

Zammad Configuration

Click on Settings --> Users --> New User and create a new user for Zabbix by filling out FIRSTNAME, LASTNAME, EMAIL, PASSWORD and setting the PERMISSIONS to Agent (Figure 1):

Figure 1

image-1609102694405.png

Once the user is created, locate the user, click the 3 dots under ACTIONS column and click on View from user's perspective option from the corresponding menu (Figure 2):

Figure 2

image-1609102424908.png

Click the user's initials on the bottom left of the screen and click on the Profile option from the corresponding menu (Figure 3):

Figure 3

image-1609102893415.png

Under the Profile section, click Token Access --> Create (Figure 4):

Figure 4

image-1609103031468.png

In the Add a Personal Access Token window, in the name field enter Zabbix, check AGENT under TICKET and click the Create button (Figure 5):

Figure 5

image-1609103192887.png

On the resulting Your New Personal Access Token window, ensure you copy the token and click the OK, I've copied my token button (Figure 6):

Figure 6

image-1609103319054.png

In the Token Access page, under the Personal Access Tokens you should see the following entry (Figure 7):

Figure 7

image-1609103581508.png

Zabbix Configuration

Click on Administration --> General -->GUI --> Macros (Figure 8):

Figure 8

image-1609103782860.png

Under the Macros page, click on the Add link to add a new Macro (Figure 9):

Figure 9

image-1609153549502.png

In the new Macro entry enter {$ZABBIX.URL} in the Macro field, enter the URL to the Zabbix serverand in the Value field and click the Update button below  (Figure 10):

Figure 10

image-1609153726997.png

Click on Administration --> Media Types --> Zammad. In the zammad_access_token field enter the Personal Access Token you created earlier the Zammad, in the zammad_url field enter the the URL to your Zammad installation, in the zammad_customer field enter the e-mail address of the Zammad user you created earlier and click the Update button (Figure 11):

Figure 11

image-1609156308393.png

Click on Administration --> Users --> Zabbix-User (where Zabbix-User is a user already created in Zabbix). Once in the User page, click on Media and then click the Add button (Figure 12):

Figure 12

image-1609156851653.png

In the Media window, in the Type field, select Zammad, in the Send to field enter AnyText (this field is not used but it's required to be filled in) and click the Add button (Figure 13):

Figure 13

image-1609156997108.png

Back in Users --> Media , you should see the newly added Zammad media type. Ensure you click the Update button to save your changes (Figure 14):

Figure 14

image-1609157186553.png

 

Docker Container Monitoring With Zabbix

Credit Dmitry Lambert Original URL: https://blog.zabbix.com/docker-container-monitoring-with-zabbix/20175/

Importing the official Docker template

image-1655138759513.png


Importing the Docker by Zabbix agent 2 template

Since we will be using the official Docker by Zabbix agent 2 template, first, we need to make sure that the template is actually available in our Zabbix instance. The template is available for Zabbix versions 5.0, 5.4, and 6.0. If you cannot find this template under Configuration – Templates, chances are that you haven’t imported it into your environment after upgrading Zabbix to one of the aforementioned versions. Remember that Zabbix does not modify or import any templates during the upgrade process, so we will have to import the template manually. If that is so, simply download the template and import it into your Zabbix instance by using the Import button in the Configuration – Templates section.

Installing and configuring Zabbix agent 2

Before we get started with configuring our host, we first have to install Zabbix agent 2 and configure it according to the template guidelines. Follow the steps in the download section of the Zabbix website and install the zabbix-agent2 package. Feel free to use any other agent deployment methods if you want to (like compiling the agent from the source files).

Plugin specific Zabbix agent 2 configuration

Zabbix agent 2 provides plugin-specific configuration parameters. Mostly these are optional parameters related to a specific plugin. You can find the full list of plugin-specific configuration parameters in the Zabbix documentation. In the newer versions of Zabbix agent 2, the plugin-specific parameters are defined in separate plugin configuration files, located in /etc/zabbix/zabbix_agent2.d/plugins.d/, while in older versions, they are defined directly in the zabbix_agent2.conf file.

Before we move on to Zabbix frontend, I would like to point your attention to the Docker socket file permission – the zabbix user needs to have access to the Docker socket file. The zabbix user should be added to the docker group to resolve the following error messages.

[Docker] cannot fetch data: Get http://1.28/info: dial unix /var/run/docker.sock: connect: permission denied
ZBX_NOTSUPPORTED: Cannot fetch data.

You can add the zabbix user to the Docker group by executing the following command:

usermod -aG docker zabbix

Configuring the docker host

image-1655138784079.png


Configuring the host representing our Docker environment

After importing the template, we have to create a host which will represent our Docker instance. Give the host a name and assign it to a Host group – I will assign it to the Linux servers host group. Assign the Docker by Zabbix agent 2 template to the host. Since the template uses Zabbix agent 2 to collect the metrics, we also have to add an agent interface on this host. The address of the interface should point to the machine running your Docker containers. Finish up the host configuration by clicking the Add button.

Docker by Zabbix agent 2 template

image-1655138802822.png


Regular docker template items

The template contains a set of regular items for the general Docker instance metrics, such as the number of available images, Docker architecture information, the total number of containers, and more.

image-1655138816577.png


Docker template Low-level discovery rules

On top of that, the template also gathers container and image-specific information by using low-level discovery rules.

Once Zabbix discovers your containers and images, these low-level discovery rules will then be used to create items, triggers, and graphs from prototypes for each of your containers and images. This way, we can monitor container or image-specific metrics, such as container memory, network information, container status, and more.

image-1655138831571.png


Docker templates Low-level discovery item prototypes

Verifying the host and template configuration

To verify that the agent and the host are configured correctly, we can use Zabbix get command-line tool and try to poll our agent. If you haven’t installed Zabbix get, do so on your Zabbix server or Zabbix proxy host:

dnf install zabbix-get

Now we can use zabbix-get to verify that our agent can obtain the Docker-related metrics. Execute the following command:

zabbix_get -s docker-host -k docker.info

Use the -s parameter to specify your agent host’s host name or IP address. The -k parameter specifies the item key for which we wish to obtain the metrics by polling the agent with Zabbix get.

zabbix_get -s 192.168.50.141 -k docker.info

{"Id":"SJYT:SATE:7XZE:7GEC:XFUD:KZO5:NYFI:L7M5:4RGO:P2KX:QJFD:TAVY","Containers":2,"ContainersRunning":2,"ContainersPaused":0,"ContainersStopped":0,"Images":2,"Driver":"overlay2","MemoryLimit":true,"SwapLimit":true,"KernelMemory":true,"KernelMemoryTCP":true,"CpuCfsPeriod":true,"CpuCfsQuota":true,"CPUShares":true,"CPUSet":true,"PidsLimit":true,"IPv4Forwarding":true,"BridgeNfIptables":true,"BridgeNfIP6tables":true,"Debug":false,"NFd":39,"OomKillDisable":true,"NGoroutines":43,"LoggingDriver":"json-file","CgroupDriver":"cgroupfs","NEventsListener":0,"KernelVersion":"5.4.17-2136.300.7.el8uek.x86_64","OperatingSystem":"Oracle Linux Server 8.5","OSVersion":"8.5","OSType":"linux","Architecture":"x86_64","IndexServerAddress":"https://index.docker.io/v1/","NCPU":1,"MemTotal":1776848896,"DockerRootDir":"/var/lib/docker","Name":"localhost.localdomain","ExperimentalBuild":false,"ServerVersion":"20.10.14","ClusterStore":"","ClusterAdvertise":"","DefaultRuntime":"runc","LiveRestoreEnabled":false,"InitBinary":"docker-init","SecurityOptions":["name=seccomp,profile=default"],"Warnings":null}

In addition, we can also use the low-level discovery key – docker.containers.discovery[false] to check the result of the low-level discovery.

zabbix_get -s 192.168.50.141 -k docker.containers.discovery[false]

[{"{#ID}":"a1ad32f5ee680937806bba62a1aa37909a8a6663d8d3268db01edb1ac66a49e2","{#NAME}":"/apache-server"},{"{#ID}":"120d59f3c8b416aaeeba50378dee7ae1eb89cb7ffc6cc75afdfedb9bc8cae12e","{#NAME}":"/mysql-server"}]

We can see that Zabbix will discover and start monitoring two containers – apache-server and mysql-server. Any agent low-level discovery rule or item can be checked with Zabbix get.

Docker template in action

image-1655138848644.png


Discovered items on our Docker host

Now that we have configured our agent and host, applied the Docker template, and verified that everything is working, we should be able to see the discovered entities in the frontend.

image-1655138862290.png


Collected Docker container metrics

In addition, our metrics should have also started coming in. We can check the Latest data section and verify that they are indeed getting collected.

image-1655138875928.png


Macros inherited from the Docker template

Lastly, we have a few additional options for further modifying the template and the results of our low-level discovery. If you open the Macros section of your host and select Inherited and host macros, you will notice that there are 4 macros inherited from the Docker template. These macros are responsible for filtering in/out the discovered containers and images. Feel free to modify these values if you wish to filter in/out the discovery of these entities as per your requirements.

Notice that the container discovery item also has one parameter, which is defined as false on the template:


Truenas Monitoring with Zabbix

Enable SNMP on Truenas Host

Figure 1

image-1655139299516.png

Figure 2

image-1655139392612.png

Figure 3

image-1655139531183.png

Download the the FREENAS-MIB Definitions File from your Truenas Host

Figure 4

image-1655146333104.png

Figure 5

image-1655146431089.png

Figure 6

image-1655146784737.png

Copy the the FREENAS-MIB Definitions File to your Zabbix Server

Import the SNMP Freenas 11 Template into Zabbix Server

Figure 7

image-1655145250022.png

Add Truenas Host to Zabbix server

Figure 8

image-1655144525187.png

Figure 9

image-1655144810693.png

Figure 10

image-1655147921858.png

 

 

Exclude Windows Services from Autodiscover in Zabbix 6.4

Navigate to Data collection ---> Templates. In the Templates page, enter Windows in the Name  field and click the Apply button. In the results listing, click on Windows by Zabbix agent

image.png

Next, click on Discovery rules tab and then click on Windows services discovery.

image.png

Next, click on Filters and then set the Label A Macro Regular expression field to @Windows service startup states for discovery and click the Update button.

image.png

Repeat the procedure above for the Windows by Zabbix Agent Active.

Next, navigate to Administration ---> General ---> Regular Expressions. In the Regular Expressions page, click on Windows service names for discovery.

image.png

In the Regular Expressions page, click on Add, set the Expression type drop-down to Result is FALSE and in the expression field enter the service name you wish to exclude between a ^ and $ characters. (For example for the VeeamVssSupport service enter ^VeeamVssSupport$), check the Case sensitive box if necessary. Continue adding lines for each service you wish to exclude and click the Update button once finished. If you wish to use a wildcard expression for a service name, enter .* at the end of the service name. (Ex: ^ServiceName.*$).

image.png

Please note that if you exclude Windows services from Autodiscover AFTER the host has already been added to Zabbix, the service names will still show up as alarms in Zabbix. You must navigate to Data collection ---> Hosts, locate and click on the Host(s) in question, click on Unlink and clear next to the Windows by Zabbix agent template and click the Update button.  Then, click on the Host(s) in question again and click on Delete and then re-add the Host(s) as necessary.