The Polkadot client consists of a single binary file. It is executed and
configured with a variety of command-line flags. Users can choose to either
compile the binary themselves or fetch a pre-compiled binary from our
releases page. A common way
to run a polkadot node is to copy this binary to
/usr/local/bin and have a
systemd service file that executes the binary with whatever command-line
flags are desired. While there are guides and advice on how to run a Polkadot
node (such as the fantastic Secure Validator
Setup), there are no
assumptions or expectations about how to invoke the binary.
Detailed instructions on how to build the polkadot binary can be found in the repository’s README.md.
Creating and securing a systemd service file
Basic Systemd Service
The majority of modern Linux distributions use
as their init system. Systemd allows us to write what it calls ‘service files’
in order to run programs as services (also known as daemons). Running the program
as a systemd service means we can offload things such as logging, error handling, etc to systemd and not worry about them too much ourselves.
The most basic systemd service file for running polkadot looks like this:
[Unit] Description=Polkadot node [Service] ExecStart=/usr/local/bin/polkadot [Install] WantedBy=multi-user.target
This will simply execute the polkadot binary with no additional command-line
flags. This gives us the advantage of logging output to
allows us to control the service with
systemctl, but there are plenty of
improvements that can be made. Let’s start with a few small ones.
[Unit] Description=Polkadot Node After=network.target Documentation=https://github.com/paritytech/polkadot [Service] EnvironmentFile=-/etc/default/polkadot ExecStart=/usr/bin/polkadot $POLKADOT_CLI_ARGS User=polkadot Group=polkadot Restart=always RestartSec=120 MemoryHigh=5400M MemoryMax=5500M [Install] WantedBy=multi-user.target
First, we add the dependency
After= which ensures the service only attempts
to start after the system’s network is up. We also specify an
EnvironmentFile, in which we can specify environment variables that are
available to the binary called in
ExecStart. Polkadot does not currently have
any support for reading its configuration from a config file, but we can use
this technique to specify a list of flags on startup. The environment file
contains the following as an example.
POLKADOT_CLI_ARGS="--database paritydb --wasm-execution Interpreted"
Flags can be added or removed to this environment variable as needed. Unfortunately,
it seems we can’t nest environment variables in this file. So for example, we can’t
have something like
We also specify a user and group to run the service as - by default, the
service will run as root. If this can be avoided, it should be. We specify a
restart policy for the service - if the program crashes, it will always be
restarted but only after a pause of 120 seconds.
Finally, we add a restriction on the max memory the process can use - if it
MemoryHigh limit, any new memory is heavily reaped. If it hits the
MemoryMax limit, the OOM killer will kill the process (thus triggering the
aforementioned restart mechanism).
This is a reasonable service file - we have some logic surrounding when to start
and restart the service, some basic privilege separation, and users can configure
the command-line flags used without having to edit the service file and run
systemctl daemon-reload every time. Now we need to make it secure!
Securing the service file
Systemd comes with a wide variety of configurable options that can be applied
to services in order to improve the security - From basic things like
disallowing the service from gaining superuser privileges, to providing a
strict set of system calls the program is allowed to make. In order to find out
what improvements are even available, we can use the excellent
Note: The service file must be in-place in order to
systemd-analyze against it. Copy it to
/etc/systemd/system/ and run
systemctl daemon-reload each time you make an edit.
% systemd-analyze security --no-pager polkadot.service NAME DESCRIPTION EXPOSURE ✗ PrivateNetwork= Service has access to the host's network 0.5 ✓ User=/DynamicUser= Service runs under a static non-root use… ✗ CapabilityBoundingSet=~CAP_SET(UID|GID|P… Service may change UID/GID identities/ca… 0.3 <SNIP> ✗ CapabilityBoundingSet=~CAP_WAKE_ALARM Service may program timers that wake up … 0.1 ✗ RestrictAddressFamilies=~AF_UNIX Service may allocate local sockets 0.1 → Overall exposure level for polkadot.service: 9.2 UNSAFE 😨
systemd-analyze generates a list of all the configuration options
that can be edited in order to reduce the exposure of the service, and sums
each unapplied option in order to generate a final exposure score. As can be
seen above, we scored 9.2/10 (lower is better) with a ranking of ‘UNSAFE’.
As an aside, I highly recommend running
systemd-analyze security with no
additional arguments. It will print an exposure summary for all running
services. You might be surprised at just how many services are considered
unsafe. I know I was.
The process of securing the service essentially consists of going through each
systemd-analyze output and deciding whether that particular
restriction can be implemented without hindering the operation of your program.
For instance, the very first suggestion is
PrivateNetwork=, which restricts
whether or not the service can communicate on the host’s network. Since
Polkadot is a network service, we don’t really want to stop it being able
to communicate over the network.
Here is the final
polkadot.service file after taking heed of
[Unit] Description=Polkadot Node After=network.target Documentation=https://github.com/paritytech/polkadot [Service] EnvironmentFile=-/etc/default/polkadot ExecStart=/usr/bin/polkadot $POLKADOT_CLI_ARGS User=polkadot Group=polkadot Restart=always RestartSec=120 MemoryHigh=5400M MemoryMax=5500M CapabilityBoundingSet= LockPersonality=true NoNewPrivileges=true PrivateDevices=true PrivateMounts=true PrivateTmp=true PrivateUsers=true ProtectClock=true ProtectControlGroups=true ProtectHostname=true ProtectKernelModules=true ProtectKernelTunables=true ProtectSystem=strict RemoveIPC=true RestrictAddressFamilies=AF_INET AF_INET6 AF_NETLINK AF_PACKET AF_UNIX RestrictNamespaces=true RestrictSUIDSGID=true SystemCallArchitectures=native UMask=0027 SystemCallFilter=@system-service SystemCallFilter=~@clock @module @mount @reboot @swap @privileged [Install] WantedBy=multi-user.target
That’s quite a few new lines! Some of them may apply to a service you are
writing, some may not. It is important to understand how adding restrictions to
the execution environment of the service will impact the execution of your
program. For example, if I had added a suggestion from
MemoryDenyWriteExecute=true (which stops the program from creating
memory mappings that are both executable and writable), Polkadot would run
just fine with the argument
--wasm-execution Interpreted but would crash when
invoking Polkadot with
--wasm-execution Compiled. It is important to
understand how adding restrictions to the execution environment of the service
will impact the execution of your program.
systemd-analyze on the updated service file gives a much better result:
% systemd-analyze security --no-pager polkadot.service | tail -n1 → Overall exposure level for polkadot.service: 1.9 OK 🙂
Now that we have Polkadot running as a service and we’re a lot more confident with the security of the service, it’s time to package and distribute.