There are many different tools available to measure the realtime performance of a system.

The cyclictest is the simplest and most usefull benchmark.

Standard tool to measure the maximum jitter of a system.

Cyclictest

The following things can influence the real-time behaviour:

• CPU
  • Mobile x86 CPU (with suffix like U, Y and M i.e i7-4600U) seems to perform much worse than non-mobile processors. Mobile processer add about 100us-200us jitter.
  • ARM processer seem to have about 60us max jitter and a high medium jitter
• Kernel version
  • 4.4.169-rt177 seems to be better than 4.19.15-rt12
• BIOS settings
• Some drivers, like WiFi driver, may negatively influence latency

• Don't start the application 'Grub customizer' while a real-time application is running. This can introduce latencies of more than 4msec.

• You should remove as many of the unknowns as possible. This means that only the necessary drivers should be loaded.
• If possible, use only a text based system. See 'Boot Ubuntu in text mode'.
• Run only software that is absolutely necessary on the real-time master.
• Deactivate automatic updates.
• A stable system can only be guaranteed if it has been sufficiently tested. Each system is different and must be tested.

At first, test the most basic system possible. With this test, you can get a baseline of the best possible real-time performance of the hardware. Every feature added will make the system more complex and add more stuff, which can increase jitter.

With this approach you can get a feeling which performance is possible and which part of the system is responsible for a high jitter.

1. Test hardware, BIOS settings and kernel version
2. Test full distro in text mode
3. Test full distro in graphical mode
4. Test full distro under load

1. Build a RT-kernel and install it.
2. Install the Cyclictest
3. Reboot with the new kernel in RunLevel 1
4. Check the real-time performance with Cyclictest

Tipp: You may want to use screen. This application provides multiple virtual terminal sessions if you want to run multiple programs in parallel..

If you the measured latencies are to high, you can tweak your system. The options which are simple but effective are listed first.

• Tune some BIOS settings
• Use a different kernel version. (4.4.169-rt177 seems to be better than 4.19.15-rt12)
• Tune some kernel settings
• If the jitter is still too high, you may have to choose a different CPU

Boot your system in text mode.

If the real-time performance is not significantly reduced, then the third step can be continued. Run a 24h test so that exceptional events can also be detected.

If the real-time performance is significantly worse, then a WLAN driver or something similar can be the cause. To further isolate the problem, the WLAN driver, or another suspected driver, can be deactivated (rmmod iwlwifi). A new test should result in an improvement of the maximum jitter.

• WLAN driver
• Ethernet driver
• Bluetooth driver
• Various peripheral devices

It is recommended that a real-time system be used only in text mode. With a graphical user interface, the system is generally more unstable.

If, however, the system is to be used with a graphical user interface, a 24h test including a graphical user interface is recommended.

High CPU load from non-RT processes does not affect the performance of RT processes very much. However, if the processor is not sufficiently cooled (e.g. in a mobile system), the processor can overheat. In such a case, the processor clock is clocked down by the system, which can severely impair real-time performance.

Check the temperature of the CPU under high load with lmsensors to avoid this problem.

If the processor does overheat:

• Disable hyper.threading in BIOS
• Deactivate Turboboost in BIOS
• Ensure better cooling

Note *: Depending on the processor, lm_sensors itself may cause high latency. Observe the Cyclictest during the sensor call to check this.

In the last test allplitkation are to be started, which are to run also later in the employment. For the cyclictest to measure correctly, its priority must be one higher than the priority of the RT application. This test should last at least 24 hours.