Unterschiede

Hier werden die Unterschiede zwischen zwei Versionen angezeigt.

Link zu dieser Vergleichsansicht

Beide Seiten der vorigen RevisionVorhergehende Überarbeitung
Nächste Überarbeitung
Vorhergehende Überarbeitung
Letzte ÜberarbeitungBeide Seiten der Revision
software:linux:realtime:benchmark:start [2019-01-24 16:21] mgehrig2software:linux:realtime:benchmark:start [2023-04-11 11:16] Urs Graf
Zeile 5: Zeile 5:
 The cyclictest is the simplest and most usefull benchmark. The cyclictest is the simplest and most usefull benchmark.
  
 +===== Tools =====
 +==== Cyclictest ====
 +Standard tool to measure the maximum jitter of a system.
  
-===== Cyclictest ===== +**[[..:cyclictest|Cyclictest]]**
-The Cyclictest starts several tasks, which are called periodically. +
-The effective period duration is measured and the jitter is displayed.+
  
-  * [[https://wiki.linuxfoundation.org/realtime/documentation/howto/tools/cyclictest/start?s[]=cyclictest | Cyclictest website]] 
  
-A presentation from the "Embedded Linux Conference 2013" offers a good introduction to the use of Cyclictest. 
-The presentation can be watched on [[https://www.youtube.com/watch?v=f_u4r6ehZKY | Youtube]]. 
-The PPT slides can be downloaded {{:software:linux:realtime:benchmark:cyclictest_frank_rowand_ppt.pdf|here}}. 
  
-==== Installation ==== +===== Influences on real-time behaviour ===== 
-<code>git clone git://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git +The following things can influence the real-time behaviour:
-sudo apt-get install build-essential libnuma-dev +
-cd rt-tests +
-git checkout -b tmp taggs/v1.0 +
-make all +
-</code>+
  
-==== Cross compile ==== +  * CPU 
-TODO+    * Mobile x86 CPU (with suffix like U, Y and M i.e i7-4600U) seems to perform much worse than non-mobile processors. Mobile processer add about 100us-200us jitter. 
 +    * ARM processer seem to have about 60us max jitter and a high medium jitter 
 +  * Kernel version 
 +    * 4.4.169-rt177 seems to be better than 4.19.15-rt12 
 +  * BIOS settings 
 +  * Some drivers, like WiFi driver, may negatively influence latency
  
-To install the Cyclictest on MPC follow [[software:linux:realtime:benchmark:cyclictest_on_mpc|this]] guid.+==== Don't ==== 
 +  * Don't start the application 'Grub customizer' while real-time application is running. This can introduce latencies of more than 4msec.
  
-==== Quickstart ==== 
-<code> 
-sudo ./cyclictest -p 80 -t5 
-</code> 
  
-5 threads are created, which are called periodically+==== Does ==== 
-After a few minutes, the maximum latency can be roughly estimated.+  * You should remove as many of the unknowns as possible. This means that only the necessary drivers should be loaded
 +  * If possibleuse only a text based system. See '[[software:linux:linux_os:text_only:start|Boot Ubuntu in text mode]]'
 +  * Run only software that is absolutely necessary on the real-time master. 
 +  * Deactivate automatic updates. 
 +  * A stable system can only be guaranteed if it has been sufficiently tested. Each system is different and must be tested.
  
  
-==== Extensive Measurement ==== +===== Methodical procedure ===== 
-In order to benchmark the realtime performance of a system well appropriately, a measurement duration of at least 24h under real or artificial load is recommended+==== Overview ==== 
-The frequency of different latencies can be counted and displayed in a histogram.+At first, test the most basic system possible. 
 +With this testyou can get baseline of the best possible real-time performance of the hardware
 +Every feature added will make the system more complex and add more stuff, which can increase jitter.
  
-=== Start Benchmark === +With this approach you can get a feeling which performance is possible and which part of the system is responsible for a high jitter.
-<code>sudo ./cyclictest -Sp80 -l86400000 -i1000 -h1000 -q > histogramm.log</code>+
  
-[{{ :software:linux:cyclictest:histograme05.jpg?400|Histogramm of a x86 system with RT Kernel. Measurement time = 1h}}] 
  
--Sp80       |S = alle cores; p = priority**-**80 (Realtime)| +  Test hardware, BIOS settings and kernel version 
-| -l86400000  |86'400'000 cycles| +  - Test full distro in text mode 
--i1000      |1000us per cycle| +  Test full distro in graphical mode 
--h1000      |Latencies up to 1000us will be stored for the histogram| +  Test full distro under load
--q          |quiet| +
-^ Measurement time: |l86400000*i1000 = 86400e6us = 86400s = 24h|+
  
-All measurements longer than ''-h1000'' do not appear in the histogramThese measurements are counted and displayed at the end of the log file as //Histogram Overflows://. The Maximum Latency is displayed as //Max Latencies://.+==== 1.) Test hardware, BIOS settings and kernel version ==== 
 +  - Build a [[software:linux:realtime:preempt_rt:build_kernel|RT-kernel]] and install it. 
 +  - Install the [[software:linux:realtime:benchmark:cyclictest|cyclictest]] 
 +  - Reboot with the new kernel in [[software:linux:linux_os:run_lvl_1|RunLevel 1]] 
 +  - Check the real-time performance with [[software:linux:realtime:benchmark:cyclictest|cyclictest]]
  
-To plot the histogram this Matlab script {{:software:linux:realtime:benchmark:plothistogram.rar|plotHistogramm.m}} can be used.+**Tipp**You may want to use [[https://wiki.archlinux.org/index.php/GNU_Screen |screen]]. 
 +This application provides multiple virtual terminal sessions if you want to run multiple programs in parallel.. 
 + 
 +=== Tuning === 
 +If you the measured latencies are to high, you can tweak your system. 
 +The options which are simple but effective are listed first. 
 + 
 +  * Tune some [[software:linux:realtime:tune_bios_settings|BIOS settings]] 
 +  * Use a different kernel version(4.4.169-rt177 seems to be better than 4.19.15-rt12) 
 +  * Tune some kernel settings TODO 
 +  * If the jitter is still too high, you may have to choose a different CPU 
 + 
 + 
 +==== 2.) Test full distro in text mode ==== 
 +Boot your system in [[software:linux:linux_os:text_only:start|text mode]]. 
 + 
 +If the real-time performance is not significantly reduced, then the third step can be continued. 
 +Run a 24h test so that exceptional events can also be detected.  
 + 
 +If the real-time performance is significantly worse, then a WLAN driver or something similar can be the cause. 
 +To further isolate the problem, the WLAN driver, or another suspected driver, can be deactivated (''rmmod iwlwifi''). 
 +A new test should result in an improvement of the maximum jitter. 
 + 
 +=== Other possible causes === 
 +  * WLAN driver 
 +  * Ethernet driver 
 +  * Bluetooth driver 
 +  * Various peripheral devices 
 + 
 + 
 +==== 3.) Test full distro in graphical mode ==== 
 +It is recommended that a real-time system be used only in text mode. 
 +With a graphical user interface, the system is generally more unstable. 
 + 
 +If, however, the system is to be used with a graphical user interface, a 24h test [[software:linux:linux_os:text_only:start|including a graphical user interface]] is recommended. 
 + 
 + 
 +==== 4.) Test full distro under load ==== 
 +=== High CPU load === 
 +High CPU load from non-RT processes does not affect the performance of RT processes very much. 
 +However, if the processor is not sufficiently cooled (e.g. in a mobile system), the processor can overheat. 
 +In such a case, the processor clock is clocked down by the system, which can severely impair real-time performance. 
 + 
 +Check the temperature of the CPU under high load with [[https://wiki.archlinux.org/index.php/lm_sensors | lmsensors]] to avoid this problem. 
 + 
 +If the processor does overheat: 
 +  * Disable hyper.threading in BIOS 
 +  * Deactivate Turboboost in BIOS 
 +  * Ensure better cooling 
 + 
 +**Note ***: Depending on the processor, //lm_sensors// itself may cause high latency. 
 +Observe the //Cyclictest// during the ''sensor'' call to check this. 
 + 
 +=== Normal Load === 
 +In the last test allplitkation are to be started, which are to run also later in the employment. 
 +For the //cyclictest// to measure correctly, its priority must be one higher than the priority of the RT application. 
 +This test should last at least 24 hours.
  
-The homepage of [[https://www.osadl.org/Latency-plots.latency-plots.0.html|OSADL]] explains well how such a histogram can be interpreted.