Lessons We Learned (common mistakes while testing applications with the RT kernel)

There is a pattern of problems that appear in customer reports when they are trying the RT kernel, be it with their applications or with scripts simulating the worst cases scenarios of their applications. Problems such as the misleading concept of "CPU isolation" on Linux or the usage of the highest available priority to run busy loops tend to be present in these tests.

There are a few subtleties in the RT kernel that, if ignored, can lead to severe performance degradation. One fine example is the contention on the mmap_sem semaphore, that when the application has a considerable amount of threads sharing memory (say 1000) will give the user the impression of a application freeze. This is known to the developers and documented on a few obscure bugzilla tickets, but seems to surprise customers.

Some of these problems are caused by stale documentation, some are caused by the lack of documentation. Others are caused by very excited people who will not waste time reading documentation.

This talk will present some of these patterns and briefly discuss how the ones that are really problems in the RT kernel are being addressed.

Topic Lead: Luis Claudio <email address hidden>
Luis Claudio is a member of the Real Time team at Red Hat. Linux user since 1994, he has already been a user, sysadmin, teacher and developer. His current interest areas are RT, HA and music.