Implement and upstream ARM 32 bit uprobes support
Implement and upstream uprobes for ARM
Blueprint information
- Status:
- Complete
- Approver:
- Jakub Pavelek
- Priority:
- High
- Drafter:
- Deepak Saxena
- Direction:
- Approved
- Assignee:
- David Long
- Definition:
- Approved
- Series goal:
- Accepted for kernel-merge-window
- Implementation:
-
Informational
- Milestone target:
-
3.13
- Started by
- David Long
- Completed by
- Jakub Pavelek
Related branches
Related bugs
Sprints
Whiteboard
Meta:
Roadmap id: CARD-153
Headline: ARM uprobes support has been upstreamed
Acceptance:
* linux-linaro Ubuntu Engineering build and build spec have been updated to include kprobes and uprobes support (Naresh/Jakub can help)
* Kprobes and Uprobes can be used in unison to trace system behaviour across the software stack (TODO - how does one prove that? Not a criterion, a requirement)
* uprobes support for ARM is merged into upstream kernel
* GIT commit link to kernel tree has been logged here
* uprobes support for ARM is merged into systemtap sources
* GIT commit link to systemtap tree has been logged here
* ci.linaro.org linux master build also execute the tests developed in this card and succeeds (to prove that the code merged upstream still passes tests) (Andrey Konovalov/Jakub can help)
[jakub-pavelek 2013-03-28] moving from 3.10 to 3.11 as current upstreaming target.
[dave-long 2013-04-30] After discussion with tixy it appears the amount of redundant code in the current approach is likely to be an issue upstream. We came up with a modified approach which I am working on now.
[jakub pavelek 2013-06-17] Moving from 3.11 to 3.12 with a bit of optimism
[jakub-pavelek 2013-06-26] Naresh got system tap test suite running - Dave please contact Naresh and get the same test suite deployed locally for your testing. Naresh will deploy the test suite to LAVA (we will make a follow-up Blueprint for him) and will run the test suite there - we want to see it passing in our CI.
[frank-eigler 2013-06-26 Please involve <email address hidden> if any systemtap-side porting work is found to be required.]
[nkambo 2013-09-04] Systemtap install on snowball and run the a simple test. Here is the log which gives complete information about
- Build kernel with following config
- kernel config file: https:/
- Must install kernel headers and modules on the target
- systemtap looks for .config and System.map file under /lib/module/'uname -r'/
- Download systemtap-2.2.1 source
- Download elfutils source
- Download dependencies build-essential aptitude m4 dejagnu
- apt-get install build-essential aptitude m4 dejagnu
- cd systemtap-2.2.1
- cp /usr/share/
- Configure systemtap
- ./configure --prefix=
- Build and Install systemtap-2.2.1
- make all; make install;
- systemtap example program
- # stap -ve 'probe begin { log("hello world") exit() }'
- https:/
- Run " make installcheck"
- Complete systemtap log can be found at: https:/
=== systemtap Summary ===
# of expected passes 770
# of unexpected failures 466
# of unexpected successes 1
# of expected failures 281
# of known failures 20
# of untested testcases 211
# of unsupported tests 5
runtest completed at Wed Sep 4 15:21:35 2013
- Observations While running this test suite on target, kill command has issues due to this reason, test process created are not killed and the inserted modules not removed from the kernel space due to this reason kernel reported "Out of memory: " many times.
- The tests keeps on adding new modules to kernel space and finally kernel space was full and stalled. I have manually killed many test processes and removed kernel modules to free the memory to continue test execution.
- "Out of memory:" dmesg log: https:/
- After the test execution test results will be available in file : systemtap.sum
- file printout : https:/
- Test results notations
- PASS
- XFAIL
- FAIL
- KFAIL
- XFAIL
- UNTESTED
- UNSUPPORTED
[jakub-pavelek 2013-09-05] Bumping from 3.12 to 3.13 - the merge window is open, do not see this being pushed in. Also we do not have Thumb implemented yet and the testing is just starting. I see many failures in the testing - do we need some SystemTap work do be done to get this working? Frank has been offering help couple of months ago - please consider keeping him in the loop on testing.
[William Cohen <email address hidden> 2013-09-05]
I have been working on making sure that SystemTap functions reasonably well on ARM. Frank Eigler mentioned that you are working on uprobes for ARM and pointed me at https:/
I can't see the details on the pastebin.linaro.org pages, but the number of failure looks unusually high and the number of tests run looks low on https:/
=== systemtap Summary ===
# of expected passes 770
# of unexpected failures 466
# of unexpected successes 1
# of expected failures 281
# of known failures 20
# of untested testcases 211
# of unsupported tests 5
runtest completed at Wed Sep 4 15:21:35 2013
One thing thing that might prevent the modules from unloading is not having the following in the kernel config:
CONFIG_
For comparison I have a samsung arm chromebook set up with Fedora 19 and a locally built linux-3.11 kernel. systemtap from a git repo checkout yielded the following results:
=== systemtap Summary ===
# of expected passes 1815
# of unexpected failures 89
# of expected failures 283
# of known failures 10
# of untested testcases 364
# of unsupported tests 10
The details on chromebook's locally built kernel are on http://
-Will
[William Cohen <email address hidden> 2013-09-06]
There has been some work at Linaro.org to include uprobes in the arm kernel:
https:/
I built a local version of a the uprobes branch kernel with uprobes enabled from:
https:/
The tests took several hours on a samsung arm chromebook
=== systemtap Summary ===
# of expected passes 3011
# of unexpected failures 168
# of expected failures 283
# of known failures 40
# of untested testcases 179
# of unsupported tests 4
The complete results are posted on dejazilla:
web.elastic.
I didn't see any regressions in a comparison between a stock linux-3.11 vs. the linux-3.11-rc5 with uprobes on the same arm machine:
https:/
-Will
[nkambo 2013-09-11]
latest systemtap results on snowball
=== systemtap Summary ===
# of expected passes 2034
# of unexpected failures 425
# of unexpected successes 1
# of expected failures 281
# of known failures 27
# of untested testcases 152
# of unsupported tests 6
[Will 2013-09-17]
On 09/10/2013 04:08 PM, William Cohen wrote:
> Last week Frank mentioned there were some patches for uprobes on ARM (https:/
>
> With the host set up I was able to set up a guest Fedora 18 vm based on the instructions at https:/
A bit more playing around with hardware virtuatlization acceleration on arm. I got a stock fedora 19 image guest image set up on the chromebook. The kvm support is working well enough to build systemtap and test systemtap within the guest. The results look pretty respectable. This is the stock fedora arm 3.9.5-301.
=== systemtap Summary ===
# of expected passes 1744
# of unexpected failures 85
# of expected failures 280
# of known failures 10
# of untested testcases 367
# of unsupported tests 12
More detailed breakdown of the results are at:
-Will
[nkambo 2013-Sept-18]
I built a local version of a the uprobes branch kernel with uprobes enabled from:
https:/
Systemtap translator/driver (version 2.2.1/0.155, non-git sources)
Thread model: posix
gcc version 4.7.3 (Ubuntu/Linaro 4.7.3-1ubuntu1)
Linux linaro-developer 3.11.0-rc5 #1 SMP PREEMPT Tue Sep 17 18:46:23 UTC 2013 ar
mv7l armv7l armv7l GNU/Linux
Results logs:
-------
http://
- systemtap.log
- systemtap.sum
- stap-report-
snowball target systemtap test results summary
=== systemtap Summary ===
# of expected passes 2089
# of unexpected failures 381
# of unexpected successes 1
# of expected failures 281
# of known failures 32
# of untested testcases 151
# of unsupported tests 6
[dave-long 2013-Sept-18] I've created another repo at:
git://git.
...which contains a "uprobes" branch with my uprobes patches. This repo is based on linaro-
Note, the following are the config differences from the default config to enable uprobes/kprobes testing:
CONFIG_
CONFIG_
CONFIG_UPROBES=y
CONFIG_
[nkambo Oct-2-2013]
Latest systemtap (2.4) and elfutils (0.157) test execution results on snowball.
kernel version: 3.11.0-rc5
systemtap version: version 2.4/0.157
=== systemtap Summary ===
# of expected passes 2309
# of unexpected failures 390
# of expected failures 280
# of known failures 41
# of untested testcases 195
# of unsupported tests 7
[jakub-pavelek 2013-10-03] Naresh/others - how should the test results look like in order to day "uprobes32 are tested and passing as expected". I'm not familiar with what issues are part of systemtap and which part of the implementation we did. Someone please spell it out here so that we have a target to go after please.
Another issue - I will be moving this blueprint from Launchpad to JIRA - as this is the last artifact that kernel working group has left behind on Launchpad. I won't be able to subscribe all the people subscribed here unfortunately. Will keep you posted.
[jakub-pavelek 20013-10-30] Moving active tracking over to Engineering card in JIRA at https:/
Work Items
Work items for 13.02:
Contact patch originator (Feb 22) : DONE
Build and test patch on 3.6 using originators test suite (Feb 25): DONE
Work items for 13.04:
Review existing uprobes patches (April 3) : DONE
Uplevel and test existing patches (April 3): DONE
Extract common uprobes and kprobes code into shared functions (April 29): DONE
Separate uprobe and kprobe parsing data and function into their own files (April 29): DONE
Internal review (new approach discussed) (4/30): DONE
Work items for 13.06:
Design and implement vector table for parser output actions (6/12): DONE
Recreate patch set for public review (June 19): DONE
Get QEMU enviroment set up and working (6/4): DONE
Uplevel work to v3.10-rc3 (6/6): DONE
Work items for 13.08:
Fix regressions (August 1): DONE
Ask Naresh to help setup systemtap locally for own testing the same way it will run in our continuous integration: DONE
Work items for 13.09:
Add thumb support for uprobes: TODO
Upstream uprobes patches to 3.12: TODO
After patchset review, ping Andrey Konovalov and Naresh to take your topic branch and setup testing in our CI - we will need that one working as acceptance criterion: TODO
Review thumb-supporting patchset: TODO
Upstream thumb-supporting patchset: TODO
Dependency tree
* Blueprints in grey have been implemented.
