kernel panic after system bootup and busybox, 0xbench, glmark2, skia, mmtest , monkeyrunner test fails on panda android jb 4.2 tracking builds

Bug #1090245 reported by Soumya Basak
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Linaro Android
Won't Fix
Undecided
Unassigned
linaro-landing-team-ti
New
Undecided
Unassigned

Bug Description

image url: https://android-build.linaro.org/jenkins/job/linaro-android_panda-jb-gcc47-tilt-tracking-blob/131/

job id: http://validation.linaro.org/lava-server/scheduler/job/40954
complete log: http://validation.linaro.org/lava-server/scheduler/job/40954/log_file
log:
Install busybox shell on device is failed. prob seems on adb-ethernet .

root@linaro# <LAVA_DISPATCHER>2012-12-05 08:35:00 AM INFO: Execute command on host: timeout 18000s lava-android-test install busybox -s 192.168.1.73:5555
ERROR: Test installation error: Failed to create directory(/data/lava-android-test/share/installed-tests/busybox) for test(busybox)
LAVA: --Start Operation: Install test (busybox) on device(192.168.1.73:5555)
<LAVA_DISPATCHER>2012-12-05 08:51:19 AM INFO: Execute adb command on host: adb disconnect 192.168.1.73
No such device 192.168.1.73:5555

<LAVA_DISPATCHER>2012-12-05 08:51:19 AM WARNING: [ACTION-E] lava_android_test_install is finished with error (Failed to install test case(busybox) on device(192.168.1.73:5555) with return value: 1).

busybox test failed on device

/srv/lava/instances/production/bin/lava-android-test
Lava failed at action lava_android_test_install with error:Failed to install test case(busybox) on device(192.168.1.73:5555) with return value: 1
Traceback (most recent call last):
  File "/srv/lava/.cache/eggs/lava_dispatcher-0.24.1-py2.7.egg/lava_dispatcher/job.py", line 178, in run
    action.run(**params)
  File "/srv/lava/.cache/eggs/lava_dispatcher-0.24.1-py2.7.egg/lava_dispatcher/actions/lava_android_test.py", line 231, in run
    "return value: %s" % (test, session.dev_name, rc))
OperationFailed: Failed to install test case(busybox) on device(192.168.1.73:5555) with return value: 1

The kernel panic happend at the device bootups

[ 120.837249] ---[ end trace fd71d128b2ae2a64 ]---
[ 120.850189] Kernel panic - not syncing: Fatal exception
[ 120.860870] CPU1: stopping

<LAVA_DISPATCHER>2012-12-05 08:54:29 AM ERROR: _enter_uboot failed
Traceback (most recent call last):
  File "/srv/lava/.cache/eggs/lava_dispatcher-0.24.1-py2.7.egg/lava_dispatcher/device/master.py", line 473, in _boot
    self._soft_reboot()
  File "/srv/lava/.cache/eggs/lava_dispatcher-0.24.1-py2.7.egg/lava_dispatcher/device/master.py", line 445, in _soft_reboot
    raise OperationFailed("Soft reboot failed")
OperationFailed: Soft reboot failed

test failed 0xbench and glmark2, skia test due to not install on system (adb-ethernet failed on device)

OperationFailed: Failed to run test case(0xbench) on device(192.168.1.73:5555) with return value: 1

OperationFailed: Failed to run test case(glmark2) on device(192.168.1.73:5555) with return value: 1

OperationFailed: Failed to run test case(skia) on device(192.168.1.73:5555) with return value: 1

mmtest failed on device

OperationFailed: Failed to run test case(v8) on device(192.168.1.73:5555) with return value: 1

OperationFailed: Failed to run test case(mmtest) on device(192.168.1.73:5555) with return value: 1

same observed with Linaro-android panda tracking builds:

image url: https://android-build.linaro.org/jenkins/job/linaro-android_panda-jb-gcc47-tilt-tracking-blob/134/
job id: http://validation.linaro.org/lava-server/scheduler/job/41102
complete log: http://validation.linaro.org/lava-server/scheduler/job/41102/log_file

image url: https://android-build.linaro.org/jenkins/job/linaro-android_panda-jb-gcc47-tilt-tracking-blob/135/
job id: http://validation.linaro.org/lava-server/scheduler/job/41159
complete log: http://validation.linaro.org/lava-server/scheduler/job/41159/log_file

image url: https://android-build.linaro.org/jenkins/job/linaro-android_panda-jb-gcc47-tilt-tracking-blob/136/
job id: http://validation.linaro.org/lava-server/scheduler/job/41360
complete log: http://validation.linaro.org/lava-server/scheduler/job/41360/log_file

image url: https://android-build.linaro.org/jenkins/job/linaro-android_panda-jb-gcc47-tilt-tracking-blob/138/
job id: http://validation.linaro.org/lava-server/scheduler/job/41590
complete log: http://validation.linaro.org/lava-server/scheduler/job/41590/log_file

image url: https://android-build.linaro.org/jenkins/job/linaro-android_panda-jb-gcc47-tilt-tracking-blob/142/
job id: http://validation.linaro.org/lava-server/scheduler/job/42136
complete log: http://validation.linaro.org/lava-server/scheduler/job/42136/log_file

Revision history for this message
Andy Doan (doanac) wrote :
Download full text (17.6 KiB)

Not sure why a kernel crash on a panda is a lava-dashboard bug. going to assign this to proper project. Here's the relevant kernel dump:

[ 118.543792] Unable to handle kernel paging request at virtual address e66107c0
[ 118.555450] pgd = e396c000
[ 118.562286] [e66107c0] *pgd=00000000
[ 118.570770] Internal error: Oops: 5 [#1] PREEMPT SMP ARM
[ 118.580322] Modules linked in: wl12xx wlcore mac80211 cfg80211 lib80211 btwilink rfcomm bnep bluetooth gator
[ 118.595001] CPU: 0 Not tainted (3.4.0-g66afb0d #1)
[ 118.604492] PC is at vmap_page_range_noflush+0xc8/0x19c
[ 118.614105] LR is at map_kernel_range_noflush+0x1c/0x20
[ 118.623687] pc : [<c00e5a3c>] lr : [<c00e6f34>] psr: 800f0113
[ 118.623718] sp : e3bb7d80 ip : e66107c0 fp : e3bb7dc4
[ 118.644805] r10: e66107c0 r9 : c0759048 r8 : c07ef0c0
[ 118.654388] r7 : c0007f98 r6 : fe7f1000 r5 : 00000000 r4 : fe7f0000
[ 118.665283] r3 : e393fc00 r2 : e66107c0 r1 : fe7f0fff r0 : fe7f0000
[ 118.676208] Flags: Nzcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[ 118.687835] Control: 10c5387d Table: a396c04a DAC: 00000015
[ 118.698822]
[ 118.698822] PC: 0xc00e59bc:
[ 118.711883] 59bc e0837182 e2842602 e2461001 e1a02aa2 e1a02a82 e242a001 e15a0001 21a02006
[ 118.724792] 59dc e3570000 e50b2030 0a00003d e5972000 e3520000 0a000007 e597c000 e7e82654
[ 118.737792] 59fc e3ccceff e3ccc00f e24cc4c0 e09cc102 1a000006 ea000032 e1a00007 e1a01004
[ 118.750823] 5a1c ebffd3f6 e3500000 0afffff2 ea00002c e59f90d4 e1a0a00c e1a0200a e51b3038
[ 118.763885] 5a3c e592e000 e28aa004 e7931105 e35e0000 0a000015 e59f00b4 e3a0106a ea000001
[ 118.776947] 5a5c e59f00a8 e3a0106c ebfd5d00 ea00001c e1a00001 e50b1040 e50b203c ebfcd4a9
[ 118.790069] 5a7c e51b203c e51b1040 e1a00002 e3a02b02 ebfcdf3d e51b3030 e2844a01 e2855001
[ 118.803253] 5a9c e1540003 1affffe3 ea00000f e3510000 0affffea e5980000 e35404bf e51b3034
[ 118.816497]
[ 118.816497] LR: 0xc00e6eb4:
[ 118.829742] 6eb4 e59f302c e5933000 e2833502 e1a03ba3 e1a03b83 e1500003 2a000001 e3a00000
[ 118.843109] 6ed4 e89da800 e35004ff 2afffffb e3a00001 e89da800 c07ef0d0 e1a0c00d e92dd800
[ 118.856475] 6ef4 e24cb004 e52de004 e8bd4000 ebfff878 e59f3008 e2800001 e583002c e89da800
[ 118.869842] 6f14 c07ef148 e1a0c00d e92dd800 e24cb004 e52de004 e8bd4000 e0811000 ebfffa8f
[ 118.883300] 6f34 e89da800 e1a0c00d e92dd830 e24cb004 e52de004 e8bd4000 e1a04000 e0815000
[ 118.896789] 6f54 e59f3030 e5933000 e3130002 1a000002 e59f3024 e5933004 e12fff33 e1a00004
[ 118.910247] 6f74 e1a01005 ebfffb41 e1a00004 e1a01005 ebfcb614 e89da830 c07558d4 c07558ac
[ 118.923767] 6f94 e1a0c00d e92dd800 e24cb004 e24dd010 e52de004 e8bd4000 e58d3000 e3a030d0
[ 118.937225]
[ 118.937225] SP: 0xe3bb7d00:
[ 118.950897] 7d00 c1112564 c07aea84 c1112560 00000001 c0ef9614 000201d2 c00e5a3c 800f0113
[ 118.964416] 7d20 ffffffff e3bb7d6c e3bb7dc4 e3bb7d38 c000df98 c0008494 fe7f0000 fe7f0fff
[ 118.977996] 7d40 e66107c0 e393fc00 fe7f0000 00000000 fe7f1000 c0007f98 c07ef0c0 c0759048
[ 118.991546] 7d60 e66107c0 e3bb7dc4 e66107c0 e3bb7d80 c00e6f34 c00e5a3c 800f0113 ffffffff
[ 119.005065] 7d80 e3bb7e0c 00000000 c07aea84 e393fc00 0000065f fe7f1000 c00...

affects: lava-dashboard → linaro-landing-team-ti
Fathi Boudra (fboudra)
affects: linaro-landing-team-ti → linaro-android
description: updated
description: updated
description: updated
description: updated
Revision history for this message
warmcat (andy-warmcat) wrote :

Is this the tilt-3.2 kernel or the tilt-3.4 based one? If it's tilt-3.2, we're probably not going to look at it.

Revision history for this message
Soumya Basak (soumya-basak) wrote :

its a tilt 3.4 based one.

Revision history for this message
Soumya Basak (soumya-basak) wrote :

image url : https://android-build.linaro.org/jenkins/job/linaro-android_panda-jb-gcc47-tilt-tracking-blob/141/
job id: http://validation.linaro.org/lava-server/scheduler/job/41947
complete log: http://validation.linaro.org/lava-server/scheduler/job/41947/log_file
log:
kernel panic will happend on reboot the system
[ 115.221618] Unable to handle kernel paging request at virtual address e66107c0
[ 115.233184] pgd = e3894000
[ 115.240447] [e66107c0] *pgd=00000000
[ 115.248352] Internal error: Oops: 5 [#1] PREEMPT SMP ARM
[ 115.257873] Modules linked in: wl12xx wlcore mac80211 cfg80211 lib80211 btwilink rfcomm bnep bluetooth gator
[ 115.272552] CPU: 0 Not tainted (3.4.0-g66afb0d #1)
[ 115.282043] PC is at vmap_page_range_noflush+0xc8/0x19c
[ 115.291625] LR is at map_kernel_range_noflush+0x1c/0x20
[ 115.301177] pc : [<c00e5a68>] lr : [<c00e6f60>] psr: 800f0113
[ 115.301177] sp : e390dd80 ip : e66107c0 fp : e390ddc4
[ 115.322204] r10: e66107c0 r9 : c0759048 r8 : c07ef0c0
[ 115.331604] r7 : c0007f98 r6 : fe7f1000 r5 : 00000000 r4 : fe7f0000
[ 115.342407] r3 : e3e34d00 r2 : e66107c0 r1 : fe7f0fff r0 : fe7f0000
[ 115.353210] Flags: Nzcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[ 115.364624] Control: 10c5387d Table: a389404a DAC: 00000015

Due to the skia test, mmtest and monkeyrunner test Installation on device is not successful on device(due to kernel panic on board bootup), so the test cases are failed.

ERROR: The test (skia) has not been installed yet.
LAVA: --Start Operation: Run test (skia) on device(192.168.30.235:5555)
OperationFailed: Failed to run test case(skia) on device(192.168.30.235:5555) with return value: 1

wifi test case fail from the log:

OperationFailed: Failed to run test case(hostshell-connect-lab-wifi) on device(192.168.30.235:5555) with return value: 1

description: updated
Revision history for this message
Jassi Brar (jassisinghbrar) wrote :

Probably not directly relevant but I also see a more fundamental sounding error :-

<LAVA_DISPATCHER>2012-12-14 09:46:58 AM INFO: Execute command on host: timeout 18000s lava-android-test install methanol -s 192.168.30.235:5555
ERROR: Test installation error: Failed to create directory(/data/lava-android-test/share/installed-tests/methanol) for test(methanol)
LAVA: --Start Operation: Install test (methanol) on device(192.168.30.235:5555)

Revision history for this message
Soumya Basak (soumya-basak) wrote :

the same behaviour is observed with the builds:
image url: https://android-build.linaro.org/jenkins/job/linaro-android_panda-jb-gcc47-tilt-tracking-blob/165/
job id: http://validation.linaro.org/lava-server/scheduler/job/44511
complete log: http://validation.linaro.org/lava-server/scheduler/job/44511/log_file

on initial attempt of image deployment is failed on the device is getting from the log:

Connecting to 192.168.1.10:80... failed: Network is unreachable.

bzip2: Compressed file ends unexpectedly;
 perhaps it is corrupted? *Possible* reason follows.
bzip2: Invalid argument
 Input file = (stdin), output file = (stdout)

It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

tar: Child returned status 2
tar: Error is not recoverable: exiting now

also 0xbenchmark is failed to complete, mmtest, monketrunner, wifi test failed on the image.

Revision history for this message
Soumya Basak (soumya-basak) wrote :

we file new bugs for each test cases (1 bug = 1 test-case),

for example: https://bugs.launchpad.net/linaro-android/+bug/1190941

so, make this bug "invalid"/ "Wont Fix".

Revision history for this message
Milosz Wasilewski (mwasilew) wrote :

Still failing tests:
mmtest: https://bugs.launchpad.net/bugs/1191668
monkeyrunner: https://bugs.launchpad.net/bugs/1191719 https://bugs.launchpad.net/bugs/1191721
Other tests are passing now. Closing.

Changed in linaro-android:
status: New → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.