Ubuntu
linux package

Kernel crash when mounting bcache on a vm with virtio

Bug #1359146 reported by Kick In on 2014-08-20

This bug affects 1 person

Affects		Status	Importance	Assigned to
linux (Ubuntu)	Fix Released	Medium	Chris J Arges	Ubuntu ubuntu-14.10
Trusty	Invalid	Undecided	Unassigned
Utopic	Fix Released	Medium	Chris J Arges	Ubuntu ubuntu-14.10

Bug Description

bcache used in a virtual machine:

If you create a bcache device with caching device and backing device on virtio disks, it will let you do your work until you mount it where it crashes the virtual machine (virtual machine is utopic, host is trusty).

For example:
if your backing device is /dev/vdb, and your caching device is /dev/vdc:

sudo make-bcache -B /dev/vdc -C /dev/vdb --writeback
is ok

mkfs.ext4 /dev/bcache0 -L bcache-test
is ok

but
mount /dev/bcache0 /media/bcache

make it crash, if I remember well one time I had a few seconds before crash, but I don't have any log files.

qemu-system-x86 is version 2.0.0+dfsg-2ubuntu1.2

Re-installing, and using IDE virtual interface seems to create and mount the bcache device is ok.

After bcache created an mounted successfully with IDE interfaces, switching to SATA, SCSI, and USB with booting and mounting seemed ok.

See original description

Tags: patch bug-exists-upstream kernel-daily-bug

Kick In (kick-d) on 2014-08-20

description:	updated
description:	updated

Revision history for this message 👁 Image

Brad Figg (brad-figg) wrote on 2014-08-20: Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1359146

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status:	New → Incomplete

Kick In (kick-d) on 2014-08-20

description:	updated
Changed in linux (Ubuntu):
status:	Incomplete → Confirmed

Revision history for this message 👁 Image

Joseph Salisbury (jsalisbury) wrote on 2014-08-20:

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.17 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.17-rc1-utopic/

Changed in linux (Ubuntu):
importance:	Undecided → Medium

Revision history for this message 👁 Image

Joseph Salisbury (jsalisbury) wrote on 2014-08-20:

Also, was there a prior release that did not exhibit this bug?

Revision history for this message 👁 Image

Kick In (kick-d) wrote on 2014-08-20: Re: [Bug 1359146] Re: Kernel crash when mounting bcache on a vm with virtio

I'll try upstream kernel, and I will try previous release. Only utopic
kernel releases, or also trusty ones?
Le 20/08/2014 18:07, Joseph Salisbury a écrit :
> Also, was there a prior release that did not exhibit this bug?
>

Revision history for this message 👁 Image

Kick In (kick-d) wrote on 2014-08-21:

I've tested with upstream kernel, and I've got also an hang of the virtual machine.
kernel tested is :
linux-image-3.17.0-031700rc1

Revision history for this message 👁 Image

Kick In (kick-d) wrote on 2014-08-21:

The virtual machine hangs some seconds after mount command of bcache device.

Revision history for this message 👁 Image

Kick In (kick-d) wrote on 2014-08-21:

At boot ime after crash I got:
http://paste.ubuntu.com/8106758/
trace begins at line 466

Revision history for this message 👁 Image

Stefan Bader (smb) wrote on 2014-08-22:

Right now 3.17-rc1 would show the virtio related issue only. With the current 3.16 we have there is still the writeback hung task issue mingled in (except with a kernel version of 3.16.0-10.15 or later), see bug #1357295.
Using Xen paravirt drives, there is no crash. This hints that it is something directly related to the combination of virtio disks and bcache. This looks to be an upstream problem still. So for this bug we should concentrate on either 3.17 or the latest Utopic kernel.

Chris J Arges (arges) on 2014-08-22

tags:

added: bug-exists-upstream

Revision history for this message 👁 Image

Chris J Arges (arges) wrote on 2014-08-22:

I can easily reproduce the issue; and it seems that this is a regression between 3.13 (where this works) and 3.16.
I'll work on doing a regression bisect. It may end up being one or two issues as there is a warn on boot before mounting, and when mounting /dev/bcache0 we get the crash.

Changed in linux (Ubuntu Trusty):
status:	New → Fix Released
Changed in linux (Ubuntu):
assignee:	nobody → Chris J Arges (arges)

Revision history for this message 👁 Image

Chris J Arges (arges) wrote on 2014-08-23:

#10

I did a bisect and found 05f1dd5315217398fc8d122bdee80f96a9f21274 to be the first bad commit.
I reverted this from 3.17, and found that it did resolve the issue. I'll continue to investigate and see if an additional patch can fix this issue instead of a revert.

Revision history for this message 👁 Image

Ming Lei (tom-leiming) wrote on 2014-09-01:

#11

block: merge segment in need Edit (1.6 KiB, text/plain)

It should be a block-core problem introduced by this commit 05f1dd531521, anyone care to test the
attached patch?

Ubuntu Foundations Team Bug Bot (crichton) on 2014-09-01

tags:

added: patch

Revision history for this message 👁 Image

Chris J Arges (arges) wrote on 2014-09-02:

#12

@tom-leiming

I've tested this patch and I am able to mount a bcache device on top of virtio disks without hangs or the original BUG_ON on startup. Please add my 'Tested-By' in the patch upstream.

Thanks,

Revision history for this message 👁 Image

Chris J Arges (arges) wrote on 2014-09-03:

#13

Ming Lei's patch is here:
https://lkml.org/lkml/2014/9/2/307

Changed in linux (Ubuntu):
status:	Confirmed → In Progress

Revision history for this message 👁 Image

Ming Lei (tom-leiming) wrote on 2014-09-14:

#14

merged to upstream as commit:

https://github.com/torvalds/linux/commit/0738854939e6ec9b9111a8cfc0ca1dfa3cff6b2e

Tim Gardner (timg-tpi) on 2014-09-15

Changed in linux (Ubuntu Utopic):
status:	In Progress → Fix Committed
Changed in linux (Ubuntu Trusty):
status:	Fix Released → Invalid

Revision history for this message 👁 Image

Launchpad Janitor (janitor) wrote on 2014-09-18:

#15

This bug was fixed in the package linux - 3.16.0-16.22

---------------
linux (3.16.0-16.22) utopic; urgency=low

[ Andy Whitcroft ]

  * Revert "SAUCE: x86/xen: Fix setup of 64bit kernel pagetables"
  * [Config] tools -- only build common tools when enabled
  * [Config] follow rename of DEB_BUILD_PROFILES

[ Tim Gardner ]

  * [Debian] set do_*_tools after stage1 or bootstrap is determined
    - LP: #1370211
  * Release Tracking Bug
    - LP: #1370535

[ Upstream Kernel Changes ]

  * x86/xen: don't copy bogus duplicate entries into kernel page tables
  * blk-merge: fix blk_recount_segments
    - LP: #1359146
  * igb: bring link up when PHY is powered up
    - LP: #1370018
  * igb: remove unnecessary break after goto
    - LP: #1370018
  * igb: remove unnecessary break after return
    - LP: #1370018
  * igb: Add message when malformed packets detected by hw
    - LP: #1370018
  * igb: bump igb version to 5.2.13
    - LP: #1370018
-- Tim Gardner <email address hidden> Tue, 16 Sep 2014 10:19:04 -0600