Mmc0 Timeout issues on Ubuntu 18.04 Kernel 4.15.0-37-generic

sossont
sossont New Member Posts: 1

Hello

I have an UP Squared which I installed Ubuntu 18.04. I have completed the install instructions including putting on the UP Kernel. But in dmesg I am getting mmc0 timeout issues. I see this was an issue in 2017, and unrecolved. Anyone got any ideas where i should be looking to try solve this?

Here is the error:
(Dmesg)

[ 23.262038] mmc0: Timeout waiting for hardware interrupt.
[ 23.262052] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
[ 23.262061] mmc0: sdhci: Sys addr: 0x00000020 | Version: 0x00001002
[ 23.262068] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000020
[ 23.262075] mmc0: sdhci: Argument: 0x0cc9b800 | Trn mode: 0x0000003b
[ 23.262082] mmc0: sdhci: Present: 0x1fff0001 | Host ctl: 0x0000003d
[ 23.262088] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000080
[ 23.262095] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00000007
[ 23.262102] mmc0: sdhci: Timeout: 0x00000004 | Int stat: 0x00000000
[ 23.262108] mmc0: sdhci: Int enab: 0x02ff000b | Sig enab: 0x02ff000b
[ 23.262115] mmc0: sdhci: AC12 err: 0x00000000 | Slot int: 0x00000000
[ 23.262122] mmc0: sdhci: Caps: 0x546ec881 | Caps_1: 0x80000807
[ 23.262128] mmc0: sdhci: Cmd: 0x0000123a | Max curr: 0x00000000
[ 23.262148] mmc0: sdhci: Host ctl2: 0x0000008d
[ 23.262155] mmc0: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x000000026e37f200
[ 23.262159] mmc0: sdhci: ============================================
[ 23.262689] mmcblk0: error -110 sending stop command, original cmd response 0x0, card status 0x400900
[ 23.262697] mmcblk0: error -110 transferring data, sector 214546432, nr 32, cmd response 0x0, card status 0x0

Kernel
$ uname -a
Linux somyth 4.15.0-37-generic #40~upboard06-Ubuntu SMP Wed Nov 27 11:29:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

More debug
$ dmesg | grep mmc
[ 2.758403] mmc0: SDHCI controller on PCI [0000:00:1c.0] using ADMA 64-bit
[ 2.762456] mmc1: SDHCI controller on PCI [0000:00:1e.0] using ADMA 64-bit
[ 2.914288] mmc0: new HS400 MMC card at address 0001
[ 2.919469] mmcblk0: mmc0:0001 DF4128 116 GiB
[ 2.920835] mmcblk0boot0: mmc0:0001 DF4128 partition 1 4.00 MiB
[ 2.922162] mmcblk0boot1: mmc0:0001 DF4128 partition 2 4.00 MiB
[ 2.923450] mmcblk0rpmb: mmc0:0001 DF4128 partition 3 4.00 MiB, chardev (244:0)
[ 2.928668] mmcblk0: p1 p2
[ 3.838087] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
[ 4.679127] EXT4-fs (mmcblk0p2): re-mounted. Opts: errors=remount-ro
[ 23.262038] mmc0: Timeout waiting for hardware interrupt.
[ 23.262052] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
[ 23.262061] mmc0: sdhci: Sys addr: 0x00000020 | Version: 0x00001002
[ 23.262068] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000020
[ 23.262075] mmc0: sdhci: Argument: 0x0cc9b800 | Trn mode: 0x0000003b
[ 23.262082] mmc0: sdhci: Present: 0x1fff0001 | Host ctl: 0x0000003d
[ 23.262088] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000080
[ 23.262095] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00000007
[ 23.262102] mmc0: sdhci: Timeout: 0x00000004 | Int stat: 0x00000000
[ 23.262108] mmc0: sdhci: Int enab: 0x02ff000b | Sig enab: 0x02ff000b
[ 23.262115] mmc0: sdhci: AC12 err: 0x00000000 | Slot int: 0x00000000
[ 23.262122] mmc0: sdhci: Caps: 0x546ec881 | Caps_1: 0x80000807
[ 23.262128] mmc0: sdhci: Cmd: 0x0000123a | Max curr: 0x00000000
[ 23.262148] mmc0: sdhci: Host ctl2: 0x0000008d
[ 23.262155] mmc0: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x000000026e37f200
[ 23.262159] mmc0: sdhci: ============================================
[ 23.262689] mmcblk0: error -110 sending stop command, original cmd response 0x0, card status 0x400900
[ 23.262697] mmcblk0: error -110 transferring data, sector 214546432, nr 32, cmd response 0x0, card status 0x0

Comments

  • DCleri
    DCleri Administrator, AAEON Posts: 1,213 admin

    Can you try to re-install the system or try with Ubuntu 16.04 if it still occurs?

    What version of UP Squared you have and what is the setup?

    -BIOS version
    -CPU
    -RAM
    -EMMC
    -Power Supply

  • yarmooh
    yarmooh New Member Posts: 7

    Hello,

    We're having the same issue on our UpSquared devices. We've got plenty of them, dispatched through our customers, and some of them started to report similar issues;

    [ 129.173111] mmc0: Timeout waiting for hardware interrupt.
    [ 129.173134] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
    [ 129.173147] mmc0: sdhci: Sys addr: 0x00000018 | Version: 0x00001002
    [ 129.173159] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000018
    [ 129.173170] mmc0: sdhci: Argument: 0x0255fa18 | Trn mode: 0x0000003b
    [ 129.173182] mmc0: sdhci: Present: 0x1fff0001 | Host ctl: 0x0000003d
    [ 129.173193] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000080
    [ 129.173204] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00000007
    [ 129.173216] mmc0: sdhci: Timeout: 0x00000004 | Int stat: 0x00000000
    [ 129.173227] mmc0: sdhci: Int enab: 0x02ff000b | Sig enab: 0x02ff000b
    [ 129.173238] mmc0: sdhci: AC12 err: 0x00000000 | Slot int: 0x00000000
    [ 129.173249] mmc0: sdhci: Caps: 0x546ec881 | Caps_1: 0x80000807
    [ 129.173261] mmc0: sdhci: Cmd: 0x0000123a | Max curr: 0x00000000
    [ 129.173293] mmc0: sdhci: Host ctl2: 0x0000008d
    [ 129.173305] mmc0: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x000000003ebcd200
    [ 129.173313] mmc0: sdhci: ============================================
    [ 129.173659] mmcblk0: error -110 sending stop command, original cmd response 0x0, card status 0x400900
    [ 129.173666] mmcblk0: error -110 transferring data, sector 39189016, nr 24, cmd response 0x0, card status 0x0
    [ 600.226392] perf: interrupt took too long (2523 > 2500), lowering kernel.perf_event_max_sample_rate to 79250
    [ 1425.649970] mmc0: Timeout waiting for hardware interrupt.
    [ 1425.649993] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
    [ 1425.650006] mmc0: sdhci: Sys addr: 0x00000008 | Version: 0x00001002
    [ 1425.650018] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000008
    [ 1425.650031] mmc0: sdhci: Argument: 0x029b9c58 | Trn mode: 0x0000003b
    [ 1425.650042] mmc0: sdhci: Present: 0x1fff0001 | Host ctl: 0x0000003d
    [ 1425.650053] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000080
    [ 1425.650064] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00000007
    [ 1425.650075] mmc0: sdhci: Timeout: 0x00000004 | Int stat: 0x00000000
    [ 1425.650087] mmc0: sdhci: Int enab: 0x02ff000b | Sig enab: 0x02ff000b
    [ 1425.650098] mmc0: sdhci: AC12 err: 0x00000000 | Slot int: 0x00000000
    [ 1425.650109] mmc0: sdhci: Caps: 0x546ec881 | Caps_1: 0x80000807
    [ 1425.650121] mmc0: sdhci: Cmd: 0x0000123a | Max curr: 0x00000000
    [ 1425.650153] mmc0: sdhci: Host ctl2: 0x0000008d
    [ 1425.650164] mmc0: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x000000003ebcd200
    [ 1425.650173] mmc0: sdhci: ============================================
    [ 1425.650538] mmcblk0: error -110 sending stop command, original cmd response 0x0, card status 0x400900
    [ 1425.650545] mmcblk0: error -110 transferring data, sector 43752536, nr 8, cmd response 0x0, card status 0x0

    Our build is running same Ubuntu, same kernel.
    BIOS: UPA1AM42
    CPU: Intel® Celeron® Dual Core N3350 up to 2.4GHz
    RAM: 2GB DDR4
    EMMC: builtin 32GB eMMC

    We'd like to get information how to fix that issue, is this hardware related, or some bug in the software/bios/kernel.

    @DCleri

  • yarmooh
    yarmooh New Member Posts: 7

    Hello,

    We're having the same issue like the original createor. We've got plenty of devices running UpSquared, dispatched to our customers all around the world, and some of them started to report this mmc0 issue:

    [ 129.173111] mmc0: Timeout waiting for hardware interrupt.
    [ 129.173134] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
    [ 129.173147] mmc0: sdhci: Sys addr: 0x00000018 | Version: 0x00001002
    [ 129.173159] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000018
    [ 129.173170] mmc0: sdhci: Argument: 0x0255fa18 | Trn mode: 0x0000003b
    [ 129.173182] mmc0: sdhci: Present: 0x1fff0001 | Host ctl: 0x0000003d
    [ 129.173193] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000080
    [ 129.173204] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00000007
    [ 129.173216] mmc0: sdhci: Timeout: 0x00000004 | Int stat: 0x00000000
    [ 129.173227] mmc0: sdhci: Int enab: 0x02ff000b | Sig enab: 0x02ff000b
    [ 129.173238] mmc0: sdhci: AC12 err: 0x00000000 | Slot int: 0x00000000
    [ 129.173249] mmc0: sdhci: Caps: 0x546ec881 | Caps_1: 0x80000807
    [ 129.173261] mmc0: sdhci: Cmd: 0x0000123a | Max curr: 0x00000000
    [ 129.173293] mmc0: sdhci: Host ctl2: 0x0000008d
    [ 129.173305] mmc0: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x000000003ebcd200
    [ 129.173313] mmc0: sdhci: ============================================
    [ 129.173659] mmcblk0: error -110 sending stop command, original cmd response 0x0, card status 0x400900
    [ 129.173666] mmcblk0: error -110 transferring data, sector 39189016, nr 24, cmd response 0x0, card status 0x0
    [ 600.226392] perf: interrupt took too long (2523 > 2500), lowering kernel.perf_event_max_sample_rate to 79250
    [ 1425.649970] mmc0: Timeout waiting for hardware interrupt.
    [ 1425.649993] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
    [ 1425.650006] mmc0: sdhci: Sys addr: 0x00000008 | Version: 0x00001002
    [ 1425.650018] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000008
    [ 1425.650031] mmc0: sdhci: Argument: 0x029b9c58 | Trn mode: 0x0000003b
    [ 1425.650042] mmc0: sdhci: Present: 0x1fff0001 | Host ctl: 0x0000003d
    [ 1425.650053] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000080
    [ 1425.650064] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00000007
    [ 1425.650075] mmc0: sdhci: Timeout: 0x00000004 | Int stat: 0x00000000
    [ 1425.650087] mmc0: sdhci: Int enab: 0x02ff000b | Sig enab: 0x02ff000b
    [ 1425.650098] mmc0: sdhci: AC12 err: 0x00000000 | Slot int: 0x00000000
    [ 1425.650109] mmc0: sdhci: Caps: 0x546ec881 | Caps_1: 0x80000807
    [ 1425.650121] mmc0: sdhci: Cmd: 0x0000123a | Max curr: 0x00000000
    [ 1425.650153] mmc0: sdhci: Host ctl2: 0x0000008d
    [ 1425.650164] mmc0: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x000000003ebcd200
    [ 1425.650173] mmc0: sdhci: ============================================
    [ 1425.650538] mmcblk0: error -110 sending stop command, original cmd response 0x0, card status 0x400900
    [ 1425.650545] mmcblk0: error -110 transferring data, sector 43752536, nr 8, cmd response 0x0, card status 0x0

    We'd like to know if this is hardware related, or some software/bios/kernel issue. We're using the same Ubuntu and same kernel version.
    BIOS: UPA1AM42
    CPU: Intel® Celeron® Dual Core N3350 up to 2.4GHz,
    RAM: 2GB DDR4
    eMMC: 32GB

  • DCleri
    DCleri Administrator, AAEON Posts: 1,213 admin

    What steps can be followed to reproduce the issue?

    Something you can try at first is to update the BIOS to the latest version available (5.2) and also the kernel to either 5.0 or better the recently released 5.4: https://wiki.up-community.org/Ubuntu_18.04#Install_Ubuntu_kernel_5.4.0_from_PPA_on_Ubuntu_18.04

  • yarmooh
    yarmooh New Member Posts: 7

    Hello,

    Setup uses Apache2,PHP, and Postgresql db, however since these are different customers I honestly don't see any common steps which can reproduce that.
    I'll try with the BIOS, and already upgraded the kernel to 5.4 - will check if this changes anything.