DCI Debug for UpSquared

jwessel
jwessel New Member Posts: 11

Does the DCI debug with the Intel System Studio work on the Up Sqaured Apollo Lake board?

I installed the latest BIOS (UP-APL01 R4.0). And went to CRB- Setup->CRB Chipset->South Cluster->Miscellaneous Configuration->DCI enable (HDCIEN) and set it to Enabled.

After saving, powering off and attaching the USB 3 debug cable between the Up Squared board and a Linux host running the Intel System Studio, I can connect and reset the board in the debugger and watch it reboot. I cannot however, get it to halt. I just get "WARNING: attempting to halt an unresponsive target, this operation may fail..."

What steps are missing to enable the debugging to work correctly?

Comments

  • Dan O'Donovan
    Dan O'Donovan Administrator, Moderator, Emutex Posts: 241 admin

    Hi @jwessel

    I tried briefly to enable DCI debug in the past with UP Squared, but encountered the same issues as you.

    In the end, I had the best results when using a special debug build of Intel's UEFI BIOS firmware for UP Squared:
    https://firmware.intel.com/projects/IntelAtomProcessorE3900
    (Look for the BIOS image named "UPBOARDA.X64.0071.D01.1809030917.bin")

    With that BIOS (which runs painfully slowly, by the way), I was able to boot as far as the EFI shell and then I stopped it there and successfully connected ISS. At that point, I could halt/resume the processor, dump registers, access the PCIe CSRs, etc.

    Please be aware that you might end up bricking the board if you try to restore the normal BIOS from AAEON again later. That's what happened me anyway when I tried to restore it. In that case, its good to have a SPI flash chip programmer nearby to reprogram the BIOS flash chip directly (its located under the heatsink).

    I'm sure it should be easier than that, but that's as far as I got with it at the time. If no-one else replies here, it may be worth raising the question on one of the relevant Intel community support forums.

    I hope that helps.

    Best regards,
    -Dan

  • jwessel
    jwessel New Member Posts: 11
    edited May 2019

    Thank for you the information. It sounds like I need to make a special cable if I am going to want to put various different BIOS on to the board and be able to recover, unless it is possible to use one of the "chip clips" to program the SPI flash after removing the heatsink. I saw a link to making a custom cable here: https://wiki.up-community.org/BIOS_chip_flashing_on_UP_Squared

    I'd like to see if we can get this feature working as an option with in the production BIOS so it can be accessible to anyone to wants to obtain an inexpensive USB 3 debug cable and install/configure ISS.

    It will take some time (possibly several weeks) before I can fully circle back to this because it looks fairly complex. I saw the code was available for the debug BIOS you referenced. Did you happen to know if the Release BIOS posted on the same page works? If not, that is probably the reason the production BIOS does not work.

    Either way if it works in one place, I would like to figure out a path to work with the production BIOS. Is there a way I can rebuild the latest production BIOS? For the Intel build you referenced, I found the instructions for the rebuild here: https://firmware.intel.com/sites/default/files/intelatome3900-0.71-buildinstructions.txt

  • jwessel
    jwessel New Member Posts: 11

    @Dan O'Donovan. Thank you for the pointers. I built a cable for the flash programmer today and used it to flash the images directly.

    The debug bios worked and it was extremely slow (nearly impossible to get into setup). I tried the release BIOS from the Intel site and the DCI debugging worked as well. In fact, the BIOS image I built from source also worked.

    I could not find any instructions for how to rebuild the production BIOS for the board so I am not sure how to further look into why the DCI debugging does not work on the production BIOS. It is also not clear to me what is functionally different between the Intel version and the production version. I haven't tried GPIOs or anything of that nature at the moment. I had only tried the kernel debugging.

    Any insights as to how to proceed are welcome.

  • Dan O'Donovan
    Dan O'Donovan Administrator, Moderator, Emutex Posts: 241 admin

    Hi @jwessel

    That's good to hear that the DCI debug worked with the release BIOS from Intel.

    Unfortunately, the production BIOS from AAEON is a closed-source BIOS. I think a large part of it is licensed from AMI, so I don't think you'll be able to get source code for that. However, you could certainly ask AAEON to investigate why DCI debugging is not working with the production BIOS.

    Can you tell us what goals you are ultimately hoping to achieve with the DCI debug functionality?

    Best regards,
    -Dan

  • fuzz
    fuzz New Member Posts: 4

    I'm also interested in DCI support. I have an N4200 and it seems the intel firmware won't work on it.

    It looks like the 0xC80 MSR is locked and doesn't have debugging bit enabled.

    There don't seem to be hidden BIOS parameters to enable silicon debugging from looking at UEFITool dump of the Setup section and IFRExtractor.

  • fuzz
    fuzz New Member Posts: 4

    I've located where the firmware sets the 0xc80 MSR. It's in the SiInit PEI module. It happens just after VT and debug trace init. I think the MSR can be unlocked and silicon debug enabled with modified NVRAM settings.

  • DownSquared
    DownSquared New Member Posts: 1
    edited March 2020

    I am confused… In the wiki for flashing the bios at:
    https://wiki.up-community.org/BIOS_chip_flashing_on_UP_Squared

    It says “This [bios] image is exactly 8388608 bytes in size” but then all tutorial screenshots show a bios image of size 16777216 bytes ! And the W25Q128.W flash memory is 16MB .

    The intel binary UPBOARDA.X64.0071.D01.1809030917.bin from intel is 8MB

    but my board, the "UP_APL01 A1.0_0_0" with an E3950 CPU has the W25Q128.W with 16MB.
    I already had another N4200 CPU UP2 board and I bought the E3950's specifically for this project.

    Was this flash memory chip changed from a previous board revision or do I need to get a specific board?

    @Dan O'Donovan @jwessel what flash chip do you have / board model / CPU ?

    Thank you for any help with this.

  • DCleri
    DCleri Administrator, AAEON Posts: 1,213 admin

    Hi @DownSquared

    The image we provide is 16MB, while Intel one is 8MB.

    Via dediprog you can erase the chip and flash a smaller BIOS, like the one from Intel.

    The official BIOS for UP Squared are available from our download section (please select the relevant platform): https://downloads.up-community.org/

  • deimos
    deimos New Member Posts: 12

    I believe the answer to the original question was said by fuzz: you also need to modify the IA32_DEBUG_INTERFACE MSR to enable CPU halting. Unfortunately, based on results from UEFITool and IFRExtractor, it does not look like there is the typical EFIvar that you can set to tell the BIOS to set this MSR this way, so yes, you will need to modify/change the bios manually.

    As for the 8MB vs 16MB bios, I am perplexed by this as well. Perhaps I need to study up how this works exactly, but for the moment I just want flashrom to be able to flash the 8MB image. It cant because of the size mismatch....

  • deimos
    deimos New Member Posts: 12
    edited May 2020

    I tried padding the end of the file with 0xFF bytes....using the release build that intel pre-built. Doesnt seem to work....

    EDIT: nvm, It worked! Just took a minute to boot up.

  • thumper
    thumper New Member Posts: 5

    @Dan O'Donovan said:

    In the end, I had the best results when using a special debug build of Intel's UEFI BIOS firmware for UP Squared:
    https://firmware.intel.com/projects/IntelAtomProcessorE3900
    (Look for the BIOS image named "UPBOARDA.X64.0071.D01.1809030917.bin")

    I just tried doing this myself with no apparent luck. I copied the image and firmware tool to a thumbdrive (FAT32 filesystem), booted the EFI shell, and ran the flashing tool (FirmwareUpdateX64.efi). The first time I tried, I got a "SPI Protocol not found" error. I tried again and was immediately returned to the EFI shell prompt. The BIOS never flashed. I've rebooted and retried several times, but I keep just getting immediatly returned to the EFI shell.

    My UP Squared board is "Fab C", and this image does state Fab A, so might that be the problem? Otherwise, is it even possible to re-flash the native UP Squared BIOS with this Intel image using the Intel tool, or am I required to use an SPI programming device? I did see Dan referencing SPI tools, so maybe he used them, and just didn't mention it explicitly?

    On a side note, I am running BIOS v21 and I have DCI debugging enabled in my BIOS, but have had no luck attaching to it. I did find a blog that referenced the BIOS history (https://wiki.up-community.org/UP_Squared_BIOS_History) which indicated DCI debugging was actually not enabled until BIOS v31, and then removed in v37. If this is true, it might explain why different people can get debugging to work, and others can't. Why the option would be visible in the BIOS when it's not active, I don't understand.

  • DCleri
    DCleri Administrator, AAEON Posts: 1,213 admin

    Hi @thumper

    The feature was removed after some instabilities and other limitations where experienced when DCI debug was enabled on a standard BIOS.

    Recently with UP Xtreme we released 2 version of the same BIOS one used by default on our boards and one with DCI debug enabled.

    We are considering to do the same also for UP Squared, but I don't have a date yet. I will keep you posted.

  • thumper
    thumper New Member Posts: 5

    Hey @DCleri,

    Thanks for the info and quick response. Much appreciated. Definitely keep me posted on the BIOS status.

    Do you happen to know if the Intel image I referenced can be flashed onto the UP^2 board using the Intel Firmware Update Utility that can also be downloaded from the same link? Some articles I read seemed to indicate there may be more involved than just running the Intel Flash utility if I want to jump from the native UP^2 BIOS to the Intel BIOS, but I'm not clear on that.

    Thanks again for the help and info.

  • deimos
    deimos New Member Posts: 12

    @thumper, What exactly does the "Fab" mean and how do you know that yours is "C"? How can I determine what mine is?

    Yes, you can flash the referenced intel firmware image, thought I don't know if you can use Intel Firmware Update Utility; I imagine that you could. I ran into the same issues as you when trying to use FirmwareUpdateX64.efi tool. It appears that the stock UP2 firmware doesn't expose a SPI Protocol for that EFI Application to use.

    In order to flash the Intel pre-built (and later on I built my own) images, I used a hardware SPI programmer. I used the "flashrom" utility in Ubuntu to write the contents. Flashrom complained since the image is 8MB and the chip is 16MB. I appended 8MB's of 0xFF bytes to the image to make it 16MB and flashrom started writing. This was kind of a guess as I'm not 100% sure if this would work. It did work and then later on I can use the FirmwareUpdateX64.efi application from the EFI shell since this bios supports the SPI protocol.

    Hope this helps. Also, be patient when booting a debug image, it can take a while. I thought I bricked my device at first but so far haven't in my experiments.

  • thumper
    thumper New Member Posts: 5

    Hey @deimos,

    Thanks for the info. I may bite the bullet and go the hardware SPI programmer route myself.

    The Fab version can be found on the Main tab of the "advanced" or engineering/development version of the BIOS (not sure what to call it). It will give you a Fab ID. As far as what it means, I don't know for sure. Though if I had to venture a guess, I would say it is probably the PCB artwork revision.

  • deimos
    deimos New Member Posts: 12

    I use a ch341Apro with a 1.8V adapter. For this SPI chip specifically, flashrom has been very weird. I had to update flashrom manually to get it to recognize the chip and even then it seems unstable. Just had many issues with detecting the chip and making sure the connection was flawless. Tried going through the header pins but that didn't work, had to use a clamp thingy. I think they are called "soic 8 clamp". This is why its nice to only need it once; after that you can use the FirmwareUpdateX64.efi.

  • thumper
    thumper New Member Posts: 5
    edited September 2020

    Got the Intel BIOS flashed. We had to remove the chip and program out of circuit, but it is booting the new BIOS now. Still not able to connect to the processor with the USB 3.0 cable. I am assuming that it does not matter which USB port I connect the USB 3.0 debug cable to on the UP Squared?

  • thumper
    thumper New Member Posts: 5

    OK. Did some more reading and according to Intel (https://www.mouser.com/datasheet/2/612/100-series-chipset-datasheet-vol-1-775591.pdf Section 33.3)

    "Direct Connect Interface (DCI) is a new debug transport technology to enable closed
    chassis debug through any of USB3 ports out from Intel silicon. "

    So I'll keep digging.