Xperia, FOTA format

Long time no see. I got new Sony Xperia Z Tablet (WiFi, not LTE one) to play with and I thought that it would be nice device to run our openSUSE with my Enlightenment on that to combine cool hardware and cool software.

It seems that there is not much project targeted on running full blown Linux distribution on Android devices. You can meet Replicant (which is great anyway), you can meet Ubuntu Touch but that is almost all. So I decided to revive my old blog and write here my notes.

I played in past with HTC Desire Z with the same desire, but as I had very little time in the end, I didn't progressed significantly beyond interesting Hackweek project. I hope that this time it will do better and at least it is not my only mobile I'd wreck :)

I started with rooting the device, which was very simple thanks to the article on unlockr and the work of DooMLoRD. I prepared then openSUSE chroot on microSD card, automated chrooting process and installed some basic tools (gcc, git, mosh + ssh) and E17. To see I have working installation I used Android's native XWindow implementation.

To boot openSUSE I'd need to boot kernel with something like

root=/dev/block/mmcblk1p4

(where my openSUSE partition is) or even provide my initramfs which will take care of mounting and other needed stuff. To be able to boot without bricking I need to find way how to provide kernel to bootloader without changing contents of flash.

fastboot

should do the job (at least it did with my HTC Desire Z attempts).

It seems that I can't influence kernel parameters for now so let's build own kernel with root device settings hardcoded. Sony provides repository with kernel source on github. Kernel configuration was not present so I took the one from Cyanogenmod project.

Let's have a look on internal flash to find some more information:

# fdisk -l /dev/block/mmcblk0
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/block/mmcblk0: 15.8 GB, 15758000128 bytes, 30777344 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: gpt


#         Start          End    Size  Type            Name
 1          256         4351      2M  unknown         TA
 2         4352         4607    128K  unknown         sbl1
 3         4608         5119    256K  unknown         sbl2
 4         5120         5631    256K  unknown         s1sbl2
 5         5632         6655    512K  unknown         sbl3
 6         6656         7679    512K  unknown         aboot
 7         7680         8703    512K  unknown         tz
 8         8704         8959    128K  unknown         alt_sbl1
 9         8960         9471    256K  unknown         alt_sbl2
10         9472         9983    256K  unknown         alt_s1sbl2
11         9984        11007    512K  unknown         alt_sbl3
12        11008        12031    512K  unknown         alt_aboot
13        12032        13055    512K  unknown         alt_tz
14        13056        14079    512K  unknown         rpm
15        14080        15103    512K  unknown         alt_rpm
16        16384        49151     16M  Linux filesyste LTALabel
17        49152        90111     20M  unknown         boot
18        90112        91671    780K  unknown         modemst1
19        94208        95767    780K  unknown         modemst2
20        98304        99863    780K  unknown         fsg
21        99864       110103      5M  unknown         ramdump
22       110104       126487      8M  Linux filesyste apps_log
23       126488       159255     16M  unknown         FOTAKernel
24       159744      4354047      2G  Linux filesyste system
25      4354048      5480447    550M  Linux filesyste cache
26      5480448      6266879    384M  Linux filesyste B2B
27      6266880     30535646   11.6G  Linux filesyste userdata

And some basic identification of the partitions:

# file -s /dev/block/mmcblk0p{1..27}
/dev/block/mmcblk0p1:  data
/dev/block/mmcblk0p2:  data
/dev/block/mmcblk0p3:  data
/dev/block/mmcblk0p4:  data
/dev/block/mmcblk0p5:  data
/dev/block/mmcblk0p6:  Hitachi SH big-endian COFF object, not stripped
/dev/block/mmcblk0p7:  data
/dev/block/mmcblk0p8:  data
/dev/block/mmcblk0p9:  data
/dev/block/mmcblk0p10: data
/dev/block/mmcblk0p11: data
/dev/block/mmcblk0p12: Hitachi SH big-endian COFF object, not stripped
/dev/block/mmcblk0p13: data
/dev/block/mmcblk0p14: data
/dev/block/mmcblk0p15: data
/dev/block/mmcblk0p16: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (extents) (large files)
/dev/block/mmcblk0p17: ELF 32-bit LSB executable, ARM, version 1, statically linked, stripped
/dev/block/mmcblk0p18: data
/dev/block/mmcblk0p19: data
/dev/block/mmcblk0p20: data
/dev/block/mmcblk0p21: data
/dev/block/mmcblk0p22: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (needs journal recovery) (extents) (large files)
/dev/block/mmcblk0p23: ELF 32-bit LSB executable, ARM, version 1, statically linked, stripped
/dev/block/mmcblk0p24: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (extents) (large files)
/dev/block/mmcblk0p25: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (needs journal recovery) (extents) (large files)
/dev/block/mmcblk0p26: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (extents) (large files)
/dev/block/mmcblk0p27: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (needs journal recovery) (extents) (large files)

I have read somewhere that there is different format used than boot.img. I also found that there is already tool for extracting this format. Unfortunately it didn't work for me.

Fortunately the format was ELF executable used as container and we have tools for analyzing or manipulating that. So, it seems that FOTAKernel is kernel and initramfs in single ELF executable in mmcblk0p23.

# readelf -a mmcblk0p23-FOTAKernel
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 61 00 00 00 00 00 00 00 00
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            ARM
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           ARM
  Version:                           0x1
  Entry point address:               0x80208000
  Start of program headers:          52 (bytes into file)
  Start of section headers:          7720040 (bytes into file)
  Flags:                             0x0
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         2
  Size of section headers:           40 (bytes)
  Number of section headers:         1
  Section header string table index: 0

Section Header:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]          LOUSER+53494e   00000000 75bc68 001000 00      0   0  0
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings)
  I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

There are no section groups in this file.

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x000074 0x80208000 0x80208000 0x581698 0x581698     0
  LOAD           0x58170c 0x82200000 0x82200000 0x1da55c 0x1da55c     0

There is no dynamic section in this file.

There are no relocations in this file.

No version information found in this file.

You can see two program headers, offsets and filesizes. First entry is kernel's. To extract kernel you can simply use dd:

$ dd if=/dev/block/mmcblk0p23 of=vmlinuz skip=$((0x74)) count=$((0x581698)) bs=1

To get to contents of initramfs you have to do some more steps:

$ dd if=/dev/block/mmcblk0p23 skip=$((0x58170c)) count=$((0x1da55c)) bs=1 | zcat | cpio -id

So, now I have original initramfs I can modify to boot from microSD card after all that HW specific black magic, I have kernel to use, so experiments with boot can start next time.

republished to get to planet.opensuse.org page

E17 0.17.4 EFL 1.7.8 update problem at start

For those, who are using E17 and run into problems after upgrade to E17 0.17.4 and EFL 1.7.8, you may find useful following script:

#!/bin/bash

TMP=$(mktemp)
TMP2=$(mktemp)
GOOD=1

use_one() {
    echo "$CFGFILE"
}

find_all() {
    find ~/.e/e/config -name e.cfg
}

if [ "$1" ]; then
    fn=use_one
    CFGFILE="$1"
else
    fn=find_all
fi

while read cfg; do
    eet -d "${cfg}" config "$TMP"

    # editation is simple
    sed 's/\([[:blank:]]*value "show_splash" int: \)1;$/\10;/' "$TMP" > "$TMP2"

    # now sanity checks
    # was there any change at all?
    if cat "$TMP" | md5sum -c <(cat "$TMP2" | md5sum ) &> /dev/null; then
        echo "${cfg} wasn't modified"
        continue
    fi

    # do we still have the same size? altering 1 --> 0 shouldn't change it
    if [ $(stat --format=%s "$TMP") != $(stat --format=%s "$TMP2") ]; then
        echo "Modifying ${cfg} caused size change, probably edited wrong :("
        echo "Skipping..."
        continue
    fi

    echo "${cfg} looks good, applying changes"
    eet -e "${cfg}" config "$TMP2" 1
done < <($fn)


rm "$TMP" "$TMP2"

It will disable splash screen on your configuration (all profiles) or one file if you specify it manually. Even though it should be harmless, you should still backup your configuration first, for example with:

cp -R ~/.e ~/.e-backup

Typical boot and cursor in console

If you have Netwalker and have you tried to switch to console (for example by pressing ctrl+alt+f1), you probably noticed that there is no cursor. Well friends, it's not bug, it's feature. This part of code in drivers/video/mxc/mxc_ipuv3_fb.c is self-explanatory...

#ifdef CONFIG_MACH_MX51_ERDOS
static int dmy_cursor(struct fb_info *info, struct fb_cursor *cursor)
{
/*
* don't display cursor for slash image.
*/
return 0;
}
#endif /* CONFIG_MACH_MX51_ERDOS */

Let's have cursor back with this patch. Well, with this you'll get cursor back, but what about removing SHARP logo? This one should give you typical Linux Tux logo instead. And for completely usual linux boot sequence we all love so much. To get all that informations running in startup, all errors and messages disable in kernel config System Type --> Freescale MXC Implementations --> MX51 Options --> printk suppress less KERN_ERR.

Good night!

Netwalker for Sleep_Walker

I recently bought new gadget - SHARP PC-Z1 Netwalker. For those who doesn't know this device it's something between netbook and PDA - good choice for every geek. It has Freescale i.MX515 ARMv7 processor with Cortex-A8 core, 512 (!) MB RAM, 4 GB internal flash storage, WiFi, 1024x600 display, touchscreen and two USB ports. With its 16 cm it can well fit my pockets. Manufacturer says that it should work about 10 hours for single charge. Cool isn't it?

One of first things I tried was to boot my Debian system I use in Treo680 and Centro and it just worked. So I prepared Debian rootfile system for you with ease. It is quite up-to-date Debian sid with Illume on top. You can find it here. I plan to use it for updates since aptitude works there fast and has no need to swap. I'll try it for building Gentoo soon.

It has Netbook remix of Ubuntu 9.04 Jaunty Jackalope out of box with GNOME environment, but I tried even KDE4 and I have to say it's great to touch applications you are using daily...

Since it has Linux inside, it also has available kernel sources, but I have to say that quality of code is really low. It often fail to build when you just change kernel configuration a bit. Good news is that Amit Kucheria from Canonical is working on adding support of i.MX chips into upstream ARM kernel.

I searched a lot for any community around this nice device but found nothing but yahoo group (intended more for users than developers) so I decided to create wiki page on Hack&Dev web as basic place for cooperation. I found some interested people on #ubuntu-arm@freenode, but it seems it will be long road to the bright future of this device. Fortunately it is quite usable even now using that kernel sources provided by manufacturer. Wiki page should contain now some basic information but I'll use it to track current status.

---

OK, this was written about two months ago. I tried some experiments with Netwalker - running Gentoo (it was quite great, but compilation on Netwalker sucks), Ubuntu 9.10 and 10.04 - it mostly sucked. I'm now using Debian with E17 and XFCE and I'm completely satisfied. Maybe I'll try openSUSE for ARM in future, but for now is Debian the most suitable distribution on Netwalker.

I had this week Hackweek 5 so I dig a bit to Netwalker's kernel and tried to port it to 2.6.31 Freescale's latest kernel. I found very awful stuff:

• Netwalker doesn't have it's own machine ID - it uses machine ID of Babbage board - one similar board.
• code is very hardly portable... there are #ifdef CONFIG_MACH_MX51_ERDOS spread in many files
  $ grep -Rl CONFIG_MACH_MX51_ERDOS /usr/src/linux-fsl-imx51-2.6.28 | wc -l
53

Try to avoid devices containing kernel code by Nissin Systems Co.. It can save you half year of work :)

Quick notes

As expected my son is occupying me a lot. But I had some Innovation Time Off accumulated at work so I worked on Palms a bit.

USB client is not working well for all PXA machines, but palmtreo.c code had missing some more functionality - connection worked only once when booting with cable attached. I fixed this with following marex's patch adding this support for PalmLD, PalmTX and PalmT5. It works now better, but it stops working after suspend/resume. I'll try to dig into code. I have found that GPIO90 works same way as GPIO1 - USB detection - weird.

Support for MAX8588 and PI2C was added and it seems to work. Funny about this is, that I tried to use CPU at 416 MHz in Linux at the same voltage as it was set in PalmOS for 312 MHz. I started to use Linux on 520 MHz and it works nice.

Camera differs from the ones found in other Palms (understand - I tried to follow marex's patch for Zire72 camera but it did not work :). I disassembled my Treo680 and tried to trace GPIOs. The only one new GPIO I have found is GPIO82. You can see photos here and here. I don't know currently what type it is and I didn't even found it on I2C bus yet. Interesting is that this module has 20 pins.

I also did some tiny code cleanups but I have to create patch set and push so you can use it too.

If you want some up-to-date distribution, you can try Ångström distribution - they started to provide online builder where Palm Treo 650 can be found. You can use such image, just put there new kernel and modules. Beware of udev and CONFIG_SYSFS_DEPRECATED. I tried OPIE (didn't work), GPE (worked somehow) using mdev instead of udev.

You can also prepare Debian root filesystem using qemu. There is prepared kernel and base system here you can use as a start. If there will be time spare I'll make some automated up-to-date Debian root filesystem generator for Palms. For getting files out of qemu disk when the virtual system is off use kpartx and raw image type.

Good night!

Palm over USB - NetworkManager without ModemManager

Maybe you found an annoying problem with recent NetworkManager with integrated ModemManager.
When you attach your Palm over USB to PC with NetworkManager, ModemManager tries to open every new serial device and it causes Palm reset.
You can workaround this problem by disabling ModemManager at all:
# mv /usr/share/dbus-1/system-services/org.freedesktop.ModemManager.service ~

I'll try to make some list of USB IDs and correct serial ports, but the best would be disable ModemManager for these devices at all.

Is there any other way to use serial port of Palm Smartphone to communicate with modem than using 'USB Modem' application? It even uses some other serial device - ttyACMx...

RING again

I don't have much time so just small overview:

• Centro patches finally applied


• suspend is broken on Treos, Centro, TX and probably others, probably git bisect is needed


• Centro has problem with framebuffer, I have some idea about possible fix, it has top priority now


• I still haven't found the way, how to run GSM when powered off in PalmOS, but you can power it on in PalmOS and use older cocoboot.prc from my page and get your RING again


• I added context menu symbol to Treo xkb map, useful when you miss right click with touchscreen, if you use it, you can update


• It seems that power consumption is lesser in Linux than in PalmOS even with higher clock! Photos on my facebook profile (yes, I was finally Facebookized too, resistance is futile ;)


• I'll be dad in a month so I'll have less time for this project, sorry.


• QtExtended support older brother of GSM (in Greenphone) chip residing Treos. It should be quite easy to add support! Volunteers needed.


• This blog entry was written on Treo680 using Illume on Debian. It's not as fast as good old Opie, but it's much more usable. I've even better impression than from SHR on FreeRunner!


• There are first attempts about Treo755p, but it will need more time.

Things in my TODO list now:

1. fix Centro framebuffer (release kernel)


2. fix GSM problem (release kernel)


3. fix gsm0710muxd and FSO


4. try to fix suspend (if Marex won't care)


5. build kernel for TX, LD, T5


6. rework install_script.sh (better rights management, support for halmount, add more options)