neděle 3. července 2016

Tmux layout checksum

One of the things I was working on during Hackweek 14 was some simple tool used internally for building packages for customers with Level 3 support contract. That is not fun at all, but I needed to split terminal into smaller windows with different roles. I chose TMUX for that - to split the screen and show progress of the builds in its panes.

For some reason split-window command worked differently for session attached and detached. Yes, I specified window size, number of lines - all this and yet it was pain to use. The only working solution I found (without using frameworks for that) was using select-layout. Command list-windows can give you string representation of the layout which you can reuse later. Format is straightforward and let me specify all the things and then rearrange panes with one command. Beautiful.

In the beginning of the string there is also checksum of the layout so you can reuse just some valid layout which existed before. I don't like this approach as it only means that they were too lazy to implement layout validation. I'm not fan of preventing users from shooting into their leg so here you can have shell version of checksum calculation.

tmux_layout_checksum() {
    local layout="$1"
    local csum=0
    local i
    for i in $(seq 0 $((${#layout} - 1))); do
        let csum=$(((csum >> 1) + ((csum & 1) << 15)))
        let csum=$((csum + $(LC_CTYPE=C printf '%d' "'${layout:$i:1}")))
    done
    printf '%x' "$((csum & 0xffff))"
}
Now you can just create your $layout and call tmux set-layout "$(tmux_layout_checksum "$layout"),$layout".

úterý 1. dubna 2014

Finally booting any kernel

Since last time I had kernel source code repository and initramfs used in stock kernel and I thought that the way to boot will be easy. Unfortunatelly there are several facts I needed to accept and tasks I needed to do. This will give you idea what have I done in the meantime:

  • to be able to flash or boot kernel I needed to unlock bootloader - Sony is respectable that it offers way to unlock your bootloader (with loosing or reducing warranty though)
    • if you want to be able to re-lock your bootloader, you need to backup your TA partition, before unlocking (you'll need root for that)
    • unlocking doesn't work well before latest Sony update (you'll loose some functionality when unlocking on 4.1)
      • updating to lastest Sony firmware will remove your root access

At the end of this saga, I was running Sony's Android 4.3 with root access and I could start playing with fastboot's boot command. After plethora of unsuccessful attempts I get to know that it is crippled in Sony's bootloader and won't work - that was dead end for me. Flashing kernel using fastboot protocol worked but my experimental kernel didn't boot. Furthermore experimenting with kernel builds and flashing each of them would kill my flash memory very soon.

I left Sony's stock firmware and installed Cyanogenmod 10.2 which works quite nice. I was able to build identical kernel by myself and could play further. I tried to enable kexec() support in kernel but as you probably expect, it didn't started the kernel correctly so this was also dead end for now.

People from FreeXperia community gave a lot of useful information about almost everything (i.e. proper location of boot kernel and meaning of other partition) and told me also about Little Kernel bootloader. Lilstevie even made port to Xperia Tablet Z already.

I need to have device initialized during boot before loading kernel and that is the job for Sony's bootloader. In LK bootloader scenario Sony's bootloader loads Little Kernel instead of Linux Kernel so Little Kernel doesn't have to care about HW initialization and can only load and run kernel. FreeXperia community used this method to have two separated images with kernels and initial ramdisk instead of one kernel+initrd which can behave both as normal boot or recovery. In this setup you don't have to worry about flashing wrong kernel anymore as it will be fixable even without computer access.

LK was already prepared so I could only grab binaries and flash it, with some minor differences - I used actual version of TeamWin recovery project, and Cyangenmod's boot.img. It worked nicely but only after applying this kernel patch I could use `adb reboot recovery' to get to recovery. Besides Little Kernel usefulness it has also one nice feature - it supports fastboot protocol as well and it's boot command implementation even works!

I spent some time on building my own LK binary (which seems to be very sensitive to toolchain used) to be able to hack it as well. There is currently missing screen support or led support for Xperia Tablet Z sp it is hard to say when you enter or leave LK. I indicate that Little Kernel is up and ready by polling using `watch -n 0 fastboot devices'. After entering LK you have ~5 seconds to press VolDown button to listen for fastboot interface.

And before end one less serious note. When I have seen logo.rle file in initrd image, I couldn't resist to play with that a bit. When you think Sony logo during boot is not nice anymore or when do you want to make it a bit more personal. You'll need installed ImageMagick, to565 (i.e. this), from565 (could be found here and ability to repack initrd image. To convert logo to some more sane format use:

$ ./from565 -rle < /Devel/xperia/logo.rle > logo.raw
$ convert -depth 8 -size 1920x1080 rgb:logo.raw logo.png
To convert sane format to the one used in initrd:
$ convert -depth 8 logo.png rgb:logo.raw
$ ./to565 -rle < logo.raw > logo.rle
Don't forget that initrd image is gzipped cpio archive of exact format:
$ cd initrd-unpacked
$ cp new-logo.rle logo.rle
$ find | cpio -o -H newc | gzip -9 > ../new-initrd.img
It seems that you can easily meet initrd size limit so don't use too complicated pictures.

That's all for today. After my second blog entry on this topic I can finally boot my own kernel (which currently panics) and have some initial terribly broken framebuffer console to even see the panic! Now I can focus on kernel itself.

Xperia, FOTA format

Long time no see. I got new Sony Xperia Z Tablet (WiFi, not LTE one) to play with and I thought that it would be nice device to run our openSUSE with my Enlightenment on that to combine cool hardware and cool software.

It seems that there is not much project targeted on running full blown Linux distribution on Android devices. You can meet Replicant (which is great anyway), you can meet Ubuntu Touch but that is almost all. So I decided to revive my old blog and write here my notes.

I played in past with HTC Desire Z with the same desire, but as I had very little time in the end, I didn't progressed significantly beyond interesting Hackweek project. I hope that this time it will do better and at least it is not my only mobile I'd wreck :)

I started with rooting the device, which was very simple thanks to the article on unlockr and the work of DooMLoRD. I prepared then openSUSE chroot on microSD card, automated chrooting process and installed some basic tools (gcc, git, mosh + ssh) and E17. To see I have working installation I used Android's native XWindow implementation.

To boot openSUSE I'd need to boot kernel with something like

root=/dev/block/mmcblk1p4
(where my openSUSE partition is) or even provide my initramfs which will take care of mounting and other needed stuff. To be able to boot without bricking I need to find way how to provide kernel to bootloader without changing contents of flash.
fastboot
should do the job (at least it did with my HTC Desire Z attempts).

It seems that I can't influence kernel parameters for now so let's build own kernel with root device settings hardcoded. Sony provides repository with kernel source on github. Kernel configuration was not present so I took the one from Cyanogenmod project.

Let's have a look on internal flash to find some more information:

# fdisk -l /dev/block/mmcblk0
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/block/mmcblk0: 15.8 GB, 15758000128 bytes, 30777344 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: gpt


#         Start          End    Size  Type            Name
 1          256         4351      2M  unknown         TA
 2         4352         4607    128K  unknown         sbl1
 3         4608         5119    256K  unknown         sbl2
 4         5120         5631    256K  unknown         s1sbl2
 5         5632         6655    512K  unknown         sbl3
 6         6656         7679    512K  unknown         aboot
 7         7680         8703    512K  unknown         tz
 8         8704         8959    128K  unknown         alt_sbl1
 9         8960         9471    256K  unknown         alt_sbl2
10         9472         9983    256K  unknown         alt_s1sbl2
11         9984        11007    512K  unknown         alt_sbl3
12        11008        12031    512K  unknown         alt_aboot
13        12032        13055    512K  unknown         alt_tz
14        13056        14079    512K  unknown         rpm
15        14080        15103    512K  unknown         alt_rpm
16        16384        49151     16M  Linux filesyste LTALabel
17        49152        90111     20M  unknown         boot
18        90112        91671    780K  unknown         modemst1
19        94208        95767    780K  unknown         modemst2
20        98304        99863    780K  unknown         fsg
21        99864       110103      5M  unknown         ramdump
22       110104       126487      8M  Linux filesyste apps_log
23       126488       159255     16M  unknown         FOTAKernel
24       159744      4354047      2G  Linux filesyste system
25      4354048      5480447    550M  Linux filesyste cache
26      5480448      6266879    384M  Linux filesyste B2B
27      6266880     30535646   11.6G  Linux filesyste userdata
And some basic identification of the partitions:
# file -s /dev/block/mmcblk0p{1..27}
/dev/block/mmcblk0p1:  data
/dev/block/mmcblk0p2:  data
/dev/block/mmcblk0p3:  data
/dev/block/mmcblk0p4:  data
/dev/block/mmcblk0p5:  data
/dev/block/mmcblk0p6:  Hitachi SH big-endian COFF object, not stripped
/dev/block/mmcblk0p7:  data
/dev/block/mmcblk0p8:  data
/dev/block/mmcblk0p9:  data
/dev/block/mmcblk0p10: data
/dev/block/mmcblk0p11: data
/dev/block/mmcblk0p12: Hitachi SH big-endian COFF object, not stripped
/dev/block/mmcblk0p13: data
/dev/block/mmcblk0p14: data
/dev/block/mmcblk0p15: data
/dev/block/mmcblk0p16: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (extents) (large files)
/dev/block/mmcblk0p17: ELF 32-bit LSB executable, ARM, version 1, statically linked, stripped
/dev/block/mmcblk0p18: data
/dev/block/mmcblk0p19: data
/dev/block/mmcblk0p20: data
/dev/block/mmcblk0p21: data
/dev/block/mmcblk0p22: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (needs journal recovery) (extents) (large files)
/dev/block/mmcblk0p23: ELF 32-bit LSB executable, ARM, version 1, statically linked, stripped
/dev/block/mmcblk0p24: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (extents) (large files)
/dev/block/mmcblk0p25: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (needs journal recovery) (extents) (large files)
/dev/block/mmcblk0p26: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (extents) (large files)
/dev/block/mmcblk0p27: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (needs journal recovery) (extents) (large files)

I have read somewhere that there is different format used than boot.img. I also found that there is already tool for extracting this format. Unfortunately it didn't work for me.

Fortunately the format was ELF executable used as container and we have tools for analyzing or manipulating that. So, it seems that FOTAKernel is kernel and initramfs in single ELF executable in mmcblk0p23.

# readelf -a mmcblk0p23-FOTAKernel
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 61 00 00 00 00 00 00 00 00
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            ARM
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           ARM
  Version:                           0x1
  Entry point address:               0x80208000
  Start of program headers:          52 (bytes into file)
  Start of section headers:          7720040 (bytes into file)
  Flags:                             0x0
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         2
  Size of section headers:           40 (bytes)
  Number of section headers:         1
  Section header string table index: 0

Section Header:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]          LOUSER+53494e   00000000 75bc68 001000 00      0   0  0
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings)
  I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

There are no section groups in this file.

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x000074 0x80208000 0x80208000 0x581698 0x581698     0
  LOAD           0x58170c 0x82200000 0x82200000 0x1da55c 0x1da55c     0

There is no dynamic section in this file.

There are no relocations in this file.

No version information found in this file.
You can see two program headers, offsets and filesizes. First entry is kernel's. To extract kernel you can simply use dd:
$ dd if=/dev/block/mmcblk0p23 of=vmlinuz skip=$((0x74)) count=$((0x581698)) bs=1
To get to contents of initramfs you have to do some more steps:
$ dd if=/dev/block/mmcblk0p23 skip=$((0x58170c)) count=$((0x1da55c)) bs=1 | zcat | cpio -id

So, now I have original initramfs I can modify to boot from microSD card after all that HW specific black magic, I have kernel to use, so experiments with boot can start next time.

republished to get to planet.opensuse.org page

úterý 6. srpna 2013

E17 0.17.4 EFL 1.7.8 update problem at start

For those, who are using E17 and run into problems after upgrade to E17 0.17.4 and EFL 1.7.8, you may find useful following script:
#!/bin/bash

TMP=$(mktemp)
TMP2=$(mktemp)
GOOD=1

use_one() {
    echo "$CFGFILE"
}

find_all() {
    find ~/.e/e/config -name e.cfg
}

if [ "$1" ]; then
    fn=use_one
    CFGFILE="$1"
else
    fn=find_all
fi

while read cfg; do
    eet -d "${cfg}" config "$TMP"

    # editation is simple
    sed 's/\([[:blank:]]*value "show_splash" int: \)1;$/\10;/' "$TMP" > "$TMP2"

    # now sanity checks
    # was there any change at all?
    if cat "$TMP" | md5sum -c <(cat "$TMP2" | md5sum ) &> /dev/null; then
        echo "${cfg} wasn't modified"
        continue
    fi

    # do we still have the same size? altering 1 --> 0 shouldn't change it
    if [ $(stat --format=%s "$TMP") != $(stat --format=%s "$TMP2") ]; then
        echo "Modifying ${cfg} caused size change, probably edited wrong :("
        echo "Skipping..."
        continue
    fi

    echo "${cfg} looks good, applying changes"
    eet -e "${cfg}" config "$TMP2" 1
done < <($fn)


rm "$TMP" "$TMP2"

It will disable splash screen on your configuration (all profiles) or one file if you specify it manually. Even though it should be harmless, you should still backup your configuration first, for example with:
cp -R ~/.e ~/.e-backup

pondělí 14. června 2010

Typical boot and cursor in console

If you have Netwalker and have you tried to switch to console (for example by pressing ctrl+alt+f1), you probably noticed that there is no cursor. Well friends, it's not bug, it's feature. This part of code in drivers/video/mxc/mxc_ipuv3_fb.c is self-explanatory...

#ifdef CONFIG_MACH_MX51_ERDOS
static int dmy_cursor(struct fb_info *info, struct fb_cursor *cursor)
{
/*
* don't display cursor for slash image.
*/
return 0;
}
#endif /* CONFIG_MACH_MX51_ERDOS */

Let's have cursor back with this patch. Well, with this you'll get cursor back, but what about removing SHARP logo? This one should give you typical Linux Tux logo instead. And for completely usual linux boot sequence we all love so much. To get all that informations running in startup, all errors and messages disable in kernel config System Type --> Freescale MXC Implementations --> MX51 Options --> printk suppress less KERN_ERR.

Good night!

pátek 11. června 2010

Netwalker for Sleep_Walker

I recently bought new gadget - SHARP PC-Z1 Netwalker. For those who doesn't know this device it's something between netbook and PDA - good choice for every geek. It has Freescale i.MX515 ARMv7 processor with Cortex-A8 core, 512 (!) MB RAM, 4 GB internal flash storage, WiFi, 1024x600 display, touchscreen and two USB ports. With its 16 cm it can well fit my pockets. Manufacturer says that it should work about 10 hours for single charge. Cool isn't it?

One of first things I tried was to boot my Debian system I use in Treo680 and Centro and it just worked. So I prepared Debian rootfile system for you with ease. It is quite up-to-date Debian sid with Illume on top. You can find it here. I plan to use it for updates since aptitude works there fast and has no need to swap. I'll try it for building Gentoo soon.

It has Netbook remix of Ubuntu 9.04 Jaunty Jackalope out of box with GNOME environment, but I tried even KDE4 and I have to say it's great to touch applications you are using daily...

Since it has Linux inside, it also has available kernel sources, but I have to say that quality of code is really low. It often fail to build when you just change kernel configuration a bit. Good news is that Amit Kucheria from Canonical is working on adding support of i.MX chips into upstream ARM kernel.

I searched a lot for any community around this nice device but found nothing but yahoo group (intended more for users than developers) so I decided to create wiki page on Hack&Dev web as basic place for cooperation. I found some interested people on #ubuntu-arm@freenode, but it seems it will be long road to the bright future of this device. Fortunately it is quite usable even now using that kernel sources provided by manufacturer. Wiki page should contain now some basic information but I'll use it to track current status.


OK, this was written about two months ago. I tried some experiments with Netwalker - running Gentoo (it was quite great, but compilation on Netwalker sucks), Ubuntu 9.10 and 10.04 - it mostly sucked. I'm now using Debian with E17 and XFCE and I'm completely satisfied. Maybe I'll try openSUSE for ARM in future, but for now is Debian the most suitable distribution on Netwalker.

I had this week Hackweek 5 so I dig a bit to Netwalker's kernel and tried to port it to 2.6.31 Freescale's latest kernel. I found very awful stuff:

  • Netwalker doesn't have it's own machine ID - it uses machine ID of Babbage board - one similar board.
  • code is very hardly portable... there are #ifdef CONFIG_MACH_MX51_ERDOS spread in many files
    $ grep -Rl CONFIG_MACH_MX51_ERDOS /usr/src/linux-fsl-imx51-2.6.28 | wc -l
    53

Try to avoid devices containing kernel code by Nissin Systems Co.. It can save you half year of work :)

pondělí 25. ledna 2010

Quick notes

As expected my son is occupying me a lot. But I had some Innovation Time Off accumulated at work so I worked on Palms a bit.

USB client is not working well for all PXA machines, but palmtreo.c code had missing some more functionality - connection worked only once when booting with cable attached. I fixed this with following marex's patch adding this support for PalmLD, PalmTX and PalmT5. It works now better, but it stops working after suspend/resume. I'll try to dig into code. I have found that GPIO90 works same way as GPIO1 - USB detection - weird.

Support for MAX8588 and PI2C was added and it seems to work. Funny about this is, that I tried to use CPU at 416 MHz in Linux at the same voltage as it was set in PalmOS for 312 MHz. I started to use Linux on 520 MHz and it works nice.

Camera differs from the ones found in other Palms (understand - I tried to follow marex's patch for Zire72 camera but it did not work :). I disassembled my Treo680 and tried to trace GPIOs. The only one new GPIO I have found is GPIO82. You can see photos here and here. I don't know currently what type it is and I didn't even found it on I2C bus yet. Interesting is that this module has 20 pins.

I also did some tiny code cleanups but I have to create patch set and push so you can use it too.

If you want some up-to-date distribution, you can try Ångström distribution - they started to provide online builder where Palm Treo 650 can be found. You can use such image, just put there new kernel and modules. Beware of udev and CONFIG_SYSFS_DEPRECATED. I tried OPIE (didn't work), GPE (worked somehow) using mdev instead of udev.

You can also prepare Debian root filesystem using qemu. There is prepared kernel and base system here you can use as a start. If there will be time spare I'll make some automated up-to-date Debian root filesystem generator for Palms. For getting files out of qemu disk when the virtual system is off use kpartx and raw image type.

Good night!