Update: as of October 2016, some or all of the plugcomputer.org links are broken. Use a search engine or web.archive.org to find the original content.
Although there are existing tutorials for upgrading a SheevaPlug to UBIFS, they require running a non-MTD/non-flash installation of Linux on the SheevaPlug, and none of them fully explain loading an UBI image from a USB flash drive. I also wanted to upgrade from Ubuntu 9.04 (jaunty) to Debian Squeeze. I had to work around a few problems along the way, including bugs in Ubuntu's mtd-utils. I didn't want to forget how I did all this, so here is exactly how I upgraded U-Boot, configured U-Boot's mtdparts, and converted my SheevaPlugs (yes, that's more than one SheevaPlug) to UBIFS, step by step. I'm using PuTTY on Ubuntu 10.10 to access the SheevaPlug's USB-serial port, but other methods (like cu and screen) and distributions will work just as well.
First, I wanted to upgrade U-Boot, as the latest U-Boot has built-in support for UBI. I tried building a few U-Boot versions from source, but unfortunately none of my builds would boot. I could load my compiled ELF image over JTAG using OpenOCD, but flashing the binary image resulted in the plug hanging at boot with no console messages. As a result, I'm using the 3.4.27+pingtoo version of U-Boot (linked from this U-Boot tutorial).
It's a good idea to back up the existing environment, so save the output of printenv to a file. Skip this step if you're running from a temporary U-Boot — you should've done this before bricking your plug ;-).
Marvell>> printenv
baudrate=115200
loads_echo=0
rootpath=/mnt/ARM_FS/
netmask=255.255.0.0
console=console=ttyS0,115200
CASset=min
MALLOC_len=1
ethprime=egiga0
bootargs_root=root=/dev/mtdblock2 ro
ethmtu=1500
usb0Mode=host
nandEcc=1bit
ethact=egiga0
serverip=10.10.0.2
ipaddr=10.10.0.1
cesvcid=ULULULULULULPPULULULULULDA
bootargs_end=:::DB88FXX81:eth0:none
image_name=uImage
standalone=fsload 0x2000000 $(image_name);setenv bootargs $(console) root=/dev/mtdblock0 rw ip=$(ipaddr):$(serverip)$(bootargs_end) $(mvPhoneConfig); bootm 0x2000000;
mvPhoneConfig=mv_phone_config=dev0:fxs,dev1:fxs
mvNetConfig=mv_net_config=(00:11:88:0f:62:81,0:1:2:3),mtu=1500
yuk_ethaddr=00:00:00:EE:51:81
netretry=no
rcvrip=169.254.100.100
loadaddr=0x02000000
autoload=no
ethaddr=nn:nn:nn:nn:nn:nn
run_diag=no
bootcmd=nand read.e 0x800000 0x100000 0x400000; bootm 0x800000
oldbootargs=console=ttyS0,115200 mtdparts=nand_mtd:0x400000@0x100000(uImage),0x1fb00000@0x500000(rootfs) rw root=/dev/mtdblock1 rw ip=10.4.50.4:10.4.50.5:10.4.50.5:255.255.255.0:DB88FXX81:eth0:none
bootargs=rootfstype=jffs2 console=ttyS0,115200 mtdparts=orion_nand:0x400000@0x100000(uImage),0x1fb00000@0x500000(rootfs) rw root=/dev/mtdblock1 ip=169.254.240.231:169.254.1.1:255.255.0.0:hostname:eth0:none
arcNumber=2097
stdin=serial
stdout=serial
stderr=serial
mainlineLinux=yes
enaMonExt=no
enaCpuStream=no
enaWrAllo=no
pexMode=RC
disL2Cache=no
setL2CacheWT=yes
disL2Prefetch=yes
enaICPref=yes
enaDCPref=yes
sata_dma_mode=yes
netbsd_en=no
vxworks_en=no
bootdelay=3
disaMvPnp=no
enaAutoRecovery=yes
filesize=73DE0Environment size: 1552/131068 bytes
Now that you're ready to flash the new version of U-Boot, initialize the USB system.
Marvell>> usb start
(Re)start USB...
USB: scanning bus for devices... 2 USB Device(s) found
scanning bus for storage devices... 1 Storage Device(s) found
Next, find the U-Boot image you want to flash (I saved the 3.4.27+pingtoo binary as u-boot-3.4.27-pingtoo.bin).
Marvell>> fatls usb 0:1 sheeva_uboot/
8 file(s), 2 dir(s)
./
../
329520 u-boot-sep2010.kwb
474592 u-boot-3.4.27-pingtoo.bin
329004 u-boot-sep2010.bin
474592 uboot3.4.27.bin
473888 u-boot.bin-3.4.19
1448428 u-boot-sep2010
233779 u-boot-sep2010.map
538 u-boot-sep2010.lds
Load the u-boot image into RAM. I'm loading it directly after the U-Boot memory space, at 0x800000 (8MB), but 0x6400000 (100MB) is also popular.
Marvell>> fatload usb 0:1 0x800000 sheeva_uboot/u-boot-3.4.27-pingtoo.bin
474592 bytes read
reading sheeva_uboot/u-boot-3.4.27-pingtoo.bin
.
.
............................................
.
Verify that the image loaded correctly by comparing what was just loaded to the file on disk.
Marvell>> md.l 0x800000 0x10
00800000: 0800008b 00073be0 00000000 00000200 .....;..........
00800010: 00600000 00670000 00000000 7f010000 ..`...g.........
00800020: 00000040 00000000 00000000 00000000 @...............
00800030: 00000000 00000000 00000000 00000000 ................
nitrogen@host:~/src/sheevaplug/uboot $ od -A x -t x4 u-boot-3.4.27-pingtoo.bin | head -n 4
000000 0800008b 00073be0 00000000 00000200
000010 00600000 00670000 00000000 7f010000
000020 00000040 00000000 00000000 00000000
000030 00000000 00000000 00000000 00000000
If you want to be absolutely sure that what you're seeing came from your USB stick, you can fill the memory area with zeros before loading the U-Boot image (keep in mind that U-Boot defaults to hexadecimal numbers).
Marvell>> mw.l 0x800000 0 0x10
Marvell>> md.l 0x800000 0x10
00800000: 00000000 00000000 00000000 00000000 ................
00800010: 00000000 00000000 00000000 00000000 ................
00800020: 00000000 00000000 00000000 00000000 ................
00800030: 00000000 00000000 00000000 00000000 ................
With the U-Boot image in RAM, it's time to write it to NAND flash.
Marvell>> nand erase 0 0x80000
NAND erase: device 0 offset 0x0, size 0x80000
Erasing at 0x60000 -- 100% complete.
OK
Marvell>> nand write.e 0x800000 0 0x80000
NAND write: device 0 offset 0x0, size 0x80000
Writing data at 0x7f800 -- 100% complete.
524288 bytes written: OK
It's time to test our new U-Boot image. We should see version 3.4.27 (instead of 3.4.16, in my case).
Marvell>> reset
U-Boot 1.1.4 (Dec 27 2009 - 22:03:21) Marvell version: 3.4.27
▒
__ __ _ _
| \/ | __ _ _ ____ _____| | |
| |\/| |/ _` | '__\ \ / / _ \ | |
| | | | (_| | | \ V / __/ | |
|_| |_|\__,_|_| \_/ \___|_|_|
_ _ ____ _
| | | | | __ ) ___ ___ | |_
| | | |___| _ \ / _ \ / _ \| __|
| |_| |___| |_) | (_) | (_) | |_
\___/ |____/ \___/ \___/ \__|
** MARVELL BOARD: SHEEVA PLUG LE
Success! If all went well, the U-Boot environment was not overwritten and if you don't interrupt the boot process the existing kernel should boot.
The next step is setting up the mtdparts variables in U-Boot, making it easier to manage Flash partitions. We'll start by getting the existing partition definitions from the kernel's boot arguments.
Marvell>> printenv bootargs
bootargs=... mtdparts=orion_nand:0x400000@0x100000(uImage),0x1fb00000@0x500000(rootfs) ...
This tells us that the kernel has a 4MB partition at offset 1MB, and the root filesystem is 507MB (i.e. 512 - 5) at offset 5MB. We also know that U-Boot is given 512KB at offset 0, and its environment takes another 128KB somewhere after that (I've yet to determine the exact offset). Since the kernel starts at 1MB, it's easiest to just give U-Boot its own 1MB partition.
In order to make U-Boot aware of our flash partitions, we need to set both the mtdids and mtdparts environment variables. The mtdids variable maps U-Boot device names to kernel device names. For the SheevaPlug, the U-Boot device name is nand0, and the kernel device name is orion_nand.
Marvell>> setenv mtdids nand0=orion_nand
Marvell>> printenv mtdids
mtdids=nand0=orion_nand
The mtdparts environment variable contains the complete mtdparts argument to the kernel (including the "mtdparts=" prefix). We'll call the filesystem area ubi now instead of rootfs, indicating that we will soon be replacing it with an UBI image.
Marvell>> setenv mtdparts mtdparts=orion_nand:1m(uboot),4m(uImage),507m(ubi)
Marvell>> printenv mtdparts
mtdparts=mtdparts=orion_nand:1m(uboot),4m(uImage),507m(ubi)
Now that U-Boot knows about our desired flash layout, we can use its mtdparts command to manipulate flash partitions. Don't forget to run saveenv before booting or rebooting!
Marvell>> mtdparts
device nand0 orion_nand, # parts = 3
#: name size offset mask_flags
0: uboot 0x00100000 0x00000000 0
1: uImage 0x00400000 0x00100000 0
2: ubi 0x1fb00000 0x00500000 0active partition: nand0,0 - (uboot) 0x00100000 @ 0x00000000defaults:
mtdids : null
mtdparts: null
The kernel needs to know about our MTD partitions as well, so we'll set up some template variables and set bootargs dynamically when U-Boot runs bootcmd. We will also add the necessary kernel parameters for UBI.
Marvell>> setenv consoleargs console=ttyS0,115200
Marvell>> setenv ubiargs ubi.mtd=ubi root=ubi0:rootfs rootfstype=ubifs
Marvell>> setenv bootcmd 'setenv bootargs $(mtdparts) $(ubiargs) $(consoleargs); nand read 0x800000 uImage 0x400000; bootm 0x800000'
Marvell>> saveenv
Saving Environment to NAND...
Erasing Nand...Writing to Nand... done
But wait, now the kernel won't boot, since UBI hasn't been set up yet! Back to the Linux workstation to install mtd-utils and build an UBI image.
You need to install a semi-recent mtd-utils. The official verison from Ubuntu 10.10 supports UBI, but generates invalid checksums for UBI images. If you try this process with a prebuilt mtd-utils and get UBI erros when booting the kernel, it's necessary to build mtd-utils from source. Here's what the errors might look like:
[ 23.425099] UBI error: vtbl_check: volume table check failed: record 0, error 9
[ 23.436775] UBI error: ubi_init: cannot attach mtd2
[ 23.442171] UBI error: ubi_init: UBI error: cannot initialize UBI, error -22
If you suspect your distribution's mtd-utils may be buggy, compile mtd-utils from source. You'll need to install git, libacl1-dev, liblzo2-dev, zlib1g-dev, uuid-dev, and build-essential (or the equivalents for your distribution) first. You might be able to use apt-get build-dep mtd-utils or aptitude build-dep mtd-utils instead of installing the packages individually.
nitrogen@host:~/src/sheevaplug $ sudo aptitude install git libacl1-dev liblzo2-dev zlib1g-dev uuid-dev build-essential
nitrogen@host:~/src/sheevaplug $ git clone git://git.infradead.org/mtd-utils.git
nitrogen@host:~/src/sheevaplug $ cd mtd-utils
nitrogen@host:~/src/sheevaplug/mtd-utils $ make -j8 PREFIX=/usr/local
nitrogen@host:~/src/sheevaplug/mtd-utils $ sudo make install PREFIX=/usr/local
Now we'll build the UBIFS and UBI images. I'm using a premade Debian Squeeze root filesystem from Mark Gillespie's Debian on SheevaPlug resources which I've saved as squeeze-rootfs.tar.gz. This root image is preconfigured for the sheevaplug, including a line in /etc/inittab that spawns getty on the serial console. After downloading the rootfs tarball, extract it to a new directory. You need to extract it as root so you can preserve file ownership and permissions (notice the p on the tar command line). You also have the option of extracting your kernel's modules now, or you can wait and extract them on the running plug.
nitrogen@host:~/src/sheevaplug/debian $ mkdir squeeze-rootfs
nitrogen@host:~/src/sheevaplug/debian $ cd squeeze-rootfs
nitrogen@host:~/src/sheevaplug/debian/squeeze-rootfs $ sudo tar -zxvpf ../squeeze-rootfs.tar.gz
PackageList.txt
bin/
bin/sync
bin/hostname
bin/bzfgrep
bin/more
bin/zless
...
nitrogen@host:~/src/sheevaplug/debian/squeeze-rootfs $ cd ..
Create the UBIFS image from the extracted rootfs tarball. The mkfs.ubifs parameters below (2KB minimum I/O size, 128K-2K logical erase block size, and up to 4096 logical erase blocks) are appropriate for the SheevaPlug's NAND flash, which has 4096 physical erase blocks of 128KB each (4056 of which are actually available for UBI in our case). The "favor_lzo" compression algorithm is used to provide a good balance between speed and storage space. mkfs.ubifs must be run as root to allow it to read non-world-readable files in the rootfs image (it's probably possible to use fakeroot bash to avoid this).
nitrogen@host:~/src/sheevaplug/debian $ sudo mkfs.ubifs -v -r squeeze-rootfs/ -m 2048 -e 129024 -c 4096 -o ubifs.img -x "favor_lzo"
mkfs.ubifs
root: squeeze-rootfs/
min_io_size: 2048
leb_size: 129024
max_leb_cnt: 4096
output: ubifs.img
jrn_size: 8388608
reserved: 0
compr: lzo
keyhash: r5
fanout: 8
orph_lebs: 1
WARNING: setting root UBIFS inode UID=GID=0 (root) and permissions to u+rwx,go+rx; use --squash-rino-perm or --nosquash-rino-perm to suppress this warning
super lebs: 1
master lebs: 2
log_lebs: 5
lpt_lebs: 2
orph_lebs: 1
main_lebs: 578
gc lebs: 1
index lebs: 12
leb_cnt: 589
UUID: 778F5852-7BF2-44CC-AF59-31E50CFD6978
Success!
The UBIFS image needs to be embedded into an UBI image that can be written directly to flash. This requires the creation of a configuration file used by the ubinize command to generate the UBI image. Since we're using UBI solely for the root filesystem, our ubi.cfg will be fairly simple. It is important that the vol_size field in ubi.cfg be larger than our UBIFS image, but smaller than the total flash space (UBI reserves extra space for flash blocks that go bad). The autoresize and dynamic flags allow the volume to grow to fill our entire 507MB flash space. Check the size of the UBIFS image, then create an appropriate ubi.cfg:
nitrogen@host:~/src/sheevaplug/debian $ ls -lh ubifs.img
-rw-r--r-- 1 nitrogen nitrogen 73M 2010-12-02 02:29 ubifs.img
nitrogen@host:~/src/sheevaplug/debian $ cat ubi.cfg
[rootfs]
mode=ubi
image=ubifs.img
vol_id=0
vol_size=200MiB
vol_type=dynamic
vol_name=rootfs
vol_flags=autoresize
With an UBIFS image and ubi.cfg file created, we are ready to create the UBI image. Just to recap, this will create an UBI image with ubifs.img as the only volume (named rootfs), using 128KB physical erase blocks. The actual file size of our UBI image will be smaller than the vol_size field specifies. The kernel will grow the image to fill all available flash when it boots for the first time.
nitrogen@host:~/src/sheevaplug/debian $ ubinize -v -o ubi.img -m 2048 -p 128KiB -s 512 ubi.cfg
ubinize: parsing section "rootfs"
ubinize: LEB size: 129024
ubinize: PEB size: 131072
ubinize: min. I/O size: 2048
ubinize: sub-page size: 512
ubinize: VID offset: 512
ubinize: data offset: 2048
ubinize: UBI image sequence number: 1690274936
ubinize: loaded the ini-file "ubi.cfg"
ubinize: count of sections: 1
ubinize: mode=ubi, keep parsing
ubinize: volume type: dynamic
ubinize: volume ID: 0
ubinize: volume size: 209715200 bytes
ubinize: volume name: rootfs
ubinize: volume alignment: 1
ubinize: autoresize flags found
ubinize: adding volume 0
ubinize: writing volume 0
ubinize: image file: ubifs.imgubinize: writing layout volume
ubinize: done
nitrogen@host:~/src/sheevaplug/debian $ ls -lh ubi.img
-rw-r--r-- 1 nitrogen nitrogen 74M 2010-12-02 02:52 ubi.img
Now copy the ubi.img file to a USB flash drive, reconnect it to the SheevaPlug, reset the SheevaPlug, enter U-Boot, and start U-Boot's USB system (see first step). We'll now load the UBI image into RAM and write it to the SheevaPlug's flash. You'll need to know the hexadecimal size of your ubi.img, which was already rounded to a multiple of the physical erase block size by ubinize. Use any multi-purpose GUI calculator (kcalc, gcalctool, etc.) or bash's printf.
nitrogen@host:~/src/sheevaplug/debian $ printf "%x %s\n" `wc -c ubi.img | tee -a /dev/stderr`
77463552 ubi.img
49e0000 ubi.img
Marvell>> fatload usb 0:1 0x800000 ubi.img
77463552 bytes read
reading ubi.img
.
.
.
Marvell>> md.l 0x800000 0x10
00800000: 23494255 00000001 00000000 00000000 UBI#............
00800010: 00020000 00080000 0315480d 00000000 .........H......
00800020: 00000000 00000000 00000000 00000000 ................
00800030: 00000000 00000000 00000000 987ff375 ............u...
Now we erase the flash and write our new image. It's essential to erase all of the flash while writing only those blocks occupied by the UBI image. This ensures that the ECC bits remain erased for unused blocks. Unfortunately this wipes out any erase counters that might've been on the flash to begin with, so this process is best done on a relatively new SheevaPlug.
Marvell>> nand erase ubi
Writing data at 0x4edf800 -- 100% complete.
NAND erase: device 0 offset 0x500000, size 0x1fb00000
Skipping bad block at 0x17d60000
Skipping bad block at 0x1a940000
Erasing at 0x1ffe0000 -- 100% complete.
OK
Marvell>> nand write.e 0x800000 ubi $(filesize)
NAND write: device 0 offset 0x500000, size 0x49e0000
77463552 bytes written: OK
If your plug's kernel supports UBI, the plug will be ready to boot into the new UBIFS filesystem running Debian Squeeze. If the plug's kernel doesn't support UBI, I recommend following the kernel upgrade procedure in my previous blog post. Since you've configured mtdparts on your plug, you'll be able to use the partition name (uImage) instead of having to remember offsets and sizes. Let's test our new system.
Marvell>> boot
Starting kernel ...Uncompressing Linux... done, booting the kernel.
NAND read: device 0 offset 0x100000, size 0x400000
4194304 bytes read: OK
## Booting image at 00800000 ...
Image Name: Linux-2.6.33.7-rt29-nl2
Created: 2010-10-31 22:11:21 UTC
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 2625744 Bytes = 2.5 MB
Load Address: 00008000
Entry Point: 00008000
Verifying Checksum ... OK
OK
[ 0.000000] Linux version 2.6.33.7-rt29-nl2 (nitrogen@n2) (gcc version 4.3.3 (Sourcery G++ Lite 2009q1-203) ) #7 PREEMPT RT Sun Oct 31 16:09:45 MDT 2010
.
.
.
[ 22.389456] 3 cmdlinepart partitions found on MTD device orion_nand
[ 22.395755] Creating 3 MTD partitions on "orion_nand":
[ 22.400923] 0x000000000000-0x000000100000 : "uboot"
[ 22.407286] 0x000000100000-0x000000500000 : "uImage"
[ 22.413384] 0x000000500000-0x000020000000 : "ubi"
[ 22.420374] UBI: attaching mtd2 to ubi0
[ 22.424228] UBI: physical eraseblock size: 131072 bytes (128 KiB)
[ 22.430523] UBI: logical eraseblock size: 129024 bytes
[ 22.435945] UBI: smallest flash I/O unit: 2048
[ 22.440672] UBI: sub-page size: 512
[ 22.445311] UBI: VID header offset: 512 (aligned 512)
[ 22.451171] UBI: data offset: 2048
[ 22.894721] UBI: attached mtd2 to ubi0
[ 22.898491] UBI: MTD device name: "ubi"
[ 22.903302] UBI: MTD device size: 507 MiB
[ 22.908291] UBI: number of good PEBs: 4054
[ 22.913016] UBI: number of bad PEBs: 2
[ 22.917482] UBI: max. allowed volumes: 128
[ 22.922120] UBI: wear-leveling threshold: 4096
[ 22.926847] UBI: number of internal volumes: 1
[ 22.931311] UBI: number of user volumes: 1
[ 22.935775] UBI: available PEBs: 0
[ 22.940241] UBI: total number of reserved PEBs: 4054
[ 22.945228] UBI: number of PEBs reserved for bad PEB handling: 40
[ 22.951351] UBI: max/mean erase counter: 2/0
[ 22.955639] UBI: image sequence number: 222827779
[ 22.960374] UBI: background thread "ubi_bgt0d" started, PID 393
.
.
.
[ 23.464563] UBIFS: mounted UBI device 0, volume 0, name "rootfs"
[ 23.470605] UBIFS: file system size: 515966976 bytes (503874 KiB, 492 MiB, 3999 LEBs)
[ 23.478651] UBIFS: journal size: 9033728 bytes (8822 KiB, 8 MiB, 71 LEBs)
[ 23.486000] UBIFS: media format: w4/r0 (latest is w4/r0)
[ 23.491859] UBIFS: default compressor: lzo
[ 23.495975] UBIFS: reserved for root: 0 bytes (0 KiB)
[ 23.501656] VFS: Mounted root (ubifs filesystem) on device 0:14.
.
.
.
Debian GNU/Linux squeeze/sid debian ttyS0debian login:
It works, but there is one more step. This step is very important! You need to set your own root password and regenerate the plug's ssh keys. Login with the default root password of nosoup4u, change the password, delete the existing ssh keys, and regenerate them.
debian:~# passwd
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
debian:~# rm /etc/ssh/ssh_host_*
debian:~# dpkg-reconfigure openssh-server
Creating SSH2 RSA key; this may take some time ...
Creating SSH2 DSA key; this may take some time ...
Restarting OpenBSD Secure Shell server: sshd.
Finally, the plug is ready. You can now extract your kernel's modules if you didn't include them in the original image, set your plug's hostname, and otherwise continue customizing your SheevaPlug. Most importantly, you can now enjoy a significantly faster boot time. I'll have a follow-up post with some recommended packages for the SheevaPlug in the near future. I welcome any suggestions or corrections you may have in the comments below.
Here is a partial list of the sources I used when figuring this stuff out for myself:
http://www.openplug.org/plugwiki/index.php/Enabling_UBIFS
http://www.denx.de/wiki/view/DULG/UBootEnvVariables
http://www.denx.de/wiki/view/DULG/UBootCmdGroupFlash#Section_5.9.3.5
http://www.linux-mtd.infradead.org/faq/ubifs.html#L_ubifs_mlc
http://www.embedian.com/index.php?main_page=ubootev
http://plugcomputer.org/plugwiki/index.php/Das_U-boot_plug_support#How_to_flash_open_u-boot_binary_to_the_on-board_NAND_flash.3F
http://chiana.homelinux.org/~marc/eib_sheeva.html
http://www.plugcomputer.org/plugwiki/index.php/Setting_Up_OpenOCD_Under_Linux#Example_1:_Running_Das_U-Boot_from_DRAM
http://www.cyrius.com/debian/kirkwood/sheevaplug/install.html
http://caller9.com/blog/3/full/
http://www.plugcomputer.org/plugwiki/index.php/Installing_Debian_To_Flash
http://www.jukie.net/bart/blog/ubifs-on-sheeva