udoo neo reconfiguration

Just a quick note on how to reconfigure udoo neo keyboard and timezone from console.

To reconfigure keyboard, from the console issue:

sudo dpkg-reconfigure console-data

give sudo password then select the last option ( ‘Select keymap from full list’). From there you can choose you actual keyboard layout and confirm with enter.

To reconfigure timezone, I think that you can manually edit /etc/timezone with an editor, changing the contents to suit your correct time zone. Otherwise you can simply issue from console:

timedatectl list-timezones

to have a list of available time zones, then

sudo timedatectl set-timezone your/timezone

to set up the correct time zone. For instance I had to issue

sudo timedatectl set-timezone Europe/Rome

and that’s all. For keyboard you should have to restart udoo neo; time zone is immediately set up, and information are update also in lxde.

Previous commands are available also for Ubuntu, from wich udoo Linux distribution comes from.

Acronis 2016 with NVMe disks

Today I helped a colleague to restore its Dell laptop from an Acronis image. The process should have been easy to perform, but there was a little problem… Its laptop was using an M.2 Toshiba SSD (specifically the model THNSN51T02DU7). If you use the Acronis launcher even in WinPE mode, the M.2 disk is not detected.

And now? What to do?

Googling around a bit I found this site which explain (in Italian, many thanks to Gianluca Cecca) how to integrate a Samsung 950 Pro M.2 disk into Acronis. The problem was: where to find THNSN51T02DU7 drivers? We had to look around again with Google, and finally we came up with this explanation:

Intel and Samsung offer a proprietary NVMe driver for their NVMe SSDs, but Toshiba does not. The XG3 utilizes the standard Microsoft NVMe driver that comes baked into Windows 8-10. This makes sense because the XG3 is an OEM SSD, and not needing an additional driver for best performance makes setup easier for OEMs.
So all we had to do was:
  1. search for the original Windows 10 drivers
  2. download them
  3. integrate them into the Acronis WinPE image
  4. cross our fingers…

Locating the driver to download wasn’t an easy taks to perform. Fortunately on Dell website there was an explanation of a similar task related to WinPE and storage controller drivers. So we had to download the Intel Rapid Storage drivers from Intel’s web site (following the lin provided into Dell’s site, we were redirected to a driver version that wasn’t the last one available; Intel site then suggested us to download the last version).

We choosed to download the package f6flpy-x64.zip from the site (f6flpy-x32.zip if your sistem is running in 32 bit mode, but it’s really improbable), then followed the instruction from the integration guide. Basically here are the instructions:

  1. Create the Acronis loader in WinPE format from within your Acronis True Image, writing it onto an USB pen drive
  2. Install Windows ADK; you might have it already installed if you are a developer. You can try to open ad administrator command prompt, then issue the following instruction. If you receive any errors, you might have to download Windows ADK
  3. Create three folders in C:\, naming them Temp, Drivers and Mount
    Copy BOOT.WIM file from the Source folder of Acronis WinPE pen drive, into C:\Temp
  4. Extract the drivers from f6flpy-x64.zip into C:\Drivers
  5. Open the Administrator command prompt
  6. Issue the following command to mount the image
    dism /mount-wim /wimfile:c:\temp\boot.wim /index:1 /mountdir:c:\mount
  7. Issue the following command to merge drivers into image
    dism /add-driver /image:c:\mount /driver:c:\drivers /forceunsigned
  8. Issue the following command to integrate the drivers into the driver library
    dism /get-drivers /image:c:\mount
  9. Issue the following command to write changes into the .WIM file and release resources
    dism /unmount-wim /mountdir:c:\mount /commit
  10. Copy the new BOOT.WIM file from C:\Temp into Sources folder of Acronis WinPE pen drive
  11. Eject pen drive and delete the Temp, Drivers and Mount folders created in point 3.

    Your disk should now be visible from Acronis loader and you can proceed to data restoration.

Further steps

If you found that your Acronis loader won’t recognize your M.2 disk, probably you don’t need to read  more, as you already succeded to boot up your PC from the USB key. But what if you don’t?

Modern PC use UEFI boot process with a secure option into BIOS. This is to avoid your PC to be booted from a device you don’t trust. It might be needed to enter the BIOS and add your Acronis WinPE boot pen drive to the trusted boot sources. How to perform this task may vary from BIOS to BIOS, and you should read your PC manual to perform these steps.

An interesting library

Searching for some nuget packet usage, I came across this really interesting and huge library: Humanizer.

From the site:

Humanizer meets all your .NET needs for manipulating and displaying strings, enums, dates, times, timespans, numbers and quantities.

I’ll keep it in my notes…

Installing ESXi 5.5 U2 on PCEngines apu1d4

AlertI started writing this article on 10th January, 2015. But I didn’t have so much time to finish it… Now time has come, so I decided to finish this work and publish it. Many time has passed by, and many things have changed. A new ESXi version has been carried out, and new APU hardware is coming soon. But I think that this article can be a good starting point also for headless installation of new ESXi and other *nix based headless devices.

Unfortunately installation was not successfully, or better speaking installation correctly ended, but device didn’t turned up as I expected it would. I attempted many ways to overcome this problem, but I haven’t found any of them to correctly work. I posted all of this history here, in the hope that someone would find what I missed.

Introduction

In this article I’m introducing a step by step guide on how to install ESXi 5.5 U2 on PCEngines apu1d4 board.
apu1d4 is a small computer board that can be used for routers, firewalls, VOIP, dedicated servers, special purpose network plumbing, education tools… It’s equipped with an AMD G series T40E, 1 GHz dual core with 64 bit support processor, 4GB RAM, three Gigabit Ethernet ports and a serial console. You can read all of its specification here.

This setup guide it’s also suitable to install many other Linux distributions.

Prerequisites

As apu1d4 does not have a video output, setup must be performed via serial console. This is the major issue when installing ESXi, as the plain setup is intended to be used with a video card. But as it is Linux-based, we can turn on the serial console redirection to handle all of the setup process from here.

In this guide I’m assuming to install ESXi on a mSATA SSD installed on the board. Please make sure the SSD is detected by board BIOS before starting the setup process (you should see your mSATA ID into the BIOS boot messages on the serial console).

You need the following hardware:

  • an apu1d4 board already boxed (for passive cooling to work), with an internal mSATA drive installed
  • an empty USB pen drive (at least 1GB)
  • a null modem cable
  • a PC with a serial port or a USB to serial converter (you may omit the null modem cable if you use a USB to DCE converter like this one, that is intended to be used to connect directly to a console port)

and the following software:

  • ESXi 5.5 U2 install ISO (you don’t need to burn it onto a CD)
  • a utility to create a bootable USB key from an ISO. There are many utilities to do that; here I’m using Rufus
  • a serial communication software which allows you to connect to console port; here I’m using PuTTY
  • a utility to open .zip files; here I’m using 7-Zip
  • a general purpose text editor; here I’m using PSPad
  • the two file contained into this ESXi setup patch: net-r816-drivers.zip

A note on the patch: apu1d4 board integrates three Gigabit Ethernet channels based on the Realtek RTL8111E chipset. As RTL8111E support has been removed from ESXi setup since version 5.5, but it was still present in 5.1 setup, we need to extract the drivers from 5.1 installer and integrate them into the 5.5 one. This is needed as without installing and activating the drivers, you will end your setup into a “No network hardware detected” screen. How to integrate this driver in the setup is better explained later.

Step by step guide

Installation process can be resumed in the following steps:

  1. create ESXi 5.5 U2 USB boot pen drive (called pen drive in the following steps)
    1. unpack ISO to pen drive and make it bootable
    2. patch pen drive to load RTL8111 drivers
    3. patch pen drive to work with serial console
  2. install ESXi
  3. finalize setup

Create ESXi 5.5 U2 USB boot pen drive

Any tool you can find to extract an ISO image to a pen drive and make it bootable is suitable to create ESXi setup pen drive. I frufusound Rufus to be easy and clear to use. It’s also OpenSource and sources can be grabbed from it’s home page. From Rufus main window you have to select the USB drive related to the pen drive you want to use, the partitioning method (usually the default value is correct) and obviously the ESXi ISO image to store on key.

If should be warned that the menu.c32 file which comes on the ISO is obsolete and you should substitute it with a freshly downloaded version. Please accept this suggestion and  let Rufus patch this file.

You should end up in few minutes with the key ready to be patched.

Patch pen drive to load RTL8111 drivers

As said ESXi 5.5 U2 setup does not include RTL8111 drivers. If you do not patch the pen drive with those drivers, installation cannot be performed and you’ll receive a “No network card present” message. Usually drivers can be added later installing the so called VIB packages, but as the network card is considered to be part of the core setup, if no network card can be detected during setup phase, process will be stopped.

To avoid this problem a clean and correct way should be to install ESXi 5.1 version and then perform a version upgrade, but this could take long. A faster-and-quite-clean method is to include ESXi 5.1 drivers directly on 5.5 U2 setup, and that’s what we are going to do now.

When googling around this problem you’ll find several .vib files packaged into a .zip file, that should unpack to upload .vibs to your ESXi machine. Basically each .vib file is in turn a .zip package, containing the driver(s) itself and some descriptors. I simply extracted and repacked the two drivers included in the following .vib:

VMware_bootbank_net-r8168_8.013.00-3vmw.510.0.0.799733.vib
VMware_bootbank_net-r8169_6.011.00-2vmw.510.0.0.799733.vib

into a single .zip file that you’ll find attached here. All you have to do is to download the .zip containing the two drivers (net-r816.v00 and net-r816.v01) and extract them onto the pen drive, in its root directory.

After that you have to edit the boot.cfg file located on the pen drive, and add those driver to load list. To do so open your favorite editor (be carefull: you should not use NotePad because it lacks support for UNIX style line endings; you’d better use a general purpose editor like PSPad). Locate the row

modules=/b.b00 --- ...

and then add the following in your favorite position (I added them after all of the net_xxx drivers):

--- /net-r816.v00 --- /net-r816.v01

Save the file and you’re done with the patch.

Patch pen drive to work with serial console

In the last preparation step we have to patch configuration files to tell loader and setup to redirect standard output and standard input over a serial console. This step involves editing three configuration files, that must be still edited with the general purpose editor.

syslinux.cfg

Patching this file only requires you to add the following two lines AT TOP of the file:

CONSOLE 0
SERIAL 0 115200

isolinux.cfg

Patching this file requires you to add the following two lines AT TOP of the file:

CONSOLE 0
SERIAL 0 115200

Then you have to locate the line

  APPEND -c boot.cfg

located in the LABEL install section of the configuration. At the end of the line you have to add the extra parameters needed to instruct loader to redirect standard input and output. Line should look like this:

  APPEND -c boot.cfg text gdbPort=none logPort=none tty2Port=com1

boot.cfg

You have again to patch boot.cfg, as done in the step before. Modification involves the kernel option parameter passed; you have to locate the line

kernelopt=runweasel

and add the extra parameters needed to redirect standard input and output. Line should look like this:

kernelopt=runweasel text nofb com1_baud=115200 com1_Port=0x3f8 tty2Port=com1 gdbPort=none logPort=none

Install ESXi

You are now ready to start ESXi setup. Put the pen drive into the USB port, connect your PC to the console port, start PuTTY and power up the board. You should notice BIOS startup in the console port:

PC Engines APU BIOS build date: Apr  5 2014
Reading data from file [bootorder]
SeaBIOS (version ?-20140405_120742-frink)
SeaBIOS (version ?-20140405_120742-frink)
Found coreboot cbmem console @ df150400
Found mainboard PC Engines APU
Relocating init from 0x000e8e71 to 0xdf1065e0 (size 39259)
Found CBFS header at 0xfffffb90
found file "bootorder" in cbmem
CPU Mhz=1000
Found 27 PCI devices (max PCI bus is 05)
Copying PIR from 0xdf160400 to 0x000f27a0
Copying MPTABLE from 0xdf161400/df161410 to 0x000f25b0 with length 1ec
Copying ACPI RSDP from 0xdf162400 to 0x000f2590
Copying SMBIOS entry point from 0xdf16d800 to 0x000f2570
Using pmtimer, ioport 0x808
Scan for VGA option rom
EHCI init on dev 00:12.2 (regs=0xf7f08420)
Found 1 lpt ports
Found 2 serial ports
AHCI controller at 11.0, iobase f7f08000, irq 11
EHCI init on dev 00:13.2 (regs=0xf7f08520)
EHCI init on dev 00:16.2 (regs=0xf7f08620)
Searching bootorder for: /rom@img/setup
Searching bootorder for: /rom@img/memtest
Searching bootorder for: /pci@i0cf8/*@11/drive@0/disk@0
AHCI/0: registering: "AHCI/0: KINGSTON SMS200S330G ATA-8 Hard-Disk (28626 MiBytes)"
OHCI init on dev 00:12.0 (regs=0xf7f04000)
OHCI init on dev 00:13.0 (regs=0xf7f05000)
OHCI init on dev 00:14.5 (regs=0xf7f06000)
OHCI init on dev 00:16.0 (regs=0xf7f07000)
Searching bootorder for: /pci@i0cf8/usb@12,2/storage@1/*@0/*@0,0
Searching bootorder for: /pci@i0cf8/usb@12,2/usb-*@1
Searching bootorder for: /pci@i0cf8/usb@16,2/storage@1/*@0/*@0,0
Searching bootorder for: /pci@i0cf8/usb@16,2/usb-*@1
USB MSC vendor='Multiple' product='Card  Reader' rev='1.00' type=0 removable=1
Device reports MEDIUM NOT PRESENT
scsi_is_ready returned -1
Unable to configure USB MSC drive.
Unable to configure USB MSC device.
USB MSC vendor='' product='' rev='PMAP' type=0 removable=1
USB MSC blksize=512 sectors=1970176
All threads complete.
Scan for option roms
Running option rom at c000:0003


iPXE (http://ipxe.org) 00:00.0 C000 PCI2.10 PnP PMMpmm call arg1=1
pmm call arg1=0
+DF0F04A0pmm call arg1=1
pmm call arg1=0
+DF04C510 C000



Searching bootorder for: /rom@genroms/pxeboot.rom

Build date: Apr  5 2014
System memory size: 4592 MB

Press F12 for boot menu.

Searching bootorder for: HALT
drive 0x000f24d0: PCHS=0/0/0 translation=lba LCHS=977/32/63 s=1970176
drive 0x000f2500: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=58626288
Space available for UMB: c1000-ee800, f0000-f24d0
Returned 253952 bytes of ZoneHigh
e820 map has 7 items:
  0: 0000000000000000 - 000000000009fc00 = 1 RAM
  1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
  2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
  3: 0000000000100000 - 00000000df14e000 = 1 RAM
  4: 00000000df14e000 - 00000000e0000000 = 2 RESERVED
  5: 00000000f8000000 - 00000000f9000000 = 2 RESERVED
  6: 0000000100000000 - 000000011f000000 = 1 RAM
enter handle_19:
  NULL
Booting from Hard Disk...
Booting from 0000:7c00

SYSLINUX 4.07 EDD 2013-07-25 Copyright (C) 1994-2013 H. Peter Anvin et al

SYSLINUX 4.07 2013-07-25 Copyright (C) 1994-2013 H. Peter Anvin et al

and then will appear ESXi installation menu.

Finalizing ??

And here we come. You expect to see ESXi installation menu… but what you see it’s only a “Relocating modules and starting up the kernel…” message. There are several reason that could lead to this error, and the most common is that you’re trying to install ESXi on a machine with a bad BIOS, which is not correctly working with your video card, or a BIOS that it is supposed to talk to a video card that it’s instead missing from a device (like APU) that is headless.

Somewhere (like here) you could be driven to press SHIFT+O on console to enter the high level command line interface, than issue some switches to properly redirect console message (that’s what we were supposing to do when modifying the boot files). I tried all of these workaround but had no success.

If you want to try… let me know your workaround, if you found any.

I’m just tinking to give over ESXi on the APU and to install a plain pfSense firewall (that is my first need on the APU, and I already installed on another APU at work).

Coming soon…

It’s time to write down something new…

I’m planning to create a map of my home network using Dia, and post it on Rate My Network Diagram web site. I just only have to find the time to do it…

News from “The Lab”… part 2

HP DL380 G4 front vewOk, finally my DL380 G4 (I talked about it here) is up and running, with VMWare ESXi 4.1 Update 3.

Strictly from an hardware point of view, this server is still a good machine: it has 8GB of RAM and only three 36GB 15000rpm SCSI hard drives. I disconnected the other three hard drives to lower power consumption, that is now attested around to 290W.

From a software point of view, I had several options to follow. My first idea was to install ESXi, and then run one or more virtual machines depending on what I needed at the moment. As current 5.5 VMWare ESXi release won’t run on this hardware (due to lacking VT support), I had to use a earlier version. The latest supported version should be 3.1 Update 5, but in several sites it can be found that also 4.1 Update 3, stated as “not supported on this hardware”, seems to run without any problem, at least if you don’t run a 64 bit host. Another option was to install a bare Linux system, and then to use VMWare Player to run different machines, but this solution it was discarded, because I wanted to to have the opportunity to test ESXi.

Installing ESXi was a simple task, with except to the HP Proliant Support Pack for VMWare (at the moment I’m not sure if it’s fully up and running with the whole option pack).

Now I’m thinking of adding a virtual appliance based on Turnkey Linux 64, and then I’ll find out soon if the 64 bit host limitation is true.

Update: I loaded TurnKey FileServer 64 bit edition in OVF format. Its deployment was succesfull, but virtual machine refused to turn on. A quick look on the event log showed that 64 bit support was disabled for the machine, because the processor has not a true 64 bit computation capability (the two provided Xeon are Nocona core based, with only EM64T instruction set extension). So finally I had to switch back to a 32 bit edition of my appliance, but that’s should not be a problem.

Solve Windows XP svchost.exe 100% CPU load

Today I think I found the perfect solution to Windows XP svchost.exe 100% CPU load problem.

Just a step behind to show what this problem is. I use some XP virtual machines (but this problem applies to real machines too) to develop and test software. A couple of them (one running in VMWare Player, the other running in Virtual PC) suddenly slowed down to a unpractical speed. “What the … is happening now” – I thought….

Running the taskmanager showed a svchost.exe process taking up the 100% of CPU time. Uhm that was really suspicious, so I had to investigate more. I used Process Explorer from SysInternals (now Microsoft) to detect who was that process and what it was doing to my poor machine.

svchost.exe is a part of XP (and others Microsoft’s NT-like OS) that takes care of running different processes as services, so it’s definitively a kind of srvany launcher (a commonly used program to start an executable as a NT service). But why was it taking up so much CPU time? Ok, to cut the longest debugging part, I stopped, one at a time, the services launched by the faulty copy of svchost, and I found that the Windows Update Service was responsible of taking up all of the CPU time.

Ok, once found were the problem was, now I had to find the solution for it. After googling a bit around, I found that this was a widely known problem afflicting XP with Internet Explorer 8, but the solution was not so obvious. Somewhere it was stated to clear the downloaded patch folder located in %WINDIR%\Software Distribution, deleting it after the service has been stopped, but that didn’t fix the problem.

The final solution that worked for me in both my virtual machines was:

  • stop Windows Update Service from management console or command line
  • start Internet Explorer and connect to http://download.microsoft.com
  • search for KB2898785, that is Cumulative Security Update for Internet Explorer 8 for Windows XP
  • download the executable
  • close Internet Explorer
  • launch the patch and wait it to end the installation (you will be warned it has to restart system)

After restart, you will be able to connect to Microsoft Update or Windows Update to download other updates, and svchost.exe should not take up the whole CPU time again.

Running commands on multiple Linux machines

The gsh software was born to run a command in parallel on multiple Linux machines.

Sincerely I did not test it, but posted this link on my blog as a future reference, it may become usefull one day..

Windows Server 2008 R2 backup (wbadmin)

It’s a long time since I dont’ write… Many things happened and many more coming..

This is just a small note to remember a couple of interesting links related to Windows Server 2008 R2 backup utility (wbadmin) troubles you may have. This blog article is a good starting point for having an idea on the problem.

If you try to backup your data to a network location hosted on a Linux-based system (but I think this problem can be found also elsewhere), wbadmin could fail with the following error:

The requested operation could not be completed
due to a file system limitation.

This happens only if you try to backup only some directories and not the entire disk(s) you have. Why this problem? This is related in how Linux manage sparse files, required for such a backup. This link explains the problem in details and a way to solve it, supposing you can modify Samba configuration on target machine. If you cannot modify your configuration, you have to backup the entire disk instead of single directories.

More information on sparse files can be found in the documentation of fsutil command.

Actualizing A-GPS data when service is broken…

Since May 15th Long Time Orbiter data provided by Global Locate lacks some files, so my iPaq cannot update GPS data using Quick GPS Connection software.

I’ve just sent an email to Global Locate to signal this problem, and I’m waiting an answer. In the meanwhile I tried to manually fix the problem.

If you check http://gllto.glpals.com/7day/2012-05-15/, you can see that at some point directory name changed: from hh:mm:ss.nnnnnn to hh:mm:ss. Also directory contents changed: svstatus.txt file is missing in the new directories, and this is the first file downloaded by Quick GPS Connection software.

So I manually downloaded lto.dat file from http://gllto.glpals.com/7day/latest/ directory, and via ActiveSync I copied it into \Application Data\Global Locate\Gpsct on the iPaq. Now Quick GPS Connection says data is valid and will expire within 6 days and 20 hours.

I’ll wait an answer from Global Locate to see if the problem will be fixed on server.

June 16th UPDATE

It seems that service status has been restored. Today I connected my iPaq and it successfully downloaded data on its own. Checking directories contents let me know that in http://gllto.glpals.com/7day/2012-06-08/22:07:31/ directory the missing svstatus.txt file has been restored, and so in all subsequent directories.