Hot adding a virtual disk to a running ESXi VM… by COMMAND!

Sometimes you need to expand the storage of a VM running into ESXi. This shouldn’t normally be a difficoult task, it’s just a matter of adding a disk like you are doing it when creating the VM.

But sometimes you can’t do this just because you’re running a buggy version of ESXi. This was my case when I wanted to add a disk residing on a NFS datastore into a 6.5.0 ESXi host. It seems that this particular version has some issues with NFS datastores, and when I was trying to browse it, it simply didn’t appeared on my choiches.

I so planned to use the old vSphere client to connect to the host and perform the needed operations, but… vSphere client was too old to allow me to perform the VM editing.

And here came in mind the useful command line. You can do almost all with it, if you know how to do it. So I had to make a search and then I found a couple of articles that explain how to create a disk and how to add it to the machine. Here they are:

And here is a recap of what you have to do to add the disk. In the following I assume that you are using ESXi version 5 or higher:

  1. log on to the ESXi host via SSH (you might have to enable it if not already done)
  2. create the disk with the following command:
    vmkfstools --createvirtualdisk <size>G --diskformat thin <location>
    where you put in size the size you want and inlocationthe location you want, remembering that all of the datastores are available in /vmfs/volumes folder and that the final part of the path will be the disk name (so for instance you will write /vmfs/volumes/mydatastore/mynewfolder/mynewdisk.vmdk)
  3. get a list of running machines with the following command:
    vim-cmd vmsvc/getallvms
    and find the VMID related to the machine you want to add the new disk to
  4. then add the newly created disk with the following:
    vim-cmd vmsvc/device.diskaddexisting vmid /vmfs/volumes/path_to_disk.vmdk 0 1
    remembering to put the VMID retrieved in step 3 and the disk name created in step 2. A quick note about the 0 and 1 at the end of the command: they are, respectively, the SCSI BUS number and the device number on which the virtual disk is to be connected, so in this example the device will be connected as SCSI 0 ID 1; you’ll have to check with your VM what bus and ID are available and adjust them accordingly.

That’s all. It’s pretty simple, once you know it.

After adding the drive, you’ll have to log on on your OS and act as needed. In my case i was adding the disk to a Windows Server 2016 VM, so I used the disk manager to bring the newly added disk online and initialize it.

Advertisements

REST client? We have it!

Sometimes you have to develop against a web service. This is really common today and you can have the needs for testing a function quickly without having a code working or a way to write it.

In this case a tool like RESTer is what can help you to successfully undestand what is happening behind the scene.

RESTer is a browser addon which can be installed in Firefox or Chrome. I personally use it in Firefox, but it shoud be the same for Chrome.

You can define a set of web service queries, each one with its method (GET/POST,…) the body request (if needed) and you can save and keep organized your queries. You can even import/export them for using on another device. Queries can be parametrics, and parameters values can be inserted when calling them.

But one of the greatest features I used today, is the Enviroment. You can define a set of variables which can be placed inside the queries and switched automatically; in this way you can define a Production environment, on which you are referencing the real server, and a Test enviroment, in which you reference your test server, and switch from the two just with one click.

It worth a try…

ESP32 Clock – triple forced pause :(

It’s a long time since I don’t update my ESP32 clock project and this blog. I have a lot of work to do, so the spare time is always little (good excuse, I always start every single article in this blog in this way).

But this time things are unfortunately going differently, and there is no good news at all. Let’s start from the beginning.

When I started the project, ESP32 libraries were provided by cloning the Github official repository. Keeping the libraries aligned was simple, as I use Git at work, but they weren’t so stable and compatible with the rest of the environment. In one occasion after an update, I wasn’t able to compile the sketch anymore, as there were conflicts with existing standard libraries. This lead to a delay in the firmware writing. And this was the first pause.

Then, about in June I think, a first public release of ESP32 libraries has been officially included into Arduino IDE project, so in this way it’s more easy and efficient the way you get it and integrate it into the development environment. And here came the first problem: when I first tried to upload the sketch on the board, I had a guru meditation, which is a way the firmware has to signal a weird condition in your program or one of the libraries. At this moment I wasn’t able to fix it, so the project has been temporarily stopped. And this was the second pause.

But first of all, I was carrying my board in my PC suitcase and I didn’t remember about it… The result? The OLED display broke at one corner, unfortunately where the glass drivers are located, and now the image is not clear anymore. Third pause…

I hope to fix the guru meditation problem as soon as possible. The broken OLED shouldn’t be a holed_displayuge problem, because debugging the firmware is still possible. In addition I ordered on Aliexpress an OLED display which is already in my hands. It should be compatible with the existing one, so I could use it instead of the original one with just some wirings.

Maybe I’ll also order a new board, so that I can have a backup option when the first one will break.

In the meantime I also received from Aliexpress a 3.7V 230mAh LiPo battery which I should have used to replace one that inflated in my smartwatch, but unfortunately that one I ordererd wasn’t correct as replacement part. But I wired it to the ESP32 board and it worked. The board has an LiPo charger controller and when powered by the USB connector it charges the battery. When disconnected the battery powers the whole board and it seems to be quit efficient.

Stay tuned!

ESXi tricks: how to copy a VM to another host – the free way

You have several ESXi hosts running the marvelous free version of this virtualization environment, and you have the need to copy or move a VM to another host. And suddenly you stumble upon the question:

How do I copy or move this 200GB virtual machine to that host as fast as I can?

If you had vCenter with vMotion feature, you don’t have any problem. It’s just a drag’n’drop operation on the interface, and the magic is done behind it. But you don’t have it…

If you had your virtual machine laying in a NFS datastore, probably this would be quite easy to do, just merely removing the machine from the inventory of the first host, eventually moving the directory to another datastore via cp or mc (or every other way offered by your NFS server), and just adding it to the other host. But you’re too lazy to use NFS…

So if you are using an internal disk as datastore, and you obviously don’t want to stop the host, take out the drive and put it into the other one (say you can’t do it…), what to do?

Here are the poor man’s option at your fingers:

  • use the ESXi web client to copy the VM folder from the datastore, but..:
    1. you need an intermediate host with free space to do it,
    2. you have to do it one file at a time,
    3. sometimes the copy doesn’t go as you expect: the web host client hides some files, so you can find yourself with weird messages when the machine tries to boot in the new host, complaining about missing files
  • connect to your ESXi host via WinSCP and transfer files using it, but…
    1. you still need an intermediate host, as WinSCP doesn’t transfers file between two different hosts
    2. you can grab a full directory but you need the double space as WinSCP copies the files in a temporary directory, and then in the destination directory of your intermediate host
  • use your ESXi backup software (you are using a backup software, aren’t you?), backup the machine and then restore it to the new datastore, but…
    1. you need the backup software
    2. you need a host onto which execute the backup
    3. you need an intermediate space to store the backup before restoration
  • connect an USB drive to your ESXi host, connect via SSH, copy files…
    No: stop immediately: ESXi doesn’t allow you to mount an USB drive and copy data from the SSH shell… this option is not feasible
  • use SCP directly from the host for a host to host data copy

There is not so much options left, so we’ll use SCP. But what is SCP? SCP stands for Secure copy, which is a protocol meant to be used to copy files from host to host in a secure way. Read more on it on Wikipedia or just Google a little bit.. But using it, at most with a user/password authentication, is really simple.

Here are in a coincise way the steps to perform. For more information on how to perform some operation, I could invite you to perform a search on Google.

  1. log on via web client on the destination host
  2. turn on SSH on the source host
  3. enter the firewall configuration and turn on SSH Client, allowing connections
  4. log on via web client on the destination host
  5. turn on SSH on the destination host
  6. select the datastore configuration and locate the correct name for your destination datastore. You can’t use the simple description, but you need something like  a GUID (i.e. 5acb6fac-40ac2eaa-24ac-99352633523.. well it’s just a weird name. You can also gain this information logging via SSH in into your ESXi destination host, moving to /vmfs/volumes and locate the datastore you want to use
  7. on the source machine, move to the /vmfs/volumes and then enter the source datastore directory
  8. insert the following command on your source host
scp -r -v ./source_directory/ user@destinationhost:/vmfs/volumes/datastorename/

Obviously you have to exchange user, destinationhost, etcetera, with the real values. During copy you’ll be informed about transfer speed, estimated time, and so on. These information are available only per file and there isn’t an overall estimation of the whole process time.

That should do the trick.

 

Onion Omega – a brief pause from ESP32

The Onion Omega

I’ve got a lot of work to do (yep and also I’m a bit tired at night when I play on my own things), so I stopped a little bit developing on the ESP32 platform. But in the meantime I had the opportunity to use a device I baked on Kickstarter in 2015.

It was a really nice project and I was so proud to have baked it, than when Onion relesed its version 2, i baked that too, and I reported its arrival in this article.

Since then, I  hadn’t so much time to spend on that project, neither I had any idea about what to build on it. But then soomething happened that made me wonder and thinking how to use it for my urgent need.

The background

I have a garage, which is connected by an Ethernet cable to my house network. In this way I can monitor everything with an IP camera, and lay some other stuff there (like my server, my OpenMediaVault NAS, …). But the garage is at floor -1, so is basically under the terrain. In this way when I’m there I cannot use my smartphone to make any call and being reachable from my wife.

So I had a thought: I grabbed an old ADSL + WIFI router with the ADSL port broken, i wired one of its LAN port to a firewall (I spoke about its hardware in this article, but I forgot to mention that as ESXi installation wasn’t gone well, I installed a plain pfSense firewall to play with) and then the WAN port of the firewall to my Ethernet cable. Just a little bit of configuration and I created a garage wireless LAN which is able to navigate in Internet using my home router, but can’t reach or surf my internal network. Awesome.

But sometimes what is working good suddenly stops working. This happened a couple of times, and just replacing the router’s power supply fixed the problem. But after three failures, I decided it was time for the router to be exchanged with something other, more cool, more modern, more… Well something I had already here without having to pay another time for a new power supply.

The Omega

Omega is an hardware platform running an OpenWrt O.S. on an Atheros chipset. You read it good: OpenWrt and Atheros. Basically we have the routing O.S. for excellence and the one of the widest available wireless platform, both at the same time in the same product. Said in another way, we have a wireless router in our hands.

The guys at Onion did a very nice work creating a project book, wich collects a wide range of designs for their hardware platform. The book is freely available here, both in the online version and in the downloadable PDF version. One of the articles available depicts how to create an Ethernet dongle, which routes the Ethernet connection to the Wifi (and vice versa). So all we need to do is to change the /etc/config/network file with a one suitable for routing and eventually configure the WiFi interface for encryption, SSID and password. Let’s start.

The first step is to connect to the board, using the preferred method. I’m just skipping the detailed instructions as they’re already listed in the Omega’s project book. Following the proposed link to Github repository, we can use the following file:

config interface 'loopback'
 option ifname 'lo'
 option proto 'static'
 option ipaddr '127.0.0.1'
 option netmask '255.0.0.0'

config globals 'globals'
 option ula_prefix 'fd1d:48c4:7633::/48'

config interface 'wlan'
 option type 'bridge'
 option ifname 'eth0'
 option proto 'static'
 option ipaddr '192.168.3.1'
 option netmask '255.255.255.0'
 option ip6assign '60'

config interface 'wan'
 option ifname 'eth0.2'
 option proto 'dhcp'

config interface 'wwan'
 option ifname 'apcli0'
 option proto 'dhcp'

After this we are almost done with the bridging. We can probably omit the IPV6 configuration at line 7, 8 and 16 (but I left it as I was not completely sure). The last step we can do is to configure the WiFi interface to expose with the wanted SSID. As I was going to replace a previous WiFi router with this one, I decided to use the same SSID and password. To configure them you have to use the following commands:

uci set wireless.@wifi-iface[0].ssid=MyPreviousSSID
uci set wireless.@wifi-iface[0].key=MyPreviousPassword
uci set wireless.@wifi-iface[0].encryption=psk2
uci commit

Then you can restart the WiFi with the

wifi

command, which will apply the new settings. The interesting part in this setup is that the Omega board is powered with an USB cable directly from the APU, so I don’t need a new power supply.

All done. I have a new WiFi router wich is smaller (probably also less powerfull but, it suits my needs). I only have to find a plastic box to wrap it around and avoid any accidental contact with something.

ESP32 Clock – Introduction

Ok, let’s start our big little project based on the ESP32.

First of all, we provide some guidelines on the project, which will be followed for the realization of the first prototype:

  1. ESP32 board has no many way to interact with a user, so at first glance our project will have only one purpose: connect to a NTP server and get the correct time;
  2. Neither it has a real RTC clock peripheral, so at the moment an RTC peripheral has to be emulated by software; when power is lost, also clock is lost, so we need to restart the board and reconnect to the NTP server to synchronize time again;
  3. To connect to the network, we have to provide a network configuration and WiFi configuration; as we don’t have a way to configure it (yet), we will hardcode the access point name and password, we will use DHCP and we will put at least one NTP server in the firmware.

So basically speaking, this would be just a proof of concept about the building of an ESP32 clock. Then we will start adding features to make this project more usable.

I decided to create a public GitHub repository to host all of the code needed so it will be easier to track down issues, bugs and improvements. You can find it at this address.

My ESP32

Several time ago I bought on Banggood an ESP32 dev board equipped with a OLED display. At a fancy 10,50€ including shipping I received:

ESP32
  1. the board itself, which consist on
    1. the main control chip using Le Xin ESP32, Tensilica LX6 dual core processor, clocked at 240MHz, computing capacity of up to 600DMIPS,
    2. 520 SRAM
    3. 802.11 KB chip b/g/n HT40 Wi-Fi transceiver, baseband, and LWIP protocol stack, Bluetooth (Bluetooth dual-mode integrated traditional and BLE low-power Bluetooth) and Wi-Fi antennas
    4. onboard 16 Mt bytes (128 Mt bit), Flash
    5. 0.96 inch White OLED display
    6. lithium battery charging circuit and interface
    7. CP2102 USB to serial chip, perfect support for Arduino development environment
  2. a couple of pin strip
  3. a heatsink for the main processor
  4. a cable to power the board with a lithium battery

Even though I don’t know what Mt means… it’s a great board. As ESP32 is widely used at the moment, I thought that it would be nice to have one to play with, especially at that price and with a display.

I started today to play with it, and thanks to Banggod comments and Q&A section I was able to get the original demo sketch, load it, modify it and make some changes. Now all I have to do is just find something to do with it. As it has WiFi and a display, a neat application could be to create a digital clock synchronized via NTP. Maybe I’ll post the code in the following days…

Server upgrade

I told about the new hardware I bought for my server in this article.

Specifically the hardware is:

  1. An HP 491838-001 network card, that is a NC375i PCIe quad gigabit port
  2. A QLogic QLE2560 SFP FCA with it’s fiber module
  3. A Brocade 4Gb SFP module

Let’s start from the first item, even though it is an HP spare part, it works fine even in my DELL. It is a PCIe x8 card and, as in my PowerEdge I only have x8 slots, I put it in one of them, using the riser card. Both the BIOS and ESXi recognized it and it started working.

The second item it’s a QLogic FCA card. It’s basically a network card but with a firmware oriented to SAN connection. At first I bought it to connect the server to the data center switch using a fiber. I wasn’t expecting a big throughput, as my switch has only 1Gb connectivity on SFP: it was just to try to use a different kind of connection. When I realized that the card was indeed unusable for my needs, I decided to let it in the server just to see how it appeared in ESXi: it is recognized as QLogic FCA as expected, but I’m going to remove it and resell, as I don’t need it anymore.

The last item is a Brocade SFP 4Gb adapter. Well I bought it to be placed into the SFP slot in the switch, and I assume it’s working. Unfortunately it can’t communicate with the QLogic’s one, as they are meant to talk different languages. But I have another SFP and I’m quite sure than soon or later I will find something to connect them to.

Back online…

Today I decided that it was time to turn on my NAS box again. Well in fact it was about two or three weeks since I bought a new USB stick (I hope it’s more reliable than the oldest one I was using) and I put on it the latest OpenMediaVault release.

In the past week I tried several time to boot the box, and all of the time it ended with the machine running for less than an hour, and then suddenly shutting down without any advice. It seemed to end up in an unknown condition, where it seems to be sleeping, the power supply has a yellow led on (instead of a green one) and network cards blinks, so it seems more a power saving feature rather than a real shut down.

Today I decided I wanted to know more… so I started up the server and waited on the console. After it booted I opened two consoles, one for upgrading the whole system and the other one to monitor it using htop, whose package is included in all of modern Linux distros, even in OpenMediaVault.

I already told that after some time of usage, video suddenly disappears. Today it didn’t happen. Video output from VGA connector worked like a charm, so I was able to upgrade the whole system from the console and finally rebooted the system. I connected through the web console and I was able to activate SMB, import the RAID filesystem and USB filesystem on an external drive, browse to the shared folders I created and then make them again available on the network.

So it seems that now my OpenMediaVault box is running again. Now I wonder when it will shut down again without any warning… Maybe in the meantime I will look for a way to proper backup setup and configuration, so that a restart could be easy to manage.

Got a new bunch of hardware to play with…

Ok, my OpenMediaVault NAS is again K.O. and I’m really upset with it, as I loaded some data on the RAID array and I want it back again soon.

But in the meantime I ordered on eBay some cool stuffs to play with my DELL server.

I grabbed a quad gigabit Ethernet card, an SFP card and a couple of SFP fiber optic adapters.

I would like to set-up an optic link between my server and my main switch, which has a couple of SFP ports. It won’t be a really fast connection, as it only support 1Gb link, but it’s just to try a different media and approach a new topic.

The quad Ethernet card will be used to route some insulated networks to test other IoT devices in a separate environment.

I really hope that both cards can be detected by ESXi and all of these stuffs works… I’m quite concerned about the quad Ethernet card: it’s an HP branded card, with some strange connectors I’ve never seen before, …

More coming soon…