Skip to content

Latest commit

 

History

History
1353 lines (924 loc) · 53.1 KB

README.adoc

File metadata and controls

1353 lines (924 loc) · 53.1 KB
Raw

Building a RHEL gold image for Azure

Table of Contents

1. Introduction and Lab Outline

This lab walks you through the process of building a RHEL "gold image" that can be used in a Microsoft Azure cloud environment.

This process is necessary if your organization wants to use Cloud Access ("Bring your own subscription"), and not Marketplace images. It is important to understand that marketplace images can only be consumed with on a Pay As You Go basis from Azure - you cannot use existing Cloud Access subscriptions.

If you’re not familiar with Cloud Access, it’s a feature of most existing Red Hat subscriptions for RHEL, JBoss, CloudForms, OpenShift and Gluster that allow Enterprise subscriptions to be moved to a public cloud provider. To find out more about cloud access and how to convert subscriptions, please look at this link; Overview to Cloud Access.

Lab Technical Difficulty:

Beginner - Basic command line knowledge is useful, and lots of help is available!

1.1. Why would I want to learn how to build a gold image for Azure?

  • Most users of Red Hat at scale have their own gold images, with requirements for standard service configuration, security settings, and pre-installed packages.

  • Most of these users already have an existing large number of Red Hat Enterprise Linux subscriptions too, and want to bring those to Azure.

  • The Azure Marketplace images cannot be customized before they are provisioned, and they also cannot be used with existing subscriptions. Therefore it is necessary to learn how to build a RHEL gold image for Azure.

1.2. What will I do in this lab?

  1. You’ll start this lab by logging into a build-workstation running in Red Hat’s cloud. You will have root on this workstation.

  2. Create a local virtual machine on this workstation, and boot it up

  3. Perform standard installation settings changes like network, bootloader and removes SSH keys necessary for Azure.

  4. Install the Windows Azure Live Agent (WALA) and configure

  5. The image is then prepared (like Windows Sysprep) so that multiple new images can be created from this template.

  6. Review how the Azure tools work

Note
 Because this lab produces a virtual machine that is approximately 11Gb in size, which is unpractical to upload to Azure during the Red Hat Summit due to bandwidth limits. This lab includes instructions on how you would upload the image using the Azure tools which you can read through. However, the actual upload to Azure is read only and you will not upload your built images to Azure. You will see how these steps are quite simple through, so you won’t miss much, and you will learn lots of Azure best practices for RHEL in this lab!

1.3. What skills do I need for this lab?

It’s recommended that you have some basic Linux skills - Red Hat Certified Systems Administrator level will be absolutely fine to complete this lab.

Please do ask if you have questions, Red Hatters should always be nearby to help out!

1.4. After completing this lab, you should be able…​

  • To understand how to build a RHEL gold image for Azure

  • To understand the reasons behind some of the changes we make for Azure

  • Be able to build your own Azure image for use in Cloud Access

1.5. Author & Support, and acknowledgments

This lab was submitted by James Read.

This lab is being supported by Götz Riger, Dan Kinkead, and Khaled Elbedri.

Here’s what we look like if you want to ask for help!

jread

goetz

James Read

Götz Rieger

EMEA Senior Solution Architect for Microsoft, Red Hat

Senior Solution Architect, Germany, Red Hat

Email: [email protected]

Email: [email protected]

dan

khaled

Dan Kinkead

Khaled Elbedri

Platform Technical Integrated Support Manager, Red Hat

Technical Sales Lead for Open Source, Microsoft

Email: [email protected]

Email: [email protected]

James would like to express his sincere thanks to Götz, Dan, Khaled and others for supporting this lab…​ and you for choose to sit here!

This lab was based around a similar excellent knowledge base article on the Red Hat customer portal. James would also like to thank the original authors and current maintainers of the knowledge base article(s).

Small changes to those instructions to that document have been made to convert it to a Red Hat Summit lab. This lab also includes additions not included in the original article about using Azure Active Directory, LVM and a bit more.

Be sure to add the following article to your bookmarks for reference after the Red Hat Summit!

article

This link is also mentioned at the end of this lab guide, along with many other relevant and useful links.

2. Lab Orientation

2.1. Environment

In this lab, we’ll be using the lab laptop and remote workstation only.

No special networking, connection Azure details are needed, no Red Hat subscriptions are needed either. Very simple really!

You may need these details later in the lab;

Activation Key

gold

Lab ID

L1071

GUID

This is generated for you automatically - 4 characters that are unique to your lab and used in your hostname.

2.2. Connecting to your lab workstation

There are 3x machines used in this lab;

  1. Your lab laptop

    1. Graphical environment

    2. No root access

  2. Your lab workstation

    1. Hosted in the cloud

    2. Can run virtual machines

    3. This is the machine that you SSH into

  3. The virtual machine gold image you will be building

You should now SSH into your lab workstation using the SSH command specified on the lab provisioning webpage. You can find the Terminal application in the Applications → Favorites top menu.

guidGrabberSsh

3. Start creating your gold image virtual machine

3.1. Welcome to your workstation!

If you logged in successfully, you should see this message;

---
  Welcome to the RHEL Gold Image Workstation!

        Red Hat Summit 2018
---
lab-user@goldimageworkstation-GUID:

3.2. Become root

You can become root on your lab workstation using sudo su. This will allow you to install packages and create virtual machines with libvirt.

lab-user@goldimageworkstation-GUID: sudo su
root@goldimageworkstation-GUID: whoami
root

3.3. Take a look around

Lets just take a look around, this is a very simple workstation. Run a couple of simple commands below to explore the lab workstation;

It should be running RHEL 7.5 Server.

root@goldimageworkstation-GUID: cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.5 (Maipo)

It should have just 8Gb of RAM;

root@goldimageworkstation-GUID: free -h
              total        used        free      shared  buff/cache   available
Mem:           7.6G        136M        7.3G         16M        195M        7.3G
Swap:            0B          0B          0B

Your workstation should be a 2x core machine;

root@goldimageworkstation-GUID: lscpu | grep -i CPU
CPU op-mode(s):        32-bit, 64-bit
CPU(s):                2
On-line CPU(s) list:   0,1
CPU family:            6
CPU MHz:               2294.872
NUMA node0 CPU(s):     0,1

This is more than enough to host a couple of small virtual machines - in this lab we just will create one virtual machine.

3.4. Configure Yum server

Warning
 The commands for yum.repos.d are no longer required after Red Hat Summit, if you are provisioning this demo from the product demo system. If you are building this demo yourself, then you need to find another way to get access to repositories.

This lab environment has stored updates and essential packages on a utility server. You need to configure your workstation to get packages from this server

root@goldimageworkstation-GUID: cd /etc/yum.repos.d/
root@goldimageworkstation-GUID: wget http://repos.example.com/lab.repo

It’s always a good idea to clean the yum cache after repos change;

root@goldimageworkstation-GUID: yum clean all

Check which repositories can be found with this command;

root@goldimageworkstation-GUID: yum repolist

You should see the 2x repos in the list;

root@goldimageworkstation-GUID: yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-m
rhel-7-server-extras-rpms
rhel-7-server-rpms
(1/2): rhel-7-server-extras-rpms/primary_db
(2/2): rhel-7-server-rpms/primary_db

3.5. Install packages needed to create Linux virtual machines

The following packages are used in this lab. Below, there is a brief description of what the packages do, and in which repository to find them.

Package

Repository

Description

libvirt

rhel-7-server-rpms

Open source API, daemon, and management tool for managing platform virtualization.

libguestfs

rhel-7-server-rpms

A library for accessing and modifying virtual machine file systems.

libguestfs-tools

rhel-7-server-rpms

System administration tools for virtual machines; includes the guestfish utility.

virt-install

rhel-7-server-rpms

Used to create a virtual machine definition and register it with libvirt.

Install the packages as follows;

root@goldimageworkstation-GUID: yum install libvirt libguestfs libguestfs-tools virt-install

3.6. Check that the processor Virtualization Extensions are enabled on your workstation

To run virtual machines on the workstations, the processor Virtualization Extensions need to be enabled in the BIOS/EFI. This will have been done for you automatically by the script that setup your workstation.

To check if the extensions are enabled, run the following command in your terminal;

root@goldimageworkstation-GUID: dmesg | grep -i kvm

You should not see the following…​ If you do, something has gone terribly wrong, please contact a lab admin!

[164324.526483] kvm: disabled by bios
[164346.574009] kvm: disabled by bios
[164964.310827] kvm: disabled by bios

You should see something like this;

...
[    0.000000] Hypervisor detected: KVM
...

3.7. Setup the libvirt daemon

We’re now going to start libvirt, which is a helper service that talks to and manages KVM.

root@goldimageworkstation-GUID: service libvirtd start
root@goldimageworkstation-GUID: chkconfig libvirtd on

3.8. Introduction to image best practices

Red Hat releases a distribution of RHEL in .qcow2 format. This is optimized for virtualization on libvirt hypervisors which is a useful starting point for building an image for Azure. Once you have the base image configured, you can use this image as a template for creating subsequent VMs in Microsoft Azure. Note the following configuration settings in the next couple of sections.

3.8.1. VM Configuration Settings

Microsoft Azure VMs must have several specific configuration settings. Some of these settings are enabled during the initial VM creation. Other settings are set when provisioning the VM image for Microsoft Azure. Keep these in mind as you move through the procedure and refer back to it if you need to.

Option Requirement

LVM

Do not use Logical Volume Management (LVM) on the operating system disk. Instead, use regular partitions and format the disk using ext3, ext4, or xfs. LVM or RAID can be used for data disks. Explain more about LVM

ssh

ssh should be enabled to provide remote access to your Azure VMs. Note that you can use Azure’s firewalling capabilities to restrict external access.

dhcp

The primary virtual adapter should be configured for dhcp (IPv4 only).

Network Manager

This service should be disabled on RHEL 6.x images.

Swap Space

Do not create a dedicated swap file or swap partition. Swap space may be configured in the Azure Linux agent.

NIC

Choose virtio device for the primary virtual network adapter.

encryption

Do not use full disk encryption for the operating system disk. Data disks can be encrypted.

3.9. Find the KVM Guest Image

We will now find the latest KVM Guest Image that is pre-downloaded for you on your workstation. Normally, you can find KVM Guest Images on the Red Hat Customer Portal, but we’ve cached them locally to speed up download times.

Note
 We recommend downloading the latest minor version of each major version of RHEL. Even though we support RHEL 6.7, we prefer you use RHEL 6.9. Likewise, even though RHEL 7.1 is supported, you should use RHEL 7.5.

RHEL 7.0 is not supported in Microsoft Azure because the driver packages for Azure’s network cards and similar were only released in 7.1 and later.

On your lab laptop, run the following command in a terminal;

root@goldimageworkstation-GUID: cd /opt/
root@goldimageworkstation-GUID: ls

You should see an image filename that was downloaded from the customer portal for you; rhel-server-7.5-x86_64-kvm.qcow2

Copy the image to /var/lib/libvirt/images/; We create a copy of this image because if we break something during the install, we can just make a new copy without having to download the image again.

root@goldimageworkstation-GUID: cp rhel-server-7.5-x86_64-kvm.qcow2 /var/lib/libvirt/images/

3.10. Create a Local RHEL VM from this image

We are now going to create a new virtual machine on the workstation, based on this standard KVM guest image. We will customize this image, then prepare it to upload to Azure.

You do not have any access to a graphical environment on your workstation, so we will create a virtual machine using command line tools virt-install and virsh (virtual shell).

root@goldimageworkstation-GUID: virt-install -n goldimage -r 2048 --os-type=linux --os-variant=rhel7 --disk /var/lib/libvirt/images/rhel-server-7.5-x86_64-kvm.qcow2,device=disk,bus=virtio -w bridge=virbr0,model=virtio --vnc --noautoconsole --import

If successful, you should see;

Starting install...
Domain creation completed.

Let’s pause for a moment to understand what this command just did;

  1. virt-install -n goldimage created a new KVM/libvirt Virtual Machine called "goldimage"

  2. -r 2048, gave this machine 2048Mb (2Gb) of RAM. Note that in Azure, this virtual machine could be configured for any Azure machine size, and would not be limited to 2Gb. We’re just using 2Gb to initially build the image as there is no workload installed.

  3. --os-type=linux --os-variant=rhel7 configures the virtual hardware choices to be the most compatible with RHEL.

  4. --disk /var/lib/libvirt/images/rhel-server-7.5-x86_64-kvm.qcow2,device=disk,bus=virtio sets the virtual machine to use the base disk image we just copied, and connects it to the virtual machine using the "virtio" interface and driver.

  5. -w bridge=virbr0,model=virtio This gives the virtual machine a local network connection so we can SSH into it later.

  6. --import This imports the virtual machine definition to the local libvirt daemon.

The virtual machine should be started automatically for you, you can check this as follows;

root@goldimageworkstation-GUID: virsh list
 Id    Name                           State
----------------------------------------------------
 2     goldimage                      running

If you see this, awesome! If you cannot see this, you should be sad and contact a lab admin :)

3.11. Connect to the VM serial console

Lets connect to our VM, but because it does not have a default network configuration, we’re going to use the serial console.

root@goldimageworkstation-GUID: virsh console goldimage

You will probably start seeing bootup warning messages like this…​ don’t panic!

[  186.475166] cloud-init[802]: 2018-05-04 03:19:50,499 - url_helper.py[WARNING]: Calling 'http://192.168.122.1/latest/meta-data/instance-id' failed [50/120s]: request error [('Connection aborted.', error(111, 'Connection refused'))]

It’s important to pause here and understand what is happening here.

3.11.1. Understanding cloud-init

cloud-init is a useful tool in environments that support it, like Red Hat OpenStack Platform. It is a utility that is pre-installed on virtual machines, and it starts up on bootup. It asks local metadata servers for configuration information - network device IP addresses, initial root passwords, initial SSH keys and various other details.

In our virtual environment, we have not got a cloud-init metadata server configured - and it would be unusual to use cloud-init in simple environments like this. Most importantly, cloud-init support for RHEL on Azure is not yet generally available (Learn more here).

We could have disabled cloud-init using bootup options, but this was hopefully a useful learning opportunity! Wait for cloud-init to time out (should take about 3-5 minutes) and you should then get a login screen eventually;

Red Hat Enterprise Linux Server 7.5 (Maipo)
Kernel 3.10.0-862.el7.x86_64 on an x86_64

localhost login:

Great, a basic login prompt! Here’s a quick checkpoint of what we’ve done so far;

  1. SSHed into the workstation that is running in Red Hat’s cloud

  2. Setup virtualization on the workstation

  3. Created a goldimage virtual machine

The problem is, because cloud-init failed, the root password has not been set, and so you cannot login…​

We’re going to set the root password manually by turning the machine off, editing it’s virtual disk, editing the shadow password file, and then starting it back up again!

We need to exit the serial console first to get back to our workstation.

  1. Press the keyboard shortcut Control + ] to exit the serial console.

  2. Power off the virtual machine; virsh destroy goldimage

Note that virsh uses the word "destroy" to mean "power off" virtual machine. We did not delete this virtual machine, and we can check it still exists with the following command;

root@goldimageworkstation-GUID: virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     goldimage                      shut off

Once you have shut down the VM, we will now set up root access to the image.

3.12. Set Up Root Access to the RHEL VM Image

So that the image can be customized for Microsoft Azure, you need to set up root access to the VM. You can do this by using a public/private key generator like ssh-keygen, or you can set up a root password by completing the steps below.

On your workstation, use openssl to generate a new encrypted password for the root account on the new VM.

root@goldimageworkstation-GUID: openssl passwd -1 yourPasswordGoesHere
   $1$bI/.EQaO$Qa.i9WtqjskncD9LgaJwq1

Copy the encrypted password string (in the example above - $1$bI/.EQaO$Qa.i9WtqjskncD9LgaJwq1 is our encrypted password).

We’re now goin to use Guestfish to open up the virtual machine disk image, edit a file, and then close the disk image. Gueshfish is a really useful tool for making either quick, or automated changes to virtual machines without them being started. It’s particulary useful in this case to reset a root password when we don’t have root access to the machine.

Alternative ways to reset the root password include;

  1. Single user boot mode

  2. virt-customize - works in a similar way to guestfish, with simple pre-prepared tasks like resetting the root password.

Launch the guestfish utility to access the etc/shadow file on the image.

root@goldimageworkstation-GUID: guestfish -a /var/lib/libvirt/images/rhel-server-7.5-x86_64-kvm.qcow2

Enter the following commands at the guestfish command prompt.

><fs> run
   100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ 00:00

This lets the Guestfish tool see the virtual machine disks.

><fs> list-filesystems
   /dev/sda1: xfs

This shows that the virtual machine has a single disk image. We can virtually mount it within Guestfish;

><fs> mount /dev/sda1 /

Within Guestfish, we can use tools like vi to then edit files directly on the disk. We’ll use this to directly edit the shadow file (which stores our Linux passwords).

Edit the shadow file using vi. Replace the root password value with the encrypted password generated by the openssl command.

Note
 vi is a Unix text editor that you may not be used to using - please don’t be afraid to ask a lab admin if you need help with using it! Unfortunately Guestfish won’t allow you to use other editors like nano that you may choose to use instead. 
><fs> vi /etc/shadow

When you edit the shadow file, you replace the root password null value that is represented by !! in the unedited file.

root:!!:17612:0:99999:7:::
bin:*:17492:0:99999:7:::
daemon:*:17492:0:99999:7:::
...

To setting the root password to our encrypted value, edit your file to look something like this;

root:$1$bI/.EQaO$Qa.i9WtqjskncD9LgaJwq1:17612:0:99999:7:::
bin:*:17492:0:99999:7:::
daemon:*:17492:0:99999:7:::
...

Save your changes to /etc/shadow and exit your text editor.

To confirm we edited this file successfully, we can just double check the first line looks something like this using cat;

><fs> cat /etc/shadow
root:$1$bI/.EQaO$Qa.i9WtqjskncD9LgaJwq1:17612:0:99999:7:::
...

Exit the guestfish utility. This will close the disk.

><fs> quit

3.13. Start the VM again

Use the following command to restart our virtual machine in libvirt;

root@goldimageworkstation-GUID: virsh start goldimage

Note that cloud-init will try again, and will slow down your virtual machine startup. Use the virsh console goldimage command to reconnect to the console - you should now be able to login using the password you just set;

Red Hat Enterprise Linux Server 7.5 (Maipo)
Kernel 3.10.0-862.el7.x86_64 on an x86_64

localhost login: root
Password:
[root@localhost ~]#

If the root password does not work, check the /etc/shadow file to make sure the password was set up properly.

Once you are logged in using the root account, you’re ready to configure the image.

4. Configure the image for Azure

Complete the procedures in the following sections to finalize the gold image for Azure.

4.1. Remove cloud-init

Lets get rid of that bootup delay from cloud-init!

If you are unfamiliar with this service, it’s used to do the initial setup of virtual machines from generic images. OpenStack, and some other cloud platforms host a metadata server, which give images their initial hostname, username, ssh keys and similar.

Red Hat Enterprise Linux and Azure does not have production support cloud-init (click here to learn why), instead the Windows Azure Live Agent (WALA) does most of the tasks that cloud-init normally does.

Stop the cloud-init service (if present).

[root@localhost ~]# systemctl stop cloud-init

Remove the cloud-init software.

[root@localhost ~]# yum remove cloud-init

4.2. Check that the Hyper-V Device Drivers are installed

Microsoft provides network and storage device drivers as part of their Linux Integration Services for Hyper-V package. Hyper-V device drivers may need to be installed on the RHEL VM prior to importing it to Microsoft Azure. Use the lsinitrd | grep hv command to verify that the drivers are installed. i

Here is how you would manually configure the Hyper-V device drivers (these next few steps should not be necessary for you to type as recent RHEL7 images include these drivers by default);

Note the spaces before and after the quotes. For example, add_drivers+=" hv_vmbus ". This ensures that unique drivers are loaded in the event that other Hyper-V drivers already exist in the environment.

On the RHEL VM, you would add the driver parameters to the /etc/dracut.conf file.

add_drivers+=" hv_vmbus "
add_drivers+=" hv_netvsc "
add_drivers+=" hv_storvsc "

Here’s how you regenerate the intramfs image.

[root@localhost ~]# dracut -f -v

Verify the configuration changes now include the hv drivers in your bootup image.

[root@localhost ~]# lsinitrd | grep hv

You should see a list of Hyper-V drivers similar to the following drivers.

virt manager 5

4.3. Enable password auth and login via SSH

Edit the /etc/ssh/sshd_config file and enable password authentication. This allows you to use SSH password authentication without using public key authentication.

PasswordAuthentication yes

Restart the SSH daemon for the configuration changes to be picked up;

[root@localhost ~]# service sshd restart

We can now exit the serial console and reconnect via SSH. First we need to find this virtual machine’s IP address on libvirt, which starts with 192.168.122.x.

    [root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:f3:b3:15 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.108/24 brd 192.168.122.255 scope global noprefixroute dynamic eth0
       valid_lft 2571sec preferred_lft 2571sec
    inet6 fe80::5054:ff:fef3:b315/64 scope link
       valid_lft forever preferred_lft forever

We can see from the above output that in this environment the IP address of our gold image is 192.168.122.108. Your IP address will be slightly different. Take a note of it.

Exit the serial console by pressing the Control + ] shortcut.

SSH from the workstation back into the gold image VM, and use the password you set earlier;

root@goldimageworkstation-GUID: ssh [email protected]
[email protected]'s password:
Last login: Fri May  4 04:08:24 2018
[root@localhost ~]#

4.4. Setup common defaults

Set a generic host name. You could of course set this to your corporate standard, like goldimage.prod.example.com.

[root@localhost ~]# hostnamectl set-hostname localhost.localdomain

Edit /etc/sysconfig/network-scripts/ifcfg-eth0 so it matches the following list of configuration details.

DEVICE="eth0"
BOOTPROTO="dhcp"
ONBOOT="yes"
TYPE="Ethernet"
USERCTL="no"
PEERDNS="yes"
IPV6INIT="no"
Note
 Azure uses DHCP for it’s network configuration of virtual machines. If you use static IP addresses from the Azure command line tools or portal, it’s still using DHCP to actually apply these changes to virtual machines. This is a common practice in cloud environments and is recommended to keep configuration simple.

Remove any persistent network device rules.

[root@localhost ~]# rm -f /etc/udev/rules.d/70-persistent-net.rules
[root@localhost ~]# rm -f /etc/udev/rules.d/75-persistent-net-generator.rules

This is necessary because when we create multiple copies of this virtual machine on Azure, it will create network cards with different MAC addresses that have not been "seen" by this virtual machine before. This would mean your first virtual machine would be assigned eth1 or em2 or similar which is undesirable.

Set the network service to start automatically.

[root@localhost ~]# chkconfig network on

Set ssh to start automatically.

[root@localhost ~]# systemctl enable sshd
[root@localhost ~]# systemctl is-enabled sshd

Modify the kernel boot parameters.

  1. Add the following options to the end of the GRUB_CMDLINE_LINUX line in the /etc/default/grub file.

    earlyprintk=ttyS0
console=ttyS0
rootdelay=300

    • The the console and earlyprintk statements allow the Azure diagnostics to pick up early bootup messages from the virtual machine in Azure. Note the Azure does not provide serial console access generally supported in every region yet.

  2. Remove the following options, if they are present.

    rhgb
quiet
crashkernel=auto

    • The rhgb statement normally is used to show a pretty/graphical boot. This won’t be seen in Azure, and the diagnostic logs are more useful to us.

    • Removing the quiet option will show us more log messages.

    • crashkernel=auto tells the kernel to use the automatic mode in a crash, rather than choosing another option.

Regenerate the grub.cfg file. This updates the grub configuration with the changes we made above.

grub2-mkconfig -o /boot/grub2/grub.cfg

In a production environment, it’s necessary to register RHEL instances using subscription-manager to receive updates. It might seem like a good idea to do this in your gold image, but for the following reasons this isn’t recommended;

  • Your machine ID will be duplicated, causing problems when you launch 2x instances.

  • A subscription will be consumed for your gold-image, which is in storage, doing nothing.

  • If your subscriptions expire or change, you would need to update your gold image.

4.5. Install the Windows Azure Linux Agent (WALinuxAgent/WALA).

Warning
 The commands for yum.repos.d are no longer required after Red Hat Summit, if you are provisioning this demo from the product demo system. If you are building this demo yourself, then you need to find another way to get access to repositories.

The goldimage virtual machine does not have any repositories configured because it is not subscribed to subscription-manager. Again, lets download the lab repo in our gold image;

[root@localhost ~]# cd /etc/yum.repos.d/
[root@localhost ~]# curl -O http://repos.example.com/lab.repo

It’s always a good idea to clean the yum cache after repos change;

[root@localhost ~]#  yum clean all
Note
 For production environments, the rhel-7-server-extras-rpms includes the Windows Azure Linux Agent.

Install the agent, and configure it to start on boot;

[root@localhost ~]# yum -y install WALinuxAgent
[root@localhost ~]# systemctl enable waagent.service

Edit the following lines in the /etc/waagent.conf file to configure swap space for provisioned VMs. Set swap space for whatever is appropriate for your provisioned VMs.

Provisioning.DeleteRootPassword=n
ResourceDisk.Filesystem=ext4
ResourceDisk.EnableSwap=y
ResourceDisk.SwapSizeMB=2048

4.6. Installing the Azure command line interface (CLI)

The Azure Command Line Interface is an optional package that is not essential to running virtual machines on Azure, but it’s very useful if you want to run Azure commands from inside your virtual machines.

Import the Microsoft repository RPM signing key;

[root@localhost ~]# sudo rpm --import https://packages.microsoft.com/keys/microsoft.asc

Create a local Azure CLI repository entry.

[root@localhost ~]# sudo sh -c 'echo -e "[azure-cli]\nname=Azure CLI\nbaseurl=https://packages.microsoft.com/yumrepos/azure-cli\nenabled=1\ngpgcheck=1\ngpgkey=https://packages.microsoft.com/keys/microsoft.asc" > /etc/yum.repos.d/azure-cli.repo'

Lets clean the yum repositories again just to be safe, and then install the azure-cli.

[root@localhost ~]# sudo yum install azure-cli

Run the az command to check it’s working. It should just print a header, lots of help about available commands and then exit.

[root@localhost ~]# az
Warning
 It would be a bad security best practice to login to your Azure account and storage your username/password credentials in your gold image. Instead, login to Azure with the az command after you create virtual machines from this template.

4.7. Install the Azure Active Directory dependencies

To configure this RHEL virtual machine to optionally connect to Azure Active Directory later, lets install the realmd and it’s dependencies so it can later be joined to a domain;

[root@localhost ~]# yum install realmd sssd krb5-workstation krb5-libs samba-common-tools

If you would like to learn more about how to configure Azure Active Directory when it is later in Azure, please see this following article; https://docs.microsoft.com/en-us/azure/active-directory-domain-services/active-directory-ds-join-rhel-linux-vm

4.8. Deprovisioning

Now many of the common tasks have been completed, we’re almost ready to "deprovision" the machine from WALA. It’s sensible to take a backup of our work so far, so that if the deprovisioning fails or we want to make changes later, we can come back to this backup.

Logout of the gold image, and go back to the workstation. Take a snapshot of the VM from the workstation;

root@goldimageworkstation-GUID: virsh snapshot-create-as --domain goldimage --name "goldimage-before-deprovision"

Validate that the snapshot was created as follows;

root@goldimageworkstation-GUID: virsh snapshot-list goldimage
 Name                 Creation Time             State
------------------------------------------------------------
 goldimage-before-provision 2018-05-09 01:01:14 +0200 running

Log back into the goldimage VM.

Prepare the VM for Azure provisioning by cleaning up the existing provisioning details; Azure will provision the VM in Azure. This command generates warnings, which is expected.

[root@localhost ~]# waagent -force -deprovision

Here is what the deprovisioning command actually did;

  1. All SSH host keys (if Provisioning.RegenerateSshHostKeyPair is 'y' in the configuration file)

  2. Nameserver configuration in /etc/resolv.conf

  3. Root password from /etc/shadow (if Provisioning.DeleteRootPassword is 'y' in the configuration file)

  4. Cached DHCP client leases

  5. Resets host name to localhost.localdomain

Clean the shell history and shut down the VM.

export HISTSIZE=0
poweroff

5. Convert the RHEL VM Image to VHD

All Azure VM images must be in vhd format. This section describes how to convert your template image from qcow2 to vhd format. Once you have converted the image to vhd using one of the conversion procedures below, proceed to the following section and authenticate your server.

Important: The resulting vhd file must be sized to the nearest 1 MB boundary for Microsoft Azure. The Indirect Method of conversion below has been thoroughly tested. You can use one of the Direct Methods for a 7.x or 6.x KVM Guest Image. Note that the Indirect Method should be used if the Azure VM does not start in Microsoft Azure after using one of the other methods of conversion.

Note
 If you do not align your image to 1mb boundaries, it may still work, but it’s likely to cause significant performance degredation in Azure due to the way Azure blob storage works.

5.1. RHEL 7.2+ 1mb boundary alignment (Direct Method)

By default, the RHEL 7.2 Guest Image and above is already sized to an even 1 MB boundary and can be converted directly from qcow2 to vhd. The qcow2 format is the format of virtual machine disks used by defualt in libvirt. The vhd format is what is supported by Azure.

Use the following command to directly convert the file.

root@goldimageworkstation-GUID: qemu-img convert -p -f qcow2 -o
   subformat=fixed -O vpc rhel-server-7.5-x86_64-kvm.qcow2 rhel-server-7.5-x86_64-kvm.vhd

Take a quick look at size of the VHD file that was just generated;

root@goldimageworkstation-GUID: ll -h rhel-server-7.5-x86_64-kvm.vhd

You should get something that looks like this;

-rw-r--r-- 1 root root 11G May  9 01:16 rhel-server-7.5-x86_64-kvm.vhd

Notice how the size of the image has radically increased from approximately ~2Gb in the qcow2 format, to ~11Gb in the VHD format. Hopefully you can see why it’s unpractical for us to upload this lab image to Azure - goodbye Summit bandwidth!!

To verify the file is resized correctly, show the virtual-size using the following command.

root@goldimageworkstation-GUID: qemu-img info --output=json -f vpc rhel-server-7.5-x86_64-kvm.vhd

You should see some output like this. Take a note of the virtual-size which you can see is 10737893376.

{
    "virtual-size": 10737893376,
    "filename": "rhel-server-7.5-x86_64-kvm.vhd",
    "format": "vpc",
    "actual-size": 1696194560,
    "dirty-flag": false
}

Divide the virtual-size value by 1024, twice. If the result is a whole number, the vhd file is aligned properly. If the result has a decimal point, the file is not sized correctly and you should use the indirect method. There is a calculator in the installed in acessories if you need it ;)

<virtual-size> / 1024 / 1024

5.2. Image is ready!

At this stage, your goldimage virtual machine disk is "ready" to be uploaded to Azure! Well done for getting this far and finishing the practical part of this lab.

Do read to the end as there are some important points you can stil learn, and a summary section.

6. Read-Only: How you could upload this machine to Azure

Note
 We cannot actually run these commands in this lab, because 20+ people uploading 11Gb images would consume all the bandwidth at Red Hat Summit, and would also take quite some time. This section of the lab is for reading only, we cannot do these steps on the laptops.

Enter az login to authenticate your Azure administration server and log in.

az login

Example:

[clouduser@localhost]$ az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code FDMSCMETZ to authenticate.
  [
	{
	  "cloudName": "AzureCloud",
	  "id": "",
	  "isDefault": true,
	  "name": "",
	  "state": "Enabled",
	  "tenantId": "",
	  "user": {
		"name": "",
		"type": "user"
	  }
	}
  ]

6.1. Read-Only: Export a Storage Account Key

Important: The following steps are only for users that have existing resources for the VM in Microsoft Azure. If you need to create new Azure resources, go to Set Up New Resources in Microsoft Azure.

Complete the steps below to get your storage account key and export it to Microsoft Azure.

Get the storage account connection string.

az storage account show-connection-string -n <storage-account-name> -g <resource-group>

Example:

[clouduser@localhost]$ az storage account show-connection-string -n azrhelclistact -g azrhelclirsgrp
{
  "connectionString": "DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=azrhelclistact;AccountKey=NreGk...=="
}

Export the connection string. Copy the connection string and paste it into the following command. This connects your system to the storage account.

export AZURE_STORAGE_CONNECTION_STRING="<storage-connection-string>"

Example:

[clouduser@localhost]$ export AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=azrhelclistact;AccountKey=NreGk...=="

Once you have exported the storage connection string, go to Upload and Provision the Azure RHEL VM.

Set Up New Resources in Microsoft Azure Complete the following steps to create resources in Microsoft Azure.

Create a resource group in an Azure region.

az group create --name <resource-group> --location <azure-region>

Example:

[clouduser@localhost]$ az group create --name azrhelclirsgrp --location southcentralus
{
  "id": "/subscriptions//resourceGroups/azrhelclirsgrp",
  "location": "southcentralus",
  "managedBy": null,
  "name": "azrhelclirsgrp",
  "properties": {
	"provisioningState": "Succeeded"
  },
  "tags": null
}

Create a storage account. Refer to Storage SKU Types for SKU type descriptions.

az storage account create -l <azure-region> -n <storage-account-name> -g <resource-group> --sku <sku_type>

Example:

[clouduser@localhost]$ az storage account create -l southcentralus -n azrhelclistact -g azrhelclirsgrp --sku Standard_LRS
{
  "accessTier": null,
  "creationTime": "2017-04-05T19:10:29.855470+00:00",
  "customDomain": null,
  "encryption": null,
  "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Storage/storageAccounts/azrhelclistact",
  "kind": "Storage",
  "lastGeoFailoverTime": null,
  "location": "southcentralus",
  "name": "azrhelclistact",
  "primaryEndpoints": {
	"blob": "https://azrhelclistact.blob.core.windows.net/",
	"file": "https://azrhelclistact.file.core.windows.net/",
	"queue": "https://azrhelclistact.queue.core.windows.net/",
	"table": "https://azrhelclistact.table.core.windows.net/"
},
"primaryLocation": "southcentralus",
"provisioningState": "Succeeded",
"resourceGroup": "azrhelclirsgrp",
"secondaryEndpoints": null,
"secondaryLocation": null,
"sku": {
  "name": "Standard_LRS",
  "tier": "Standard"
},
"statusOfPrimary": "available",
"statusOfSecondary": null,
"tags": {},
  "type": "Microsoft.Storage/storageAccounts"
}

Get the storage account connection string.

az storage account show-connection-string -n <storage-account-name> -g <resource-group>

Example:

[clouduser@localhost]$ az storage account show-connection-string -n azrhelclistact -g azrhelclirsgrp
{
  "connectionString": "DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=azrhelclistact;AccountKey=NreGk...=="
}

Export the connection string. Copy the connection string and paste it into the following command. This connects your system to the storage account.

export AZURE_STORAGE_CONNECTION_STRING="<storage-connection-string>"

Example:

[clouduser@localhost]$ export AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=azrhelclistact;AccountKey=NreGk...=="

Create the storage container.

$ az storage container create -n <container-name>

Example:

[clouduser@localhost]$ az storage container create -n azrhelclistcont
{
  "created": true
}

Create a virtual network.

az network vnet create -g <resource group> --name <vnet-name> --subnet-name <subnet-name>

Example:

[clouduser@localhost]$ az network vnet create --resource-group azrhelclirsgrp --name azrhelclivnet1 --subnet-name azrhelclisubnet1
{
  "newVNet": {
	"addressSpace": {
	  "addressPrefixes": [
	  "10.0.0.0/16"
	  ]
  },
  "dhcpOptions": {
	"dnsServers": []
  },
  "etag": "W/\"\"",
  "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Network/virtualNetworks/azrhelclivnet1",
  "location": "southcentralus",
  "name": "azrhelclivnet1",
  "provisioningState": "Succeeded",
  "resourceGroup": "azrhelclirsgrp",
  "resourceGuid": "0f25efee-e2a6-4abe-a4e9-817061ee1e79",
  "subnets": [
	{
	  "addressPrefix": "10.0.0.0/24",
	  "etag": "W/\"\"",
	  "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Network/virtualNetworks/azrhelclivnet1/subnets/azrhelclisubnet1",
	  "ipConfigurations": null,
	  "name": "azrhelclisubnet1",
	  "networkSecurityGroup": null,
	  "provisioningState": "Succeeded",
	  "resourceGroup": "azrhelclirsgrp",
	  "resourceNavigationLinks": null,
	  "routeTable": null
	}
  ],
  "tags": {},
  "type": "Microsoft.Network/virtualNetworks",
  "virtualNetworkPeerings": null
  }
}

6.2. Read-Only: Upload and Provision the Azure RHEL VM

Complete the following steps to upload and provision the VM. Note that the exported storage connection string does not persist after a system reboot. If any of commands in the following steps fail, export the storage connection string again. (See Steps 3 and 4 in the previous section.)

Upload the image to the storage container. It may take several minutes.

Note: Enter az storage container list to get the list of storage containers.

az storage blob upload --account-name <storage-account-name> --container-name <container-name> --type page --file <path-to-vhd> --name <image-name>.vhd

Example:

[clouduser@localhost]$ az storage blob upload --account-name azrhelclistact --container-name azrhelclistcont --type page --file rhel-image-7.3.vhd --name rhel-image-7.3.vhd
Percent complete: %100.0

Get the URL for the uploaded vhd file. You will need to use this URL in the following step.

az storage blob url -c <container-name> -n <image-name>.vhd

Example:

[clouduser@localhost]$ az storage blob url -c azrhelclistcont -n rhel-image-7.3.vhd
"https://azrhelclistact.blob.core.windows.net/azrhelclistcont/rhel-image-7.3.vhd"

6.3. Read-Only: Create the VM

Note: The following command uses the option --generate-ssh-keys, which creates a private/public key pair. The private and public key files are created in ~/.ssh on your local machine. The public key is added to the authorized_keys file on the VM for the user specified by the --admin-username option.

az vm create --resource-group <resource-group> --location <azure-region> --use-unmanaged-disk --name <vm-name> --storage-account <storage-account-name> --os-type linux --admin-username <administrator-name> --generate-ssh-keys --image <URL>

Example:

[clouduser@localhost]$ az vm create --resource-group azrhelclirsgrp --location southcentralus --use-unmanaged-disk --name rhel-azure-vm-1 --storage-account azrhelclistact --os-type linux --admin-username clouduser --generate-ssh-keys --image https://azrhelclistact.blob.core.windows.net/azrhelclistcont/rhel-image-7.3.vhd
{
  "fqdns": "",
  "id": "/subscriptions//resourceGroups/azrhelclirsgrp/providers/Microsoft.Compute/virtualMachines/rhel-azure-vm-1",
  "location": "southcentralus",
  "macAddress": "",
  "powerState": "VM running",
  "privateIpAddress": "10.0.0.4",
  "publicIpAddress": "12.84.121.147",
  "resourceGroup": "azrhelclirsgrp"

Note the public IP address. You will need this to log in to the VM in the next step.

Start an SSH session and log in to the appliance.

ssh -i <path-to-ssh-key> <admin-username@public-IP-address>

Example:

[clouduser@localhost]$ ssh  -i /home/clouduser/.ssh/id_rsa [email protected]
The authenticity of host '12.84.121.147' can't be established.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '12.84.121.147' (ECDSA) to the list of known hosts.
[clouduser@rhel-azure-vm-1 ~]$

If you see your user login, you have successfully deployed your Azure RHEL VM.

You can now go to the Microsoft Azure portal and check the audit logs and properties of your resources. You can manage your VMs directly in the Microsoft Azure portal. If you are managing multiple VMs, you should use the ARM CLI. The ARM CLI provides a powerful interface to your resources in Microsoft Azure. Enter az --help in the CLI or go to Azure CLI 2.0 Command Reference to learn more about the commands you use to manage your VMs in Microsoft Azure.

Using other Authentication Methods While recommended for increased security, the use of the Azure-generated public key file in the example above is not a requirement. The following examples show two other methods for SSH authentication.

Example 1: These command options provision a new Azure VM without generating a public key file. They allow SSH authentication using a password.

az vm create --resource-group <resource-group> --location <azure-region> --use-unmanaged-disk --name <vm-name> --storage-account <storage-account-name> --os-type linux --admin-username <administrator-name> --admin-password <ssh-password> --image <URL>

Authentication command: ssh <admin-username@public-ip-address>

Example 2: These command options provision a new Azure VM that you can use the SSH protocol to access using an existing public key file.

az vm create --resource-group <resource-group> --location <azure-region> --use-unmanaged-disk --name <vm-name> --storage-account <storage-account-name> --os-type linux --admin-username <administrator-name> --ssh-dest-key-path <path-to-existing-ssh-key> --image <URL>

Authentication command: ssh -i <path-to-existing-ssh-key> <admin-username@public-ip-address>

7. Summary

7.1. I still have time left!

If you still have some time left in this lab (and you have some time free because we didn’t want you to run out of time) then here’s some extras you could do;

  1. Make use of the lab assistants to ask your Azure questions! We use Azure a lot, and one of us will hopefully know the answer!

  2. Try automating some of these steps - as they would be necessary when new releases of RHEL come out, like 7.6, 7.7, etc.

  3. What else do you do for a corporate standard build? Configure auditing, logging, do you often install a webserver? Do that on this image, use the WAAgent again to deprovision, realign the image if needed.

  4. Spend time reading through the recommended links! Really there is a lot of useful stuff there to learn.

7.2. What did we learn in this lab?!

In this lab we’ve run through most of the basic steps needed to create a Red Hat Enterprise Linux image for Azure.

  1. We setup virtualisation support on a RHEL7 server in order to create a "goldimage" virtual machine.

  2. The goldimage virtual machine was downloaded from the Red Hat customer portal, and we customized this image to boot up in our local environment.

  3. We removed cloud-init, which is not yet supported on Azure, and setup the Windows Azure Live Agent.

  4. There were read-only instructions on how you would upload this image to Azure later.

We hope that you are now more familiar with the process, please do review the recommended links below to learn more.

Any feedback, comments about this lab guide, please email [email protected] , and enjoy the rest of the Red Hat Summit!

7.3. What if I want to use this lab after Summit?

Sure! It’s on GitHub, check out: https://github.com/redhatdemocentral/rhsummitlabs-2018/blob/master/rhsummitlabs/Building_a_RHEL_gold_image_for_Azure/README.adoc . Obviously the lab environment you used at Red Hat Summit will not be available, but you can easily create your own workstation and then many of the instructions are similar.

7.4. Goodbye, and thanks for your time!

Thanks so much for taking the time to sit in this lab. It’s super, mega, uber important to fill out the survey in the mobile app. It’s like TripAdvisor for Red Hat Summit labs. Give us 5 stars and we’re happy. Anything less and we’re sad :( (but welcome your honest feedback!)

8. Appendixes

8.2. Logical Volume Manager (LVM) in Azure

If you inspect the root disk of these machines, you’ll notice that Logical Volume Management (LVM) is not being used. LVM is very common in physical machines and on-premise virtual machines to configure OS and Data disks in software, rather than in hardware.

While LVM is a useful tool still for Data disks in Azure, the Red Hat recommendation is not to use LVM for Operating System disks at this time (mount points like /, /bin, /usr, /var, etc).

  1. Azure’s disk resizing capabilities have no understanding of LVM, and risk damaging disks if resized.

  2. If LVM has issues during bootup, it would pause the bootup before SSH becomes available, rendering the machine impossible to login to. Note that some Azure regions have early support for serial consoles that may make this limitation less of an issue in the future. However, it is not yet available in all regions for all machine types.

8.4. Indirect Method of converting images

Note
 RHEL 7.1 images are not aligned. For RHEL 7.1 the following instructions are required, and if you image somehow became unaligned.

We need to convert the qcow2 image to raw format, re-size it, and convert the raw image to vhd.

Convert the image from qcow2 to raw.

root@goldimageworkstation-GUID: qemu-img convert -f qcow2 -O raw <image-xxx.qcow2> <image-xxx.raw>

Save the following as a script. (These steps use aligned-size.sh.) The script will calculate the size of the raw image to the nearest 1 MB boundary.

#!/bin/bash
rawdisk="image-xxx.raw"
MB=$((1024 * 1024))
size=$(qemu-img info -f raw --output json "$rawdisk" | gawk 'match($0, /"virtual-size": ([0-9]+),/, val) {print val[1]}')
rounded_size=$((($size/$MB + 1) * $MB))
echo "rounded size = $rounded_size"
export rounded_size

Run the script.

root@goldimageworkstation-GUID: sh aligned-size.sh

Resize the raw image using the rounded size.

root@goldimageworkstation-GUID: qemu-img resize -f raw <image-xxx.raw> <rounded-size>

Convert the raw disk image to vhd format.

Important: qemu-img version 1.5.3 is used in this procedure. Check the qemu-img version using yum info qemu-img (or dnf info qemu-img for Fedora 22 or later). If the version is 2.2.1 or later, add the option force_size in the conversion command, for example, subformat=fixed,force_size. All other command options remain the same.

root@goldimageworkstation-GUID: qemu-img convert -f raw -o subformat=fixed -O vpc <image-xxx.raw> <image-xxx.vhd>

To verify the file is resized correctly, show the virtual-size using the following command.

root@goldimageworkstation-GUID: qemu-img info --output=json -f vpc <path-to-image>
Divide the virtual-size value by 1024, twice. If the result is a whole number, the vhd file is aligned properly.
<virtual-size> / 1024 / 1024