From 5066a3c692d53d12c58f4013bcea67e949d33335 Mon Sep 17 00:00:00 2001 From: Gregor Riepl Date: Thu, 28 May 2020 18:30:22 +0200 Subject: [PATCH 1/5] Update and clean up the cloud-init documentation --- .../adminguide/virtual_machines/user-data.rst | 107 +++++++++++------- 1 file changed, 67 insertions(+), 40 deletions(-) diff --git a/source/adminguide/virtual_machines/user-data.rst b/source/adminguide/virtual_machines/user-data.rst index 3a84ea16fd..81082c3385 100644 --- a/source/adminguide/virtual_machines/user-data.rst +++ b/source/adminguide/virtual_machines/user-data.rst @@ -17,35 +17,38 @@ User-Data and Meta-Data ----------------------- -CloudStack provides API access to attach up to 2KB of data after base64 encoding -to a deployed VM. Using HTTP POST(via POST body), you can send up to 32K of data -after base64 encoding. Deployed VMs also have access to instance metadata via -the virtual router. +CloudStack provides APIs to attach up to 32KB of user-data to a deployed VM. -Create virtual machine thru the API: `deployVirtualMachine `_ -using the parameter ``userdata=`` to include user-data formated in -`base64 `_. +There are two CloudStack APIs that can be used to store user-data: +`deployVirtualMachine `_ +and +`updateVirtualMachine `_ +They both support the parameter ``userdata=``. The value for this parameter +must be `base64 `_-encoded. -Accessed user-data from VM. Once the IP address of the virtual router is -known, use the following steps to retrieve user-data: +HTTP GET parameters are limited to a length of 2048 bytes, but it is possible +to store larger user-data blobs by sending them in the body via HTTP POST +instead of GET. -#. Run the following command to find the virtual router. - - .. code:: bash +From inside the VM, the user-data is accessible via the virtual router, +if the UserData service is enabled on the network offering. - # cat /var/lib/dhcp/dhclient.eth0.leases | grep dhcp-server-identifier | tail -1 +If you are using the DNS service of the virtual router, a special hostname +called `data-server.` is provided, that will point to a valid user-data server. -#. Access user-data by running the following command using the result of - the above command +Otherwise you have to determine the virtual router address via other means, +such as DHCP leases. Be careful to scan all routers if you have multiple +networks attached to a VM, in case not all of them have the UserData service +enabled. - .. code:: bash +User-data is available from the URL ``http://data-server./latest/user-data`` +and can be fetched via curl or other HTTP client. - # curl http://10.1.1.1/latest/user-data +It is also possible to fetch VM metadata from the same service, via the URL +``http://10.1.1.1/latest/{metadata type}``. For backwards compatibility, +the previous URL ``http://10.1.1.1/latest/{metadata type}`` is also supported. -Meta Data can be accessed similarly, using a URL of the form -``http://10.1.1.1/latest/meta-data/{metadata type}``. (For backwards -compatibility, the previous URL ``http://10.1.1.1/latest/{metadata type}`` -is also supported.) For metadata type, use one of the following: +For metadata type, use one of the following: - ``service-offering``. A description of the VMs service offering @@ -55,23 +58,41 @@ is also supported.) For metadata type, use one of the following: - ``local-hostname``. The hostname of the VM -- ``public-ipv4``. The first public IP for the router. (E.g. the first IP - of eth2) +- ``public-ipv4``. The first public IP for the router. - ``public-hostname``. This is the same as public-ipv4 - ``instance-id``. The instance name of the VM -Using Cloud-Init +Determining the virtual router address without DNS +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If can't or don't want to use the virtual router's DNS service, it's also +possible to determine the user-data server from a DHCP lease. + +#. Run the following command to find the virtual router. + + .. code:: bash + + # cat /var/lib/dhclient/dhclient-eth0.leases | grep dhcp-server-identifier | tail -1 + +#. Access the data-server via its IP + + .. code:: bash + + # curl http://10.1.1.1/latest/user-data + + +Using cloud-init ~~~~~~~~~~~~~~~~ -`Cloud-Init `_ can be use to access -an interpret user-data from virtual machines. Cloud-Init be installed into -templates and also require CloudStack password and sshkey scripts (:ref:`adding-password-management-to-templates` and `using ssh keys `_). User password management and -``resetSSHKeyForVirtualMachine`` API are not yet supported by cloud-init. +`cloud-init `_ can be used to access +and interpret user-data inside virtual machines. If you install cloud-init into your +VM templates, it will allow you to store SSH keys and user passwords on each new +VM deployment automatically (:ref:`adding-password-management-to-templates` and `using ssh keys `_). -#. Install cloud-init package into a template: +#. Install cloud-init package into a VM template: .. code:: bash @@ -79,7 +100,7 @@ templates and also require CloudStack password and sshkey scripts (:ref:`adding- or $ sudo apt-get install cloud-init -#. Create datasource configuration file: ``/etc/cloud/cloud.cfg.d/99_cloudstack.cfg`` +#. Create a datasource configuration file in the VM template: ``/etc/cloud/cloud.cfg.d/99_cloudstack.cfg`` .. code:: yaml @@ -90,13 +111,16 @@ templates and also require CloudStack password and sshkey scripts (:ref:`adding- - CloudStack -user-data example -~~~~~~~~~~~~~~~~~ +Custom user-data example +~~~~~~~~~~~~~~~~~~~~~~~~ -This example use cloud-init to Upgrade Operating-System of the newly created VM: +This example uses cloud-init to automatically update all OS packages on the first launch. -.. code:: yaml +#. Create user-data and encode in base64: +.. code:: bash + + base64 <`_ -documentation for latest capabilities. Cloud-Init and Cloud-Init CloudStack +Refer to the `cloud-init CloudStack datasource `_ +documentation for latest capabilities. cloud-init and the cloud-init CloudStack datasource are not supported by Apache CloudStack community. - From c47103f2d92911fa0fd5e73084b588591df79f4e Mon Sep 17 00:00:00 2001 From: Gregor Riepl Date: Thu, 28 May 2020 18:35:47 +0200 Subject: [PATCH 2/5] Replace more instances of explicit IPs --- source/adminguide/virtual_machines/user-data.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/adminguide/virtual_machines/user-data.rst b/source/adminguide/virtual_machines/user-data.rst index 81082c3385..af01f43b89 100644 --- a/source/adminguide/virtual_machines/user-data.rst +++ b/source/adminguide/virtual_machines/user-data.rst @@ -45,8 +45,8 @@ User-data is available from the URL ``http://data-server./latest/user-data`` and can be fetched via curl or other HTTP client. It is also possible to fetch VM metadata from the same service, via the URL -``http://10.1.1.1/latest/{metadata type}``. For backwards compatibility, -the previous URL ``http://10.1.1.1/latest/{metadata type}`` is also supported. +``http://data-server./latest/{metadata type}``. For backwards compatibility, +the previous URL ``http://data-server./latest/{metadata type}`` is also supported. For metadata type, use one of the following: From 1e2fbb6b9320575c264361080e07538681bcada0 Mon Sep 17 00:00:00 2001 From: Gregor Riepl Date: Thu, 28 May 2020 18:45:24 +0200 Subject: [PATCH 3/5] Document getVirtualMachineUserData usage --- source/adminguide/virtual_machines/user-data.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/source/adminguide/virtual_machines/user-data.rst b/source/adminguide/virtual_machines/user-data.rst index af01f43b89..1704279202 100644 --- a/source/adminguide/virtual_machines/user-data.rst +++ b/source/adminguide/virtual_machines/user-data.rst @@ -84,6 +84,18 @@ possible to determine the user-data server from a DHCP lease. # curl http://10.1.1.1/latest/user-data +Fetching user-data via the API +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +User-data is not included with the normal VM state for historic reasons. +To read out the base64-encoded user-data via the API, use the `getVirtualMachineUserData `_ +API call: + +.. code:: bash + + cmk get virtualmachineuserdata virtualmachineid=8fd996b6-a102-11ea-ba47-23394b299ae9 + + Using cloud-init ~~~~~~~~~~~~~~~~ From 5df23cf5505dd2dab5c55bd6a59c9a03838cef50 Mon Sep 17 00:00:00 2001 From: Gregor Riepl Date: Fri, 29 May 2020 13:50:53 +0200 Subject: [PATCH 4/5] Add MIME header to example user-data --- source/adminguide/virtual_machines/user-data.rst | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/source/adminguide/virtual_machines/user-data.rst b/source/adminguide/virtual_machines/user-data.rst index 1704279202..22c80a5047 100644 --- a/source/adminguide/virtual_machines/user-data.rst +++ b/source/adminguide/virtual_machines/user-data.rst @@ -24,7 +24,8 @@ There are two CloudStack APIs that can be used to store user-data: and `updateVirtualMachine `_ They both support the parameter ``userdata=``. The value for this parameter -must be `base64 `_-encoded. +must be a `base64 `_-encoded multi-part MIME +message. See further below for an example of what this should look like. HTTP GET parameters are limited to a length of 2048 bytes, but it is possible to store larger user-data blobs by sending them in the body via HTTP POST @@ -128,11 +129,20 @@ Custom user-data example This example uses cloud-init to automatically update all OS packages on the first launch. -#. Create user-data and encode in base64: +#. Create user-data, wrapped into a multi-part MIME message and encoded in base64: .. code:: bash base64 < Date: Sun, 10 Jan 2021 14:33:42 +0100 Subject: [PATCH 5/5] Adapt caf374e167940b65a695b741a4ec263b474d2016 (DHCP lease path) for user-data overhaul --- source/adminguide/virtual_machines/user-data.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/adminguide/virtual_machines/user-data.rst b/source/adminguide/virtual_machines/user-data.rst index 22c80a5047..24285e9246 100644 --- a/source/adminguide/virtual_machines/user-data.rst +++ b/source/adminguide/virtual_machines/user-data.rst @@ -76,7 +76,7 @@ possible to determine the user-data server from a DHCP lease. .. code:: bash - # cat /var/lib/dhclient/dhclient-eth0.leases | grep dhcp-server-identifier | tail -1 + # cat /var/lib/dhcp/dhclient.eth0.leases | grep dhcp-server-identifier | tail -1 #. Access the data-server via its IP