Skip to content

[WIP] installguide: doc improvements around building from source #164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
falcon78921 wants to merge 1 commit into from
Closed

[WIP] installguide: doc improvements around building from source #164

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
173 changes: 87 additions & 86 deletions source/installguide/building_from_source.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,64 +14,70 @@
under the License.


############
Introduction
------------
############

The official CloudStack release is always in source code form. You will
likely be able to find "convenience binaries," the source is the
canonical release. In this section, we'll cover acquiring the source
release and building that so that you can deploy it using Maven or
create Debian packages or RPMs.
likely be able to find "convenience binaries", where the source is the
canonical release. In this section, we will cover acquiring and building
the source release so that you can deploy using Maven or
build packages for distribution (e.g. ``.deb``, ``.rpm``).

Note that building and deploying directly from source is typically not
the most efficient way to deploy an IaaS. However, we will cover that
method as well as building RPMs or Debian packages for deploying
CloudStack.
.. note::

Building and deploying directly from source is typically not
the most efficient way to deploy CloudStack. Also, please be aware that
development branches may contain unstable code. If you run into any issues
during the build process, please check the CloudStack `Issues
<https://github.com/apache/cloudstack/issues>`_ page on GitHub. You may find
that the issue you are experiencing has already been reported.

The instructions here are likely version-specific (i.e. Building 4.7.x
is different from building 4.2.x).

The instructions here are likely version-specific. That is, the method
for building from source for the 4.7.x series is different from the
4.2.x series.
If you are working with an unreleased version of CloudStack, please read the
``INSTALL.md`` file in the top-level directory of the release.

If you are working with a unreleased version of CloudStack, see the
INSTALL.md file in the top-level directory of the release.


Downloading the release
-----------------------
Downloading the Release
=======================

You can download the latest CloudStack release from the `Apache
CloudStack project download page
CloudStack project download page
<http://cloudstack.apache.org/downloads.html>`_.

Prior releases are available via archive.apache.org as well. See the
downloads page for more information on archived releases.
Prior releases are available via `archive.apache.org
<https://archive.apache.org>`_ as well. Please see the
Downloads page for more information on archived releases.

You'll notice several links under the 'Latest release' section. A link
to a file ending in ``tar.bz2``, as well as a PGP/GPG signature, MD5,
You will notice several links under the *Latest CloudStack Releases* section.
A link to a file ending in ``tar.bz2``, as well as a PGP/GPG signature, MD5,
and SHA512 file.

- The ``tar.bz2`` file contains the Bzip2-compressed tarball with the
- The ``tar.bz2`` file contains the bzip2-compressed tarball with the
source code.

- The ``.asc`` file is a detached cryptographic signature that can be
used to help verify the authenticity of the release.
used to verify release authenticity.

- The ``.md5`` file is an MD5 hash of the release to aid in verify the
validity of the release download.
- The ``.md5`` file is a MD5 hash that can be used to verify release
authenticity.

- The ``.sha`` file is a SHA512 hash of the release to aid in verify
the validity of the release download.
- The ``.sha`` file is a SHA512 hash that can be used to verify release
authenticity.


Verifying the downloaded release
--------------------------------
Verifying the Downloaded Release
================================

There are a number of mechanisms to check the authenticity and validity
of a downloaded release.


Getting the KEYS
~~~~~~~~~~~~~~~~
----------------

To enable you to verify the GPG signature, you will need to download the
`KEYS <http://www.apache.org/dist/cloudstack/KEYS>`_ file.
Expand All @@ -85,7 +91,7 @@ You next need to import those keys, which you can do by running:


GPG
~~~
---

The CloudStack project provides a detached GPG signature of the release.
To check the signature, run the following command:
Expand All @@ -94,47 +100,46 @@ To check the signature, run the following command:

$ gpg --verify apache-cloudstack-|release|-src.tar.bz2.asc

If the signature is valid you will see a line of output that contains
'Good signature'.
If the signature is valid, you will see a line of output that contains
``Good signature``.


MD5
~~~
---

In addition to the cryptographic signature, CloudStack has an MD5
checksum that you can use to verify the download matches the release.
checksum that you can use to verify the downloaded release's validity.
You can verify this hash by executing the following command:

.. parsed-literal::

$ gpg --print-md MD5 apache-cloudstack-|release|-src.tar.bz2 | diff - apache-cloudstack-|release|-src.tar.bz2.md5

If this successfully completes you should see no output. If there is any
output from them, then there is a difference between the hash you
generated locally and the hash that has been pulled from the server.
Seeing no output means that the MD5 check completed successfully.
If there is any output, please make sure that the hash you generated locally and
the hash that has been pulled from the server are consistent.


SHA512
~~~~~~
------

In addition to the MD5 hash, the CloudStack project provides a SHA512
cryptographic hash to aid in assurance of the validity of the downloaded
cryptographic hash, which can assist in verifying the validity of the downloaded
release. You can verify this hash by executing the following command:

.. parsed-literal::

$ gpg --print-md SHA512 apache-cloudstack-|release|-src.tar.bz2 | diff - apache-cloudstack-|release|-src.tar.bz2.sha
$ gpg --print-md SHA512 apache-cloudstack-|release|-src.tar.bz2 | diff - apache-cloudstack-|release|-src.tar.bz2.sha512

If this command successfully completes you should see no output. If
there is any output from them, then there is a difference between the
hash you generated locally and the hash that has been pulled from the
server.
Seeing no output means that the SHA512 check completed successfully.
If there is any output, please make sure that the hash you generated locally and
the hash that has been pulled from the server are consistent.


Prerequisites for building Apache CloudStack
--------------------------------------------
Prerequisites for Building Apache CloudStack
============================================

There are a number of prerequisites needed to build CloudStack. This
There are a number of prerequisites for building CloudStack. This
document assumes compilation on a Linux system that uses RPMs or DEBs
for package management.

Expand All @@ -155,8 +160,8 @@ You will need, at a minimum, the following to compile CloudStack:
#. rpmbuild or dpkg-dev


Extracting source
-----------------
Extracting Source
=================

Extracting the CloudStack release is relatively simple and can be done
with a single command as follows:
Expand All @@ -171,53 +176,50 @@ You can now move into the directory:

$ cd ./apache-cloudstack-|release|-src

Install new MySQL connector
---------------------------

Install Python MySQL connector using the official MySQL packages repository.

Installing MySQL Connector
==========================

MySQL connector APT repository
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(Ubuntu/Debian)
---------------

.. note::

Procedure below should work for Ubuntu 14.04 and 16.04.
For Ubuntu 18.04, please read later below.
The following procedure should work for Ubuntu 14.04 and 16.04.
Please continue on for information regarding Ubuntu 18.04.

Install the following package provided by MySQL to enable official repositories:

.. note::

If the download fails check the current version number. The version available may
If the download fails, check the current version number. The version available may
have been updated.

.. parsed-literal::

wget http://dev.mysql.com/get/mysql-apt-config_0.7.3-1_all.deb
sudo dpkg -i mysql-apt-config_0.7.3-1_all.deb

Make sure to activate the repository for MySQL connectors.
Please make sure to activate the repository for MySQL connectors:

.. parsed-literal::

sudo apt-get update
sudo apt-get install mysql-connector-python
sudo apt-get install mysql-connector-python

.. note::

Below is given a bit different procedure if you are compiling on Ubuntu 18.04

Due to default python version changes (and some others) in Ubuntu 18.04 version, we will need to install python 2.7, python-mysql.connector from Universe repo (instead from official MySQL repo) and later make sure we are using Java 8, since Java 10 comes as default
In Ubuntu 18.04, there are differences with the default Python.
We will need to install ```python-mysql.connector``` for Python 2.7 from the ```Universe```
repo, instead of from the official MySQL repo. We also need to make sure that
Java 11 is configured, since Ubuntu 18.04 ships with Java 10 by default.

.. parsed-literal::

apt-add-repository universe
apt-get install python-mysql.connector python-setuptools dh-systemd
(installs python 2.7 with needed dependencies)

MySQL connector RPM repository
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(RHEL/CentOS)
-------------

Add a new yum repo ``/etc/yum.repos.d/mysql.repo``:

Expand All @@ -238,7 +240,7 @@ Install mysql-connector

.. _building_deb_packages:

Building DEB packages
Building DEB Packages
---------------------

In addition to the bootstrap dependencies, you'll also need to install
Expand All @@ -250,7 +252,7 @@ several other dependencies. Note that we recommend using Maven 3.
$ sudo apt-get install python-software-properties
$ sudo apt-get update
$ sudo apt-get install debhelper openjdk-11-jdk libws-commons-util-java genisoimage libcommons-codec-java libcommons-httpclient-java liblog4j1.2-java maven

.. note::

If on Ubuntu 18.04, in above command, please replace "python-software-properties" with "software-properties-common"
Expand Down Expand Up @@ -397,10 +399,10 @@ You can now move on to the instructions under Install on Ubuntu.
Building RPMs from Source
-------------------------

As mentioned previously in `“Prerequisites for building Apache CloudStack”
<#prerequisites-for-building-apache-cloudstack>`_, you will need to install
As mentioned previously in `“Prerequisites for building Apache CloudStack”
<#prerequisites-for-building-apache-cloudstack>`_, you will need to install
several prerequisites before you can build packages for CloudStack. Here we'll
assume you're working with a 64-bit build of CentOS or Red Hat Enterprise
assume you're working with a 64-bit build of CentOS or Red Hat Enterprise
Linux.

.. parsed-literal::
Expand Down Expand Up @@ -449,16 +451,16 @@ minutes.

.. note::

Packaging has changed. If you've created packages for CloudStack
previously, you should be aware that the process has changed considerably
since the project has moved to using Apache Maven. Please be sure to follow
Packaging has changed. If you've created packages for CloudStack
previously, you should be aware that the process has changed considerably
since the project has moved to using Apache Maven. Please be sure to follow
the steps in this section closely.


Generating RPMS
~~~~~~~~~~~~~~~

Now that we have the prerequisites and source, you will cd to the
Now that we have the prerequisites and source, you will cd to the
`packaging/` directory.

.. parsed-literal::
Expand All @@ -470,7 +472,7 @@ Generating RPMs is done using the ``package.sh`` script:
.. parsed-literal::

$ ./package.sh -d centos63

For other supported options(like centos7), run ``./package.sh --help``

That will run for a bit and then place the finished packages in
Expand Down Expand Up @@ -537,19 +539,19 @@ to build from source.

.. warning::

Some of the plugins supported by CloudStack cannot be distributed with
CloudStack for licensing reasons. In some cases, some of the required
libraries/JARs are under a proprietary license. In other cases, the
required libraries may be under a license that's not compatible with
`Apache's licensing guidelines for third-party products
Some of the plugins supported by CloudStack cannot be distributed with
CloudStack for licensing reasons. In some cases, some of the required
libraries/JARs are under a proprietary license. In other cases, the
required libraries may be under a license that's not compatible with
`Apache's licensing guidelines for third-party products
<http://www.apache.org/legal/resolved.html#category-x>`_.

#. To build the Non-OSS plugins, you'll need to have the requisite JARs
installed under the ``deps`` directory.

Because these modules require dependencies that can't be distributed
with CloudStack you'll need to download them yourself. Links to the
most recent dependencies are listed on the `*How to build CloudStack*
most recent dependencies are listed on the `*How to build CloudStack*
<https://cwiki.apache.org/confluence/display/CLOUDSTACK/How+to+build+CloudStack>`_
page on the wiki.

Expand All @@ -567,6 +569,5 @@ to build from source.
$ mvn install -Dnoredist

#. Once you've built CloudStack with the ``noredist`` profile, you can
package it using the `“Building RPMs from Source” <#building-rpms-from-source>`_
package it using the `“Building RPMs from Source” <#building-rpms-from-source>`_
or `“Building DEB packages” <#building-deb-packages>`_ instructions.