|
|
|
@@ -0,0 +1,209 @@ |
|
|
|
Building ``dds`` from Source |
|
|
|
############################ |
|
|
|
|
|
|
|
While prebuilt ``dds`` executables are `available on the GitHub page |
|
|
|
<releases_>`_, one may wish to build ``dds`` from source. |
|
|
|
|
|
|
|
.. _releases: https://github.com/vector-of-bool/dds/releases |
|
|
|
|
|
|
|
The ``dds`` build process is designed to be as turn-key simple as possible. |
|
|
|
|
|
|
|
|
|
|
|
Platform Support |
|
|
|
**************** |
|
|
|
|
|
|
|
``dds`` aims to be as cross-platform as possible. It currently build and |
|
|
|
executes on Windows, macOS, Linux, and FreeBSD. Support for additional |
|
|
|
platforms is possible but will require modifications to ``bootstrap.py`` that |
|
|
|
will allow it to be built on such platforms. |
|
|
|
|
|
|
|
|
|
|
|
Build Requirements |
|
|
|
****************** |
|
|
|
|
|
|
|
Building ``dds`` has a simple set of requirements: |
|
|
|
|
|
|
|
- **Python 3.6** or newer to run the bootstrap/CI scripts. |
|
|
|
- A C++ compiler that has rudimentary support for C++20 concepts. Newer |
|
|
|
releases of Visual C++ that ship with **VS 2019** will be sufficient on |
|
|
|
Windows, as will **GCC 9** with ``-fconcepts`` on other platforms. |
|
|
|
|
|
|
|
.. note:: |
|
|
|
On Windows, you will need to execute the build from within a Visual C++ |
|
|
|
enabled environment. This will involve launching the build from a Visual |
|
|
|
Studio Command Prompt. |
|
|
|
|
|
|
|
.. note:: |
|
|
|
At the time of writing, C++20 Concepts has not yet been released in Clang, |
|
|
|
but should be available in LLVM/Clang 11 and newer. |
|
|
|
|
|
|
|
|
|
|
|
Build Scripts and the CI Process |
|
|
|
******************************** |
|
|
|
|
|
|
|
The main CI process is driven by Python. The root CI script is ``tools/ci.py``, |
|
|
|
and it accepts several command-line parameters. Only a few of are immediate |
|
|
|
interest: |
|
|
|
|
|
|
|
``--bootstrap-with=<method>`` or ``-B <method>`` |
|
|
|
Tell ``ci.py`` how to obtain the previous ``dds`` executable that can build |
|
|
|
the *current* ``dds`` source tree. This accepts one of three values: |
|
|
|
``skip``, ``download``, or ``build``. Refer to :ref:`bootstrapping`. |
|
|
|
|
|
|
|
``--build-only`` |
|
|
|
A flag that tells ``ci.py`` to exit after it has successfully built the |
|
|
|
current source tree, and to not execute the phase-2 build nor the automated |
|
|
|
tests. |
|
|
|
|
|
|
|
``--toolchain=<path>`` or ``-T <path>`` |
|
|
|
Tell ``ci.py`` what toolchain to give to the prior ``dds`` to build the |
|
|
|
current ``dds``. |
|
|
|
|
|
|
|
The ``ci.py`` script performs the following actions, in order: |
|
|
|
|
|
|
|
#. Prepare the build output directory |
|
|
|
#. Bootstrap the prior version of ``dds`` that will build the current version. |
|
|
|
#. Import the embedded ``catalog.json`` into a catalog database stored within |
|
|
|
``_build/``. This will be used to resolve the third-party packages that |
|
|
|
``dds`` itself uses. |
|
|
|
#. Invoke the build of ``dds`` using the prebuilt ``dds`` from the prior |
|
|
|
bootstrap phase. If ``--build-only`` was specified, the CI script stops |
|
|
|
here. |
|
|
|
#. Use the new ``dds`` executable to rebuild itself *again* (phase-2 self-build |
|
|
|
test). A bit of a "sanity test." |
|
|
|
#. Execute the test suite using ``pytest``. |
|
|
|
|
|
|
|
|
|
|
|
.. _bootstrapping: |
|
|
|
|
|
|
|
Bootstrapping ``dds`` |
|
|
|
********************* |
|
|
|
|
|
|
|
In the beginning, ``dds`` was built by a Python script that globbed the sources |
|
|
|
and invoked the compiler+linker on those sources. Once ``dds`` was able to |
|
|
|
build and link itself, this Python script was replaced instead with ``dds`` |
|
|
|
building itself. ``dds`` has never used another build system. |
|
|
|
|
|
|
|
The ``ci.py`` script accepts one of three methods for the ``--bootstrap-with`` |
|
|
|
flag: ``skip``, ``download``, or ``build``. |
|
|
|
|
|
|
|
Once bootstrapping is complete, a ``dds`` executable will be written to |
|
|
|
``_prebuilt/dds``. This executable refers to a **previous** version of ``dds`` |
|
|
|
that is able to build the newer ``dds`` source tree. |
|
|
|
|
|
|
|
.. note:: |
|
|
|
For all development work on ``dds``, the ``_prebuilt/dds`` executable should |
|
|
|
always be used. This means that newer ``dds`` features are not available |
|
|
|
for use within the ``dds`` repository. |
|
|
|
|
|
|
|
|
|
|
|
Bootstrap: ``skip`` |
|
|
|
=================== |
|
|
|
|
|
|
|
If given ``skip``, ``ci.py`` will not perform any bootstrapping steps. It will |
|
|
|
assume that there is an existing ``_prebuilt/dds`` executable. This option |
|
|
|
should be used once bootstrapping has been performed at least once with another |
|
|
|
method, as this is much faster than rebuilding/redownloading every time. |
|
|
|
|
|
|
|
|
|
|
|
Bootstrap: ``download`` |
|
|
|
======================= |
|
|
|
|
|
|
|
The ``ci.py`` script has a reference to a download URL of the prior version of |
|
|
|
``dds`` that has been designated for the bootstrap. These executables originate |
|
|
|
from `the GitHub releases <releases_>`_ page. |
|
|
|
|
|
|
|
If given ``download``, then ``ci.py`` will download a predetermined ``dds`` |
|
|
|
executable and use it to perform the remainder of the build. |
|
|
|
|
|
|
|
|
|
|
|
Bootstrap: ``build`` |
|
|
|
==================== |
|
|
|
|
|
|
|
Another script, ``tools/bootstrap.py`` is able to build ``dds`` from the ground |
|
|
|
up. It works by progressively cloning previous versions of the ``dds`` |
|
|
|
repository and using them to build the next commit in the chain. |
|
|
|
|
|
|
|
While this is a neat trick, it isn't necessary for most development, as the |
|
|
|
resulting executable will be derived from the same commit as the executable |
|
|
|
that would be obtained using the ``download`` method. This is also more fragile |
|
|
|
as the past commits may make certain assumptions about the system that might |
|
|
|
not be true outside of the CI environment. The build process may be tweaked in |
|
|
|
the future to correct these assumptions. |
|
|
|
|
|
|
|
|
|
|
|
Selecting a Build Toolchain |
|
|
|
*************************** |
|
|
|
|
|
|
|
``dds`` includes three toolchains that it uses to build itself in its CI |
|
|
|
environment: ``tools/gcc-9.jsonc`` for Linux and macOS, |
|
|
|
``tools/freebsd-gcc-9.jsonc`` for FreeBSD, and ``tools/msvc.jsonc`` for |
|
|
|
Windows. |
|
|
|
|
|
|
|
While these toolchains will work perfectly well in CI, you may need to tweak |
|
|
|
these for your build setup. For example: ``gcc-9.jsonc`` assumes that the GCC 9 |
|
|
|
executables are named ``gcc-9`` and ``g++-9``, which is incorrect on some |
|
|
|
Linux distributions. |
|
|
|
|
|
|
|
It is recommended to tweak these files as necessary to get the build working on |
|
|
|
your system. However, do not include those tweaks in a commit unless they are |
|
|
|
necessary to get the build running in CI. |
|
|
|
|
|
|
|
|
|
|
|
Giving a Toolchain to ``ci.py`` |
|
|
|
=============================== |
|
|
|
|
|
|
|
Just like passing a toolchain to ``dds``, ``ci.py`` also requires a toolchain. |
|
|
|
Simply pass the path to your desired toolchain using the ``--toolchain``/ |
|
|
|
``-T`` argument: |
|
|
|
|
|
|
|
.. code-block:: bash |
|
|
|
|
|
|
|
$ python3 tools/ci.py [...] -T tools/gcc-9.jsonc |
|
|
|
|
|
|
|
|
|
|
|
Building for Development |
|
|
|
************************ |
|
|
|
|
|
|
|
While ``ci.py`` is rigorous in maintaining a clean and reproducible environment, |
|
|
|
we often don't need such rigor for a rapid development iteration cycle. Instead |
|
|
|
we can invoke the build command directly in the same way that ``ci.py`` does |
|
|
|
it: |
|
|
|
|
|
|
|
.. code-block:: bash |
|
|
|
|
|
|
|
$ _prebuilt/dds build -t [toolchain] \ |
|
|
|
--catalog _build/catalog.db \ |
|
|
|
--repo-dir _build/ci-repo |
|
|
|
|
|
|
|
The ``--catalog`` and ``--repo-dir`` arguments are not strictly necessary, but |
|
|
|
help to isolate the ``dds`` dev environment from the user-local ``dds`` |
|
|
|
environment. This is important if modifications are made to the catalog |
|
|
|
database schema that would conflict with the one of an external ``dds`` |
|
|
|
version. |
|
|
|
|
|
|
|
.. note:: |
|
|
|
You'll likely want to run ``ci.py`` *at least once* for it to prepare the |
|
|
|
necessary ``catalog.db``. |
|
|
|
|
|
|
|
.. note:: |
|
|
|
As mentioned previously, if using MSVC, the above command must execute with |
|
|
|
the appropriate VS development environment enabled. |
|
|
|
|
|
|
|
|
|
|
|
Running the Test Suite |
|
|
|
********************** |
|
|
|
|
|
|
|
The ``--build-only`` flag for ``ci.py`` will disable test execution. When this |
|
|
|
flag is omitted, ``ci.py`` will execute a self-build sanity test and then |
|
|
|
execute the main test suite, which is itself written as a set of ``pytest`` |
|
|
|
tests in the ``tests/`` subdirectory. |
|
|
|
|
|
|
|
|
|
|
|
Unit Tests |
|
|
|
========== |
|
|
|
|
|
|
|
Various pieces of ``dds`` contain unit tests. These are stored within the |
|
|
|
``src/`` directory itself in ``*.test.cpp`` files. They are built and executed |
|
|
|
by the bootstrapped ``dds`` executable unconditionally. These tests execute |
|
|
|
in milliseconds and do not burden the development iteration cycle. |
vector-of-bool
4 years ago