Update user docs for alpha.6

docs/design.rst

@@ -3,13 +3,14 @@
 
 ``dds`` has been designed from the very beginning as an extremely opinionated
 hybrid *build system* and *package manager*. Unlike most build systems however,
-``dds`` has a hyper-specific focus on a particular aspect of software
-development: C and C++ libraries.
+``dds`` has a strong focus on a particular aspect of software development: C and
+C++ libraries.
 
 This may sound pointless, right? Libraries are useless unless we can use them
 to build applications!
 
-Indeed, applications *are* essential, but that is "not our job" with ``dds``.
+Indeed, applications *are* essential, and ``dds`` is able to build those as
+well.
 
 Another design decision is that ``dds`` is built to be driven by automated
 tools as well as humans. ``dds`` is not designed to entirely replace existing
@@ -32,8 +33,8 @@ incredible implementation challenges.
 
 Despite the vast amount of work put into build systems and tooling, virtually
 all developers are using them *incorrectly* and/or *dangerously* without
-realizing it. Despite this work, we seem to be a great distance from a unified
-library package distribution and consumption mechanism.
+realizing it, and we seem to be still a great distance from a unified library
+package distribution and consumption mechanism.
 
 
 Tabula Rasa
@@ -46,7 +47,7 @@ If you opt-in to have your library built by ``dds``, you forgoe
 *customizability* in favor of *simplicity* and *ease*.
 
 ``dds`` takes a look at what is needed to build and develop *libraries* and
-hyper-optimizes for that use case. It is also built with a very strong, very
+optimizes for that use case. It is also built with a very strong, very
 opinionated idea of *how* libraries should be constructed and used. These
 prescriptions are not at all arbitrary, though. They are built upon the
 observations of the strengths and weaknesses of build systems in use throughout
@@ -69,14 +70,14 @@ different, despite both using the same underlying "Build System."
 
 ``dds`` takes a massive divergence at this point. One project using ``dds`` as
 their build system has a nearly identical build process to every other project
-using ``dds``. Simply running :code:`dds build -t <toolchain>` should be enough
+using ``dds``. Simply running ``dds build`` should be enough
 to build *any* ``dds`` project.
 
 In order to reach this uniformity and simplicity, ``dds`` drops almost all
 aspects of project-by-project customizability. Instead, ``dds`` affords the
 developer a contract:
 
-    If you play by my rules, you get to play in my space.
+    If you play by the rules, you get to play in this space.
 
 
 .. _design.rules:
@@ -91,7 +92,7 @@ imposes, but what are they?
 .. _design.rules.not-apps:
 
 ``dds`` Is not Made for Complex Applications
-===============================================
+============================================
 
 Alright, this one isn't a "rule" as much as a recommendation: If you are
 building an application that *needs* some build process functionality that
@@ -105,22 +106,28 @@ violate any of the other existing rules.
     customization features to permit the rules to be bent arbitrarily: Read
     on.
 
-``dds`` contains a minimal amount of functionality for building simple
-applications, but it is certainly not its primary purpose.
+``dds`` *does* contain functionality for building applications, but they must
+also play by the rules.
+
+If you want to build a complex application with ``dds`` that uses lots of
+platform-specific sources, code generation, and conditional components, a good
+option is to use an external build script that prepares the project tree before
+invoking ``dds``.
 
 
 .. _design.rules.change:
 
-*Your* Code Should Be Changed Before ``dds`` Should Be Changed
-=================================================================
+*Your Code* Should Be Changed Before ``dds`` Should Be Changed
+==============================================================
 
-The wording of this rule means that the onus is on the library developer to
-meet the expectations that ``dds`` prescribes in order to make the build
-work.
+The wording of this rule means that the onus is on the developer to meet the
+expectations that ``dds`` prescribes in order to make the build work.
 
-If your library meets all the requirements outlined in this document but you
-still find trouble in making your build work, this is grounds for change in
-``dds``, either in clarifying the rules or tweaking ``dds`` functionality.
+If your project meets all the requirements outlined in this document but you
+still find trouble in making your build work, or if you *cannot* see *any*
+possible way for your project to be built by ``dds`` regardless of what changes
+you make, then it this is grounds for change in ``dds``, either in clarifying
+the rules or tweaking ``dds`` functionality
 
 
 .. _design.rules.layout:
@@ -154,9 +161,8 @@ conditional compilation.
 All Code Must Be in Place Before Building
 =========================================
 
-``dds`` does not provide code-generation functionality. Instead, any
-generated code should be generated and committed to the repository to be only
-ever modified through such generation scripts.
+``dds`` does not provide code-generation functionality. Instead, any generated
+code should be generated by separate build steps before ``dds`` is executed.
 
 
 .. _design.rules.one-binary-per-src:
@@ -176,7 +182,7 @@ No Arbitrary ``#include`` Directories
 =====================================
 
 Only ``src/`` and ``include/`` will ever be used as the basis for header
-resolution while building a library, so all ``#include`` directives should be
+resolution while building a project, so all ``#include`` directives should be
 relative to those directories. Refer to :ref:`pkg.source-root`.
 
 
@@ -185,7 +191,7 @@ relative to those directories. Refer to :ref:`pkg.source-root`.
 All Files Compile with the Same Options
 =======================================
 
-When DDS compiles a library, every source file will be compiled with an
+When DDS compiles a project, every source file will be compiled with an
 identical set of options. Additionally, when DDS compiles a dependency tree,
 every library in that dependency tree will be compiled with an identical set of
 options. Refer to the :doc:`guide/toolchains` page for more information.

docs/err/git-url-ref-mutual-req.rst

@@ -12,4 +12,4 @@ as the ``ref`` requires support from the remote Git server, and it is often
 unavailable in most setups). Using a Git tag is strongly recommended.
 
 .. seealso::
-    Refer to the documentation on :doc:`/guide/catalog`.
+    Refer to the documentation on :doc:`/guide/remote-pkgs`.

docs/err/invalid-catalog-json.rst

@@ -1,9 +0,0 @@
-Error: Invalid catalog JSON
-###########################
-
-This error occurs when the JSON data given to import into the package catalog
-is in some way invalid. Refer to the catalog documentation for a description of
-the proper JSON format.
-
-.. seealso::
-    :ref:`catalog.adding`

docs/err/invalid-repo-transform.rst

@@ -1,10 +0,0 @@
-Error: A repository filesystem transformation is invalid
-########################################################
-
-In ``dds``, a catalog entry can have a list of attached "transforms" that will
-be applies to the root directory of the package before ``dds`` tries to build
-and use it.
-
-.. seealso::
-    For information on the shape and purpose of transforms, refer to
-    :ref:`catalog.fs-transform` on the :doc:`/guide/catalog` page.

docs/err/no-catalog-remote-info.rst

@@ -6,5 +6,3 @@ requires some information regarding how to actually *acquire* that package
 when it is requested.
 
 If such information is not provided, ``dds`` will issue an error.
-
-.. seealso:: :ref:`catalog.adding`.

docs/err/no-such-catalog-package.rst

@@ -8,10 +8,3 @@ in the catalog.
 It is possible that the intended package *does exist* but that the spelling of
 the package name or version number is incorrect. Firstly, check your spelling
 and that the version number you have requested is correct.
-
-In another case, it is possible that the package *exists somewhere*, but has
-not been loaded into the local catalog. As of this writing, ``dds`` does not
-automatically maintain the catalog against a central package repository, so
-package entries must be loaded and imported manually. If you believe this to be
-the case, refer to the section on the :doc:`/guide/catalog`, especially the
-section :ref:`catalog.adding`.

docs/err/sdist-exists.rst

@@ -19,7 +19,7 @@ write a source distribution to the named path, it would be required to delete
 whatever exists there before creating the source distribution.
 
 .. warning::
-    When using ``dds sdist create`` with the ``--out <path>`` parameter, the
+    When using ``dds pkg create`` with the ``--out <path>`` parameter, the
     ``<path>`` given **is not the directory in which to place the source
     distribution, but the filepath to the source distribution itself**!
 
@@ -27,7 +27,7 @@ whatever exists there before creating the source distribution.
     distribution in that directory, **the following command is incorrect**::
 
         # Do not do this:
-        dds sdist create --out foo/
+        dds pkg create --out foo/
 
     If you pass ``--replace`` to the above command, ``dds`` will **destroy the
     existing directory** and replace it with the source distribution!
@@ -35,4 +35,4 @@ whatever exists there before creating the source distribution.
     You **must** provide the full path to the source distribution::
 
         # Do this:
-        dds sdist create --out foo/my-project.dsd
+        dds pkg create --out foo/my-project.tar.gz

docs/guide/build-deps.rst

@@ -0,0 +1,187 @@
+Building and Using ``dds`` in Another Build System
+##################################################
+
+One of ``dds``'s primary goals is to inter-operate with other build systems
+cleanly. One of ``dds``'s primary outputs is *libman* package indices. These
+package indices can be imported into other build systems that support the
+`libman`_ format. (At the time of writing there is a CMake module which can do
+the import, but other build systems are planned.)
+
+.. _libman: https://api.csswg.org/bikeshed/?force=1&url=https://raw.githubusercontent.com/vector-of-bool/libman/develop/data/spec.bs
+
+.. _PMM: https://github.com/vector-of-bool/PMM
+
+.. _CMakeCM: https://github.com/vector-of-bool/CMakeCM
+
+.. _lm-cmake: https://raw.githubusercontent.com/vector-of-bool/libman/develop/cmake/libman.cmake
+
+
+.. _build-deps.gen-libman:
+
+Generating a libman Index
+*************************
+
+Importing libman packages into a build system requires that we have a libman
+index generated on the filesystem. **This index is not generated globally**: It
+is generated on a per-build basis as part of the build setup. The index will
+describe in build-system-agnostic terms how to include a set of packages and
+libraries as part of a build.
+
+``dds`` has first-class support for generating this index. The ``build-deps``
+subcommand of ``dds`` will download and build a set of dependencies, and places
+an ``INDEX.lmi`` file that can be used to import the built results.
+
+
+Declaring Dependencies
+======================
+
+``dds build-deps`` accepts a list of dependency statements as command line
+arguments, but it may be useful to specify those requirements in a file.
+
+``dds build-deps`` accepts a JSON5 file describing the dependencies of a
+project as well. This file is similar to a very stripped-down version of a
+``dds`` :ref:`package manifest <pkgs.pkgs>`, and only includes the ``depends``
+key. (The presence of any other key is an error.)
+
+Here is a simple dependencies file that declares a single requirement:
+
+.. code-block:: js
+  :caption: ``dependencies.json5``
+
+  {
+    depends: [
+      'neo-sqlite3^0.2.0',
+    ]
+  }
+
+
+Building Dependencies and the Index
+===================================
+
+We can invoke ``dds build-deps`` and give it the path to this file:
+
+.. code-block:: bash
+
+  $ dds build-deps --deps-file dependencies.json5
+
+When finished, ``dds`` will write the build results into a subdirectory called
+``_deps`` and generate a file named ``INDEX.lmi``. This file is ready to be
+imported into any build system that can understand libman files.
+
+.. note::
+  The output directory and index filepath can be controlled with the
+  ``--out`` and ``--lmi-path`` flags, respectively.
+
+
+Importing an Index: CMake
+*************************
+
+.. highlight:: cmake
+
+.. note::
+
+  This section discusses how to import ``INDEX.lmi`` into CMake, but ``dds``
+  also has built-in support for generating a CMake targets file. See
+  :doc:`cmake` and :ref:`cmake.pmm` for even simpler integration steps.
+
+Supposed that we've generated a libman index and set of packages, and we want to
+import them into CMake. CMake doesn't know how to do this natively, but there
+exists a single-file module for CMake that allows CMake to import libraries from
+libman indices without any additional work.
+
+The module is not shipped with CMake, but is available online as a single
+stand-alone file. The `libman.cmake <lm-cmake_>`_ file can be downloaded and
+added to a project directly, or it can be obtained automatically through a
+CMake tool like `PMM`_ (recommended).
+
+
+Getting ``libman.cmake`` via PMM
+================================
+
+Refer to the ``README.md`` file in `the PMM repo <PMM_>`_ for information on how
+to get PMM into your CMake project. In short, download and place the
+``pmm.cmake`` file in your repository, and ``include()`` the file near the top
+of your ``CMakeLists.txt``::
+
+  include(pmm.cmake)
+
+Once it has been included, you can call the ``pmm()`` function. To obtain
+*libman*, we need to start by enabling `CMakeCM`_::
+
+  pmm(CMakeCM ROLLING)
+
+.. warning::
+  It is not recommended to use the ``ROLLING`` mode, but it is the easiest to
+  use when getting started. For reproducible and reliable builds, you should
+  pin your CMakeCM version using the ``FROM <url>`` argument.
+
+Enabling CMakeCM will make available all of the CMake modules available in `the
+CMakeCM repository <CMakeCM_>`_, which includes `libman.cmake <lm-cmake_>`_.
+
+After the call to ``pmm()``, simply ``include()`` the ``libman`` module::
+
+  include(libman)
+
+That's it! The only function from the module that we will care about for now
+is the ``import_packages()`` function.
+
+
+Importing Our Dependencies' Packages
+====================================
+
+To import a package from a libman tree, we need only know the *name* of the
+package we wish to import. In our example case above, we depend on
+``neo-sqlite3``, so we simply call the libman-CMake function
+``import_packages()`` with that package name::
+
+  import_packages("neo-sqlite3")
+
+You'll note that we don't request any particular version of the package: All
+versioning resolution is handled by ``dds``. You'll also note that we don't
+need to specify our transitive dependencies: This is handled by the libman
+index that was generated by ``dds``: It will automatically ``import_packages()``
+any of the transitive dependencies required.
+
+More than one package name can be provided to a single call to
+``import_packages()``, and ``import_packages()`` may be called multiple times
+within a CMake project.
+
+
+Using Our Dependencies' Libraries
+=================================
+
+Like with ``dds``, CMake wants us to explicitly declare how our build targets
+*use* other libraries. When we import a package from a libman index, the
+import will generate CMake ``IMPORTED`` targets that can be linked against.
+
+In ``dds`` and in libman, a library is identified by a combination of
+*namespace* and *name*, joined together with a slash ``/`` character. This
+*qualified name* of a library is decided by the original package author, and
+should be documented. In the case of ``neo-sqlite3``, the only library is
+``neo/sqlite3``.
+
+When the libman CMake module imports a library, it creates a qualified name
+using a double-colon "``::``" instead of a slash. As such, our ``neo/sqlite3``
+is imported in CMake as ``neo::sqlite3``. We can link against it as we would
+with any other target::
+
+  add_executable(my-application app.cpp)
+  target_link_libraries(my-application PRIVATE neo::sqlite3)
+
+Altogether, here is the final CMake file:
+
+.. code-block::
+  :caption: ``CMakeLists.txt``
+  :linenos:
+
+  cmake_minimum_required(VERSION 3.15)
+  project(MyApplication VERSION 1.0.0)
+
+  include(pmm.cmake)
+  pmm(CMakeCM ROLLING)
+
+  include(libman)
+  import_packages("neo-sqlite3")
+
+  add_executable(my-application app.cpp)
+  target_link_libraries(my-application PRIVATE neo::sqlite3)

docs/guide/catalog.rst

@@ -1,286 +0,0 @@
-The Package Catalog
-###################
-
-``dds`` stores a catalog of available packages, along with their dependency
-statements and information about how a source distribution thereof may be
-maintained.
-
-
-Viewing Catalog Contents
-************************
-
-The default catalog database is stored in a user-local location, and the
-package IDs available can be listed with ``dds catalog list``. This will only
-list the IDs of the packages, but none of the additional metadata about them.
-
-
-.. _catalog.adding:
-
-Adding Packages to the Catalog
-******************************
-
-The ``dds catalog import`` supports a ``--json`` flag that specifies a JSON5
-file from which catalog entries will be generated.
-
-.. note::
-    The ``--json`` flag can be passed more than once to import multiple JSON
-    files at once.
-
-The JSON file has the following structure:
-
-.. code-block:: javascript
-
-  {
-    // Import version spec.
-    version: 1,
-    // Packages section
-    packages: {
-      // Subkeys are package names
-      "acme-gadgets": {
-        // Keys within the package names are the versions that are
-        // available for each package.
-        "0.4.2": {
-          // `depends` is an array of dependency statements for this
-          // particular version of the package. (Optional)
-          depends: [
-            "acme-widgets^1.4.1"
-          ],
-          // `description` is an attribute to give a string to describe
-          // the package. (Optional)
-          description: "A collection of useful gadgets.",
-          // Specify the Git remote information
-          git: {
-            // `url` and `ref` are required.
-            url: "http://example.com/git/repo/acme-gadgets.git",
-            ref: "v0.4.2-stable",
-            // The `auto-lib` is optional, to specify an automatic
-            // library name/namespace pair to generate for the
-            // root library
-            "auto-lib": "Acme/Gadgets",
-            // List of filesystem transformations to apply to the repository
-            // (optional)
-            transform: [
-              // ... (see below) ...
-            ]
-          }
-        }
-      }
-    }
-  }
-
-
-.. _catalog.fs-transform:
-
-Filesystem Transformations
-**************************
-
-.. note::
-  Filesystem transformations is a transitional feature that is likely to be
-  removed in a future release, and replaced with a more robust system when
-  ``dds`` has a better way to download packages. Its aim is to allow ``dds``
-  projects to use existing libraries that might not meet the layout
-  requirements that ``dds`` imposes, but can otherwise be consumed by ``dds``
-  with a few tweaks.
-
-A catalog entry can have a set of filesystem transformations attached to its
-remote information (e.g. the ``git`` property). When ``dds`` is obtaining a
-copy of the code for the package, it will apply the associated transformations
-to the filesystem based in the directory of the downloaded/cloned directory. In
-this way, ``dds`` can effectively "patch" the filesystem structure of a project
-arbitrarily. This allows many software projects to be imported into ``dds``
-without needing to patch/fork the original project to support the required
-filesystem structure.
-
-.. important::
-  While ``dds`` allows you to patch directories downloaded via the catalog, a
-  native ``dds`` project must still follow the layout rules.
-
-  The intention of filesystem transformations is to act as a "bridge" that will allow ``dds`` projects to more easily utilize existing libraries.
-
-
-Available Transformations
-=========================
-
-At time of writing, there are five transformations available to catalog entries:
-
-``copy`` and ``move``
-  Copies or moves a set of files/directories from one location to another. Allows the following options:
-
-  - ``from`` - The path from which to copy/move. **Required**
-  - ``to`` - The destination path for the copy/move. **Required**
-  - ``include`` - A list of globbing expressions for files to copy/move. If
-    omitted, then all files will be included.
-  - ``exclude`` - A list of globbing expressions of files to exclude from the
-    copy/move. If omitted, then no files will be excluded. **If both** ``include`` and ``exclude`` are provided, ``include`` will be checked *before* ``exclude``.
-  - ``strip-components`` - A positive integer (or zero, the default). When the
-    ``from`` path identifies a directory, its contents will be copied/moved
-    into the destination and maintain their relative path from the source path as their relative path within the destination. If ``strip-components`` is set to an integer ``N``, then the first ``N`` path components of that relative path will be removed when copying/moving the files in a directory. If a file's relative path has less than ``N`` components, then that file will be excluded from the ``copy/move`` operation.
-
-``remove``
-  Delete files and directories from the package source. Has the following options:
-
-  - ``path`` - The path of the file/directory to remove. **Required**
-  - ``only-matching`` - A list of globbing expressions for files to remove. If omitted and the path is a directory, then the entire directory will be deleted. If at least one pattern is provided, then directories will be left intact and only non-directory files will be removed. If ``path`` names a non-directory file, then this option has no effect.
-
-``write``
-  Write the contents of a string to a file in the package source. Has the following options:
-
-  - ``path`` - The path of the file to write. **Required**
-  - ``content`` - A string that will be written to the file. **Required**
-
-  If the file exists and is not a directory, the file will be replaced. If the
-  path names an existing directory, an error will be generated.
-
-``edit``
-  Modifies the contents of the files in the package.
-
-  - ``path`` - Path to the file to edit. **Required**
-  - ``edits`` - An array of edit objects, applied in order, with the following
-    keys:
-
-    - ``kind`` - One of ``insert`` or ``delete`` to insert/delete lines,
-      respectively. **Required**
-    - ``line`` - The line at which to perform the insert/delete. The first line
-      of the file is line one, *not* line zero. **Required**
-    - ``content`` - For ``insert``, the string content to insert into the file.
-      A newline will be appended after the content has been inserted.
-
-Transformations are added as a JSON array to the JSON object that specifies
-the remote information for the package. Each element of the array is an
-object, with one or more of the keys listed above. If an object features more
-than one of the above keys, they are applied in the same order as they have
-been listed.
-
-
-Example: Crypto++
-=================
-
-The following catalog entry will build and import `Crypto++`_ for use by a
-``dds`` project. This uses the unmodified Crypto++ repository, which ``dds``
-doesn't know how to build immediately. With some simple moving of files, we
-end up with something ``dds`` can build directly:
-
-.. code-block:: javascript
-
-  "cryptopp": {
-    "8.2.0": {
-      "git": {
-        "url": "https://github.com/weidai11/cryptopp.git",
-        "ref": "CRYPTOPP_8_2_0",
-        "auto-lib": "cryptopp/cryptopp",
-        "transform": [
-          {
-            // Crypto++ has no source directories at all, and everything lives
-            // at the top level. No good for dds.
-            //
-            // Clients are expected to #include files with a `cryptopp/` prefix,
-            // so we need to move the files around so that they match the
-            // expected layout:
-            "move": {
-              // Move from the root of the repo:
-              "from": ".",
-              // Move files *into* `src/cryptopp`
-              "to": "src/cryptopp",
-              // Only move the C++ sources and headers:
-              "include": [
-                "*.c",
-                "*.cpp",
-                "*.h"
-              ]
-            }
-          }
-        ]
-      }
-    }
-  }
-
-
-Example: libsodium
-==================
-
-For example, this catalog entry will build and import `libsodium`_ for use in
-a ``dds`` project. This uses the upstream libsodium repository, which does not
-meet the layout requirements needed by ``dds``. With a few simple
-transformations, we can allow ``dds`` to build and consume libsodium
-successfully:
-
-.. code-block:: javascript
-
-  "libsodium": {
-    "1.0.18": {
-      "git": {
-        "url": "https://github.com/jedisct1/libsodium.git",
-        "ref": "1.0.18",
-        "auto-lib": "sodium/sodium",
-        /// Make libsodium look as dds expects of a project.
-        "transform": [
-          // libsodium has a `src` directory, but it does not look how dds
-          // expects it to. The public `#include` root of libsodium lives in
-          // a nested subdirectory of `src/`
-          {
-            "move": {
-              // Move the public header root out from that nested subdirectory
-              "from": "src/libsodium/include",
-              // Put it at `include/` in the top-level
-              "to": "include/"
-            }
-          },
-          // libsodium has some files whose contents are generated by a
-          // configure script. For demonstration purposes, we don't need most
-          // of them, and we can just swipe an existing pre-configured file
-          // that is already in the source repository and put it into the
-          // public header root.
-          {
-            "copy": {
-              // Generated version header committed to the repository:
-              "from": "builds/msvc/version.h",
-              // Put it where the configure script would put it:
-              "to": "include/sodium/version.h"
-            }
-          },
-          // The subdirectory `src/libsodium/` is no good. It now acts as an
-          // unnecessary layer of indirection. We want `src/` to be the root.
-          // We can just "lift" the subdirectory:
-          {
-            // Up we go:
-            "move": {
-              "from": "src/libsodium",
-              "to": "src/"
-            },
-            // Delete the now-unused subdirectory:
-            "remove": {
-              "path": "src/libsodium"
-            }
-          },
-          // Lastly, libsodium's source files expect to resolve their header
-          // paths differently than they expect of their clients (Bad!!!).
-          // Fortunately, we can do a hack to allow the files in `src/` to
-          // resolve its headers. The source files use #include as if the
-          // header root was `include/sodium/`, rather than `include/`.
-          // To work around this, generate a copy of each header file in the
-          // source root, but remove the leading path element.
-          // Because we have a separate `include/` and `src/` directory, dds
-          // will only expose the `include/` directory to clients, and the
-          // header copies in `src/` are not externally visible.
-          //
-          // For example, the `include/sodium/version.h` file is visible to
-          // clients as `sodium/version.h`, but libsodium itself tries to
-          // include it as `version.h` within its source files. When we copy
-          // from `include/`, we grab the relative path to `sodium/version.h`,
-          // strip the leading components to get `version.h`, and then join that
-          // path with the `to` path to generate the full destination at
-          // `src/version.h`
-          {
-            "copy": {
-              "from": "include/",
-              "to": "src/",
-              "strip-components": 1
-            }
-          }
-        ]
-      }
-    }
-  }
-
-.. _libsodium: https://doc.libsodium.org/
-.. _Crypto++: https://cryptopp.com/

docs/guide/cmake.rst

@@ -1,240 +1,134 @@
 .. highlight:: cmake
 
-Using ``dds`` Packages in a CMake Project
-#########################################
+Easy Mode: Using ``dds`` in a CMake Project
+###########################################
 
 One of ``dds``'s primary goals is to inter-operate with other build systems
-cleanly. One of ``dds``'s primary outputs is *libman* package indices. These
-package indices can be imported into other build systems that support the
-*libman* format.
+cleanly. Because of CMakes ubiquity, ``dds`` includes built-in support for
+emitting files that can be imported into CMake.
 
-.. note::
-    ``dds`` doesn't (yet) have a ready-made central repository of packages that
-    can be downloaded. You'll need to populate the local package catalog
-    appropriately. The default catalog file contains a limited set of useful
-    packages, but you may wish to add more for yourself.
-
-    .. seealso:: Refer to :doc:`catalog` for information about remote packages.
-
-.. _PMM: https://github.com/vector-of-bool/PMM
-
-.. _CMakeCM: https://github.com/vector-of-bool/CMakeCM
-
-.. _lm-cmake: https://raw.githubusercontent.com/vector-of-bool/libman/develop/cmake/libman.cmake
-
-
-Generating a libman Index
-*************************
-
-Importing libman packages into a build system requires that we have a libman
-index generated on the filesystem. **This index is not generated globally**: It
-is generated on a per-build basis as part of the build setup. The index will
-describe in build-system-agnostic terms how to include a set of packages and
-libraries as part of a build.
-
-``dds`` has first-class support for generating this index. The ``build-deps``
-subcommand of ``dds`` will download and build a set of dependencies, and places
-an ``INDEX.lmi`` file that can be used to import the built results.
-
-
-Declaring Dependencies
-======================
-
-``dds build-deps`` accepts a list of dependency statements as commnad line
-arguments, but it may be useful to specify those requirements in a file.
-
-``dds build-deps`` accepts a JSON5 file describing the dependencies of a
-project as well. This file is similar to a very stripped-down version of a
-``dds`` :ref:`package manifest <pkgs.pkgs>`, and only includes the ``depends``
-key. (The presence of any other key is an error.)
-
-Here is a simple dependencies file that declares a single requirement:
-
-.. code-block:: js
-    :caption: ``dependencies.json5``
-
-    {
-        depends: [
-            'neo-sqlite3^0.2.0',
-        ]
-    }
-
-
-Building Dependencies and the Index
-===================================
-
-We can invoke ``dds build-deps`` and give it the path to this file:
-
-.. code-block:: bash
-
-    $ dds build-deps --deps dependencies.json5
+.. seealso::
 
-When finished, ``dds`` will write the build results into a subdirectory called
-``_deps`` and generate a file named ``INDEX.lmi``. This file is ready to be
-imported into any build system that can understand libman files (in our case,
-CMake).
+  Before reading this page, be sure to read the :ref:`build-deps.gen-libman`
+  section of the :doc:`build-deps` page, which will discuss how to use the
+  ``dds build-deps`` subcommand.
 
 .. note::
-    The output directory and index filepath can be controlled with the
-    ``--out`` and ``--lmi-path`` flags, respectively.
 
+  We'll first look as *easy mode*, but there's also an *easiest mode* for a
+  one-line solution: :ref:`see below <cmake.pmm>`.
 
-Importing into CMake
-********************
-
-We've generated a libman index and set of packages, and we want to import
-them into CMake. CMake doesn't know how to do this natively, but there exists a
-single-file module for CMake that allows CMake to import libraries from libman
-indices without any additional work.
-
-The module is not shipped with CMake, but is available online as a single
-stand-alone file. The `libman.cmake <lm-cmake_>`_ file can be downloaded and
-added to a project directly, or it can be obtained automatically through a
-CMake tool like `PMM`_ (recommended).
-
-
-Enabling *libman* Support in CMake via PMM
-==========================================
-
-Refer to the ``README.md`` file in `the PMM repo <PMM_>`_ for information on how
-to get PMM into your CMake project. In short, download and place the
-``pmm.cmake`` file in your repository, and ``include()`` the file near the top
-of your ``CMakeLists.txt``::
-
-    include(pmm.cmake)
-
-Once it has been included, you can call the ``pmm()`` function. To obtain
-*libman*, we need to start by enabling `CMakeCM`_::
-
-    pmm(CMakeCM ROLLING)
-
-.. warning::
-    It is not recommended to use the ``ROLLING`` mode, but it is the easiest to
-    use when getting started. For reproducible and reliable builds, you should
-    pin your CMakeCM version using the ``FROM <url>`` argument.
-
-Enabling CMakeCM will make available all of the CMake modules available in `the
-CMakeCM repository <CMakeCM_>`_, which includes `libman.cmake <lm-cmake_>`_.
+.. _PMM: https://github.com/vector-of-bool/PMM
 
-After the call to ``pmm()``, simply ``include()`` the ``libman`` module::
 
-    include(libman)
+Generating a CMake Import File
+******************************
 
-That's it! The only function from the module that we will care about for now
-is the ``import_packages()`` function.
+``build-deps`` accepts an ``--lmi-path`` argument, but also accepts a
+``--cmake=<path>`` argument that serves a similar purpose: It will write a CMake
+file to ``<path>`` that can be ``include()``'d into a CMake project:
 
+.. code-block:: bash
 
-Importing Our Dependencies' Packages
-====================================
+  $ dds build-deps "neo-sqlite3^0.2.0" --cmake=deps.cmake
 
-To import a package from a libman tree, we need only know the *name* of the
-package we wish to import. In our example case above, we depend on
-``neo-sqlite3``, so we simply call the libman-CMake function
-``import_packages()`` with that package name::
+This will write a file ``./deps.cmake`` that we can ``include()`` from a CMake
+project, which will then expose the ``neo-sqlite3`` package as a set of imported
+targets.
 
-    import_packages("neo-sqlite3")
 
-You'll note that we don't request any particular version of the package: All
-versioning resolution is handled by ``dds``. You'll also note that we don't
-need to specify our transitive dependencies: This is handled by the libman
-index that was generated by ``dds``: It will automatically ``import_packages()``
-any of the transitive dependencies required.
+Using the CMake Import File
+===========================
 
+Once we have generated the CMake import file using ``dds build-deps``, we can
+simply import it in our ``CMakeLists.txt``::
 
-Using Our Dependencies' Libraries
-=================================
+  include(deps.cmake)
 
 Like with ``dds``, CMake wants us to explicitly declare how our build targets
-*use* other libraries. When we import a package from a libman index, the
-import will generate CMake ``IMPORTED`` targets that can be linked against.
+*use* other libraries. When we ``include()`` the generated CMake file, it will
+generate ``IMPORTED`` targets that can be linked against.
 
-In ``dds`` and in libman, a library is identified by a combination of
+In ``dds`` (and in libman), a library is identified by a combination of
 *namespace* and *name*, joined together with a slash ``/`` character. This
 *qualified name* of a library is decided by the original package author, and
-should be documented. In the case of ``neo-sqlite3``, the only target is
+should be documented. In the case of ``neo-sqlite3``, the only library is
 ``neo/sqlite3``.
 
-When the libman CMake module imports a library, it creates a qualified name
+When the generated import file imports a library, it creates a qualified name
 using a double-colon "``::``" instead of a slash. As such, our ``neo/sqlite3``
 is imported in CMake as ``neo::sqlite3``. We can link against it as we would
 with any other target::
 
-    add_executable(my-application app.cpp)
-    target_link_libraries(my-application PRIVATE neo::sqlite3)
-
-Altogether, here is the final CMake file:
+  add_executable(my-application app.cpp)
+  target_link_libraries(my-application PRIVATE neo::sqlite3)
 
-.. code-block::
-    :caption: ``CMakeLists.txt``
-    :linenos:
 
-    cmake_minimum_required(VERSION 3.15)
-    project(MyApplication VERSION 1.0.0)
+.. _cmake.pmm:
 
-    include(pmm.cmake)
-    pmm(CMakeCM ROLLING)
+*Easiest* Mode: PMM Support
+***************************
 
-    include(libman)
-    import_packages("neo-sqlite3")
-
-    add_executable(my-application app.cpp)
-    target_link_libraries(my-application PRIVATE neo::sqlite3)
+`PMM`_ is the *package package manager*, and can be used to control and access
+package managers from within CMake scripts. This includes controlling ``dds``.
+With PMM, we can automate all of the previous steps into a single line.
 
+Refer to the ``README.md`` file in `the PMM repo <PMM>`_ for information on how
+to get PMM into your CMake project. In short, download and place the
+``pmm.cmake`` file in your repository, and ``include()`` the file near the top
+of your ``CMakeLists.txt``::
 
-Additional PMM Support
-**********************
+  include(pmm.cmake)
 
-The ``pmm()`` function also supports ``dds`` directly, similar to ``CMakeCM``
-mode. This will automatically download a prebuilt ``dds`` for the host platform
-and invoke ``dds build-deps`` in a single pass as part of CMake's configure
-process. This is especially useful for a CI environment where you want to have
-a stable ``dds`` version and always have your dependencies obtained
-just-in-time.
+The ``pmm()`` function also supports ``dds`` directly, and will automatically
+download a prebuilt ``dds`` for the host platform and invoke ``dds build-deps``
+in a single pass as part of CMake's configure process. This is especially useful
+for a CI environment where you want to have a stable ``dds`` version and always
+have your dependencies obtained just-in-time.
 
 To start, pass the ``DDS`` argument to ``pmm()`` to use it::
 
-    pmm(DDS)
+  pmm(DDS)
 
 .. note::
-    The ``_deps`` directory and ``INDEX.lmi`` file will be placed in the CMake
-    build directory, out of the way of the rest of the project.
+  The ``_deps`` directory and generated CMake imports file will be placed in
+  the CMake build directory, out of the way of the rest of the project.
 
 .. note::
-    The version of ``dds`` that PMM downloads depends on the version of PMM
-    that is in use.
+  The version of ``dds`` that PMM downloads depends on the version of PMM
+  that is in use.
 
 This alone won't do anything useful, because you'll need to tell it what
 dependencies we want to install::
 
-    pmm(DDS DEP_FILES dependencies.json5)
+  pmm(DDS DEP_FILES dependencies.json5)
 
-You can also list your dependencies as an inline string in your CMakeLists.txt
+You can also list your dependencies as inline strings in your CMakeLists.txt
 instead of a separate file::
 
-    pmm(DDS DEPENDS neo-sqlite3^0.2.2)
+  pmm(DDS DEPENDS neo-sqlite3^0.2.2)
 
-Since you'll probably want to be using ``libman.cmake`` at the same time, the
-calls for ``CMakeCM`` and ``DDS`` can simply be combined. This is how our new
-CMake project might look:
+This invocation will run ``build-deps`` with the build options, generate a CMake
+imports file, and immediately ``include()`` it to import the generated CMake
+targets. ``pmm(DDS)`` will also generate a ``dds`` :doc:`toolchain <toolchains>`
+based on the current CMake build environment, ensuring that the generated
+packages have matching build options to the rest of the project. Refer to the
+PMM README for more details.
 
 .. code-block::
-    :caption: ``CMakeLists.txt``
-    :linenos:
-
-    cmake_minimum_required(VERSION 3.15)
-    project(MyApplication VERSION 1.0.0)
+  :caption: ``CMakeLists.txt``
+  :linenos:
+  :emphasize-lines: 4,5
 
-    include(pmm.cmake)
-    pmm(CMakeCM ROLLING
-        DDS DEPENDS neo-sqlite3^0.2.2
-        )
+  cmake_minimum_required(VERSION 3.15)
+  project(MyApplication VERSION 1.0.0)
 
-    include(libman)
-    import_packages("neo-sqlite3")
+  include(pmm.cmake)
+  pmm(DDS DEPENDS neo-sqlite3^0.2.2)
 
-    add_executable(my-application app.cpp)
-    target_link_libraries(my-application PRIVATE neo::sqlite3)
+  add_executable(my-application app.cpp)
+  target_link_libraries(my-application PRIVATE neo::sqlite3)
 
 This removes the requirement that we write a separate dependencies file, and we
 no longer need to invoke ``dds build-deps`` externally, as it is all handled
-by ``pmm``.
+by ``pmm()``.

docs/guide/index.rst

@@ -9,7 +9,8 @@ User Guide
     packages
     toolchains
     source-dists
-    repo
-    catalog
+    pkg-cache
+    remote-pkgs
     interdeps
+    build-deps
     cmake

docs/guide/interdeps.rst

@@ -85,8 +85,8 @@ versions of the dependency are supported.
     Refer to: :ref:`deps.ranges.why-lowest`.
 
 ``dds`` compatible-version ranges are similar to the shorthand range specifiers
-supported by ``npm`` and ``npm``-like tools. There are five (and a half)
-version range formats available, listed in order of most-to-least restrictive:
+supported by ``npm`` and ``npm``-like tools. There are four version range kinds
+available, listed in order of most-to-least restrictive:
 
 Exact: ``@1.2.3``
     Specifies an *exact* requirement. The dependency must match the named

docs/guide/packages.rst

@@ -56,7 +56,7 @@ If a file's extension is not listed in the table above, ``dds`` will ignore it.
 .. note::
     Although headers are not compiled, this does not mean they are ignored.
     ``dds`` still understands and respects headers, and they are collected
-    together as part of *source distribution*.
+    together as part of a *source distribution*.
 
 
 .. _pkgs.apps-tests:
@@ -65,15 +65,42 @@ Applications and Tests
 **********************
 
 ``dds`` will recognize certain compilable source files as belonging to
-applications and tests. If a compilable source file stem ends with ``.main`` or
-``.test``, that source file is assumed to correspond to an executable to
-generate. The filename stem before the ``.main`` or ``.test`` will be used as
-the name of the generated executable. For example:
+applications and tests, depending on the filenames "stem," which is the part of
+the filename not including the outer-most file extension. If a compilable source
+filename stem ends with ``.main`` or ``.test``, that source file is assumed to
+correspond to an executable to generate. The filename second-inner stem before
+the ``.main`` or ``.test`` will be used as the name of the generated executable.
+For example:
 
-- ``foo.main.cpp`` will generate an executable named ``foo``.
-- ``bar.test.cpp`` will generate an executable named ``bar``.
-- ``cat-meow.main.cpp`` will generate an executable named ``cat-meow``.
-- ``cats.musical.test.cpp`` will generate an executable named ``cats.musical``.
+- Given ``foo.main.cpp``
+
+  - The stem is ``foo.main``, whose extension is ``.main``, so we will generate
+    an application.
+  - The stem of ``foo.main`` is ``foo``, so the executable will be named
+    ``foo``.
+
+- Given ``bar.test.cpp``
+
+  - The stem is ``bar.test``, whose extension is ``.test``, so we will generate
+    a test.
+  - The stem of ``bar.test`` is ``bar``, so will generate an executable named
+    ``bar``.
+
+- Given ``cat-meow.main.cpp``
+
+  - The stem is ``cat-meow.main``, which has extension ``.main``, so it is an
+    application.
+  - The stem of ``cat-meow.main`` is ``cat-meow``, so will generate an
+    executable named ``cat-meow``.
+
+- Given ``cats.musical.test.cpp``
+
+  - The stem is ``cats.musical.test``, which has extension ``.test``, so this is
+    a text executable.
+  - The stem of ``cats.musical.test`` is ``cats.musical``, so we will generate
+    an executable named ``cats.musical``.
+  - Note that the dot in ``cats.musical`` is not significant, as ``dds`` does
+    strip any further extensions.
 
 .. note::
     ``dds`` will automatically append the appropriate filename extension to the
@@ -161,7 +188,7 @@ In order for any code to compile and resolve these ``#include`` directives, the
 ``src/`` directory must be added to their *include search path*.
 
 Because the ``#include`` directives are based on the *portable* source root,
-the exactly location of ``src/`` is not important to the content of the
+the exact location of ``src/`` is not important to the content of the
 consuming source code, and can thus be relocated and renamed as necessary.
 Consumers only need to update the path of the *include search path* in a single
 location rather than modifying their source code.
@@ -285,8 +312,8 @@ The primary distribution format of packages that is used by ``dds`` is the
 
 Packages are identified by a name/version pair, joined together by an ``@``
 symbol. The version of a package must be a semantic version string. Together,
-the ``name@version`` string forms the *package ID*, and it must be unique
-within a repository or package catalog.
+the ``name@version`` string forms the *package ID*, and it must be unique within
+a repository or local package cache.
 
 In order for a package to be exported by ``dds`` it must have a
 ``package.json5`` file at its package root. Three keys are required to be

docs/guide/pkg-cache.rst

@@ -0,0 +1,90 @@
+The Local Package Cache
+#######################
+
+``dds`` maintains a local cache of packages that it has obtained at the request
+of a user. The packages themselves are stored as
+:doc:`source distributions <source-dists>` (``dds`` does not store the binaries
+that it builds within this package cache).
+
+
+Reading Repository Contents
+***************************
+
+Most times, ``dds`` will manage the cache content silently, but it may be useful
+to see what ``dds`` is currently storing away.
+
+The content of the cache can be seen with the ``pkg ls`` subcommand::
+
+> dds pkg ls
+
+This will print the names of packages that ``dds`` has downloaded, as well as
+the versions of each.
+
+
+Obtaining Packages
+******************
+
+.. seealso:: See also: :doc:`remote-pkgs`
+
+When ``dds`` builds a package, it will also build the dependency libraries of
+that package. In order for the dependency build to succeed, it must have a
+local copy of the source distribution of that dependency.
+
+When ``dds`` performs dependency resolution, it will consider both locally
+cached packages, as well as packages that are available from any
+:doc:`remote packages <remote-pkgs>`. If the dependency solution requires any
+packages that are not in the local cache, it will use the information in its
+catalog database to obtain a source distribution for each missing package. These
+source distributions will automatically be added to the local cache, and later
+dependency resolutions will not need to download that package again.
+
+This all happens automatically when a project is built: There is **no**
+"``dds install``" subcommand.
+
+
+Manually Downloading a Dependency
+=================================
+
+It may be useful to obtain a copy of the source distribution of a package
+from a remote. The ``pkg get`` command can be used to do this::
+
+> dds pkg get <name>@<version>
+
+This will obtain the source distribution of the package matching the given
+package ID and place that distribution in current working directory, using the
+package ID as the name of the source distribution directory::
+
+    $ dds pkg get spdlog@1.4.2
+    [ ... ]
+
+    $ ls .
+    .
+    ..
+    spdlog@1.4.2
+
+    $ ls ./spdlog@1.4.2/
+    include/
+    src/
+    library.json5
+    package.json5
+
+
+.. _repo.import-local:
+
+Exporting a Project into the Repository
+***************************************
+
+``dds`` can only use packages that are available in the local cache. For
+packages that have a listing in the catalog, this is not a problem. But if one
+is developing a local package and wants to allow it to be used in another local
+package, that can be done by importing that project into the package cache as a
+regular package, as detailed in :ref:`sdist.import`::
+
+> dds pkg import /path/to/project
+
+This command will create a source distribution and place it in the local cache.
+The package is now available to other projects on the local system.
+
+.. note::
+    This doesn't import in "editable" mode: A snapshot of the package root
+    will be taken and imported to the local cache.

docs/guide/remote-pkgs.rst

@@ -0,0 +1,231 @@
+Remote Packages and Repositories
+################################
+
+.. highlight:: bash
+
+``dds`` stores a local database of available packages, along with their
+dependency statements and information about how a source distribution thereof
+may be obtained.
+
+Inside the database are *package repositories*, which are remote servers that
+contain their own database of packages, and may also contain the packages
+themselves. An arbitrary number of package repositories may be added to the
+local database. When ``dds`` updates its package information, it will download
+the package database from each registered remote and import the listings into
+its own local database, making them available for download.
+
+
+Viewing Available Packages
+**************************
+
+The default catalog database is stored in a user-local location, and the
+available packages can be listed with ``dds pkg search``::
+
+  $ dds pkg search
+      Name: abseil
+  Versions: 2018.6.0, 2019.8.8, 2020.2.25
+      From: repo-1.dds.pizza
+
+      Name: asio
+  Versions: 1.12.0, 1.12.1, 1.12.2, 1.13.0, 1.14.0, 1.14.1, 1.16.0, 1.16.1
+      From: repo-1.dds.pizza
+
+      Name: boost.leaf
+  Versions: 0.1.0, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5, 0.3.0
+      From: repo-1.dds.pizza
+
+      Name: boost.mp11
+  Versions: 1.70.0, 1.71.0, 1.72.0, 1.73.0
+      From: repo-1.dds.pizza
+
+Optionally, one can search with a glob/fnmatch-style pattern::
+
+  $ dds pkg search 'neo-*'
+      Name: neo-buffer
+  Versions: 0.2.1, 0.3.0, 0.4.0, 0.4.1, 0.4.2
+      From: repo-1.dds.pizza
+
+      Name: neo-compress
+  Versions: 0.1.0, 0.1.1, 0.2.0
+      From: repo-1.dds.pizza
+
+      Name: neo-concepts
+  Versions: 0.2.2, 0.3.0, 0.3.1, 0.3.2, 0.4.0
+      From: repo-1.dds.pizza
+
+
+Remote Repositories
+*******************
+
+A remote package repository consists of an HTTP(S) server serving the following:
+
+1. An accessible directory containing source distributions of various packages,
+   and
+2. An accessible database file that contains a listing of packages and some
+   repository metadata.
+
+The exact details of the directory layout and database are not covered here, and
+are not necessary to make use of a repository.
+
+When ``dds`` uses a repository, it pulls down the database file and imports its
+contents into its own local database, associating the imported package listings
+with the remote repository which provides them. Pulling the entire database at
+once allows ``dds`` to perform much faster dependency resolution and reduces
+the round-trips associated with using a dynamic package repository.
+
+
+Adding a Repository
+===================
+
+Adding a remote repository to the local database is a simple single command::
+
+  $ dds pkg repo add "https://repo-1.dds.pizza"
+  [info ] Pulling repository contents for repo-1.dds.pizza [https://repo-1.dds.pizza/]
+
+This will tell ``dds`` to add ``https://repo-1.dds.pizza`` as a remote
+repository and immediately pull its package listings for later lookup. This
+initial update can be suppressed with the ``--no-update`` flag.
+
+.. note::
+
+  After the initial ``pkg repo add``, the repository is *not* identified by its
+  URL, but by its *name*, which is provided by the repository itself. The name
+  is printed the first time it is added, and can be seen using ``pkg repo ls``.
+
+
+Listing Repositories
+====================
+
+A list of package repositories can be seen with the ``pkg repo ls`` subcommand::
+
+  $ dds pkg repo ls
+
+
+Removing Repositories
+=====================
+
+A repository can be removed by the ``pkg repo remove`` subcommand::
+
+  $ dds pkg repo remove <name>
+
+Where ``<name>`` is given as the *name* (not URL!) of the repository.
+
+**Note** that removing a repository will make all of its corresponding remote
+packages unavailable, while packages that have been pulled into the local cache
+will remain available even after removing a repository.
+
+
+Updating Repository Data
+========================
+
+Repository data and package listings can be updated with the ``pkg repo update``
+subcommand::
+
+  $ dds pkg repo update
+
+This will pull down the databases of all registered remote repositories. If
+``dds`` can detect that a repository's database is unchanged since a prior
+update, that update will be skipped.
+
+
+The Default Repository
+**********************
+
+When ``dds`` first initializes its local package database, it will add a single
+remote repository: ``https://repo-1.dds.pizza/``, which has the name
+``repo-1.dds.pizza``. At the time of writing, this is the only official ``dds``
+repository, and is populated sparsely with hand-curated and prepared packages.
+In the future, the catalog of packages will grow and be partially automated.
+
+There is nothing intrinsically special about this repository other than it being
+the default when ``dds`` first creates its package database. It can be removed
+as any other, should one want tighter control over package availability.
+
+
+Managing a Repository
+*********************
+
+A ``dds`` repository is simply a directory of static files, so any HTTP server
+that can serve from a filesystem can be used as a repository. ``dds`` also
+ships with a subcommand, ``repoman``, that can be used to manage a repository
+directory.
+
+
+Initializing a Repository
+=========================
+
+Before anything can be done, a directory should be converted to a repository by
+using ``repoman init``::
+
+  $ dds repoman init ./my-repo-dir --name=my-experimental-repo
+
+This will add the basic metadata into ``./my-repo-dir`` such that ``dds`` will
+be able to pull package data from it.
+
+The ``--name`` argument should be used to give the repository a unique name. The
+name should be globally unique to avoid collisions: When ``dds`` pulls a
+repository that declares a given name, it will *replace* the package listings
+associated with any repository of that name. As such, generic names like
+``main`` or ``packages`` shouldn't be used in production.
+
+
+Listing Contents
+================
+
+The packages in a repository can be listed using ``dds repoman ls <repo-dir>``.
+This will simply print each package identifier that is present in the
+repository.
+
+
+Importing Source Distributions
+==============================
+
+If you have a source distribution archive, it can be imported with the
+appropriately name ``dds repoman import`` command::
+
+  $ dds repoman import ./my-repo some-pkg@1.2.3.tar.gz
+
+Multiple archive paths may be provided at once to import them all at once.
+
+
+Adding a Package by URL
+=======================
+
+A repository can also list packages that it does not host itself. Such a package
+listing can be added "by URL," where the URL tells ``dds`` how to pull the
+source distribution of the package. Beyond basic HTTP(S) URLs, ``dds`` can also
+clone packages via ``git``::
+
+  $ dds repoman add ./my-repo git+https://github.com/vector-of-bool/neo-fun.git#0.5.2
+
+The above URL tells ``dds`` that it can use ``git clone`` against
+``https://github.com/vector-of-bool/neo-fun.git`` and ask for tag ``0.5.2`` to
+get a source distribution directory that can be imported. Note the fragment on
+``git`` URLs! The fragment is required to specify the branch or tag to clone.
+
+If the package is available on GitHub, ``dds`` has a shorthand URL for that::
+
+  $ dds repoman add ./my-repo github:vector-of-bool/neo-fun/0.6.0
+
+The ``github:`` URL scheme tells ``dds`` to clone from GitHub. A ``github:`` URL
+must have exactly three path elements to determine *what* to download:
+``github:{owner}/{repository}/{branch-or-tag}``.
+
+.. note::
+
+  The ``github:`` URL lacks an *authority* element, and as such *does not* use
+  the double-slash.
+
+.. note::
+
+  ``repoman add`` will immediately attempt to pull a source distribution from
+  the given URL so that it may import the package's metadata into its listing
+  database. You cannot add a URL that is not already accessible.
+
+
+Removing Packages
+=================
+
+A package can be removed from a repository with
+``dds repoman remove <repo-dir> <pkg-id>``, where ``<pkg-id>`` is the
+``<name>@<version>`` of the package to remove.

docs/guide/repo.rst

@@ -1,92 +0,0 @@
-The Local Package Repository
-############################
-
-``dds`` maintains a local repository of packages that it has obtained at the
-request of a user. The packages themselves are stored as
-:doc:`source distributions <source-dists>` (``dds`` does not store the binaries
-that it builds within the repository).
-
-
-Reading Repository Contents
-***************************
-
-Most times, ``dds`` will manage the repository content silently, but it may be
-useful to see what ``dds`` is currently storing away.
-
-The content of the repostiory can be seen with the ``repo`` subcommand::
-
-> dds repo ls
-
-This will print the names of packages that ``dds`` has downloaded, as well as
-the versions of each.
-
-
-Obtaining Packages
-******************
-
-.. seealso:: See also: :doc:`catalog`
-
-When ``dds`` builds a package, it will also build the dependency libraries of
-that package. In order for the dependency build to succeed, it must have a
-local copy of the source distribution of that dependency.
-
-When ``dds`` performs dependency resolution, it will consider both existing
-packages in the local repository, as well as packages that are available from
-the :doc:`package catalog <catalog>`. If the dependency solution requires any
-packages that are not in the local repository, it will use the information in
-the catalog to obtain a source distribution for each missing package. These
-source distributions will automatically be added to the local repository, and
-later dependency resolutions will not need to download that package again.
-
-
-Manually Downloading a Dependency
-=================================
-
-It may be useful to obtain a copy of the source distribution of a package
-contained in the catalog. The ``catalog get`` command can be used to do this::
-
-> dds catalog get <name>@<version>
-
-This will obtain the source distribution of the package matching the named
-identifier and place that distribution in current working directory, using the
-package ID as the name of the source distribution directory::
-
-    $ dds catalog get spdlog@1.4.2
-    [ ... ]
-
-    $ ls .
-    .
-    ..
-    spdlog@1.4.2
-
-    $ ls ./spdlog@1.4.2/
-    include/
-    src/
-    library.json5
-    package.json5
-
-
-.. _repo.export-local:
-
-Exporting a Project into the Repository
-***************************************
-
-``dds`` can only use packages that are available in the local repository. For
-packages that have a listing in the catalog, this is not a problem. But if one
-is developing a local package and wants to allow it to be used in another local
-package, that can be done by exporting a source distribution from the package
-root::
-
-> dds sdist export
-
-This command will create a source distribution and place it in the local
-repository. The package is now available to other projects on the local system.
-
-.. note::
-    This doesn't export in "editable" mode: A snapshot of the package root
-    will be taken and exported to the local repository.
-
-If one tries to export a package root into a repository that already contains
-a package with a matching identifier, ``dds`` will issue an error. If the
-``--replace`` flag is specified with ``sdist export``, then ``dds`` will
-forcibly replace the package in the local repository with a new copy.

docs/guide/source-dists.rst

@@ -1,9 +1,9 @@
 Source Distributions
 ####################
 
-A *source distribution* is ``dds``'s primary format for consuming and
-distributing packages. A source distribution, in essence, is a
-:ref:`package root <pkgs.pkg-root>` archive that contains only the files
+A *source distribution* (often abbreviated as "sdist") is ``dds``'s primary
+format for consuming and distributing packages. A source distribution, in
+essence, is a :ref:`package root <pkgs.pkg-root>` that contains only the files
 necessary for ``dds`` to reproduce the full build of all libraries in the
 package. The source distribution retains the directory structure of every
 :ref:`source root <pkgs.source-root>` of the original package, and thus retains
@@ -18,7 +18,7 @@ Generating a Source Distribution
 Generating a source distribution from a project directory is done with the
 ``sdist`` subcommand::
 
-> dds sdist create
+> dds pkg create
 
 The above command can be executed within a package root, and the result will be
 a gzip'd tar archive that reproduces the package's filesystem structure, but
@@ -26,8 +26,8 @@ only maintaining the files that are necessary for ``dds`` to reproduce a build
 of that package.
 
 The ``--project=<dir>`` flag can be provided to override the directory that
-``dds`` will use as the package root. The default is the working directory of
-the project.
+``dds`` will use as the package root. The default is the current working
+directory.
 
 The ``--out=<path>`` flag can be provided to override the destination path of
 the archive. The path should not name an existing file or directory. By default,
@@ -37,10 +37,32 @@ then ``dds`` will overwrite the destination if it names an existing file or
 directory.
 
 
-Importing a Source Ditsribution
+.. _sdist.import:
+
+Importing a Source Distribution
 *******************************
 
 Given a source distribution archive, one can import the package into the local
-repository with a single command::
+package cache with a single command::
+
+> dds pkg import some-package@1.2.3.tar.gz
+
+You can also specify an HTTP or HTTPS URL to download a source distribution
+archive to import without downloading it separately::
+
+> dds pkg import https://example.org/path/to/sdist.tar.gz
+
+Alternatively, if a directory correctly models a source distribution, then
+that directory can be imported in the same manner::
+
+> dds pkg import /path/to/some/project
+
+Importing a package will allow projects on the system to use the imported
+package as a dependency.
+
+.. note::
 
-> dds repo import some-package@1.2.3.tar.gz
+    If one tries to import a package root into the cache that already contains a
+    package with a matching identifier, ``dds`` will issue an error. This
+    behavior can be overridden by providing ``--if-exists=replace`` on the
+    command-line.

docs/guide/toolchains.rst

@@ -299,6 +299,16 @@ Specify *additional* compiler options, possibly per-language.
 
 Specify *additional* link options to use when linking executables.
 
+.. note::
+
+    ``dds`` does not invoke the linker directly, but instead invokes the
+    compiler with the appropriate flags to perform linking. If you need to pass
+    flags directly to the linker, you will need to use the compiler's options to
+    direct flags through to the linker. On GNU-style, this is
+    ``-Wl,<linker-option>``. With MSVC, a separate flag ``/LINK`` must be
+    specified, and all following options are passed to the underlying
+    ``link.exe``.
+
 
 ``optimize``
 ------------

docs/tut/hello-test.rst

@@ -98,7 +98,7 @@ leave the condition the same, though:
 Now running ``dds build`` will print more output that Catch has generated as
 part of test execution, and we can see the reason for the failing test::
 
-    [16:41:45] [error] Test <root>/_build/test/hello/strings failed! Output:
+    [error] Test <root>/_build/test/hello/strings failed! Output:
 
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     strings is a Catch v2.10.2 host application.

docs/tut/hello-world.rst

@@ -168,9 +168,9 @@ If you run the ``dds build`` command again, you will now see an error:
 
 .. code-block:: text
 
-    [12:55:25] [info ] [dds-hello] Link: hello-world
-    [12:55:25] [info ] [dds-hello] Link: hello-world                    -     57ms
-    [12:55:25] [error] Failed to link executable '<root>/_build/hello-world'.
+    [info ] [dds-hello] Link: hello-world
+    [info ] [dds-hello] Link: hello-world                    -     57ms
+    [error] Failed to link executable '<root>/_build/hello-world'.
     ...
     <additional lines follow>
 
@@ -238,6 +238,10 @@ package root:
 
 Rebuilding the project will show no difference at the moment.
 
+.. note::
+    You may also use a ``.jsonc`` or ``.json`` file extension. ``dds`` will
+    search for all of these files, but they will all be parsed as JSON5.
+
 .. seealso::
     Creating a single application executable is fine and all, but what if we
     want to create libraries? See the next page: :doc:`hello-lib`