GitLab

gitlab-ci.yaml: add tests

See merge request !2

To support scenarios where the `runs.as` or `lives.as` user/group
already exists in the base image, pair the calls to `useradd` and
`groupadd` with `getent` conditionals.

Bug: T268819
Change-Id: I3aef722ac57b38695c7411e64b41589fad16e95b

The `apt.proxies` field is now merged by appending second set of
proxies to the original.

Example:

    apt:
      proxies:
        - url: http://proxy.example:8080
    variants:
      foo:
        apt:
          proxies:
            - url: http://proxy.example:8081
              source: http://security.debian.org

In the above example, the effective `apt.proxies` configuration for the
`foo` variant would include both the base `proxy.example:8080` proxy and
the `proxy.example:8081` proxy specific to the `security.debian.org`
source.

Bug: T277109
Change-Id: I64bad1eb1e6fb6712a5de178be3aa273936a2df5

Defines a new `apt.proxies` field that allows users to specify
http/https proxy URLs that apply to one or all APT sources. APT
configuration for the proxies will be written prior to package
installation and then removed.

Both a shorthand and longhand configuration are supported.

Example:

    apt:
      packages: [foo]
      proxies: [http://proxy.example:8080]

Is a shorthand equivalent of:

    apt:
      packages: [foo]
      proxies:
        - url: http://proxy.example:8080

In which case, the following APT config is written before installing
packages, proxying all http sources via `http://proxy.example:8080`.

    Acquire::http::Proxy "http://proxy.example:8080";

Example:

    apt:
      packages: [foo]
      proxies:
        - url: https://proxy.example:8081
          source: https://security.debian.org

In which case, the following APT config is written before installing
packages, proxying only https requests to security.debian.org via
`https://pr...

To support image compositions that rely on copying files from one or
more external images, allow `ArtifactsConfig.From` to be any valid image
reference.

Generalize `baseimage` validator as `imageref` and create a new
`artifactfrom` validator alias for `imageref|variantref`. Use the new
validator for `From`.

Additionally, omits non-variant `From` references from the dependency
graph.

Change-Id: I3b92d815b62856fe5a8df836b37cce0043a74ffa

A `python` config block already triggered installing pip via
easy_install which is pretty specific to python2 era usage. It then used
pip to install/update pypi packages for setuptools, wheel, and tox. This
change adds the pip package to that set of pypi install/update packages.
This is useful to ensure that the resulting image is capable of
installing manylinux2014 formatted wheels (pip >= 19.3). Debian Buster's
python3-pip package is pip 18.1 which is too old to look for
manylinux2014 wheels when installing packages such as cryptography which
have stopped publishing wheels with the older manylinux1 and
manylinux2010 platform tags.

Existing pinning of pip to a version less than 21.0 is preserved for
those poor, unloved python2 projects that linger in an ever colder and
darker world hostile to their kind.

Bug: T274435
Change-Id: I81f0af69dbd4f9202bc9736faf653813c81fc0ad

Pip 21 has dropped support for Python 2 but is still strangely
selected for installation when running setuptools with a python2 binary.
Let's pin pip when a `version` is specified that starts with "python2".

Note this will require python 2 users to explicitly specify `version:
python2` even if their default `python` binary in the base image is for
python 2.

Bug: T273793
Change-Id: Id7d4315e345657ce7ac2efdd0b065d693e47b2f5

Kubernetes non-root verification only supports numeric UIDs. With
usernames (string) used, it can't verify that the user is not root and
therefore will reject running the image if RunAsNonRoot is enforced.

Bug: T228967
Change-Id: Iab59a28291c3712a135aa6aa21e4b338efe3ec43

In order to allow packages from backports to be downloaded, the ability
to target a release when running apt-get install is required.

apt.packages now takes the previously defined list of string or a map
of targets and packages to install. When using a map, in order to
install the packages for the base image, the 'default' key should be
used:

packages:
  default: ["libfoo", "libbar"]
  buster-backports: ["libbaaz"]

Bug: T272759
Change-Id: I435c47794ffbc0264e0a440bfbd23e11570645d1

There is not reason to stick to stretch. We already have a buster image
with golang 1.13-3, let's switch to it.

Bug: T271901
Change-Id: I231940c92354ddbaa3e486f5f2a93cadce3c489a

Copying of required files is now a generic operation implemented by
`config.RequirementsConfig`. Other builder types should re-use this
implementation.

To support this change, some `RequirementsConfig` implementation around
source and destination paths was moved to functions of
`ArtifactsConfig`.

Build macro functions `SortFilesByDir` and `SyncFiles` are no longer
needed and were removed.

Change-Id: Ieb0cd2a0b1e4f11b05d19c63bea9bdf9d578e3ea
Follows-up: If813829bdace6851bdba56abcdfcab1cd967df03

composer allows packages to run code after downloading them. In nearly every
case this is never needed and just opens up room for arbitrary code
execution.

Bug: T270207
Change-Id: Ic7db66f6e00d82ae70078eab333736f107ca5a48

Flip the order of `path.Clean()` and `path.Dir()` in the NewFromShort()
constructor. The prior order was stripping directory indicators from the
source path prematurely.

Bug: T263597
Change-Id: I0d76b4632660a8c32fb1c8a7a90b287546271348

Destination is now an optional field, defaulting to the source path
when omitted.

This makes for less redundancy when defining cross-variant builder
requirements and copies in cases where the application directories
are the same between variants.

Change-Id: I44cc3fdff260670b07b6cb3d82d4d7a49ff221f7

Allow `builder.requirements` to use a long form similar to `copies` when
the user needs to copy files from one variant to another as part of the
builder stage. This allows multi-stage builds to generate some artifact
in stage "A" and then copy that artifact into stage "B" in the
PreInstall phase before running `builder.command`.

Usage example:
```
variants:
  A:
    apt:
      packages:
        - wget
    builder:
      command:
        - /bin/bash
        - -c
        - >-
          wget https://example.org/some_cool_utility.tgz


          && tar xzvf some_cool_utility.tgz
          && rm some_cool_utility.tgz
  B:
    builder:
      requirements:
        - src
        - from: A
          source: some_cool_utility
          destination: .
      command:
        - some_cool_utility --do-something-cool src/
```

Bug: T263597
Co-authored-by: Dan Duvall <dduvall@wikimedia.org>
Change-Id: If813829bdace6851bdba56abcdfcab1cd967df03