Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • W WMF Data Workflow Utils
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Custom issue tracker
    • Custom issue tracker
  • Merge requests 1
    • Merge requests 1
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Artifacts
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Terraform modules
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Activity
  • Graph
  • Jobs
  • Commits
Collapse sidebar
  • repos
  • data-engineeringdata-engineering
  • WMF Data Workflow Utils
  • Merge requests
  • !14

Resolve "[FIXME] large conda envs cannot be shared in Gitlab"

  • Review changes

  • Download
  • Patches
  • Plain diff
Merged Gmodena requested to merge 2-fixme-large-conda-envs-cannot-be-shared-in-gitlab into main Apr 07, 2022
  • Overview 4
  • Commits 7
  • Pipelines 5
  • Changes 1

Closes #2

Tagging as draft to start a discussion, and iron out the wrinkles.

This MR combines the build and publish job together. When archives exceed a certain size they cannot be cached as job artifacts, therefore making publication impossible.

Based on what we discussed with @otto, I'd like to propose a workflow that allows:

  • building and publishing automatically on tagged commits.
  • allowing manual build and publication on regular commits.

To support local development and testing, a dev should run conda-dist manually. See for example: https://gitlab.wikimedia.org/repos/research/knowledge-gaps/-/blob/5-add-ci-integration-package-conda-env-wip/Makefile#L50

Edited Apr 11, 2022 by Gmodena
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: 2-fixme-large-conda-envs-cannot-be-shared-in-gitlab