Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • I ImageMatching
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Custom issue tracker
    • Custom issue tracker
  • Merge requests 2
    • Merge requests 2
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Artifacts
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Terraform modules
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Activity
  • Graph
  • Jobs
  • Commits
Collapse sidebar
  • repos
  • generated-data-platform
  • ImageMatching
  • Merge requests
  • !29

Initial draft of refactoring efforts

  • Review changes

  • Download
  • Patches
  • Plain diff
Open Gmodena requested to merge github/fork/clarakosi/refactoring into main Sep 01, 2021
  • Overview 0
  • Commits 1
  • Pipelines 0
  • Changes 1

Created by: clarakosi

Changes:

  • Adds spark udf
  • Modifies schema for top_candidates column to now view null image suggestions as an empty array
  • Saves output as parquet in hdfs

Does not enable cluster mode in spark because it does not appear to be possible with jupyter notebooks

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: github/fork/clarakosi/refactoring