Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • I ImageMatching
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Custom issue tracker
    • Custom issue tracker
  • Merge requests 2
    • Merge requests 2
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Activity
  • Graph
  • Jobs
  • Commits
Collapse sidebar
  • repos
  • generated-data-platform
  • ImageMatching
  • Merge requests
  • !29

Initial draft of refactoring efforts

  • Review changes

  • Download
  • Email patches
  • Plain diff
Open Gmodena requested to merge github/fork/clarakosi/refactoring into main Sep 01, 2021
  • Overview 0
  • Commits 1
  • Pipelines 0
  • Changes 1

Created by: clarakosi

Changes:

  • Adds spark udf
  • Modifies schema for top_candidates column to now view null image suggestions as an empty array
  • Saves output as parquet in hdfs

Does not enable cluster mode in spark because it does not appear to be possible with jupyter notebooks

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: github/fork/clarakosi/refactoring