Skip to content
  • STran's avatar
    Guard against already-run imports · 5e245b7e
    STran authored
    The import script as-is isn't guaranteed to run 24+ hours in the
    future and therefore unique imports per run are also not guaranteed.
    Additionally, the script will be expected to run every few hours
    to check for batch errors, exacerbating this problem.
    
    `main.sh` should therefore be able to check if the import it wants
    to run has already been run and abort if it has.
    
    - Add support for running the script multiple times a day by aborting
      the import if it's already been completed for the given feeds
    - Add support for retrying an import with failed batches
    - Add support for retrying an import that didn't complete (eg. database
      disconnect)
    - make --batchsize non-optional interally, as non-integer values from
      other parameters were being passed if a batchsize wasn't passed. This
      value still defaults to 10000.
    
    Bug: T344941
    Change-Id: If648356e2b62b2c310d9adcef7db361040c29dac
    5e245b7e