Commit f9e6d11c authored by Gmodena's avatar Gmodena
Browse files

Add linting build step

parent 2aa412e2
...@@ -20,6 +20,9 @@ jobs: ...@@ -20,6 +20,9 @@ jobs:
- name: Install dependencies - name: Install dependencies
run: | run: |
make venv make venv
- name: Lint python files with flake8
run: |
make flake8
- uses: olafurpg/setup-scala@v10 - uses: olafurpg/setup-scala@v10
with: with:
java-version: adopt@1.8 java-version: adopt@1.8
......
from pyspark.sql import SparkSession from pyspark.sql import SparkSession
from pyspark.sql.types import ArrayType, StructType, StringType, DoubleType, IntegerType from pyspark.sql.types import StructType, StringType, IntegerType
from pyspark.sql import Column, DataFrame from pyspark.sql import Column, DataFrame
from pyspark.sql import functions as F from pyspark.sql import functions as F
......
pytest==6.2.2 pytest==6.2.2
pytest-spark==0.6.0 pytest-spark==0.6.0
pytest-cov==2.10.1 pytest-cov==2.10.1
\ No newline at end of file flake8==3.8.4
from etl.transform import RawDataset, ImageRecommendation from etl.transform import ImageRecommendation
def test_etl(raw_data): def test_etl(raw_data):
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment