LoslegenKostenlos starten

Pairing inputs with zip

Your team has a pipeline that uploads files to specific cloud storage locations. A colleague uses zip() with expand_kwargs() to pair each file with its cloud storage destination:

@task
def get_files():
    return ["users.csv", "orders.csv", "products.csv"]

@task
def get_destinations():
    return ["s3://bronze/users/", "s3://bronze/orders/", "s3://bronze/products/"]

@task
def upload(file_to_upload):
    # Simulate uploading a file to cloud storage
    print(f"Uploading {file_to_upload[0]} to {file_to_upload[1]}")

files = get_files()
destinations = get_destinations()
upload.expand(file_to_upload=files.zip(destinations))

How many instances of upload will Airflow create?

Diese Übung ist Teil des Kurses

<Kurs>Building Data Pipelines with Airflow</Kurs>
Kurs ansehen

Interaktive praktische Übung

Verwandle Theorie mit einer unserer interaktiven Übungen in die Praxis

Übung starten