Conclusion

1. Conclusion

Let's reflect for a moment. It's incredible to look back on how far you've come in such a short amount of time. I promised that by the end of this course, you'd know enough about data engineering with Snowflake to be dangerously good at applying these concepts to build data pipelines on your own. To meet this objective, we designed this course to cover the most essential and practical Snowflake features within the ITD framework. We covered all sorts of ways to ingest data into Snowflake, from loading data from the Snowflake Marketplace, to using the web interface and command line, to knowing how to ingest data in cloud object storage into Snowflake with just a bit of SQL. These are core techniques and concepts you'll frequently encounter and use. For transforming data, we covered quite a bit, specifically how to use SQL or Snowpark for Python to perform data transformations, how to reuse logic by creating UDFs and stored procedures, and how to use streams and dynamic tables for incremental processing of data. You even learned how to use Snowflake Notebooks and the VS Code extension for Snowflake to do these things. For delivery, you learned all sorts of techniques for delivering valuable insights to consumers, from data sharing on the Snowflake Marketplace, to building and sharing apps in your account using Streamlit in Snowflake, to sharing apps beyond your account with Snowflake native applications. And last, but certainly not least, you learned how to add automation to pipelines using tasks, and you also learned how to chain tasks together to create DAGs that help with the broader orchestration of things in your pipeline. Having covered these concepts, I feel pretty good about the promise of you being dangerously good enough to start building pipelines on your own. Now here's what I didn't promise. I didn't promise that you'd be an expert on data engineering by the end of the course. And here's why I didn't promise that. First, I'm not so sure that any online course could really make a promise like that. And second, it's because so much of your expertise with building data pipelines is going to come from you going out and tinkering with things. And when you do that, you'll end up doing a few things. You'll make mistakes. You'll run into errors. You'll spend time debugging issues. You'll peruse technical documentation. And you'll go to all sorts of places online to learn more. And these are the sorts of things that will round out and enhance your expertise. Those are the things that will also exercise your mental model of data engineering and continuously strengthen it. So my advice to you, go out and put these concepts to use as soon as you can. Let go of any reservations you might have and get hands on. Try things, break things, fix things. Constantly push the limits of what you know and of what's possible. This will only help to level up your understanding and expertise. I know that you can do this. And I know that you can build data pipelines with Snowflake. I'm Gilberto Hernandez, developer advocate at Snowflake. Thanks for joining me and see you in future courses.

2. Let's practice!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.