CoCo in pipelines
1. CoCo in pipelines
Welcome back to the course, where CoCo builds work that runs after you close the terminal.2. From answers to pipelines
So far, each answer from CoCo was a snapshot of the data.3. From answers to pipelines
A script repeats the work, but someone still has to run it.4. From answers to pipelines
A pipeline runs on its own, inside Snowflake, with nobody pressing run.5. Dynamic tables
One kind of pipeline is a dynamic table. It starts from a source table, here our store's reviews.6. Dynamic tables
On top sits a query, and ours uses AI classify to give each review a topic.7. Dynamic tables
Snowflake saves the result as a new table and keeps it up to date.8. Dynamic tables
A target lag sets how far behind the source the result is allowed to fall.9. Dynamic tables
When a review arrives, only that one goes through AI classify, so the function runs once, not 178 times.10. Ask for the outcome
Let's have CoCo build one from a description.11. Ask for the outcome
Each review gets one of five topics,12. Ask for the outcome
and the table should stay up to date within two minutes of a new review, without anyone rerunning anything.13. Review before approving
CoCo asks before creating anything, so read its SQL first. It chose a dynamic table,14. Review before approving
with a target lag of two minutes, as we asked,15. Review before approving
AI classify assigns our five topics,16. Review before approving
and it reads the reviews table, so we approve.17. A new review arrives
Then a review arrives: the USB-C hub came late, in a crushed box.18. A new review arrives
A minute later, with nobody rerunning anything, we ask CoCo whether it's labeled.19. A new review arrives
It's there already, with the topic delivery.20. Check the refresh history
To check it runs, ask CoCo for the refresh history.21. Check the refresh history
The first run, at creation, labeled all 177 reviews.22. Check the refresh history
The next found no new data, so it did nothing.23. Check the refresh history
The one after that added a single row, our new review.24. A scheduled task
Now let's set up a scheduled task.25. A scheduled task
The product team wants a report every Monday at nine, New York time,26. A scheduled task
summing up the week's reviews in two sentences, saved to a table.27. Inside the task
CoCo answers with a task, which is SQL that Snowflake runs on a schedule.28. Inside the task
The schedule is written in cron, which lists the minute, hour, day, month, and weekday. This one means nine o'clock every Monday, New York time, and CoCo writes it for you, so there's nothing to memorize.29. Inside the task
On each run, AI agg sums up the week's reviews in one new row.30. Turn the task on
A new task starts switched off, so CoCo asks before turning it on.31. Test it once
Rather than wait for Monday, we ask CoCo to run it once.32. Test it once
It succeeded in about three seconds,33. Test it once
and the summary flags the hub overheating and the speakerphone sounding distant.34. Pause when needed
If you don't need a pipeline for a while, CoCo can pause it.35. Pause when needed
It suspends both, and neither runs again until you resume them.36. CoCo inside the pipeline
You don't need to remember any of this SQL, because CoCo writes it. What matters is the bigger picture: so far, CoCo built each pipeline, and Snowflake ran it.37. CoCo inside the pipeline
CoCo can also be the job itself, in two common ways.38. Automations
The first is an automation, created in the terminal with cortex automation create.39. Automations
You give it a name,40. Automations
the prompt CoCo should run, here a check for low-rated products,41. Automations
and a weekday schedule. From then on, Snowflake runs it in a sandbox, even with your laptop closed.42. A report in your inbox
After each run, you may also get an email.43. A report in your inbox
Its subject line sums up the result,44. A report in your inbox
and the body lists the hub and the speakerphone, with what customers complained about.45. CoCo in GitHub Actions
The second way is GitHub Actions, which can run jobs whenever a team's code changes.46. Snowflake example workflow
Snowflake publishes an example workflow, which is a YAML file that GitHub runs.47. Snowflake example workflow
Its key step runs cortex exec with a prompt file, just as you did in the shell.48. The full picture
To sum up, dynamic tables keep results current as new data arrives,49. The full picture
tasks run SQL at fixed times,50. The full picture
and automations and GitHub Actions run CoCo itself, on a schedule or when your code changes.51. Let's practice!
Now it's your turn to build a pipeline with CoCo.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.