All Data and AI Weekly #189 - May 12, 2025

All Data and AI Weekly

( AI, Data, NiFi, Iceberg, Polaris, Streamlit, Flink, Kafka, Python, Java, SQL, Unstructured Data )

#189 - May 12, 2025

https://bsky.app/profile/paasdev.bsky.social

NiFi + AI + AI Data Cloud + Iceberg.

https://www.reddit.com/r/DataEngineeringForAI/hot/

Boston May 14 2025 https://www.dbta.com/DataSummit/2025/Timothy-Spann.aspx

https://github.com/sfc-gh-tspann/DataAIDemos/blob/main/airquality.sql

https://www.slideshare.net/slideshow/14may2025_tspann_fromairqualityunstructureddata-pdf/277680861

https://medium.com/@tim.spann_50517/populating-an-open-lakehouse-with-codeless-data-streams-9395a30a2d4f

https://medium.com/@tim.spann_50517/real-time-enrichment-of-air-quality-data-26564464b2a5

https://www.youtube.com/watch?v=YJhRcXFNv2M

Monthly NYC and Youtube Events

https://lu.ma/PINSAI

Snowflake Tips

Tim, I need to backup some data in Snowflake. Just make sure you have retention time up, usually 30-60 days makes sense. You could want 90 days. Make a clone at your point and time so you can instantly compare any changes to what it was at that point you are concerned for. You can also export your data to cloud storage if you wish. You can also replicate it to other accounts. Lots of options here, no worry about data loss. Just travel back in time.

Zero Copy Clone

Storage Considerations

• https://docs.snowflake.com/en/user-guide/tables-storage-considerations#label-cloning-tables

Create clones of databases at/before a table

• https://docs.snowflake.com/en/sql-reference/sql/create-clone

• https://www.youtube.com/watch?v=uGCpwoQOQzQ

Time Travel to Clone Databases / Schemas / Tables at a Point in Time

• https://docs.snowflake.com/en/user-guide/data-time-travel

Set your retention time in days (up to 90 days)

• https://docs.snowflake.com/en/sql-reference/parameters#label-data-retention-time-in-days

Replicate Across Accounts / Regions / Clouds

• https://docs.snowflake.com/en/user-guide/account-replication-intro

Export Data to Cloud Storage

If you wish to export the data to an S3 stage, you can do that as well.

• https://community.snowflake.com/s/article/How-to-backup-the-Snowflake-database

• https://community.snowflake.com/s/article/backups

Cool Stuff of the week

⚡️ https://www.youtube.com/watch?v=v3Anx71WNm0&t=568s&pp=ygULIlRpbSBTcGFubiI%3D

❄️ https://medium.com/snowflake/ai-infused-pipelines-with-snowflake-cortex-6a7954f2078d

⚡️ https://medium.com/@orellabac/querying-data-from-neo4j-to-snowflake-1c1ee537aeb6

❄️ https://www.snowflake.com/en/blog/auto-manufacturers-drive-innovation-snowflake/

❄️ https://medium.com/@orellabac/ingest-external-data-into-snowflake-with-snowpark-and-jdbc-leveraging-parquet-part-2-5cb2d29f9749

❄️ https://medium.com/@tim.spann_50517/building-rag-applications-with-cortex-ai-bf0a3d2202db

❄️ https://quickstarts.snowflake.com/guide/optimizing-network-operations-with-cortex-ai-call-transcripts-and-tower-data-analysis/index.html?index=..%2F..index#3

⚡️ https://github.com/yuanze-lin/Olympus

⚡️ https://github.com/slidevjs/slidev

❄️ https://pytorch.org/blog/press-release-pytorch-foundation-expands-welcomes-projects-vllm-deepspeed/

❄️ https://docs.snowddl.com/getting-started

❄️ https://github.com/sfc-gh-praj/app-app-communication

❄️ https://medium.com/snowflake/building-microservices-architecture-patterns-in-snowflake-using-native-app-5c68fb6446f1

❄️ https://quickstarts.snowflake.com/guide/getting_started_with_ai_observability/#0

❄️ https://medium.com/@peter.horrigan/so-you-have-your-pat-in-vault-now-what-5757632f8d51

❄️ https://medium.com/snowflake/viewing-data-as-the-visiting-user-in-streamlit-in-snowflake-07eee0d6a5a3

❄️ https://medium.com/snowflake/snowflake-how-to-document-database-objects-by-an-empowered-snowflake-cortex-ai-process-2bee594bf66c

❄️ https://www.snowflake.com/en/blog/new-regions-egress-cost-optimizer/

❄️ https://docs.snowflake.com/en/user-guide/warehouses-gen2

⚡️ https://github.com/emcie-co/parlant

New Models

❄️ https://www.snowflake.com/en/blog/meta-llama-4-now-available-snowflake-cortex-ai/

⚡️ https://huggingface.co/docs/transformers/main/en/model_doc/d_fine

❄️ https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2

⚡️ https://huggingface.co/nvidia/OpenCodeReasoning-Nemotron-7B

Marketplace

⚡️ https://app.snowflake.com/marketplace/providers/GZSTZJL5LMG/Coresignal

Upcoming

May 15 - Overview of Snowflake https://www.snowflake.com/webinars/product-demo/data-cloud-demo-2025-05-15/

May 21 - Zero to Snowflake Hands on Lab https://www.snowflake.com/webinars/virtual-hands-on-labs/zero-to-snowflake-2025-05-21/

May 28 - Transforming Text https://www.snowflake.com/webinar/virtual-hands-on-labs/transforming-text-with-snowflake-cortex-building-intelligent-applications-apac-20250528/

June 19 - Northstar Intro to Snowflake Data Engineering https://www.snowflake.com/webinars/northstar-virtual-2025-06-19/

June 21 - Hybrid Tables for Real-Time https://www.snowflake.com/webinars/product-demo/harnessing-real-time-data-with-snowflake-hybrid-tables-101-2025-05-21

June 25 - Build data engineering pipelines https://www.snowflake.com/webinars/virtual-hands-on-labs/build-data-engineering-pipelines-using-snowpark-in-snowflake-notebooks-2025-06-25/

June 26 - Build a GenAI App https://www.snowflake.com/webinars/virtual-hands-on-labs/build-a-gen-ai-app-in-10-min-with-snowflake-2025-06-26/

In-Person

June 2 -5 Snowflake Summit - SF https://www.snowflake.com/en/summit/?utm_cta=website-events-featured

Very soon:

📊 May 14, 2025 - Boston - https://www.dbta.com/DataSummit/2025/default.aspx

📊 May 22, 2025 - New York City - https://events.sigmacomputing.com/mergespringnyc Sigma Computing - 9th floor

https://sessionize.com/tspann

https://github.com/timothyspann

Recent Tim Stuff

💻 Video IoT

https://www.youtube.com/watch?v=4Ojue8TWv6A

Apps, Demos, Examples, Models, Notebooks and Projects

© 2020-2025 Tim Spann https://www.youtube.com/@FLaNK-Stack

(AI + Vectors + LLM + Streaming + IoT)