Volker Janz
Mar 11, 2024

--

Quick follow-up on the article:

There's one more thing regarding data modeling for cost-optimized BigQuery usage I didn't mention in this article. BigQuery offers smart tuning via materialized views, which means queries are automatically rewritten by the engine using materialized views when certain conditions are met.

That way, common aggregations, JOINs, and other operations can be pre-computed in materialized views to significantly reduce costs. The best practice here is to apply this kind of optimization after the release of the project to stakeholders to first gather data on how they utilize your model. Then, try to identify common cost-intensive patterns in the queries and create materialized views for these parts.

I might cover this aspect in a future article.

More information: https://cloud.google.com/bigquery/docs/materialized-views-intro

Enjoy reading! ✌️

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

--

--

Volker Janz
Volker Janz

Written by Volker Janz

Expert Data Engineer @ InnoGames | 👨‍💻 Data Engineering, SQL, Python, Java, Kafka, Flink, Hadoop, Cloud | 🎙️ Passionate Speaker | 🧩 I Bring Big Data to life

No responses yet

Write a response