Google BigQuery basics

Google BigQuery is a cloud-based big data analytics service. You can start querying and analyzing a huge amount of data within minutes. BigQuery is like a warehouse where you can store a huge amount of read-only data and analyze them.

BigQuery is a fully managed service in Google Cloud Platform.

BigQuery is a fully managed service in Google Cloud Platform.

Why use BigQuery?

• Ease of use. Simple UI and settings
• Querying is very fast
• You can scale your process in no-time
• Use of SQL type queries
• It is very cost-effective
• Save your queries and results are cached for 24 hours
• Easy Error handling

Structure of BigQuery

• Projects – Your Google Cloud Project
• Datasets – Datasets contain your tables. So, create a dataset first
• Tables – Your data as you know
• Views – You can partition data as you like for easy usage
• Jobs – SQL queries that are processed

Loading Data

You can load data to BigQuery from your local machine or you can upload data to Google Cloud Storage and load from there. BigQuery supports CSV, JSON file formats, it has schema auto-detection so you don’t have to provide all the details for your columns. I prefer to use a CSV file format as I can validate the files with MS Acess to address the errors

Querying Data

You can use Web UI to run your queries or you can make programmatic queries from your Application. You can save your queries and views for improving the speed and efficiency of your work.

Interactive Query – This type of Query is run with high priority
Batch Query – This type of Query is run when resource is available

Pricing

BigQuery is a serverless data warehouse so you can store data at a very very low cost and you are charged for the bytes processed by your queries. Google Cloud provides 1TB FREE data processing per month.

Error Handling

You can get all the required details for every job that is processed. You can use these details to further address the errors caused while loading or querying data.