Sponsored material
How much time do you spend fighting your tools instead of solving problems? Every data scientist is there: Looking down the datastas down because it does not fit in memory or does not have a business user to interact with the machine learning model.
The ideal environment is removed from the path so you can focus on analysis. This article covers eight practical methods in the Big Cory that are designed for the same way, from using AI -powered agents to the service of the ML model to direct ML model.
1. Learning the machine in your spreadsheet
BQML training and forecasts from Google Sheet
Many data conversations begin and end in a spreadsheet. They are familiar, easy to use, and mutual support is the best. But what happens when your data is too big, or when you want to predict without writing a group of code? Attached Sheets The Google Sheets interface helps you analyze the large queue data of billions of rows. All calculations, charts and axes are strengthened behind the curtains.
Moving it a step further, you can also access the models with whom you have made Big Cory Machine Learning (BQML). Imagine that you have a BQML model that predicts housing prices. With attached sheets, a business user can open the sheet, enter data for new property (square footage, bedroom number, location), and call a BQML model to return a formula price estimate. No need for an aggregate or API swallowing – just a sheets formula called the model. This machine is a powerful way to expose learning in front of non -technical teams.
2. No price is a large price sandbox and Kolab notebook
Starting with enterprise data warehouses often involves friction, such as setting a billing account. Big Cory Sandbox It removes the barrier, and allows you to inquire up to 1 terabyte data per month. No credit card is needed. This is a huge, without costing to learn and start experimenting with large -scale analytics.
As a data scientist, you can access one from your larger sandbox COLAB Notebook. With just a few lines of the verification code, you can run SQL questions from a notebook and draw the results of the analysis into the data frame. The same notebook environment can also work as AI partner to help you plan to write your analysis and code.
3. Your AI -powered fellow in the Kolab Notebook
Data Science Agent in Kolab Notebook (Setting Settlement, Examples Objects Results)
Kolab Notebooks are now AI’s first experience Designed to accelerate your workflow. You can create a natural language code, get automatic error specifications, and chat with an assistant along with your code.
The Kolab notebook also contains a built -in data science agent. Think about it as an ML expert you can cooperate with. Start with Dataset – like local CSV or a large table – and a high -level goal, such as “Create a model to predict customer Mandor”. The agent develops a plan with the recommended steps (such as data cleaning, feature engineering, model training) and writes the code.
And you are always in control. The agent produces a direct code in the notebook cells, but does not run anything itself. You can review and modify each cell before deciding, or also ask the agent to re -consider his approach and try various techniques.
4. Make your pandas’s workflow with Big Coyry Data Fames
Many data scientists live in the notebook and use Pandas data frames for data manipulation. But it has a well -known limit: all your data is following your machine’s memory. MemoryError The exceptions are all very common, which makes you forced to reduce your data initially.
This is the same problem Big Query Data Fames The solution provides a deliberate API, like pandas. Instead of running locally, it translates your orders into the SQL and performs them on the Big Cairy Engine. This means that you can work with your notebook with terabyte scale datases, with a familiar API, and no worries about memory barriers. The same concept applies to the model training, which includes the Skate Learn -like API, which leads the model training toward the Big Query ML.
5. Big Curi Studio in the notebook spark ML
Sample spark ML Notebook in Big Cory Studio
Apache Spark Feature is a useful tool from engineering to model training, but management of infrastructure has always been a challenge. Server lace for Apache spark Allows you to run a spark code, including jobs, such as XG boosts, piturichs, and transformers, without providing any cluster. You can develop Interactive From a notebook directly within the Big Query, let you focus on the development of the model, while the Big Courie handles infrastructure.
You can use server lace sparks to work on the same data (and one governance model) in your large warehouse.
6. Add the outdoor context with public datases
Top 5 Trending Terms in the Los Angeles area in early July 2025
Your first party data tells you what happened, but can’t always explain it. To find out this context, you can join your data with a large reservoir of public datases available in the Big Query.
Imagine that you are a data scientist for a retail brand. You see the increase in the sale of rain coat in the Pacific. Was it your recent marketing campaign, or something else? By joining your sales data Google trends datasit In the Big Coyry, you can quickly see that searching for “waterproof jacket” has also increased in the same region and period.
Or we say you are planning a new store. You can use Visual Dataset Places To analyze traffic patterns and business density in potential palaces, you to choose the best location on your own customer’s information. These public datases allow you to make rich models calculate real -world factors.
7. Geographical analytics on a scale
Using color to indicate radius and wind speed, a large geographical map of a hurricane
Building features familiar with location of a model can be complicated, but Big Cory has made it easier by supporting one GEOGRAPHY The type of data And standard GIS works within SQL. This can help you with local features on the source. For example, if you are creating a model to predict the property unacceptable prices, you can use a function like ST_DWITHIN to calculate the number of public transit stops in a mile circle for each property. You can then use this value directly as input into your model.
You can take it more Google Earth Engine The integration, which brings satellite imagination and environmental figures to the Big Cory. The same property of the Unconstitutional Model, you can inquire from the Earth Engine data to add the risk of a historical flood or the density of the tree covering. This helps you create a much more rich model by increasing your business data with environmental information on a planet scale.
8. Log data.
Most people think of the Big Big Corey of analytical data, but it is also a powerful destination for operational data. You can root all of your Cloud Logging Data in Big QueryTransforming non -structured text logs into question resources. This allows you to run the SQL in all your services logs to diagnose problems, track performance, or analyze security events.
Data Scientists This, this is a great source for cloud logging data predictions. Imagine an investigation into a reduction in user activity. After indicating an error message in the logs, you can use Looking for Big Cory Vector To find similar logs, even if they do not have the exact text. This can help to display related issues, such as “user token wrong” and “verification failed”, which are part of the same main reason. You can then use the Use for the training of the labeled data, which actively flags the samples.
Conclusion
Hopefully, these examples have given rise to some new ideas for your next project. Pandas data frames, from scaling to engineering features with geography data, to help you work on a scale of familiar tools.
Are ready to give a shot? You can start searching for today without a price Big Cory Sandbox!
Author: Jeff Nelson, Developer Relationship Engineer