Unlock your data on AI platform: Generative AI for multi -modal analytics

Sponsored material

Traditional data platforms have long performed well on structural questions on tabler data – think “How many units did the Western region sell in the last quarter?” This basic relative base is powerful. But with the increasing amount and importance of multi -modal data (such as images, audio, non -imposed text), the answers to proportional spiritual questions have become an important obstacle, relying on traditional, outdoor machine learning pipelines.

Consider a typical e -commerce scenario: “Identify the high return rate electronics products associated with photos of consumers, which show a sign of damage when it is.” Historically, this means using SQL for structural product data, sending pictures to a separate ML pipeline for analysis, and eventually trying to combine different results. Instead of being integrated locally in a multi -faceted, time -consuming process where the analytical environment, AI was primarily bothered on data flu.

Imagine dealing with this task – combining structural data with non -imposed visual media insights – using the same beautiful SQL statement. This jump is possible by integrating production AI directly into the core part of the modern data platform. It introduces a new era where sophisticated, multi -modal analysis can be implemented with the SQL familiar.

Let’s discover how the Generative AI is primarily renewing the data platform and allows practitioners to provide multi -modal insights with SQL verse.

Related algebra meets Generatito AI

Traditional data warehouses have gained their strength from a foundation in a relative algebra. It provides a mathematical and permanent framework to question the structural, tabler data, where the schemes are well explained.

But multi -modal data contains full spiritual content that can not directly interpret relative algebra, itself. Generative AI integration works as a spiritual bridge. This enables questions that include AI’s ability to translate complex signals embedded in multi -modal data, which makes a lot of reasoning as a human, and thus cross the types of traditional data and the obstacles of SQL functions.

To fully appreciate this evolution, first first find the architecture components that enable these abilities.

Generative AI in the process

From modern data, AI platforms allow business to interact with data by adding generative AI capabilities to their core. Instead of ETL pipelines for outdoor services, functions such as Big Curi AI.GENERATE And AI.GENERATE_TABLE Allow users to take advantage of the powerful large language model (LLM) using familiar SQL. These functions collect data from an existing table, as well as the user defined, found in the LLM, and returns the answer.

Analysis of non -imposed text

Consider the e -commerce business, which includes reviews of millions of products in thousands of items. Manual analysis is prohibited in this volume to understand customer’s opinion. Instead, AI functions can automatically remove key topics from each review and create comprehensive summary. These summary can provide immediate and insightful reviews to potential customers.

Multi -modal analysis

And these functions go beyond non -tabler data. Modern LLM can withdraw from multi -modal data. This data usually lives in cloud objects like Google Cloud Storage (GCS). Big Coyry simplifies access to these items ObjectRef. ObjectRef The column in GCS for analysis lives inside the standard Big Cairy Tables and the reference to the reference to safely.

Consider the possibility of connecting structural and non -structures for e -commerce.

• Identifying all phones sold in 2024 with repeated customer complaints of “Bluetooth pair problems” and the product user manual (PDF) for cross reference whether the defects are missing.
• List shipping carriers are often analyzed by the customer’s storage storage, showing affiliates related to the Western region, showing affiliate damage to users.

To solve conditions where insight depends on external file analysis, as well as structured table data, as well as the bug Corey uses ObjectRef. Let’s see how ObjectRef Increases a standard bug coeries table. Consider a table with basic product information:

We can easily add one ObjectRef Name of the column manuals In this example, the GCS -stored official product manual to refer to the PDF. It allows ObjectRef Structural data to live together:

This integration strengthens the latest multi -modal analysis. Let’s take a look at an instance where we develop a question and answer pairs of questioning (text) (text) and product manual (PDF):

SQL 

SELECT
product_id,
product_name,
question_answer
FROM
  AI.GENERATE_TABLE(
    MODEL `my_dataset.gemini`,
    (SELECT product_id, product_name,
    ('Use reviews and product manual PDF to generate common question/answers',
    customer_reviews, 
    manuals
    ) AS prompt, 
    FROM `my_dataset.reviews_multimodal`
    ),
  STRUCT("question_answer ARRAY" AS output_schema)
);

Instant argument of AI.GENERATE_TABLE This question has used three major inputs:

• A modest recipe to the model to create the questions usually asked
• customer_reviews Column (a string with a gross medical interpretation)
• manuals ObjectRef Column, directly connecting the product manual PDF

Function uses non -structured text columns And The basic PDF is preserved in GCS to carry out AI operation. Output is a combination of valuable question and answer pairs that help potential users better understand the product.

Increase the utility of Object Reef

We can easily add additional multi -modal assets by adding more ObjectRef Columns on our table. Continuing with e -commerce scenario, we add a ObjectRef Called column product_imageWhich refers to the official product image shown on the website.

And since ObjectRefThere are types of structure data, they support nesting with rows. It is particularly powerful for scenario where a basic record is related to numerous non -structural objects. For example, A customer_images The column can be a row ObjectRefS, each indicates a different customer uploaded product image stored in GCS.

The ability to make more than one to one relationship flexible between structural records and various non -structural data (inside the Big Curry and the use of SQL) opens up an analytical possibility that requires several external tools.

Type specific AI functions

AI.GENERATE Functions offer flexibility in the output schemes, but that the general analytical tasks that require strictly typed output, the bugquer type provides specific AI functions. These functions can analyze the text or ObjectRefWith SLLM and return the reaction directly into the Big Corridor.

Here are some examples:

• Ai.generate_BooL: The process returns input (text or object refills) and a bol value, which is useful for emotion analysis or any right/wrong commitment.
• ai.generate_int: The data returns a number useful for extracting numerical attributes according to numeric, rating, or quantity.
• ai.generate_double: Return a floating point number useful for extracting scores, measurements, or financial values.

The main advantage of these types of specific functions is to enforce their output data types, which is to ensure predicted scaler results (such as boys, investors, doubles) using a simple SQL.

For example of our e -commerce, imagine that we want to flag product reviews quickly that mention shipping or packaging issues. We can use AI.GENERATE_BOOL For this binary rating:

SQL

SELECT *
FROM `my_dataset.reviews_table`
AI.GENERATE_BOOL(
   prompt => ("The review mentions a shipping or packaging problem", customer_reviews),
   connection_id => "us-central1.conn");

The inquiries filter the records and return the rows, which mentions shipping or packaging matters. Note that we did Not Keywords (such as “broken”, “bad”) must be explained – this spiritual meaning within each review is reviewed by LLM.

Bring it all together: a united multi -moodle query

We have discovered how the Generative AI data platform enhances the capabilities. Now, we review the e -commerce challenge presented in the introduction: “Identify high return rate electronics products associated with consumer images that show signs of damage when it is shown.” Historically, it requires separate pipelines and often spread many individuals (data scientists, data analysts, data engineers).

With integrated AI capabilities, a beautiful SQL inquiry can now resolve this question:

This united inquiry shows an important evolution of how data platforms work. Instead of storing and recovering different data types, the platform becomes an active environment where users can ask business questions and return answers by direct analysis with structural and non -imposed data using familiar SQL interface. This integration offers a more direct path to insight that requires special skills and tooling first.

AI Question Engine with meaningful reasoning (coming soon)

While the functions like AI.GENERATE_TABLE Rove AI is powerful for processing (reinforcing individual records or developing new data), the Big Curie’s purpose is to integrate more comprehensive, spiritual reasoning with AI inquiry engine (AIQE).

The purpose of the AIQ is to empower data analysts, even without deep AI skills, to perform complex term arguments throughout the datases. AIQE acquires it by summarizing complexities like quick engineering and allows consumers to focus on business logic.

Sample AIQE functions may include:

• ai.if: For cement filtering. An LLM estimates that if a row data is immediately associated with a natural language state (such as “return product reviews that raise high heat concerns”).
• Ai.join: Natural language -expressed spiritual similarities or relationships are included in tables – not just clearly equations (such as “link customer support tickets from relevant parts on the basis of your product knowledge”)
• Ai.score: Rows or rows of orders are how well they meet a spiritual condition, which is useful for “top-K” scenario (such as “Top 10 Find Best Customer Support Calls”).

Results: Data Platforms developed

The data platforms are permanently in a state of evolution. From structural, related data management Organization, they now accept opportunities offered by non -imposed, multi -modal data. Support for Direct integration of AI -powered SQL operators and discretionary files in Object Stores with such mechanisms and such mechanisms ObjectRef We represent the fundamental change in how we interact with the data.

Since the lines between data management and the AI ​​are combined with each other, the data warehouse is the focus of enterprise data-now it is affected by the ability to understand the more, more human, such as more human. Complex multi -modal questions that once needed different tools and widely AI skills can now be focused with more simplicity. This evolution towards more capable data platforms is continuing to democratic sophisticated analytics and allow the wider range of SQL professional users to gain deep insights.

To discover these abilities and start working with multi -modal data in Big Query:

Author: Jeff Nelson, developer relationship engineer, Google Cloud