BigQuery ObjectRef Brings Multimodal Data and AI into SQL Workflows

Oct 21, 2026 535 views

BigQuery wants unstructured data in the same workflow

For teams working with images, video, audio and documents, the hard part has often been the plumbing. Files sit in Cloud Storage; models run elsewhere; and their results have to be stitched back into a warehouse before analysts can use them. That handoff adds operational work and can leave database records out of sync. BigQuery’s ObjectRef and multimodal analytics features are now generally available. The announcement’s larger claim is that BigQuery can serve as a single platform for structured and unstructured data, analytics and AI agents. That’s a meaningful shift in where teams can run these workflows, though the announcement doesn’t quantify how much time or cost they’ll save. ObjectRef is a built-in STRUCT that points from a BigQuery row to an object stored outside BigQuery. Teams can keep the file in Cloud Storage and place its reference alongside ordinary relational columns. BigQuery can then pass multiple columns, including multimodal inputs, to Gemini in one SQL query—or through a natural-language query in conversational analytics. Its integrations with BigQuery AI, BigQuery conversational analytics and Gemini Enterprise can combine files with database context such as metadata, timestamps and custom agent instructions. The pitch is that teams can build and govern pipelines that turn unstructured data into structured results, transform it into other unstructured data, or work across modalities, using BigQuery’s serverless compute and built-in inference without separate compute clusters. Keeping the ObjectRef in the same row as extracted features and annotations also preserves access to the source file. That matters: a table of labels and scores can support reporting, but it may lose context that an image, recording or document still contains. The post says customers already use the approach to generate embeddings for multimodal vector search, classify data at scale with generative AI, and extract data and features from large document and media archives—all through standard SQL. One simple example asks BigQuery to summarize the main categories in a public product image dataset: ```sql SELECT AI.AGG(image_ref, "Summarize the major categories of pictures") FROM `bigquery-public-data.cymbal_pets.products`; ```

References stored with the data

ObjectRef is a predefined BigQuery STRUCT containing a URI, an authorizer, a version and a details field with Cloud Storage metadata: ```sql -- ObjectRef is a special STRUCT in BigQuery STRUCT { uri STRING, authorizer STRING, version STRING, details JSON { gcs_metadata JSON } } ``` Storing these references in regular BigQuery tables makes files usable alongside conventional columns, while keeping the file itself in Cloud Storage. The two systems can follow independent data lifecycles: changes to a BigQuery table don’t alter the underlying Cloud Storage object, and changes to Cloud Storage don’t alter the table. That design brings several practical benefits. It avoids copying high-volume media into the warehouse, so teams can query files where they’re stored. They can work through SQL or BigQuery DataFrames, and pass ObjectRefs to BigQuery Python UDFs to run open-source libraries such as OpenCV on the data. Version tracking records the Cloud Storage generation, letting a query refer to the precise file version associated with a row even after the source object changes. And BigQuery’s existing security policies and IAM controls can govern, audit and share access to both the table columns and their related files. Since the preview, users have adopted a pattern the post describes as full-fidelity ETL in SQL. BigQuery’s serverless setup lets teams run AI and machine learning in their data environment without separate clusters for moving data and running inference. Treating Cloud Storage objects as BigQuery columns also lets data engineers apply familiar practices—idempotent pipelines, reproducible runs and backfills—to images, video, audio and documents. Here’s the useful distinction: teams can extract structured features while retaining the original ObjectRef in the same row. Those features support filtering, reports and BI dashboards; the source file remains available for analysis that needs more context. For example, sentiment scores and topic tags from customer calls could flag at-risk accounts in a dashboard. A conversational analytics agent could then inspect the original recordings to assess how customers and service representatives handled escalations. The source audio can retain nuance that a table of extracted fields cannot. image1

Where customers are applying it

At Snap, the feature has been used to change how reviewers sample public posts. Carlos Ortiz, a data scientist at Snap, described the shift:
“At Snap, the safety of our community is a top priority, and we aim to limit users’ exposure to content that violates our Community Guidelines. We regularly review samples of public posts, and by leveraging ObjectRef with BigQuery AI, we transitioned from simple random sampling to post-stratification sampling. This transition improved the efficiency of our human review processes, enabling us to reinvest reclaimed capacity into new and expanded workflows that further enhance our ability to detect and address potentially harmful content.” — Carlos Ortiz, Data Scientist, Snap
The post says preview customers have moved experimental pipelines into production across several workloads: - **Content moderation:** Online platforms combine multimedia assets with telemetry in one BigQuery pipeline to improve moderation and safety outcomes. - **Threat detection:** Enterprise security teams analyze visual and structural web components alongside network logs to identify threats proactively. - **Inventory compliance:** Online retailers search large image repositories for text and context that may reveal regulatory or brand compliance issues. - **Customer experience:** Service organizations analyze customer and support-agent audio with BigQuery AI to assess sentiment and improve service delivery.

What changed at general availability

The GA release adds three capabilities. First, ObjectRefs support End User Credentials (EUC), so people can explore unstructured data for ad hoc analysis using their own IAM identities. Explicit BigQuery connections, which were required during preview, remain supported. The post recommends them for production pipelines that use service accounts, although setting them up takes extra work. Second, ObjectRef can use Cloud Storage metadata from BigQuery object tables, Cloud Storage insights datasets and raw string URIs in standard BigQuery tables. Third, it supports ObjectRefs in arrays, opening up workflows such as comparing documents, analyzing image sequences and batching items in one semantic call.

Example: classify call-center audio

Call analysis has traditionally involved sending recordings to separate transcription and language-processing clusters. With ObjectRef, teams can classify audio and write the results into ordinary relational columns. The GA `OBJ.MAKE_REF()` function detects the audio format from a URI. BigQuery’s [managed AI functions](https://docs.cloud.google.com/bigquery/docs/generative-ai-overview#managed_ai_functions) include `AI.CLASSIFY()` for known tasks such as assigning call types. By default, BigQuery uses [Optimized Mode for AI Functions](https://docs.cloud.google.com/bigquery/docs/optimize-ai-functions) to reduce token consumption. ```sql CREATE OR REPLACE TABLE call_center.annotated_call_logs AS SELECT customer_id, call_date, OBJ.MAKE_REF(audio_uri) AS ref, AI.CLASSIFY( OBJ.MAKE_REF(audio_uri), [ "Lost or stolen card", "Declined card payment", "Activate my card", "Order physical card", "Cancel transfer", "Password or pin forgotten", "Virtual card not working", "Card swallowed by ATM", "Balance not updated after bank transfer", "Balance not updated after check or cash deposit", "Extra charge on statement", "Transfer fee charged" ] ) AS customer_issue_type FROM call_center.raw_call_logs; ``` The resulting table can feed a BigQuery conversational analytics agent. The agent turns natural-language questions into SQL and can use BigQuery AI functions to inspect recordings when it needs more information.

Example: filter product uploads with AI

For large collections of images or documents, engineers may want to filter by content without generating and parsing long text responses. BigQuery’s declarative `AI.IF()` operator can sit in a SQL `WHERE` clause, and its default [Optimized Mode](https://docs.cloud.google.com/bigquery/docs/optimize-ai-functions) reduces token consumption. ```sql SELECT product_id, seller_id FROM ecommerce.vendor_uploads WHERE upload_date >= CURRENT_DATE() - 1 AND AI.IF( "Does this product image contain a restricted item or policy violation?", OBJ.MAKE_REF(image_uri) ); ```

Spot phishing pages by comparing screenshots

The bigger shift is that BigQuery can now reason across several files in one Gemini call. With native array support generally available, you can pass multiple `ObjectRef` values together and compare their contents directly. For a security team, that could mean matching a trusted login page against a suspicious one. The SQL below asks Gemini to flag visual differences, missing logos, and fake elements, then return a phishing verdict and an explanation: ```sql SELECT log_id, suspect_domain, assessment.is_phishing, assessment.visual_discrepancies FROM ( SELECT log_id, suspect_domain, AI.GENERATE( ( "Compare these two web UI screenshots. The first is our official, trusted login page. The second is a suspected spoofed page. Identify any visual discrepancies, missing logos, or fake elements, and determine if it is a phishing attempt.", [OBJ.MAKE_REF(trusted_ui_uri), OBJ.MAKE_REF(suspect_ui_uri)] ), output_schema => 'is_phishing BOOLEAN, visual_discrepancies STRING' ) AS assessment FROM cybersecurity.network_ui_logs ); ``` This puts comparative multimodal analysis inside a SQL workflow, which could make it easier to apply across large collections of network logs. The example shows how to produce a structured assessment; it doesn’t establish how reliably the model detects phishing, so teams would still need to validate results against their own pages and threat data.

Get started with BigQuery ObjectRef

BigQuery ObjectRef is generally available across all BigQuery editions and on-demand pricing tiers. If you’re running separate infrastructure to perform machine learning inference on files in Cloud Storage, bringing that work into BigQuery may simplify the pipeline and let your data engineering practices handle more of the workflow. You can try the approach with the [hands-on tutorial](https://docs.cloud.google.com/bigquery/docs/multimodal-data-sql-tutorial), which uses synthetic data to generate image captions and assign product categories. Cloud Next also has demos on [unifying unstructured and structured data](https://www.youtube.com/watch?v=hBg8FB6vCQ8&t=1s) and [generating text with LLMs and performing vector search](https://www.youtube.com/watch?v=dPcta6Jvuvs&t=2s). For implementation details, see the BigQuery documentation on [working with ObjectRef](https://docs.cloud.google.com/bigquery/docs/work-with-objectref).
Source: Guarav Soni · cloud.google.com

Comments

Sign in to comment.
No comments yet. Be the first to comment.

Related Articles

BigQuery ObjectRef now GA: Build multimodal data platform...