Google Cloud debuts new AI tools to boost data science productivity

On a mission to lighten the workload for data scientists, Google LLC’s cloud division today announced a wave of new artificial intelligence tools designed to help them build the next generation of AI agents while supporting their day-to-day work.

At Big Data London, an annual data and AI conference, Google Cloud unveiled enhancements to its data tools aimed at cutting down context switching, the constant need to move between different interfaces and tools to get a job done. For data scientists, that often means juggling SQL queries, Python code and data visualizations. Each switch can break focus and slow progress.

“Our priority is to eliminate this friction by creating the single, intelligent environment an architect needs to engineer, build, and deploy — not just run predictive models,” said Yasmeen Ahmad, managing director of Data Cloud at Google.

The enhancements are for Colab Enterprise notebooks, a data engineering environment that allows users to write Python code to analyze and process large amounts of data in BigQuery, the company’s cloud-native analytics database, and Vertex AI, a managed platform for building and deploying generative AI applications.

Supporting this unified environment, Google Cloud released Native SQL Cells in preview, the tool allows data scientists to iterate on SQL queries and Python code in the same place. As they go, they can pipe results directly into BigQuery DataFrame to build models in Python. DataFrame allows users to interact with massive datasets efficiently using familiar data science frameworks to perform machine learning tasks.

Also in preview, data scientists will be able to automatically generate richly interactive, editable charts from the data. Google calls the new feature set Rich Interactive Visualization Cells, which take raw data and transform it into visualizations that make it easy to understand the underlying information.

These two updates make it possible to work with SQL, Python and visualization all in one development environment for data science tasks.

Google also enhanced its Data Science Agent, an AI assistant that responds to natural language prompts. Now in preview, the agent supports advanced tool usage, meaning it can incorporate Google’s data science platforms directly into its workflow planning.

Powered by Google’s flagship Gemini AI models, the DSA can autonomously construct end-to-end analytical pipelines, from exploratory data analysis and data cleaning to machine learning predictions. It can now integrate with tools such as BigQuery ML for AI training and deployment and BigQuery DataFrames for Python-based analysis.

Building AI agents with unstructured data

Many high-value AI applications — including e-commerce, finance, customer support and logistics — rely on real-time access to unstructured data. Yet much of this data resides in separate systems such as event streams and specialized databases.

“To address this challenge, we are making real-time streams and unstructured data more accessible for data science teams,” said Ahmad.

To that end, Google announced stateful processing for BigQuery continuous queries. This gives SQL queries “memory,” enabling them to detect evolving patterns over time. Instead of returning isolated transactions, queries can now analyze live event data in real time.

For instance, a data engineer could ask: “Has this credit card’s average transaction value over the last five minutes spiked more than 300%?” An AI agent monitoring financial transactions could detect the anomaly and trigger protective actions, such as flagging the transaction for review or placing a temporary block.

Vector databases give AI models memory, but updating them can be a slow process where large amounts of data are added to them in chunks at a time, especially multimodal data such as images, video and audio. An update to Big Query Vector Search will now take care of updating vector databases automatically with new data as it arrives and as users chat with AI agents or use search systems.

Google also introduced an update that builds on BigQuery Vector Search, addressing one of the pain points of working with vector databases. While these databases give AI models and applications long-term memory, updating them can be slow, especially with multimodal data such as images, video and audio. The new capability enables automatic, continuous updates to vector databases as new data streams in, whether from user interactions with AI agents or from search activity.

Image: SiliconANGLE/Microsoft Designer, Google