Teradata Embraces Open Table Formats, Iceberg and Delta Lake, to Deliver the Most Open and Connected Ecosystem for Trusted AI
Teradata (NYSE: TDC) today announced a uniquely open and connected approach to supporting open table formats (OTFs) Apache Iceberg and Linux Foundation Delta Lake, embracing the industry pivot toward open-source technologies and offering unparalleled customer choice in data management. This move adds a forward-looking dimension to Teradata VantageCloud Lake, Teradata’s next-generation, cloud-native analytics and data platform for AI, as well as Teradata AI Unlimited, an on-demand and cloud-native AI/ML engine – which will move to public preview on both AWS and Azure Marketplaces beginning in Q2 2024. Teradata’s fully open and connected approach is designed to be future-ready and allow enterprises to employ a modern data strategy with unmatched agility and flexibility for executing Trusted AI at scale.
OTFs represent a significant change from proprietary data storage to more flexible storage that can be used across platforms. The vision is greater interoperability, cost efficiency, and choice. While some companies claim to offer the openness afforded by OTFs, they lock users into preferred formats that stifle the creativity and innovation early adopters want and future users will expect. Teradata’s agnostic OTF support and open catalog integration is designed to enable the platform to read various catalogs with predictable execution. And with its unique implementation and approach to parallel processing, workload management, and query optimization of shared data, the platform is designed to deliver the best performance in the market. Across its cloud-native, multi-cloud, and on-demand offerings, Teradata provides broad choice for data management, without sacrificing the powerful and scalable engine that differentiates Teradata in its ability to optimize value and business outcomes.
“In today’s data landscape, we’re seeing wide adoption of open table formats with 51% of organizations actively adopting Delta tables and 27% adopting Apache Iceberg. This trend reflects the industry’s focus on a single source of data and the ability to leverage multiple engines against that data,” said David Menninger, Executive Director at Ventana Research, part of ISG. “Teradata’s commitment to open table formats, alongside the introduction of VantageCloud Lake and AI Unlimited engines working with these formats, empowers customers to combine low-cost storage with enterprise performance and capabilities.”
Teradata’s platform, the industry’s most open, offers first-party services for Apache Iceberg and Linux Foundation Delta Lake OTFs with full support for cross-read and cross-write data stored in multiple OTFs. This interoperability extends to AWS Glue, Unity, and Apache Hive catalogs and works in multi-cloud and multi-data lake environments, which is designed to deliver a connected, harmonized data experience to Teradata customers without the need to move or transform data.
“The future is open. Teradata has cultivated a thriving ecosystem for many years now and our embrace of OTFs, as well as the expansion of our on-demand AI/ML offering, shows just how serious we are about providing organizations the most open and connected ecosystem for cloud analytics and data,” said Hillary Ashton, Chief Product Officer at Teradata. “This is especially critical as companies work to incorporate Trusted AI into their ecosystems because an open platform like ours is intended to deliver massive benefits – accelerated innovation, improved productivity and more – but with robust support for the guardrails required for trust, like transparency and explainability.”
AI adoption has driven a rapid consolidation of data warehouses, analytics, and data science workloads into unified lakehouses. OTF support further enhances Teradata’s lakehouse capabilities, providing a storage abstraction layer that’s designed to be flexible, cost-efficient, and easy-to-use. Users can leverage AI Unlimited and other analytical engines against their own datasets, in their own cloud object stores – enabling them to take an agile, best-of-breed approach without the challenge and expense of moving data. Secure, shared access to large datasets enables Teradata customers to build data pipelines, score and train models, improve analytics and decision-making, and much more.
Teradata AI Unlimited
AI datasets are one of the most common ways organizations are using OTFs today, so it’s critical that platforms meant to accelerate AI use cases, like Teradata AI Unlimited, support this data format. Teradata AI Unlimited is an on-demand and cloud-native AI/ML engine that’s fully open and connected – now with support for OTFs – which is intended to make AI innovation faster, easier, and more cost-effective. Soon- to-be-available in public preview on AWS and Azure Marketplaces, AI Unlimited allows data scientists, data engineers, and developers to explore, experiment, and operationalize AI projects at scale.
Teradata AI Unlimited is designed to provide high-performance compute, massive parallel processing, and the advanced in-engine analytics required to execute and operationalize AI workloads. AWS SageMaker and Azure Machine Learning can be integrated into AI Unlimited workloads, and Azure OpenAI Services provide large language models (LLMs) that can be used to develop generative AI applications. Best of all, work prototyped in AI Unlimited can be smoothly promoted to production in VantageCloud, which Teradata believes will provide quick and substantial value to enterprises.
AI Unlimited also includes the robust capabilities of Teradata’s industry-leading ClearScape Analytics, which is designed to provide organizations the transparency, explainability, and repeatability needed for Trusted AI.
New Integrations
OTFs and AI Unlimited can reduce the cost and complexity of AI workloads by minimizing data storage costs, as well as the need for data movement or replication. Teradata has also added new integrations with third-party tools – Airbyte Cloud, Apache Airflow™, and dbt™ (data build tool) – a part of the modern data stack. These integrations are intended to accelerate the building of data pipelines, increase productivity by freeing data engineers from repetitive work, and drive profitable growth.
- Airbyte Cloud: Streamline data ingestion into VantageCloud with a fully managed and hosted service that eliminates the need for time consuming infrastructure setup and management.
- Apache Airflow: Programmatically author, schedule, and monitor workflows.
- dbt (data build tool): Manages the T in ELT (Extract Load Transform) and is a commonly used tool for data transformation in databases, data lakes, and data warehouses.
Availability
OTF support will be available for VantageCloud Lake and AI Unlimited on AWS and Azure in June 2024.
AI Unlimited is available for purchase in Q2, as a public preview, on the AWS and Azure Marketplaces.
Integrations with Airbyte Cloud, Apache Airflow, and dbt are available now. More information can be found on this blog post.
This release contains forward-looking statements within the meaning of Section 21E of the Securities and Exchange Act of 1934. Forward-looking statements generally relate to opinions, beliefs, and projections of expected future financial and operating performance, business trends, liquidity, and market conditions, among other things. These forward-looking statements are based upon current expectations and assumptions and often can be identified by words such as “expect,” “strive,” “looking ahead,” “anticipate,” “continue,” “plan,” “believe,” “will,” “would,” “likely,” “intend,” “potential,” “designed,” or similar expressions. Forward-looking statements in this release include the availability, capabilities, and benefits provided by the integration of open table formats with Teradata’s VantageCloud platform and Teradata AI Unlimited. Forward-looking statements involve risks and uncertainties that could cause actual results to differ materially, including those relating to the rapidly changing and intensely competitive nature of the information technology industry, the data analytics business, and artificial intelligence capabilities; the timely and successful development, production or acquisition, availability and/or market acceptance of new and existing products, product features and services, including for artificial intelligence; and other factors described from time to time in Teradata’s filings with the U.S. Securities and Exchange Commission, including its most recent annual report on Form 10-K, and subsequent quarterly reports on Forms 10-Q or current reports on Forms 8-K, as well as Teradata’s annual report to stockholders. Teradata does not undertake any obligation to publicly update or revise any forward-looking statements, whether as a result of new information, future events or otherwise, except as required by law.
