Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
San Francisco-based Monte Carlo Data, a provider of automated data observability capabilities for enterprises, today expanded its footprint by announcing a no-code integration with data ingestion major Fivetran. The move will enable joint customers of the companies to monitor the health of their data pipelines end-to-end.
Monte Carlo has already announced integrations with multiple players in the data ecosystem, including Snowflake and BigQuery. The integration with Fivetran marks another step for Monte Carlo toward complete data observability for the modern data stack, right from the orchestration layer to data stores. The company competes with players like Acceldata and BigEye in the space.
The data game
Enterprises today are bullish on data-driven initiatives. They’re ingesting more and more information from different sources, making pipelines complex and increasing the chances of data downtime — periods when data is inaccurate or erroneous. Bad data can directly harm applications and users, leading to inaccurate decision-making and eventually days’ worth of fire-fighting.
Monte Carlo, founded in 2019, addresses this challenge with its machine learning-powered platform that provides enterprise data analysts with a holistic view of data reliability for critical business and data product use cases in near real time. The platform detects and sends alerts about incidents faster, allowing teams to address data downtime before it has a significant impact on the business.
Fivetran integration for pipeline observability
With its engagement with companies like Snowflake, Databricks and Google BigQuery, Monte Carlo addressed the challenge of monitoring data where it’s stored, but that did not include the complete ecosystem. So the company started expanding to other layers, plugging into business intelligence tools like Looker, Tableau and Mode as well as ETL offerings such as dbt and Airflow.
“We’ve seen incredible adoption of our dbt integration, which has demonstrated to us that customers require more and more visibility into the orchestration layer. And with Fivetran being one of our customers’ favorite ELT tools, this new integration will be transformative and give them the ability to detect and troubleshoot [data pipeline] issues faster,” Lior Gavish, CEO and co-founder of Monte Carlo, said.
Gavish told Venturebeat that the quick, no-code integration ensures data pipelines built with Fivetran connectors are monitored as soon as they are built. Observability is built right into them, giving a clear view of upstream problems — from issues related to data freshness, volume and distribution to schema and lineage gaps — before they impact downstream products.
Notably, as and when an issue occurs, notifications are surfaced within the Monte Carlo UI and sent as alerts in Slack, Microsoft Teams, etc. This takes the user directly to all the information and context, like Fivetran logs, that they need to address the problem.
“The integration is the latest in a long list of solutions and platforms that Monte Carlo has available … that scale observability across the modern data stack. Our solution has helped companies using Fivetran reduce data incidents by nearly 90% and save over 100 hours per quarter in data engineering time spent detecting and resolving data quality issues,” Gavish added.
The offering has been beta-tested by multiple enterprises, including Backcountry.com, Checkout.com and MasterClass, and can now be added to the stack free of charge by joint customers of the two companies.
