Google iconGoogleSep 4, 2026 ~1 min source read

How Yahoo optimizes resources with flexible VMs in Managed Service for Apache Spark

As a global media and technology company connecting hundreds of millions of users to finance, sports, and entertainment platforms, Yahoo operates a massive data infrastructure where analytics workloads must run continuously at high speed. Yahoo utilizes flexible VMs in Managed Service for Apache Spark clusters to automatically absorb these resource fluctuations by defining a ranked list of acceptable VM shapes.

How Yahoo optimizes resources with flexible VMs in Managed Service for Apache Spark

Share this story

Send the public story page.

Useful takeaways from this story.

Yahoo utilizes flexible VMs in Managed Service for Apache Spark clusters to automatically absorb these resource fluctuations by defining a ranked list of acceptable VM shapes.

This optimization builds on Yahoo's broader data modernization journey, which involved migrating on-premises Hadoop and big data estates directly to Google Cloud.

As a global media and technology company connecting hundreds of millions of users to finance, sports, and entertainment platforms, Yahoo operates a massive data infrastructure where analytics workloads must...

Building the complete brief

The page is ready to read now. The fuller skim-friendly version will appear here automatically.

The useful part

Yahoo utilizes flexible VMs in Managed Service for Apache Spark clusters to automatically absorb these resource fluctuations by defining a ranked list of acceptable VM shapes. This allows the system to dynamically search regional zones and maintain pipeline execution without manual intervention. To search for capacity across a region, teams must also enable Auto-Zone placement.

How it works

  • This optimization builds on Yahoo's broader data modernization journey, which involved migrating on-premises Hadoop and big data estates directly to Google Cloud.
  • By transitioning those legacy workloads, the team established a cloud foundation capable of running high-scale batch and streaming analytics with dynamic resource flexibility.
  • As a global media and technology company connecting hundreds of millions of users to finance, sports, and entertainment platforms, Yahoo operates a massive data infrastructure where analytics workloads must...

Keep reading in the app

Open the app view to save this story, compare related coverage, and continue from the same source.

Open in app