CloudPro

We’re leaving Kubernetes -GitpodCloudPro #77: How to ace (CKS 2.0) Certified Kubernetes Security Specialist ExamStop worrying about your to-do list.Zapier connects the apps you use every day, so you can focus on what matters most.Start working more efficiently - Create your free account today.Get started for free⭐MasterclassWe’re leaving Kubernetes -GitpodHow to ace (CKS 2.0) Certified Kubernetes Security Specialist ExamCreating alerts from panels in Kubernetes Monitoring: An overlooked, powerhouse featureManaged DevOpsPoolsOptimizing Kubernetes Costs with Multi-Tenancy and Virtual Clusters🔍Secret KnowledgeI followed the official AWS Amplify guide and was charged $1,100What I wish someone told me about PostgresChoose the Right Instance Size for AWS RDSBuilding databases over a weekendMigrating billions of records: moving our active DNS database while it’s in use⚡TechwaveStreamline Kubernetes cluster management with new Amazon EKS Auto ModeOpenTelemetry for Generative AISimplify AWS governance with declarative policiesIntroducing Buy with AWSAWS Database Migration Service now automates time-intensive schema conversion tasks using generative AIAmazon DynamoDB reduces prices🛠️HackhubPkdnsis a DNS server providing self-sovereign and censorship-resistant domain names.Macosprovides a way to run macOS inside a Docker container using KVM acceleration.Pgpdfis an extension for PostgreSQL that provides a pdf data type and assorted functions.Kloudliteis an open-source platform designed to provide seamless and secure development environments for building distributed applications.OpenObserveis a cloud-native observability platform built specifically for logs, metrics, traces, analytics, and RUM designed to work at a petabyte scale.This is our final edition of CloudPro for 2024, but don’t worry—we’ll be back with more insights and updates in January 2025. In the meantime, we’ve got a little holiday treat for you!Packt has some exciting offers lined up to help you boost your tech skills and get ready for an amazing new year! It’s the perfect opportunity to relax, learn something new, and stay ahead in your field. Keep an eye out for these special holiday deals!From all of us at the Packt Newsletters team, we wish you a joyful holiday season and a fantastic start to 2025. See you next year!Cheers,Shreyans SinghEditor-in-ChiefMastering Software Deployments at the Edge: A User’s Guide to Diverting DisasterSoftware delivery to dedicated edge devices is one of the most complex challenges faced by IT professionals today. While edge deployments come with inherent complications, it’s possible to avoid the pitfalls. With this guide in hand, a little planning, and the right tools and strategies in place, you can be confident you’ll never push a faulty update at scale.Read the Guide⭐MasterClass: Tutorials & GuidesWe’re leaving Kubernetes -GitpodGitpod decided to move away from Kubernetes after realizing it isn't ideal for cloud-based development environments due to their unique demands: they are highly stateful, interactive, resource-intensive, and require broad system permissions. Despite Kubernetes' strengths in scalability and orchestration for production workloads, Gitpod faced challenges with performance, security, and resource management at scale, particularly with CPU and memory usage, storage, autoscaling, and startup times. Extensive experimentation with custom solutions for these issues proved complex and limited. While Kubernetes excels for controlled, predictable application workloads, Gitpod’s experience highlighted the mismatched fit for development environments, leading to a shift toward more tailored infrastructure.How to ace (CKS 2.0) Certified Kubernetes Security Specialist ExamTo ace the Certified Kubernetes Security Specialist (CKS) 2.0 exam, start by ensuring you have a solid understanding of Kubernetes fundamentals and meet the prerequisite of obtaining the Certified Kubernetes Administrator (CKA) certification. The exam focuses on hands-on, performance-based tasks across key domains like cluster setup, hardening, system and supply chain security, and runtime monitoring. Utilize trusted study materials such as the Kubernetes documentation, platforms like KodeKloud, and mock exams from Killer.sh, which are often harder than the real exam. During the test, manage time effectively by tackling easier questions first, using aliases and shortcuts for command-line tasks, and referring to allowed documentation for efficient problem-solving.Creating alerts from panels in Kubernetes Monitoring: An overlooked, powerhouse featureGrafana Cloud's new alerting feature for Kubernetes Monitoring simplifies incident management by letting users create alerts directly from app panels. This powerful yet understated tool pulls queries from panels, lets you set thresholds, and sends notifications when they're exceeded. Ideal for tracking metrics like CPU usage, costs, and network health, it allows teams to manage infrastructure proactively.Managed DevOpsPoolsManaged DevOps Pools (MDP) simplify Azure DevOps agent management by providing a Microsoft-managed platform (PaaS) that integrates seamlessly with Azure DevOps to create scalable, secure, and customizable agent pools. Acting as a wrapper around Virtual Machine Scale Sets, MDP automates infrastructure management, allowing agents to be spun up on demand based on workload needs. It supports various image types, including preconfigured Microsoft Azure Pipeline Images, and offers private networking options for enhanced security.Optimizing Kubernetes Costs with Multi-Tenancy and Virtual ClustersManaging Kubernetes costs effectively requires innovative approaches, especially as organizations scale. Traditional methods like resource quotas, autoscaling, and cost monitoring help, but they fall short when dealing with the inefficiencies of running numerous underutilized clusters. Multi-tenancy with virtual Kubernetes clusters offers a cost-efficient solution by enabling multiple teams or applications to share a single host cluster while maintaining strong isolation and flexibility. Virtual clusters act like fully functional Kubernetes clusters within a host cluster, reducing redundancies and management fees while improving resource utilization.🔍Secret Knowledge: Learning ResourcesI followed the official AWS Amplify guide and was charged $1,100The author followed an AWS Amplify guide to integrate OpenSearch and ended up with a shocking $1,100 bill due to unexpected behaviors in the setup. Specifically, the guide’s default configurations created high-cost OpenSearch instances without making costs transparent, and resources were not properly deleted when shutting down the environment. The author identified issues like persistent OpenSearch domains and a lack of warnings about default expensive configurations. AWS refunded the charges and advised setting up budget alerts, but the problematic behavior in the guide still exists. The post cautions developers about potential pitfalls when using AWS Amplify with OpenSearch and highlights the importance of understanding AWS costs and configurations.What I wish someone told me about PostgresPostgres is a powerful but complex database system, and its vast official documentation can be overwhelming. Key tips for getting started include normalizing your data to avoid redundancy, except when performance optimizations (denormalization) are necessary. Understand SQL quirks like handling NULL as "unknown" and utilizing functions like COALESCE. Enhance the usability of psql by configuring features like pagers and shortcuts (e.g., \x for expanded view). Use indexes wisely, considering their order and suitability for different queries (e.g., prefix searches need text_pattern_ops). Be cautious with locks during operations like ALTER TABLE, as long-held locks can disrupt other processes. Embrace tools like query plans (EXPLAIN) to optimize performance, and always start with the Postgres documentation and community advice for best practices.Choose the Right Instance Size for AWS RDSTo choose the right AWS RDS instance size, start by evaluating your workload's needs in terms of CPU, memory, storage, and network bandwidth. Use AWS instance families to match these requirements, with memory-optimized instances for RAM-intensive tasks and burstable instances for cost-sensitive, sporadic workloads. Monitor key performance metrics, like CPU utilization, freeable memory, and network throughput, using AWS CloudWatch, and adjust the instance size based on consistent patterns—scale down if utilization is low and up if demands are high. Optimize performance with database tuning and continuously revisit your setup to balance cost, scalability, and performance effectively.Building databases over a weekendBuilding a database over a weekend is made feasible with tools like Apache DataFusion, which simplifies creating custom database functionalities. DataFusion provides a modular framework where you can extend or replace components like query parsing, logical and physical planning, and execution engines. By leveraging its SQL and DataFrame interfaces, you can implement custom operators, such as a streaming window operator for handling infinite data streams, by defining execution plans and integrating them into the planning pipeline. Through logical and physical optimizations, you ensure efficient query execution tailored to your use case.Migrating billions of records: moving our active DNS database while it’s in useCloudflare recently migrated its active DNS database to a new cluster to handle increasing data volumes and improve performance. Originally, DNS records were stored in a primary Postgres database alongside other services, but as Cloudflare scaled, this became increasingly problematic. The migration involved separating DNS records from other data, implementing a new gRPC API for better control, and using a Change Data Capture and Transfer Service to move data efficiently with minimal downtime. The new setup, which included better indexing and partitioning, reduced API latency and improved overall performance.⚡TechWave:CloudNews & AnalysisStreamline Kubernetes cluster management with new Amazon EKS Auto ModeWith EKS Auto Mode, AWS simplifies Kubernetes cluster management, automating compute, storage, and networking, enabling higher agility and performance while reducing operational overhead.OpenTelemetry for Generative AIOpenTelemetry is being enhanced to support observability for generative AI applications, ensuring reliable performance, cost efficiency, and safety. It introduces Semantic Conventions to standardize telemetry data across platforms and an Instrumentation Library to automate data collection, initially focusing on the OpenAI Python API. Key signals like Traces, Metrics, and Events provide insights into model behavior, usage, and interactions, aiding in debugging, optimization, and performance tuning. Developers can easily integrate this observability into applications using the provided Python library, enabling monitoring of model inputs, outputs, and operational details.Simplify AWS governance with declarative policiesAWS Declarative Policies simplify governance by enabling organizations to define and enforce cloud resource configurations centrally and at scale. Administrators can set standards, like blocking public access to VPCs or requiring specific Amazon Machine Images (AMIs), which are automatically applied across accounts, including new ones joining the organization. These policies reduce complexity by maintaining configurations even as AWS services evolve, providing actionable error messages to users for non-compliant actions. Initially supporting Amazon EC2, VPC, and EBS, declarative policies are managed via AWS Organizations and other AWS tools.Introducing Buy with AWSAWS introduces "Buy with AWS," a new feature that streamlines the procurement of cloud solutions by integrating AWS Marketplace purchasing directly into AWS Partner websites. Customers can discover, try, and purchase solutions with their AWS accounts, benefiting from simplified billing, centralized subscription management, and cost optimization tools. For example, users can start free trials or request private offers for products like Wiz or Databricks directly from Partner sites, with seamless transitions to co-branded procurement pages. Partners, in turn, can enhance their customer experience with AWS Marketplace APIs to showcase products, provide filters, and track metrics for engagement and sales.AWS Database Migration Service now automates time-intensive schema conversion tasks using generative AIAWS Database Migration Service (AWS DMS) now uses generative AI to automate up to 90% of schema conversion tasks, simplifying migrations from commercial databases to PostgreSQL. Powered by large language models hosted on Amazon Bedrock, this feature tackles complex code conversions like stored procedures and proprietary functions that traditional methods often struggle with. It reduces migration costs, accelerates timelines, and allows users to focus on optimizing their applications post-migration.Amazon DynamoDB reduces pricesAmazon DynamoDB, a serverless NoSQL database with high performance and scalability, has significantly reduced its pricing: on-demand throughput costs are now 50% lower, and global table replicated writes are up to 67% cheaper. These changes make on-demand mode—ideal for scaling serverless applications without capacity planning—the default and most cost-effective option for many workloads, even those with steady usage. Additionally, global tables now offer the same pricing for multi-Region and single-Region writes, simplifying cost management for globally distributed applications.🛠️HackHub: Best Tools for CloudPkdnsis a DNS server providing self-sovereign and censorship-resistant domain names.Macosprovides a way to run macOS inside a Docker container using KVM acceleration.Pgpdfis an extension for PostgreSQL that provides a pdf data type and assorted functions.Kloudliteis an open-source platform designed to provide seamless and secure development environments for building distributed applications.OpenObserveis a cloud-native observability platform built specifically for logs, metrics, traces, analytics, and RUM designed to work at a petabyte scale.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

A clean way to mount read-only image data in KubernetesCloudPro #90: Running as root without being root: Kubernetes finally gets user namespaces rightLive Webinar | Scale AppSec with Security Champions – May 15Security Champions programs are a proven way to scale AppSec across dev teams. Join Snyk’s live webinar on May 15 @ 11AM ET where we’ll cover👇✓ Defining the role of security champions✓ Designing a scalable, tailored program✓ Recognizing, rewarding & growing your champions🎓 BONUS: Earn CPE credits for attending!Save your spot!🔐 Cloud SecurityWe once let a vendor upload files directly into our S3 bucket. It was fine, until it wasn’t.If you’re building GenAI on AWS and winging the security architecture, start here instead.How I use LLMs as a staff engineerOur infra used to depend on whoever remembered how they set it up last time. This fixed it. How stable is your cloud infrastructure security?[Sponsored] When attacking digital wallets and SoftPOS mobile apps, threat actors target more than just data. Learn how to protect your digital wallets and SoftPOS apps.⚙️Infrastructure & DevOpsWe killed long-lived EC2 access and didn’t lose a minute of on-call response time. We were bleeding CloudWatch costs and had no idea who was hitting GetMetricData. This finally gave us the receipts.Built an HTTP1 server inside HTTP2, inside HTTP3, all for funIf SQL ever felt backwards to read, BigQuery’s pipe syntax fixes that. We needed rotating IPs and secure access for outbound traffic, This Squid on Fargate setup nailed it📦Kubernetes & Cloud NativeRunning as root without being root: Kubernetes finally gets user namespaces rightA clean way to mount read-only image data in KubernetesWhat’s new in CNCF Sandbox? 14 fresh projects shaping cloud native in 2024Securing the Kubernetes host operating systemNever Trust the Pod: Protecting Your Kubernetes Host the Right Way[Sponsored]Learn practical mobile app security tips to help mitigate attacks on SoftPOS and digital wallets.🔍 Observability & SREIf your ETL pipeline breaks and you’re still guessing where- read thisDon’t wait for your nodes to fail at 2AM- EKS can fix them for you nowWe ditched Prometheus for VictoriaMetrics: cut costs by 30%, and our slowest queries dropped from 30s to 3Making observability work: How platform engineers can improve monitoringSmall teams need PaaS-Ops, not DevOpsCheers,Shreyans SinghEditor-in-ChiefNew developer products provide a glimpse into the future of app building on HubSpot, including deeper extensibility, flexible UI, modern development tools, and moreHubSpot’s AI-powered ecosystem presents a global opportunity projected to reach $10.2 billion by 2028.To capitalize on that growth potential, we are opening our platform more, starting with expanded APIs, customizable app UI, and tools that better support a unified data stra`tegy.Start Building TodayForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Github Copilot Autofix: Secure code 3x faster CloudPro #61: How Figma Migrated onto K8s in Less Than 12 months ⭐Masterclass: From Docker Compose to Kubernetes Manifests A hard look at GuardDuty shortcomings Streamlining Keycloak in Kubernetes The hater’s guide to Kubernetes A skeptic's first contact with Kubernetes 🔍Secret Knowledge: Enhancing Bitnami Helm Charts Security Cloudflare adopted OpenTelemetry for logging pipeline Josh Grose on LinkedIn: I spent the last 3 yrs outside of observability Did you know the CNCF has an actual cookbook? Not metaphorically! Unfashionably secure: why we use isolated VMs ⚡Techwave: How Figma Migrated onto K8s in Less Than 12 months Github Copilot Autofix: Secure code 3x faster New Kubernetes CPUManager Static Policy: Distribute CPUs Across Cores Announcing mandatory multi-factor authentication for Azure sign-in GitHub scales on demand with Azure Functions 🛠️HackHub: Best Tools for the Cloud Web tool for database management The devs are over here at devzat, chat over SSH! CloudFormation_To_Terraform Debugging tool for Kubernetes which tests and displays connectivity between nodes in the cluster. Kubernetes network solution Cheers, Shreyans Singh Editor-in-Chief Forward to a Friend ⭐MasterClass: Tutorials & Guides From Docker Compose to Kubernetes Manifests This blog post provides a beginner-friendly guide for developers transitioning from Docker Compose to Kubernetes manifests, using Minikube for local Kubernetes development. It walks through setting up Minikube, deploying a sample application using Docker Compose, converting Docker Compose files into Kubernetes manifests with Kompose, and finally deploying the application on Kubernetes. The guide emphasizes practical steps, like generating and applying Kubernetes manifests, and validating deployments using the Minikube dashboard. A hard look at GuardDuty shortcomings AWS GuardDuty, while a cornerstone in cloud threat detection, isn't without its flaws. It offers good coverage and deep integration with AWS services, but its limitations in service support, detection latency, and cost can leave gaps in your security posture. Adversarial simulations and benchmarks reveal that GuardDuty can miss critical threats, and its detection can be slow, especially for high-impact, low-volume attacks like S3 ransomware. Streamlining Keycloak in Kubernetes In this blog post, the author, a DevOps Engineer at Tikal, shares how they automated the deployment and management of Keycloak, an open-source identity and access management solution, within a Kubernetes environment. By leveraging Kubernetes’ native capabilities, Helm, and Python, they streamlined the complex configuration process, which typically requires extensive manual adjustments. This approach not only ensures consistency and reduces manual efforts but also enables scalable and repeatable deployments. The hater’s guide to Kubernetes Kubernetes often gets a bad rap for being overly complex, especially for startups with small teams. Critics argue it’s over-engineering for tasks that don't need such a heavyweight solution. The key to avoiding its complexity is to use only the necessary features and ignore the rest. While Kubernetes isn’t for everyone, especially for those needing quick, ephemeral workloads, it's a solid choice if you need the robustness it offers and are careful in its application. A skeptic's first contact with Kubernetes The author’s first real exploration of Kubernetes revealed its core concepts like control loops, services, and workload management, which actually simplify and automate many tasks traditionally done manually. Kubernetes uses controllers to ensure that workloads meet desired states, services to manage network traffic efficiently, and storage management to handle data persistence across pods. While the system has some quirks and limitations, its approach to automating and scaling workloads has proven to be a valuable evolution in managing modern infrastructure. Quick Start Kubernetes The course prepares you to leverage Kubernetes for continuous development and deployment. Whether you're scaling applications to meet demand or ensuring seamless updates with minimal downtime, you'll be equipped with the skills necessary for efficient and effective Kubernetes management. This course is your gateway to becoming proficient in one of the most essential tools in the DevOps toolkit. 🔍Secret Knowledge: Learning Resources Related Titles Enhancing Bitnami Helm Charts Security Bitnami enhanced the security of its Helm charts using Kubescape, an open-source Kubernetes security tool that identifies misconfigurations by comparing configurations to industry best practices. By integrating Kubescape into their build pipelines, Bitnami made significant improvements such as eliminating group root dependencies, configuring immutable filesystems, and reducing misconfigured resources. Cloudflare adopted OpenTelemetry for logging pipeline Cloudflare recently transitioned its logging pipeline from syslog-ng to OpenTelemetry Collector to enhance performance, maintainability, and telemetry insights. This move allowed the team to leverage Go, a language more familiar to their engineers, and integrate better observability through Prometheus metrics. Despite challenges like minimizing downtime during the switch and ensuring compatibility with existing infrastructure, the migration has opened up opportunities for further improvements, such as better log sampling and migration to the OpenTelemetry Protocol (OTLP). Josh Grose on LinkedIn: I spent the last 3 yrs outside of observability Josh Grose (ex-Principal PM, Splunk), after three years away from the observability space, was surprised to find that despite companies spending around 30% of their cloud budgets on monitoring, reliability hasn't improved significantly. He observed that even when Service Level Agreements (SLAs) are met, it often comes at the cost of developer productivity and experience. Engineering leaders are frustrated with the high costs and limited improvements in key metrics like Mean Time to Recovery (MTTR) and development speed, leading to the perception that observability has become an expensive and ineffective necessity. Did you know the CNCF has an actual cookbook? Not metaphorically! The "Cloud Native Community Cookbook" is a unique collection of recipes put together by the CNCF and Equinix Metal, born out of the increased time people spent at home during the COVID-19 pandemic. Instead of focusing on cloud technologies, this cookbook brings together food recipes shared by members of the Cloud Native community, originally exchanged in Equinix Metal's Slack channel. Unfashionably secure: why we use isolated VMs While modern cloud architectures often favor shared, multi-tenant environments for efficiency and scalability, Thinkst Canary opts for a less trendy but highly secure approach by using isolated virtual machines (VMs) for each customer. This choice prioritizes security by ensuring that each customer's data and services are completely separated, reducing the risk of cross-customer data breaches. Although this method comes with higher operational costs and complexity, it provides a stronger security boundary, making it easier to manage risks and sleep better at night. ⚡TechWave: Cloud News & Analysis How Figma Migrated onto K8s in Less Than 12 months Figma completed its migration to Kubernetes in under a year by meticulously planning and executing a well-scoped transition. Initially running services on AWS's ECS, Figma faced limitations such as complex stateful workloads and limited auto-scaling. The decision to move to Kubernetes (EKS) was driven by its broader functionality, including support for StatefulSets, Helm charts, and advanced scaling options from the CNCF ecosystem. By Q1 2024, Figma had migrated most core services with minimal impact on users, resulting in enhanced reliability, reduced costs, and a more flexible compute platform. Github Copilot Autofix: Secure code 3x faster Copilot Autofix, now available in GitHub Advanced Security, is an AI-powered tool designed to help developers fix code vulnerabilities more than three times faster than manual methods. It analyzes vulnerabilities, explains their significance, and offers code suggestions for quick remediation. This accelerates the fixing process for both new vulnerabilities and existing security debt, significantly reducing the time and effort required for secure coding. Copilot Autofix is included by default for GHAS customers and also available for open source projects starting in September. New Kubernetes CPUManager Static Policy: Distribute CPUs Across Cores Kubernetes v1.31 introduces a new alpha feature called "distribute-cpus-across-cores" for the CPUManager's static policy. This option aims to enhance performance by spreading CPUs more evenly across physical cores, rather than clustering them on fewer cores. This reduces contention and resource sharing between CPUs on the same core, which can boost performance for CPU-intensive applications. To use this feature, users need to adjust their Kubernetes configuration to enable it. Currently, it cannot be combined with other CPUManager options, but future updates will address this limitation. Announcing mandatory multi-factor authentication for Azure sign-in Microsoft is making multi-factor authentication (MFA) mandatory for all Azure sign-ins to enhance security and protect against cyberattacks. Starting in the latter half of 2024, Azure users will need to use MFA to access the Azure portal and admin centers, with broader enforcement for other Azure tools like CLI and PowerShell set for early 2025. MFA, which adds an extra layer of security by requiring more than just a password, is shown to block over 99% of account compromises. GitHub scales on demand with Azure Functions GitHub faced scalability issues with its internal data pipeline, which struggled to handle the massive amount of data it collects daily. To address this, GitHub partnered with Microsoft to use Azure Functions' new Flex Consumption plan, which allows serverless functions to scale dynamically based on demand. This solution has enabled GitHub to efficiently process up to 1.6 million events per second, addressing their growth challenges and improving performance with minimal overhead. 🛠️HackHub: Best Tools for Cloud commandprompt/pgmanage PgManage is a modern graphical database client for PostgreSQL, focusing on management features and built on the now-dormant OmniDB project. quackduck/devzat Devzat is a chat service accessible via SSH that replaces the traditional shell prompt with a chat interface, allowing you to connect from any device with SSH capabilities. aperswal/CloudFormation_To_Terraform The CloudFormation to Terraform Converter is a tool that automates the migration of AWS CloudFormation templates to Terraform configuration files. bloomberg/goldpinger Goldpinger monitors Kubernetes networking by making calls between its instances and providing Prometheus metrics for visualization and alerts. ZTE/Knitter Knitter is a Kubernetes CNI plugin that supports multiple network interfaces for pods, allowing custom network configurations across various cloud environments. Buy now at $16.99 $10.99 Buy now at $39.99 $27.98 Buy now at $24.99 $16.99 📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us. If you have any comments or feedback, just reply back to this email. Thanks for reading and have a great day! *{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Datadog Acquires QuickwitCloudPro #78: Kubernetes health checks: Best practices for configuringCloud Conversations: A Fireside Chat with Forrest Brazeal and RubrikJoin us on Jan. 28th @ 10 AM PST for a captivating fireside chat where storytelling meets cloud innovation. Forrest Brazeal—acclaimed cloud architect, author, and the creative mind behind cloud computing's most beloved cartoons—teams up with Rubrik’s Chief Business Officer, Mike Tornincasa to explore the evolving challenges of data protection in a multi-cloud world.Save Your Spot⭐MasterclassKubernetes health checks: Best practices for configuringHow to manage secrets with Azure Key Vault in Kubernetes?Self-Hosting a Container RegistryHow I tuned my CI/CD pipeline to be done in 60 secondsWhat Karpenter v1.0.0 means for Kubernetes autoscaling🔍Secret KnowledgeFive Lessons from a Minor Production IncidentMaking a Postgres Compound Index 50x FasterSQLite Index VisualizationNetworking Costs CalculatorWriting secure Go code⚡TechwaveDatadog Acquires QuickwitAzure Storage—A look back and a look forwardOpenTelemetry and Grafana Labs: what’s new and what’s next in 2025Introducing Amazon Nova foundation models: Frontier intelligence and industry leading price performanceIntroducing the next generation of Amazon SageMaker: The center for all your data, analytics, and AI🛠️HackhubGoliat Dashboard: Manage, visualize, and optimize Terraform deploymentspv-migrate:CLI tool to easily migrate Kubernetes persistent volumesGit-remote-s3:Library that enables using Amazon S3 as a git remote and LFS serverToolGit:Git Productivity ToolkitDatabend: Modern alternative to SnowflakeCheers,Shreyans SinghEditor-in-ChiefWorld’s first 16 Hour LIVE Training to become an AI-Powered human in 2025 🤖The world of AI is evolving at lightning speed, and the only way to stay relevant is to MASTER AI before it masters you.Join the World’s first 2-Day Mastermind Challenge to learn the Tools, Tactics, and Strategies to Automate Your Work Like Never Before!Best part? It is usually for $395, but the first 100 of you get in for free.Claim your FREE spot now!⭐MasterClass: Tutorials & GuidesKubernetes health checks: Best practices for configuringKubernetes health checks are essential for maintaining the reliability, performance, and availability of applications. They use probes to monitor container health and take corrective actions when necessary. The three main types of probes—Liveness, Readiness, and Startup—serve distinct purposes. Liveness probes ensure the application is running and can restart containers in case of failure. Readiness probes determine if a container is ready to handle traffic, temporarily removing it from service if it fails. Startup probes focus on verifying successful initialization for slow-starting applications. Probes can use methods like HTTP, TCP, commands, or gRPC to perform health checks.How to manage secrets with Azure Key Vault in Kubernetes?To manage secrets with Azure Key Vault in Kubernetes, you can use tools like the External Secrets Operator (ESO) and a service principal for authentication. Start by creating an Azure Key Vault, adding your sensitive data (e.g., API tokens) as secrets, and assigning the required permissions to a service principal. Install ESO on your Kubernetes cluster to synchronize secrets from Azure Key Vault to Kubernetes secrets. Then, configure a SecretStore resource in Kubernetes to connect to the Key Vault, using the service principal credentials for authentication. With this setup, applications running in Kubernetes can securely access secrets from Azure Key Vault without exposing sensitive data.Self-Hosting a Container RegistryA self-hosted container registry allows you to store and manage container images on your own infrastructure, giving you full control and independence from third-party services. It involves setting up a server with Docker, configuring a container to run the registry, securing it with user authentication (e.g., via htpasswd), and enabling HTTPS using Nginx and SSL certificates. Once configured, you can push and pull images securely from your registry. While self-hosting ensures privacy and compliance with strict regulations, it requires maintaining and securing the system yourself, making it ideal for enterprises needing tight control over their containerized workflows.How I tuned my CI/CD pipeline to be done in 60 secondsThe process of optimizing my CI/CD pipeline to run in under 60 seconds involved strategic improvements in parallelization, caching, and job refinement. Initially, my pipeline was a simple setup that took over five minutes to execute, which hampered my productivity. I split the pipeline into multiple parallel jobs, grouped similar tasks to save cost and debug time, and leveraged GitHub's caching for dependencies, linting tools, and test data to drastically reduce redundant downloads and processing. By using a Makefile for local testing, I accelerated iterations and ensured the GitHub YAML was simple and reliable. Further tuning, like combining related jobs and adding task-specific cache keys, helped balance speed and cost. These optimizations allowed me to reduce the runtime for building, testing, linting, and deploying my Golang app to under a minute, making the pipeline more efficient and developer-friendly.What Karpenter v1.0.0 means for Kubernetes autoscalingKarpenter v1.0.0 marks a significant milestone for Kubernetes autoscaling, offering a mature and stable solution for dynamic node lifecycle management. As an open-source tool designed to optimize workload placement and reduce costs, Karpenter automatically provisions and deprovisions nodes based on application demands and Kubernetes scheduling constraints. With its vendor-neutral design and integration with cloud-specific APIs like AWS, Azure, and GCP, Karpenter enhances scalability, cost-efficiency, and ease of management across diverse cloud environments. The 1.0 release ensures API stability, supports features like workload consolidation and rolling updates for node images, and enables seamless integration with other CNCF tools, empowering organizations to build intelligent and scalable cloud-native infrastructure.🔍Secret Knowledge: Learning ResourcesFive Lessons from a Minor Production IncidentA minor production incident in the AWS News platform highlighted five key lessons about software operations. First, investing in observability early paid off, as comprehensive dashboards allowed for quick identification and resolution of the issue within an hour. Second, a robust software architecture and testing regime enabled safe and confident adjustments to the system during a crisis. Third, the YAGNI principle (You Aren't Gonna Need It) has trade-offs; while simpler designs work initially, anticipating growth with safeguards like alarms could prevent issues. Fourth, bugs often travel in pairs, as one problem often uncovers or triggers another, underscoring the need for thorough debugging processes. Lastly, data lineage simplifies troubleshooting, as stored intermediate data made it easy to pinpoint and fix the root causes. These lessons underscore the importance of building resilient systems even for small-scale projects.Making a Postgres Compound Index 50x FasterOptimizing a compound index reduced query latency by 50x, showcasing the importance of index field order in PostgreSQL. Initially, a query filtering by status and event_type, and sorting by occurred_at, was slow due to an index ordered by occurred_at first. This structure forced PostgreSQL to scan millions of rows inefficiently. By reordering the index to prioritize filter fields (status, event_type) before the sort field (occurred_at), the search space narrowed significantly, enabling PostgreSQL to process only relevant subsets. This simple yet impactful adjustment improved endpoint latency from ~500ms to under 10ms, highlighting how understanding index design can drastically enhance database performance.SQLite Index VisualizationSQLite uses a B-Tree structure to organize indexes, ensuring efficient data storage and quick searches. A B-Tree consists of nodes, with each node storing cells that contain the indexed data, a row ID, and links to child nodes. The data is saved on pages, which have fixed sizes, and every index is structured hierarchically for balance and fast lookups. Using tools like sqlite3_analyzer, we can inspect indexes and visualize their layout, which includes pages, cells, and relationships. For better understanding, visualizations can be created from index data dumps, showcasing how SQLite handles different types of indexes (e.g., ASC/DESC, multi-column, and unique indexes) and optimizations through commands like VACUUM or REINDEX. This approach makes it possible to compare index designs, analyze efficiency, and explore SQLite’s inner workings.Networking Costs CalculatorThe Networking Costs Calculator is a self-hosted tool designed to estimate AWS networking costs. It includes a serverless backend that fetches updated prices for networking services using AWS Price List Query APIs, storing them in a DynamoDB table, and a ReactJS frontend hosted on S3 and CloudFront for user interaction. Users can select an AWS region, specify services, and input data transfer details to view estimated monthly costs. Deployment requires a Linux OS, NodeJS, AWS CLI, and AWS CDK, with setup guided by a provided script. The tool helps users calculate costs for features like Data Transfer, NAT Gateways, and Transit Gateway Attachments.Writing secure Go codeWriting secure Go code involves following best practices to ensure that your code is robust, secure, and performs well. Key steps include staying informed about security updates by subscribing to the Go mailing list, keeping Go versions up to date for security patches, and regularly checking for vulnerabilities using tools like go vet, staticcheck, and golangci-lint. It's also important to test code for race conditions using Go’s built-in race detector and scan for known vulnerabilities with tools like govulncheck and gosec. Regular fuzz testing and keeping dependencies updated can help prevent security issues and improve the overall quality of your code.⚡TechWave: Cloud News & AnalysisDatadog Acquires QuickwitDatadog has acquired Quickwit, an open-source, cloud-native search engine designed for fast, scalable, and cost-effective log management. This acquisition will help Datadog address the needs of organizations in regulated industries, such as finance and healthcare, that must meet strict data residency, privacy, and regulatory requirements. By integrating Quickwit, Datadog aims to provide seamless observability and real-time insights without compromising data ownership or requiring multiple logging tools. Quickwit will continue to support its open-source community with a major update under the Apache License 2.Azure Storage—A look back and a look forwardAzure Storage has played a critical role in supporting AI advancements and cloud adoption in 2024, with innovations like Azure Blob Storage enabling large-scale AI model training and Azure Elastic SAN providing cloud-native SAN capabilities. Key highlights include rapid growth in Premium SSD v2 adoption, enhanced Kubernetes support through Azure Container Storage, and improved security measures like Microsoft Defender for Storage. Looking ahead to 2025, Azure Storage aims to empower businesses with smarter data solutions, including seamless integration of unstructured data with AI services, advanced disaster recovery options, and optimized storage for mission-critical workloads, all while collaborating with key partners to drive innovation.OpenTelemetry and Grafana Labs: what’s new and what’s next in 2025OpenTelemetry, a rapidly growing open-source observability project, achieved major milestones in 2024, including support for profiling, stability for the Spring Boot starter, and updates to Semantic Conventions for databases, AI, and more. Grafana Labs actively contributed to OpenTelemetry advancements, integrating it with Prometheus and introducing tools like Grafana Alloy and Beyla for enhanced compatibility and eBPF-based auto-instrumentation. Looking ahead to 2025, the OpenTelemetry Collector is expected to reach stability with its v1 release, signaling long-term support, while new innovations like expanded eBPF capabilities and enhanced protocol support aim to simplify trace-to-profile correlation and drive broader adoption across the observability ecosystem.Introducing Amazon Nova foundation models: Frontier intelligence and industry leading price performanceAmazon Nova is Amazon's latest suite of advanced foundation models available on Amazon Bedrock, designed for both text and multimodal (text, image, and video) tasks. With models tailored for understanding (like text analysis, document processing, and multimodal reasoning) and creative content generation (producing images and videos), Nova combines top-tier intelligence with cost efficiency. Models like Nova Micro, Lite, and Pro cater to diverse business needs, from fast, low-cost tasks to complex, high-accuracy workflows, and all support extensive customization for specific industries.Introducing the next generation of Amazon SageMaker: The center for all your data, analytics, and AIAmazon SageMaker has launched its next-generation platform, integrating tools for data exploration, analytics, machine learning (ML), and generative AI into a unified environment. The revamped platform features the SageMaker Unified Studio (preview), which consolidates data and AI workflows, enabling users to process data, develop ML models, and create generative AI applications seamlessly. It introduces key capabilities like the SageMaker Lakehouse for unified data access, a visual ETL tool for data transformation, and the Amazon Bedrock IDE for building advanced generative AI solutions.🛠️HackHub: Best Tools for CloudGoliat Dashboard:The Goliat Dashboard is an open-source project built with Astro that provides an interactive interface for managing Terraform Cloud resources. It integrates seamlessly with the Terraform Cloud API to display real-time metrics and organize projects and workspaces for better resource visibility. The dashboard also supports the DigitalOcean API and plans to add Azure, AWS, and OpenAI integrations for enhanced insights. With dynamic routes and automatic updates, no additional configuration is needed after API connections.pv-migrate:pv-migrate is a command-line tool and kubectl plugin designed to simplify the migration of Kubernetes PersistentVolumeClaim (PVC) data. It addresses challenges in renaming, resizing, or moving PVCs between namespaces, clusters, or cloud providers by securely transferring data using rsync over SSH. With support for in-cluster and cross-cluster migrations, customizable manifests, and multiple migration strategies, pv-migrate enables efficient and flexible volume data handling. It supports various architectures, including arm64 and amd64, and offers shell completions for popular terminals like bash and zsh.Git-remote-s3:git-remote-s3 is a Python-based tool that enables using Amazon S3 as a Git remote and Git LFS (Large File Storage) server. It provides a seamless way to manage Git repositories and LFS files directly on S3 buckets. Users can push, pull, and manage branches in their repositories stored on S3 while ensuring encryption for security. The tool also integrates with AWS services like CodePipeline by allowing zipped repository archives for pipeline source actions. It supports concurrent users, IAM-based access control, and debug logging, making it versatile for managing versioned code or assets on AWS.ToolGit:ToolGit is a productivity toolkit for Git that extends its functionality with various custom commands and aliases to simplify and automate common Git tasks. It includes utilities for cleaning up branches, force-pulling remote changes, restoring file modes, managing branch history, and more. Easy to install, ToolGit integrates seamlessly into your workflow by adding its scripts to your PATH environment variable, enabling them as Git sub-commands. Each command comes with detailed help text for user-friendly operation, making it a practical enhancement for developers seeking efficiency in version control.Databend:Databend is an open-source cloud data warehouse built in Rust, designed as a cost-effective alternative to Snowflake. It focuses on high-speed query execution and data ingestion, supporting complex analysis of large datasets. Databend offers features such as full ACID compliance, schema flexibility, advanced indexing, and real-time data updates. It can be deployed on both cloud and on-prem environments, providing enterprise-level performance with reduced costs.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

How Postgres stores data on diskCloudPro #75: A Guide to Kubernetes Network Policies⭐MasterclassA Guide to Kubernetes Network PoliciesDockerfile Instructions - ADD vs. COPYHow to add new worker node to existing Kubernetes clusterHow I Reduced Docker Image Size from 588 MB to Only 47.7 MBAmbient mesh: Can sidecar-less Istio make your application faster?🔍Secret KnowledgeOops, I Deleted the AWS Auth RolesRising Incidents on Git PlatformsHow Postgres stores data on diskHow We Integrate a New Service in Under 1 Hour for 25 ClustersEleventeen ways to delete an AWS resource🛠️HackhubApeman: AWS attack path management toolCyphernetes: A Kubernetes Query LanguageDesed:A command-line tool for complex sed scriptsKueue: Kubernetes-native Job QueueingAWS CloudFormation StarterkitCheers,Shreyans SinghEditor-in-Chief⭐MasterClass: Tutorials & GuidesA Guide to Kubernetes Network PoliciesIn Kubernetes, network policies control the traffic between pods, ensuring secure communication within the cluster. There are two main types: Layer 4 (L4) and Layer 7 (L7) policies. L4 policies manage traffic at the transport layer (e.g., TCP/UDP) based on IP addresses and ports, while L7 policies operate at the application layer (e.g., HTTP) with more fine-grained control over communication between services. L7 policies often require a service mesh like Linkerd, which adds features like mutual TLS (mTLS) for encrypted communication.Dockerfile Instructions - ADD vs. COPY`COPY` is simple and secure, only transferring files from the local build context to the image. In contrast, `ADD` offers extra functionality, such as downloading files from URLs or automatically extracting compressed archives. However, this added flexibility introduces complexity and potential security risks. Best practice recommends using `COPY` for most cases due to its straightforwardness, reserving `ADD` for situations where its unique features are necessary.How to add new worker node to existing Kubernetes clusterTo add a new worker node to an existing Kubernetes cluster, start by setting up a new Ubuntu 24.04 instance and configuring its hostname and `/etc/hosts` file. Disable swap memory, load necessary kernel modules, and install containerd as the container runtime. Add the Kubernetes APT repository, then install Kubernetes components like kubeadm, kubelet, and kubectl. On the control plane node, generate a kubeadm join command with a token. Run this command on the new worker node to join the cluster. Finally, verify the addition by checking the nodes from the control plane using `kubectl get nodes`.How I Reduced Docker Image Size from 588 MB to Only 47.7 MBTo significantly reduce a Docker image size, using multi-stage builds is key. In this case, a Flask app's image size was reduced from 588 MB to just 47.7 MB by switching to the lightweight Python 3.9-alpine image and using a multi-stage build approach. Multi-stage builds allow you to separate the build and runtime environments, keeping only essential runtime dependencies in the final image. Additionally, minimizing the number of layers by combining commands, using a `.dockerignore` file to exclude unnecessary files, and optimizing the Dockerfile structure contributed to this impressive 91.89% reduction.Ambient mesh: Can sidecar-less Istio make your application faster?Ambient mode in Istio, introduced in 2022, allows a sidecar-less architecture that can sometimes make applications faster. In traditional service meshes, adding latency is expected, but tests with ambient mode showed slightly improved performance in some cases, like the Bookinfo application's details service. This is partly because of more efficient connection handling and reduced syscalls in ambient mode, which offsets the overhead of extra hops via lightweight ztunnels.🔍Secret Knowledge: Learning ResourcesOops, I Deleted the AWS Auth RolesThe author, while managing an EKS (Elastic Kubernetes Service) cluster using Terraform, accidentally deleted the AWS authentication roles, which are crucial for accessing the cluster. This resulted in losing access to the EKS cluster. The fix involved manually restoring access by modifying the EKS API access configuration via the AWS Console, re-adding the necessary admin roles, and regenerating the `aws-auth` config map.Rising Incidents on Git PlatformsIn 2023, incidents affecting popular DevOps platforms like GitHub, Bitbucket, GitLab, and Jira increased, with issues such as RepoJacking, security vulnerabilities, and performance disruptions. GitHub saw a rise in attacks, with hackers exploiting vulnerabilities and hosting malware. Atlassian products like Bitbucket and Jira faced security flaws, with Jira experiencing a significant increase in incidents. GitLab suffered from performance issues and security breaches, including a major Proxyjacking attack.How Postgres stores data on diskPostgres stores data on disk in a well-organized, file-based structure within a directory, typically located at `/var/lib/postgresql/data`. Inside this directory, you'll find folders like `base/`, where actual database data for each database is stored, and `pg_wal/`, which holds the Write-Ahead Log (WAL) files that help recover data after crashes. Each table and database object is ultimately represented by files in these directories. PostgreSQL uses clever abstractions to manage data, such as snapshots for transactions, dynamic shared memory for handling multiple processes, and special mechanisms like tablespaces for physically separating certain data.How We Integrate a New Service in Under 1 Hour for 25 ClustersThe article describes how a team integrated a new service called Otterize across 25 clusters in under an hour, emphasizing that while the technical setup was quick, the lengthy licensing process took over four months. The integration involved automating several steps using GitOps and tools like Argo CD to avoid manual errors. Key tasks included creating an organization and environment, inviting users, integrating with Kubernetes, securely managing credentials, and deploying the setup through a script.Eleventeen ways to delete an AWS resourceOur goal is to reduce AWS costs, but the deletion methods vary widely, often leaving users frustrated. They categorize deletion patterns, from simple one-click deletes to more complex confirmations that require typing specific phrases or acknowledging consequences. Ultimately, AWS should standardize its deletion processes to improve user experience and security, and they call for more data on user behavior during these actions.🛠️HackHub: Best Tools for CloudApeman: AWS attack path management toolProject Apeman is an AWS attack path management tool that helps analyze and manage AWS security data. To set it up, you need Docker, Python, and a virtual environment. Once the system is initialized, Apeman gathers AWS account data, including authorization details and ARNs, which are then ingested into a graph database for analysis.Cyphernetes: A Kubernetes Query LanguageCyphernetes is a Cypher-inspired query language for Kubernetes, simplifying complex Kubernetes operations with intuitive, SQL-like queries. It allows developers to easily manage Kubernetes resources by expressing relationships between them, such as connecting deployments to services and ingresses.Desed:A command-line tool for complex sed scriptsDesed is a command-line tool designed to help debug and understand complex `sed` scripts. It allows users to step through their scripts, both forwards and backwards, preview how substitute commands will affect the pattern space, and set breakpoints to examine the program's state. Desed also supports hot reloading, so changes to the source code can be instantly applied without restarting the debugger.Kueue: Kubernetes-native Job QueueingKueue is a Kubernetes-native job queueing system that manages when jobs start and stop based on a variety of factors, such as priorities and resource availability. It offers features like job management with FIFO strategies, resource fair sharing, dynamic resource reclaim, and integration with popular job types like BatchJob and Kubeflow training jobs.AWS CloudFormation StarterkitAn AWS CloudFormation starterkit including CI/CD and dev tools that allow you to securely and quickly deploy CloudFormation stacks on your AWS account.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

This Guide Helped Us Actually Save Money With Kubernetes Rightsizing- Start HereCloudPro #91: This Update Makes Kubernetes Smarter About Volume Limits Real stories from real engineers. How they blew their Azure budget on networking, what actually worked for IAM trust policies, how they enforced container image standards without killing velocity.That’s what this issue is about.🔐 Cloud SecurityThe One Thing You Should Never Skip When Accepting Public File UploadsHow to Keep IAM Trust Working. Without Breaking on Role RecreationNo One Tells You This About Building Cybersecurity Products: You Become the TargetWhat a Real Secure GenAI Deployment Looks Like on AWSThe One Thing You Should Never Do When Building a Platform: Rely on Tickets, Raw Cloud, or Rigid Templates⚙️ Infrastructure & DevOpsHow AWS Fixed the Hardest Part of Data Lake ManagementHow We Enforced Image Standards in ECS Without Slowing Down DevsWe Blew Our Azure Budget on Networking- Here’s the Guide I Wish We HadHow a Simple Misconfig Let Us Pull Images from Any S3 Bucket. No CVE NeededIf You’re Still Writing IaC by Hand, Read This Firstgit-whois a command-line tool for answering that eternal question: Who wrote this code?📦 Kubernetes & Cloud NativeThis Update Makes Kubernetes Smarter About Volume LimitsIf You’re Still Treating Sidecars Like Init Containers, You’re Doing It WrongThis Guide Helped Us Actually Save Money With Kubernetes Rightsizing- Start HereWhat a Real-World Ingress Setup Looks Like on AWS EKSDocker Engine v28: Hardening container networking by default🔍 Observability & SREGrafana 12 release: observability as code, dynamic dashboards, new Grafana Alerting tools, and more | Grafana LabsCloudWatch Dashboard (Over)Sharing | WithSecure™ LabsPrezi's Journey from Prometheus to VictoriaMetrics - InfoQOpenTelemetry: A Guide to Observability with Go | Blog | Luca CavallinKHIis a rich log visualization tool for Kubernetes clusters. KHI transforms vast quantities of logs into an interactive, comprehensive timeline view.Cheers,Shreyans SinghEditor-in-ChiefNew developer products provide a glimpse into the future of app building on HubSpot, including deeper extensibility, flexible UI, modern development tools, and moreHubSpot’s AI-powered ecosystem presents a global opportunity projected to reach $10.2 billion by 2028.To capitalize on that growth potential, we are opening our platform more, starting with expanded APIs, customizable app UI, and tools that better support a unified data strategy.Start Building TodayForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Google Kubernetes Engine supports 65,000-node clustersCloudPro #76: Kubernetes Gateway API v1.2: WebSockets, Timeouts, Retries, and MoreMastering Software Deployments at the Edge: A User’s Guide to Diverting DisasterSoftware delivery to dedicated edge devices is one of the most complex challenges faced by IT professionals today. While edge deployments come with inherent complications, it’s possible to avoid the pitfalls. With this guide in hand, a little planning, and the right tools and strategies in place, you can be confident you’ll never push a faulty update at scale.Read the Guide⭐MasterclassI have asked this SSH question in every AWS interviewHow to Ace (CKAD) Certified Kubernetes Application Developer examKubernetes resource model, controller pattern and operator SDK refresherHow we avoided an outage caused by running out of IPs in EKSDeploying a Serverless REST API🔍Secret KnowledgeHow to Differentiate Manual Changes from Terraform Changes in S3 BucketManaging AWS EKS access entries with Terraform and OpenTofuUber’s billion trips migration setup30 Days of CNCF projects | Day 5: What is Crossplane + demoCI/CD automation with Tekton: GitHub PR pipeline guide⚡TechwaveKubernetes Gateway API v1.2: WebSockets, Timeouts, Retries, and MoreGoogle Kubernetes Engine supports 65,000-node clustersContainer Insights with enhanced observability now available in Amazon ECSNew Amazon S3 Tables: Storage optimized for analytics workloadsGrafana 11.4 release: Introducing support for OpenSearch PPL and OpenSearch SQL in the AWS CloudWatch data source plugin🛠️HackhubNovascans your cluster for installed Helm charts and then cross-checks them against all known Helm repositories.Pglite-fusionallows you to embed an SQLite database in your PostgreSQL table. AKA multitenancy has been solved.Drasiis a data processing platform that simplifies detecting changes in data and taking immediate action.SonarIACis a static code analyzer for Infrastructure-as-Code languages such as CloudFormation and Terraform, as well as DevOps like Docker and Kubernetes.Pg_flois a CLI to move and transform data between PostgreSQL databases using Logical Replication.Cheers,Shreyans SinghEditor-in-ChiefLearn Million Dollar AI Strategies & Tools in this 3 hour AI Training for Free.This 3 hour power packed workshop that will teach you 30+ AI Tools, make you a master of prompting & talk about hacks, strategies & secrets that only the top 1% know of.By the way, here’s sneak peek into what’s inside the training:-Making money using AI-The latest AI developments, like GPT o1-Creating an AI clone of yourself, that functions exactly like YOU-10 BRAND new AI tools to automate your work & cut work time by 50%1.5 Million people are already RAVING about this hands-on Training on AI Tools. Don’t take our word for it? Attend for yourself and see.Register here⭐MasterClass: Tutorials & GuidesI have asked this SSH question in every AWS interviewIn AWS interviews, a popular and insightful question is: "You're trying to SSH into an EC2 instance, but it’s failing. How would you troubleshoot?" While it seems simple, the question evaluates a candidate's problem-solving approach, understanding of AWS infrastructure, and real-world experience. A key expectation is that candidates check security groups first since they act as firewalls controlling traffic. Surprisingly, many overlook this basic yet crucial step, diving into more complex areas instead. The question thus highlights how well candidates understand AWS fundamentals and prioritize troubleshooting steps effectively.How to Ace (CKAD) Certified Kubernetes Application Developer examThe Certified Kubernetes Application Developer (CKAD) exam is a practical certification focused on Kubernetes application deployment, maintenance, and troubleshooting. Ideal for engineers managing containerized applications in Kubernetes, it tests real-world problem-solving skills across topics like application design, deployment strategies, observability, security, and networking. The exam includes hands-on tasks performed in a live Kubernetes cluster and allows access to documentation during the test. It's considered pre-professional in difficulty, with a 66% passing score and retake opportunities. Preparation involves mastering Kubernetes CLI commands, understanding concepts like pods, deployments, and ConfigMaps, and practicing with tools like Killer.sh to simulate the exam experience.Kubernetes resource model, controller pattern and operator SDK refresher] The Resource Model uses etcd as the state store, with resources defined by objects like Kind, Group, Version, and Resource, which are mapped to API endpoints (e.g., /apis/apps/v1/deployments). Informers and SharedInformers optimize resource management by efficiently watching changes in objects, reducing API server load. Informers utilize Reflectors to fetch and cache data, Listers to retrieve objects from the cache, and Workqueues to process events like Add, Update, or Delete. Controllers act as loops that continuously reconcile the current state of resources (from their status) with the desired state (defined in their spec).How we avoided an outage caused by running out of IPs in EKSAdevinta's platform team tackled the critical issue of IP exhaustion in their EKS clusters by implementing custom networking with a secondary CIDR to allocate additional IPs, avoiding potential outages. The problem stemmed from the VPC-CNI plugin's default behavior of assigning an IP address per pod, which strained available IPs in their VPC as clusters scaled. While alternatives like switching to Cilium or enabling IPv6 were explored, the chosen solution balanced speed and reliability, enabling the team to complete their migration to EKS. By carefully testing and rolling out custom networking, the team stabilized IP usage, avoided service disruptions, and ensured seamless scaling for their multi-tenant cluster architecture.Deploying a Serverless REST APIThis guide walks you through deploying a REST API using AWS services like API Gateway, Lambda, DynamoDB, and Cognito with Terraform. The project involves creating an API that allows users to manage a list of Sicilian dishes. It starts with configuring AWS as the provider and setting up an S3 bucket to store Terraform state files. You then create an IAM role with the necessary permissions for Lambda to interact with DynamoDB. The Lambda function itself is written in Python, with methods to handle CRUD operations on the DynamoDB table based on the incoming HTTP requests. Authentication is added via Amazon Cognito to secure write operations. Finally, the API routes (GET, POST, PATCH, DELETE) are implemented to handle the dish data, including a recursive scan function to fetch all dishes from the table.🔍Secret Knowledge: Learning ResourcesHow to Differentiate Manual Changes from Terraform Changes in S3 BucketTo differentiate manual changes from Terraform changes in an S3 bucket managed by Terraform, you can use AWS CloudTrail, EventBridge, Lambda, and SNS notifications. CloudTrail logs all S3 API actions, including manual and automated changes. EventBridge filters these logs for specific events (e.g., uploads or deletions) and triggers a Lambda function. The Lambda function processes the events to exclude actions initiated by Terraform (using the IAM role or userIdentity details associated with Terraform). It sends SNS notifications only for manual changes, ensuring Terraform modifications do not trigger alerts.Managing AWS EKS access entries with Terraform and OpenTofuManaging AWS EKS access entries with Terraform and OpenTofu simplifies authentication and authorization for Kubernetes clusters by replacing the outdated aws-auth ConfigMap with a more scalable and robust EKS API. Access entries allow direct API-based management of IAM users, roles, and predefined policies, eliminating manual ConfigMap updates prone to errors. With tools like Terraform and OpenTofu, you can define access entries as Infrastructure as Code (IaC), enabling automated and secure access control at scale. This method seamlessly integrates AWS IAM for authentication with Kubernetes RBAC for authorization.Uber’s billion trips migration setupUber successfully migrated its complex trip fulfillment infrastructure to a hybrid cloud environment without downtime by employing innovative strategies. To ensure uninterrupted service for millions of global users, Uber implemented a backward compatibility layer, maintaining support for existing APIs during the transition. They also used shadow validation, mirroring requests and comparing responses between old and new systems to identify discrepancies.30 Days of CNCF projects | Day 5: What is Crossplane + demoCI/CD automation with Tekton: GitHub PR pipeline guideAutomating CI/CD with Tekton involves leveraging its Kubernetes-native framework to define pipelines for building, testing, and deploying code. Using tools like Minikube, kubectl, and Ngrok, you can set up a local Tekton environment and integrate pre-built tasks such as git-clone and kaniko for cloning repositories and building Docker images. To automate workflows triggered by GitHub pull requests, Tekton Triggers can be configured to listen for webhooks, validate events, and execute pipelines, ensuring CI/CD tasks like building, testing, and updating GitHub statuses happen seamlessly.⚡TechWave:CloudNews & AnalysisKubernetes Gateway API v1.2: WebSockets, Timeouts, Retries, and MoreGateway API v1.2 introduces significant updates and improvements to Kubernetes networking, focusing on WebSocket support, HTTPRoute timeouts, retries, and more robust infrastructure annotations. It graduates several features, like HTTPRoute timeouts and backend protocol support, to the stable Standard channel, ensuring better resilience and backward compatibility. Notably, the release brings two breaking changes: the removal of outdated v1alpha2 versions for GRPCRoute and ReferenceGrant, and a shift in .status.supportedFeatures structure for greater future flexibility.Google Kubernetes Engine supports 65,000-node clustersGoogle Kubernetes Engine (GKE) now supports clusters of up to 65,000 nodes, a scale designed to meet the growing computational demands of massive AI workloads, including training and serving trillion-parameter AI models. This enhancement allows for faster training times, larger model scalability, and flexibility in resource allocation for diverse tasks. GKE achieves this through innovations like transitioning to a Spanner-based key-value store for enhanced reliability and a revamped control plane for faster scaling and operations.Container Insights with enhanced observability now available in Amazon ECSAmazon ECS now features enhanced observability with Container Insights, helping users monitor and troubleshoot container workloads more effectively. This capability offers detailed metrics, logs, and visual dashboards to quickly identify root causes of issues, reduce detection and repair times, and improve application performance. It supports granular resource monitoring, proactive issue management, cross-account observability, and seamless integration with CloudWatch services like Application Signals and Logs.New Amazon S3 Tables: Storage optimized for analytics workloadsAmazon S3 Tables are a new storage option optimized for analytics workloads, supporting tabular data in Apache Iceberg format. This managed service provides faster query performance (up to 3x) and handles higher transactions per second (up to 10x) compared to self-managed storage. S3 Tables integrate seamlessly with query engines like Amazon Athena and Apache Spark and include features like automatic maintenance (e.g., compaction, snapshot management) and logical grouping with namespaces.Grafana 11.4 release: Introducing support for OpenSearch PPL and OpenSearch SQL in the AWS CloudWatch data source pluginGrafana 11.4 now supports OpenSearch Piped Processing Language (PPL) and OpenSearch SQL in its AWS CloudWatch data source plugin, allowing AWS users more flexibility in querying CloudWatch Logs. These new query options join the existing Logs Insights QL, enabling users to filter and aggregate logs using their preferred language without duplicating data. Features like syntax highlighting, live code completion, and sample queries enhance usability.🛠️HackHub: Best Tools for CloudNovascans your cluster for installed Helm charts and then cross-checks them against all known Helm repositories.Pglite-fusionallows you to embed an SQLite database in your PostgreSQL table. AKA multitenancy has been solved.Drasiis a data processing platform that simplifies detecting changes in data and taking immediate action.SonarIACis a static code analyzer for Infrastructure-as-Code languages such as CloudFormation and Terraform, as well as DevOps like Docker and Kubernetes.Pg_flois a CLI to move and transform data between PostgreSQL databases using Logical Replication.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

European grocery store becomes cloud services providerCloudPro #65: IBM acquires Kubecost[Sponsored] Use AI to 10X your productivity & efficiency at work with AI (free bonus)Save your free spot here (seats are filling fast!) ⏰⭐MasterclassA Guide to Kubernetes Network PoliciesDockerfile Instructions - ADD vs. COPYHow to add new worker node to existing Kubernetes clusterHow I Reduced Docker Image Size from 588 MB to Only 47.7 MBAmbient mesh: Can sidecar-less Istio make your application faster?🔍Secret KnowledgeOops, I Deleted the AWS Auth RolesRising Incidents on Git PlatformsHow Postgres stores data on diskHow We Integrate a New Service in Under 1 Hour for 25 ClustersEleventeen ways to delete an AWS resource⚡TechwaveEuropean grocery store becomes cloud services providerIBM acquires KubecostIntroducing Pulumi Insights 2.0Linus Torvalds advises open-source developers to pursue meaningful projects, not hypeJFrog Extends GitHub Alliance to Provide Unified Dashboard🛠️HackhubApeman: AWS attack path management toolCyphernetes: A Kubernetes Query LanguageDesed:A command-line tool for complex sed scriptsKueue: Kubernetes-native Job QueueingAWS CloudFormation Starterkit💡Recommended Reading: Implementing GitOps with KubernetesCheers,Shreyans SinghEditor-in-ChiefJoin Roman Lavrik from Deloitte Snyk hosted DevSecCon 2024Snyk is thrilled to announce DevSecCon 2024, Developing AI Trust Oct 8-9, a FREE virtual summit designed for DevOps, developer and security pros of all levels.Join Roman Lavrik from Deloitte, among many others, and learn some presciptive DevSecOps methods for AI-powered development.Save your spot⭐MasterClass: Tutorials & GuidesA Guide to Kubernetes Network PoliciesIn Kubernetes, network policies control the traffic between pods, ensuring secure communication within the cluster. There are two main types: Layer 4 (L4) and Layer 7 (L7) policies. L4 policies manage traffic at the transport layer (e.g., TCP/UDP) based on IP addresses and ports, while L7 policies operate at the application layer (e.g., HTTP) with more fine-grained control over communication between services. L7 policies often require a service mesh like Linkerd, which adds features like mutual TLS (mTLS) for encrypted communication.Dockerfile Instructions - ADD vs. COPY`COPY` is simple and secure, only transferring files from the local build context to the image. In contrast, `ADD` offers extra functionality, such as downloading files from URLs or automatically extracting compressed archives. However, this added flexibility introduces complexity and potential security risks. Best practice recommends using `COPY` for most cases due to its straightforwardness, reserving `ADD` for situations where its unique features are necessary.How to add new worker node to existing Kubernetes clusterTo add a new worker node to an existing Kubernetes cluster, start by setting up a new Ubuntu 24.04 instance and configuring its hostname and `/etc/hosts` file. Disable swap memory, load necessary kernel modules, and install containerd as the container runtime. Add the Kubernetes APT repository, then install Kubernetes components like kubeadm, kubelet, and kubectl. On the control plane node, generate a kubeadm join command with a token. Run this command on the new worker node to join the cluster. Finally, verify the addition by checking the nodes from the control plane using `kubectl get nodes`.How I Reduced Docker Image Size from 588 MB to Only 47.7 MBTo significantly reduce a Docker image size, using multi-stage builds is key. In this case, a Flask app's image size was reduced from 588 MB to just 47.7 MB by switching to the lightweight Python 3.9-alpine image and using a multi-stage build approach. Multi-stage builds allow you to separate the build and runtime environments, keeping only essential runtime dependencies in the final image. Additionally, minimizing the number of layers by combining commands, using a `.dockerignore` file to exclude unnecessary files, and optimizing the Dockerfile structure contributed to this impressive 91.89% reduction.Ambient mesh: Can sidecar-less Istio make your application faster?Ambient mode in Istio, introduced in 2022, allows a sidecar-less architecture that can sometimes make applications faster. In traditional service meshes, adding latency is expected, but tests with ambient mode showed slightly improved performance in some cases, like the Bookinfo application's details service. This is partly because of more efficient connection handling and reduced syscalls in ambient mode, which offsets the overhead of extra hops via lightweight ztunnels.🔍Secret Knowledge: Learning ResourcesOops, I Deleted the AWS Auth RolesThe author, while managing an EKS (Elastic Kubernetes Service) cluster using Terraform, accidentally deleted the AWS authentication roles, which are crucial for accessing the cluster. This resulted in losing access to the EKS cluster. The fix involved manually restoring access by modifying the EKS API access configuration via the AWS Console, re-adding the necessary admin roles, and regenerating the `aws-auth` config map.Rising Incidents on Git PlatformsIn 2023, incidents affecting popular DevOps platforms like GitHub, Bitbucket, GitLab, and Jira increased, with issues such as RepoJacking, security vulnerabilities, and performance disruptions. GitHub saw a rise in attacks, with hackers exploiting vulnerabilities and hosting malware. Atlassian products like Bitbucket and Jira faced security flaws, with Jira experiencing a significant increase in incidents. GitLab suffered from performance issues and security breaches, including a major Proxyjacking attack.How Postgres stores data on diskPostgres stores data on disk in a well-organized, file-based structure within a directory, typically located at `/var/lib/postgresql/data`. Inside this directory, you'll find folders like `base/`, where actual database data for each database is stored, and `pg_wal/`, which holds the Write-Ahead Log (WAL) files that help recover data after crashes. Each table and database object is ultimately represented by files in these directories. PostgreSQL uses clever abstractions to manage data, such as snapshots for transactions, dynamic shared memory for handling multiple processes, and special mechanisms like tablespaces for physically separating certain data.How We Integrate a New Service in Under 1 Hour for 25 ClustersThe article describes how a team integrated a new service called Otterize across 25 clusters in under an hour, emphasizing that while the technical setup was quick, the lengthy licensing process took over four months. The integration involved automating several steps using GitOps and tools like Argo CD to avoid manual errors. Key tasks included creating an organization and environment, inviting users, integrating with Kubernetes, securely managing credentials, and deploying the setup through a script.Eleventeen ways to delete an AWS resourceOur goal is to reduce AWS costs, but the deletion methods vary widely, often leaving users frustrated. They categorize deletion patterns, from simple one-click deletes to more complex confirmations that require typing specific phrases or acknowledging consequences. Ultimately, AWS should standardize its deletion processes to improve user experience and security, and they call for more data on user behavior during these actions.⚡TechWave: Cloud News & AnalysisEuropean grocery store becomes cloud services providerLidl, through its parent company Schwarz Group, unintentionally entered the competitive world of cloud computing when it built its own cloud system in 2021 to meet internal needs. As other German businesses sought alternatives to U.S. and Chinese cloud providers, Schwarz Group recognized a demand for data services with a focus on European data privacy standards. This led to the creation of Schwarz Digits, which now provides cloud and cybersecurity services, attracting major clients like SAP and Bayern Munich. While competing with giants like Amazon and Google, Schwarz Digits differentiates itself with a focus on digital sovereignty and data protection.IBM acquires KubecostIBM has acquired Kubecost, a startup that helps companies optimize and monitor their Kubernetes clusters for cost efficiency. Kubecost, known for its widely adopted Kubernetes cost management tool and its open-source project OpenCost, will enhance IBM’s FinOps capabilities. Kubecost will likely be integrated into IBM's FinOps Suite and potentially its OpenShift platform.Introducing Pulumi Insights 2.0Pulumi Insights 2.0 expands beyond just Pulumi-managed infrastructure to provide visibility into all cloud resources, offering powerful tools for assessing security, efficiency, and management. It introduces new features like comprehensive infrastructure scanning, visual explorers, and dashboards to help organizations manage their cloud environments more effectively. Insights 2.0 integrates with Pulumi’s Infrastructure-as-Code (IaC) tools, making it easier to bring unmanaged infrastructure under IaC.Linus Torvalds advises open-source developers to pursue meaningful projects, not hypeAt the Open Source Summit Europe, Linus Torvalds encouraged open-source developers to focus on meaningful projects rather than chasing trends and hype. While discussing the latest Linux kernel updates, he emphasized that progress in Linux remains steady, even if not always exciting, with a focus on reliability. Torvalds also praised the ongoing evolution of Linux and the wider open-source ecosystem, noting its democratizing effect for new developers.JFrog Extends GitHub Alliance to Provide Unified DashboardJFrog and GitHub have expanded their partnership to provide developers with a unified platform for better security and productivity. This integration offers a consolidated view of project statuses and security through tools like GitHub's Copilot chat and JFrog’s Advanced Security features. Developers can now get insights on third-party packages, track vulnerabilities earlier, and navigate between code and the binaries it produces seamlessly.🛠️HackHub: Best Tools for CloudApeman: AWS attack path management toolProject Apeman is an AWS attack path management tool that helps analyze and manage AWS security data. To set it up, you need Docker, Python, and a virtual environment. Once the system is initialized, Apeman gathers AWS account data, including authorization details and ARNs, which are then ingested into a graph database for analysis.Cyphernetes: A Kubernetes Query LanguageCyphernetes is a Cypher-inspired query language for Kubernetes, simplifying complex Kubernetes operations with intuitive, SQL-like queries. It allows developers to easily manage Kubernetes resources by expressing relationships between them, such as connecting deployments to services and ingresses.Desed:A command-line tool for complex sed scriptsDesed is a command-line tool designed to help debug and understand complex `sed` scripts. It allows users to step through their scripts, both forwards and backwards, preview how substitute commands will affect the pattern space, and set breakpoints to examine the program's state. Desed also supports hot reloading, so changes to the source code can be instantly applied without restarting the debugger.Kueue: Kubernetes-native Job QueueingKueue is a Kubernetes-native job queueing system that manages when jobs start and stop based on a variety of factors, such as priorities and resource availability. It offers features like job management with FIFO strategies, resource fair sharing, dynamic resource reclaim, and integration with popular job types like BatchJob and Kubeflow training jobs.AWS CloudFormation StarterkitAn AWS CloudFormation starterkit including CI/CD and dev tools that allow you to securely and quickly deploy CloudFormation stacks on your AWS account.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Elasticsearch is Open Source, AgainCloudPro #64: Introducing OpenAI o1Hack the Cybersecurity InterviewPrepare for cybersecurity job interviews across various roles, from entry-level to expert positions.It covers topics like answering technical and behavioral questions, understanding different cybersecurity roles and developing important soft skills.It includes information on salaries, career paths, and how to find jobs in tough markets.Get It for $35.99 $24.99⭐MasterclassDeploying Databases in K8s with PostgreSQL, CloudNative-PG, and Ceph Rook on Amazon EKSKubernetes and Access Management API, the new authentication in EKSHow Netflix solved the issue with Java 21 virtual threadsDoes Talos Kubernetes and Omni live up to the hype?Understand your Kubernetes cost drivers and the best ways to rein in spending🔍Secret KnowledgeHack the Cybersecurity InterviewTroubleshooting: Terminal LagMonitor these Kubernetes signals to help rightsize your fleetGetting Started with Cilium Service Mesh on Amazon EKSHow AppsFlyer migrated from Kafka to Kubernetes using Karpenter⚡TechwaveIntroducing OpenAI o1Elasticsearch is Open Source, AgainOracle to offer 131,072 Nvidia Blackwell GPUs via its cloudWhy eBPF is critical and how it’s getting betterJuniper adds AI cloud services to its Apstra data center software🛠️HackhubHigh-performance server for NATS.ioA collection of Bash One-Liners and terminal tricksdistributed key value NoSQL database that uses RocksDB as storage engineBuild, Share and Run Both Your Kubernetes Cluster and Distributed ApplicationsRun your deep learning workloads on Kubernetes more easily and efficientlyCheers,Shreyans SinghEditor-in-ChiefForward to a Friend⭐MasterClass: Tutorials & GuidesDeploying Databases in K8s with PostgreSQL, CloudNative-PG, and Ceph Rook on Amazon EKSThis article explains how to deploy and manage a PostgreSQL database on Kubernetes using Amazon EKS. It combines CloudNativePG, a PostgreSQL operator, with Ceph Rook, a storage orchestrator, to ensure data persistence and high availability in a Kubernetes environment. A step-by-step guide is provided for deploying and configuring these tools, including using Helm charts, setting up storage with Ceph, and monitoring the database cluster.Kubernetes and Access Management API, the new authentication in EKSIn AWS EKS (Elastic Kubernetes Service), the new authentication and access management system simplifies how users and worker nodes access Kubernetes clusters. Previously, access was managed using an "aws-auth" ConfigMap, which could be complex and prone to errors. Now, AWS introduces the EKS Access Management API, allowing users to authenticate through AWS IAM directly and authorize Kubernetes actions via Kubernetes RBAC. This approach removes the need for managing ConfigMaps manually, offers predefined EKS Access Policies, and enhances security by eliminating hidden root users. Additionally, integration with Terraform makes access control easier to implement and manage.How Netflix solved the issue with Java 21 virtual threadsIn Java 21, Netflix encountered an issue with virtual threads, which are lightweight threads designed to improve concurrency by suspending and resuming automatically. The problem involved some Netflix services using Java 21, Spring Boot 3, and Tomcat, where servers stopped processing requests due to sockets stuck in a `CLOSE_WAIT` state. Virtual threads were getting "pinned" to operating system threads while waiting to acquire locks. Since all OS threads became blocked, Tomcat couldn't process incoming requests, causing the system to hang. The underlying issue was traced to locking mechanisms in virtual threads, leading to thread contention and blocked OS threads.Does Talos Kubernetes and Omni live up to the hype?Talos Kubernetes and Sidero Omni live up to the hype by providing an intuitive and efficient way to set up and manage Kubernetes clusters. With Omni, you can easily create a Talos cluster without needing to access your virtual machines directly, making the process more streamlined. Setting up clusters, scaling nodes, and even upgrading Kubernetes versions are straightforward, with minimal manual intervention required. While there are some areas for improvement, like simplifying static IP configuration, the overall experience is highly positive.Understand your Kubernetes cost drivers and the best ways to rein in spendingTo reduce Kubernetes-related costs, it's important to monitor key cost drivers such as CPU, memory, storage, and networking. Costs are driven by resource usage and the rate at which they are consumed, so reducing unnecessary usage and optimizing resource allocation is key. Over-provisioning, idle resources, and inefficient scaling are common cost culprits. Regularly adjusting resource requests, leveraging auto-scaling tools like Horizontal Pod Autoscaler, and monitoring metrics with tools like Grafana and Prometheus can help optimize usage.🔍Secret Knowledge: Learning ResourcesHack the Cybersecurity Interview"Hack the Cybersecurity Interview, Second Edition" is a comprehensive guide designed to help individuals prepare for interviews across a wide range of cybersecurity roles. The book covers technical and behavioral interview questions for positions like cybersecurity engineer, penetration tester, and CISO, while also offering tips on personal branding, stress management, and negotiation. It provides real-world advice and industry insights, making it an essential resource for anyone looking to succeed in the competitive field of cybersecurity.Troubleshooting: Terminal LagIn this troubleshooting session, Tavis Ormandy investigates why launching the xterm terminal on his Windows machine is significantly slower compared to Fedora. He identifies that Windows applies an animation effect that delays interaction with the terminal. Through a series of tests and debugging, he discovers that the X server software (X410) adds unnecessary animation effects, which can’t be disabled directly. He uses a debugger to bypass the issue, improving the performance slightly. After further optimizations with features and caching processes, he brings the Windows terminal's performance closer to Fedora’s, significantly reducing the lag.Monitor these Kubernetes signals to help rightsize your fleetTo ensure your Kubernetes environment is both cost-efficient and sustainable, it's crucial to monitor signals like CPU, memory, disk I/O, and network utilization. Over-provisioning leads to wasted resources and high costs, while under-provisioning can degrade performance. Watch for indicators such as high resource usage, slow application performance, or low utilization to fine-tune your setup. Tools like Prometheus and Grafana, along with autoscaling, can help you dynamically adjust resources, ensuring optimal balance, reduced costs, and improved sustainability.Getting Started with Cilium Service Mesh on Amazon EKSThe blog post explains how Cilium, an open-source networking and security solution powered by eBPF, enhances network connectivity between workloads in Amazon EKS (Elastic Kubernetes Service). Cilium provides advanced networking, load balancing, encryption, and observability without the need for sidecar proxies. It integrates seamlessly with Amazon EKS to improve microservice communication, multi-cluster networking, and network policy enforcement. Cilium Service Mesh, built into Cilium, leverages eBPF and Envoy to offer high performance and low overhead for traffic management, security, and monitoring.How AppsFlyer migrated from Kafka to Kubernetes using KarpenterAppsFlyer, a global leader in mobile attribution, migrated their Kafka infrastructure to Kubernetes using Amazon EKS, simplifying management and improving performance. By switching from EC2 instances to Graviton-powered nodes, they achieved a 75% increase in throughput, 58% better write I/O, and reduced costs by 30%. AWS solutions like Strimzi Kafka Operator, Rancher’s Local Path Provisioner, and Karpenter autoscaler helped optimize local storage management and scaling. This transition cut CPU core usage in half and enhanced AppsFlyer's Kafka cluster’s scalability, efficiency, and resilience.⚡TechWave: Cloud News & AnalysisIntroducing OpenAI o1OpenAI has introduced the "OpenAI o1" series, a new set of AI models designed to focus more on reasoning through complex problems, such as those in science, coding, and math. These models think more carefully before responding and perform significantly better than previous models in areas like math, coding competitions, and complex scientific tasks. Alongside the main "o1-preview" model, there is also a smaller, cheaper "o1-mini" model aimed at developers.Elasticsearch is Open Source, AgainElasticsearch is officially open source again as Elastic has added the AGPL license alongside its existing licenses (ELv2 and SSPL). This move allows Elasticsearch to be called open source under an OSI-approved license, clearing up any confusion caused when Elastic changed its licensing three years ago due to conflicts with AWS. While the license change led to a fork by Amazon, Elastic's partnership with AWS has strengthened, and now users have more licensing options without any impact on current usage.Oracle to offer 131,072 Nvidia Blackwell GPUs via its cloudOracle has announced it will offer 131,072 Nvidia Blackwell GPUs via its Oracle Cloud Infrastructure (OCI) Supercluster, starting in 2025, to support large language model (LLM) training and other AI use cases. This offering aims to meet the growing demand for GPUs, which are essential for generative AI development but in short supply due to limited availability of high-bandwidth memory (HBM). .Why eBPF is critical and how it’s getting bettereBPF (extended Berkeley Packet Filter) is a crucial open-source technology for Linux, providing powerful capabilities for networking, monitoring, and security by allowing safe execution of code in the kernel. It enhances network visibility, reduces patching cycles, and improves performance monitoring. Netflix, for example, uses eBPF for efficient traffic management and security.Juniper adds AI cloud services to its Apstra data center softwareJuniper Networks has updated its Apstra data center software with new AI-powered features, including a cloud-based suite called Apstra Cloud Services and the new 5.0 version of the software. Apstra uses AI to manage network configurations, ensure security policies, and monitor performance across both physical and virtual infrastructures. It now includes App/Service Awareness and Impact Analysis to help data center operators monitor application performance and quickly address issues.🛠️HackHub: Best Tools for Cloudnats-io/nats-serverNATS is a high-performance, cloud-native messaging system designed for modern distributed systems, offering secure and efficient communication between digital services, devices, and systems. It supports over 40 client languages.onceupon/Bash-Oneliner"Bash-Oneliner" is a blog focusing on simple and effective bash commands for tasks like data parsing and Linux system maintenance. The commands are compatible with systems like Ubuntu, Amazon Linux, RedHat, Linux Mint, Mac, and CentOS. The blog covers topics such as variable manipulation, system management, math operations, and networking.apache/kvrocksApache Kvrocks is a distributed key-value NoSQL database that uses RocksDB as its storage engine and is compatible with the Redis protocol. It aims to reduce memory costs and increase capacity compared to Redis. Kvrocks supports key features like asynchronous replication, high availability with Redis Sentinel, and a centralized cluster management system that works with any Redis cluster client.sealerio/sealerSealer is a tool that simplifies the delivery of distributed applications by packaging a Kubernetes cluster and all application dependencies into a "ClusterImage." A ClusterImage functions similarly to a Docker image, containing everything needed to run the application, such as container images, YAML files, and Helm charts. Users can write a "Kubefile" to build this image and a "Clusterfile" to describe how to run it.kubedl-io/kubedlKubeDL is a CNCF sandbox project that simplifies running deep learning workloads on Kubernetes. It offers features like unified scheduling for training and inference, advanced optimization, and native model tracking using Kubernetes Custom Resource Definitions (CRDs).📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Bestselling Cloud Titles specially curated for you Are you ready to enhance your expertise and stay ahead of the curve in the latest tech trends? Dive into cutting-edge resources designed to elevate your skills. Whether you're exploring cloud computing, refining your techniques, or mastering devops, we have the perfect reads for you. BESTSELLERS OF THE WEEK Linux Kernel Programming By Kaiwan N. Billimoria Discover how to write Linux kernel and module code for real-world products Implement industry-grade techniques in real-world scenarios for fast, efficient memory allocation and data synchronization Understand and exploit kernel architecture, CPU scheduling, and kernel synchronization techniques eBook: $39.99 $27.98 Print: $49.99 Mastering PowerShell Scripting By Chris Dent Key benefits: Explores PowerShell as a programming language Take advantage of the features built into the PowerShell language in day-to-day automation Automation of complex tasks, data manipulation, and environment security eBook: $35.99 $17.99 Print: $44.99 $30.99 Mastering Active Directory, Third Edition By Dishan Francis Key benefits Design and update your identity infrastructure by utilizing the latest Active Directory features and core capabilities Overcome migration challenges as you update to Active Directory Domain Services 2022 Establish a strong identity foundation in the cloud by consolidating secure access eBook: $43.99 $29.99 Print: $54.99 Automating DevOps with GitLab CI/CD Pipelines By Christopher Cowell, Nicholas Lotz, Chris Timberlake Key benefits Reap the power of GitLab CI/CD pipelines at every stage of your software development lifecycle Learn how GitLab makes Git easier to use and more powerful when committing and reviewing code Cement your understanding using hands-on tutorials and extensive self-assessment exercises eBook: $35.99$24.99 Print: $44.99 PowerShell Automation and Scripting for Cybersecurity By Miriam C. Wiesner Key benefits Master PowerShell for security by configuring, auditing, monitoring, exploiting, and bypassing defenses Research and develop methods to bypass security features and use stealthy tradecraft Explore essential security features in PowerShell and protect your environment against exploits and bypasses eBook: $39.99 $27.98 Print: $49.99 Want even more resources? Start a free trial and explore our entire library! From cloud solutions to system programming, gain unlimited access to the latest in tech. Start your free trial today. DISCOVER TRENDING TITLES Thanks, Packt Copyright (C) 2024 Packt Publishing. All rights reserved. Our mailing address is: Packt Publishing, Grosvenor House, 11 St Paul's Square, Birmingham, West Midlands, B3 1RB, United Kingdom Want to change how you receive these emails? You can update your preferences or unsubscribe *{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Infamous DevOps roadmapCloudPro #66: Grafana Soars Past $250M ARRWhat changed in the way you code for 2024? What has happened in the tech world in the last months? Take this shorter version of the Developer Nation survey, learn about new tools, influence the future of development and share your insights with the world!TAKE THE SURVEY⭐MasterclassInfamous DevOps roadmapKubernetes Open Source Limits & Requests Configuration OptimizationA guide to modern Kubernetes network policiesUsing Python Virtual Environments in DockerHow to terminate Go programs elegantly – a guide to graceful shutdowns🔍Secret KnowledgeHow Meta Enforces Purpose Limitation at ScaleWhy I Use Nim Instead of Python for Data ProcessingConvert OpenTelemetry Traces to Metrics using SpanMetrics ConnectorWhat happens when bucket.grantRead() in AWS CDKPreventing the Risk of Request Collapsing in Web Caching⚡TechwaveGrafana Labs Soars Past $250M ARR and 5,000 Customers, Completes $270M funding round, and Named a Leader in the Gartner Magic Quadrant for Observability PlatformsCockroachDB retires its free "Core" versionOpenMetrics is Archived, Merged into PrometheusAnnouncing Storage Browser for Amazon S3 for your web applications (alpha release)Juniper jumps into Wi-Fi 7 with enterprise switches, access points🛠️HackhubKardinal: lightest-weight way to spin up dev and test environments in KubernetesKubeblocks: control plane software that runs and manages databases, message queues on K8s.Flipt: Enterprise-ready, GitOps enabled, CloudNative feature management solutionKubecolor: Colorize your kubectl outputAWS-mine: AWS honey token manager💡Recommended Learning: Continuous Integration Mastery with JenkinsCheers,Shreyans SinghEditor-in-ChiefIntroducing A Market-Changing Approach to Mobile App Protection by GuardsquareMobile applications face constant, evolving threats.To address these challenges, Guardsquare is proud to announce the launch of our innovative guided configuration approach to mobile app protection.Learn More⭐MasterClass: Tutorials & GuidesInfamous DevOps roadmapThis roadmap provides community-driven guides, resources, and roadmaps to help developers grow in their careers, focusing on different fields like DevOps, backend development, and various programming languages. It offers step-by-step instructions for learning new skills, tracking progress, and staying updated with industry best practices.Kubernetes Open Source Limits & Requests Configuration OptimizationThis article provides a step-by-step guide on using Kexa, an open-source tool for optimizing Kubernetes resource limits and requests through monitoring and alerting, with Grafana for visualization. It explains how to install Kexa using Helm, set up necessary credentials, connect it to databases like Postgres or MySQL, and configure rules to monitor CPU and memory consumption. It then walks through the setup of a Grafana dashboard to display and optimize pod performance.A guide to modern Kubernetes network policiesIn Kubernetes, network policies are rules that control traffic flow between pods in a cluster. They define which traffic is allowed to enter (ingress), exit (egress), or move between pods, helping secure communication within the cluster. These policies fall into two categories based on the OSI model: Layer 4 (L4) policies, which control traffic using IP addresses and ports, and Layer 7 (L7) policies, which offer finer control at the application level (e.g., HTTP routes). By combining both, Kubernetes can implement robust, zero-trust security models.Using Python Virtual Environments in DockerThe author explains that despite the trend of simplifying Python Docker workflows by avoiding virtual environments, they continue using them for several key reasons. Virtual environments provide predictability, a well-defined structure, and consistency across projects, which simplifies communication and management in team environments. By isolating the Python environment, it helps prevent complex import issues and makes the codebase more reliable and easier to debug.How to terminate Go programs elegantly – a guide to graceful shutdownsBy handling termination signals like SIGTERM, Go applications can stop accepting new requests while allowing in-flight processes to finish, utilizing tools such as `signal.NotifyContext` and `sync.WaitGroup` to manage concurrency. This approach helps maintain data integrity and smooth operations during shutdowns, particularly in orchestrated environments where unexpected terminations can otherwise lead to issues.🔍Secret Knowledge: Learning ResourcesHow Meta Enforces Purpose Limitation at ScaleMeta enforces purpose limitation at scale using its Privacy Aware Infrastructure (PAI) through technologies like Policy Zones. Policy Zones ensure that data is processed only for its intended purposes by labeling and tracking data assets across systems. It integrates real-time checks during data flow, preventing unauthorized uses by monitoring the movement and processing of data in different environments like function-based or batch-processing systems. This approach provides granular control over data use while scaling across Meta’s complex infrastructure.Why I Use Nim Instead of Python for Data ProcessingThe author chooses Nim over Python for data processing because it offers the simplicity of Python with the speed of C, making it ideal for handling large datasets without complex optimization. In a comparison of processing a 150 MB genome file, Nim significantly outperforms Python, running 30 times faster with nearly identical code. While Nim requires a few syntax changes, such as using `var` for variables and `echo` for output, its faster compilation and execution make it a powerful alternative for tasks like analyzing DNA sequences.Convert OpenTelemetry Traces to Metrics using SpanMetrics ConnectorThe SpanMetrics Connector in OpenTelemetry allows you to convert trace data into actionable metrics, addressing the lack of native metrics support in some languages. It works by aggregating key metrics like request counts, errors, and durations (R.E.D metrics) from trace spans. By configuring it in the OpenTelemetry Collector, you can generate useful performance insights without adding extra instrumentation for metrics.What happens when bucket.grantRead() in AWS CDKWhen you call `bucket.grantRead()` in AWS CDK, it grants read permissions to an IAM role or user by either updating identity-based policies (attached to the IAM principal) or resource-based policies (attached to the S3 bucket). If the IAM role was created within the same CDK stack, identity-based policies are updated. However, if the role or bucket is just a reference (using interfaces like `IRole` or `IBucket`), CDK cannot modify existing policies, and the grant may not work.Preventing the Risk of Request Collapsing in Web CachingRequest collapsing is a caching feature where multiple identical requests for the same resource are combined, so only one is sent to the origin server to reduce load. However, this can cause security issues when dealing with sensitive data, as the response to the first request might be mistakenly sent to other users who made the same request. Even if a server uses `Cache-Control: no-cache`, request collapsing may still send cached responses to multiple users. To prevent this, it's crucial to use strict cache policies, such as disabling caching for certain patterns and configuring both the cache and origin server to avoid caching sensitive data.⚡TechWave: Cloud News & AnalysisGrafana Labs Soars Past $250M ARR and 5,000 Customers, Completes $270M Primary and Secondary Transaction, and Named a Leader in the Gartner® Magic Quadrant™ for Observability PlatformsCockroachDB retires its free "Core" versionCockroachDB is evolving its self-hosted offering by retiring the free "Core" version and consolidating all users into a single "Enterprise" version that provides full access to its advanced features. This change, starting with version 24.3, ensures that individuals, students, and small businesses (under $10M annual revenue) can still use CockroachDB Enterprise for free with community support, while larger businesses will need a paid license.OpenMetrics is Archived, Merged into PrometheusThe OpenMetrics project, originally created to spin off Prometheus' metrics format into an independent specification, has been archived and merged back into Prometheus as of July 2024. While OpenMetrics aimed to become a universal format for exporting metrics, it struggled to gain adoption outside the Prometheus ecosystem, where Prometheus had already become the de facto standard for cloud-native observability.Announcing Storage Browser for Amazon S3 for your web applications (alpha release)Amazon S3 has released an alpha version of Storage Browser for S3, an open-source component that lets web applications provide a simple interface for users to browse, download, and upload S3-stored data. It integrates with AWS Amplify's JavaScript and React libraries, allowing developers to control access based on user identity and customize the design to fit their app's branding.Juniper jumps into Wi-Fi 7 with enterprise switches, access pointsJuniper has introduced new EX Series switches and Mist Wi-Fi 7 access points for enterprise wireless networks, offering higher speeds, lower latency, and broader range. The EX4400 switches support both Wi-Fi 6E and Wi-Fi 7 and are managed via the AI-powered Mist Cloud, which helps detect and resolve network issues. Juniper’s new AP47 Series access points offer advanced features like dual-5GHz or dual-6GHz operation and AI-based channel management.🛠️HackHub: Best Tools for Cloudkardinal: lightest-weight way to spin up dev and test environments in KubernetesCreate lightweight, temporary development environments within a shared Kubernetes cluster, making testing and development more efficient. It allows developers to spin up tailored, on-demand "flows"—ephemeral environments that use minimal resources by deploying only the necessary services for feature development.kubeblocks: control plane software that runs and manages databases, message queues on K8s.KubeBlocks is an open-source control plane software designed to simplify the management of multiple database engines on Kubernetes (K8s). It uses a unified set of APIs to manage various types of databases, such as MySQL, PostgreSQL, Redis, and Kafka, reducing the need to learn individual database operators.flipt: Enterprise-ready, GitOps enabled, CloudNative feature management solutionFlipt is a cloud-native, GitOps-enabled feature management solution designed to help organizations separate feature releases from deployments, allowing for safer, more controlled updates. It can be integrated into existing infrastructure to avoid third-party latency and is built with high-performance DevOps teams in mind.kubecolor: Colorize your kubectl outputKubecolor is a simple wrapper for the kubectl command-line tool that adds color to its output, making it easier to read and interpret. It enhances the standard kubectl by colorizing logs, tables, and other outputs without changing the actual content. Kubecolor supports custom themes, including options for light backgrounds and colorblind-friendly themes.aws-mine: AWS honey token manageraws-mine is a project designed to create "honey tokens" for AWS, which are fake AWS access keys placed in various locations to lure potential attackers. If someone uses these keys, the system sends a notification within about four minutes, allowing you to investigate the possible compromise. Built with AWS Amplify for easy deployment, users can manage their access through Amazon Cognito and receive alerts via Amazon SNS when the keys are accessed.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Understand your Kubernetes cost drivers and the best ways to rein in spendingCloudPro #74: How Netflix solved the issue with Java 21 virtual threadsScale your scrapers with Apify’s Black Friday Boost planGet a 30% prepaid usage bonus on Apify this Black Friday.Scrape data for LLMs, machine learning, competitive intelligence, product mapping, or any AI use cases.Use ready-made scrapers or build your own.The Boost plan ends December 5 - grab it before it’s gone!Claim your bonus now⭐MasterclassDeploying Databases in K8s with PostgreSQL, CloudNative-PG, and Ceph Rook on Amazon EKSKubernetes and Access Management API, the new authentication in EKSHow Netflix solved the issue with Java 21 virtual threadsDoes Talos Kubernetes and Omni live up to the hype?Understand your Kubernetes cost drivers and the best ways to rein in spending🔍Secret KnowledgeHack the Cybersecurity InterviewTroubleshooting: Terminal LagMonitor these Kubernetes signals to help rightsize your fleetGetting Started with Cilium Service Mesh on Amazon EKSHow AppsFlyer migrated from Kafka to Kubernetes using Karpenter🛠️HackhubHigh-performance server for NATS.ioA collection of Bash One-Liners and terminal tricksdistributed key value NoSQL database that uses RocksDB as storage engineBuild, Share and Run Both Your Kubernetes Cluster and Distributed ApplicationsRun your deep learning workloads on Kubernetes more easily and efficientlyCheers,Shreyans SinghEditor-in-ChiefForward to a Friend⭐MasterClass: Tutorials & GuidesDeploying Databases in K8s with PostgreSQL, CloudNative-PG, and Ceph Rook on Amazon EKSThis article explains how to deploy and manage a PostgreSQL database on Kubernetes using Amazon EKS. It combines CloudNativePG, a PostgreSQL operator, with Ceph Rook, a storage orchestrator, to ensure data persistence and high availability in a Kubernetes environment. A step-by-step guide is provided for deploying and configuring these tools, including using Helm charts, setting up storage with Ceph, and monitoring the database cluster.Kubernetes and Access Management API, the new authentication in EKSIn AWS EKS (Elastic Kubernetes Service), the new authentication and access management system simplifies how users and worker nodes access Kubernetes clusters. Previously, access was managed using an "aws-auth" ConfigMap, which could be complex and prone to errors. Now, AWS introduces the EKS Access Management API, allowing users to authenticate through AWS IAM directly and authorize Kubernetes actions via Kubernetes RBAC. This approach removes the need for managing ConfigMaps manually, offers predefined EKS Access Policies, and enhances security by eliminating hidden root users. Additionally, integration with Terraform makes access control easier to implement and manage.How Netflix solved the issue with Java 21 virtual threadsIn Java 21, Netflix encountered an issue with virtual threads, which are lightweight threads designed to improve concurrency by suspending and resuming automatically. The problem involved some Netflix services using Java 21, Spring Boot 3, and Tomcat, where servers stopped processing requests due to sockets stuck in a `CLOSE_WAIT` state. Virtual threads were getting "pinned" to operating system threads while waiting to acquire locks. Since all OS threads became blocked, Tomcat couldn't process incoming requests, causing the system to hang. The underlying issue was traced to locking mechanisms in virtual threads, leading to thread contention and blocked OS threads.Does Talos Kubernetes and Omni live up to the hype?Talos Kubernetes and Sidero Omni live up to the hype by providing an intuitive and efficient way to set up and manage Kubernetes clusters. With Omni, you can easily create a Talos cluster without needing to access your virtual machines directly, making the process more streamlined. Setting up clusters, scaling nodes, and even upgrading Kubernetes versions are straightforward, with minimal manual intervention required. While there are some areas for improvement, like simplifying static IP configuration, the overall experience is highly positive.Understand your Kubernetes cost drivers and the best ways to rein in spendingTo reduce Kubernetes-related costs, it's important to monitor key cost drivers such as CPU, memory, storage, and networking. Costs are driven by resource usage and the rate at which they are consumed, so reducing unnecessary usage and optimizing resource allocation is key. Over-provisioning, idle resources, and inefficient scaling are common cost culprits. Regularly adjusting resource requests, leveraging auto-scaling tools like Horizontal Pod Autoscaler, and monitoring metrics with tools like Grafana and Prometheus can help optimize usage.🔍Secret Knowledge: Learning ResourcesHack the Cybersecurity Interview"Hack the Cybersecurity Interview, Second Edition" is a comprehensive guide designed to help individuals prepare for interviews across a wide range of cybersecurity roles. The book covers technical and behavioral interview questions for positions like cybersecurity engineer, penetration tester, and CISO, while also offering tips on personal branding, stress management, and negotiation. It provides real-world advice and industry insights, making it an essential resource for anyone looking to succeed in the competitive field of cybersecurity.Troubleshooting: Terminal LagIn this troubleshooting session, Tavis Ormandy investigates why launching the xterm terminal on his Windows machine is significantly slower compared to Fedora. He identifies that Windows applies an animation effect that delays interaction with the terminal. Through a series of tests and debugging, he discovers that the X server software (X410) adds unnecessary animation effects, which can’t be disabled directly. He uses a debugger to bypass the issue, improving the performance slightly. After further optimizations with features and caching processes, he brings the Windows terminal's performance closer to Fedora’s, significantly reducing the lag.Monitor these Kubernetes signals to help rightsize your fleetTo ensure your Kubernetes environment is both cost-efficient and sustainable, it's crucial to monitor signals like CPU, memory, disk I/O, and network utilization. Over-provisioning leads to wasted resources and high costs, while under-provisioning can degrade performance. Watch for indicators such as high resource usage, slow application performance, or low utilization to fine-tune your setup. Tools like Prometheus and Grafana, along with autoscaling, can help you dynamically adjust resources, ensuring optimal balance, reduced costs, and improved sustainability.Getting Started with Cilium Service Mesh on Amazon EKSThe blog post explains how Cilium, an open-source networking and security solution powered by eBPF, enhances network connectivity between workloads in Amazon EKS (Elastic Kubernetes Service). Cilium provides advanced networking, load balancing, encryption, and observability without the need for sidecar proxies. It integrates seamlessly with Amazon EKS to improve microservice communication, multi-cluster networking, and network policy enforcement. Cilium Service Mesh, built into Cilium, leverages eBPF and Envoy to offer high performance and low overhead for traffic management, security, and monitoring.How AppsFlyer migrated from Kafka to Kubernetes using KarpenterAppsFlyer, a global leader in mobile attribution, migrated their Kafka infrastructure to Kubernetes using Amazon EKS, simplifying management and improving performance. By switching from EC2 instances to Graviton-powered nodes, they achieved a 75% increase in throughput, 58% better write I/O, and reduced costs by 30%. AWS solutions like Strimzi Kafka Operator, Rancher’s Local Path Provisioner, and Karpenter autoscaler helped optimize local storage management and scaling. This transition cut CPU core usage in half and enhanced AppsFlyer's Kafka cluster’s scalability, efficiency, and resilience.🛠️HackHub: Best Tools for Cloudnats-io/nats-serverNATS is a high-performance, cloud-native messaging system designed for modern distributed systems, offering secure and efficient communication between digital services, devices, and systems. It supports over 40 client languages.onceupon/Bash-Oneliner"Bash-Oneliner" is a blog focusing on simple and effective bash commands for tasks like data parsing and Linux system maintenance. The commands are compatible with systems like Ubuntu, Amazon Linux, RedHat, Linux Mint, Mac, and CentOS. The blog covers topics such as variable manipulation, system management, math operations, and networking.apache/kvrocksApache Kvrocks is a distributed key-value NoSQL database that uses RocksDB as its storage engine and is compatible with the Redis protocol. It aims to reduce memory costs and increase capacity compared to Redis. Kvrocks supports key features like asynchronous replication, high availability with Redis Sentinel, and a centralized cluster management system that works with any Redis cluster client.sealerio/sealerSealer is a tool that simplifies the delivery of distributed applications by packaging a Kubernetes cluster and all application dependencies into a "ClusterImage." A ClusterImage functions similarly to a Docker image, containing everything needed to run the application, such as container images, YAML files, and Helm charts. Users can write a "Kubefile" to build this image and a "Clusterfile" to describe how to run it.kubedl-io/kubedlKubeDL is a CNCF sandbox project that simplifies running deep learning workloads on Kubernetes. It offers features like unified scheduling for training and inference, advanced optimization, and native model tracking using Kubernetes Custom Resource Definitions (CRDs).📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Path Traversal vulnerability found in Grafana versions 8.xCloudPro #68: New Telemetry Feature in Go 1.23Notion for StartupsThousands of startups use Notion as a connected workspace to create and share docs, take notes, manage projects, and organize knowledge—all in one place.We’re offering 6 months of new Plus plans, including unlimited Notion AI so you can try it all for free!Redemption InstructionsTo redeem the Notion for Startups offer:1. Submit an application using our custom link and select Packt on the partner list.2. Include our partner key, STARTUP4110P19151.Free 6-Month Notion Plus Access! Use Our Packt Partner Key!Next month, Packt is hosting an AI conference. 3 days of LIVE sessions with 20+ top experts and unlock the full potential of Generative AI. If this sounds interesting, check out the conference here.Today we will talk about:⭐MasterclassA guide to modern Kubernetes network policiesKubernetes 1.31: Pod Failure Policy for Jobs Goes GAHot Reloading in KubernetesRunning application on Docker Swarm with Docker SecretsWhy is Browser Observability Hard🔍Secret KnowledgeTelemetry in Go 1.23 and beyondNginx Logging - A Comprehensive GuideMy Methodology to AWS Detection Engineering (Part 2: Risk Assignment)Comparison of Serverless Development and Hosting PlatformsMaking sense of secrets management on Amazon EKS for regulated institutions⚡TechwaveKubecost 2.4 Release HighlightsAmazon S3 Express One Zone now supports AWS KMS with customer managed keysAmazon RDS for MySQL zero-ETL integration with Amazon Redshift, now generally available, enables near real-time analytics🛠️HackhubPath Traversal vulnerability found in Grafana versions 8.xK4all: A Kubernetes installer 4 ALLValidate-aws-policies: Python CLI to validate aws policies using boto3 and Access Analyzer APIBoxxy: boxxy puts bad Linux applications in a box with only their filesRuncvm: experimental open-source Docker container runtime, for launching standard container workloadsCheers,Shreyans SinghEditor-in-ChiefYour cloud deserves dedicated data protectionHere’s a handy resource you’ll want with you as you map out your plan:Orchestrating the Symphony of Cloud Data Security.You’ll learn how to:-Overcome the challenges of securing data in the cloud-Navigate multi cloud data security-Balance data security with cloud economicsClick below to download your complimentary copy.Download Now⭐MasterClass: Tutorials & GuidesA guide to modern Kubernetes network policiesKubernetes network policies are essential for controlling traffic in a cluster, ensuring security and communication management. They allow administrators to define which traffic is allowed to enter (ingress), exit (egress), or move between pods. Network policies are divided into two main types: Layer 4 (L4) and Layer 7 (L7). L4 policies operate at the transport layer, focusing on IP addresses and ports, while L7 policies offer more granular control at the application layer, managing protocols like HTTP and gRPC.Kubernetes 1.31: Pod Failure Policy for Jobs Goes GAKubernetes 1.31 introduces the Pod failure policy for Jobs, which helps manage pod failures more efficiently. This policy allows users to differentiate between retriable and non-retriable pod failures, providing more control over how Jobs handle failures. It complements the existing backoffLimit by letting users ignore transient errors or fail Jobs immediately upon serious errors. Users can define rules based on pod conditions or exit codes and specify actions like ignoring the failure or terminating the entire Job.Hot Reloading in KubernetesHot reloading in Kubernetes allows developers to instantly see changes in their code without restarting or rebuilding the application, enhancing productivity. Tools like Tilt enable this by streamlining the deployment process, making it easier to update Kubernetes applications in real time. Using Tilt with tools like K3d (for lightweight Kubernetes clusters) and ttl.sh (for ephemeral Docker image storage), developers can efficiently test changes in a cloud-native environment. This setup is especially useful for complex microservices architectures, where hot reloading minimizes the need for extensive test code or mocks.Running application on Docker Swarm with Docker SecretsThis article explains how to run an application on Docker Swarm using Docker Secrets to securely manage sensitive information such as database credentials. Docker Swarm is a container orchestration tool that manages multiple Docker nodes (servers) as a single system. The article demonstrates how to create a Docker Swarm cluster, define services in a `docker-compose.yml` file, and use Docker Secrets to manage sensitive data like usernames and passwords securely.Why is Browser Observability HardBrowser observability is challenging because the asynchronous, event-driven nature of front-end systems doesn't fit well with OpenTelemetry’s linear lifecycle model, which works best for synchronous, short-lived processes. React adds complexity by extending component lifetimes unpredictably and lacking lifecycle hooks to track spans effectively. Additionally, browsers face issues like no gRPC support, data loss, and limitations in efficiently handling telemetry data without increasing page load or draining user resources.🔍Secret Knowledge: Learning ResourcesTelemetry in Go 1.23 and beyondGo 1.23 introduces a new feature allowing users to enable telemetry, which helps the Go team collect data about toolchain usage to improve performance and fix bugs. By default, telemetry data is only stored locally, but users can choose to upload it by enabling the option. This feature started with Go's language server and has already helped identify and fix bugs.Nginx Logging - A Comprehensive GuideNginx logging involves recording crucial information such as client requests and errors to help monitor and manage a web server's performance. Logs are stored in two main files: the access log (records requests and their details) and the error log (captures issues encountered during operations). Nginx logs can be customized for clarity or to capture specific details using the `log_format` directive, and can be stored locally or managed through Docker for ease of access. Structured logging with JSON format can also be implemented to streamline the analysis, making it easier to debug or monitor Nginx's performance efficiently.My Methodology to AWS Detection Engineering (Part 2: Risk Assignment)In Part 2 of his AWS Detection Engineering series, the author explains how to assign risk scores to AWS-specific alerts using Splunk's Risk-Based Alerting (RBA). The methodology involves filtering AWS detections, assigning default severity and fidelity scores, and expanding the risk object to cover multiple data fields (like instance IDs or IPs). A base risk score is then calculated by multiplying the severity score with fidelity. The collected data is sent to a risk index for analysis. The author also shares alternative risk assignment methods and emphasizes the importance of tuning detection rules to prevent score inflation.Comparison of Serverless Development and Hosting PlatformsThe post outlines a typical workflow for deploying an application using services like AWS Amplify, which integrates CI/CD processes, secret management, and connections to other AWS resources. Additionally, the author provides a comparison of various serverless platforms regarding supported programming languages, frameworks, and security features, ultimately recommending serverless solutions for simplifying the development lifecycle of cloud-native applications.Making sense of secrets management on Amazon EKS for regulated institutionsAWS provides tools like Kubernetes Secrets, AWS Secrets Manager, and open-source solutions (e.g., Sealed Secrets) to safeguard sensitive data like passwords and API keys. However, Kubernetes' native secrets management has limitations, as secrets are only base64-encoded, not encrypted. To meet regulatory requirements (e.g., PCI DSS, HIPAA), regulated industries often use enhanced solutions like the External Secrets Operator (ESO), AWS Secrets Store CSI Driver, and Sealed Secrets to encrypt and securely manage secrets, ensuring compliance and operational security.⚡TechWave: Cloud News & AnalysisKubecost 2.4 Release HighlightsKubecost 2.4 introduces several key features, including new tools for GPU cost monitoring and efficiency, helping teams optimize their spending on AI/ML hardware. It adds support for Oracle Cloud, allowing users to monitor costs across multiple cloud providers in one place. The release also brings enhanced cost aggregation and filtering options, as well as the ability to include idle costs in budgeting reports. Additional updates include more granular cluster rightsizing recommendations and various enhancements to improve Kubernetes cost tracking and management.Amazon S3 Express One Zone now supports AWS KMS with customer managed keysAmazon S3 Express One Zone now supports AWS KMS for server-side encryption using customer-managed keys. This feature allows users to encrypt data at rest with their own keys, offering an additional layer of security and compliance without impacting performance. It ensures high-performance, single-digit millisecond data access while reducing AWS KMS requests by up to 99%, thanks to the automatic use of S3 Bucket Keys.Amazon RDS for MySQL zero-ETL integration with Amazon Redshift, now generally available, enables near real-time analyticsAmazon RDS for MySQL now offers a zero-ETL integration with Amazon Redshift, allowing near real-time data replication for analytics without the need to manually build and manage ETL pipelines. This integration makes it easy to move data from MySQL databases to Amazon Redshift within seconds, enabling quick analysis of transactional data. New features include data filtering, support for multiple integrations, and the ability to configure these integrations in AWS CloudFormation. This simplifies operations, reduces costs, and helps businesses get insights faster with minimal setup and maintenance.🛠️HackHub: Best Tools for CloudPath Traversal vulnerability found in Grafana versions 8.xCVE-2021-43798 is a path traversal vulnerability found in Grafana versions 8.x that allows attackers to access files on the server without authentication. This is due to improper sanitization of file paths provided by users in the Grafana public API. Attackers can exploit this vulnerability by manipulating the file path to access sensitive files, such as configuration files or databases, on the server. They can use HTTP requests or scripts like the one in the repository to dump critical data, potentially leading to further attacks such as database extraction or password decryption.K4all: A Kubernetes installer 4 ALLThe k4all project provides a pre-configured Fedora CoreOS ISO designed for setting up Kubernetes clusters, particularly for home servers or virtual machines. It includes essential Kubernetes tools like the Calico networking solution, a metrics server, NGINX as an Ingress controller, and the Logical Volume Manager (LVM) for managing persistent storage.Validate-aws-policies: Python CLI to validate aws policies using boto3 and Access Analyzer APIThe "validate-aws-policies" project is a Python command-line tool designed to scan and validate AWS Service Control Policies (SCPs) using the AWS IAM Access Analyzer API. It generates reports in both HTML and PDF formats, allowing users to review the compliance and structure of their AWS policies.Boxxy: boxxy puts bad Linux applications in a box with only their filesBoxxy is a Linux-only tool that helps organize misbehaving applications by redirecting where they store their files and directories, without using symlinks. It uses Linux namespaces to control this behavior, allowing users to specify custom rules for file locations. For example, you can force AWS CLI to store its configuration in a different directory than the default. Boxxy is particularly useful for keeping home directories tidy by redirecting application data to more appropriate locations.Runcvm: experimental open-source Docker container runtime, for launching standard container workloadsRunCVM allows users to run containerized applications inside lightweight virtual machines (VMs) using Docker. It simplifies the process of launching both standard container workloads and system-level tasks (like Systemd and Docker) in VMs, making it as easy as running a regular container. With RunCVM, you can use commands like `docker run` to start VMs directly from container images.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Chrome Vulnerability Reward Program (VRP) has updated its rewardsCloudPro #67: Supercharge Your Kubernetes Workflow with Essential Tools: Starship, Kubectx, Kubecolor, and K9sThis 3 hour power packed workshop that will teach you 25+ AI Tools, make you a master of prompting & talk about hacks, strategies & secrets that only the top 1% know of.Best thing? It's usually $399, but it's absolutely free for the first 100 readers.Save your seat now (Offer valid for 24 hours only)⭐Masterclass[Sponsored] Become an AI Powered Professional. Free 3-hour ChatGPT and AI workshop for ProfessionalsPreemptible pods: Optimizing Kubernetes node utilizationSupercharge Your Kubernetes Workflow with Essential Tools: Starship, Kubectx, Kubecolor, and K9sExploring Helm template dictionary objects: Syntax evolution and best practicesDockerizing a Golang API with MySQL and adding Docker Compose SupportKarmada: Deep dive into managing multiple AKS clusters🔍Secret KnowledgeZero Downtime Deployment in AWS with TofuCron Jobs on LinuxHow To Run Migrations Across 2,800 MicroservicesTransform AWS exam generator architecture to open sourceHow to Run WebAssembly on Amazon EKS⚡TechwaveChrome Vulnerability Reward Program (VRP) has updated its rewardsHow misconfigured AWS IAM roles using GitLab's OpenID Connect (OIDC) can allow unauthorized users to assume rolesPreview Release of the Migration Tool for the AWS SDK for Java 2.xAmazon’s Exabyte-Scale Migration from Apache Spark to Ray on Amazon EC2Unlock 1 Million RPS: Experience Triple the Speed with Valkey🛠️Hackhubkubeai: Private Open AI on Kubernetescyphernetes: A Kubernetes Query Languagechartdb: Free and open-source database diagrams editor, visualize and design your DB with a single query.stack-auth: Open-source Auth0/Clerk alternativemariadb-operator: Run and operate MariaDB in a cloud native way💡Get 30% off on CloudPro Book of the Week: AWS DevOps SimplifiedCheers,Shreyans SinghEditor-in-ChiefLast Chance! For the next 48 hours only, save $150 on your full event pass!Imagine being part of 10+ Power Talks, 12+ Hands-On Workshops, and 3 Interactive Roundtables—while networking with 30+ top industry leaders and hundreds of tech professionals from across the globe. This is your opportunity to dive into cutting-edge AI solutions at the Generative AI in Action 2024 Conference.It's all happening on November 11-13 (LIVE, Virtual) - prices increase permanently on Saturday!BOOK YOUR SEAT NOW before prices go up!Use code LASTCHANCE40 at checkoutBOOK NOW AT $399.99 $239.99⭐MasterClass: Tutorials & GuidesPreemptible pods: Optimizing Kubernetes node utilizationPreemptible Pods in Kubernetes enable efficient resource management by allowing you to assign priorities to different workloads through pod priority and preemption mechanisms. This means that critical applications are guaranteed the resources they need because higher-priority pods can preempt, or evict, lower-priority ones when resources are scarce. By implementing PriorityClasses and configuring pods accordingly, you ensure that essential services remain responsive and that your cluster optimizes node utilization.Supercharge Your Kubernetes Workflow with Essential Tools: Starship, Kubectx, Kubecolor, and K9sTo enhance your Kubernetes workflow, using tools like Starship, Kubectx, Kubecolor, and K9s can significantly improve efficiency. Starship provides a customizable, fast shell prompt that shows key info like cluster and namespace, while Kubectx and Kubens allow quick switching between clusters and namespaces. Kubecolor adds color to kubectl output for better readability, and K9s offers a terminal-based UI to manage and visualize Kubernetes resources easily.Exploring Helm template dictionary objects: Syntax evolution and best practicesHelm, the Kubernetes package manager, uses dictionary objects in its templating system to manage key-value pairs for application deployment. Initially, Helm syntax allowed for creating dictionaries in a single line, but this became cumbersome when handling many properties. Over time, a more efficient syntax evolved, using the `set` function to incrementally add properties to a dictionary without recreating it. Best practices for using Helm dictionaries include adding properties incrementally, avoiding reassignment to prevent data loss, maintaining consistent naming conventions, and thoroughly testing templates to ensure correct Kubernetes manifest generation.Dockerizing a Golang API with MySQL and adding Docker Compose SupportDockerizing a Golang API with MySQL simplifies the process of developing and testing APIs locally by containerizing both the API and database. First, you create a Dockerfile for the Go API using best practices like lightweight base images, multi-stage builds, creating a binary, and optimizing Docker layers. This ensures a smaller and more efficient container. Then, to streamline managing both the API and MySQL containers, Docker Compose is used. A `compose.yml` file sets up both services, ensuring the API only starts once the MySQL database is ready, avoiding connection issues. This setup makes local development smoother and easier to replicate.Karmada: Deep dive into managing multiple AKS clustersKarmada (Kubernetes Armada) is a tool that simplifies managing multiple AKS (Azure Kubernetes Service) clusters by treating them like a single entity. It helps deploy applications across clusters while handling tasks like scheduling, resource propagation, and ensuring consistency. Karmada’s components—such as the API Server, Controller Manager, Scheduler, and Agent—work together to automate the deployment process. It supports advanced strategies like multi-cluster deployments, disaster recovery, and canary releases.🔍Secret Knowledge: Learning ResourcesZero Downtime Deployment in AWS with TofuZero Downtime Deployment in AWS is a strategy to update applications without causing service interruptions. By leveraging tools like OpenTofu, Terraform, and AWS SAM, developers can ensure seamless updates. Techniques like instance refreshes in Auto Scaling Groups (using OpenTofu), immutable infrastructure (Terraform + Ansible), and advanced deployment strategies like Blue/Green and Canary deployments enable applications to be updated while keeping them available to users. These approaches allow for gradual testing, automated rollbacks, and maintaining reliability.Cron Jobs on LinuxCron jobs in Linux are scheduled tasks that automate running scripts or commands at specific times or intervals, managed by the cron daemon. Common use cases include backups, updates, and system health checks. Users can create, view, or edit cron jobs using the `crontab` command. Cron jobs are defined using a simple time-based syntax, where each job can run on a specific schedule (e.g., hourly, daily, or weekly). Cron jobs can be user-specific or system-wide, and their syntax supports flexible timing options like ranges, lists, and intervals.How To Run Migrations Across 2,800 MicroservicesTo handle migrations across 2,800 microservices, we use a centrally driven approach where a single team manages the entire process. This allows us to keep libraries up-to-date, maintain consistency, and automate the bulk of the changes, reducing coordination overhead and minimizing risks of failure. Our strategy relies on a monorepo structure, consistent technology (like Go), and powerful mass deployment tooling. We start by wrapping old libraries, automate common updates, handle edge cases manually, and control rollouts via config changes to ensure smooth transitions without downtime.Transform AWS exam generator architecture to open sourceIn this series, we aim to transform a serverless AWS architecture for an exam generator app into an open-source version. The original solution helps educators create curriculum-aligned assessments quickly, while students can take personalized quizzes with instant feedback. We'll replace key AWS services like Cognito, Lambda, DynamoDB, and Fargate with open-source alternatives and host everything on a Kubernetes cluster.How to Run WebAssembly on Amazon EKSThe article outlines the process of setting up a Wasm environment on Amazon EKS using tools like HashiCorp Packer and Terraform to create custom Amazon Machine Images (AMIs) and manage the infrastructure. It details how to build an EKS cluster, deploy example workloads using different Wasm runtimes (Spin and WasmEdge), and check if everything is working correctly. Finally, it offers instructions for cleaning up the resources after running the applications.⚡TechWave: Cloud News & AnalysisChrome Vulnerability Reward Program (VRP) has updated its rewardsGoogle's Chrome Vulnerability Reward Program (VRP) has updated its reward structure to encourage deeper research into Chrome's security vulnerabilities. As Chrome becomes more secure, finding impactful bugs has become harder. The new structure separates memory corruption bugs from other vulnerability types and offers higher rewards for more complex, well-documented reports, such as those demonstrating remote code execution (RCE) or memory corruption. The top reward for an RCE in a non-sandboxed process is now $250,000. These changes aim to incentivize thorough and high-quality bug reporting, ensuring Chrome remains secure.How misconfigured AWS IAM roles using GitLab's OpenID Connect (OIDC) can allow unauthorized users to assume rolesThe article by Nick Frichette explains how misconfigured AWS IAM roles using GitLab's OpenID Connect (OIDC) can allow unauthorized users to assume roles. This occurs when the trust policy lacks restrictions on which specific GitLab groups or projects can access the role. By default, the AWS Console creates a vulnerable trust policy, making it possible for any GitLab user to exploit the misconfiguration. The article walks through how to generate a GitLab OIDC token and use it to assume a misconfigured role, highlighting the risks of default settings in AWS.Preview Release of the Migration Tool for the AWS SDK for Java 2.xAWS has released a preview of a migration tool to help developers transition from AWS SDK for Java 1.x to 2.x, as 1.x is now in maintenance mode. This tool uses OpenRewrite, an open-source code refactoring tool, to automate much of the migration process. It currently supports most service SDK clients, except for AmazonS3Client, TransferManager, and DynamoDBMapper, and helps reduce the time and effort needed for the upgrade. Developers can use this tool with Maven or Gradle projects, choosing between preview (dryRun) or actual (run) modes to apply the changes.Amazon’s Exabyte-Scale Migration from Apache Spark to Ray on Amazon EC2Amazon’s Business Data Technologies (BDT) team is migrating from Apache Spark to Ray on Amazon EC2 to handle exabyte-scale data more efficiently. The switch is driven by the need to reduce data processing costs and time for their large business intelligence datasets. Apache Spark, though powerful, had started to show limitations with scalability and performance as their data grew. Ray, initially known for machine learning tasks, offered a more flexible and cost-effective solution with its distributed compute capabilities, reducing processing costs by 82% and improving data processing speeds significantly.Unlock 1 Million RPS: Experience Triple the Speed with ValkeyValkey 8.0, set for release in September 2024, introduces a new multi-threaded architecture that significantly boosts performance, increasing throughput by 230% to over 1 million requests per second and reducing latency by nearly 70%. This is achieved through an innovative I/O threading system, where dedicated worker threads handle tasks like reading commands and writing responses, freeing up the main thread to focus on executing commands. Valkey 8.0 also supports larger shards, improving performance for workloads that don't scale well horizontally, but comes with trade-offs like increased complexity in managing larger nodes.🛠️HackHub: Best Tools for Cloudkubeai: Private Open AI on KubernetesKubeAI is an open-source tool that allows users to run AI models like LLMs (Large Language Models), embeddings, and speech-to-text on Kubernetes. It provides an API compatible with OpenAI, letting users serve and scale models like Whisper and vLLM across CPU, GPU, and soon TPU infrastructure.cyphernetes: A Kubernetes Query LanguageCyphernetes is a query language for Kubernetes inspired by Cypher (from Neo4j) that simplifies managing Kubernetes resources. Instead of complex `kubectl` commands, Cyphernetes lets users perform operations like finding and modifying deployments, services, and ingresses with clear, SQL-like syntax.chartdb: Free and open-source database diagrams editor, visualize and design your DB with a single query.ChartDB is an open-source, web-based tool for creating and editing database diagrams. With a single "Smart Query," users can instantly visualize their database schema, making it easy to understand and document database structures. It supports multiple databases like PostgreSQL, MySQL, and SQLite.stack-auth: Open-source Auth0/Clerk alternativeChartDB is a free, open-source tool for creating and editing database diagrams. It allows users to instantly visualize their database schema with a single query and supports databases like PostgreSQL, MySQL, and SQLite. Users can interactively edit schemas, export SQL scripts, and even use AI to generate migration scripts for switching between databases.mariadb-operator: Run and operate MariaDB in a cloud native wayThe MariaDB Operator allows users to manage MariaDB databases in a cloud-native environment using Kubernetes. It simplifies tasks like deploying and operating MariaDB instances through Custom Resource Definitions (CRDs), enabling features like high availability, automated backups, and flexible storage options.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

CloudPro Special EditionCloudPro Special: Mastering Serverless Architecture- A Concise GuideAfter the last CloudPro Special, a lot of subscribers messaged me that I should do it more often. So here we are, with the next CloudPro Special. Today, I’ll talk about Serverless Architecture.There are eight sections:1. Introduction to Serverless Architecture2. Designing Serverless Solutions3. Using Serverless with AWS4. Serverless in the Bigger Picture5. Real-world Use Cases6. Monitoring Serverless Apps7. Pros and Cons8. Conclusion`Each section has additional learning resources:Cloud Computing Demystified for Aspiring ProfessionalsArchitecting Cloud-Native Serverless SolutionsMulti-Cloud Strategy for Cloud ArchitectsAWS CDK in PracticeMulti-Cloud Handbook for DevelopersAWS for Solutions ArchitectsMastering Amazon EC2Solutions Architect's HandbookAWS Certified Developer Associate Certification and BeyondAWS Observability HandbookAny feedback or questions, just reply back to this email and let me know. Without further delay, let's jump into today's CloudPro Special!Cheers,Shreyans SinghEditor in ChiefIntroduction to Serverless ArchitectureServerless architecture is a new way to build and run apps without worrying about servers. Despite its name, there are still servers involved, but cloud providers like Amazon or Google manage them for you. This means you can focus on writing code instead of managing hardware.Serverless is the latest step in cloud computing. It started with physical machines, then moved to services where you rent virtual servers, and now we have serverless where you just run your code without thinking about the servers at all.💡Learning ResourceTo learn more about serverless and cloud computing, check out "Cloud Computing Demystified for Aspiring Professionals" ($24.99 $35.99). It explains these concepts in simple terms.Learn more about cloud computing and serverlessDesigning Serverless SolutionsWhen building serverless apps, keep these things in mind:📌Make your functions independent (they shouldn't rely on saved information)📌Build your app around events and triggers📌Break your app into small, separate services📌Be aware of "cold starts" which can slow things down📌Make sure your functions finish quickly (there are time limits)Remember, serverless isn't always the cheapest option. It works best for apps with unpredictable usage. For apps with steady, predictable usage, traditional servers might be cheaper.💡Learning ResourceTo learn more about designing serverless apps, I recommend "Architecting Cloud-Native Serverless Solutions" ($24.99 $35.99). It's a helpful guide for building serverless apps on different cloud platforms.Learn more about designing serverless solutionsIf you want to use serverless with multiple cloud providers, check out "Multi-Cloud Strategy for Cloud Architects" ($29.99 $43.99). It helps you understand how to use serverless across different providers.Learn about using multiple cloud providersUsing Serverless with AWSAWS offers many serverless tools. Here are the main ones:📌AWS Lambda: Run your code📌Amazon API Gateway: Create and manage APIs📌AWS Step Functions: Coordinate multiple functions📌Amazon EventBridge: Build apps that respond to events📌AWS SAM: Make serverless development easierOne useful tool for deploying serverless apps on AWS is the AWS Cloud Development Kit (CDK). It lets you set up your cloud infrastructure using regular programming languages.💡Learning ResourceTo learn how to use AWS CDK, I suggest "AWS CDK in Practice" ($27.98 $39.99). It teaches you how to build complex serverless apps easily.Learn about AWS CDKTo compare serverless options across different providers, check out "Multi-Cloud Handbook for Developers" ($27.98 $39.99).Learn about different cloud providersServerless in the Bigger PictureWhile serverless is great, it's not perfect for every situation. Sometimes, traditional cloud services might work better. For example, if you have long-running processes or steady workloads, using regular servers or containers might be better.To understand when to use serverless and when to use other options, we recommend these resources:💡Learning ResourceLearn about AWS architectureLearn about traditional cloud computingLearn about different cloud architecturesReal-world Use CasesMany companies use serverless successfully. Here are a few examples:📌Coca-Cola: Used serverless for vending machines, cutting costs by 65%.📌Netflix: Uses serverless for tasks like processing videos, handling billions of events daily.📌Zalora: Switched to serverless, reducing infrastructure costs by 60%.These companies learned some important lessons:📌Start small and gradually move more parts of your app to serverless📌Use good monitoring tools📌Use code to manage your infrastructure📌Keep improving your functions' performance and cost💡Learning ResourceTo learn more about real-world serverless uses, check out "AWS Certified Developer Associate Certification and Beyond" ($27.98$39.99). It gives practical insights into building serverless solutions on AWS.Learn about serverless development on AWSMonitoring Serverless AppsKeeping an eye on serverless apps is different from traditional apps. You need to watch:📌How different functions work together📌How long it takes for functions to start and run📌How much your functions cost to run📌Logs from all your functions💡Learning ResourceTo learn how to monitor serverless apps on AWS, I recommend the "AWS Observability Handbook" ($27.98 $39.99) It shows you how to use AWS tools to watch, track, and get alerts about your serverless apps.Learn about AWS monitoringPros:📌Can be cheaper for some types of apps📌Automatically handles more users📌Lets you release new features faster📌Lets developers focus on code📌No need to manage serversCons:📌Can be slow to start sometimes📌Limited run time (usually max 15 minutes)📌Can be harder to find and fix problems📌Might be hard to switch to a different provider📌Less control over the underlying systemConclusionServerless is a powerful way to build apps that can handle any number of users and potentially save you money. It lets developers focus on writing code instead of managing servers.Remember, serverless isn't always the best choice. Always consider your specific needs when deciding whether to use serverless.To learn more, check out the learning resources mentioned in this guide. They'll help you become an expert in serverless computing.Happy learning, and good luck with your serverless projects!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}