Tag: Internet

Amazon SageMaker HyperPod now supports Managed Tiered KV Cache and Intelligent Routing for large language model (LLM) inference, enabling customers to optimize inference performance for long-context prompts and multi-turn conversations. Customers deploying production LLM applications need fast response times while processing lengthy documents or maintaining conversation context, but traditional inference approaches require recalculating attention mechanisms for all previous tokens with each new token generation, creating computational overhead and escalating costs. Managed Tiered KV Cache addresses this challenge by intelligently caching and reusing computed values, while Intelligent Routing directs requests to optimal instances. These capabilities deliver up to 40% latency reduction, 25% throughput improvement, and 25% cost savings compared to baseline configurations. The Managed Tiered KV Cache feature uses a two-tier architecture combining local CPU memory (L1) with disa

Amazon SageMaker HyperPod now supports custom Kubernetes labels and taints, enabling customers to control pod scheduling and integrate seamlessly with existing Kubernetes infrastructure. Customers deploying AI workloads on HyperPod clusters orcehstrated with EKS need precise control over workload placement to prevent expensive GPU resources from being consumed by system pods and non-AI workloads, while ensuring compatibility with custom device plugins such as EFA and NVIDIA GPU operators. Previously, customers had to manually apply labels and taints using kubectl and reapply them after every node replacement, scaling, or patching operation, creating significant operational overhead. This capability allows you to configure labels and taints at the instance group level through the CreateCluster and UpdateCluster APIs, providing a managed approach to defining and maintaining scheduling policies across the entire node lifecycle. Using the new KubernetesConfig parameter, you can

AWS announces a new warm storage tier for Amazon Kinesis Video Streams (Amazon KVS), delivering cost-effective storage for extended media retention. The standard Amazon KVS storage tier, now designated as the hot tier, remains optimized for real-time data access and short-term storage. The new warm tier enables long-term media retention with sub-second access latency at reduced storage costs. The warm storage tier enables developers of home security and enterprise video monitoring solutions to cost-effectively stream data from devices, cameras, and mobile phones while maintaining extended retention periods for video analytics and regulatory compliance. Moreover, developers now have the flexibility to configure fragment sizes based on their specific requirements — selecting smaller fragments for lower latency use cases or larger fragments to reduce ingestion costs. Both hot and warm storage tiers integrate seamles

Amazon Aurora PostgreSQL-Compatible Edition has added support for PostgreSQL versions 17.6, 16.10, 15.14, 14.19, and 13.22. The update includes the PostgreSQL community's product improvements and bug fixes, and also includes Aurora-specific enhancements. Dynamic Data Masking (DDM) (16.10 and 17.6 only) is a new database-level security feature that protects sensitive data like personally identifiable information by masking column values dynamically at query time based on role-based policies, without altering the actual stored data. This release also includes a shared plan cache, improved performance and recovery-time-objective (RTO) and improvement for Global Database switchovers. To use the new versions, create a new Aurora PostgreSQL-compatible database with just a few clicks in the Amazon RDS Management Console. You can also upgrade your existing d

Today, Amazon SageMaker HyperPod announces the general availability of new APIs that enable programmatic rebooting and replacement of SageMaker HyperPod cluster nodes. SageMaker HyperPod helps you provision resilient clusters for running machine learning (ML) workloads and developing state-of-the-art models such as large language models (LLMs), diffusion models, and foundation models (FMs). The new BatchRebootClusterNodes and BatchReplaceClusterNodes APIs enable customers to programmatically reboot or replace unresponsive or degraded cluster nodes, providing a consistent, orchestrator agnostic approach to node recovery operations. The new APIs enhance node management capabilities for both Slurm and EKS orchestrated clusters complementing existing node reboot and replacement workflows. Existing orchestrator-specific methods, such as Kubernetes labels for EKS clusters and Slurm commands for Slurm clusters, remain available alongside the newly introduced programmatic capabilit

Amazon CloudWatch now offers configuring deletion protection on your CloudWatch log groups, helping customers safeguard their critical logging data from accidental or unintended deletion. This feature provides an additional layer of protection for logs maintaining audit trails, compliance records, and operational logs that must be preserved. With deletion protection enabled, administrators can prevent unintended deletions of their most important log groups. Once enabled, log groups cannot be deleted until the protection is explicitly turned off, helping safeguard critical operational, security, and compliance data. This protection is particularly valuable for preserving audit logs and production application logs needed for troubleshooting and analysis. Log group deletion protection is available in all AWS commercial Regions. You can enable deletion protection d

Today, AWS announces that AWS Compute Optimizer now supports idle resource recommendations for NAT Gateways. With this new recommendation type, you will be able to identify NAT Gateways that are unused, resulting in cost savings. With the new unused NAT Gateway recommendation, you will be able to identify NAT Gateways that show no traffic activity over a 32-day analysis period. Compute Optimizer analyzes CloudWatch metrics including active connection count, incoming packets from source, and incoming packets from destination to validate if NAT Gateways are truly unused. To avoid recommending critical backup resources, Compute Optimizer also examines if the NAT Gateway resource is associated in any AWS Route Tables. You can view the total savings potential of these unused NAT Gateways and access detailed utilization metrics to verify unused conditions before taking action. This new feature is available in all

AWS Health now includes two new properties in its event schema - actionability and persona - enabling customers to identify the most relevant events. These properties allow organizations to programmatically identify events requiring customer action and direct them to relevant teams. The enhanced event schema is accessible through both the AWS Health API and Health EventBridge communication channels, improving operational efficiency and team coordination. AWS customers receive various operational notifications and scheduled changes, including Planned Lifecycle Events. With the new actionability property, teams can quickly distinguish between events requiring action and those shared for awareness. The persona property streamlines event routing and visibility to specific teams like security and billing, ensuring critical information reaches appropriate stakeholders. These structured properties streamline integration with existing operational tools, allowing teams to effectivel

Amazon S3 Block Public Access (BPA) now allows organization-level control through AWS Organizations, allowing you to standardize and enforce S3 public access settings across all accounts in your AWS organization through a single policy configuration. S3 Block Public Access at the organization level uses a single configuration that controls all public access settings across accounts within your organization. When you attach the policy at the root or Organizational Unit (OU)-level of your organization, it propagates to all sub-accounts within that scope, and new member accounts automatically inherit the policy. Alternatively, you can choose to apply the policy to specific accounts for more granular control. To get started, navigate to the AWS Organizations console and use the "Block all public access" checkbox or JSON editor. Additionally, you can use AWS CloudTrail to audit or keep track of policy attachment as well as enforcement for member accounts. This feature i

AWS now supports deletion vectors and row lineage as defined in the Apache Iceberg Version 3 (V3) specification. These new features are available with Apache Spark on Amazon EMR 7.12, AWS Glue, Amazon SageMaker notebooks, Amazon S3 Tables, and the AWS Glue Data Catalog. These Iceberg V3 capabilities help customers build petabyte-scale data lakes with improved performance for data modifications and functionality to easily track changed records. Deletion vectors write optimized delete files that speed up data pipelines and reduce data compaction costs. Row lineage provides metadata fields on each record to track changes with a simple SQL query, eliminating the computational expense of finding small changes in large tables. Get started creating V3 tables by setting the table property to 'format-version = 3' in the CREATE TABLE command in Spark or a SageMaker notebook. To upgrade existing tables, simply update the table property in metadata with the new format version.