AMI, EC2 Image Builder, Local EC2 Instance Store, EFS, Shared Responsibility, FSx

 AMI

  • AMI = Amazon Machine Image
  • AMI are a customization of an EC2 instance
    • You add your own software, configuration, operating system, monitoring...
    • Faster boot/ configuration time because all your software is pre-packaged
  • AMI are built for a specific region ( and can be copied across regions)
  • You can launch EC2 instances from:
    • A public AMI: AWS provided
    • Your own AMI: you make and maintain them yourself
    • An AWS Marketplace AMI: an AMI someone else made (and potentially sells)

AMI Process (from an EC2 instance)

  • Start an EC2 instance and customize it
  • Stop the instance (for data integrity)
  • Build an AMI - this will also create EBS snapshots
  • Launch instances from other AMIs

EC2 Image Builder

  • Used to automate the creation of Virtual Machines or container images
  • Automate the creation, maintain, validate and test EC2 AMIs
  • Can be run on a schedule (weekly, whenever packages are updated, etc....)
  • Free service (only pay for the underlying resources)

EC2 Instance Store

  • EBS volumes are network drives with good but "limited" performance
  • If you need a high-performance hardware disk, use EC2 Instance Store
  • Better I/O performance
  • EC2 Instance Store lose their storage if they're stopped (ephermeral)
  • Good for buffer/ cache/ scratch data/ temporary content
  • Risk of data loss if hardware fails
  • Backups and Replication are your responsibility

Local EC2 Instance Store


EFS - Elastic File System

  • Managed NFS (network file system) that can be mounted on 100s of EC2
  • EFS works with Linux EC2 instances in multi-AZ
  • Highly available, scalable, expensive (3x gp2), pay per use, no capacity planning

EBS vs EFS

EFS Infrequent Access (EFS-IA)

  • Storage class that is cost-optimized for files not accessed every day
  • Up to 92% lower cost compared to EFS Standard
  • EFS will automatically move your files to EFS-IA based on the last time they were accessed
  • Enable EFS-IA with a Lifecycle Policy
  • Example: move files that are not accessed for 60 days to EFS-IA
  • Transparent to the applications accessing EFS

Shared Responsibility Model for EC2 Storage

  • AWS responsibility
    • Infrastructure
    • Replication for data for EBS volumes & EFS drives
    • Replacing faulty hardware
    • Ensuring their employees cannot access your data
  • User responsibility
    • Setting up backup/ snapshot procedures
    • Setting up data encryption
    • Responsibility of any data on the drives
    • Understanding the risk of using EC2 Instance Store

Amazon FSx:

  • Launch 3rd party high-performance file systems on AWS
  • Fully managed service
  • FSx for Lustre
  • FSx for Windows File Server
  • FSx for NetApp ONTAP

Amazon FSx for Windows File Server

  • A fully managed, highly reliable and scalable Windows native shared file system
  • Built on Windows File Server
  • Supports SMB protocol & Windows NTFS
  • Integrated with Microsoft Active Directory
  • Can be accessed from AWS or your on-premise infrastructure

Amazon FSx for Lustre

  • A fully managed, high-performance, scalable file storage for High Performance Computing (HPC)
  • The name Lustre is derived from "Linux" and "cluster"
  • Machine Learning, Analytics, Video Processing, Financial Modeling....
  • Scaled up to 100 GB/s, millions of IOPS, sub-ms latencies


EC2 Instance Storage - Summary

  • EBS volumes:
    • network drives attached to one EC2 instance at a time
    • Mapped to an Availability Zones
    • Can use EBS Snapshots for backups / transferring EBS volumes across AZ
  • AMI: create ready-to-use EC2 instances with out customizations
  • EC2 Image Builder: automatically build, test and distribute AMIs
  • EC2 Instance Store:
    • High Performance hardware disk attached to our EC2 instance
    • Lost if our instance is stopped / terminated
  • EFS: network file system, can be attached to 100s of instances in a region
  • EFS-IA: cost-optimization storage class for infrequent accessed files
  • FSx for Windows: Network File System for Windows servers
  • FSx for Lustre: High Performance Computing Linus file system

Comments

Post a Comment

Popular posts from this blog

Machine Learning

Image
 Machine Learning Amazon Rekognition Find objects, people, text, scenes in images and videos using ML Facial analysis and facial search to do user verification, people counting Create a database of "familiar faces" or compare against celebrities Use cases: Labeling Content Moderation Text Detection Face Detection and Analysis (gender, age ranges, emotions...) Face Search and Verification Celebrity Recognition Pathing (ex: for sports game analysis) Amazon Transcribe Automatically convert speech to text Uses a deep learning process called automatic speech recognition (ASR) to convert speech to text quickly and accurately Automatically remove Personally Identifiable Information (PII) using Redaction Supports Automatic Language Identification for multi-lingual audio Use cases: transcribe customer service calls automate closed captioning and subtitling generate metadata for medis assets to create a fully searchable archive Amazon Polly Turn text into lifelike speech using deep lea...
Read more

Cloud Computing and IT

Image
Cloud Computing and IT How Website work?   Server composed of: Compute, memory, storage(file or database), network - Routers, switch, DNS server. IT terminology: Network: cables, routers and servers connected with each other. Router: A networking device that forwards data packets between computer networks. They know where to send your packets on the internet. Switch: takes a packet and send it to the correct server/client on your network. Problems: Pay rent for datacenter Pay for power, cooling and maintenance Adding and replacing hardware takes time Scaling is limited Hire 24/7 team to monitor the infrastructure Plan for taking care of disasters. eg: Earth-Quake, Power-Supply issue etc... Demand: How to externalize all above task? Solution: Cloud Cloud Computing: (on-demand/pay-as-you-go/) Cloud computing is the on-demand delivery of compute power, database storage, applications and other IT resources. Cloud services are pay-as-you-go pricing. Provision exactly the right type and ...
Read more

Cloud Monitoring

Image
 Cloud Monitoring Amazon CloudWatch Metrics CloudWatch provides metrics for every services in AWS Metric is a variable to monitor ( CPUUtilization, Networking...) Metrics have timestamps Can create CloudWatch dashboards of metrics Important Metrics EC2 instances: CPU Utilization, Status Checks, Network (not RAM) Default metrics every 5 minutes Option for Detailed Monitoring ($$$): metrics every 1 minute EBS volumes: Disk Read/Writes S3 buckets: BucketSizeBytes, NumberOfObjects, AllRequests Billing: Total Estimated Charge (only in us-east-1) Service Limits: how many you've been using a service API Custom mertics: push your own metrics Amazon CloudWatch Alarms Alarms are used to trigger notifications for any metric Alarms actions Auto Scaling: increase or decrease EC2 instances "desired" count EC2 Actions: stop, terminate, reboot or recover an EC2 instance SNS notifications: send a notification into an SNS topic Various options (sampling, %, max, min, etc...) Can choose the...
Read more