AMI Build Guide#

本章详细介绍了作为一个开发者, 应该如何使用这个库来管理复杂的 packer build AMI 项目.

The workflow Folder#

Folder Structure

在这个 Repo 的根目录下有一个 workflow 目录. 里面包含一个 workflow_param.json 和一堆文件夹. 里面的目录结构大致长下面这样.

/workflow/
/workflow/{step1_workspace}/
/workflow/{step2_workspace}/
/workflow/.../
/workflow/find_root_base_image_id.py
/workflow/workflow_param.json

如果你还记得 Workflow and Step Strategy 中提到的我们将一个 AMI 的多个步骤拆分的策略, 整个 workflow 就是一个 workflow. 而这里的每个子目录就是一个 Step.

Step1

而 step1 是一个特殊的 Step. 它是这个 workflow 中的第一个 step, 同时它给出了一个典型的 step 的目录下的代码结构. 所有真正要用的 step 都是用这个 step1 作为模板来创建的.

Find root base image id script

通常一个 workflow 起始于一个 base image, 它被称为整个 workflow 中所有 step 的 root base image. find_root_base_image_id.py 是一个脚本筛选 base image 的. 在这个例子中, 我们筛选出指定 ubuntu 发行版中的最新版本作为 root base image. 获得了 image id 和 image name 之后我们就可以将其填入 workflow_param.json 文件中 (详情请看下一节).

find_root_base_image_id.py
 1# -*- coding: utf-8 -*-
 2
 3"""
 4这个脚本能帮你找到合适的由 ubuntu 官方提供的 AWS AMI 作为 base image. 一旦找到之后,
 5就可以将 id 和 name 填入 ``workflow_param.json`` 文件中了.
 6
 7你可以参考 ubuntu 官方的
 8`Find Ubuntu images on AWS <https://documentation.ubuntu.com/aws/en/latest/aws-how-to/instances/find-ubuntu-images/>`_
 9文档来了解具体方法. 本脚本只是自动化了这个方法的.
10
11下面是我在 2024-06-13 从上面的文档中找到的一些信息, 我自己留个档:
12
13The format for the parameter is:
14
15    ubuntu/$PRODUCT/$RELEASE/stable/current/$ARCH/$VIRT_TYPE/$VOL_TYPE/ami-id
16
17- PRODUCT: server, server-minimal or pro-server
18- RELEASE: jammy, 22.04, focal, 20.04, bionic, 18.04, xenial, or 16.04
19- ARCH: amd64 or arm64
20- VIRT_TYPE: pv or hvm
21- VOL_TYPE: ebs-gp3 (for >=23.10), ebs-gp2 (for <=23.04), ebs-io1, ebs-standard, or instance-store
22"""
23
24from pathlib_mate import Path
25from boto_session_manager import BotoSesManager
26from rich import print as rprint
27
28import packer_ami_workflow.api as paw
29
30# ------------------------------------------------------------------------------
31# 根据 ubuntu 的版本, 以及 arch (AMD64 还是 ARM) 来找到合适的 root base ami
32# 其中 owner_account_id 来自于本脚本最前面的 reference 文档
33aws_profile = "bmt_app_dev_us_east_1"
34root_base_ami_name = "ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"
35root_base_ami_owner_account_id = "099720109477"
36# ------------------------------------------------------------------------------
37bsm = BotoSesManager(profile_name=aws_profile)
38
39# locate the latest root base ami, this is my own implementation of the
40# packer's source_ami_filter feature, for better control
41images = paw.find_root_base_ami(
42    ec2_client=bsm.ec2_client,
43    source_ami_name=root_base_ami_name,
44    source_ami_owner_account_id=root_base_ami_owner_account_id,
45)
46latest_root_base_ami = images[0]
47ami_catalog_url = (
48    f"https://{bsm.aws_region}.console.aws.amazon.com/ec2"
49    f"/home?region={bsm.aws_region}#AMICatalog:"
50)
51print("Root base AMI details:")
52rprint(latest_root_base_ami)
53print(f"Root base AMI id = {latest_root_base_ami.id}")
54print(f"Root base AMI name = {latest_root_base_ami.name}")
55print(
56    f"Enter the AMI id in ami catalog url to see the details "
57    f"(in Community AMIs tab): {ami_catalog_url}"
58)

Workflow Parameter JSON File

这些 Step 的 packer template 中都会有很多 parameter, 而这里很多 Step 的 parameter 都是一样的. 而 /workflow/workflow_param.json 就保存了这些通用的 parameter 的值.

workflow_param.json
 1// read :class:`acore_ami.workspace.WorkflowParam` for more details
 2{
 3    "workflow_id": "paw-2024-06-13-12-00-00",
 4    "profile_name": "bmt_app_dev_us_east_1",
 5    "vpc_name": "aws_landing_zone-dev-vpc",
 6    "is_default_vpc": "false", // please use string
 7    "subnet_name":  "aws_landing_zone-dev/public/1",
 8    "security_group_name": "aws_landing_zone-dev/sg/allow-restricted-traffic-from-authorized-ip",
 9    "ec2_iam_role_name": "ec2-admin-role",
10    "root_base_ami_id": "ami-0975ad60e5054592a",
11    "root_base_ami_name": "ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-20240607",
12    "aws_tags": {
13        "tech:project_name": "packer_ami_workflow",
14        "tech:env_name": "devops",
15        "tech:version": "v1",
16        "tech:human_creator": "Sanhe Hu",
17        "tech:machine_creator": "packer",
18        "auto:delete_at": "na",
19        "bus:ou": "app",
20        "bus:team": "app",
21        "bus:project": "packer_ami_workflow",
22        "bus:owner": "Sanhe Hu"
23    }
24}

Per Step Folder#

下面我们进到一个 具体的 Step1 目录里 看看每个 Step 的 packer template 应该怎么写. 下面列出了 Step 的 workspace 的目录结构.

# 核心文件
/workspace/
/workspace/templates/
/workspace/templates/.pkr.hcl
/workspace/templates/.pkrvars.hcl
/workspace/templates/.variables.pkr.hcl
/workspace/.gitignore
/workspace/packer_build.py # <--- 用这个脚本运行 packer build
/workspace/param.json

# 这个例子展示了在无需任何 python 库的情况下, 使用 python shell script 来实现 provision 逻辑
/workspace/zero_deps_script.py
# 这个例子展示了在需要少量 python 库的情况下, 使用 python shell script 来实现 provision 逻辑
/workspace/some_deps_script.py
# 这个例子展示了如何实现非常复杂的 provision 逻辑
# 其核心就是用 .sh 来给 Python 安装依赖
# 然后用 .py 来实现复杂的 provision 逻辑
/workspace/complicated_script.py
/workspace/complicated_script.sh
/workspace/README.rst

在详细展开之前, 我们先来了解一下 /workspace/templates/ 目录:

Prepare Packer Templates#

Packer 原生的 Template 本质上相当于一个 declaration (声明式) 的脚本. 这有点类似于 CloudFormation, 它不是面像过程, 而是声明式的. 但是它有着声明式脚本的通用缺点, 自动化程度不高, 参数化系统不够灵活, 你无法基于 parameter 来用 if else, for loop 等对整个 template 的结构进行控制. 所以我在 Template 上又用 jinja2 模板引擎封装了一层 (这跟我初期改进 CloudFormation 流程的做法类似). 具体来说整个开发流程是这样的:

  1. 用 jinja2 语言写 hcl 模板. 其中使用一个 params Python 对象作为所有的 parameter 的 container, 然后用 {{ params.parameter_name }} 这样的语法来插入参数. 所有的 jinja2 模板都放在 templates 目录下.

  2. 在 Python 脚本中生成 params 对象, 至于 params 的数据放在哪里由开发者自己决定. 一般是放在 JSON 里.

  3. 用 jinja2 语言 render 最终的 hcl 文件, 并将其放在 Step 的目录下.

其中在 #1 这一步, 我们有三个关键文件:

在编写 *.pkr.hcl 的时候, 所有在 packer template 中以 string replacement 存在的参数 (例如 ami_name      = var.output_ami_name) 都需要在 *.variables.pkr.hcl 中定义. 这样能充分利用 packer 的 declaration 语法记录每个 variable 是用来干什么的. 请不要用 {{ param.output_ami_name }} 这样的语法直接替换掉里面的值, 这样做会降低代码的可维护性. 而如果是用来控制 template 结构的参数我们就不要放在 *.variables.pkr.hcl 中了. 我认为不应该用 jinja2 template 来完全替代 packer 的 variables 系统, 因为 jinja2 主要是一个 string template engine, 插入值的时候并不会检查类型, 所以我们只用 jinja2 来做 string manipulation, if/else, for loop.

下面我们给出了在 step1 中的这三个关键文件的源代码:

Important

.pkr.hcl 最为重要, 请仔细阅读其中的注释. 特别是里面关与如何用复杂的 Python 自动化脚本来执行 provision 的相关介绍.

.pkr.hcl
  1packer {
  2  required_plugins {
  3    amazon = {
  4      source  = "github.com/hashicorp/amazon"
  5      version = "~> 1"
  6    }
  7  }
  8}
  9
 10source "amazon-ebs" "ubuntu20" {
 11  ami_name      = var.output_ami_name
 12  instance_type = "t2.micro"
 13  region        = var.aws_region
 14  ssh_username  = "ubuntu"
 15
 16  /*----------------------------------------------------------------------------
 17  You can either explicitly specify the ``source_ami`` field or use the ``source_ami_filter``
 18  to find the AMI ID automatically. I personal prefer to provide the ``source_ami``
 19  explicitly for better control.
 20  https://developer.hashicorp.com/packer/integrations/hashicorp/amazon/latest/components/builder/ebs
 21  ----------------------------------------------------------------------------*/
 22  source_ami = var.source_ami_id
 23
 24#  source_ami_filter {
 25#    filters = {
 26#      name                = var.source_ami_name
 27#      root-device-type    = "ebs"
 28#      virtualization-type = "hvm"
 29#    }
 30#    most_recent = true
 31#    owners      = [var.source_ami_owner_account_id]
 32#  }
 33
 34  /*----------------------------------------------------------------------------
 35  If you want to build on a custom VPC, you can uncomment the following block
 36  ----------------------------------------------------------------------------*/
 37  # if none default VPC, you need to explicitly set this to true
 38  associate_public_ip_address = true
 39
 40  # you don't have to set the VPC explicitly if you specified the subnet.
 41#  vpc_filter {
 42#    filters = {
 43#      "tag:Name": var.vpc_name,
 44#      "isDefault": var.is_default_vpc,
 45#    }
 46#  }
 47
 48  # make sure you are using a public subnet
 49  subnet_filter {
 50    filters = {
 51      "tag:Name": var.subnet_name,
 52    }
 53    most_free = true
 54    random = false
 55  }
 56
 57  # make sure the security group has ssh inbound rule
 58  security_group_filter {
 59    filters = {
 60      "tag:Name": var.security_group_name,
 61    }
 62  }
 63
 64  /*----------------------------------------------------------------------------
 65  If you want to use a custom IAM role, you can use ``iam_instance_profile``
 66  ----------------------------------------------------------------------------*/
 67  iam_instance_profile = var.ec2_iam_role_name
 68
 69  /*----------------------------------------------------------------------------
 70  If you need to add additional volume to your AMI, you can do it here
 71  ----------------------------------------------------------------------------*/
 72#  launch_block_device_mappings {
 73#      device_name = "/dev/sda1"
 74#      # in the most of the cases, you should set delete_on_termination = true
 75#      # the AMI has the snapshot of the volume already. When you use the output
 76#      # of this build as a image, it will create a ebs volume from the snapshot.
 77#      # If you set delete_on_termination = false, you will end up with a volume
 78#      # after the build and you have to clean up your self
 79#      delete_on_termination = true
 80#      /*
 81#      gp3 would be the optimal choice for most of the cases since Dec 2020
 82#
 83#      reference:
 84#
 85#      - Introducing new Amazon EBS general purpose volumes, gp3: https://aws.amazon.com/about-aws/whats-new/2020/12/introducing-new-amazon-ebs-general-purpose-volumes-gp3/
 86#      - Migrate your Amazon EBS volumes from gp2 to gp3 and save up to 20% on costs: https://aws.amazon.com/blogs/storage/migrate-your-amazon-ebs-volumes-from-gp2-to-gp3-and-save-up-to-20-on-costs/
 87#      */
 88#      volume_type = "gp3"
 89#      volume_size = 30
 90#  }
 91#
 92#  ami_block_device_mappings {
 93#     device_name = "/dev/sda1"
 94#     delete_on_termination = true
 95#     volume_type = "gp3"
 96#  }
 97}
 98
 99build {
100  name    = "install python"
101  sources = [
102    "source.amazon-ebs.ubuntu20"
103  ]
104
105  provisioner "shell" {
106    inline = [
107      "sleep 10",
108      # verify ebs attachment
109      "lsblk",
110      "df -h",
111    ]
112  }
113
114
115  /*----------------------------------------------------------------------------
116  if you need to sudo install something, do it in the ``inline`` block
117  ----------------------------------------------------------------------------*/
118  provisioner "shell" {
119    inline = [
120      "sudo apt-get install -y curl",
121      "sudo apt-get install -y wget",
122      "sudo apt-get install -y git",
123      "sudo apt-get install -y unzip",
124    ]
125  }
126
127
128  /*----------------------------------------------------------------------------
129  if you need to run complicate logic in Python, and it has zero dependency
130  and fits in one file, you can upload the script to the server and run it.
131  make sure you have the right shebang ``#!/usr/bin/env python`` in the first line
132  ----------------------------------------------------------------------------*/
133  provisioner "shell" {
134    script = "zero_deps_script.py"
135  }
136
137
138  /*----------------------------------------------------------------------------
139  if your Python script has some simple dependencies, please pre-configure
140  the pyenv https://github.com/pyenv/pyenv and then use the pyenv to install
141  some user Python versions, and then use the user Python to install dependencies
142  and run code. DON't directly install anything to the system Python
143  ----------------------------------------------------------------------------*/
144#  provisioner "file" {
145#    source = "requirements.txt"
146#    destination = "/tmp/requirements.txt"
147#  }
148#
149#  provisioner "file" {
150#    source = "some_deps_script.py"
151#    destination = "/tmp/some_deps_script.py"
152#  }
153#
154#  provisioner "shell" {
155#    inline = [
156#      "~/.pyenv/shims/pip install -r /tmp/requirements.txt",
157#      "~/.pyenv/shims/python /tmp/some_deps_script.py",
158#    ]
159#  }
160
161
162  /*----------------------------------------------------------------------------
163  if you need to run super complicate logic in Python, and you need split your code
164  into modules and create a Python library for it, this is the solution.
165
166  You should create a bash script to prepare the Python virtualenv,
167  then explicitly use the virtualenv Python interpreter to run your scripts.
168  Please read the sample ``complicated_script.sh`` for more details.
169  ----------------------------------------------------------------------------*/
170#  provisioner "shell" {
171#    script = "complicated_script.sh"
172#  }
173}
.pkrvars.hcl
1source_ami_id = "{{ builder.source_ami_id }}"
2output_ami_name = "{{ builder.output_ami_name }}"
3aws_region = "{{ builder.workflow_param.aws_region }}"
4vpc_name = "{{ builder.workflow_param.vpc_name }}"
5is_default_vpc = "{{ builder.workflow_param.is_default_vpc }}"
6subnet_name = "{{ builder.workflow_param.subnet_name }}"
7security_group_name = "{{ builder.workflow_param.security_group_name }}"
8ec2_iam_role_name = "{{ builder.workflow_param.ec2_iam_role_name }}"
.variables.pkr.hcl
 1variable "source_ami_id" {
 2  type        = string
 3  description = "Base AMI ID to use for building this AMI, I prefer to explicitly provide this value."
 4}
 5
 6variable "output_ami_name" {
 7  type        = string
 8  description = "The generated AMI name, it has to be unique in a region."
 9}
10
11
12variable "aws_region" {
13  type        = string
14  description = "The AWS region where the AMI will be created."
15}
16
17variable "vpc_name" {
18  type        = string
19  description = "The VPC name where the packer build will run."
20}
21
22variable "is_default_vpc" {
23  type        = string
24  description = "are we using default VPC? use false or true (string, not boolean)."
25}
26
27variable "subnet_name" {
28  type        = string
29  description = "The Subnet name where the packer build will run."
30}
31
32variable "security_group_name" {
33  type        = string
34  description = "The Security name where the packer build will use."
35}
36
37variable "ec2_iam_role_name" {
38  type        = string
39  description = "The IAM role name that the packer build will use."
40}

Step Level Parameter#

和前面 workflow_param.json 类似, step_param.json 保存了跟这个 step 相关的一些参数. 其中最关键的就是这一步的 step id 和前一步的 step id. 如果当前 step 就是第一步, 那么 previous_step_id 就是 None.

step_param.json
1{
2    "step_id": "step1",
3    "previous_step_id": null,
4    "metadata": {
5        "what_is_included": "step 1 stuff"
6    }
7}

Manage AMIs#

AWS 官方有很多 AMI API 可以进行 list, get details 等操作. 但是灵活性还是远远不如用数据库来管理 metadata. 所以在这个项目中我们会用 DynamoDB 来管理 AMI 的 metadata, 使得我们可以更方便地操作 AMI.

AmiData 是一个 ORM 类, 它能让开发者用 Pythonic 的方式操作 DynamoDB, 并封装了常用的 query pattern, 例如:

Packer Build Script#

Important

这一步就是我们真正作为一个 AMI 的维护着要动手写的部分了.

这个 packer_ami_workflow/tests/example.py 是一个非常薄的 wrapper, 把 packer_ami_workflow 库的 utility 扩展, 并封装了一下. 它展示了你如何扩展默认的 WorkflowParam 和 StepParam 类, 如何指定 AmiData DynamoDB Table 的名字.

packer_ami_workflow/tests/example.py
 1# -*- coding: utf-8 -*-
 2
 3import dataclasses
 4from pathlib_mate import Path
 5import pynamodb_mate.api as pm
 6
 7import packer_ami_workflow.api as paw
 8
 9
10@dataclasses.dataclass
11class WorkflowParam(paw.WorkflowParam):
12    pass
13
14
15@dataclasses.dataclass
16class StepParam(paw.StepParam):
17    pass
18
19
20class AmiData(paw.AmiData):
21    class Meta:
22        table_name = "packer_ami_workflow_example"
23        billing_mode = pm.constants.PAY_PER_REQUEST_BILLING_MODE
24
25
26class AmiBuilder(paw.AmiBuilder):
27    @classmethod
28    def make_builder(
29        cls,
30        dir_step: Path,
31    ):
32        return cls.make(
33            dir_step=dir_step,
34            table_class=AmiData,
35            workflow_param_class=WorkflowParam,
36            step_param_class=StepParam,
37        )

有了这个 wrapper 之后, 开发者唯一要做的事情就只有三个:

  1. 编写 /workflow/step1/template 中的 packer template 的逻辑. 具体语法和细节你可以参考 packer 的官方文档.

  2. 填写 /workflow/workflow_param.json 和 /workflow/step1/step_param.json 配置文件.

  3. 运行 /workflow/step1/packer_build.py 脚本.

下面我们来详细讲一讲 packer_build.py 脚本的结构. 首先, 我们来看一下这个脚本的源码.

Important

packer_build.py 也是我们的核心脚本之一, 我建议仔细阅读 packer_build.py 源码中的注释来了解这个脚本的逻辑.

packer_build.py
 1# -*- coding: utf-8 -*-
 2
 3from pathlib_mate import Path
 4from packer_ami_workflow.tests.example import AmiBuilder
 5
 6dir_here = Path.dir_here(__file__)
 7builder = AmiBuilder.make_builder(dir_step=dir_here)
 8
 9# ------------------------------------------------------------------------------
10# Use this to create a new AMI image if using packer build
11#   dry_run = True: NOTHING happen, dry_run = False: run packer build
12# ------------------------------------------------------------------------------
13builder.run_packer_build_workflow(dry_run=True)
14builder.tag_ami()
15# ------------------------------------------------------------------------------
16# Use this to create a new AMI image from a stopped EC2 instance
17#   instance_id: the EC2 instance id
18# ------------------------------------------------------------------------------
19# builder.create_image_manually(instance_id="i-1a2b3c", wait=True)
20# ------------------------------------------------------------------------------
21# Create a new item in the DynamoDB table
22# ------------------------------------------------------------------------------
23builder.create_dynamodb_item()
24# ------------------------------------------------------------------------------
25# Deregister the AMI image and delete the associated snapshot (optional)
26# ------------------------------------------------------------------------------
27# builder.delete_ami(delete_snapshot=False, skip_prompt=False)

这个脚本的内容很简单:

  1. 创建一个 AmiBuilder 对象, 这个对象在前面提到的 packer_ami_workflow/tests/example.py wrapper 中已经写好了.

builder = AmiBuilder.make_builder(dir_step=dir_here)

  1. 用 packer build 命令创建 AMI.

# dry_run is True = NOTHING happen, False = run packer build
builder.run_packer_build_workflow(dry_run=True)

  1. 给 AMI 打 AWS Tags.

builder.tag_ami()

  1. 在 DynamoDB 中创建一条记录.

builder.create_dynamodb_item()

  1. (Optional) 删除 AMI, 并可以选择是否同时删除 snapshot.

builder.delete_ami(delete_snapshot=False, skip_prompt=False)

还有一种特殊情况是, 这个 packer template 中有一些步骤真的无法通过自动化完成, 那么你可以手动用前一步的 AMI 创建 EC2, 然后 SSH 进去, 手动 provision 环境, 退出然后 stop instance, 手动 create image, 然后 terminate instance. (我这里有个小工具可以方便的 SSH 到 EC2 ssh2awsec2). 下面是一个例子:

# 手动填写这个 ec2 instance id
builder.create_image_manually(instance_id="i-a1b2c3d4")