AWS CLI -- 第三步 EC2 定時自動開關機

 

 3.1創立 IAM 角色

開一個給lamda用的role

aws iam create-role --role-name lambda-ex --assume-role-policy-document '{"Version": "2012-10-17","Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'

arn:aws:iam::1234:role/lambda-ex 

創立一個json (定義ec2開關機權限)

vi policy.json 

{ 
  "Version": "2012-10-17", 
  "Statement": 
[ 
 { 
   "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], 
    "Resource": "arn:aws:logs:*:*:*" }, 
 { 
   "Effect": "Allow", "Action": [ "ec2:Start*", "ec2:Stop*" ], "Resource": "*" 
 } 
] 
}

 創立一個policy 

aws iam create-policy --policy-name ec2-start-stop-policy --policy-document file://policy.json 

 arn:aws:iam::1234:policy/ec2-start-stop-policy

新增這個policy到role裡

aws iam attach-role-policy --role-name lambda-ex --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

aws iam attach-role-policy --role-name lambda-ex --policy-arn  arn:aws:iam::1234:policy/ec2-start-stop-policy

記下他的ARN

3.2 新增lamda 函式

vi start.py

 

import boto3

region = 'ap-northeast-1'

instances = ['i-1234']

 

def handler (event, context):

    ec2 = boto3.client('ec2', region_name=region)

    ec2.start_instances(InstanceIds=instances)

    print('started your instances: ' + str(instances))

 

vi stop.py

 

import boto3

region = 'ap-northeast-1'

instances = ['i-1234']

 

def handler (event, context):

    ec2 = boto3.client('ec2', region_name=region)

    ec2.stop_instances (InstanceIds=instances)

    print('stopped your instances: ' + str(instances)) 

 

zip start.zip start.py 

zip stop.zip stop.py 

 

 使用3.1的ARN

aws lambda create-function --function-name start_function \
--zip-file fileb://start.zip --handler start.handler --runtime python3.7 \
--role arn:aws:iam::1234:role/lambda-ex 

 aws lambda create-function --function-name stop_function \
--zip-file fileb://stop.zip --handler stop.handler --runtime python3.7 \
--role arn:aws:iam::1234:role/lambda-ex 

3.3 新增排程

 1-5 早上八點開機

aws events put-rule --name "start_function" --schedule-expression "cron(0 0 ? * MON-FRI *)" 

  1-5 下午六點關機

 aws events put-rule --name "stop_function" --schedule-expression "cron(0 10 ? * MON-FRI *)" 

AWS CLI -- 第二步 建立 Ec2

 

 2.0 確認網路環境

 

( VPC 是AWS的一種虛擬網路服務

確認要使用的VPC id及其 subnet id << 要跟AMI同一地區

 

subnet-1234

vpc-1234

 2.1創立keypair

aws ec2 create-key-pair --key-name serverKeyPair

把顯示的文字存成pem檔

aws ec2 describe-key-pairs --key-name serverKeyPair

2.2 創立Security Group

 

 

創立Security Group

aws ec2 create-security-group --group-name serverSecurityGroup --description "server security group" --vpc-id vpc-1234

>>會回你創好的SGid

我們把這組SG的 1883 + 22 port 打開

 aws ec2 authorize-security-group-ingress --group-id sg-1234--protocol tcp --port 1883 --cidr 0.0.0.0/0

 aws ec2 authorize-security-group-ingress --group-id sg-1234--protocol tcp --port 22 --cidr 0.0.0.0/0

檢查一下現在的設定值 

aws ec2 describe-security-groups --group-ids sg-1234

2.3 搜尋要用的作業系統

找到自己要用的AMI

https://cloud-images.ubuntu.com/locator/

Amazon AWS

ap-northeast-1

bionic

18.04

amd64

hvm-ssd

20220610

ami-0a2e24115d3ca5d76

2.4 創立EC2 instances

aws ec2 run-instances \
    --image-id ami-0a2e24115d3ca5d76 \
    --instance-type t2.micro \
    --subnet-id subnet-1234\
    --security-group-ids sg-1234\
    --associate-public-ip-address \
    --key-name serverKeyPair

 >>查詢其中的instance-ids

i-1234

aws ec2 describe-instances \
    --instance-ids i-1234

 aws ec2 describe-instances --instance-ids i-1234--query 'Reservations[].Instances[].PublicDnsName'

>>查詢其中的 PublicDnsName 

1234.ap-northeast-1.compute.amazonaws.com

2.5 登入並使用server

 還記得2.1用的pem檔嗎

 

可以使用linux mac 內建的ssh指令

 ssh -i "serverKeyPair.pem" ubuntu@1234.ap-northeast-1.compute.amazonaws.com

 

或是 windows 使用putty 做登入 

 (要先使用puttygen 轉換pem 為ppk檔

 hostname :

ubuntu@1234.ap-northeast-1.compute.amazonaws.com

 port:  22

Connection >> SSH >> Auth >> 選擇PPK檔

好用的指令

開機 

aws ec2 start-instances --instance-ids i-1234

關機 

aws ec2 stop-instances --instance-ids i-1234

重開機 

aws ec2 reboot-instances --instance-ids i-1234

AWS CLI -- 第一步 新增IAM 帳號

可以local 使用AWS shell

https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html

或是直接用cloudshell

https://us-east-1.console.aws.amazon.com/cloudshell
 

1. 創建admin account

 

1.1  創建admin group

aws iam create-group --group-name Admins

aws iam list-groups 

1.2  連接admin group policy

aws iam attach-group-policy --group-name Admins --policy-arn arn:aws:iam::aws:policy/AdministratorAccess 

aws iam list-attached-group-policies --group-name Admins 

1.3 創建admin user

aws iam create-user --user-name admin 

1.4 創建password

串建空白範本 

aws iam create-login-profile --generate-cli-skeleton > create-login-profile.json 

修改為需要的帳號密碼(請注意密碼有最低安全限制

 vi  create-login-profile.json 

{
    "UserName": "admin",
    "Password": "admin",
    "PasswordResetRequired": true
} 

aws iam create-login-profile --cli-input-json file://create-login-profile.json 

 1.5 加入user group

aws iam add-user-to-group --user-name admin --group-name Admins 

如何在一般docker container 裡用USB硬碟?

首先拿到USB硬碟mount完的路徑

device 是 /dev/usb/bus/001/014

但是我們要拿的是可以存取的資料夾路徑

譬如 /media/lab734/Kingston

在docker create時加上 -v /media/lab734/Kingston:/mnt/usb/Kingston

container裡面會有/mnt/usb/Kingston 裡面就是USB硬碟的資料了

一般docker 操作

去找到你要複製的image 和TAG

tensorflow/tensorflow:2.0.0b1-gpu-py3-jupyter

創一個新的container

sudo nvidia-docker create

--name lab734docker //名子

-p 7777:8888 //本機跟虛擬機的port-forwaording
--privileged=true //開放真的root權限

--device /dev/nvidia-uvm:/dev/nvidia-uvm //把顯卡映射進來

-it //來個可以互動的命令列 方便之後操作

-v /media/lab734/Kingston:/mnt/usb/Kingston

//映射資料夾 這裡是用上面的USB硬碟

tensorflow/tensorflow:2.0.0b1-gpu-py3-jupyter

//來源的image

sudo nvidia-docker create --name lab734docker -p 7777:8888 --privileged=true --device /dev/nvidia-uvm:/dev/nvidia-uvm --device /dev/nvidia0:/dev/nvidia0 --device /dev/nvidiactl:/dev/nvidiactl -it -v /home/lab734:/home/usr tensorflow/tensorflow:2.0.0b1-gpu-py3-jupyter 

docker start lab734docker

//開機~

docker ps//現在開了哪幾台

docker exec -it lab734docker /bin/bash
//進入shell打指令 ctrl +D 退出

docker stop lab734docker
//停掉

docker rm lab734docker

//移除container 
apt-get install update

pip3 install --upgrade pip
pip install matplotlib pillow opencv-python==3.4.2.16
apt-get install libgtk2.0-dev


docker exec -it CH /bin/bash

全新電腦灌ubuntu 18.04和一般深度學習的DOCKER

先到ubuntu 官網抓18.04

灌到USB裡 重灌電腦

網路設定好

sudo apt-get update

sudo apt-get install openssh-server

sudo reboot

sudo apt install gcc



sudo apt install build-essential
去官網載新版的driver
sudo chmod +x N@@@@@
./N@@@@@@ 
sudo reboot
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

sudo apt-get install docker-ce docker-ce-cli containerd.io

sudo docker run hello-world







curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
  sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update



sudo apt-get install -y nvidia-docker2
sudo pkill -SIGHUP dockerd



sudo usermod -a -G docker $USER



docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi




sudo nvidia-docker create --name lab734docker  -p 7777:8888 -e PASSWORD=pass --privileged=true --device /dev/nvidia-uvm:/dev/nvidia-uvm --device /dev/nvidia0:/dev/nvidia0  --device /dev/nvidiactl:/dev/nvidiactl -it -v /home/lab734:/home/usr tensorflow/tensorflow:2.0.0b1-gpu-py3-jupyter 


docker exec -it lab734docker /bin/bash


 jupyter notebook password 
top
kill 1
docker start lab734docker