Developing On AWS Memo I
這篇紀錄Developing On AWS Memo I。
Day1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
| APM : Application Performance Monitor
SigV4 : https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html (檢查對時)
檢查時間戳記是用來避免RePlay造成DDos攻擊
Throttling
使用SDKs操作S3或 DynamoDB是非同步的 需做狀態檢查
Pagination support : DyamoDB下載限制1MB , SDKs才有續傳功能
Low-level api 需自行close connection
CLI TIP:
aws —cli-auto-prompt
alias aws=‘aws —cli-auto-prompt’
aws lambda get-function —function-name
Wrapper : while loop包裝在方法裡面
The service client APIs provide a higher-level abstraction than the low-level
calls made by client APIs.
Each AWS SDK implements automatic retry logic. (AWS SDK會自動retry 2 times)
3A : atthentication authZ accountry
IAM = authentication + authZ
Accountry -> cloudTrail
Identity-based policy : Statement -> Effect Action Resource
ARN : Amazon Resource Name <—> bucket ex: “arn:{partition}:{service name}:
{region}:{account}:{bucket name or table name}”
{partition} : global = aws
{region} : us-west-z AP-northeast-1
{account} : 12碼 account id
*=wildcard
S3 bucket 有兩個component -> object & bucket (both has actions)
Bucket level + object level
Resource-based policy考試會考判讀 : Statement -> Principal (Who)
考題會問～
Required : Effect Action Resource
Optional : Principal Condition
Permission boundaries : 限縮開放的權限規則
Effect + Permission boundaries 的交集
一般不建議一個系統開一個IAM帳號 取而代之會使用Role account
Why use user role? Role-based access control : 一個使用者在特定時間當下只能
使用一個role (避免人在Multitask時犯錯)
Assume role
Assume back -> remove QA role back to develop role
Default -> always Deny
當logic conflict -> 有 Deny & Allow -> always deny
AWS Credentials : aws s3 ls —profile user1
IaC : infrastructure access code
Security credentials 建議使用instance profile -> role -> permission -> STS -> AK/SK
*Provisioned(預先配置) IOPS -> QoS
Unix -> NFS Network File System
FSx for Windows File Server (Samba) versus FSx for Lustre (DFS) = Distribution File System
EFS Standard expensive than EFS Infrequent access
S3 object include 3 parts : content meta key = abs path
S3 object I/O always equal max volume
S3 (disk=real time access) versus S3 Glacier (Tape)
30天沒被access 會從Standard to Standard-IA
One-Zone-IA : 1 copy -> 1 zone => cost down
1 data store S3 standard for 3 AZ (3 Redudant) for 99.999999999%
S3 : AWS Managed Service
https://{bucket name}.{service code}.{Region}.amazonaws.com/{Prefix}/{Name} => 沒有Folder
Object = Prefix (Create Folder) + Name
Prefix : Dev/ + Delimiter : / => Dev/a.txt (corrent)
S3 Versioning default close -> file replace
DELETE - insert Delete Marker
S3 access points
Dualstack support both IPv4 and IPv6
S3 bucket’s naming is Global but Region has to set.
Create S3 bucket is async action.
Many endpoints by functions.
Head-bucket is meta-data.
—generate-cli-skeleton output
S3 versioning 一但Enabled 就不能disabled , 只能暫時suspended
Single upload Size <= 5GB
Multipart upload : Split file part <= 5GB (local) -> initial Retrieve UploadID
超過500MB檔案就建議使用Multipart upload
Multipart upload 有續傳的能力
S3 Select 用於.csv .json 這種 semi-structure data 可以做select from where…. (不支援Group by)
Presigned URL : 在時效內(expiration date)提供URL供對方存取
IsTruncated : true -> 資料有分頁
S3 Batch : support Server to Server copy
CORS setting : AllowedOrigin
S3 Select for object -> subset -> return client.
|