Code

Data warehouse for access logs#46

Code

Merged

kikuomax merged 41 commits intocodemonger-io:maincodemonger-io/codemonger:mainfrom kikuomax:access-logskikuomax/codemonger:access-logsCopy head branch name to clipboardOct 19, 2022

+4,316-1,300

Commits

Commits on Sep 24, 2022

• feat(cdk): output access logs bucket

  Show description for 3ca9dec
  kikuomax

  committed
  3ca9dec

• chore(cdk-ops): update CDK

  Show description for 1a13ccf
  kikuomax

  committed
  1a13ccf

• fix(cdk-ops): name of the main stack

  Show description for ae2887c
  kikuomax

  committed
  ae2887c

• feat(cdk-ops): resolve access logs bucket

  Show description for 16042a7
  kikuomax

  committed
  16042a7

• feat(cdk-ops): mask access logs

  Show description for 0211abb
  kikuomax

  committed
  0211abb

• feat(cdk-ops): provision AccessLogsMasking

  Show description for 827846f
  kikuomax

  committed
  827846f

• feat(cdk-ops): process S3 event via SQS

  Show description for 714c370
  kikuomax

  committed
  714c370

• feat(cdk-ops): ignores non-existing logs

  Show description for e459feb
  kikuomax

  committed
  e459feb

• feat(cdk-ops): delete masked access logs

  Show description for 3af141e
  kikuomax

  committed
  3af141e

• chore(cdk-ops): rename AccessLogsMasking

  Show description for 3433cd6
  kikuomax

  committed
  3433cd6

Commits on Sep 26, 2022

• feat(cdk-ops): prepend prefix to masked logs

  Show description for 1fffb18
  kikuomax

  committed
  1fffb18

• feat(cdk-ops): prefix date

  Show description for 58327f3
  kikuomax

  committed
  58327f3

Commits on Oct 1, 2022

• feat(cdk-ops): configure env

  Show description for 344b6d3
  kikuomax

  committed
  344b6d3

• feat(cdk-ops): latest boto3 layer

  Show description for 91dfd32
  kikuomax

  committed
  91dfd32

• feat(cdk-ops): provision data warehouse for access logs

  Show description for b2b6987
  kikuomax

  committed
  b2b6987

Commits on Oct 8, 2022

• chore(cdk-ops): install cdk2-python-library-layer

  Show description for 268279f
  kikuomax

  committed
  268279f

• feat(cdk-ops): add libdatawarehouse

  Show description for 570d151
  kikuomax

  committed
  570d151

• feat(cdk-ops): use libdatawarehouse

  Show description for 29b0b25
  kikuomax

  committed
  29b0b25

• feat(cdk-ops): load access logs

  Show description for a9796ab
  kikuomax

  committed
  a9796ab

• feat(cdk-ops): add sequential numbers to rows

  Show description for 68efb84
  kikuomax

  committed
  68efb84

Commits on Oct 9, 2022

• fix(cdk-ops): check existence of access logs

  Show description for 27d4910
  kikuomax

  committed
  27d4910

• chore(cdk-ops): add thoughts about VACUUM

  Show description for b0d9808
  kikuomax

  committed
  b0d9808

Commits on Oct 11, 2022

• feat(cdk-ops): schedule loading access logs

  Show description for b669dc0
  kikuomax

  committed
  b669dc0

• fix(cdk-ops): ignore invalid keys

  Show description for ae03435
  kikuomax

  committed
  ae03435

• feat(cdk-ops): add vacuum workflow

  Show description for e7a8ea8
  kikuomax

  committed
  e7a8ea8

• feat(cdk-ops): start VACUUM after load

  Show description for 8a58bc9
  kikuomax

  committed
  8a58bc9

• chore(cdk-ops): output ARN of load-access-logs

  Show description for 605f682
  kikuomax

  committed
  605f682

• chore(cdk-ops): install @aws-sdk/client-lambda

  Show description for c60c541
  kikuomax

  committed
  c60c541

• feat(cdk-ops): add data warehouse population script

  Show description for 6d7ac60
  kikuomax

  committed
  6d7ac60

• feat(cdk-ops): add production data warehouse

  Show description for 27d794b
  kikuomax

  committed
  27d794b

Commits on Oct 14, 2022

• fix(cdk-ops): intuitive timeout

  Show description for 00f8ea7
  kikuomax

  committed
  00f8ea7

• feat(cdk-ops): enable enhanced VPC routing

  Show description for 284ae05
  kikuomax

  committed
  284ae05

Commits on Oct 15, 2022

• fix(cdk-ops): NULL user_agent

  Show description for 00ab4f6
  kikuomax

  committed
  00ab4f6

Commits on Oct 18, 2022

• feat(cdk-ops): enable enhancedVpcRouting

  Show description for 39d1779
  kikuomax

  committed
  39d1779

• docs(cdk-ops): add data warehouse architecture

  Show description for 7973ee5
  kikuomax

  committed
  7973ee5

Commits on Oct 19, 2022

• docs(cdk-ops): update README

  Show description for 4b7a7b9
  kikuomax

  committed
  4b7a7b9

• docs(cdk-ops): refine data-warehouse.md

  Show description for 9b61fec
  kikuomax

  committed
  9b61fec

• docs(cdk-ops): translate data-warehouse.md → ja

  Show description for 81cea1e
  kikuomax

  committed
  81cea1e

• docs(cdk-ops): translate README → ja

  Show description for b2b47ad
  kikuomax

  committed
  b2b47ad

• docs(cdk-ops): link to Redshift admin info

  Show description for 8ba5e96
  kikuomax

  committed
  8ba5e96

• docs(cdk-ops): update README

  Show description for c658d26
  kikuomax

  committed
  c658d26