Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: adds concurrent_insurance_buy go script #155

Merged
merged 3 commits into from
Aug 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 12 additions & 10 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,11 @@ __pycache__
.idea
.vscode
*.egg-info
/vendor
bin
node_modules
target
venv
*.sw*
*results.csv
/.dummy
/.editorconfig
/.eslintignore
/.eslintrc
/.flake8
/.golangci.yml
Expand All @@ -20,11 +19,14 @@ venv
/easycop.yml
/easypost_java_style.xml
/layout_rules.xml
/official/tools/build_doc_json_responses/responses/
/official/tools/build_doc_json_responses/tests/cassettes/
/phpcs.xml
/pyproject.toml
/style_suppressions.xml
/.dummy
/.eslintignore
*.sw*
/official/tools/build_doc_json_responses/responses/
/official/tools/build_doc_json_responses/tests/cassettes/
bin
bulkins
node_modules
target
vendor
venv
27 changes: 27 additions & 0 deletions community/golang/concurrent_insurance_buy/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Concurrently Buy Insurance

This script allows you to concurrently buy EasyPost Insurance. It works when insuring EasyPost Shipments or when buying standalone Insurance for a package bought outside of the EasyPost ecosystem. Use the `sample.csv` file as a template, fill in the details (do not delete the header), and run the script.

## Things to Know

- You must follow the `sample.csv` template or the script will not work correctly. Do not delete the header row
- Only run the script once! Running the script multiple times with the same data will create duplicate insurance objects and fees
- The script will spin up 20 concurrent requests at a time
- Because requests are sent concurrently, they may complete in a different order than they were specified in the original CSV. Sorting the results CSV by Tracking Code and the original `sample.csv` will allow you to match up requests with input data for debugging purchases in the event of errors
- The status of each request will be saved to a CSV once the script is complete. Each line will include error messages if there were any, the time each request took, and the status of the request
- If there are errors reported in the results CSV, correct input data as necessary and run the script again
- NOTE: Ensure you delete successful rows from the `sample.csv` file before running the script again. Failure to do so will result in duplicate insurance objects and fees (eg: If I had 5 rows, 3 succeeded, 2 failed - I would remove the 3 rows with a success status of true so that my sample CSV contained the 2 rows that initially failed, I'd correct the data as needed, and re-run the script with only those two failed rows)
- It's recommended to run the `sample.csv` file as-is with a test API key to get a feel for how it works prior to loading real data and using a production API key

## Usage

```shell
EASYPOST_API_KEY=123... CSV=path/to/sample.csv go run concurrent_insurance_buy.go
```

## Development

```shell
# To build a standalone binary of this tool (eg: call it bulkins)
go build -o bulkins
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
package main

import (
"encoding/csv"
"errors"
"fmt"
"io"
"math"
"os"
"strconv"
"time"

"github.com/EasyPost/easypost-go/v4"
)

// Concurrently buy EasyPost Insurance via a CSV file
// Usage: EASYPOST_API_KEY=123... CSV=sample.csv go run concurrent_insurance_buy.go

func main() {
scriptStart := time.Now()

records := getCsvRecords()

apiKeyParam := os.Getenv("EASYPOST_API_KEY")
if apiKeyParam == "" {
handleGoErr(errors.New("EASYPOST_API_KEY param not set"))
}

client := easypost.New(apiKeyParam)
numOfGoroutines := int(math.Min(float64(len(records)), 20))
semaphore := make(chan bool, numOfGoroutines)
lineMessageList := make([][]string, 0)
lineMessageList = append(lineMessageList, []string{"Tracking Code", "Reference", "Time Elapsed", "Success", "Message"})

// Iterate over our set of data and create an Insurance record for each line in the CSV
for i, line := range records {
semaphore <- true

go func(lineNumber int, currentLine []string) {
goroutineStartTime := time.Now()

tracking_code := currentLine[0]
reference := currentLine[1]
carrier_string := currentLine[2]
amount := currentLine[3]
to_address_id := currentLine[4]
from_address_id := currentLine[5]

fmt.Printf("Sending request for %s...\n", tracking_code)
success, message := createInsurance(client, tracking_code, reference, carrier_string, amount, to_address_id, from_address_id)

elapsedTime := time.Since(goroutineStartTime)
lineMessage := []string{tracking_code, reference, elapsedTime.String(), success, message}
lineMessageList = append(lineMessageList, lineMessage)

<-semaphore
}(i, line)
}

// Gather up goroutines
for i := 0; i < cap(semaphore); i++ {
semaphore <- true
}

file, err := os.Create("insurance_buy_results.csv")
handleGoErr(err)
defer file.Close()
writer := csv.NewWriter(file)
defer writer.Flush()
writer.WriteAll(lineMessageList)

elapsedTotal := time.Since(scriptStart)
fmt.Printf("\nTotal time elapsed: %s\n", elapsedTotal)
}

// getCsvRecords builds the set of data without including the header and validates it
func getCsvRecords() [][]string {
csvParam := os.Getenv("CSV")
if csvParam == "" {
handleGoErr(errors.New("CSV parameter not set"))
}

file, err := os.Open(csvParam)
handleGoErr(err)
defer file.Close()
reader := csv.NewReader(file)

lineNumber := 0
records := make([][]string, 0)
for {
record, err := reader.Read()
// Ensure rows have valid data
if err != nil {
if err == io.EOF {
break
}
handleGoErr(err)
}
if len(record) != 6 {
handleGoErr(err)
}

// Skip header
if lineNumber == 0 {
lineNumber++
continue
}

// Ensure `Amount` column is a valid float
_, err = strconv.ParseFloat(record[3], 64)
handleGoErr(err)

lineNumber++
records = append(records, record)
}

return records
}

// handleGoErr prints the error to stderr and exits, should be used for Go errors and not EasyPost errors
func handleGoErr(err error) {
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}

// createInsurance creates an Insurance based on if the Tracker has a Shipment or not
func createInsurance(client *easypost.Client, tracking_code string, reference string, carrier_string string, amount string, to_address_id string, from_address_id string) (string, string) {
// We first get the tracker to know if there is a Shipment or not so we know which Insurance endpoint to hit
tracker, err := client.GetTracker(tracking_code)
var eperr *easypost.APIError
if errors.As(err, &eperr) {
return "false", eperr.Message
}

if tracker.ShipmentID != "" {
_, err := client.InsureShipment(tracker.ShipmentID, amount)
var eperr *easypost.APIError
if errors.As(err, &eperr) {
return "false", eperr.Message
}

return "true", ""
} else {
// If no shipment ID, it's a standalone insurance
_, err := client.CreateInsurance(
&easypost.Insurance{
ToAddress: &easypost.Address{
ID: to_address_id,
},
FromAddress: &easypost.Address{
ID: from_address_id,
},
Reference: reference,
Carrier: carrier_string,
TrackingCode: tracking_code,
Amount: amount,
},
)
var eperr *easypost.APIError
if errors.As(err, &eperr) {
return "false", eperr.Message
}

return "true", ""
}
}
5 changes: 5 additions & 0 deletions community/golang/concurrent_insurance_buy/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module github.com/EasyPost/examples/community/golang/concurrent_insurance_buy

go 1.16

require github.com/EasyPost/easypost-go/v4 v4.6.0
56 changes: 56 additions & 0 deletions community/golang/concurrent_insurance_buy/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
github.com/EasyPost/easypost-go/v4 v4.6.0 h1:wxMK4wkGEG5vW/4Vdy3rwE9iqww1eQ1xS6oYWUZbhrc=
github.com/EasyPost/easypost-go/v4 v4.6.0/go.mod h1:WGoS4tmjHquhooMNmY6RirP+KWeYV/akcf/Jg9Q6fsk=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dnaeon/go-vcr v1.2.0/go.mod h1:R4UdLID7HZT3taECzJs4YgbbH6PIGXB6W/sc5OLb6RQ=
github.com/google/go-cmp v0.5.2 h1:X2ev0eStA3AbceY54o37/0PQ/UWqKEiiO2dKL5OPaFM=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/modocache/gover v0.0.0-20171022184752-b58185e213c5/go.mod h1:caMODM3PzxT8aQXRPkAt8xlV/e7d7w8GM5g0fa5F0D8=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.11.0 h1:LAntKIrcmeSKERyiOh0XMV39LXS8IE9UL2yP7+f5ij4=
golang.org/x/text v0.11.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
8 changes: 8 additions & 0 deletions community/golang/concurrent_insurance_buy/sample.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Tracking Code,Reference (optional),Carrier String,Amount,To Address ID (optional),From Address ID (optional)
EZ1000000001,ref_1,USPS,100.00,,
EZ2000000002,ref_2,USPS,200.00,,
EZ3000000003,,USPS,300.00,,
EZ4000000004,,USPS,400.00,,
EZ5000000005,,USPS,500.00,,
EZ6000000006,,USPS,600.00,,
EZ7000000007,,USPS,700.00,,