Skip to content

Commit

Permalink
Merge pull request #222 from UW-Macrostrat/typescript
Browse files Browse the repository at this point in the history
Typescript
  • Loading branch information
davenquinn authored Jan 17, 2024
2 parents 64a7251 + 633b3d0 commit ce4cfd3
Show file tree
Hide file tree
Showing 201 changed files with 40,943 additions and 12,404 deletions.
2 changes: 1 addition & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
node_modules
credentials.js
credentials.ts
23 changes: 9 additions & 14 deletions .github/workflows/build-dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,19 @@ name: Build Development
on:
push:
tags:
- v[0-9]+.[0-9]+.[0-9]+-** # Semver Pre-Release
- v[0-9]+.[0-9]+.[0-9]+-** # Semver Pre-Release
pull_request:
branches: [ main ]
branches: [main]

jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Checkout
- name: Checkout
uses: actions/checkout@v3
with:
submodules: 'recursive'
-
name: Docker meta
with:
submodules: "recursive"
- name: Docker meta
id: meta
uses: docker/metadata-action@v4
with:
Expand All @@ -28,19 +26,16 @@ jobs:
type=raw,value={{tag}}-{{date 'YYYYMMDDHHmmss'}}
type=raw,value=latest-itb-{{date 'YYYYMMDDHHmmss'}}
type=raw,value=sha-{{sha}}
-
name: Set up Docker Buildx
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

-
name: Login to OSG DockerHub
- name: Login to OSG DockerHub
uses: docker/login-action@v2
with:
registry: hub.opensciencegrid.org
username: ${{ vars.HARBOR_CLI_NAME }}
password: ${{ secrets.HARBOR_CLI_SECRET }}
-
name: Build and push
- name: Build and push
uses: docker/build-push-action@v3
with:
context: .
Expand Down
21 changes: 8 additions & 13 deletions .github/workflows/build-prod.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,17 @@ name: Build Production
on:
push:
tags:
- v[0-9]+.[0-9]+.[0-9]+ # Semver Release
- v[0-9]+.[0-9]+.[0-9]+ # Semver Release

jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Checkout
- name: Checkout
uses: actions/checkout@v3
with:
submodules: 'recursive'
-
name: Docker meta
with:
submodules: "recursive"
- name: Docker meta
id: meta
uses: docker/metadata-action@v4
with:
Expand All @@ -24,18 +22,15 @@ jobs:
type=semver,pattern={{version}}
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=sha-{{sha}}
-
name: Set up Docker Buildx
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Login to OSG DockerHub
- name: Login to OSG DockerHub
uses: docker/login-action@v2
with:
registry: hub.opensciencegrid.org
username: ${{ vars.HARBOR_CLI_NAME }}
password: ${{ secrets.HARBOR_CLI_SECRET }}
-
name: Build and push
- name: Build and push
uses: docker/build-push-action@v3
with:
context: .
Expand Down
6 changes: 3 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ v1/npm-debug.log

v1/utilities/scripts/credentials.py

credentials.js
!v1/credentials.js
!v2/credentials.js
credentials.ts
!v1/credentials.ts
!v2/credentials.ts
v2/utilities/scripts/credentials.py

.DS_Store
Expand Down
6 changes: 3 additions & 3 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"<node_internals>/**",
"<rootDir>/node_modules/**",
"<rootDir>/v1/node_modules/**",
"<rootDir>/v2/node_modules/**",
"<rootDir>/v2/node_modules/**"
]
},
}
}
}
5 changes: 3 additions & 2 deletions NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

## Debugging with VS Code

- Added [VSCode debugger configuration](https://code.visualstudio.com/docs/nodejs/nodejs-debugging)
- Added
[VSCode debugger configuration](https://code.visualstudio.com/docs/nodejs/nodejs-debugging)
- Enable 'Auto attach' in the VSCode configuration
- Run `npm start` *in VS Code's integrated terminal*.
- Run `npm start` _in VS Code's integrated terminal_.
90 changes: 61 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,59 +1,88 @@
# Macrostrat API

The API for SCIENCE

## About
The Macrostrat API provides diverse macrostratigraphic and geologic data in a concise, queryable format.

The Macrostrat API provides diverse macrostratigraphic and geologic data in a
concise, queryable format.

## Prerequisites

#### Software
MariaDB, PostgreSQL, Nodejs, Python, and [Macrostrat CLI](https://github.com/UW-Macrostrat/utils) are required. Redis is optional. When the API starts requests for all columns and all units are cached. If Redis is available it will be used, otherwise they are cached in the application memory.

MariaDB, PostgreSQL, Nodejs, Python, and
[Macrostrat CLI](https://github.com/UW-Macrostrat/utils) are required. Redis is
optional. When the API starts requests for all columns and all units are cached.
If Redis is available it will be used, otherwise they are cached in the
application memory.

#### Data
The following databases must be in place:
+ MariaDB - macrostrat
+ Postgres - burwell, alice, wof, elevation

The following databases must be in place:

- MariaDB - macrostrat
- Postgres - burwell, alice, wof, elevation

## Setup

First clone the repo:
````

```
git clone https://github.com/UW-Macrostrat/macrostrat-api.git
cd macrostrat-api
````
```

Next, install dependencies:
````

```
npm install
````
NOTE: Postgres connections are completely broken in node v14 and v15 (as of 2021-03-16). Use node v12!
```

NOTE: Postgres connections are completely broken in node v14 and v15 (as of
2021-03-16). Use node v12!

This will also run the script `postinstall.sh` which copies credentials files into place.
This will also run the script `postinstall.sh` which copies credentials files
into place.

For `v1/credentials.js` fill in your MariaDB and PostgreSQL user information. Do the same for `v2/credentials.js`, but also update the port for Redis if necessary, and follow the inline instructions for generating a cache refresh key. The cache refresh key is used as a secret parameter to programmatically refresh the column cache, such as in situations in which the underlying data has been changed (editing, adding, etc).
For `v1/credentials.js` fill in your MariaDB and PostgreSQL user information. Do
the same for `v2/credentials.js`, but also update the port for Redis if
necessary, and follow the inline instructions for generating a cache refresh
key. The cache refresh key is used as a secret parameter to programmatically
refresh the column cache, such as in situations in which the underlying data has
been changed (editing, adding, etc).

## Running
To start the API simply run `node server.js` which will start a process on port 5050. Note that the default port was changed from 5000 to 5050 to avoid a port conflict introduced in recent versions of commonly used OSs. To use a different port, you can specify it during startup as so: `node server.js 5151`.

To start the API simply run `node server.js` which will start a process on
port 5050. Note that the default port was changed from 5000 to 5050 to avoid a
port conflict introduced in recent versions of commonly used OSs. To use a
different port, you can specify it during startup as so: `node server.js 5151`.

For production use `pm2` is recommended. To start as a single process:
````

```
pm2 start server.js --name macrostrat-api
````
```

or, to start in a load balanced mode with two processes:
````
pm2 start server.js --name macrostrat-api -i 2
````

For more information about managing processes with pm2 see the [Macrostrat Wiki](https://github.com/UW-Macrostrat/lab/wiki/Nodejs-based-application-management)
```
pm2 start server.js --name macrostrat-api -i 2
```

For more information about managing processes with pm2 see the
[Macrostrat Wiki](https://github.com/UW-Macrostrat/lab/wiki/Nodejs-based-application-management)

## Other useful things

#### Caching

The Macrostrat API caches three specific request responses:
+ All columns with geometry (`/columns?all&format=geojson_bare`)
+ All columns without geometry (`/columns?all`)
+ The summary of all units used in the response of the above routes

- All columns with geometry (`/columns?all&format=geojson_bare`)
- All columns without geometry (`/columns?all`)
- The summary of all units used in the response of the above routes

To refresh this cache without restarting the API you can make an HTTP GET
request to `/api/v2/columns/refresh-cache?cacheRefreshKey=<cacheRefreshKey>`
Expand All @@ -64,19 +93,22 @@ This is primarily used by scripts in the Macrostrat CLI to ensure that the API
is reporting the most recent data after columns have been added, removed, or
otherwise edited.



### Test
````
npm test
````

```
npm test
```

### Organization
Each version of the API functions as a self-contained module, and therefore has its own ````package.json```` and dependencies.

Each version of the API functions as a self-contained module, and therefore has
its own `package.json` and dependencies.

### Authors
[John J Czaplewski](https://github.com/jczaplew) and [Shanan E Peters](http://strata.geoglogy.wisc.edu)

[John J Czaplewski](https://github.com/jczaplew) and
[Shanan E Peters](http://strata.geoglogy.wisc.edu)

### License

CC0 for all code unique to this API.
33 changes: 0 additions & 33 deletions credentials.example.js

This file was deleted.

32 changes: 32 additions & 0 deletions credentials.example.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
// A file mirroring this structure should be placed at /code/credentials.ts
// in the Docker container.

exports.mysql = {
host: "localhost",
user: "user",
password: "password",
database: "macrostrat",
socketPath: "/tmp/mysql.sock",
// socketPath: '/var/tmp/mariadb.sock'
};

exports.pg = {
host: "localhost",
port: "5432",
user: "postgres",
password: "",
};

exports.postgresDatabases = {
burwell: "burwell",
geomacro: "geomacro",
};

// This is the default Redis port
// NOTE: Redis is not configured at the moment
exports.redis = {
port: 6379,
};

// Generate a hash by running: node -e "console.log(require('uuid/v4')())"
exports.cacheRefreshKey = "put-hash-here";
Loading

0 comments on commit ce4cfd3

Please sign in to comment.