Skip to content

Commit

Permalink
Initial public release, 0.1.5
Browse files Browse the repository at this point in the history
  • Loading branch information
gshively11 committed Aug 3, 2017
0 parents commit a376f49
Show file tree
Hide file tree
Showing 48 changed files with 2,712 additions and 0 deletions.
10 changes: 10 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
logs
*.log
pids
*.pid
*.seed
build/Release
node_modules
.idea/*
dist
sandbox.js
114 changes: 114 additions & 0 deletions API.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
<a name="module_connection-pool-party"></a>

## connection-pool-party

* [connection-pool-party](#module_connection-pool-party)
* [.ConnectionPoolParty](#module_connection-pool-party.ConnectionPoolParty) ⇐ <code>EventEmitter</code>
* [new ConnectionPoolParty(config, [cb])](#new_module_connection-pool-party.ConnectionPoolParty_new)
* [.warmup([cb])](#module_connection-pool-party.ConnectionPoolParty+warmup) ⇒ <code>Promise</code>
* [.request()](#module_connection-pool-party.ConnectionPoolParty+request) ⇒ <code>mssql.Request</code>
* [.close([cb])](#module_connection-pool-party.ConnectionPoolParty+close) ⇒ <code>Promise</code>
* [.stats()](#module_connection-pool-party.ConnectionPoolParty+stats) ⇒ <code>Object</code>
* [.forceFqdnConnectionPoolFactory(suffix)](#module_connection-pool-party.forceFqdnConnectionPoolFactory) ⇒ <code>Promise</code>

<a name="module_connection-pool-party.ConnectionPoolParty"></a>

### connection-pool-party.ConnectionPoolParty ⇐ <code>EventEmitter</code>
**Kind**: static class of [<code>connection-pool-party</code>](#module_connection-pool-party)
**Extends**: <code>EventEmitter</code>

* [.ConnectionPoolParty](#module_connection-pool-party.ConnectionPoolParty) ⇐ <code>EventEmitter</code>
* [new ConnectionPoolParty(config, [cb])](#new_module_connection-pool-party.ConnectionPoolParty_new)
* [.warmup([cb])](#module_connection-pool-party.ConnectionPoolParty+warmup) ⇒ <code>Promise</code>
* [.request()](#module_connection-pool-party.ConnectionPoolParty+request) ⇒ <code>mssql.Request</code>
* [.close([cb])](#module_connection-pool-party.ConnectionPoolParty+close) ⇒ <code>Promise</code>
* [.stats()](#module_connection-pool-party.ConnectionPoolParty+stats) ⇒ <code>Object</code>

<a name="new_module_connection-pool-party.ConnectionPoolParty_new"></a>

#### new ConnectionPoolParty(config, [cb])
Class representing a ConnectionPoolParty, which manages one or more ConnectionPool instance(s).
ConnectionPoolParty extends the mssql package to provide failover between ConnectionPools,
reconnets/retries, and basic health/statistics reporting.


| Param | Type | Default | Description |
| --- | --- | --- | --- |
| config | <code>object</code> | | Configuration for ConnectionPoolParty |
| [config.reconnects] | <code>number</code> | <code>0</code> | The number of times a request will be retried against ALL pools. A heal operation is attempted before a reconnect. Total request attempts is calculated using: pools * (1+reconnects) * (1+retries) |
| [config.retries] | <code>number</code> | <code>0</code> | The number of times a request will be retried against a single pool. Each pool is retried separately. Total request attempts is calculated using: pools * (1+reconnects) * (1+retries) |
| [config.dsn] | <code>object</code> | | A single DSN, matches the configuration object expected by the mssql package. Required if dsns and dsnProvider are not provided. |
| [config.dsns] | <code>array</code> | | An array of DSNs, each entry should match the configuraiton object expected by the mssql package. Overrides config.dsn. Required if dsn and dsnProvider are not provided. |
| [config.dsnProvider] | <code>function</code> | | A function returning a promise that resolves with an array of dsn object(s). This option will override config.dsn and config.dsns. Required if dsn and dsns are not provided. |
| [config.connectionPoolFactory] | <code>function</code> | | A function that receives the dsn objects from the dnsProvider and returns a promise that resolves with *connected* instance(s) of ConnectionPool. Use this option if you want to customize how mssql ConnectionPools are instantiated and connected. |
| [config.connectionPoolConfig] | <code>object</code> | | An object containing any configuration you want to attach to the config provided when creating an mssql ConnectionPool. This is useful if you don't want to create a custom dsnProvider or connectionPoolFactory to modify the configuration used to create ConnectionPools. Just keep in mind that any config set here will override the config set in the dsnProvider. Also keep in mind that node-mssql expects some configuration to exists on an "options" property (like timeouts). Check node-mssql README.md for more information. |
| [cb] | <code>function</code> | | Optional callback interface, providing this automatically calls warmup. It is preferable to use the Promise-based interface and call warmup explicitly. |

<a name="module_connection-pool-party.ConnectionPoolParty+warmup"></a>

#### connectionPoolParty.warmup([cb]) ⇒ <code>Promise</code>
Retrieve the dsn(s) from the dsnProvider, create and connect the ConnectionPool
instance(s) using the connectionPoolFactory. Returns a promise. Can be called
to explicitly warmup database connections. Called implicitly when submitting
any requests. After a successful warmup, subsequent calls will not warmup again.

**Kind**: instance method of [<code>ConnectionPoolParty</code>](#module_connection-pool-party.ConnectionPoolParty)
**Returns**: <code>Promise</code> - A promise indicating that a warmup was successful. This promise
cannot reject, but errors during warmup will result in the cached warmup promise
being removed, which will allow warmup to be re-attempted.

| Param | Type | Description |
| --- | --- | --- |
| [cb] | <code>function</code> | An optional callback interface. It is preferable to use the Promise-based interface. |

<a name="module_connection-pool-party.ConnectionPoolParty+request"></a>

#### connectionPoolParty.request() ⇒ <code>mssql.Request</code>
Retrieve a new Request instance. This is the same Request provided by the mssql
package, but it's specially extended to interact with ConnectionPoolParty.

**Kind**: instance method of [<code>ConnectionPoolParty</code>](#module_connection-pool-party.ConnectionPoolParty)
**Returns**: <code>mssql.Request</code> - An extended instance of mssql.Request.
<a name="module_connection-pool-party.ConnectionPoolParty+close"></a>

#### connectionPoolParty.close([cb]) ⇒ <code>Promise</code>
Close all pools associated with this instance of ConnectionPoolParty

**Kind**: instance method of [<code>ConnectionPoolParty</code>](#module_connection-pool-party.ConnectionPoolParty)
**Returns**: <code>Promise</code> - A Promise that resolves when all pools are closed. Will also
resolve if there is an error encountered while closing the pools.

| Param | Type | Description |
| --- | --- | --- |
| [cb] | <code>function</code> | An optional callback interface. It is preferable to use the Promise-based interface. |

<a name="module_connection-pool-party.ConnectionPoolParty+stats"></a>

#### connectionPoolParty.stats() ⇒ <code>Object</code>
Retrieve health and statistics for this ConnectionPoolParty and its associated
pools.

**Kind**: instance method of [<code>ConnectionPoolParty</code>](#module_connection-pool-party.ConnectionPoolParty)
**Returns**: <code>Object</code> - An object containing a bunch of health/stats data for this instance
of ConnectionPoolParty and its associated pools.
<a name="module_connection-pool-party.forceFqdnConnectionPoolFactory"></a>

### connection-pool-party.forceFqdnConnectionPoolFactory(suffix) ⇒ <code>Promise</code>
This connection pool factory is only needed for a niche use case, but it
serves as an example of what is possible when creating a custom
connectionPoolFactory.

If your dsn provider returns servers as hostnames instead of FQDNs or IPs,
you may have systems that are unable to resolve the hostnames due to
misconfigured DNS settings. If you are unable to fix the DNS resolution for
whatever reason, and you know what the FQDN suffix is, you can use this
connectionPoolFactory to add the suffix.

**Kind**: static method of [<code>connection-pool-party</code>](#module_connection-pool-party)
**Returns**: <code>Promise</code> - A promise that uses a dsn provided by the dsnProvider to create
an mssql ConnectionPool.

| Param | Type | Description |
| --- | --- | --- |
| suffix | <code>string</code> | The FQDN suffix to use if your dsn's server is provided as a hostname. |

42 changes: 42 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# 0.1.5 (July 21th, 2017)

## Other

- Improved debug logs
- All debug logs go to stdout
- No changes in production functionality

# 0.1.4 (June 27th, 2017)

## Bug Fixes

- Fixed error when running `connection.request().query('...')` commands that did not return a recordset (e.g. `TRUNCATE TABLE ...`)

## Other

- Added a high-write integration test
- Added unit test
- Added a bit more debug logging

# 0.1.3 (April 5th, 2017)

## Bug Fixes

- Removed pool.dsn.password from PoolError to avoid passwords showing up in error logging

# 0.1.2 (April 3rd, 2017)

## Bug Fixes

- Fixed returnValue not being return when calling execute
- Fixed broken execute callback tests

# 0.1.1 (April 3rd, 2017)

## Features

- Added `connectionPoolConfig` option to the ConnectionPoolParty constructor. This new option lets you set configuration options that will be passed to the mssql ConnectionPool constructor. This provides a way to set things like timeouts without having to create a custom connectionPoolFactory or dsnProvider. _Note: any options set here will override options created by the dsnProvider_

# 0.1.0 (April 3rd, 2017)

First Release!
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
The MIT License (MIT)

Copyright (c) 2017 GoDaddy Operating Company, LLC.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
151 changes: 151 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# mssql-pool-party

ES6+ extension of [mssql](https://github.com/patriksimek/node-mssql) that provides:

- Failover between multiple connection pools
- A mechanism to load DSNs ([Data Source Names](https://en.wikipedia.org/wiki/Data_source_name), think connection strings) asynchronously and recreate connection pools accordingly
- Various retry options for requests, transactions, and prepared statements
- Health and statistics of connection pools
- Only minor changes to the existing mssql API

Jump on in, the water's fine.

### Why does this package exist?

- Disaster recovery option when you're not using Availability Groups (e.g. Database Mirroring)
- If you store/manage database credentials using an external service, the `dsnProvider` option lets you automatically grab those changes and recreate connection pools.
- Stats/health reporting, so you can see what databases your app is talking to and whether or not the connections are healthy.
- Built in retry/reconnect options

### Usage

#### Simple single connection pool

```js
// my-db.js
import sql from 'mssql-pool-party';

const config = {
// See configuration section below for more information
dsn: {/* ... */}
};

const connection = new sql.ConnectionPoolParty(config);

connection.on('error', console.err);

export default connection;
```

```js
// call-proc.js
import sql from 'mssql-pool-party';
import myDb from './my-db';

export default function callProc() {
return myDb.request()
.input('some_param', sql.Int, 10)
.execute('my_proc')
.then(console.dir)
.catch(console.error);
}
```

#### Using a DsnProvider to retrieve one or more dsn(s) dynamically

```js
// my-db.js
import sql from 'mssql-pool-party';
import jsonFileDsnProvider from 'my-example-dsn-provider';

// grab DSN(s) from a json file
const dsnProvider = jsonFileDsnProvider('/etc/secrets/my_db.json');

const config = {
dsnProvider,
retries: 2,
reconnects: 1,
};

const connection = new sql.ConnectionPoolParty(config);

// this attempts to connect each ConnectionPool before any requests are made.
// returns a promise, so you can use it during an API's warmup phase before
// starting any listeners
connection.warmup();

connection.on('error', console.error);

// logging connection pool stats to console every 60 seconds
setInterval(() => console.log(connection.stats()), 60000);

export default connection;
```

```js
// run-query.js
import sql from 'mssql-pool-party';
import myDb from './my-db';

export default function runQuery(id) {
return myDb.request()
.input('some_param', sql.Int, id)
.query('select * from mytable where id = @some_param')
.then(console.dir)
.catch(console.error);
}
```

### Configuration

Check out the [detailed API documentation](API.md#new-connectionpoolpartyconfig).

### Events

- `error` - This event is fired whenever errors are encountered that DO NOT result in a rejected promise, stream error, or callback error that can be accessed/caught by the consuming app, which makes this event the only way to respond to such errors. Example: An app initiates a query, the first attempt fails, but a retry is triggered and the second attempt succeeds. From the apps perspective, the retry attempt isn't visible and their promised query is resolved. However, apps may want to know when a query requires a retry to succeed, so we emit the error using this event. **TAKE NOTE**: Unlike the mssql package, failing to subscribe to the error event will not result in an unhandled exception and subsequent process crash.

### Streaming

The mssql package's streaming API is supported in mssql-pool-party, with a substantial caveat. By their nature, streams attempt to minimize the amount of memory in use by "streaming" chunks of the data from one source to another. Their behavior makes them not play well with retry/reconnect logic, because the stream destination would need to understand when a retry/reconnect happens and abandon any previously received data in preparation for data coming from another attempt. Otherwise, you're going to end up just caching all the chunks in memory, negating the benefits of using a stream. Observe:

```js
import myDb from './my-db';

const request = myDb.request();
request.stream = true;
let currentAttempt = 0;
let rows;
let errors;
const resetDataIfNeeded = (attemptNumber) => {
if (attemptNumber > currentAttempt) {
rows = [];
errors = [];
currentAttempt += 1;
}
};
request.query('select * from SomeHugeTable');
request.on('error', (err, attemptNumber) => {
resetDataIfNeeded(attemptNumber);
errors.push(err);
});
request.on('rows', (row, attemptNumber) => {
resetDataIfNeeded(attemptNumber);
rows.push(row);
});
// NOTE: done is only called after the final attempt
request.on('done', (rowsAffected, attemptNumber) => {
if (errors.length) {
errors.forEach(console.log);
throw new Error('Unhandled errors. Handle them!');
}
console.log(rows);
});
```

In this example, we are storing errors/rows in memory. If we notice that the attemptNumber increments, we throw away our cached data and start over. Once the done event fires, we check to see if there are any errors and if not, return the results. This is a poor use of streams, because we are caching the entire result set in memory. To use streams properly, we would need to take the data provided by the `rows` event and shuttle it off somewhere else. The problem is that where we are shuttling it off to needs something similar to the `resetDataIfNeeded` function in the example above.

One thing to note is mssql-pool-party does allow you to use the vanilla streaming API and avoid the concerns of juggling the attempt number, but you'll need to set `retries` and `reconnects` to 0.

### A quick note on caching

When a ConnectionPoolParty is instantiated, it will internally cache one or more instance(s) of mssql.Connection. You should only create one instance of ConnectionPoolParty per set of DSNs and cache it for use in other modules (as seen in the examples above).
3 changes: 3 additions & 0 deletions jsdoc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"plugins": ["node_modules/jsdoc-babel"]
}
Loading

0 comments on commit a376f49

Please sign in to comment.