Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

typescript #92

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .eslintrc.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
// @ts-check
const { defineConfig } = require('eslint-define-config');

module.exports = defineConfig({
extends: "eslint:recommended",
parserOptions: {
ecmaVersion: 8,
sourceType: "module",
ecmaFeatures: {},
},
env: {
node: true,
mocha: true,
es6: true,
},
});
13 changes: 0 additions & 13 deletions .eslintrc.js

This file was deleted.

4 changes: 2 additions & 2 deletions .github/workflows/npm-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
with:
node-version: 12
node-version: 14.16
- run: npm ci
- run: npm test

Expand All @@ -25,7 +25,7 @@ jobs:
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
with:
node-version: 12
node-version: 14.16
registry-url: https://registry.npmjs.org/
- run: npm ci
- run: npm publish
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:

strategy:
matrix:
node-version: [10.x, 12.x, 14.x]
node-version: [14.16, 16.x, 18.x]

steps:
- uses: actions/checkout@v2
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/version-bump.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
nodeVersion: ${{ steps.bump_version.outputs.version }}
strategy:
matrix:
node-version: [10.x]
node-version: [14.16]
steps:
- uses: actions/checkout@v2
- name: Use Node.js ${{ matrix.node-version }}
Expand All @@ -38,7 +38,7 @@ jobs:
needs: build
strategy:
matrix:
node-version: [10.x]
node-version: [14.16]
steps:
- name: draft release
id: draft_release
Expand Down
118 changes: 54 additions & 64 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,52 +12,30 @@
![Test](https://github.com/seantomburke/sitemapper/workflows/test/badge.svg?branch=master&event=push)

Parse through a sitemaps xml to get all the urls for your crawler.
## Version 2

## Version 4

> Version 4.0.0 introduces breaking changes and supports ECMAScript Modules (ESM)
If upgrading to 4.0.0 you will not be able to use require() to import the dependency.
You must use import() to import the dependencies.
You will also need to upgrade to `Node >= 14.16`

### Installation
```bash
npm install sitemapper --save
npm install sitemapper@latest --save
```

### Simple Example
```javascript
const Sitemapper = require('sitemapper');
import Sitemapper from 'sitemapper';

const sitemap = new Sitemapper();

sitemap.fetch('https://wp.seantburke.com/sitemap.xml').then(function(sites) {
console.log(sites);
});

```
### Examples in ES6
```javascript
import Sitemapper from 'sitemapper';

(async () => {
const Google = new Sitemapper({
url: 'https://www.google.com/work/sitemap.xml',
timeout: 15000, // 15 seconds
});

try {
const { sites } = await Google.fetch();
const { sites } = await sitemap.fetch('https://wp.seantburke.com/sitemap.xml');
console.log(sites);
} catch (error) {
console.log(error);
}
})();

// or

const sitemapper = new Sitemapper();
sitemapper.timeout = 5000;

sitemapper.fetch('https://wp.seantburke.com/sitemap.xml')
.then(({ url, sites }) => console.log(`url:${url}`, 'sites:', sites))
.catch(error => console.log(error));
```

# Options

You can add options on the initial Sitemapper object when instantiating it.
Expand Down Expand Up @@ -101,56 +79,68 @@ const sitemapper = new Sitemapper({

```

### Examples in ES5
### Examples in ESM
```javascript
var Sitemapper = require('sitemapper');

var Google = new Sitemapper({
url: 'https://www.google.com/work/sitemap.xml',
timeout: 15000 //15 seconds
});
import Sitemapper from 'sitemapper';

Google.fetch()
.then(function (data) {
console.log(data);
})
.catch(function (error) {
console.log(error);
(async () => {
const Google = new Sitemapper({
url: 'https://www.google.com/work/sitemap.xml',
timeout: 15000, // 15 seconds
});

try {
const { sites } = await Google.fetch();
console.log(sites);
} catch (error) {
console.log(error);
}
})();

// or


var sitemapper = new Sitemapper();

const sitemapper = new Sitemapper();
sitemapper.timeout = 5000;
sitemapper.fetch('https://wp.seantburke.com/sitemap.xml')
.then(function (data) {
console.log(data);
})
.catch(function (error) {
console.log(error);
});

sitemapper.fetch('https://wp.seantburke.com/sitemap.xml')
.then(({ url, sites }) => console.log(`url:${url}`, 'sites:', sites))
.catch(error => console.log(error));
```

## Version 1

## Version 3

> Works for `Node 10.X` to `Node 12.X`

### Installation
```bash
npm install sitemapper@1.1.1 --save
npm install sitemapper@3.2 --save
```

### Simple Example
```javascript
const Sitemapper = require('sitemapper');

const sitemap = new Sitemapper();

sitemap.fetch('https://wp.seantburke.com/sitemap.xml').then(function(sites) {
console.log(sites);
});

### Example in JS
```javascript
var Sitemapper = require('sitemapper');

var sitemapper = new Sitemapper();

sitemapper.getSites('https://wp.seantburke.com/sitemap.xml', function(err, sites) {
if (!err) {
console.log(sites);
}
var Google = new Sitemapper({
url: 'https://www.google.com/work/sitemap.xml',
timeout: 15000 //15 seconds
});
```

Google.fetch()
.then(function (data) {
console.log(data);
})
.catch(function (error) {
console.log(error);
});

41 changes: 0 additions & 41 deletions example.es6.js

This file was deleted.

77 changes: 31 additions & 46 deletions example.js
Original file line number Diff line number Diff line change
@@ -1,56 +1,41 @@
var Sitemapper = require('sitemapper');
import Sitemapper from 'sitemapper';

// Instantiate an instance with options
var Google = new Sitemapper({
url: 'https://www.google.com/work/sitemap.xml',
debug: false,
timeout: 15000 //15 seconds
});
(async () => {
const sitemapper = new Sitemapper();

// Then fetch
Google.fetch()
.then(function (data) {
console.log(data);
})
.catch(function (error) {
console.log(error);
});

// Instantiate an instance with no options
var sitemapper = new Sitemapper();
sitemapper.timeout = 5000;

sitemapper.fetch('https://wp.seantburke.com/sitemap.xml')
.then(function (data) {
console.log(data);
})
.catch(function (error) {
console.log(error);
const Google = new Sitemapper({
url: 'https://www.google.com/work/sitemap.xml',
debug: false,
timeout: 15000, // 15 seconds
});

sitemapper.fetch('http://www.cnn.com/sitemaps/sitemap-index.xml')
.then(function (data) {
console.log('sites:', data.sites, 'url', data.url);
})
.catch(function (error) {
try {
const data = await Google.fetch();
console.log(data.sites);
} catch(error) {
console.log(error);
});
}

sitemapper.fetch('http://www.stubhub.com/new-sitemap/us/sitemap-US-en-index.xml')
.then(function (data) {
console.log('sites:', data.sites, 'url', data.url);
})
.catch(function (error) {
console.log(error);
});
sitemapper.timeout = 5000;

// Version 1.0.0 example which has been deprecated.
sitemapper.getSites('https://wp.seantburke.com/sitemap.xml', function (err, sites) {
if (!err) {
console.log(sites);
try {
const { url, sites } = await sitemapper.fetch('https://wp.seantburke.com/sitemap.xml');
console.log(`url:${url}`, 'sites:', sites);
} catch(error) {
console.log(error)
}
else {
console.log(err);

try {
const { url, sites } = await sitemapper.fetch('http://www.cnn.com/sitemaps/sitemap-index.xml');
console.log(`url:${url}`, 'sites:', sites);
} catch(error) {
console.log(error)
}
});

try {
const { url, sites } = await sitemapper.fetch('http://www.stubhub.com/new-sitemap/us/sitemap-US-en-index.xml');
console.log(`url:${url}`, 'sites:', sites);
} catch(error) {
console.log(error)
}
})();
Loading