Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support ascii no trailing blanks for v2 b #99

Merged
merged 3 commits into from
Jan 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ await connection.uploadDataset(input, 'HLQ.HOSTS', "LRECL=80 RECFM=FB");
##### Parameter

* dsn - _string_ - Specify a full qualified dataset name, or USS file name. It **CAN NOT** contain any wildcard (*).
* transferMode - _TransferMode_ - `TransferMode.ASCII`, `TransferMode.BINARY`, `TransferMode.ASCII_STRIP_EOL`, `TransferMode.ASCII_RDW`, or `TransferMode.BINARY_RDW`. When downloading a text dataset, transferMode should be either `TransferMode.ASCII` or `TransferMode.ASCII_STRIP_EOL` so that z/OS FTP service converts `EBCDIC` characters to `ASCII`. `TransferMode.ASCII_STRIP_EOL` asks z/OS FTP service not to append a `CLRF` to the end of each record. `TransferMode.ASCII_RDW` and `TransferMode.BINARY_RDW` support to download variable length dataset, which add 4-byte Record Description Word (RDW) at the beginning of each record.
* transferMode - _TransferMode_ - `TransferMode.ASCII`, `TransferMode.BINARY`, `TransferMode.ASCII_STRIP_EOL`, `TransferMode.ASCII_RDW`, `TransferMode.ASCII_NO_TRAILING_BLANKS`, or `TransferMode.BINARY_RDW`. The `TransferMode.ASCII`, `TransferMode.ASCII_STRIP_EOL`, `TransferMode.ASCII_RDW`, or `TransferMode.ASCII_NO_TRAILING_BLANKS` asks z/OS FTP service to convert `EBCDIC` characters to `ASCII`. The `TransferMode.ASCII_STRIP_EOL` asks z/OS FTP service not to append a `CLRF` to the end of each record. The `TransferMode.ASCII_NO_TRAILING_BLANKS` asks z/OS FTP service to remove trailing blanks. The `TransferMode.ASCII_RDW` or `TransferMode.BINARY_RDW` supports to download variable length dataset, which add 4-byte Record Description Word (RDW) at the beginning of each record.
* stream - _boolean_ - `true` if you want to obtain a [ReadableStream](https://nodejs.org/api/stream.html#stream_readable_streams) of the data set content, or `false` to read a full dataset into memory (in Buffer). The buffer accepts up to 4MB data. For large dataset, use `stream=true` instead.
* siteParams - _string_ - Add extra site parameters, for example, 'sbd=(IBM-1047,ISO8859-1)' for encoding setting.

Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "zos-node-accessor",
"version": "2.0.8",
"version": "2.0.9",
"description": "Accessing z/OS dataset and interacting with JES in NodeJS way",
"main": "./lib/zosAccessor.js",
"types": "./lib/zosAccessor.d.ts",
Expand Down
18 changes: 18 additions & 0 deletions src/__test__/downloadDataset.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,18 @@ describe('The method of downloadDataset()', () => {
}
});

async function reconnectFTPServer() {
if (accessor) {
await accessor.close();
}
accessor = await connectFTPServer();
}

it('can get text contents from dataset with the default TransferMode.ASCII', async () => {
await accessor.uploadDataset('hello\nworld', dsn);
const contents1 = await accessor.downloadDataset(dsn, TransferMode.ASCII);
expect(contents1.toString().trim()).toBe('hello\nworld');
await reconnectFTPServer()
const contents2 = await accessor.downloadDataset(dsn, TransferMode.ASCII);
expect(contents2.toString().trim()).toBe('hello\nworld');
});
Expand All @@ -78,10 +86,20 @@ describe('The method of downloadDataset()', () => {
await accessor.uploadDataset('hello\r\nworld', dsn);
const contents1 = await accessor.downloadDataset(dsn, TransferMode.ASCII);
expect(contents1.toString().trim()).toBe('hello\r\nworld');
await reconnectFTPServer()
const contents2 = await accessor.downloadDataset(dsn, TransferMode.ASCII_STRIP_EOL);
expect(contents2.toString().trim()).toBe('helloworld');
});

it('can get text contents from dataset with TransferMode.ASCII_NO_TRAILING_BLANKS', async () => {
await accessor.uploadDataset('hello ', dsn, TransferMode.ASCII, { RECfm: 'FB', LRECL: 80 });
const contents1 = await accessor.downloadDataset(dsn, TransferMode.ASCII);
expect(contents1.toString()).toBe('hello \r\n');
await reconnectFTPServer()
const contents2 = await accessor.downloadDataset(dsn, TransferMode.ASCII_NO_TRAILING_BLANKS);
expect(contents2.toString()).toBe('hello\r\n');
});

it('can get text contents from dataset with TransferMode.BINARY', async () => {
await accessor.uploadDataset('hello\r\nworld', dsn);
const contents2 = await accessor.downloadDataset(dsn, TransferMode.BINARY);
Expand Down
39 changes: 29 additions & 10 deletions src/zosAccessor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,11 +71,16 @@ export enum TransferMode {
ASCII = 'ascii',

/**
* The data is transferre in text mode, in which encoding conversion like ASCII/EBCDIC will happen, and
* The data is transferred in text mode, in which encoding conversion like ASCII/EBCDIC will happen, and
* the EOL of each record in dataset will be removed.
*/
ASCII_STRIP_EOL = 'ascii_strip_eol',

/**
* The data is transferred in text mode, in which the trailing blanks will be removed.
*/
ASCII_NO_TRAILING_BLANKS = 'ascii_no_trailing_blanks',

/**
* The data is transferred in binary mode, in which no encoding conversion will happen.
*/
Expand Down Expand Up @@ -435,23 +440,34 @@ class ZosAccessor {

// We still need this check in case the other TransferMode will be added in future.
if (transferMode !== TransferMode.ASCII &&
transferMode !== TransferMode.BINARY &&
transferMode !== TransferMode.ASCII_STRIP_EOL) {
transferMode !== TransferMode.ASCII_NO_TRAILING_BLANKS &&
transferMode !== TransferMode.ASCII_STRIP_EOL &&
transferMode !== TransferMode.BINARY) {
throw new Error('Unsupported data type: ' + transferMode);
}
let ftpCommand = ftpClient.binary.bind(ftpClient);
if (transferMode === TransferMode.ASCII ||
transferMode === TransferMode.ASCII_NO_TRAILING_BLANKS ||
transferMode === TransferMode.ASCII_STRIP_EOL) {
ftpCommand = ftpClient.ascii.bind(ftpClient);
}
let sbsendeol = 'SBSENDEOL=CRLF';
if (transferMode === TransferMode.ASCII_STRIP_EOL) {
sbsendeol = 'SBSENDEOL=NONE';
transferMode = TransferMode.ASCII;
}

let site = 'FILETYPE=SEQ TRAILINGBLANKS ' + sbsendeol;
let trailingBlanks = 'TRAILINGBLANKS';
if (transferMode === TransferMode.ASCII_NO_TRAILING_BLANKS) {
trailingBlanks = '';
}
let site = 'FILETYPE=SEQ ' + trailingBlanks + ' ' + sbsendeol;
if (siteParams) {
site += ' ' + siteParams;
}

// ftpClient.ascii() or ftpClient.binary()
ftpClient[transferMode]((err: Error) => {
ftpCommand((err: Error) => {
if (err) {
return deferred.reject(err);
}
Expand Down Expand Up @@ -495,11 +511,14 @@ class ZosAccessor {
* Downloads the specified dataset or member of patition dataset.
*
* @param dsn - Dataset name
* @param transferMode - TransferMode.ASCII, TransferMode.BINARY, TransferMode.ASCII_RDW, TransferMode.BINARY_RDW
* or TransferMode.ASCII_STRIP_EOL. When downloading a text dataset, transferMode should be
* either `TransferMode.ASCII` or `TransferMode.ASCII_STRIP_EOL` so that z/OS FTP service
* converts `EBCDIC` characters to `ASCII`. `TransferMode.ASCII_STRIP_EOL` asks z/OS FTP
* service not to append a `CLRF` to the end of each record. `TransferMode.ASCII_RDW` and
* @param transferMode - TransferMode.ASCII, TransferMode.ASCII_RDW, TransferMode.ASCII_STRIP_EOL,
* TransferMode.ASCII_NO_TRAILING_BLANKS, TransferMode.BINARY, or TransferMode.BINARY_RDW.
* The `TransferMode.ASCII`, ` TransferMode.ASCII_RDW`, `TransferMode.ASCII_STRIP_EOL`,
* or TransferMode.ASCII_NO_TRAILING_BLANKS asks z/OS FTP service to
* convert `EBCDIC` characters to `ASCII`. `TransferMode.ASCII_STRIP_EOL` asks z/OS FTP
* service not to append a `CLRF` to the end of each record. The
* `TransferMode.ASCII_NO_TRAILING_BLANKS` asks z/OS FTP service to remove trailing blanks.
* `TransferMode.ASCII_RDW` and
* `TransferMode.BINARY_RDW` support to download variable length dataset, which add 4-byte
* Record Description Word (RDW) at the beginning of each record.
* @param {boolean} stream - `true` if you want to obtain a `ReadStream` of the data set content, or `false`
Expand Down
Loading