Skip to content
This repository has been archived by the owner on Oct 14, 2020. It is now read-only.

WIP on schemas for the experiment APIs - for review by all #135

Merged
merged 38 commits into from
Apr 20, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
4388b6a
WIP re #121
gregglind Apr 4, 2018
7b10237
Building schema.json during npm run build
motin Apr 5, 2018
5a7f984
Not tracking schema.json (for making yaml-focused diffs easier to rev…
motin Apr 5, 2018
8705e5e
WIP: added sample study using the api
gregglind Apr 9, 2018
c798997
Minimal changes to reach valid syntax
motin Apr 9, 2018
91a83af
Npm run format
motin Apr 9, 2018
3c18e13
Added minimal boilerplate to be able to launch examples/small-study
motin Apr 9, 2018
6f15bf9
WIP to make fakeApi
gregglind Apr 10, 2018
2e4bd9a
WIP on revised api
gregglind Apr 13, 2018
837d18c
More WIP. Not happy with how it's going
gregglind Apr 16, 2018
a9ffa82
WIP to fix linting. Local tests fail.
gregglind Apr 16, 2018
305eba5
format results
gregglind Apr 16, 2018
7ca4198
WIP, tests still broken. Moving toward one file for study, one for b…
gregglind Apr 16, 2018
8d79f5e
Continuing work on simple study, including schema work
gregglind Apr 16, 2018
c310a22
Simpler activation
gregglind Apr 16, 2018
34fa991
Vastly simplified API, including new background and study files. All…
gregglind Apr 17, 2018
fd0f4c6
WIP some schema naming, added wee-schema validation
gregglind Apr 17, 2018
bc1db5b
Candidate fix #121.
gregglind Apr 17, 2018
e7e4161
RC2 of api.
gregglind Apr 17, 2018
a57ba30
Known issue with schema for studySetup, will fix in morning.
gregglind Apr 17, 2018
d8b43d3
WIP, rc3
gregglind Apr 18, 2018
910ff11
WIP on verifying schemas. Fx is still not liking the validation
gregglind Apr 19, 2018
84cd21e
Fixed all schemas.
gregglind Apr 19, 2018
4cdba0a
Added 'browser.study.log(stringOrArray)'
gregglind Apr 19, 2018
24e6841
Documentation for the api, autogenerate by `npm run apiDocs`
gregglind Apr 19, 2018
59ebeff
First attempt at README improvements. Not sure if relative links work
gregglind Apr 19, 2018
31e6fe9
Readme, docs, minor nits and typos
gregglind Apr 19, 2018
f22b267
Fixing the header table
gregglind Apr 19, 2018
608f8b9
Linting and formatting
motin Apr 20, 2018
fc8a2ea
Using npm prepare instead of prepack (https://github.com/npm/npm/issu…
motin Apr 20, 2018
e03ffe1
Updated list of files to include in npm package
motin Apr 20, 2018
6828cc7
Gitignore cleanup
motin Apr 20, 2018
c13d6cf
Updated package-lock.json
motin Apr 20, 2018
436aed6
Make webExtensionApis/api.js and schema.json included in the npm package
motin Apr 20, 2018
1fe9073
Mention of how to get working develop version using npm install
gregglind Apr 20, 2018
6bc9bc9
new tool: copyStudyUtils, as a command. Final in 121
gregglind Apr 20, 2018
3b5d9df
Post format
gregglind Apr 20, 2018
4f1f117
README fixes
gregglind Apr 20, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions bin/schemaToInterface.js
Original file line number Diff line number Diff line change
Expand Up @@ -85,5 +85,6 @@ this.${ns} = class extends ExtensionAPI {
}`);
}
}

const path = require("path");
schema2shim(require(path.resolve(process.argv[2])));
65 changes: 65 additions & 0 deletions bin/verifyWeeSchema.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@

/**
* given a propose wee interface schama.json, lint and check it for
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aswan Is there a standardized/documented process to validate WEE API schema.json?

* validity
*/
// given a proposed WEE schema attept to validate it, including.

const path = require("path");

const proposed = require(path.resolve(process.argv[2]));
// const weeSchemaSchema = require(path.resolve("./wee-schema-schema.json"));

const ajv = new require("ajv")()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to add allErrors to the constructor here?

allErrors: check all rules collecting all errors. Default is to return after the first error.
— via https://github.com/epoberezkin/ajv#options


// 1. is eevery 'type' and every 'parameter' valid
for (let i in proposed) {
let ns = proposed[i];
for (let j in ns.types || []) {
let type = ns.types[j];
let valid = ajv.validateSchema(type);
if (!valid) {
console.error(`# ERRORS IN ${i}:${j} ${ns.namespace}.types[${j}] "${type.id}"`)
console.error(ajv.errors)
}

// checking test cases if any
if (!type.testcase) continue
valid = ajv.validate(type, type.testcase);
if (!valid) {
console.error(`# testcase failed IN ${i}:${j} ${ns.namespace}.types[${j}] "${type.id}"`)
console.error(ajv.errors)
}
}

}


// 2. is every 'parameter' valid
for (let i in proposed) {
let ns = proposed[i];
for (let j in ns.functions || []) {
let type = ns.functions[j];
for (let k in type.parameters) {
let parameter = type.parameters[k]
let valid = ajv.validateSchema(parameter);
if (!valid) {
console.error(`# ERRORS IN ${i}:${j} ${type.name} ${ns.namespace}.functions[${j}].paramters[${k}]`)
console.error(ajv.errors)
debugger;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rogue debugger statement?

}
}
}
for (let j in ns.events || []) {
let type = ns.events[j];
for (let k in type.parameters) {
let parameter = type.parameters[k]
let valid = ajv.validateSchema(parameter);
if (!valid) {
console.error(`# ERRORS IN ${i}:${j} ${type.name} ${ns.namespace}.events[${j}].parameters[${k}]`)
console.error(ajv.errors)
}
}
}
}

2 changes: 1 addition & 1 deletion bin/wee-schema-schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
"type": "string"
},
"type": {
"type": ["string", "array"]
"type": ["string", "array", "object"]
},
"$ref": {
"type": "string"
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@
"docformat": "doctoc --title '**Contents**' docs/*.md && prettier '**/*.md' --write",
"eslint": "eslint . --ext jsm --ext js --ext json",
"eslint-fix": "npm run eslint -- --fix",
"fakeApi": "cd webExtensionApis/study && yaml2json schema.yaml -p > schema.json && ajv -s ../../bin/wee-schema-schema.json -d schema.json && node ../../bin/schemaToInterface.js ./schema.json > fakeApi.js",
"fakeApi": "cd webExtensionApis/study && yaml2json schema.yaml -p > schema.json && node ../../bin/verifyWeeSchema.js schema.json && ajv -s ../../bin/wee-schema-schema.json -d schema.json && node ../../bin/schemaToInterface.js ./schema.json > fakeApi.js",
"format": "prettier '**/*.{css,js,jsm,json,md}' --trailing-comma=all --ignore-path=.eslintignore --write",
"lint": "npm-run-all lint:*",
"lint:eslint": "npm run eslint",
Expand Down
204 changes: 98 additions & 106 deletions webExtensionApis/study/fakeApi.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ this.study = class extends ExtensionAPI {
getAPI(context) {
return {
study: {
/* Attempt an setup/enrollment, with these effects:

/* Attempt an setup/enrollment, with these effects:

- sets 'studyType' as Shield or Pioneer
- affects telemetry
Expand Down Expand Up @@ -55,12 +56,12 @@ Preferences set
Note:
1. allowEnroll is ONLY used during first run (install)
*/
setup: async function setup(studySetup) {
console.log("called setup studySetup");
return undefined;
},
setup: async function setup ( studySetup ) {
console.log("called setup studySetup");
return undefined;
},

/* Signal to browser.study that it should end.
/* Signal to browser.study that it should end.

Usage scenarios:
- addons defined
Expand Down Expand Up @@ -99,12 +100,12 @@ Note:
1. calling this function multiple time is safe.
`browser.study` will choose the
*/
endStudy: async function endStudy(anEndingAlias, anEndingObject) {
console.log("called endStudy anEndingAlias, anEndingObject");
return { urls: ["url1", "url2"], endingName: "some-reason" };
},
endStudy: async function endStudy ( anEndingAlias, anEndingObject ) {
console.log("called endStudy anEndingAlias, anEndingObject");
return {"urls":["url1","url2"],"endingName":"some-reason"};
},

/* current study configuration, including
/* current study configuration, including
- variation
- activeExperimentName
- timeUntilExpire
Expand All @@ -115,28 +116,18 @@ But not:

Throws Error if called before `browser.study.setup`
*/
info: async function info() {
console.log("called info ");
return {
variation: "styleA",
firstRunTimestamp: 1523968204184,
activeExperimentName: "some experiment",
timeUntilExpire: null,
};
},

/* object of current dataPermissions with keys shield, pioneer, telemetry, 'ok' */
getDataPermissions: async function getDataPermissions() {
console.log("called getDataPermissions ");
return {
shield: true,
pioneer: false,
telemetry: true,
alwaysPrivateBrowsing: false,
};
},

/* Send Telemetry using appropriate shield or pioneer methods.
info: async function info ( ) {
console.log("called info ");
return {"variation":"styleA","firstRunTimestamp":1523968204184,"activeExperimentName":"some experiment","timeUntilExpire":null};
},

/* object of current dataPermissions with keys shield, pioneer, telemetry, 'ok' */
getDataPermissions: async function getDataPermissions ( ) {
console.log("called getDataPermissions ");
return {"shield":true,"pioneer":false,"telemetry":true,"alwaysPrivateBrowsing":false};
},

/* Send Telemetry using appropriate shield or pioneer methods.

shield:
- `shield-study-addon` ping, requires object string keys and string values
Expand All @@ -153,12 +144,12 @@ Note:

TBD fix the parameters here.
*/
sendTelemetry: async function sendTelemetry(payload) {
console.log("called sendTelemetry payload");
return "undefined";
},
sendTelemetry: async function sendTelemetry ( payload ) {
console.log("called sendTelemetry payload");
return "undefined";
},

/* Filter locally stored telemetry pings using these fields (if set)
/* Filter locally stored telemetry pings using these fields (if set)

n:
if set, no more than `n` pings.
Expand All @@ -175,82 +166,83 @@ Usage scenarios:
- enrollment / eligiblity using recent Telemetry behaviours or client environment
- addon testing scenarios
*/
filterTelemetry: async function filterTelemetry(filterTelemetryQuery) {
console.log("called filterTelemetry filterTelemetryQuery");
return [{ pingType: "main" }];
},
filterTelemetry: async function filterTelemetry ( filterTelemetryQuery ) {
console.log("called filterTelemetry filterTelemetryQuery");
return [{"pingType":"main"}];
},

/* Choose a element from `weightedVariations` array
/* Choose a element from `weightedVariations` array
based on various hashes of clientId

- shield: TBD
- pioneer: TBD
*/
deterministicVariation: async function deterministicVariation(
weightedVariations,
algorithm,
) {
console.log(
"called deterministicVariation weightedVariations, algorithm",
);
return "styleA";
},

/* Format url with study covariate queryArgs appended / mixed in.
deterministicVariation: async function deterministicVariation ( weightedVariations, algorithm ) {
console.log("called deterministicVariation weightedVariations, algorithm");
return "styleA";
},

/* Format url with study covariate queryArgs appended / mixed in.

Use this for constructing midpoint surveys.
*/
surveyUrl: async function surveyUrl(baseUrl, additionalFields) {
console.log("called surveyUrl baseUrl, additionalFields");
return "https://example.com?version=59.0&branch=studyA";
},

/* Using AJV, do jsonschema validation of an object. Can be used to validate your arguments, packets at client. */
validateJSON: async function validateJSON(someJson, jsonschema) {
console.log("called validateJSON someJson, jsonschema");
return { valid: true, errors: [] };
},

// https://firefox-source-docs.mozilla.org/toolkit/components/extensions/webextensions/events.html
/* Fires whenever any 'dataPermission' changes, with the new dataPermission object. Allows watching for shield or pioneer revocation. */
onDataPermissionsChange: new EventManager(
context,
"study.onDataPermissionsChange",
fire => {
const callback = value => {
fire.async(value);
};
// RegisterSomeInternalCallback(callback);
return () => {
// UnregisterInternalCallback(callback);
};
},
).api(),

// https://firefox-source-docs.mozilla.org/toolkit/components/extensions/webextensions/events.html
/* Fires when the study is 'ready' for the feature to startup. */
onReady: new EventManager(context, "study.onReady", fire => {
const callback = value => {
fire.async(value);
};
// RegisterSomeInternalCallback(callback);
return () => {
// UnregisterInternalCallback(callback);
};
}).api(),

// https://firefox-source-docs.mozilla.org/toolkit/components/extensions/webextensions/events.html
/* Listen for when the study wants to end */
onEndStudy: new EventManager(context, "study.onEndStudy", fire => {
const callback = value => {
fire.async(value);
};
// RegisterSomeInternalCallback(callback);
return () => {
// UnregisterInternalCallback(callback);
};
}).api(),
surveyUrl: async function surveyUrl ( baseUrl, additionalFields ) {
console.log("called surveyUrl baseUrl, additionalFields");
return "https://example.com?version=59.0&branch=studyA";
},
};

/* Using AJV, do jsonschema validation of an object. Can be used to validate your arguments, packets at client. */
validateJSON: async function validateJSON ( someJson, jsonschema ) {
console.log("called validateJSON someJson, jsonschema");
return {"valid":true,"errors":[]};
},


// https://firefox-source-docs.mozilla.org/toolkit/components/extensions/webextensions/events.html
/* Fires whenever any 'dataPermission' changes, with the new dataPermission object. Allows watching for shield or pioneer revocation. */
onDataPermissionsChange: new EventManager(
context,
"study.onDataPermissionsChange", fire => {
const callback = value => {
fire.async(value);
};
// RegisterSomeInternalCallback(callback);
return () => {
// UnregisterInternalCallback(callback);
};
}).api(),


// https://firefox-source-docs.mozilla.org/toolkit/components/extensions/webextensions/events.html
/* Fires when the study is 'ready' for the feature to startup. */
onReady: new EventManager(
context,
"study.onReady", fire => {
const callback = value => {
fire.async(value);
};
// RegisterSomeInternalCallback(callback);
return () => {
// UnregisterInternalCallback(callback);
};
}).api(),


// https://firefox-source-docs.mozilla.org/toolkit/components/extensions/webextensions/events.html
/* Listen for when the study wants to end */
onEndStudy: new EventManager(
context,
"study.onEndStudy", fire => {
const callback = value => {
fire.async(value);
};
// RegisterSomeInternalCallback(callback);
return () => {
// UnregisterInternalCallback(callback);
};
}).api(),

}
}
}
};
}
Loading