Skip to content

Commit

Permalink
Remove references to Mozilla (#965)
Browse files Browse the repository at this point in the history
  • Loading branch information
englehardt authored Dec 20, 2021
1 parent 3354bc1 commit 8b62f18
Show file tree
Hide file tree
Showing 18 changed files with 45 additions and 46 deletions.
42 changes: 21 additions & 21 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,35 +107,35 @@ There has been a lot happening in OpenWPM over the last three months.

Here are the highlights:

- Introduced `seed_profile` ([#735](https://github.com/mozilla/OpenWPM/issues/735))
- Introduced `seed_profile` ([#735](https://github.com/openwpm/OpenWPM/issues/735))
As part of our long-standing effort to restore stateful crawling support we now allow
specifying a `seed_profile` that gets loaded for each fresh start of a browser.
More documentation can be found [here](docs/Configuration.md#load-a-profile).

- Added new WebExtension instrument `dns_instrument` ([#721](https://github.com/mozilla/OpenWPM/issues/721))
- Added new WebExtension instrument `dns_instrument` ([#721](https://github.com/openwpm/OpenWPM/issues/721))
This instrument was contributed by [@turban1988](https://github.com/turban1988) .
It allows to log the resolution of DNS requests. Unfortunately it is currently
undocumented. Adding docs is tracked in [#758](https://github.com/mozilla/OpenWPM/issues/758).
undocumented. Adding docs is tracked in [#758](https://github.com/openwpm/OpenWPM/issues/758).

- Moved `automation` to `openwpm` ([#793](https://github.com/mozilla/OpenWPM/issues/793))
- Moved `automation` to `openwpm` ([#793](https://github.com/openwpm/OpenWPM/issues/793))
This change might have the biggest impact on users as we changed the name of the
top-level package. We apologize for the inconvenience caused but felt it was
a good change overall as the new name is a lot more meaningful.

Internal changes:

- We replaced flake8 with black as our formatting tool [#740](https://github.com/mozilla/OpenWPM/issues/740)
- [@Metropass](https://github.com/Metropass) removed the built in extensions as they were horribly out of date [#754](https://github.com/mozilla/OpenWPM/issues/754)
- [@Ankushduacodes](https://github.com/Ankushduacodes) removed the `browser_settings` as they required a lot of code for very little benefit [#775](https://github.com/mozilla/OpenWPM/issues/775)
- [@Ankushduacodes](https://github.com/Ankushduacodes) made the `memory_watchdog` and `process_watchdog` part of the `manager_params` [#785](https://github.com/mozilla/OpenWPM/issues/785) [#787](https://github.com/mozilla/OpenWPM/issues/787)
- We replaced flake8 with black as our formatting tool [#740](https://github.com/openwpm/OpenWPM/issues/740)
- [@Metropass](https://github.com/Metropass) removed the built in extensions as they were horribly out of date [#754](https://github.com/openwpm/OpenWPM/issues/754)
- [@Ankushduacodes](https://github.com/Ankushduacodes) removed the `browser_settings` as they required a lot of code for very little benefit [#775](https://github.com/openwpm/OpenWPM/issues/775)
- [@Ankushduacodes](https://github.com/Ankushduacodes) made the `memory_watchdog` and `process_watchdog` part of the `manager_params` [#785](https://github.com/openwpm/OpenWPM/issues/785) [#787](https://github.com/openwpm/OpenWPM/issues/787)

Thanks to all the external contributors that worked on this release.
Besides the people mentioned above we also merged contributions from:

- [@jyothisjagan](https://github.com/jyothisjagan) [#769](https://github.com/mozilla/OpenWPM/issues/769)
- [@Prajwal7842](https://github.com/Prajwal7842) [#760](https://github.com/mozilla/OpenWPM/issues/760)
- [@7brokenmirrors](https://github.com/7brokenmirrors) [#776](https://github.com/mozilla/OpenWPM/issues/776)
- [@LordReigns](https://github.com/LordReigns) [#801](https://github.com/mozilla/OpenWPM/pull/801)
- [@jyothisjagan](https://github.com/jyothisjagan) [#769](https://github.com/openwpm/OpenWPM/issues/769)
- [@Prajwal7842](https://github.com/Prajwal7842) [#760](https://github.com/openwpm/OpenWPM/issues/760)
- [@7brokenmirrors](https://github.com/7brokenmirrors) [#776](https://github.com/openwpm/OpenWPM/issues/776)
- [@LordReigns](https://github.com/LordReigns) [#801](https://github.com/openwpm/OpenWPM/pull/801)

## v0.12.0 - 2020-08-26

Expand All @@ -144,9 +144,9 @@ Firefox 80.0.0 release
There have been no new features added in this release.
However there are two significant bugfixes worth highlighting:

- We hopefully fixed [a bug when hashing content](https://github.com/mozilla/OpenWPM/issues/711) where the same file could have multiple hashes
If you ran a big crawl and could repeat @birdsarah's analysis, we'd be grateful if you reported your results [here](https://github.com/mozilla/OpenWPM/issues/719)
- Fixed longstanding bug when [propagating exceptions from the BrowserManager to the TaskManager](https://github.com/mozilla/OpenWPM/issues/547) you should now be seeing
- We hopefully fixed [a bug when hashing content](https://github.com/openwpm/OpenWPM/issues/711) where the same file could have multiple hashes
If you ran a big crawl and could repeat @birdsarah's analysis, we'd be grateful if you reported your results [here](https://github.com/openwpm/OpenWPM/issues/719)
- Fixed longstanding bug when [propagating exceptions from the BrowserManager to the TaskManager](https://github.com/openwpm/OpenWPM/issues/547) you should now be seeing
the exception that happened in the BrowserManager in your logs

__NOTE:__ Please be aware that this release contains a regression related to https://bugzilla.mozilla.org/show_bug.cgi?id=1656405 and https://bugzilla.mozilla.org/show_bug.cgi?id=1599160.
Expand All @@ -169,14 +169,14 @@ New features:
new ``js_instrument_settings`` are in the
[Instrumentation and Data Access section of the README](./README.md#instrumentation-and-data-access).

- [Issue 641](https://github.com/mozilla/OpenWPM/issues/641)
- [PR 642](https://github.com/mozilla/OpenWPM/pull/642)
- [Issue 641](https://github.com/openwpm/OpenWPM/issues/641)
- [PR 642](https://github.com/openwpm/OpenWPM/pull/642)

Minor fixes:

- Asserting that unpickled exception is an exception [PR 705](https://github.com/mozilla/OpenWPM/pull/705)
- Minor fixes [PR 703](https://github.com/mozilla/OpenWPM/pull/703)
- Better crawling experience [PR 696](https://github.com/mozilla/OpenWPM/pull/696)
- Asserting that unpickled exception is an exception [PR 705](https://github.com/openwpm/OpenWPM/pull/705)
- Minor fixes [PR 703](https://github.com/openwpm/OpenWPM/pull/703)
- Better crawling experience [PR 696](https://github.com/openwpm/OpenWPM/pull/696)

No OpenWPM release was made with Firefox 78.0.

Expand All @@ -200,7 +200,7 @@ Changes:
* Refactor extension instumentation to live in a separate module
* Re-write logger and add support for logging to sentry
* Add a crawler.py crawl script that can be used for cloud deployments like
the type documented in https://github.com/mozilla/openwpm-crawler
the type documented in https://github.com/openwpm/OpenWPM-crawler
* Add support for Firefox's native headless mode alongside XVFB
* Add Dockerfile and automatically deploy builds to dockerhub
* Add a Navigation instrument that records navigation events
Expand Down
2 changes: 1 addition & 1 deletion Extension/firefox/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# OpenWPM Client Extension

Used by the OpenWPM platform. This extension implements the OpenWPM instrumentation module (https://github.com/mozilla/openwpm-webext-instrumentation/) in a WebExtension.
Used by the OpenWPM platform. This extension implements the OpenWPM instrumentation module (https://github.com/openwpm/OpenWPM-webext-instrumentation/) in a WebExtension.
5 changes: 2 additions & 3 deletions Extension/firefox/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@openwpm/webext-firefox",
"description": "OpenWPM Client extension",
"author": "Mozilla",
"author": "OpenWPM Contributors",
"dependencies": {
"openwpm-webext-instrumentation": "../webext-instrumentation"
},
Expand Down Expand Up @@ -35,14 +35,13 @@
"homepage": "",
"keywords": [
"firefox",
"mozilla",
"openwpm"
],
"license": "MPL-2.0",
"private": true,
"repository": {
"type": "git",
"url": "git+https://github.com/mozilla/OpenWPM.git"
"url": "git+https://github.com/openwpm/OpenWPM.git"
},
"scripts": {
"prebuild": "cd ../webext-instrumentation && npm run build && cd - && webpack",
Expand Down
4 changes: 2 additions & 2 deletions Extension/webext-instrumentation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,15 @@ The instrumentation leverages the available [JavaScript APIs for WebExtensions](
- Response body content
- Cookie Access (Experimental)

More specifically, all packets sent by the instrumentation conform to [these interfaces](https://github.com/mozilla/OpenWPM/tree/master/Extension/webext-instrumentation/src/schema.ts).
More specifically, all packets sent by the instrumentation conform to [these interfaces](https://github.com/openwpm/OpenWPM/tree/master/Extension/webext-instrumentation/src/schema.ts).

## Usage

The instrumentation is designed to invoke a `dataReceiver` object whenever a packet or log entry is available.

Pending proper documentation, the best way to see how this library is used is to check how the instrumentation is incorporated into the following extensions:

* https://github.com/mozilla/OpenWPM/tree/master/Extension/firefox
* https://github.com/openwpm/OpenWPM/tree/master/Extension/firefox
* https://github.com/mozilla/jestr-pioneer-shield-study

## Npm publishing
Expand Down
2 changes: 1 addition & 1 deletion Extension/webext-instrumentation/src/lib/js-instruments.ts
Original file line number Diff line number Diff line change
Expand Up @@ -564,7 +564,7 @@ export function getInstrumentJS(eventId: string, sendMessagesToLogger) {
logSettings,
);
// Restore the original prototype and constructor so that instrumented classes remain intact
// TODO: This may have introduced prototype pollution as per https://github.com/mozilla/OpenWPM/issues/471
// TODO: This may have introduced prototype pollution as per https://github.com/openwpm/OpenWPM/issues/471
if (origProperty.prototype) {
instrumentedFunctionWrapper.prototype = origProperty.prototype;
if (origProperty.prototype.constructor) {
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

OpenWPM
[![Documentation Status](https://readthedocs.org/projects/openwpm/badge/?version=latest)](https://openwpm.readthedocs.io/en/latest/?badge=latest)
[![Build Status](https://github.com/mozilla/openwpm/workflows/Tests%20and%20linting/badge.svg?branch=master)](https://github.com/mozilla/openwpm/actions?query=branch%3Amaster)
[![Build Status](https://github.com/openwpm/OpenWPM/workflows/Tests%20and%20linting/badge.svg?branch=master)](https://github.com/openwpm/OpenWPM/actions?query=branch%3Amaster)
[![OpenWPM Matrix Channel](https://img.shields.io/matrix/OpenWPM:mozilla.org?label=Join%20us%20on%20matrix&server_fqdn=mozilla.modular.im)](https://matrix.to/#/#OpenWPM:mozilla.org?via=mozilla.org) <!-- omit in toc -->
=======

Expand Down
2 changes: 1 addition & 1 deletion crawler.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@
# Loads the default manager params
# We can't use more than one browser per instance because the job management
# code below requires blocking commands. For more context see:
# https://github.com/mozilla/OpenWPM/issues/470
# https://github.com/openwpm/OpenWPM/issues/470
NUM_BROWSERS = 1
manager_params = ManagerParams()
browser_params = [BrowserParams() for _ in range(NUM_BROWSERS)]
Expand Down
6 changes: 3 additions & 3 deletions docs/Configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ left out of this section.
"headless" browsing because you do not need to be in a full X environment e.g. on a
server. `headless` mode is supported on all platforms and is implemented by the browser
but has some differences. For example WebGL is not supported in headless mode.
<https://github.com/mozilla/OpenWPM/issues/448> discusses additional factors to consider
<https://github.com/openwpm/OpenWPM/issues/448> discusses additional factors to consider
when picking a `display_mode`.
- `browser`
- Used to specify which browser to launch. Currently, only `firefox` is
Expand Down Expand Up @@ -116,7 +116,7 @@ They allow collecting various types of data that is labeled per visit and aim to
much of a website's behaviour as we can.

If you feel that we are missing a fundamental instrument and are willing to implement it,
please [file an issue](https://github.com/mozilla/OpenWPM/issues/new?labels=feature-request),
please [file an issue](https://github.com/openwpm/OpenWPM/issues/new?labels=feature-request),
and we'll try to assist you in writing that instrument.

Below you'll find a description for every single instrument, however if you
Expand Down Expand Up @@ -283,7 +283,7 @@ or by manually tarring a firefox profile directory.
> Please note that you must tar the contents of the profile directory
> and not the directory itself.
> (For an example of the difference please see
> [here](https://github.com/mozilla/OpenWPM/issues/790#issuecomment-791316632))
> [here](https://github.com/openwpm/OpenWPM/issues/790#issuecomment-791316632))
The profile will be automatically extracted and loaded into the browser
instance for which the configuration parameter was set.
Expand Down
2 changes: 1 addition & 1 deletion docs/Release-Checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ We aim to release a new version of OpenWPM with each new Firefox release (~1 rel
5. Increment the version number in [VERSION](../VERSION)
6. Add a summary of changes since the last version to [CHANGELOG](../CHANGELOG.md)
7. Squash and merge the release PR to master.
8. Publish a new release from https://github.com/mozilla/OpenWPM/releases:
8. Publish a new release from https://github.com/openwpm/OpenWPM/releases:
1. Click "Draft a new release".
2. Enter the "Tag version" and "Release title" as `vX.X.X`.
3. In the description:
Expand Down
2 changes: 1 addition & 1 deletion docs/Using_OpenWPM.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ for _ in range(num_browsers):

Please have a look at [`custom_command.py`](../custom_command.py). Note that custom commands must be
defined in a separate module and imported. They can't be defined within the main crawl script.
See [#837](https://github.com/mozilla/OpenWPM/issues/837).
See [#837](https://github.com/openwpm/OpenWPM/issues/837).

## Running a simple analysis

Expand Down
2 changes: 1 addition & 1 deletion install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ eval "$(conda shell.bash hook)"
if [ "$1" != "--skip-create" ]; then
echo 'Creating / Overwriting openwpm conda environment.'
# `PYTHONNOUSERSITE` set so python ignores local user site libraries when building the env
# See: https://github.com/mozilla/OpenWPM/pull/682#issuecomment-645648939
# See: https://github.com/openwpm/OpenWPM/pull/682#issuecomment-645648939
PYTHONNOUSERSITE=True conda env create --force -q -f environment.yaml
fi

Expand Down
2 changes: 1 addition & 1 deletion openwpm/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,7 @@ def validate_browser_params(browser_params: BrowserParams) -> None:
raise ConfigError(
"The callstacks instrument currently doesn't work without "
"the JS instrument enabled. see: "
"https://github.com/mozilla/OpenWPM/issues/557"
"https://github.com/openwpm/OpenWPM/issues/557"
)

if not isinstance(browser_params.save_content, bool) and not isinstance(
Expand Down
2 changes: 1 addition & 1 deletion openwpm/deploy_browsers/configure_firefox.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from ..config import BrowserParams

# TODO: Remove hardcoded geckodriver default preferences. See
# https://github.com/mozilla/OpenWPM/issues/867
# https://github.com/openwpm/OpenWPM/issues/867
# Source of preferences:
# https://hg.mozilla.org/mozilla-central/file/tip/testing/geckodriver/src/prefs.rs
# https://hg.mozilla.org/mozilla-central/file/tip/testing/geckodriver/src/marionette.rs
Expand Down
6 changes: 3 additions & 3 deletions openwpm/deploy_browsers/deploy_firefox.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def deploy_firefox(
# Set a custom profile that is used in-place and is not deleted by geckodriver.
# https://firefox-source-docs.mozilla.org/testing/geckodriver/CrashReports.html
# Using FirefoxProfile breaks stateful crawling:
# https://github.com/mozilla/OpenWPM/issues/423#issuecomment-521018093
# https://github.com/openwpm/OpenWPM/issues/423#issuecomment-521018093
fo.add_argument("-profile")
fo.add_argument(str(browser_profile_path))

Expand Down Expand Up @@ -117,7 +117,7 @@ def deploy_firefox(

# Geckodriver currently places the user.js file in the wrong profile
# directory, so we have to create it manually here.
# TODO: See https://github.com/mozilla/OpenWPM/issues/867 for when
# TODO: See https://github.com/openwpm/OpenWPM/issues/867 for when
# to remove this workaround.
# Load existing preferences from the profile's user.js file
prefs = configure_firefox.load_existing_prefs(browser_profile_path)
Expand Down Expand Up @@ -162,7 +162,7 @@ def deploy_firefox(
firefox_binary=fb,
options=fo,
log_path=interceptor.fifo,
# TODO: See https://github.com/mozilla/OpenWPM/issues/867 for
# TODO: See https://github.com/openwpm/OpenWPM/issues/867 for
# when to remove this
service_args=["--marionette-port", str(marionette_port)],
)
Expand Down
2 changes: 1 addition & 1 deletion openwpm/storage/storage_controller.py
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ async def _handle_meta(self, visit_id: VisitId, data: Dict[str, Any]) -> None:
signal that a visit_id is complete.
- initialize: TODO: Start complaining if we receive data for a visit_id
before the initialize event happened.
See also https://github.com/mozilla/OpenWPM/issues/846
See also https://github.com/openwpm/OpenWPM/issues/846
"""
action: str = data["action"]
if action == ACTION_TYPE_INITIALIZE:
Expand Down
2 changes: 1 addition & 1 deletion openwpm/storage/storage_providers.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ class UnstructuredStorageProvider(StorageProvider):
This is primarily content loaded by websites.
Don't make any assumptions about the data (especially don't assume it's valid unicode)
In the future this interface will be expanded to address the needs of https://github.com/mozilla/OpenWPM/issues/232
In the future this interface will be expanded to address the needs of https://github.com/openwpm/OpenWPM/issues/232
"""

@abstractmethod
Expand Down
2 changes: 1 addition & 1 deletion scripts/repin.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ eval "$(conda shell.bash hook)"

# Create openwpm env with unpinned yaml file
# `PYTHONNOUSERSITE` set so python ignores local user site libraries when building the env
# See: https://github.com/mozilla/OpenWPM/pull/682#issuecomment-645648939
# See: https://github.com/openwpm/OpenWPM/pull/682#issuecomment-645648939
PYTHONNOUSERSITE=True conda env create --force -q -f environment-unpinned.yaml

# Activate
Expand Down
4 changes: 2 additions & 2 deletions test/manual_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ def cleanup_server():
fo = Options()
fo.add_argument("-profile")
fo.add_argument(str(browser_profile_path))
# TODO: See https://github.com/mozilla/OpenWPM/issues/867 for when
# TODO: See https://github.com/openwpm/OpenWPM/issues/867 for when
# to remove manually creating user.js
prefs = configure_firefox.load_existing_prefs(browser_profile_path)
prefs.update(configure_firefox.DEFAULT_GECKODRIVER_PREFS)
Expand All @@ -139,7 +139,7 @@ def cleanup_server():
firefox_binary=fb,
options=fo,
# Use the default Marionette port.
# TODO: See https://github.com/mozilla/OpenWPM/issues/867 for
# TODO: See https://github.com/openwpm/OpenWPM/issues/867 for
# when to remove this
service_args=["--marionette-port", "2828"],
)
Expand Down

0 comments on commit 8b62f18

Please sign in to comment.