v1.4.15_Production

Aggregate v1.4.15

(July 2017)

1. Update javarosa to the new opendatakit-javarosa-2.3.0.jar
2. Fix: rev 210 sync protocol. User permissions were incorrectly being computed and filtered. This prevented resetting the server with new content from the device (but syncing with existing content worked fine). The rev 210 sync protocol is incompatible with anything prior to rev 210.

Upgrading

1. Do not upgrade if you are using ODK 2.0 rev 208 or earlier.
2. See v1.4.13 upgrade notes.

v1.4.14_Production: checkin for 1.4.14 - mixed Java 7 and Java 8 build.

Aggregate v1.4.14

(June 2017)

1. Update javarosa to the new opendatakit-javarosa-2.2.0.jar
2. Fix for JSON publisher correctly handling UTF-8
3. Added SHA-1 library to browser so that Basic Auth can now be configured and will now work (this capability is not exposed in the wizard-based installer/configurer; existing passwords need to be changed before they can be used in a Basic Auth configuration)
4. Revised sync protocol and database schema for ODK 2.0. This is incompatible with anything prior to rev 210 (not yet available).
5. Microsoft SQLServer configurations can now run on Linux and MacOSX boxes.
6. Updated to the latest AppEngine SDK.
7. Updated build environment to a mixed Java 7 and Java 8 environment.

Upgrading

1. Do not upgrade if you are using ODK 2.0 rev 208 or earlier.
2. See v1.4.13 upgrade notes.

v1.4.13_Production

Aggregate v1.4.13

(January 2017)

1. Add support for Microsoft SQL Server database and Azure SQL Server
2. Updated to expect Tomcat 8 and JDBC 4 libraries. This extensively impacted the datastore layer.
3. Rewrote the Google Sheets publisher to use a newer API.
4. Changed the PAUSED publisher state to expose a restart button. Fixed bug that prevented ABANDONED button from working. When a publisher enters the Paused state, this button triggers ODK Aggregate to resume from that state. This happens automatically every 8 minutes or so, but having a resume button simplifies integration testing. i.e., when a publisher is in the Paused state, you do NOT need to click this button -- but if you are testing a publisher to your own server, it can be useful to have.
5. Fixes for ODK 2.0 datatype form.
6. NaN +Infinity and -Infinity are now handled within Google AppEngine. These cannot be supported on systems using MySQL and SQL Server (a fundamental limitation of those databases). On PostgreSQL systems, if you alter your tables to use REAL instead of DECIMAL(38,0) columns, it will be able to store and process these special values.
7. Updated a multitude of jars (libraries).
8. If this is a first-time install using PostgreSQL, please see the upgrade steps below for the additional commands you need to execute to complete the PostgreSQL configuration. These commands are missing from the create_db_and_user.sql but are required for PostgreSQL operations.

Upgrading

1. If publishing to Google Sheets, you must go to the API Manager tab and enable the Google Sheets API.
2. Tomcat installs now require Tomcat 8 or higher. The war file produced by the installer will work as-is on those systems -- no need for any manual modifications to get things to work (though MySQL still requires downloading and copying the MySQL Connector/J, and SQL Server also has a manual configuration step).
3. If using an older MySQL database, please upgrade to MySQL 5.7 or higher; note that some MySQL releases expire all database passwords after 360 days. Please verify the behavior of your version of MySQL and either change the password expiration policy or create a calendar reminder to change the password before it expires. For ODK Aggregate, you will need to re-run the installer to specify any new password. For more information, see the MySQL documentation. e.g., MySQL password expiration policy.
4. The ODK 2.0 sync protocol in this release is compatible with the ODK 2.0 Rev 200, 202, 204, 206 and 208 tools. Note that you cannot just upgrade from a server earlier than 1.4.10; you have to delete everything and start over.
5. See v1.4.12 upgrade steps if upgrading from an earlier ODK Aggregate version.
6. If using an older PostgreSQL database, please upgrade to at least 9.4. For all PostgreSQL servers, you must execute an additional command to grant permissions to the schema on the server. This is a change from the earlier 9.1 install and the additional command is missing from the create_db_and_user.sql script. The commands to be executed in the pgAdmin query window are as follows, with your_database, your_schema and database_username replaced with the values you specified when you ran the installer:

\c "your_database";
alter schema "your_schema" owner to "database_username";

v1.4.12_Production

Aggregate v1.4.12

(September 2016)

1. update to installer with new language and links to documentation.
2. revise upload tool for Google AppEngine deployments to more-consistently display the token-entry dialog.
3. add .csv-based downloading and bulk updating of configured users and their capabilities.
4. change MySQL ODK 1.x and ODK 2.0 data table constructions to use MEDIUMBLOB columns for media attachments. Fix several issues around accessing attachments. See Updating instructions (below) and MySQL configuration instructions (on opendatakit.org) for configuration that is REQUIRED for MySQL.
5. tweak Google AppEngine configuration to reduce frontend usage hours and thereby reduce costs.
6. performance improvements to Google AppEngine task locks (primarily for ODK 2.0 support).
7. add "/users/list" URL that returns the list of all configured users and the roles they have been granted. Access to this URL requires authentication and is restricted to ODK Aggregate usernames and Google accounts (anonymousUser access is forbidden). If the authenticated user does not have Tables Super-user, Administer Tables, or Site Administrator permissions, a singleton list is returned that contains only information about their own identity and its capabilities.
8. changes to support row-level filtering in ODK 2.0.
9. implement a paginated view of ODK 2.0 data rows; 100 records per page.
10. upgrade to Google AppEngine 1.9.42 SDK

Known Issues

1. the ODK 2.0 "datatypes" example form and table (in the app-designer repository) fail to upload to ODK Aggregate. This will be addressed in a future release.

Upgrading

1. For MySQL installations, please make sure you have this server configuration (if this is a new database install, you can reduce max_allowed_packet to 16842752):

character_set_server=utf8
collation_server=utf8_unicode_ci
max_allowed_packet=1073741824

1. For ODK 1.x uses, no special upgrade steps are required.
2. If you are using ODK 2.0 features, you must visit the Site Admin / Permissions page and click Save Changes to complete the upgrade to this release (you do not need to modify anything on that page, just click the button).
3. The ODK 2.0 sync protocol in this release is compatible with the ODK 2.0 Rev 200, 202, 204, 206 and 208 tools. Note that you cannot just upgrade from a server earlier than 1.4.10; you have to delete everything and start over.
4. See v1.4.11 upgrade steps if upgrading from an earlier ODK Aggregate version.

v1.4.11_Production

Aggregate v1.4.11

(July 2016)

1. add a mutex around ODK 1.x form submission creation and updating. This should eliminate submission data corruption.
2. if only ODK 2.0 privileges are assigned, hide the Submissions tab.
3. fix: advisory memcache interaction to delete the entry when rolling back the datastore update.
4. add "Tables Super-user" as a configurable user capability. Users with this capability will be able to modify the table-level and row-level privileges on ODK 2.0 rev 206 data tables. These privileges are advisory and do not provide strong access or revision control.
5. add "/roles/granted" URL that returns the roles granted to an authenticated username or google account. This will be used by ODK 2.0 rev 206 tools to determine and save the capabilities of the configured user on a device. Those capabilities are then used to apply advisory access controls on the tables and rows.

Upgrading

1. For MySQL installations, please make sure you have this server configuration:

character_set_server=utf8
collation_server=utf8_unicode_ci
max_allowed_packet=1073741824

1. For ODK 1.x uses, no special upgrade steps are required.
2. If you are using ODK 2.0 features, you must visit the Site Admin / Permissions page and click Save Changes to complete the upgrade to this release (you do not need to modify anything on that page, just click the button).
3. The ODK 2.0 sync protocol in this release is compatible with the ODK 2.0 Rev 200, 202 and 204 tools. Note that you cannot just upgrade from a server earlier than 1.4.10; you have to delete everything and start over.
4. See v1.4.10 upgrade steps if upgrading from an earlier ODK Aggregate version.

v1.4.10_Production

Aggregate v1.4.10

(July 2016)

1. on AppEngine, add advisory memcache interactions inside database mutex implementation (TaskLockImpl) to detect and thereby reduce race conditions that could lead to database corruption. These changes do not alter the fundamental mutex mechanism. They may reduce the likelihood of a mutex failure.
2. fix ODK 2.0 sync: table-specific pre-loaded instance attachments under config/assets/csv/tableid/instances/... were not being identified as table-specific.
3. fix ODK 2.0 sync: add mutex around app-level file, table-level file and row-level attachment REST interactions. Eliminates the possibility of joint-updating that could corrupt the app-, table- or row-level manifests.
4. fix ODK 2.0 sync: change row-level file attachment bulk-update to not refetch the manifest after adding each file attachment. This had caused 1% failure rate on row-level attachment syncing when there were large numbers of file attachments (30-60) for each row.
5. fix ODK 2.0 sync: when authorization fails (permission denied), return UNAUTHORIZED response rather than DENIED. This reports an authorization failure on the client vs. a protocol error.

Upgrading

1. For MySQL installations, please make sure you have this server configuration:

character_set_server=utf8
collation_server=utf8_unicode_ci
max_allowed_packet=1073741824

1. For ODK 1.x uses, no special upgrade steps are required.
2. The ODK 2.0 sync protocol is compatible with the ODK 2.0 Rev 200, 202 and 204 tools.
3. For ODK 2.0 uses, if you have pre-loaded datasets (via tables.init) that have row-level attachments (e.g., geotagger example dataset), you should:
   1. remove these incorrectly-categorized files from the server (anything under assets/csv/table_id/instances/... on the Manage App Level Files sub-tab)
   1. deploy your device configuration to a clean device
   1. Reset App Server to correctly upload these files so that they appear under the Manage Table Level Files sub-tab.

v1.4.9_Production

Aggregate v1.4.9

(May 2016)

1. update to javarosa 2016-04-21
   * add XPath 3.1 math functions (e.g., exp, log, sin, cos, etc.)
2. fix AppEngine adaptation layer for Apache HttpClient so that Oauth2 authentications work
3. change version string in ODKAggregateAppEngineUpdater.jar to be a date stamp (2016-04-21)
4. fix ODK 2.0 interfaces to support ODK Aggregate username / password for authentication.
5. fix ODK 2.0 Sync so that a delete-table request can be repeated until successful or Not Found (404) when it times out.

Upgrading

1. For MySQL installations, please make sure you have this server configuration:

character_set_server=utf8
collation_server=utf8_unicode_ci
max_allowed_packet=1073741824

1. For all other installations, there are no special upgrade steps required.
2. The ODK 2.0 sync protocol is compatible with the ODK 2.0 Rev 200, 202 and 204 tools.

v1.4.8_Production

Aggregate v1.4.8

(April 2016)

1. Submission Filters are once again working.
2. Update javarosa library with fixes for:
   • fix date, time, dateTime handling of time zones (in collaboration with SurveyCTO)
   • fix itemset choice lists -- various issues.
   • fix jr:choice-name() context resolution so that it can work with relative paths and repeat groups (SurveyCTO contribution).
   • add CONTROL_FILE_CAPTURE for future arbitrary-file-attachment handling
   • use consistent UTF-8 treatment when writing and reading files
   • better detection and handling of binary-file-format changes across versions
   • build with Java 7
3. Add support for geotrace and geoshape to Export-to-KML (for ODK 1.x).
4. Update Apache HTTP Client libraries to 4.5.2; this should support SNI protocol interactions of external publishers (ODK 1.x - untested).
5. Change Google App Engine update mechanism to use new wizard update tool.
6. Change Google App Engine code to use EAR / module format.
7. Widespread jar update.
8. Correct bug in csv-export library shared with ODK 2.0 tools for exporting datasets in those tools (does not impact Export as CSV in the 1.x toolchain).
9. Add new bulk attachment-POST APIs for ODK 2.0 sync protocol.
10. Rework ui tests to use Chrome for the Selenium web testing.
11. Rework all of the eclipse projects and add documentation for setting up a tomcat eclipse debug environment.
12. Rework the Google App Engine maven projects to use the EAR / module format.
13. Update maven plugins and tools to newer versions.

Upgrading

1. For MySQL installations, please make sure you have this server configuration:

character_set_server=utf8
collation_server=utf8_unicode_ci
max_allowed_packet=1073741824

1. For all other installations, there are no special upgrade steps required.
2. The ODK 2.0 sync protocol is compatible with the ODK 2.0 Rev 200 and higher tools.

v1.4.7_Production

Aggregate v1.4.7

(May 2015)
🔻 Google disabled functionality used by this installer to upload ODK Aggregate to Google App Engine. See here for the work-around.

1. NOTE: Submission Filters are broken in this release.
2. when submissions rows are corrupted, consistently report the information necessary for end-users to delete or repair the corrupted submission. 4 error messages needed to be enhanced.
3. correct ODK 2.0 delete-table functionality so that it does not generate errors when it is not running on AppEngine. The code had been performing an unguarded explicit cast to an AppEngine-only implementation class.

Upgrading

1. For MySQL installations, please make sure you have this server configuration:

character_set_server=utf8
collation_server=utf8_unicode_ci
max_allowed_packet=1073741824

1. For all other installations, there are no special upgrade steps required.

v1.4.6_Production

Aggregate v1.4.6

(April 2015)
🔻 Google disabled functionality used by this installer to upload ODK Aggregate to Google App Engine. See here for the work-around.

1. NOTE: Submission Filters are broken in this release.
2. fix for Google Sheets publisher-creation problem caused by April 20th deprecation of the Google Docs API. We were making a single call to that obsolete API to create a blank Google Sheets document.
3. update to javarosa-2015-04-17 jar. That jar adds OpenStreetMap support to javarosa.
4. removal of the "Sign in with Google" mechanisms that used OpenID.
5. widespread update to jars, custom repository configuration, and the elimination of several custom-built jars because of the elimination of OpenID and the maturity of Maven as a version repository.

Upgrading

1. No special upgrade steps required.