v2.0.1

This is a major rework of Vizier, featuring a significant update to the Vizier frontend, as well as substantial behind-the-scenes infrastructure changes.

Full Details: https://github.com/VizierDB/vizier-scala/milestone/6

Installing

Prerequisites

• JDK 11 (later versions break Spark)
• Python3.9+

Manual

• Download the vizier release binary and add it to in your path

Updating

Note: This release WILL very likely break the 1.x vizier.db file format. To migrate to the 2.x format:

• Export your 1.x project using Vizier 1.x
  • Go to Export Project in the Project menu, OR
  • use the vizierdb export command
• Import your project with Vizier 2.x

There is a small chance that your project will work without issues. Make sure to back up your vizier.db directory. You will almost certainly need to re-execute the entire workflow.

Release Summary

User-Facing Features

• A completely redesigned, cleaner, snappier, faster interface
• The interface is now reactive. See cell output as it is generated, and get immediate feedback when a cell completes.
• Vizier is now based off of Spark 3.2 (#103)
• "infinite-scroll" style spreadsheets and data tables (#51 #8 #44)
• Fully-featured spreadsheets w/arbitrary cell references (#55)
• No longer a single-page app. That means, among other things, that browser-based navigation works correctly.
• Table of Contents is now faster and more prominent.
• Quick-access to artifacts and related actions (e.g., download, jump-to-cell, etc...)
• Significant performance improvements in the backend (#182 )
• Python cells now have a 'environment' option, allowing them to be run in non-system environments configured by vizier (#58 #164 #58)
• Python cells now try to infer the return type of an exported method for use with Spark from type annotations (#219 )
• Completely redesigned 'New Cell' UX, using icons to highlight prominent cell types. (#191 )
• Support for JDK 11 (#214 )
• Added a Documentation area (#278 )
• Data visualization tools are now quicker and easier to access; Standard plotting types are now directly accessible from the new cell picker, each with plot-specific interfaces. (#264 , #281, #233 ; thanks to @marysunc , @cameronjtoy, @naiyyirh )

Bug-Fixes

• Exported Python functions now work with SQL cells (#143)
• No more crashes when displaying doubles
• Python module warnings no longer include spurious blanks (#144)
• When downloading a dataset as CSV, convert Geometry objects to WKT strings (#138)

Developer-Facing Features

• All code now uses a single language (Scala), with a single source of truth about API formats
• Migrated to mill (#96). It's faster, and writing build scripts does not require learning a brand new DSL written over scala.
• Eliminated dependency on mimir-api (#85 #147). Bugs with lenses no longer need fiddly lock-step updates to multiple packages.
• Migrated the server backend to akka-http. This should break jetty dependency hell, and make integration with Vizier Server easier (#149 #216 )

Features Migrated from v1.2

• History Support (#152 #175)
• Code Snippets (#168 )
• Re-introduced read-only workflows (#175 )

v2.0.0-rc3

Misc Bugfixes

• Fixed a typo in the Python 'get artifact' snippet.
• Load dataset now infers clean names (Fixes #297 , should make #298 less painful).
• Set working directory properly in scala cells.
• Datasets exported to the current working directory are now moved into place using Files.move() instead of the deprecated File.renameTo().
• CDF plots now correctly put the CDF attribute on the y axis.

v2.0-rc2

This is a major rework of Vizier, featuring a significant update to the Vizier frontend, as well as substantial behind-the-scenes infrastructure changes.

https://github.com/VizierDB/vizier-scala/milestone/6

Note: This release WILL very likely break the 1.x vizier.db file format. To migrate to the 2.x format:

• Export your 1.x project using Vizier 1.x
  • Go to Export Project in the Project menu, OR
  • use the vizierdb export command
• Import your project with Vizier 2.x

There is a small chance that your project will work without issues. Make sure to back up your vizier.db directory. You will almost certainly need to re-execute the entire workflow.

User-Facing Features

• A completely redesigned, cleaner, snappier, faster interface
• The interface is now reactive. See cell output as it is generated, and get immediate feedback when a cell completes.
• Vizier is now based off of Spark 3.2 (#103)
• "infinite-scroll" style spreadsheets and data tables (#51 #8 #44)
• Fully-featured spreadsheets w/arbitrary cell references (#55)
• No longer a single-page app. That means, among other things, that browser-based navigation works correctly.
• Table of Contents is now faster and more prominent.
• Quick-access to artifacts and related actions (e.g., download, jump-to-cell, etc...)
• Significant performance improvements in the backend (#182 )
• Python cells now have a 'environment' option, allowing them to be run in non-system environments configured by vizier (#58 #164 #58)
• Python cells now try to infer the return type of an exported method for use with Spark from type annotations (#219 )
• Completely redesigned 'New Cell' UX, using icons to highlight prominent cell types. (#191 )
• Support for JDK 11 (#214 )
• Added a Documentation area (#278 )
• Data visualization tools are now quicker and easier to access; Standard plotting types are now directly accessible from the new cell picker, each with plot-specific interfaces. (#264 , #281, #233 ; thanks to @marysunc , @cameronjtoy, @naiyyirh )

Bug-Fixes

• Exported Python functions now work with SQL cells (#143)
• No more crashes when displaying doubles
• Python module warnings no longer include spurious blanks (#144)
• When downloading a dataset as CSV, convert Geometry objects to WKT strings (#138)

Developer-Facing Features

• All code now uses a single language (Scala), with a single source of truth about API formats
• Migrated to mill (#96). It's faster, and writing build scripts does not require learning a brand new DSL written over scala.
• Eliminated dependency on mimir-api (#85 #147). Bugs with lenses no longer need fiddly lock-step updates to multiple packages.
• Migrated the server backend to akka-http. This should break jetty dependency hell, and make integration with Vizier Server easier (#149 #216 )

Features Migrated from v1.2

• History Support (#152 #175)
• Code Snippets (#168 )
• Re-introduced read-only workflows (#175 )

v1.2.0

Major Features

• Support triggering workflow execution from the command line (#113 )
• Support rendering vector and raster data via Leaflet in a GeoJson column (#124 ; #172 by @bicknrown )
• A new scheduler opportunistically runs workflow cells in parallel when static inference of the cell permits it (#137 )
• A new vizier run command allows workflows to be re-run from the command line without interactions thorugh the UI. (#113 )
• Image data is now supported throughout the Vizier stack. The UI displays images inline; Python treats images as binary blobs, and Spark internals can interact with them as normal data. (#63 contributed by @sohampatel12 )

Bugfixes

• -p flag is no longer ignored ( 466955b by @lordpretzel )
• Python code snippet cleanup ( c7ca63bda0fc026e6bde5360122ab8c817c1f5d2 by @lordpretzel )

New Cells

• A new Split Dataset transformation partitions data into two separate datasets (#29 )

Papercuts

• Auto-launch UI on app start (#117)
• A new command lets you quickly attach a geometry to any dataset with lat/lon coordinates in numeric columns.

Internals

• Javascript cell messages can provide CSS dependencies (#137 )

v1.1.1

• UI updates needed to support updated defaults. (closes #108 )
• Python cells no longer barf when trying to insert rows with non-string types (@sohampatel12 )
• Python cells no longer barf on date and datetime types (closes #130 )
• Python cells now work with non-shapely geometry formats that support the geo-datatype standard (closes #129 )
• Vizual (resp python w/ use_deltas = true) cells no longer barf on geometry types (closes #131 )
  ` Spark session is now accessible through Execution context instead of having to fish around through system classes (closes #127 )
• Readded support for publish dataset locally (closes #133 )
• Caveats 1.3.4 fixes a bug with propagating caveats through EXISTS queries (closes #135)

v1.1.0

Major Features

• The UpdateCell and Script Vizual commands now have a comment field that, if set, will apply a caveat to cells affected by an UpdateCell vizual operation (closes #3)
• Add support for parameter artifacts (closes #77)
• Added an "Aggregate Dataset" command (closes #25)
• Added a "Filter Dataset" command (closes #26)
• Added support for freezing individual cells (closes #82)
• Main menu (Branch menu) now has a "Cancel workflow" option (closes #73)

Minor Features

• Add cache reset toggle to the geocoding lens (closes #79)
• Make LOAD DATASET summary prettier for URLs (closes #40)
• Added support for pickled python objects as artifacts (closes #14)
• Profiler now generates a null value count for all columns and UI displays it (closes #72)
• Plot command now provides a "Lines Without Points" option (closes #57)
• Added a materialize option for Sampling cells (closes #91)
• If the first line of a python script is a comment, use it as the cell's ToC title.
• LoadDataset now defaults to infer types, assume headers, and load csvs with error reporting (closes #108 )

Bugfixes

• Modify vizier to use relative paths (closes #70)
• Rewrote bokeh support to lazily load dependencies and use unique cell IDs (closes #84, closes #69)
• PythonUDF support now implemented in vizier-scala (closes #88)
• Plot now filters out nulls on the x axis and properly encodes y-axis nulls (closes #71)
• Spreadsheet navigation buttons no longer break when used with "Edit as Spreadsheet" (closes #4)
• The state management system has been overhauled (closes #6, closes #7)
• Upstream fixes to fix caveat enumeration for individual cells on queries with 3+ way joins. (closes #90)
• LoadDataset now properly reports errors instead of failing with a null pointer exception (re: #111)
• LoadDataset now correctly works with http/s URLs (re: #111)
• Missing Value Lens caveats are now displayed correctly in caveat lists (closes #115 )
• Full caveat list no longer crashes the UI (closes #109 )

Infrastructure

• Added a server mode flag (closes #81)
• Add support for javascript cell outputs (closes #45)
• Added a vizier doctor command to sanity check the database (closes #75)
• Added a websocket endpoint w/ enough state to generate and maintain the frontend view (closes #59); PoC Client Here: https://github.com/VizierDB/scala-ui/
• Python cells now default to applying updates via Vizual rather than by providing a dataset literal to overwrite the existing dataset (closes #38 , closes #74)
• Provide a --working-directory parameter to override the current working directory in relative URLs. (closes #105)

v1.0.0

v1.0