We need to set the CHARTS_FIELDS for our custom handler, that autogenerate documentation
for chart types fields.
The CKAN instance will be initialized, because we will call validators and helpers in this code. Therefore, you'll need +another environment variable to set the CKAN configuration file path:
+export CKAN_INI=$PWD/config/ckan.ini
+The config file must enable the ckanext-charts plugin:
ckan.plugins = ... charts_view charts_builder_view ...
+If you're working on the documentation, you can run the following command to start a live-reloading server without gathering the chart types fields data. It will speed up the process significantly, as we won't need to wait for the CKAN initialization:
diff --git a/search/search_index.json b/search/search_index.json index aa0e7d3..b5c063c 100644 --- a/search/search_index.json +++ b/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-\\.\\_]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":""},{"location":"#ckanext-charts","title":"ckanext-charts","text":"This extension, ckanext-charts, provides additional functionality for working with charts in CKAN. It allows users to create, manage, and visualize charts based on data stored in CKAN datasets.
The extension includes features such as chart creation, chart editing, chart embedding, and chart sharing. It also supports various chart types, including bar charts, line charts, pie charts, and more.
With ckanext-charts, users can easily generate interactive and visually appealing charts to enhance data analysis and presentation in CKAN.
See the documentation for more information.
"},{"location":"#quick-start","title":"Quick start","text":"PyPi with pip install ckanext-charts[pyarrow]charts_view to the list of plugins in your CKAN config (ckan.plugins = charts_view charts_builder_view)To install ckanext-charts for development, activate your CKAN virtualenv and do:
git clone https://github.com/DataShades/ckanext-charts.git\ncd ckanext-charts\npip install -e '.[dev]'\nTo run the tests, do:
pytest --ckan-ini=test.ini\nWe are using mkdocs to build the documentation. To build and deploy the documentation, do:
CHARTS_FIELDS=1 mkdocs build && mkdocs gh-deploy\nWe need to set the CHARTS_FIELDS for our custom handler, that autogenerate documentation for chart types fields.
If you're working on the documentation, you can run the following command to start a live-reloading server without gathering the chart types fields data. It will speed up the process significantly, as we won't need to wait for the CKAN initialization:
mkdocs serve -a 127.0.0.1:8001\nAGPL
"},{"location":"configuration/","title":"Configuration","text":"Configuration options
There is a number of configuration options available for the extension. You can set them in the CKAN configuration file or using the admin configuration page.
"},{"location":"configuration/#configuration-options","title":"Configuration options","text":""},{"location":"configuration/#cache-strategy","title":"Cache strategy","text":"ckanext.charts.cache_strategy [optional]
Cache strategy for chart data.
Options: redis, file_orc, file_csv
Type: str
Default: redis
ckanext.charts.redis_cache_ttl [optional]
Time to live for the Redis cache in seconds. Set to 0 to disable cache.
Type: int
Default: 3600
ckanext.charts.file_cache_ttl [optional]
Time to live for the File cache in seconds. Set to 0 to disable cache.
Type: int
Default: 3600
ckanext.charts.enable_cache [optional]
Enable cache for the charts.
WarningIf the cache is disabled, the chart data will be fetched from the data source every time the chart is rendered. It could lead to performance issues if the data source is slow or the data is too large.
Type: bool
Default: true
ckanext.charts.include_htmx_asset [optional]
Include HTMX assets. Could be disabled if another plugins are including it.
Type: bool
Default: true
ckanext.charts.reinit_ckan_js_modules [optional]
Reinit CKAN JS modules after HTMX swap.
Type: bool
Default: false
ckanext.charts.allow_anon_building_charts [optional]
By default, only authenticated users can build charts. If you want to allow anonymous users to build charts, set this option to true.
Type: bool
Default: false
The extension provides an admin configuration page where you can set all the listed configuration options. The admin page available only using an extension ckanext-admin-panel. The admin panel is a separate extension that provides an alternative admin interface for CKAN. It allows you to manage CKAN settings and other extensions settings through the web interface.
This page could be accessed by the following URL: /admin-panel/charts/config, but it will be registered only if the ckanext-admin-panel is installed and enabled.
charts_allow_anon_building_charts()","text":"Check if anonymous users are allowed to build charts.
RETURNS DESCRIPTIONbool True if anonymous users are allowed to build charts, False otherwise.
TYPE: bool
ckanext/charts/helpers.py def charts_allow_anon_building_charts() -> bool:\n \"\"\"Check if anonymous users are allowed to build charts.\n\n Returns:\n bool: True if anonymous users are allowed to build charts, False otherwise.\n \"\"\"\n return config.allow_anon_building_charts()\ncharts_get_resource_columns(resource_id)","text":"Get the columns of the given resource.
PARAMETER DESCRIPTIONresource_id Resource ID
TYPE: str
str JSON string of columns options
TYPE: str
ckanext/charts/helpers.py def charts_get_resource_columns(resource_id: str) -> str:\n \"\"\"Get the columns of the given resource.\n\n Args:\n resource_id: Resource ID\n\n Returns:\n str: JSON string of columns options\n \"\"\"\n fetcher = DatastoreDataFetcher(resource_id)\n\n return json.dumps(\n [{\"id\": col, \"title\": col} for col in fetcher.fetch_data().columns],\n )\ncharts_include_htmx_asset()","text":"Checks if the HTMX asset should be included.
RETURNS DESCRIPTIONbool True if the HTMX asset should be included, False otherwise.
TYPE: bool
ckanext/charts/helpers.py def charts_include_htmx_asset() -> bool:\n \"\"\"Checks if the HTMX asset should be included.\n\n Returns:\n bool: True if the HTMX asset should be included, False otherwise.\n \"\"\"\n return config.include_htmx_asset()\ncharts_reinit_ckan_js_modules()","text":"Checks if CKAN JS modules should be reinitialized.
RETURNS DESCRIPTIONbool True if CKAN JS modules should be reinitialized, False otherwise.
TYPE: bool
ckanext/charts/helpers.py def charts_reinit_ckan_js_modules() -> bool:\n \"\"\"Checks if CKAN JS modules should be reinitialized.\n\n Returns:\n bool: True if CKAN JS modules should be reinitialized, False otherwise.\n \"\"\"\n return config.reinit_ckan_js_modules()\ncharts_user_is_authenticated()","text":"Check if the user is authenticated.
RETURNS DESCRIPTIONbool True if the user is authenticated, False otherwise.
TYPE: bool
ckanext/charts/helpers.py def charts_user_is_authenticated() -> bool:\n \"\"\"Check if the user is authenticated.\n\n Returns:\n bool: True if the user is authenticated, False otherwise.\n \"\"\"\n return tk.current_user.is_authenticated\nget_available_chart_engines_options()","text":"Get the available chart engines.
RETURNS DESCRIPTIONlist[dict[str, str]] List of chart engines options
Source code inckanext/charts/helpers.py def get_available_chart_engines_options() -> list[dict[str, str]]:\n \"\"\"Get the available chart engines.\n\n Returns:\n List of chart engines options\n \"\"\"\n return [{\"value\": engine, \"text\": engine} for engine in get_chart_engines()]\nget_file_cache_size()","text":"Get the size of the file cache in a human-readable format.
RETURNS DESCRIPTIONstr Human-readable file cache size
TYPE: str
ckanext/charts/helpers.py def get_file_cache_size() -> str:\n \"\"\"Get the size of the file cache in a human-readable format.\n\n Returns:\n str: Human-readable file cache size\n \"\"\"\n return utils.printable_file_size(count_file_cache_size())\nget_redis_cache_size()","text":"Get the size of the Redis cache in a human-readable format.
RETURNS DESCRIPTIONstr Human-readable Redis cache size
TYPE: str
ckanext/charts/helpers.py def get_redis_cache_size() -> str:\n \"\"\"Get the size of the Redis cache in a human-readable format.\n\n Returns:\n str: Human-readable Redis cache size\n \"\"\"\n return utils.printable_file_size(count_redis_cache_size())\nRequires Redis 7+
Compatibility with core CKAN versions:
CKAN version Compatible? 2.9 and earlier no 2.10+ yes"},{"location":"install/#installation_1","title":"Installation","text":"Install the extension from PyPI:
pip install ckanext-charts\nIf you want to use ORC file cache, you have to install the extension with the pyarrow extra:
pip install ckanext-charts[pyarrow]\nEnable the view and builder plugins in your CKAN configuration file (e.g. ckan.ini or production.ini):
ckan.plugins = ... charts_view charts_builder_view ...\nThe extension requires the following CKAN extensions to be installed and enabled:
ckanext-scheming: We're using the scheming extension to create custom forms for the chart builders.
ckanext-admin-panel (optional): If you want to use the admin configuration page, you need to install and enable this extension. The admin panel is a separate extension that provides an alternative admin interface for CKAN. It allows you to manage CKAN settings and other extensions settings through the web interface and significantly extends the default CKAN admin interface.
ImportError: lxml.html.clean module is now a separate project lxml_html_clean
Install lxml[html_clean] or lxml_html_clean directly using pip.
This extension offers two plugins: charts_view and charts_builder_view. The first plugin is a standard chart builder designed for system administrators or anyone with permission to edit or create a resource view. Creating a preview using this plugin generates a static chart with saved settings. This chart retains the settings and will always be rendered in the same way.
The charts_builder_view plugin does not have any special settings during the creation or editing stages. Instead, it renders a chart builder JS widget on a resource page, allowing any user to interact with the resource data and create a chart that meets their needs. In the future, we plan to implement a feature that will allow users to save their charts.
The user chart builder could be restricted to only authenticated users. See the config settings section for more information. In this case, user will be forced to log in to build a chart. After the user logs in, user will be redirected to the chart builder page back.
"},{"location":"utils/","title":"Utility Functions","text":""},{"location":"utils/#charts.utils.build_chart_for_data","title":"build_chart_for_data(settings, data)","text":"Build chart for the given dataframe and settings.
PARAMETER DESCRIPTIONsettings Chart settings
TYPE: dict[str, Any]
data Dataframe with data
TYPE: DataFrame
str | None Chart config as JSON string
Source code inckanext/charts/utils.py def build_chart_for_data(settings: dict[str, Any], data: pd.DataFrame) -> str | None:\n \"\"\"Build chart for the given dataframe and settings.\n\n Args:\n settings: Chart settings\n data: Dataframe with data\n\n Returns:\n Chart config as JSON string\n \"\"\"\n return _build_chart(settings, data)\nbuild_chart_for_resource(settings, resource_id)","text":"Build chart for the given resource ID.
Uses a DatastoreDataFetcher to fetch data from the resource.
PARAMETER DESCRIPTIONsettings Chart settings
TYPE: dict[str, Any]
resource_id Resource ID
TYPE: str
str | None str | None: Chart config as JSON string or None if the chart can't be built
Source code inckanext/charts/utils.py def build_chart_for_resource(settings: dict[str, Any], resource_id: str) -> str | None:\n \"\"\"Build chart for the given resource ID.\n\n Uses a DatastoreDataFetcher to fetch data from the resource.\n\n Args:\n settings: Chart settings\n resource_id: Resource ID\n\n Returns:\n str | None: Chart config as JSON string or None if the chart can't be built\n \"\"\"\n settings.pop(\"__extras\", None)\n\n try:\n df = DatastoreDataFetcher(resource_id).fetch_data()\n except tk.ValidationError:\n return None\n\n return _build_chart(settings, df)\ncan_view(data_dict)","text":"Check if the resource can be viewed as a chart.
For now, we work only with resources stored with the DataStore.
PARAMETER DESCRIPTIONdata_dict Resource data dictionary
TYPE: dict[str, Any]
bool True if the resource can be viewed as a chart, False otherwise
TYPE: bool
ckanext/charts/utils.py def can_view(data_dict: dict[str, Any]) -> bool:\n \"\"\"Check if the resource can be viewed as a chart.\n\n For now, we work only with resources stored with the DataStore.\n\n Args:\n data_dict: Resource data dictionary\n\n Returns:\n bool: True if the resource can be viewed as a chart, False otherwise\n \"\"\"\n if data_dict[\"resource\"].get(\"datastore_active\"):\n return True\n\n # TODO: Add support for XML, XLS, XLSX, and other formats tabular data?\n # if data_dict[\"resource\"][\"format\"].lower() == \"xml\":\n # return True\n\n return False\nget_chart_form_builder(engine, chart_type)","text":"Get form builder for the given engine and chart type.
Source code inckanext/charts/utils.py def get_chart_form_builder(engine: str, chart_type: str):\n \"\"\"Get form builder for the given engine and chart type.\"\"\"\n builders = get_chart_engines()\n\n if engine not in builders:\n raise NotImplementedError(f\"Engine {engine} is not supported\")\n\n return builders[engine].get_form_for_type(chart_type)\nget_column_options(resource_id)","text":"Get column options for the given resource.
PARAMETER DESCRIPTIONresource_id Resource ID
TYPE: str
list[dict[str, str]] List of column options
Source code inckanext/charts/utils.py def get_column_options(resource_id: str) -> list[dict[str, str]]:\n \"\"\"Get column options for the given resource.\n\n Args:\n resource_id: Resource ID\n\n Returns:\n List of column options\n \"\"\"\n df = DatastoreDataFetcher(resource_id).fetch_data()\n\n return [{\"text\": col, \"value\": col} for col in df.columns]\nprintable_file_size(size_bytes)","text":"Convert file size in bytes to human-readable format.
PARAMETER DESCRIPTIONsize_bytes File size in bytes
TYPE: int
str Human-readable file size
TYPE: str
Examples:
>>> printable_file_size(123456789)\n'117.7 MB'\n>>> printable_file_size(7777)\n'7.6 KB'\nckanext/charts/utils.py def printable_file_size(size_bytes: int) -> str:\n \"\"\"Convert file size in bytes to human-readable format.\n\n Args:\n size_bytes: File size in bytes\n\n Returns:\n str: Human-readable file size\n\n Examples:\n >>> printable_file_size(123456789)\n '117.7 MB'\n\n >>> printable_file_size(7777)\n '7.6 KB'\n \"\"\"\n if size_bytes == 0:\n return \"0 bytes\"\n\n size_name = (\"bytes\", \"KB\", \"MB\", \"GB\", \"TB\")\n i = int(math.floor(math.log(size_bytes, 1024)))\n p = math.pow(1024, i)\n s = round(float(size_bytes) / p, 1)\n\n return f\"{s} {size_name[i]}\"\ncharts_if_empty_same_as(other_key)","text":"A custom version of if_empty_same_as validator for charts.
This validator is used to set the value of a field to the value of another field if it is empty or missing.
PARAMETER DESCRIPTIONother_key The key of the field to copy the value from
TYPE: str
Callable[..., Any] Callable[..., Any]: The validator function
Source code inckanext/charts/logic/validators.py def charts_if_empty_same_as(other_key: str) -> Callable[..., Any]:\n \"\"\"A custom version of if_empty_same_as validator for charts.\n\n This validator is used to set the value of a field to the value of another\n field if it is empty or missing.\n\n Args:\n other_key (str): The key of the field to copy the value from\n\n Returns:\n Callable[..., Any]: The validator function\n \"\"\"\n\n def callable(key, data, errors, context):\n value = data.get(key)\n if not value or value is tk.missing:\n try:\n data[key] = data[key[:-1] + (other_key,)]\n except KeyError:\n data[key] = data.get((\"__extras\",), {}).get(other_key, \"\")\n\n return callable\ncharts_list_length_validator(max_length)","text":"A validator to check the length of a list.
PARAMETER DESCRIPTIONmax_length The maximum length of the list
TYPE: int
Callable[..., Any] Callable[..., Any]: The validator function
Source code inckanext/charts/logic/validators.py def charts_list_length_validator(max_length: int) -> Callable[..., Any]:\n \"\"\"A validator to check the length of a list.\n\n Args:\n max_length (int): The maximum length of the list\n\n Returns:\n Callable[..., Any]: The validator function\n \"\"\"\n def callable(\n key: types.FlattenKey,\n data: types.FlattenDataDict,\n errors: types.FlattenErrorDict,\n context: types.Context,\n ):\n if len(data[key]) > max_length:\n raise tk.Invalid(tk._(\"Length must be less than {0}\").format(max_length))\n\n return callable\ncharts_list_to_csv(data)","text":"Convert a list of strings to a CSV string.
PARAMETER DESCRIPTIONdata The data to convert
TYPE: list[str] | str
str The comma separated string
TYPE: str
ckanext/charts/logic/validators.py def charts_list_to_csv(data: list[str] | str) -> str:\n \"\"\"Convert a list of strings to a CSV string.\n\n Args:\n data (list[str] | str): The data to convert\n\n Returns:\n str: The comma separated string\n \"\"\"\n if not isinstance(data, list):\n return data\n\n return \", \".join(data)\ncharts_strategy_support(strategy)","text":"Check if the cache strategy is supported.
PARAMETER DESCRIPTIONstrategy The cache strategy
TYPE: str
str The cache strategy if it is supported
TYPE: str
Invalid If the cache strategy is not supported
Source code inckanext/charts/logic/validators.py def charts_strategy_support(strategy: str) -> str:\n \"\"\"Check if the cache strategy is supported.\n\n Args:\n strategy (str): The cache strategy\n\n Returns:\n str: The cache strategy if it is supported\n\n Raises:\n tk.Invalid: If the cache strategy is not supported\n \"\"\"\n if strategy not in const.SUPPORTED_CACHE_STRATEGIES:\n raise tk.Invalid(tk._(\"Invalid cache strategy\"))\n\n if strategy == const.CACHE_FILE_ORC:\n try:\n from pyarrow import orc as _ # noqa\n except ImportError:\n raise tk.Invalid(\n tk._(\"Can't use File Orc cache strategy. PyArrow is not installed\"),\n ) from None\n\n if not strategy:\n return const.DEFAULT_CACHE_STRATEGY\n\n return strategy\ncharts_to_list_if_string(value)","text":"Convert a string to a list.
PARAMETER DESCRIPTIONvalue The value to convert
TYPE: Any
Any list[Any]: The value in a list
Source code inckanext/charts/logic/validators.py def charts_to_list_if_string(value: Any) -> Any:\n \"\"\"Convert a string to a list.\n\n Args:\n value (Any): The value to convert\n\n Returns:\n list[Any]: The value in a list\n \"\"\"\n if isinstance(value, str):\n return [value]\n\n return value\ncharts_validate_extras(key, data, errors, context)","text":"Validate charts settings according to the chart type and engine schema.
PARAMETER DESCRIPTIONkey The key of the field
TYPE: FlattenKey
data The data to validate
TYPE: FlattenDataDict
errors The errors dict
TYPE: FlattenErrorDict
context The context
TYPE: Context
ckanext/charts/logic/validators.py def charts_validate_extras(\n key: types.FlattenKey,\n data: types.FlattenDataDict,\n errors: types.FlattenErrorDict,\n context: types.Context,\n):\n \"\"\"Validate charts settings according to the chart type and engine schema.\n\n Args:\n key (types.FlattenKey): The key of the field\n data (types.FlattenDataDict): The data to validate\n errors (types.FlattenErrorDict): The errors dict\n context (types.Context): The context\n \"\"\"\n settings = _extract_setting(data)\n\n if \"engine\" not in settings or \"type\" not in settings:\n builder = DEFAULT_CHART_FORM\n else:\n builder = utils.get_chart_form_builder(settings[\"engine\"], settings[\"type\"])\n\n settings, err = tk.navl_validate(\n settings,\n builder(settings[\"resource_id\"]).get_validation_schema(\n context.get(\"_for_show\", False),\n ),\n {},\n )\n\n # TODO: do we have a better way to handle this? Seems like a hack\n for k, v in settings.items():\n data[(k,)] = v\n\n for k, v in settings.pop(\"__extras\", {}).items():\n data[(k,)] = v\n\n for k, v in err.items():\n errors[(k,)] = v\nfloat_validator(value)","text":"A validator for decimal numbers.
PARAMETER DESCRIPTIONvalue The value to validate
TYPE: Any
float The value as a float
TYPE: float
ckanext/charts/logic/validators.py def float_validator(value: Any) -> float:\n \"\"\"A validator for decimal numbers.\n\n Args:\n value (Any): The value to validate\n\n Returns:\n float: The value as a float\n \"\"\"\n try:\n return float(value)\n except ValueError:\n raise tk.Invalid(tk._(\"Must be a decimal number\")) from None\nThe extension implement a cache strategy to store the data fetched from the different sources.
There are three cache strategies available:
redisfile_orcfile_csv.The file cache works by storing the data in an orc or csv file in the filesystem. The redis cache stores the data in a Redis database. The cache strategy can be changed at the CKAN configuration level through the admin interface or in a configuration file.
The file-type cache strategy stores the data in a file in the filesystem. The file cache is stored in the ckanext-charts directory in the CKAN storage path. The file cache is stored in an orc or csv file format.
Using file_orc cache strategy requires the pyarrow python library to be installed.
The redis cache strategy stores the data in a Redis database.
Each redis key has a ckanext-charts:* prefix and store the data as a CSV string.
You need to have a Redis server running to use the redis cache strategy.
The cache TTL can be set in the CKAN configuration file. The default value is 3600 seconds (1 hour). The cache TTL can be set to 0 to disable the cache.
The redis and file-type cache has separate TTL settings:
redis cache TTL can be set with the ckanext.charts.redis_cache_ttl configuration option.file cache TTL can be set with the ckanext.charts.file_cache_ttl configuration option.Cache could be disabled by setting ckanext.charts.enable_cache to false. In this case the data will be fetched from the source every time the chart is rendered. It could be useful for debugging purposes. But using it in production is not recommended, as it could lead to performance issues.
To implement a custom cache strategy, you need to create a new class that extends the CacheStrategy class and implement the abstract methods.
See a naive implementation of a memcached cache strategy below:
from io import BytesIO\n\nfrom pymemcache.client import base\nimport pandas as pd\n\nimport ckanext.charts.config as config\nfrom ckanext.charts.cache import CacheStrategy\n\nclass MemcachedCache(CacheStrategy):\n \"\"\"Cache data to Memcached\"\"\"\n\n def __init__(self):\n self.client = base.Client(('localhost', 11211))\n\n def get_data(self, key: str) -> pd.DataFrame | None:\n \"\"\"Return data from cache if exists\"\"\"\n try:\n raw_data = self.client.get(key)\n\n if not raw_data:\n return None\n\n return pd.read_csv(BytesIO(raw_data))\n except Exception:\n log.exception(f\"Failed to get data for key: {key}\")\n return None\n\n def set_data(self, key: str, data: pd.DataFrame):\n \"\"\"Serialize data and save to Memcached\"\"\"\n cache_ttl = config.get_memcached_cache_ttl()\n\n try:\n serialized_data = data.to_csv(index=False).encode('utf-8')\n self.client.set(key, serialized_data, expire=cache_ttl)\n except Exception:\n log.exception(f\"Failed to save data to Memcached for key: {key}\")\n\n def invalidate(self, key: str):\n \"\"\"Invalidate cache by key\"\"\"\n try:\n self.client.delete(key)\n except Exception:\n log.exception(f\"Failed to invalidate cache for key: {key}\")\n Bases: ABC
Cache strategy interface.
Defines the abstracts methods for cache strategies.
"},{"location":"caching/base/#charts.cache.CacheStrategy.get_data","title":"get_data(key) abstractmethod","text":"Return data from cache if exists.
PARAMETER DESCRIPTIONkey The cache key to retrieve the data.
TYPE: str
DataFrame | None The data if exists, otherwise None.
"},{"location":"caching/base/#charts.cache.CacheStrategy.invalidate","title":"invalidate(key) abstractmethod","text":"Invalidate cache by key.
PARAMETER DESCRIPTIONkey The cache key to invalidate.
TYPE: str
set_data(key, data) abstractmethod","text":"Store data to cache.
PARAMETER DESCRIPTIONkey The cache key to store the data.
TYPE: str
data The data to be stored.
TYPE: DataFrame
FileCache","text":" Bases: CacheStrategy
Cache data as file.
We store the cached files in a separate folder in the CKAN storage.
"},{"location":"caching/file/#charts.cache.FileCache.generate_unique_consistent_filename","title":"generate_unique_consistent_filename(key)","text":"Generate unique and consistent filename based on the key.
PARAMETER DESCRIPTIONkey The cache key to generate the filename.
TYPE: str
str The filename.
"},{"location":"caching/file/#charts.cache.FileCache.get_data","title":"get_data(key)","text":"Return data from cache if exists.
PARAMETER DESCRIPTIONkey The cache key to retrieve the data.
TYPE: str
DataFrame | None The data if exists, otherwise None.
"},{"location":"caching/file/#charts.cache.FileCache.invalidate","title":"invalidate(key)","text":"Remove data from cache.
PARAMETER DESCRIPTIONkey The cache key to invalidate.
TYPE: str
is_file_cache_expired(file_path) staticmethod","text":"Check if file cache is expired.
If TTL is 0 then cache never expires.
PARAMETER DESCRIPTIONfile_path The path to the file.
TYPE: str
bool True if file cache is expired, otherwise False.
"},{"location":"caching/file/#charts.cache.FileCache.make_file_path_from_key","title":"make_file_path_from_key(key)","text":"Generate file path based on the key
PARAMETER DESCRIPTIONkey The cache key to generate the file path.
TYPE: str
str The file path.
"},{"location":"caching/file/#charts.cache.FileCache.read_data","title":"read_data(file) abstractmethod","text":"Read cached data from a file object.
PARAMETER DESCRIPTIONfile The file object to read the data.
TYPE: IO
DataFrame | None The data if exists, otherwise None.
"},{"location":"caching/file/#charts.cache.FileCache.set_data","title":"set_data(key, data)","text":"Store data to cache.
PARAMETER DESCRIPTIONkey The cache key to store the data.
TYPE: str
data The data to be stored.
TYPE: DataFrame
write_data(file_path, data) abstractmethod","text":"Defines how to write data to a file.
PARAMETER DESCRIPTIONfile_path The path to the file.
TYPE: str
data The data to be stored.
TYPE: DataFrame
FileCacheORC","text":" Bases: FileCache
Cache data as ORC file
"},{"location":"caching/file/#charts.cache.FileCacheORC.read_data","title":"read_data(file)","text":"Read cached data from an ORC file.
PARAMETER DESCRIPTIONfile The file object to read the data.
TYPE: IO
DataFrame | None The data if exists, otherwise None.
"},{"location":"caching/file/#charts.cache.FileCacheORC.write_data","title":"write_data(file_path, data)","text":"Write data to an ORC file.
PARAMETER DESCRIPTIONfile_path The path to the file.
TYPE: str
data The data to be stored.
TYPE: DataFrame
FileCacheCSV","text":" Bases: FileCache
Cache data as CSV file
"},{"location":"caching/file/#charts.cache.FileCacheCSV.read_data","title":"read_data(file)","text":"Read cached data from a CSV file.
PARAMETER DESCRIPTIONfile The file object to read the data.
TYPE: IO
DataFrame | None The data if exists, otherwise None.
"},{"location":"caching/file/#charts.cache.FileCacheCSV.write_data","title":"write_data(file_path, data)","text":"Write data to a CSV file.
PARAMETER DESCRIPTIONfile_path The path to the file.
TYPE: str
data The data to be stored.
TYPE: DataFrame
Bases: CacheStrategy
Cache data to Redis as a CSV string
"},{"location":"caching/redis/#charts.cache.RedisCache.get_data","title":"get_data(key)","text":"Return data from cache if exists.
PARAMETER DESCRIPTIONkey The cache key to retrieve the data.
TYPE: str
DataFrame | None The data if exists, otherwise None.
"},{"location":"caching/redis/#charts.cache.RedisCache.invalidate","title":"invalidate(key)","text":"Remove data from cache.
PARAMETER DESCRIPTIONkey The cache key to invalidate.
TYPE: str
set_data(key, data)","text":"Serialize data and save to Redis.
PARAMETER DESCRIPTIONkey The cache key to store the data.
TYPE: str
data The data to be stored.
TYPE: DataFrame
Exception If failed to save data to Redis.
"},{"location":"engines/","title":"Index","text":"The ckanext-charts implements supports different chart engines, such as plotly, observable and chartjs.
Each engine consists of two classes - chart builder and form builder. The chart builder is responsible for generating a JSON-string representation of the chart data, that will be passed to a respective JS module, that will render a chart based on the data.
The form builder is responsible for generating a form that allows users to configure the chart from the UI.
Also, you can implement support of a chart library of your choice by creating a custom chart engine. Read more about it here.
"},{"location":"engines/chartjs/","title":"Chartjs","text":"The Chart.JS chart engine supports the following chart types:
ckanext/charts/chart_builders/chartjs.py class ChartJsBuilder(BaseChartBuilder):\n @classmethod\n def get_supported_forms(cls) -> list[type[Any]]:\n return [\n ChartJSBarForm,\n ChartJSHorizontalBarForm,\n ChartJSLineForm,\n ChartJSPieForm,\n ChartJSDoughnutForm,\n ChartJSScatterForm,\n ChartJSBubbleForm,\n ChartJSRadarForm,\n ]\n\n def _create_zoom_and_title_options(self, options: dict[str, Any]) -> dict[str, Any]:\n \"\"\"Add zoom and title plugin options to the provided options dictionary\"\"\"\n if \"plugins\" not in options:\n options[\"plugins\"] = {}\n\n options[\"plugins\"].update(\n {\n \"zoom\": {\n \"zoom\": {\n \"wheel\": {\"enabled\": True},\n \"pinch\": {\"enabled\": True},\n \"drag\": {\"enabled\": True},\n \"mode\": \"xy\",\n },\n \"pan\": {\n \"enabled\": True,\n \"modifierKey\": \"shift\",\n \"mode\": \"xy\",\n },\n },\n \"title\": {\n \"display\": True,\n \"position\": \"bottom\",\n },\n },\n )\n return options\nThe bar chart is a chart with rectangular bars with lengths proportional to the values that they represent. The bars can be plotted vertically or horizontally. For a horizontal bar chart, use the Horizontal bar chart chart type.
The horizontal bar chart is a chart with rectangular bars with lengths proportional to the values that they represent. The bars are plotted horizontally.
"},{"location":"engines/chartjs/#line-chart","title":"Line chart","text":"The line chart is a chart that displays information as a series of data points called 'markers' connected by straight line segments. It is useful for showing trends over time.
"},{"location":"engines/chartjs/#pie-chart","title":"Pie chart","text":"The pie chart is a circular statistical graphic that is divided into slices to illustrate numerical proportions. The arc length of each slice is proportional to the quantity it represents.
"},{"location":"engines/chartjs/#doughnut-chart","title":"Doughnut chart","text":"The doughnut chart is a variant of the pie chart, with a hole in the center. It is useful for showing the relationship of parts to a whole.
"},{"location":"engines/chartjs/#scatter-plot","title":"Scatter plot","text":"The scatter plot is a chart that uses Cartesian coordinates to display values for two variables for a set of data. The data points are represented as individual dots.
"},{"location":"engines/chartjs/#bubble-chart","title":"Bubble chart","text":"The bubble chart is a chart that displays data points as bubbles. The size of the bubble represents a third dimension of the data.
"},{"location":"engines/chartjs/#radar-chart","title":"Radar chart","text":"The radar chart is a chart that displays multivariate data in the form of a two-dimensional chart of three or more quantitative variables represented on axes starting from the same point. The data points are connected by a line to form a polygon.
"},{"location":"engines/custom/","title":"Custom chart engine","text":""},{"location":"engines/custom/#implementing-new-chart-engines-support","title":"Implementing new chart engines support","text":"Implementing support for a new chart engine includes multiple steps and changes in Python, HTML, and JavaScript. Starting from the Python code:
Create a new builder class at ckanext.charts.chart_builder that inherits from BaseChartBuilder and implements the get_supported_forms method. This method must return a list of classes that represent supported chart types forms.
Each form type builder must be connected with a respective chart type builder.
The chart type builder must implement a to_json method that will return a dumped JSON data, which will be passed to a JS script.
The form type builder must implement a get_form_fields method that will return a list of all form fields that will be rendered for the user, allowing them to provide all the necessary information for a chart.
Register your chart engine by adding the builder class to get_chart_engines in ckanext.charts.chart_builder.__init__.py.
A full example of an implementation of bar chart for obvervable plot library.
from __future__ import annotations\n\nimport json\nfrom typing import Any\n\nimport ckanext.charts.exception as exception\nfrom ckanext.charts.chart_builders.base import BaseChartBuilder, BaseChartForm\n\n\nclass ObservableBuilder(BaseChartBuilder):\n @classmethod\n def get_supported_forms(cls) -> list[type[Any]]:\n return [ObservableBarForm]\n\n\nclass ObservableBarBuilder(ObservableBuilder):\n def to_json(self) -> str:\n return json.dumps(\n {\n \"type\": \"bar\",\n \"data\": self.df.to_dict(orient=\"records\"),\n \"settings\": self.settings,\n }\n )\n\n\nclass ObservableBarForm(BaseChartForm):\n name = \"Bar\"\n builder = ObservableBarBuilder\n\n def fill_field(self, choices: list[dict[str, str]]) -> dict[str, str]:\n field = self.color_field(choices)\n field.update({\"field_name\": \"fill\", \"label\": \"Fill\"})\n\n return field\n\n def get_form_fields(self):\n columns = [{\"value\": col, \"label\": col} for col in self.df.columns]\n chart_types = [\n {\"value\": form.name, \"label\": form.name}\n for form in self.builder.get_supported_forms()\n ]\n\n return [\n self.title_field(),\n self.description_field(),\n self.engine_field(),\n self.type_field(chart_types),\n self.x_axis_field(columns),\n self.y_axis_field(columns),\n self.fill_field(columns),\n self.opacity_field(),\n self.limit_field(),\n ]\nAnother step is to register JS/CSS vendor libraries of the chart you want to use. Refer to CKAN documentation to read about adding CSS and JavaScript files using Webassets.
You also will need a CKAN JS module, that will be responsible for rendering the Chart. This module will work with the vendor library and will be responsible for rendering the chart in the container.
This module must be registered inside a webassets.yml as well.
ckan.module(\"charts-render-observable\", function ($, _) {\n \"use strict\";\n\n return {\n options: {\n config: null\n },\n\n initialize: function () {\n $.proxyAll(this, /_/);\n\n if (!this.options.config) {\n console.error(\"No configuration provided\");\n return;\n }\n\n var plot;\n\n switch (this.options.config.type) {\n case \"bar\":\n plot = Plot.barY(this.options.config.data, this.options.config.settings).plot();\n break;\n default:\n return;\n }\n\n this.el[0].replaceChildren(plot);\n }\n };\n });\nAnd an HTML file, that will provide a proper container and include your JS module with data-module.
{% asset \"charts/observable\" %}\n\n {% if chart %}\n <div id=\"chart-container\" data-module=\"charts-render-observable\" data-module-config=\"{{ chart }}\"></div>\n {% else %}\n <p class=\"text-muted\">\n {{ _(\"Cannot build chart with current settings\") }}\n </p>\n {% endif %}\nNote, that we should add {% asset \"charts/observable\" %} not only here, but in charts_form.html too.
The reason for having a separate HTML file and JS module is that different libraries may require different types of container elements (such as div, canvas, etc.) to initialize or may need additional boilerplate code to build a chart. There's no easy way to abstract this, so you have to implement these things yourself.
The Observable Plot chart engine supports the following chart types:
ckanext/charts/chart_builders/observable.py class ObservableBuilder(BaseChartBuilder):\n @classmethod\n def get_supported_forms(cls) -> list[type[Any]]:\n return [\n ObservableBarForm,\n ObservableHoriontalBarForm,\n ObservableLineForm,\n ObservablePieForm,\n ObservableScatterForm,\n ObservableAutoForm,\n ]\nThe bar chart is a chart with rectangular bars with lengths proportional to the values that they represent. The bars can be plotted vertically or horizontally. For a horizontal bar chart, use the Horizontal bar chart chart type.
The horizontal bar chart is a chart with rectangular bars with lengths proportional to the values that they represent. The bars are plotted horizontally.
"},{"location":"engines/observable/#line-chart","title":"Line chart","text":"The line chart is a chart that displays information as a series of data points called 'markers' connected by straight line segments. It is useful for showing trends over time.
"},{"location":"engines/observable/#pie-chart","title":"Pie chart","text":"The pie chart is a circular statistical graphic that is divided into slices to illustrate numerical proportions. The arc length of each slice is proportional to the quantity it represents.
"},{"location":"engines/observable/#scatter-plot","title":"Scatter plot","text":"The scatter plot is a chart that uses Cartesian coordinates to display values for two variables for a set of data. The data points are represented as individual dots.
"},{"location":"engines/observable/#auto-chart","title":"Auto chart","text":"The auto chart is a chart that automatically selects the best chart type based on the data. It is useful for quickly visualizing data without having to manually select a chart type.
"},{"location":"engines/plotly/","title":"Plotly","text":"The Plotly chart engine supports the following chart types:
Base class for Plotly chart builders.
Defines supported chart types for Plotly engine.
Source code inckanext/charts/chart_builders/plotly.py class PlotlyBuilder(BaseChartBuilder):\n \"\"\"Base class for Plotly chart builders.\n\n Defines supported chart types for Plotly engine.\n \"\"\"\n\n @classmethod\n def get_supported_forms(cls) -> list[type[Any]]:\n return [\n PlotlyBarForm,\n PlotlyHoriontalBarForm,\n PlotlyPieForm,\n PlotlyLineForm,\n PlotlyScatterForm,\n ]\nThe bar chart is a chart with rectangular bars with lengths proportional to the values that they represent. The bars can be plotted vertically or horizontally. For a horizontal bar chart, use the Horizontal bar chart chart type.
The horizontal bar chart is a chart with rectangular bars with lengths proportional to the values that they represent. The bars are plotted horizontally.
"},{"location":"engines/plotly/#line-chart","title":"Line chart","text":"The line chart is a chart that displays information as a series of data points called 'markers' connected by straight line segments. It is useful for showing trends over time.
"},{"location":"engines/plotly/#pie-chart","title":"Pie chart","text":"The pie chart is a circular statistical graphic that is divided into slices to illustrate numerical proportions. The arc length of each slice is proportional to the quantity it represents.
"},{"location":"engines/plotly/#scatter-plot","title":"Scatter plot","text":"The scatter plot is a chart that uses Cartesian coordinates to display values for two variables for a set of data. The data points are represented as individual dots.
"},{"location":"fetchers/","title":"Fetchers","text":"Fetchers are responsible for fetching data from different sources (DataStore, URL, file system, hardcoded data).
For the current implementation, we're working with resources that are uploaded to the DataStore, so the fetcher will be responsible for fetching the data from the DataStore.
But it might come in handy to have fetchers for other sources, like URL, file system, etc.
"},{"location":"fetchers/#implementing-new-fetchers","title":"Implementing new fetchers","text":"To register new fetchers, you need to create a new class that inherits from DataFetcherStrategy and implements the fetch_data and make_cache_key methods.
The fetch_data method should return a pandas DataFrame object with the data that should be displayed in the chart.
The make_cache_key method should return a unique string that will be used as a key to store the data in the cache.
See the base class DataFetcherStrategy for more information.
All fetchers must inherit from the DataFetcherStrategy class. This class provides the basic structure for fetchers and defines the methods that must be implemented by all fetchers.
fetch_data() abstractmethod","text":"This method should implement the data fetch logic.
All the necessary information should be provided in the constructor.
RETURNS DESCRIPTIONDataFrame pd.DataFrame: The fetched data
"},{"location":"fetchers/base/#charts.fetchers.DataFetcherStrategy.get_cached_data","title":"get_cached_data()","text":"Fetch data from the cache.
RETURNS DESCRIPTIONDataFrame | None pd.DataFrame | None: The cached data or None if not found
"},{"location":"fetchers/base/#charts.fetchers.DataFetcherStrategy.invalidate_cache","title":"invalidate_cache()","text":"Invalidate the cache for the data fetcher.
"},{"location":"fetchers/base/#charts.fetchers.DataFetcherStrategy.make_cache_key","title":"make_cache_key() abstractmethod","text":"This method should generate a cache key for the fetched data.
Every data fetcher should implement this method to support caching.
RETURNS DESCRIPTIONstr The cache key
TYPE: str
Fetch dataset resource data from the DataStore.
This fetcher is used to fetch data from the DataStore using the resource ID.
"},{"location":"fetchers/datastore/#charts.fetchers.DatastoreDataFetcher.__init__","title":"__init__(resource_id, limit=2000000, cache_strategy=None)","text":"Initialize the DatastoreDataFetcher.
PARAMETER DESCRIPTIONresource_id The ID of the resource to fetch data for.
TYPE: str
limit The maximum number of rows to fetch.
TYPE: int DEFAULT: 2000000
cache_strategy The cache strategy to use. If not provided, the configured cache strategy will be used.
TYPE: str DEFAULT: None
fetch_data()","text":"Fetch data from the DataStore.
RETURNS DESCRIPTIONDataFrame pd.DataFrame: Data from the DataStore
"},{"location":"fetchers/datastore/#charts.fetchers.DatastoreDataFetcher.make_cache_key","title":"make_cache_key()","text":"Generate a cache key for the DataStore data fetcher.
Uses the resource ID as the part of a cache key.
RETURNS DESCRIPTIONstr The cache key
TYPE: str
Bases: DataFetcherStrategy
Fetch data from the file system.
This fetcher is used to fetch data from a file on the file system.
Supported formatsCSVXLSXXLSXML__init__(file_path, file_format='csv', cache_strategy=None)","text":"Initialize the FileSystemDataFetcher.
PARAMETER DESCRIPTIONfile_path The path to the file.
TYPE: str
file_format The format of the file.
TYPE: str DEFAULT: 'csv'
cache_strategy The cache strategy to use. If not provided, the configured cache strategy will be used.
TYPE: str DEFAULT: None
fetch_data()","text":"Fetch data from the file system.
RETURNS DESCRIPTIONDataFrame pd.DataFrame: Data fetched from the file system
"},{"location":"fetchers/file/#charts.fetchers.FileSystemDataFetcher.make_cache_key","title":"make_cache_key()","text":"Generate a cache key for the FileSystem data fetcher.
Uses the file path as the part of a cache key.
RETURNS DESCRIPTIONstr The cache key
TYPE: str
Bases: DataFetcherStrategy
Fetch hardcoded data.
This fetcher is used to make a dataframe from hardcoded data, so you can build a chart from it.
"},{"location":"fetchers/hardcode/#charts.fetchers.HardcodedDataFetcher.__init__","title":"__init__(data)","text":"Initialize the HardcodedDataFetcher.
PARAMETER DESCRIPTIONdata The hardcoded data.
TYPE: dict[str, list[Any]]
fetch_data()","text":"Transform arbitrary data into a dataframe.
RETURNS DESCRIPTIONDataFrame pd.DataFrame: The hardcoded data as a dataframe
"},{"location":"fetchers/hardcode/#charts.fetchers.HardcodedDataFetcher.invalidate_cache","title":"invalidate_cache()","text":"Hardcoded data is not cached
"},{"location":"fetchers/hardcode/#charts.fetchers.HardcodedDataFetcher.make_cache_key","title":"make_cache_key()","text":"Hardcoded data is not cached
"},{"location":"fetchers/url/","title":"URL Data Fetcher","text":" Bases: DataFetcherStrategy
Fetch data from a URL.
This fetcher is used to fetch data from a URL.
Supported formatsCSVXLSXXLSXML__init__(url, file_format='csv', timeout=0, cache_strategy=None)","text":"Initialize the URLDataFetcher.
PARAMETER DESCRIPTIONurl The URL to fetch data from.
TYPE: str
file_format The format of the file.
TYPE: str DEFAULT: 'csv'
timeout The timeout for the request in seconds.
TYPE: int DEFAULT: 0
cache_strategy The cache strategy to use. If not provided, the configured cache strategy will be used.
TYPE: str DEFAULT: None
fetch_data()","text":"Fetch data from the URL.
RETURNS DESCRIPTIONDataFrame pd.DataFrame: Data fetched from the URL
"},{"location":"fetchers/url/#charts.fetchers.URLDataFetcher.make_cache_key","title":"make_cache_key()","text":"Generate a cache key for the URL data fetcher.
Uses the URL as the part of a cache key.
RETURNS DESCRIPTIONstr The cache key
TYPE: str
make_request()","text":"Make a request to the URL and return the response content.
RETURNS DESCRIPTIONbytes The response content
TYPE: bytes
DataFetchError If an error occurs during the request
"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-\\.\\_]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":""},{"location":"#ckanext-charts","title":"ckanext-charts","text":"This extension, ckanext-charts, provides additional functionality for working with charts in CKAN. It allows users to create, manage, and visualize charts based on data stored in CKAN datasets.
The extension includes features such as chart creation, chart editing, chart embedding, and chart sharing. It also supports various chart types, including bar charts, line charts, pie charts, and more.
With ckanext-charts, users can easily generate interactive and visually appealing charts to enhance data analysis and presentation in CKAN.
See the documentation for more information.
"},{"location":"#quick-start","title":"Quick start","text":"PyPi with pip install ckanext-charts[pyarrow]charts_view to the list of plugins in your CKAN config (ckan.plugins = charts_view charts_builder_view)To install ckanext-charts for development, activate your CKAN virtualenv and do:
git clone https://github.com/DataShades/ckanext-charts.git\ncd ckanext-charts\npip install -e '.[dev]'\nTo run the tests, do:
pytest --ckan-ini=test.ini\nWe are using mkdocs to build the documentation. To build and deploy the documentation, do:
CHARTS_FIELDS=1 mkdocs build && mkdocs gh-deploy\nWe need to set the CHARTS_FIELDS for our custom handler, that autogenerate documentation for chart types fields.
The CKAN instance will be initialized, because we will call validators and helpers in this code. Therefore, you'll need another environment variable to set the CKAN configuration file path:
export CKAN_INI=$PWD/config/ckan.ini\nThe config file must enable the ckanext-charts plugin:
ckan.plugins = ... charts_view charts_builder_view ...\nIf you're working on the documentation, you can run the following command to start a live-reloading server without gathering the chart types fields data. It will speed up the process significantly, as we won't need to wait for the CKAN initialization:
mkdocs serve -a 127.0.0.1:8001\nAGPL
"},{"location":"configuration/","title":"Configuration","text":"Configuration options
There is a number of configuration options available for the extension. You can set them in the CKAN configuration file or using the admin configuration page.
"},{"location":"configuration/#configuration-options","title":"Configuration options","text":""},{"location":"configuration/#cache-strategy","title":"Cache strategy","text":"ckanext.charts.cache_strategy [optional]
Cache strategy for chart data.
Options: redis, file_orc, file_csv
Type: str
Default: redis
ckanext.charts.redis_cache_ttl [optional]
Time to live for the Redis cache in seconds. Set to 0 to disable cache.
Type: int
Default: 3600
ckanext.charts.file_cache_ttl [optional]
Time to live for the File cache in seconds. Set to 0 to disable cache.
Type: int
Default: 3600
ckanext.charts.enable_cache [optional]
Enable cache for the charts.
WarningIf the cache is disabled, the chart data will be fetched from the data source every time the chart is rendered. It could lead to performance issues if the data source is slow or the data is too large.
Type: bool
Default: true
ckanext.charts.include_htmx_asset [optional]
Include HTMX assets. Could be disabled if another plugins are including it.
Type: bool
Default: true
ckanext.charts.reinit_ckan_js_modules [optional]
Reinit CKAN JS modules after HTMX swap.
Type: bool
Default: false
ckanext.charts.allow_anon_building_charts [optional]
By default, only authenticated users can build charts. If you want to allow anonymous users to build charts, set this option to true.
Type: bool
Default: false
The extension provides an admin configuration page where you can set all the listed configuration options. The admin page available only using an extension ckanext-admin-panel. The admin panel is a separate extension that provides an alternative admin interface for CKAN. It allows you to manage CKAN settings and other extensions settings through the web interface.
This page could be accessed by the following URL: /admin-panel/charts/config, but it will be registered only if the ckanext-admin-panel is installed and enabled.
charts_allow_anon_building_charts()","text":"Check if anonymous users are allowed to build charts.
RETURNS DESCRIPTIONbool True if anonymous users are allowed to build charts, False otherwise.
TYPE: bool
ckanext/charts/helpers.py def charts_allow_anon_building_charts() -> bool:\n \"\"\"Check if anonymous users are allowed to build charts.\n\n Returns:\n bool: True if anonymous users are allowed to build charts, False otherwise.\n \"\"\"\n return config.allow_anon_building_charts()\ncharts_get_resource_columns(resource_id)","text":"Get the columns of the given resource.
PARAMETER DESCRIPTIONresource_id Resource ID
TYPE: str
str JSON string of columns options
TYPE: str
ckanext/charts/helpers.py def charts_get_resource_columns(resource_id: str) -> str:\n \"\"\"Get the columns of the given resource.\n\n Args:\n resource_id: Resource ID\n\n Returns:\n str: JSON string of columns options\n \"\"\"\n fetcher = DatastoreDataFetcher(resource_id)\n\n return json.dumps(\n [{\"id\": col, \"title\": col} for col in fetcher.fetch_data().columns],\n )\ncharts_include_htmx_asset()","text":"Checks if the HTMX asset should be included.
RETURNS DESCRIPTIONbool True if the HTMX asset should be included, False otherwise.
TYPE: bool
ckanext/charts/helpers.py def charts_include_htmx_asset() -> bool:\n \"\"\"Checks if the HTMX asset should be included.\n\n Returns:\n bool: True if the HTMX asset should be included, False otherwise.\n \"\"\"\n return config.include_htmx_asset()\ncharts_reinit_ckan_js_modules()","text":"Checks if CKAN JS modules should be reinitialized.
RETURNS DESCRIPTIONbool True if CKAN JS modules should be reinitialized, False otherwise.
TYPE: bool
ckanext/charts/helpers.py def charts_reinit_ckan_js_modules() -> bool:\n \"\"\"Checks if CKAN JS modules should be reinitialized.\n\n Returns:\n bool: True if CKAN JS modules should be reinitialized, False otherwise.\n \"\"\"\n return config.reinit_ckan_js_modules()\ncharts_user_is_authenticated()","text":"Check if the user is authenticated.
RETURNS DESCRIPTIONbool True if the user is authenticated, False otherwise.
TYPE: bool
ckanext/charts/helpers.py def charts_user_is_authenticated() -> bool:\n \"\"\"Check if the user is authenticated.\n\n Returns:\n bool: True if the user is authenticated, False otherwise.\n \"\"\"\n return tk.current_user.is_authenticated\nget_available_chart_engines_options()","text":"Get the available chart engines.
RETURNS DESCRIPTIONlist[dict[str, str]] List of chart engines options
Source code inckanext/charts/helpers.py def get_available_chart_engines_options() -> list[dict[str, str]]:\n \"\"\"Get the available chart engines.\n\n Returns:\n List of chart engines options\n \"\"\"\n return [{\"value\": engine, \"text\": engine} for engine in get_chart_engines()]\nget_file_cache_size()","text":"Get the size of the file cache in a human-readable format.
RETURNS DESCRIPTIONstr Human-readable file cache size
TYPE: str
ckanext/charts/helpers.py def get_file_cache_size() -> str:\n \"\"\"Get the size of the file cache in a human-readable format.\n\n Returns:\n str: Human-readable file cache size\n \"\"\"\n return utils.printable_file_size(count_file_cache_size())\nget_redis_cache_size()","text":"Get the size of the Redis cache in a human-readable format.
RETURNS DESCRIPTIONstr Human-readable Redis cache size
TYPE: str
ckanext/charts/helpers.py def get_redis_cache_size() -> str:\n \"\"\"Get the size of the Redis cache in a human-readable format.\n\n Returns:\n str: Human-readable Redis cache size\n \"\"\"\n return utils.printable_file_size(count_redis_cache_size())\nRequires Redis 7+
Compatibility with core CKAN versions:
CKAN version Compatible? 2.9 and earlier no 2.10+ yes"},{"location":"install/#installation_1","title":"Installation","text":"Install the extension from PyPI:
pip install ckanext-charts\nIf you want to use ORC file cache, you have to install the extension with the pyarrow extra:
pip install ckanext-charts[pyarrow]\nEnable the view and builder plugins in your CKAN configuration file (e.g. ckan.ini or production.ini):
ckan.plugins = ... charts_view charts_builder_view ...\nThe extension requires the following CKAN extensions to be installed and enabled:
ckanext-scheming: We're using the scheming extension to create custom forms for the chart builders.
ckanext-admin-panel (optional): If you want to use the admin configuration page, you need to install and enable this extension. The admin panel is a separate extension that provides an alternative admin interface for CKAN. It allows you to manage CKAN settings and other extensions settings through the web interface and significantly extends the default CKAN admin interface.
ImportError: lxml.html.clean module is now a separate project lxml_html_clean
Install lxml[html_clean] or lxml_html_clean directly using pip.
This extension offers two plugins: charts_view and charts_builder_view. The first plugin is a standard chart builder designed for system administrators or anyone with permission to edit or create a resource view. Creating a preview using this plugin generates a static chart with saved settings. This chart retains the settings and will always be rendered in the same way.
The charts_builder_view plugin does not have any special settings during the creation or editing stages. Instead, it renders a chart builder JS widget on a resource page, allowing any user to interact with the resource data and create a chart that meets their needs. In the future, we plan to implement a feature that will allow users to save their charts.
The user chart builder could be restricted to only authenticated users. See the config settings section for more information. In this case, user will be forced to log in to build a chart. After the user logs in, user will be redirected to the chart builder page back.
"},{"location":"utils/","title":"Utility Functions","text":""},{"location":"utils/#charts.utils.build_chart_for_data","title":"build_chart_for_data(settings, data)","text":"Build chart for the given dataframe and settings.
PARAMETER DESCRIPTIONsettings Chart settings
TYPE: dict[str, Any]
data Dataframe with data
TYPE: DataFrame
str | None Chart config as JSON string
Source code inckanext/charts/utils.py def build_chart_for_data(settings: dict[str, Any], data: pd.DataFrame) -> str | None:\n \"\"\"Build chart for the given dataframe and settings.\n\n Args:\n settings: Chart settings\n data: Dataframe with data\n\n Returns:\n Chart config as JSON string\n \"\"\"\n return _build_chart(settings, data)\nbuild_chart_for_resource(settings, resource_id)","text":"Build chart for the given resource ID.
Uses a DatastoreDataFetcher to fetch data from the resource.
PARAMETER DESCRIPTIONsettings Chart settings
TYPE: dict[str, Any]
resource_id Resource ID
TYPE: str
str | None str | None: Chart config as JSON string or None if the chart can't be built
Source code inckanext/charts/utils.py def build_chart_for_resource(settings: dict[str, Any], resource_id: str) -> str | None:\n \"\"\"Build chart for the given resource ID.\n\n Uses a DatastoreDataFetcher to fetch data from the resource.\n\n Args:\n settings: Chart settings\n resource_id: Resource ID\n\n Returns:\n str | None: Chart config as JSON string or None if the chart can't be built\n \"\"\"\n settings.pop(\"__extras\", None)\n\n try:\n df = DatastoreDataFetcher(resource_id).fetch_data()\n except tk.ValidationError:\n return None\n\n return _build_chart(settings, df)\ncan_view(data_dict)","text":"Check if the resource can be viewed as a chart.
For now, we work only with resources stored with the DataStore.
PARAMETER DESCRIPTIONdata_dict Resource data dictionary
TYPE: dict[str, Any]
bool True if the resource can be viewed as a chart, False otherwise
TYPE: bool
ckanext/charts/utils.py def can_view(data_dict: dict[str, Any]) -> bool:\n \"\"\"Check if the resource can be viewed as a chart.\n\n For now, we work only with resources stored with the DataStore.\n\n Args:\n data_dict: Resource data dictionary\n\n Returns:\n bool: True if the resource can be viewed as a chart, False otherwise\n \"\"\"\n if data_dict[\"resource\"].get(\"datastore_active\"):\n return True\n\n # TODO: Add support for XML, XLS, XLSX, and other formats tabular data?\n # if data_dict[\"resource\"][\"format\"].lower() == \"xml\":\n # return True\n\n return False\nget_chart_form_builder(engine, chart_type)","text":"Get form builder for the given engine and chart type.
Source code inckanext/charts/utils.py def get_chart_form_builder(engine: str, chart_type: str):\n \"\"\"Get form builder for the given engine and chart type.\"\"\"\n builders = get_chart_engines()\n\n if engine not in builders:\n raise NotImplementedError(f\"Engine {engine} is not supported\")\n\n return builders[engine].get_form_for_type(chart_type)\nget_column_options(resource_id)","text":"Get column options for the given resource.
PARAMETER DESCRIPTIONresource_id Resource ID
TYPE: str
list[dict[str, str]] List of column options
Source code inckanext/charts/utils.py def get_column_options(resource_id: str) -> list[dict[str, str]]:\n \"\"\"Get column options for the given resource.\n\n Args:\n resource_id: Resource ID\n\n Returns:\n List of column options\n \"\"\"\n df = DatastoreDataFetcher(resource_id).fetch_data()\n\n return [{\"text\": col, \"value\": col} for col in df.columns]\nprintable_file_size(size_bytes)","text":"Convert file size in bytes to human-readable format.
PARAMETER DESCRIPTIONsize_bytes File size in bytes
TYPE: int
str Human-readable file size
TYPE: str
Examples:
>>> printable_file_size(123456789)\n'117.7 MB'\n>>> printable_file_size(7777)\n'7.6 KB'\nckanext/charts/utils.py def printable_file_size(size_bytes: int) -> str:\n \"\"\"Convert file size in bytes to human-readable format.\n\n Args:\n size_bytes: File size in bytes\n\n Returns:\n str: Human-readable file size\n\n Examples:\n >>> printable_file_size(123456789)\n '117.7 MB'\n\n >>> printable_file_size(7777)\n '7.6 KB'\n \"\"\"\n if size_bytes == 0:\n return \"0 bytes\"\n\n size_name = (\"bytes\", \"KB\", \"MB\", \"GB\", \"TB\")\n i = int(math.floor(math.log(size_bytes, 1024)))\n p = math.pow(1024, i)\n s = round(float(size_bytes) / p, 1)\n\n return f\"{s} {size_name[i]}\"\ncharts_if_empty_same_as(other_key)","text":"A custom version of if_empty_same_as validator for charts.
This validator is used to set the value of a field to the value of another field if it is empty or missing.
PARAMETER DESCRIPTIONother_key The key of the field to copy the value from
TYPE: str
Callable[..., Any] Callable[..., Any]: The validator function
Source code inckanext/charts/logic/validators.py def charts_if_empty_same_as(other_key: str) -> Callable[..., Any]:\n \"\"\"A custom version of if_empty_same_as validator for charts.\n\n This validator is used to set the value of a field to the value of another\n field if it is empty or missing.\n\n Args:\n other_key (str): The key of the field to copy the value from\n\n Returns:\n Callable[..., Any]: The validator function\n \"\"\"\n\n def callable(key, data, errors, context):\n value = data.get(key)\n if not value or value is tk.missing:\n try:\n data[key] = data[key[:-1] + (other_key,)]\n except KeyError:\n data[key] = data.get((\"__extras\",), {}).get(other_key, \"\")\n\n return callable\ncharts_list_length_validator(max_length)","text":"A validator to check the length of a list.
PARAMETER DESCRIPTIONmax_length The maximum length of the list
TYPE: int
Callable[..., Any] Callable[..., Any]: The validator function
Source code inckanext/charts/logic/validators.py def charts_list_length_validator(max_length: int) -> Callable[..., Any]:\n \"\"\"A validator to check the length of a list.\n\n Args:\n max_length (int): The maximum length of the list\n\n Returns:\n Callable[..., Any]: The validator function\n \"\"\"\n def callable(\n key: types.FlattenKey,\n data: types.FlattenDataDict,\n errors: types.FlattenErrorDict,\n context: types.Context,\n ):\n if len(data[key]) > max_length:\n raise tk.Invalid(tk._(\"Length must be less than {0}\").format(max_length))\n\n return callable\ncharts_list_to_csv(data)","text":"Convert a list of strings to a CSV string.
PARAMETER DESCRIPTIONdata The data to convert
TYPE: list[str] | str
str The comma separated string
TYPE: str
ckanext/charts/logic/validators.py def charts_list_to_csv(data: list[str] | str) -> str:\n \"\"\"Convert a list of strings to a CSV string.\n\n Args:\n data (list[str] | str): The data to convert\n\n Returns:\n str: The comma separated string\n \"\"\"\n if not isinstance(data, list):\n return data\n\n return \", \".join(data)\ncharts_strategy_support(strategy)","text":"Check if the cache strategy is supported.
PARAMETER DESCRIPTIONstrategy The cache strategy
TYPE: str
str The cache strategy if it is supported
TYPE: str
Invalid If the cache strategy is not supported
Source code inckanext/charts/logic/validators.py def charts_strategy_support(strategy: str) -> str:\n \"\"\"Check if the cache strategy is supported.\n\n Args:\n strategy (str): The cache strategy\n\n Returns:\n str: The cache strategy if it is supported\n\n Raises:\n tk.Invalid: If the cache strategy is not supported\n \"\"\"\n if strategy not in const.SUPPORTED_CACHE_STRATEGIES:\n raise tk.Invalid(tk._(\"Invalid cache strategy\"))\n\n if strategy == const.CACHE_FILE_ORC:\n try:\n from pyarrow import orc as _ # noqa\n except ImportError:\n raise tk.Invalid(\n tk._(\"Can't use File Orc cache strategy. PyArrow is not installed\"),\n ) from None\n\n if not strategy:\n return const.DEFAULT_CACHE_STRATEGY\n\n return strategy\ncharts_to_list_if_string(value)","text":"Convert a string to a list.
PARAMETER DESCRIPTIONvalue The value to convert
TYPE: Any
Any list[Any]: The value in a list
Source code inckanext/charts/logic/validators.py def charts_to_list_if_string(value: Any) -> Any:\n \"\"\"Convert a string to a list.\n\n Args:\n value (Any): The value to convert\n\n Returns:\n list[Any]: The value in a list\n \"\"\"\n if isinstance(value, str):\n return [value]\n\n return value\ncharts_validate_extras(key, data, errors, context)","text":"Validate charts settings according to the chart type and engine schema.
PARAMETER DESCRIPTIONkey The key of the field
TYPE: FlattenKey
data The data to validate
TYPE: FlattenDataDict
errors The errors dict
TYPE: FlattenErrorDict
context The context
TYPE: Context
ckanext/charts/logic/validators.py def charts_validate_extras(\n key: types.FlattenKey,\n data: types.FlattenDataDict,\n errors: types.FlattenErrorDict,\n context: types.Context,\n):\n \"\"\"Validate charts settings according to the chart type and engine schema.\n\n Args:\n key (types.FlattenKey): The key of the field\n data (types.FlattenDataDict): The data to validate\n errors (types.FlattenErrorDict): The errors dict\n context (types.Context): The context\n \"\"\"\n settings = _extract_setting(data)\n\n if \"engine\" not in settings or \"type\" not in settings:\n builder = DEFAULT_CHART_FORM\n else:\n builder = utils.get_chart_form_builder(settings[\"engine\"], settings[\"type\"])\n\n settings, err = tk.navl_validate(\n settings,\n builder(settings[\"resource_id\"]).get_validation_schema(\n context.get(\"_for_show\", False),\n ),\n {},\n )\n\n # TODO: do we have a better way to handle this? Seems like a hack\n for k, v in settings.items():\n data[(k,)] = v\n\n for k, v in settings.pop(\"__extras\", {}).items():\n data[(k,)] = v\n\n for k, v in err.items():\n errors[(k,)] = v\nfloat_validator(value)","text":"A validator for decimal numbers.
PARAMETER DESCRIPTIONvalue The value to validate
TYPE: Any
float The value as a float
TYPE: float
ckanext/charts/logic/validators.py def float_validator(value: Any) -> float:\n \"\"\"A validator for decimal numbers.\n\n Args:\n value (Any): The value to validate\n\n Returns:\n float: The value as a float\n \"\"\"\n try:\n return float(value)\n except ValueError:\n raise tk.Invalid(tk._(\"Must be a decimal number\")) from None\nThe extension implement a cache strategy to store the data fetched from the different sources.
There are three cache strategies available:
redisfile_orcfile_csv.The file cache works by storing the data in an orc or csv file in the filesystem. The redis cache stores the data in a Redis database. The cache strategy can be changed at the CKAN configuration level through the admin interface or in a configuration file.
The file-type cache strategy stores the data in a file in the filesystem. The file cache is stored in the ckanext-charts directory in the CKAN storage path. The file cache is stored in an orc or csv file format.
Using file_orc cache strategy requires the pyarrow python library to be installed.
The redis cache strategy stores the data in a Redis database.
Each redis key has a ckanext-charts:* prefix and store the data as a CSV string.
You need to have a Redis server running to use the redis cache strategy.
The cache TTL can be set in the CKAN configuration file. The default value is 3600 seconds (1 hour). The cache TTL can be set to 0 to disable the cache.
The redis and file-type cache has separate TTL settings:
redis cache TTL can be set with the ckanext.charts.redis_cache_ttl configuration option.file cache TTL can be set with the ckanext.charts.file_cache_ttl configuration option.Cache could be disabled by setting ckanext.charts.enable_cache to false. In this case the data will be fetched from the source every time the chart is rendered. It could be useful for debugging purposes. But using it in production is not recommended, as it could lead to performance issues.
To implement a custom cache strategy, you need to create a new class that extends the CacheStrategy class and implement the abstract methods.
See a naive implementation of a memcached cache strategy below:
from io import BytesIO\n\nfrom pymemcache.client import base\nimport pandas as pd\n\nimport ckanext.charts.config as config\nfrom ckanext.charts.cache import CacheStrategy\n\nclass MemcachedCache(CacheStrategy):\n \"\"\"Cache data to Memcached\"\"\"\n\n def __init__(self):\n self.client = base.Client(('localhost', 11211))\n\n def get_data(self, key: str) -> pd.DataFrame | None:\n \"\"\"Return data from cache if exists\"\"\"\n try:\n raw_data = self.client.get(key)\n\n if not raw_data:\n return None\n\n return pd.read_csv(BytesIO(raw_data))\n except Exception:\n log.exception(f\"Failed to get data for key: {key}\")\n return None\n\n def set_data(self, key: str, data: pd.DataFrame):\n \"\"\"Serialize data and save to Memcached\"\"\"\n cache_ttl = config.get_memcached_cache_ttl()\n\n try:\n serialized_data = data.to_csv(index=False).encode('utf-8')\n self.client.set(key, serialized_data, expire=cache_ttl)\n except Exception:\n log.exception(f\"Failed to save data to Memcached for key: {key}\")\n\n def invalidate(self, key: str):\n \"\"\"Invalidate cache by key\"\"\"\n try:\n self.client.delete(key)\n except Exception:\n log.exception(f\"Failed to invalidate cache for key: {key}\")\n Bases: ABC
Cache strategy interface.
Defines the abstracts methods for cache strategies.
"},{"location":"caching/base/#charts.cache.CacheStrategy.get_data","title":"get_data(key) abstractmethod","text":"Return data from cache if exists.
PARAMETER DESCRIPTIONkey The cache key to retrieve the data.
TYPE: str
DataFrame | None The data if exists, otherwise None.
"},{"location":"caching/base/#charts.cache.CacheStrategy.invalidate","title":"invalidate(key) abstractmethod","text":"Invalidate cache by key.
PARAMETER DESCRIPTIONkey The cache key to invalidate.
TYPE: str
set_data(key, data) abstractmethod","text":"Store data to cache.
PARAMETER DESCRIPTIONkey The cache key to store the data.
TYPE: str
data The data to be stored.
TYPE: DataFrame
FileCache","text":" Bases: CacheStrategy
Cache data as file.
We store the cached files in a separate folder in the CKAN storage.
"},{"location":"caching/file/#charts.cache.FileCache.generate_unique_consistent_filename","title":"generate_unique_consistent_filename(key)","text":"Generate unique and consistent filename based on the key.
PARAMETER DESCRIPTIONkey The cache key to generate the filename.
TYPE: str
str The filename.
"},{"location":"caching/file/#charts.cache.FileCache.get_data","title":"get_data(key)","text":"Return data from cache if exists.
PARAMETER DESCRIPTIONkey The cache key to retrieve the data.
TYPE: str
DataFrame | None The data if exists, otherwise None.
"},{"location":"caching/file/#charts.cache.FileCache.invalidate","title":"invalidate(key)","text":"Remove data from cache.
PARAMETER DESCRIPTIONkey The cache key to invalidate.
TYPE: str
is_file_cache_expired(file_path) staticmethod","text":"Check if file cache is expired.
If TTL is 0 then cache never expires.
PARAMETER DESCRIPTIONfile_path The path to the file.
TYPE: str
bool True if file cache is expired, otherwise False.
"},{"location":"caching/file/#charts.cache.FileCache.make_file_path_from_key","title":"make_file_path_from_key(key)","text":"Generate file path based on the key
PARAMETER DESCRIPTIONkey The cache key to generate the file path.
TYPE: str
str The file path.
"},{"location":"caching/file/#charts.cache.FileCache.read_data","title":"read_data(file) abstractmethod","text":"Read cached data from a file object.
PARAMETER DESCRIPTIONfile The file object to read the data.
TYPE: IO
DataFrame | None The data if exists, otherwise None.
"},{"location":"caching/file/#charts.cache.FileCache.set_data","title":"set_data(key, data)","text":"Store data to cache.
PARAMETER DESCRIPTIONkey The cache key to store the data.
TYPE: str
data The data to be stored.
TYPE: DataFrame
write_data(file_path, data) abstractmethod","text":"Defines how to write data to a file.
PARAMETER DESCRIPTIONfile_path The path to the file.
TYPE: str
data The data to be stored.
TYPE: DataFrame
FileCacheORC","text":" Bases: FileCache
Cache data as ORC file
"},{"location":"caching/file/#charts.cache.FileCacheORC.read_data","title":"read_data(file)","text":"Read cached data from an ORC file.
PARAMETER DESCRIPTIONfile The file object to read the data.
TYPE: IO
DataFrame | None The data if exists, otherwise None.
"},{"location":"caching/file/#charts.cache.FileCacheORC.write_data","title":"write_data(file_path, data)","text":"Write data to an ORC file.
PARAMETER DESCRIPTIONfile_path The path to the file.
TYPE: str
data The data to be stored.
TYPE: DataFrame
FileCacheCSV","text":" Bases: FileCache
Cache data as CSV file
"},{"location":"caching/file/#charts.cache.FileCacheCSV.read_data","title":"read_data(file)","text":"Read cached data from a CSV file.
PARAMETER DESCRIPTIONfile The file object to read the data.
TYPE: IO
DataFrame | None The data if exists, otherwise None.
"},{"location":"caching/file/#charts.cache.FileCacheCSV.write_data","title":"write_data(file_path, data)","text":"Write data to a CSV file.
PARAMETER DESCRIPTIONfile_path The path to the file.
TYPE: str
data The data to be stored.
TYPE: DataFrame
Bases: CacheStrategy
Cache data to Redis as a CSV string
"},{"location":"caching/redis/#charts.cache.RedisCache.get_data","title":"get_data(key)","text":"Return data from cache if exists.
PARAMETER DESCRIPTIONkey The cache key to retrieve the data.
TYPE: str
DataFrame | None The data if exists, otherwise None.
"},{"location":"caching/redis/#charts.cache.RedisCache.invalidate","title":"invalidate(key)","text":"Remove data from cache.
PARAMETER DESCRIPTIONkey The cache key to invalidate.
TYPE: str
set_data(key, data)","text":"Serialize data and save to Redis.
PARAMETER DESCRIPTIONkey The cache key to store the data.
TYPE: str
data The data to be stored.
TYPE: DataFrame
Exception If failed to save data to Redis.
"},{"location":"engines/","title":"Index","text":"The ckanext-charts implements supports different chart engines, such as plotly, observable and chartjs.
Each engine consists of two classes - chart builder and form builder. The chart builder is responsible for generating a JSON-string representation of the chart data, that will be passed to a respective JS module, that will render a chart based on the data.
The form builder is responsible for generating a form that allows users to configure the chart from the UI.
Also, you can implement support of a chart library of your choice by creating a custom chart engine. Read more about it here.
"},{"location":"engines/chartjs/","title":"Chartjs","text":"The Chart.JS chart engine supports the following chart types:
ckanext/charts/chart_builders/chartjs.py class ChartJsBuilder(BaseChartBuilder):\n @classmethod\n def get_supported_forms(cls) -> list[type[Any]]:\n return [\n ChartJSBarForm,\n ChartJSHorizontalBarForm,\n ChartJSLineForm,\n ChartJSPieForm,\n ChartJSDoughnutForm,\n ChartJSScatterForm,\n ChartJSBubbleForm,\n ChartJSRadarForm,\n ]\n\n def _create_zoom_and_title_options(self, options: dict[str, Any]) -> dict[str, Any]:\n \"\"\"Add zoom and title plugin options to the provided options dictionary\"\"\"\n if \"plugins\" not in options:\n options[\"plugins\"] = {}\n\n options[\"plugins\"].update(\n {\n \"zoom\": {\n \"zoom\": {\n \"wheel\": {\"enabled\": True},\n \"pinch\": {\"enabled\": True},\n \"drag\": {\"enabled\": True},\n \"mode\": \"xy\",\n },\n \"pan\": {\n \"enabled\": True,\n \"modifierKey\": \"shift\",\n \"mode\": \"xy\",\n },\n },\n \"title\": {\n \"display\": True,\n \"position\": \"bottom\",\n },\n },\n )\n return options\nThe bar chart is a chart with rectangular bars with lengths proportional to the values that they represent. The bars can be plotted vertically or horizontally. For a horizontal bar chart, use the Horizontal bar chart chart type.
The horizontal bar chart is a chart with rectangular bars with lengths proportional to the values that they represent. The bars are plotted horizontally.
"},{"location":"engines/chartjs/#line-chart","title":"Line chart","text":"The line chart is a chart that displays information as a series of data points called 'markers' connected by straight line segments. It is useful for showing trends over time.
"},{"location":"engines/chartjs/#pie-chart","title":"Pie chart","text":"The pie chart is a circular statistical graphic that is divided into slices to illustrate numerical proportions. The arc length of each slice is proportional to the quantity it represents.
"},{"location":"engines/chartjs/#doughnut-chart","title":"Doughnut chart","text":"The doughnut chart is a variant of the pie chart, with a hole in the center. It is useful for showing the relationship of parts to a whole.
"},{"location":"engines/chartjs/#scatter-plot","title":"Scatter plot","text":"The scatter plot is a chart that uses Cartesian coordinates to display values for two variables for a set of data. The data points are represented as individual dots.
"},{"location":"engines/chartjs/#bubble-chart","title":"Bubble chart","text":"The bubble chart is a chart that displays data points as bubbles. The size of the bubble represents a third dimension of the data.
"},{"location":"engines/chartjs/#radar-chart","title":"Radar chart","text":"The radar chart is a chart that displays multivariate data in the form of a two-dimensional chart of three or more quantitative variables represented on axes starting from the same point. The data points are connected by a line to form a polygon.
"},{"location":"engines/custom/","title":"Custom chart engine","text":""},{"location":"engines/custom/#implementing-new-chart-engines-support","title":"Implementing new chart engines support","text":"Implementing support for a new chart engine includes multiple steps and changes in Python, HTML, and JavaScript. Starting from the Python code:
Create a new builder class at ckanext.charts.chart_builder that inherits from BaseChartBuilder and implements the get_supported_forms method. This method must return a list of classes that represent supported chart types forms.
Each form type builder must be connected with a respective chart type builder.
The chart type builder must implement a to_json method that will return a dumped JSON data, which will be passed to a JS script.
The form type builder must implement a get_form_fields method that will return a list of all form fields that will be rendered for the user, allowing them to provide all the necessary information for a chart.
Register your chart engine by adding the builder class to get_chart_engines in ckanext.charts.chart_builder.__init__.py.
A full example of an implementation of bar chart for obvervable plot library.
from __future__ import annotations\n\nimport json\nfrom typing import Any\n\nimport ckanext.charts.exception as exception\nfrom ckanext.charts.chart_builders.base import BaseChartBuilder, BaseChartForm\n\n\nclass ObservableBuilder(BaseChartBuilder):\n @classmethod\n def get_supported_forms(cls) -> list[type[Any]]:\n return [ObservableBarForm]\n\n\nclass ObservableBarBuilder(ObservableBuilder):\n def to_json(self) -> str:\n return json.dumps(\n {\n \"type\": \"bar\",\n \"data\": self.df.to_dict(orient=\"records\"),\n \"settings\": self.settings,\n }\n )\n\n\nclass ObservableBarForm(BaseChartForm):\n name = \"Bar\"\n builder = ObservableBarBuilder\n\n def fill_field(self, choices: list[dict[str, str]]) -> dict[str, str]:\n field = self.color_field(choices)\n field.update({\"field_name\": \"fill\", \"label\": \"Fill\"})\n\n return field\n\n def get_form_fields(self):\n columns = [{\"value\": col, \"label\": col} for col in self.df.columns]\n chart_types = [\n {\"value\": form.name, \"label\": form.name}\n for form in self.builder.get_supported_forms()\n ]\n\n return [\n self.title_field(),\n self.description_field(),\n self.engine_field(),\n self.type_field(chart_types),\n self.x_axis_field(columns),\n self.y_axis_field(columns),\n self.fill_field(columns),\n self.opacity_field(),\n self.limit_field(),\n ]\nAnother step is to register JS/CSS vendor libraries of the chart you want to use. Refer to CKAN documentation to read about adding CSS and JavaScript files using Webassets.
You also will need a CKAN JS module, that will be responsible for rendering the Chart. This module will work with the vendor library and will be responsible for rendering the chart in the container.
This module must be registered inside a webassets.yml as well.
ckan.module(\"charts-render-observable\", function ($, _) {\n \"use strict\";\n\n return {\n options: {\n config: null\n },\n\n initialize: function () {\n $.proxyAll(this, /_/);\n\n if (!this.options.config) {\n console.error(\"No configuration provided\");\n return;\n }\n\n var plot;\n\n switch (this.options.config.type) {\n case \"bar\":\n plot = Plot.barY(this.options.config.data, this.options.config.settings).plot();\n break;\n default:\n return;\n }\n\n this.el[0].replaceChildren(plot);\n }\n };\n });\nAnd an HTML file, that will provide a proper container and include your JS module with data-module.
{% asset \"charts/observable\" %}\n\n {% if chart %}\n <div id=\"chart-container\" data-module=\"charts-render-observable\" data-module-config=\"{{ chart }}\"></div>\n {% else %}\n <p class=\"text-muted\">\n {{ _(\"Cannot build chart with current settings\") }}\n </p>\n {% endif %}\nNote, that we should add {% asset \"charts/observable\" %} not only here, but in charts_form.html too.
The reason for having a separate HTML file and JS module is that different libraries may require different types of container elements (such as div, canvas, etc.) to initialize or may need additional boilerplate code to build a chart. There's no easy way to abstract this, so you have to implement these things yourself.
The Observable Plot chart engine supports the following chart types:
ckanext/charts/chart_builders/observable.py class ObservableBuilder(BaseChartBuilder):\n @classmethod\n def get_supported_forms(cls) -> list[type[Any]]:\n return [\n ObservableBarForm,\n ObservableHoriontalBarForm,\n ObservableLineForm,\n ObservablePieForm,\n ObservableScatterForm,\n ObservableAutoForm,\n ]\nThe bar chart is a chart with rectangular bars with lengths proportional to the values that they represent. The bars can be plotted vertically or horizontally. For a horizontal bar chart, use the Horizontal bar chart chart type.
The horizontal bar chart is a chart with rectangular bars with lengths proportional to the values that they represent. The bars are plotted horizontally.
"},{"location":"engines/observable/#line-chart","title":"Line chart","text":"The line chart is a chart that displays information as a series of data points called 'markers' connected by straight line segments. It is useful for showing trends over time.
"},{"location":"engines/observable/#pie-chart","title":"Pie chart","text":"The pie chart is a circular statistical graphic that is divided into slices to illustrate numerical proportions. The arc length of each slice is proportional to the quantity it represents.
"},{"location":"engines/observable/#scatter-plot","title":"Scatter plot","text":"The scatter plot is a chart that uses Cartesian coordinates to display values for two variables for a set of data. The data points are represented as individual dots.
"},{"location":"engines/observable/#auto-chart","title":"Auto chart","text":"The auto chart is a chart that automatically selects the best chart type based on the data. It is useful for quickly visualizing data without having to manually select a chart type.
"},{"location":"engines/plotly/","title":"Plotly","text":"The Plotly chart engine supports the following chart types:
Base class for Plotly chart builders.
Defines supported chart types for Plotly engine.
Source code inckanext/charts/chart_builders/plotly.py class PlotlyBuilder(BaseChartBuilder):\n \"\"\"Base class for Plotly chart builders.\n\n Defines supported chart types for Plotly engine.\n \"\"\"\n\n @classmethod\n def get_supported_forms(cls) -> list[type[Any]]:\n return [\n PlotlyBarForm,\n PlotlyHoriontalBarForm,\n PlotlyPieForm,\n PlotlyLineForm,\n PlotlyScatterForm,\n ]\nThe bar chart is a chart with rectangular bars with lengths proportional to the values that they represent. The bars can be plotted vertically or horizontally. For a horizontal bar chart, use the Horizontal bar chart chart type.
The horizontal bar chart is a chart with rectangular bars with lengths proportional to the values that they represent. The bars are plotted horizontally.
"},{"location":"engines/plotly/#line-chart","title":"Line chart","text":"The line chart is a chart that displays information as a series of data points called 'markers' connected by straight line segments. It is useful for showing trends over time.
"},{"location":"engines/plotly/#pie-chart","title":"Pie chart","text":"The pie chart is a circular statistical graphic that is divided into slices to illustrate numerical proportions. The arc length of each slice is proportional to the quantity it represents.
"},{"location":"engines/plotly/#scatter-plot","title":"Scatter plot","text":"The scatter plot is a chart that uses Cartesian coordinates to display values for two variables for a set of data. The data points are represented as individual dots.
"},{"location":"fetchers/","title":"Fetchers","text":"Fetchers are responsible for fetching data from different sources (DataStore, URL, file system, hardcoded data).
For the current implementation, we're working with resources that are uploaded to the DataStore, so the fetcher will be responsible for fetching the data from the DataStore.
But it might come in handy to have fetchers for other sources, like URL, file system, etc.
"},{"location":"fetchers/#implementing-new-fetchers","title":"Implementing new fetchers","text":"To register new fetchers, you need to create a new class that inherits from DataFetcherStrategy and implements the fetch_data and make_cache_key methods.
The fetch_data method should return a pandas DataFrame object with the data that should be displayed in the chart.
The make_cache_key method should return a unique string that will be used as a key to store the data in the cache.
See the base class DataFetcherStrategy for more information.
All fetchers must inherit from the DataFetcherStrategy class. This class provides the basic structure for fetchers and defines the methods that must be implemented by all fetchers.
fetch_data() abstractmethod","text":"This method should implement the data fetch logic.
All the necessary information should be provided in the constructor.
RETURNS DESCRIPTIONDataFrame pd.DataFrame: The fetched data
"},{"location":"fetchers/base/#charts.fetchers.DataFetcherStrategy.get_cached_data","title":"get_cached_data()","text":"Fetch data from the cache.
RETURNS DESCRIPTIONDataFrame | None pd.DataFrame | None: The cached data or None if not found
"},{"location":"fetchers/base/#charts.fetchers.DataFetcherStrategy.invalidate_cache","title":"invalidate_cache()","text":"Invalidate the cache for the data fetcher.
"},{"location":"fetchers/base/#charts.fetchers.DataFetcherStrategy.make_cache_key","title":"make_cache_key() abstractmethod","text":"This method should generate a cache key for the fetched data.
Every data fetcher should implement this method to support caching.
RETURNS DESCRIPTIONstr The cache key
TYPE: str
Fetch dataset resource data from the DataStore.
This fetcher is used to fetch data from the DataStore using the resource ID.
"},{"location":"fetchers/datastore/#charts.fetchers.DatastoreDataFetcher.__init__","title":"__init__(resource_id, limit=2000000, cache_strategy=None)","text":"Initialize the DatastoreDataFetcher.
PARAMETER DESCRIPTIONresource_id The ID of the resource to fetch data for.
TYPE: str
limit The maximum number of rows to fetch.
TYPE: int DEFAULT: 2000000
cache_strategy The cache strategy to use. If not provided, the configured cache strategy will be used.
TYPE: str DEFAULT: None
fetch_data()","text":"Fetch data from the DataStore.
RETURNS DESCRIPTIONDataFrame pd.DataFrame: Data from the DataStore
"},{"location":"fetchers/datastore/#charts.fetchers.DatastoreDataFetcher.make_cache_key","title":"make_cache_key()","text":"Generate a cache key for the DataStore data fetcher.
Uses the resource ID as the part of a cache key.
RETURNS DESCRIPTIONstr The cache key
TYPE: str
Bases: DataFetcherStrategy
Fetch data from the file system.
This fetcher is used to fetch data from a file on the file system.
Supported formatsCSVXLSXXLSXML__init__(file_path, file_format='csv', cache_strategy=None)","text":"Initialize the FileSystemDataFetcher.
PARAMETER DESCRIPTIONfile_path The path to the file.
TYPE: str
file_format The format of the file.
TYPE: str DEFAULT: 'csv'
cache_strategy The cache strategy to use. If not provided, the configured cache strategy will be used.
TYPE: str DEFAULT: None
fetch_data()","text":"Fetch data from the file system.
RETURNS DESCRIPTIONDataFrame pd.DataFrame: Data fetched from the file system
"},{"location":"fetchers/file/#charts.fetchers.FileSystemDataFetcher.make_cache_key","title":"make_cache_key()","text":"Generate a cache key for the FileSystem data fetcher.
Uses the file path as the part of a cache key.
RETURNS DESCRIPTIONstr The cache key
TYPE: str
Bases: DataFetcherStrategy
Fetch hardcoded data.
This fetcher is used to make a dataframe from hardcoded data, so you can build a chart from it.
"},{"location":"fetchers/hardcode/#charts.fetchers.HardcodedDataFetcher.__init__","title":"__init__(data)","text":"Initialize the HardcodedDataFetcher.
PARAMETER DESCRIPTIONdata The hardcoded data.
TYPE: dict[str, list[Any]]
fetch_data()","text":"Transform arbitrary data into a dataframe.
RETURNS DESCRIPTIONDataFrame pd.DataFrame: The hardcoded data as a dataframe
"},{"location":"fetchers/hardcode/#charts.fetchers.HardcodedDataFetcher.invalidate_cache","title":"invalidate_cache()","text":"Hardcoded data is not cached
"},{"location":"fetchers/hardcode/#charts.fetchers.HardcodedDataFetcher.make_cache_key","title":"make_cache_key()","text":"Hardcoded data is not cached
"},{"location":"fetchers/url/","title":"URL Data Fetcher","text":" Bases: DataFetcherStrategy
Fetch data from a URL.
This fetcher is used to fetch data from a URL.
Supported formatsCSVXLSXXLSXML__init__(url, file_format='csv', timeout=0, cache_strategy=None)","text":"Initialize the URLDataFetcher.
PARAMETER DESCRIPTIONurl The URL to fetch data from.
TYPE: str
file_format The format of the file.
TYPE: str DEFAULT: 'csv'
timeout The timeout for the request in seconds.
TYPE: int DEFAULT: 0
cache_strategy The cache strategy to use. If not provided, the configured cache strategy will be used.
TYPE: str DEFAULT: None
fetch_data()","text":"Fetch data from the URL.
RETURNS DESCRIPTIONDataFrame pd.DataFrame: Data fetched from the URL
"},{"location":"fetchers/url/#charts.fetchers.URLDataFetcher.make_cache_key","title":"make_cache_key()","text":"Generate a cache key for the URL data fetcher.
Uses the URL as the part of a cache key.
RETURNS DESCRIPTIONstr The cache key
TYPE: str
make_request()","text":"Make a request to the URL and return the response content.
RETURNS DESCRIPTIONbytes The response content
TYPE: bytes
DataFetchError If an error occurs during the request
"}]} \ No newline at end of file