diff --git a/2023-10-xx-graphs-Open Ecosystem.ipynb b/2023-10-xx-graphs-Open Ecosystem.ipynb index 69819b2..11c0ad1 100644 --- a/2023-10-xx-graphs-Open Ecosystem.ipynb +++ b/2023-10-xx-graphs-Open Ecosystem.ipynb @@ -11,6 +11,43 @@ "## POSTER" ] }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "skip" + } + }, + "source": [ + "Structure:\n", + "- title\n", + "- two main sentences\n", + "- picture/table\n", + "- reference\n", + "\n", + "Suggestions:\n", + "- 1 every 5 slides should be fun (color, gif, joke)\n", + "- infographic: how does each topic relate to graphs\n", + "- concepts should be easy\n", + " - simple: family connections\n", + " - simple: family networks - which group of people stuck together most?\n", + " \n", + " - OSM: traffic expected\n", + " - OSM: bridge built next after disaster\n", + " \n", + " - wiki: automatically changing related people and events\n", + " - wiki: ?\n", + " \n", + " - psyc: leadership\n", + " - psyc: loyalty of the voters\n", + " \n", + " - npsyc: diseases like Parkinsons, ASD\n", + " - npsyc: events like Stroke-Aphasia, Stroke-RightH\n", + " \n", + " - biomed: drug repurposing\n", + " - biomed: ??" + ] + }, { "cell_type": "markdown", "metadata": { @@ -26,8 +63,7 @@ "2. How graphs build OSM networks?\n", "3. How graphs build wikipedia knowledge?\n", "4. Graphs and open neuropsychology\n", - "5. Graphs and open biomedicine\n", - "6. Graphs and AI" + "5. Graphs and open biomedicine" ] }, { @@ -51,7 +87,9 @@ }, "source": [ "### Nodes and edges\n", - "*I am a graph, you are a graph, we are a graph*" + "*How family trees start becoming useful ... \n", + "Who are the third cousins that already have babies (and not married to their own cousin)?*\n", + "\n" ] }, { @@ -82,23 +120,6 @@ "*new properties like betweenness, centrality, x*" ] }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Language as a graph\n", - "*Also language can be thought of as a graph, \n", - "with nodes as words and grammatical relations as edges.*\n", - "\n", - "\n", - "\n", - "*Ref: Spacy model for language, https://demos.explosion.ai/displacy*" - ] - }, { "cell_type": "markdown", "metadata": { @@ -128,300 +149,6 @@ "*Ref: Tirana in OSM, https://www.openstreetmap.org/relation/1250106*" ] }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "slideshow": { - "slide_type": "skip" - } - }, - "outputs": [], - "source": [ - "import warnings\n", - "warnings.filterwarnings('ignore')\n", - "std_kwargs = {\n", - " \"figsize\":(12, 12),\n", - " \"legend\":True,\n", - " \"legend_kwds\":dict(loc=\"upper left\", ncol=3, bbox_to_anchor=(1, 1)),\n", - "}" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Nodes and edges in OSM" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "outputs": [ - { - "data": { - "text/html": [ - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
yxstreet_counthighwaygeometry
osmid
1546083041.32737519.8218874NaNPOINT (19.82189 41.32737)
1549271841.32441019.8221174traffic_signalsPOINT (19.82212 41.32441)
1549271941.32397019.8222223NaNPOINT (19.82222 41.32397)
\n", - "
" - ], - "text/plain": [ - " y x street_count highway \\\n", - "osmid \n", - "15460830 41.327375 19.821887 4 NaN \n", - "15492718 41.324410 19.822117 4 traffic_signals \n", - "15492719 41.323970 19.822222 3 NaN \n", - "\n", - " geometry \n", - "osmid \n", - "15460830 POINT (19.82189 41.32737) \n", - "15492718 POINT (19.82212 41.32441) \n", - "15492719 POINT (19.82222 41.32397) " - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
osmidnamehighwayonewayreversedlengthlanesgeometrymaxspeedwidthrefserviceaccessjunctiontunnelbridge
uvkey
1546083059202674580469986175Rruga Murat ToptaniresidentialFalseTrue5.153NaNLINESTRING (19.82189 41.32737, 19.82190 41.32733)NaNNaNNaNNaNNaNNaNNaNNaN
82346833980623138259NaNpedestrianFalseFalse7.587NaNLINESTRING (19.82189 41.32737, 19.82187 41.32744)NaNNaNNaNNaNNaNNaNNaNNaN
2581695450624093873Rruga Abdi ToptanisecondaryTrueFalse73.8262LINESTRING (19.82189 41.32737, 19.82205 41.327...NaNNaNNaNNaNNaNNaNNaNNaN
\n", - "
" - ], - "text/plain": [ - " osmid name highway oneway \\\n", - "u v key \n", - "15460830 5920267458 0 469986175 Rruga Murat Toptani residential False \n", - " 8234683398 0 623138259 NaN pedestrian False \n", - " 258169545 0 624093873 Rruga Abdi Toptani secondary True \n", - "\n", - " reversed length lanes \\\n", - "u v key \n", - "15460830 5920267458 0 True 5.153 NaN \n", - " 8234683398 0 False 7.587 NaN \n", - " 258169545 0 False 73.826 2 \n", - "\n", - " geometry \\\n", - "u v key \n", - "15460830 5920267458 0 LINESTRING (19.82189 41.32737, 19.82190 41.32733) \n", - " 8234683398 0 LINESTRING (19.82189 41.32737, 19.82187 41.32744) \n", - " 258169545 0 LINESTRING (19.82189 41.32737, 19.82205 41.327... \n", - "\n", - " maxspeed width ref service access junction tunnel \\\n", - "u v key \n", - "15460830 5920267458 0 NaN NaN NaN NaN NaN NaN NaN \n", - " 8234683398 0 NaN NaN NaN NaN NaN NaN NaN \n", - " 258169545 0 NaN NaN NaN NaN NaN NaN NaN \n", - "\n", - " bridge \n", - "u v key \n", - "15460830 5920267458 0 NaN \n", - " 8234683398 0 NaN \n", - " 258169545 0 NaN " - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "import osmnx as ox\n", - "place_name = \"Njësia Bashkiake Nr. 2, Tirane, Albania\"\n", - "graph = ox.graph_from_place(place_name)\n", - "nodes, edges = ox.graph_to_gdfs(graph)\n", - "display(nodes.head(3))\n", - "$display(edges.head(3))" - ] - }, { "cell_type": "markdown", "metadata": { @@ -469,36 +196,30 @@ ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, - "outputs": [], "source": [] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, - "outputs": [], "source": [] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, - "outputs": [], "source": [] }, { @@ -521,10 +242,11 @@ } }, "source": [ - "### Networks of traits \n", + "### Networks of psychology\n", + "*How do traits affect things like leadership or voting loyalty?*\n", "\n", "\n", - "\n", + "\n", "\n", "*Ref: https://www.hindawi.com/journals/complexity/2019/9424605/*" ] @@ -617,12 +339,13 @@ } }, "source": [ - "### Building precision medicine\n", - "\n", + "### Language as a graph\n", + "*Also language can be thought of as a graph, \n", + "with nodes as words and grammatical relations as edges.*\n", "\n", - "\n", + "\n", "\n", - "*Ref: https://www.nature.com/articles/s41597-023-01960-3*" + "*Ref: Spacy model for language, https://demos.explosion.ai/displacy*" ] }, { @@ -633,9 +356,12 @@ } }, "source": [ - "### Data: Ontologies\n", + "### Building precision medicine\n", "\n", - "*Ref: MOX*" + "\n", + "\n", + "\n", + "*Ref: https://www.nature.com/articles/s41597-023-01960-3*" ] }, { @@ -646,28 +372,27 @@ } }, "source": [ - "### Tools: Open Targets\n", + "### Data: Ontologies\n", "\n", - "*Ref: https://platform.opentargets.org/*" + "*Ref: MOX*" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { - "slide_type": "slide" + "slide_type": "subslide" } }, "source": [ - "# Graphs and AI\n", - "*GNN vs GPTs*" + "### Tools: Open Targets\n", + "\n", + "*Ref: https://platform.opentargets.org/*" ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": {}, - "outputs": [], "source": [] } ], diff --git a/2023-11-04-cryptoparty-OSINT for Journalism.ipynb b/2023-11-04-cryptoparty-OSINT for Journalism.ipynb new file mode 100644 index 0000000..a3e3c37 --- /dev/null +++ b/2023-11-04-cryptoparty-OSINT for Journalism.ipynb @@ -0,0 +1,190 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "## Table of Contents\n", + "\n", + "- Why OSINT?\n", + "- Useful tools\n", + " - open-data \n", + " - fishing documents in websites\n", + "- OSINT in the Albanian Context\n", + " - OpenCorporates\n", + " - LibreOffice + PMPC + networks/graphs\n", + " - GeoSpatial Intelligence\n", + "- Practice" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "# Why OSINT?\n", + "\n", + "*how to get from one place to next?* \n", + "*use OSMNX https://osmnx.readthedocs.io/en/stable/*\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "# Useful tools\n", + "\n", + "*how to get from one place to next?* \n", + "*use OSMNX https://osmnx.readthedocs.io/en/stable/*\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "## The map of OSINT tools\n", + "\n", + "*how to get from one place to next?* \n", + "*use OSMNX https://osmnx.readthedocs.io/en/stable/*\n", + "\n", + "\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "# OSINT in the Albanian context\n", + "\n", + "*how to get from one place to next?* \n", + "*use OSMNX https://osmnx.readthedocs.io/en/stable/*\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "## OpenCorporates\n", + "\n", + "*Track businesses.* \n", + "*use https://opencorporates.al/*\n", + "\n", + "\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "# Practice\n", + " \n", + "*use OSMNX https://osmnx.readthedocs.io/en/stable/*\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "## Analyzing trips with OSS\n", + "*Analyzing gps data for people, bikes, buses, cars, birds, football players* \n", + "*using movingpandas https://movingpandas.org/*\n", + "\n", + "| People moving | Birds moving | Football | \n", + "|---| --| --|\n", + "| | | |" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "# References\n", + "\n", + "- https://wiki.openlabs.cc/index.php?title=CryptoParty_Tirana_04/11/2023\n", + "- https://pad.openlabs.cc/NUCxcxzvT1imddByZ5oXEw\n", + "- https://en.wikipedia.org/wiki/Bellingcat" + ] + } + ], + "metadata": { + "celltoolbar": "Slideshow", + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + }, + "vscode": { + "interpreter": { + "hash": "e7370f93d1d0cde622a1f8e1c04877d8463912d04d973331ad4851f04de6915a" + } + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/2023-11-04-cryptoparty-OSINT for Journalism.slides.html b/2023-11-04-cryptoparty-OSINT for Journalism.slides.html new file mode 100644 index 0000000..bb1e6a8 --- /dev/null +++ b/2023-11-04-cryptoparty-OSINT for Journalism.slides.html @@ -0,0 +1,15398 @@ + + + + + + + + + +2023-11-04-cryptoparty-OSINT for Journalism slides + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
+
+
+ + + + + + + + + + + diff --git a/docs/crypto/OSINT-all.png b/docs/crypto/OSINT-all.png new file mode 100644 index 0000000..15efb75 Binary files /dev/null and b/docs/crypto/OSINT-all.png differ diff --git a/docs/crypto/opencorporates.png b/docs/crypto/opencorporates.png new file mode 100644 index 0000000..60a4eb8 Binary files /dev/null and b/docs/crypto/opencorporates.png differ diff --git a/docs/crypto/poster-crypto.png b/docs/crypto/poster-crypto.png new file mode 100644 index 0000000..a84d4c7 Binary files /dev/null and b/docs/crypto/poster-crypto.png differ diff --git a/docs/crypto/timeline.jpeg b/docs/crypto/timeline.jpeg new file mode 100644 index 0000000..0965588 Binary files /dev/null and b/docs/crypto/timeline.jpeg differ diff --git a/graphs.html b/graphs.html new file mode 100644 index 0000000..ab82408 --- /dev/null +++ b/graphs.html @@ -0,0 +1,392 @@ + + + + + + + + + + + + + + + + + + + +
+ + + + + + \ No newline at end of file diff --git a/notes-foss4g.md b/notes-foss4g.md new file mode 100644 index 0000000..531884c --- /dev/null +++ b/notes-foss4g.md @@ -0,0 +1,29 @@ +# Notes + +FOSS4G: Open Source conference. + +### Interesting papers + +- Open Geospatial Software and DATA: https://www.mdpi.com/2220-9964/9/2/90 +- CLIMATE CHANGE: https://www.mdpi.com/2220-9964/10/2/94 +- REVIEW: https://www.mdpi.com/2220-9964/11/2/85 +- SMART CITY and BIG DATA: https://www.mdpi.com/2220-9964/10/1/13 +- EPIDEMIOLOGY and GIS: https://www.mdpi.com/2220-9964/9/6/352 + +### Extra + +- Crowd Flow Analysis: https://www.mdpi.com/2220-9964/8/10/440 +- Meaning of the word SPACE: https://www.mdpi.com/2220-9964/9/1/58 +- Big data and Disaster management - Tech: https://www.mdpi.com/2220-9964/8/9/387 +- Input data management: https://www.mdpi.com/2220-9964/8/9/376 +- Sustainability assessment: https://www.mdpi.com/2220-9964/11/11/569 +- Historic Building Information Modelling: https://www.mdpi.com/2220-9964/11/9/472 +- POLICE PATROL: https://www.mdpi.com/2220-9964/9/3/157 + +### References + +- https://2023.foss4g.org/call-for-papers/ +- https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLVIII-4-W1-2022/ +- https://www.mdpi.com/journal/ijgi +- https://www.mdpi.com/search?sort=pubdate&page_count=50&journal=ijgi&featured=&article_types=review-article%2Ccase-report%2Cproject-report%2Csystematic_review&countries= + diff --git a/setup/build.sh b/setup/build.sh index 5c5a4cb..986c04e 100644 --- a/setup/build.sh +++ b/setup/build.sh @@ -1,3 +1,4 @@ #/bin/bash jupyter nbconvert --to html index.ipynb +jupyter nbconvert --to slides "2023-11-04-cryptoparty-OSINT for Journalism.ipynb" --reveal-prefix=reveal.js