diff --git a/README.md b/README.md index 563ec62..c3514ee 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ # OpenFDA BigData Pipeline -**OpenFDA BigData Pipeline** enables collection, processing and real-time presentation of data on adverse drug events from [openFDA](https://open.fda.gov/apis/drug/) project database. +**OpenFDA BigData Pipeline** enables collection, processing, and real-time presentation of data - on adverse drug events from the [openFDA](https://open.fda.gov/apis/drug/) project database. -The solution uses [Apache Kafka](https://kafka.apache.org/) as a message bus, [Mongo DB](https://www.mongodb.com) as a document storage, [Spring Boot](https://spring.io/projects/spring-boot) for services and is Dockerized. +The solution uses [Apache Kafka](https://kafka.apache.org/) as a message broker, [Mongo DB](https://www.mongodb.com) as a document storage, [Spring Boot](https://spring.io/projects/spring-boot) for services and is Dockerized. ## Contents @@ -60,14 +60,11 @@ In addition, you can access Jupyter Notebook `jupyter-notebook` via a browser un ## Example graphs -### Top patient reactions reported between 2020-01-01 and 2022-01-01 -![Top patient reactions](results/top-patient-reactions-2020-2022.png) +### Top 20 patient reactions reported since 2004 +![Top patient reactions](results/top-patient-reactions-all.png) -### Top patient medical products reported between 2020-01-01 and 2022-01-01 -![Top medical products](results/top-medical-products-2020-2022.png) - -### Number of patient reactions grouped by month reported between 2020-01-01 and 2022-01-01 -![Top reactions grouped by month](results/reactions-by-month-2020-2022.png) +### Top 20 patient medical products reported since 2004 +![Top medical products](results/top-medical-products-all.png) ## Issues and contribution diff --git a/pipeline-architecture-pl.drawio b/pipeline-architecture-pl.drawio index 75e7f43..1073546 100644 --- a/pipeline-architecture-pl.drawio +++ b/pipeline-architecture-pl.drawio @@ -1 +1 @@ -7Vxrc6u4Gf41nmk/OMNNGH+M7aQ727PbtDmd7vabDLJNjBEVOA759dUNkEB28Inj5CTOzubAqwvwXh89iAzc6ebpbwRmq99whJKBY0VPA3c2cBwbBGP6D5OUQjIeuUKwJHEkOzWC+/gZSaElpds4QrnWscA4KeJMF4Y4TVFYaDJICN7p3RY40a+awSXqCO5DmHSl/4mjYiWltj9uGn5B8XIlLx04I9GwgVVn+ST5CkZ4p4jcm4E7JRgX4mjzNEUJU16lFzHudk9rfWMEpUWfAQ/u8p/gHw7+96j08H+fx/mSTIeemOURJlv5wPJmi7LSAIqoQuQpJsUKL3EKk5tGOiF4m0aIXcaiZ02fbxhnVGhT4QMqilJaF24LTEWrYpPIVnFNdqG9zyZFOd6SEB14oMpHIFmi4kA/p7YAdV2EN6ggJR1HUAKL+FG/Dyh9aFn3a9RMD6Smj9B65SOK2rPtPInX2wc0cPyE3vhkTujRkh1FMEVGq3yDcxptmiZhEi9TehxSxSFCBY+IFDF152vZsImjSBgN5fEznPP5mNkyHKcFf1AwGYBZbRU2AXoaGGJNDm48XLXXfm/rKl3OPrSuLDCS15DpQvpnb7PIye/Ywygzu9qkw/YMeLHIqbe0zVrfosnSeP7A8g196gSGaEXzCiKtlOR9LzN5yzhD6SKCw4zgaBtSy8geU5wWME6ZqZrgowNRuEpxgpfyju8zEqdL2jRh6cKh92L9Ch9h1X2G8pDEWRHjVPTfxCHBOSKPcShdJxG+Ii4iPGyBuY4ap/L/t8VVwzDnwXpNO9h+9tQ0Vl5Z+2kjAPKJHVA1Up2Jq+hjovixFgExsFEEHc0u68MNTR6TdJ5n9eSNVmgnMFOuos7YCSDTZY/VgX1IB3KWkN4b0WYZOO6Nz/4zjQW64V7UWvsxnZfSjK1EhExLSgah9Yh75wo9wSVznIl43Bng6QCRmDo/d0zZ464S0Uknu1VcoPsM8mS8o7Vfz0OL+AlV1dzWSgQ74844geF6yeVTqbcUp4iPTZJprUrXca+d2VhquZrSkedKvwX/MeXAvCB4jZSu1s1ods1SHH0ceBPFhbwvPQdOrCtqkqnF/2feNqUym51dOcAgNMlGulB0snWZsaNhRqt1aZ6k22l8b9LuFNO9edi3tGzpuvJ81+Afx5KylQJ97JF1dL2kpzKRHlE+gf0hUEtOQUZxzTBm47tcdhuzx+PTojSqeoQJzPM4FELZ5cTgx+kJfsDbgJ/jSuIaLtZ1BetZB69pwlnRAmP9XR3cLX8ozxmud6z5Nr/Uv69a/1SMv7/+hWUS00RA3KYC2oBri1BvFj5FI4+lvpeq3lyklG/zSqDnmEvZ+8hlr65zVd0DoFv3PFPdc85V96yPV/eaqqaWPvvspQ/0LH2jj1D6qtUgXUjn281lNXiphmeohiroO2I16F9Wg1+6LPrOR18OOu4HroInLHGjniXO3mPQV5KottXyBGC1DCzuTA5rbEyVBEulm4yr/hfyPavlM2LKnkxtTy8CHXC1K/PnreDkrQimZbjqVhPapxxMwSBwIIuJMo3XsdEdfzqyXoTV3rxA85HnBXpqGJyErKeZQ5t26IyAPsnb8/UbnC7xkbDsNzZmNtkLwyIcUrDHHzWC1CAwv6CxL4vG1HXI8dzEhYf4xIDLapXZwAC43pWHqN7KnhdwUfWR8g82/gpUp38OKpqdnsyetLNSPbtTFjCNnU4Cy4K+sOyVqOx1Ow78DrrJyDNSsYvYZtDOz798/343YI5120371r9u7r9/ErDz4tYERwtK5yRQx9Xxk38elPM73Cg81O3sugt/bp6ojWhwUvF9mRdosx/UkC1jomBErcHwjIUeEUf374Nx+LNp1XpuKNU9cU8FmH4SNBPyn3dBM8FBNKPTPS9gl6PASTANvLF/DBiBJKxAjGXCJiPXHwVOF5scTkqTGhh04UZbNjo9qhkZYI0J6ohLnBL/nADveDrcsa2PBne87m6998A/H2+7gd13s6W9h0o4rny+mkdyrdYO2SMHHEs8/dhLoYQqZBjBfDXHkERHchC31Opr8UpoRqcQR3cldbZ0bwnfoTnTapYlFI6JAnFhJb4mK6FRx/tpic3Tku32vwq9K+o8E7bLHhHFQ8/JT9h2cD2dXfiJs/ETQdBKkqYXQsaC7b+6YB+3vNmyxe1kWhZrvEvjdTfl3tHRBxLjZeXy+VcuGjTpZjz1JbaSX9g5K9F1yuIJL9cTXitRWYF7OwLGfOO7NpgeSFmGvJtx13VM+exHs9aHWGq8VdYat6CdZ0haxrfYwZlWGa5pU7NwaupXqTEAZTCx4CPL+V8sTtQxxKcc/ZUdcraOh+sCbuKkFGNqyER9zfVYiUbJI2KkXKdFn6Qd81qbuEvWmGKy4RyS0ryTumXtnrhP3pjQtQ8iQxZCbG+TaTw1fjGU3s6apcNrzextSSqnt5Rb440FgWm+oJNW0zPCU3bYYRLpV1eHz2swMmzp3GG7BKWuW8eN5qM4pwVLaj1Ok1i58CLBsFBvqJ0En2EW5/VLaANFq+Q+4Slq8lNyGdPBy9jsBLE2au2kBE432AJDrLnHh1rP2HI6sbXmW+8+z/d2L3AswfhqPPbHVMUj+tvReWfvkOL3LJVtoE0x9E5AXvc0ZpeO2eA0LjAxWnNXrgl7z/G5DGpbgFm0+dEt6oy4vW3gBh7/3du+lT3H5zKn141No60+xnauHgSb8p7SsvQ3lVe+Pxq879tK77ycXSdxBOMW3dteFO7ZRHYyd/N/bu86uSuc3sItQt+3QC8LH0vvtlGGMzrM7gb+6/pXz3EWMvhhm5UMD6e4QHOMa+6iJw38qxhO235vTWAgOawdmg/Zm9qIo1M6Dobc/qxQbbJtIT42oPddcPB8YYe/KDusJYwLO3xhh02712w9a/bevPZ6crgnAHANePMUwVvsMHkuu9mCrt/LAqYx3BuO5nX6Gy1KGDjJm09xdPc2BJEUncA12gXVtru+4Rtcw3srWqDavv7jlBs0c26DPuwbXbRiwRjv4dZ6ElDsKSrSyRsrBBS7uKFKtP9YF6AaY1L+l6jqs0qDgOuQSmbsmE0OmJYA2w/4Ql+77ls5yQ9N4zTTCEPVLcr3GYDbjbbwwAP8adkZ/zzDAVOa9wDf5giajzSqWYWR61kbYoBNwKOQtdn8VEQiO6/YR9CORtYo45E1NhHJGiwuq6CtuCRHLCxPA26f6llFqJm0qdIHBpV2tF0PrRXdBCdg4Vn3DCxlmlKxwdi6qqgE9rvpBPOm11KZtW1cflpbWBXqfif7dRy0H8X5s5M6vk7q6BwbEJSP5PDc9rKqB6lzgs9oen6p3iV14JKg5SEa+1N8SfWigS2DSdW4Oh4CNX9BcNCKHPfm/w== \ No newline at end of file +7Vxbd6O6Ff41Xqt9cBY3YfwY20nb0zmn0+Z09fImg2wzxogKHIf8+m5dAAlwgiceTybjnHUyaOu+r582IiN3vnv6E8PZ5lcakWTkWNHTyF2MHMdGwRT+4ZRSUqYTVxLWLI5Uo4bwED8TRbQUdR9HJDcaFpQmRZyZxJCmKQkLg4YZowez2Yom5qwZXpMO4SHESZf6rzgqNopq+9Om4s8kXm/U1IEzkRU7XDVWO8k3OKIHjeTejdw5o7SQT7unOUk48yq+yH73R2rrhTGSFkM6fHHXf0d/c+g/J6VH//s8zddsPvbkKI842asNq8UWZcUBEgFDVJGyYkPXNMXJXUOdMbpPI8KnsaDUtPlEaQZEG4hfSFGUSrp4X1AgbYpdomrlnHyio3tTpJzuWUhe2FClI5itSfFCO6eWAKguoTtSsBL6MZLgIn4014GVDq3rdg2b4UFx+gSuVzqisT3bL5N4u/9CRo6fwMJnSwZPa/4U4ZT0SuUTXoK1GZzESbxO4TkExhEGhEfCihjU+VZV7OIokkIjefyMl2I8LraMxmkhNopmI7SopcIHIE+jHltTnRsN1+V1XNu6TFejj60bC03UHMpdKP0cLBY1+Ge+GW1k1xh03B6BrlY5aEtbrPUS+yRNl1+4v4FdJzgkG/ArhLVckvd7makl04ykqwiPM0ajfQiSUS3mNC1wnHJRNcYHHUm4SWlC12rFDxmL0zVUzbi7cGAt1i/4EVfNFyQPWZwVMU1l+10cMpoT9hiHSnUSqStyEqlhKyp41CiV/789rSrGuTDWW2hg+9lTU1lpZa2nDQGpHTuoqgSeyVnMPlH8WJOQ7NgwAnrzaX28A+cxS5d5Vg/ecAUaoYU2iz5ix4D6pj2VB/ZLPFCjhLA2Zowyctw7n//X1xeZgnuVa+1tOq+5GVuzCOWWNA8C8Uho54Y84TVXnJnc7gIJd0BYDMovFFO1+FyRYNDZYRMX5CHDwhkfIPabfmgVP5EqmttGiOAloYwzHG7Xgj5XfEtpSkTfJJnXrHQd99ZZTBWXqyEdVdbarcRPnw/MC0a3RGtq3U0Wt9zFwXbwXRQXal2mD5xZNyCSuSX+59o2B5rNSzcO6iH20SYmUTayTVpvw54RrdbUwkm33fhRp90Jpkf9sG8Z3tJ1VfnQ4B/HUrSNBn3siXVyvISicqQnhE9kvwvUkgPIKG45xmx0V9DuY749MSxJo6pFmOA8j0NJVE3ODH6cgeAHfRvwc1pI3OLVto5gA+PgLTicDQQY66965274I3nOcb1jLff5Nf79rPFPx/jH419YJjE4AuY2EdBGglsMtFnqFFged32vRb2ldCmflhXB9DHXsPeew14d56q4h1A37nl9cc+5VNyz3l/ca6KaHvrsi4c+NDD0Td5D6KtOg3CQzve762nwGg0vEA110HfCadC/ngZ/6rDoO+/9OOi47zgKnjHETQaGOPuIQN+YRLWtliYgqyVguTLVrZExMAmXWjNlV8Mn8j2rpTNyyIGZ2oFahDrg6lDmz3uZk7dkEr4dS6BFOZqjUeBgbhFlGm/jXmX84VL10qiOegXwRp4XmI5hdJZUPfgNY9ixM0HmIN8+W7+j6ZqeCMp+5X0Ws6MgLKIhQD2x1QiDQHB+xWI/LRbTTyGnZyauWYgPDLesVpANeuDWd81CVO9kLwu3gH2s/Dfvf4Oq4n9GVZIdCosno1Tqpc/a8aWR01lAWTAUlL0Rk73tvoHfwTYZeyY6dunHN/+4e/gdKm8//+WDoJpXbyA4hvU5Z8E0rgmU/MvAmd/wTks33S9uuzjn7glkBFYI5IcyL8juOHphe55wwhFIgwMXizwSAeK/D5gRezPC8rInJg8EOBUy+kFgSyh+vgtsCV6ELWZW5xWQchIKCeaBN/VPQR2YhRVasfpAyMT1J4HTBSEvO6VZjQC6uKJNm5wfvkx68EsfppFTnBPonAHYeCausa33hmu87qW87wF03t+tAnvonUr7SM7gtPD55nSRa7Uuwp7Y4dT80te9+0mAIeMI55slxSw6MdlwD1Lfyjc/CxhCPn0uQdnSoyH8QJacq1mWAByTAeKafvg50w9Ghvh4/mH3tOaX+m9C7waUZ8Yv0xOmaeglExG2HdzOF9dExMUSEUHQcpJ97316A7b/5oB92vFmz0+xs3lZbOkhjbddlwvH//wFx3g9uXz8k4sBTboeT39XrfkXXuYhunZZwuHlpsNrOSorcO8nqNff+K6N5i+4rB6/mwnVdfr82dd6rXdx1PhWXmvagnZej9PqfVkdXOiU4fbdXZZKDXqV9hqgMiZufGy9/AOs1JKIT3v6I3/kLLSEua7wLk5K2aeGTKBrrsdDNEkeCU/KdWrMQdo2b9TJVfLKlLKdyCFp1QfFW17vyXWKygTOPoSNuQnxK0x9/UH4xVhpO69WCm9U89ciqRre0pYmKguG03wFg1bD88ymanCgLDJn17svazAybvHc4ZcBFa9bzw3noziHgKW4HqdJrE28Sigu9AW1neAzzuL8pXfNmu+TmqI7P82XcR68js3OYGuT1oVJ5HSNLeixNfd0UxtoW07Htrbiht3H+azulRxLML2ZTv0psHgCvx0z7+y9xPgjR2UbGUOMvTMkrwcKs5uO2dE0Lijrleah3DL+QuNjCdS2EJdo82NK1JkIedvIDTzxe7B8K3lOLyVOr2ubvbJ6H7e2BiTYtBeSlmW+krzx/cno+76W9C6bs+s4jmDaSve2D4VH7oqdTd38H1u7zq4K55dwK6HvW2iQhE9N77ZRhjN5Obsb+G9rX+3jIsngL/us5Hg4pQVZUlrnLgamgX+R3aHut9YAPUkO60CWY/6mNhLoFPrhUMifB6pdti/kNwWw7kKA52t2+CfNDhsO45odvmaH+66p2abXHHxL7e3J4YEAwO3Bm+cw3uJA2XPZ9RZwfi8LnMb4qDn2n9O/0aGEg5O8+eLGVO8eI1KkM6hGO6Dadlc3/B7V8L5VWqC6p/71KTfcn3MbDcm+waGVyozxkdzawAQU30WVdPKmWgKKT94TJdp/kwsBxzhV/MGpulRxEAkeAmXBn/ngiHMJAeNfa2vXbSsl+aphnGYYKai6RvsMAwm5QY0wPCR2y0viOwwHzcHvIXGfETVfY1SjSiHXozaJAT6AsEJeZ4uitERerrKPqG2NvFLZI69sLJJXWIJWQVs5pUAs3E8jIZ9qr9LU+rippw96WNrhdt21ZnRjnIibZ90ysLRhSk0GU+umSiXw300jnDet1tqobeGKYi1hnWjqnWrXUdBhKc4fPanjm0kdM8eGZMpH5fDc9rFqQFLnDN/LDPwgvZvUwWtG1h/+k6lXBWz1iFS3q9MhUPOHAkcty3Hv/g8= \ No newline at end of file diff --git a/pipeline-architecture.png b/pipeline-architecture.png index fe255d9..a767f4c 100644 Binary files a/pipeline-architecture.png and b/pipeline-architecture.png differ diff --git a/results/top-medical_products-all.png b/results/top-medical-products-all.png similarity index 100% rename from results/top-medical_products-all.png rename to results/top-medical-products-all.png