-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
5ecaef7
commit 72a2397
Showing
2 changed files
with
192 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
pacman::p_load(tidyverse, pins, connectapi) | ||
|
||
# Download the file from google drive | ||
sdrive <- shared_drive_find("byuids_data") # This will ask for authentication. | ||
google_file <- drive_ls(sdrive) |> | ||
filter(stringr::str_detect(name, "carwash")) | ||
tempf <- tempfile() | ||
drive_download(google_file, tempf) | ||
car_wash <- read_csv(tempf) | ||
|
||
# Publish the data to the server with Bro. Hathaway as the owner. | ||
board <- board_connect() | ||
pin_write(board, car_wash, type = "parquet", access_type = "all") | ||
|
||
pin_name <- "car_wash" | ||
meta <- pin_meta(board, paste0("hathawayj/", pin_name)) | ||
client <- connect() | ||
my_app <- content_item(client, meta$local$content_id) | ||
set_vanity_url(my_app, paste0("data/", pin_name)) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,173 @@ | ||
--- | ||
title: "Carwash" | ||
author: "DS 350" | ||
date: "2024-05-08" | ||
categories: [businesses] | ||
description: "Carwash sales over time" | ||
params: | ||
source: "" | ||
pins_name: 'hathawayj/car_wash' | ||
available: all | ||
--- | ||
|
||
|
||
```{r} | ||
#| echo: false | ||
title <- params$title | ||
description <- params$description | ||
source <- params$source | ||
pins_url <- params$pins_url | ||
``` | ||
|
||
# `r params$title` | ||
|
||
`r params$description` | ||
|
||
```{python} | ||
#| tags: [parameters] | ||
#| echo: false | ||
title = r.title | ||
description = r.description | ||
source = r.source | ||
pins_url = r.pins_url | ||
``` | ||
|
||
|
||
## Data details | ||
|
||
```{r} | ||
#| echo: false | ||
#| warning: false | ||
# https://aeturrell.github.io/skimpy/ python skimr for Python | ||
library(pins) | ||
library(tidyverse) | ||
library(skimr) | ||
library(arrow) | ||
board <- board_connect() | ||
dat <- pin_read(board, params$pins_name) | ||
meta <- pin_meta(board, params$pins_name) | ||
url_path <- meta$local$url | ||
file_name <- meta$file | ||
object_name <- rev(unlist(str_split(meta$name, "/")))[[1]] | ||
url_download <- paste0( | ||
str_remove( | ||
url_path, | ||
paste0( | ||
rev(unlist(str_split(url_path, "/")))[2], | ||
"/")), | ||
file_name) | ||
``` | ||
|
||
There are __`r prettyNum(nrow(dat), big.mark = ",", scientific = FALSE)` rows__ and __`r ncol(dat)` columns__. The data source[^1] is used to create our data that is stored in our [pins table](`r meta$local$url`). You can access this pin from a connection to [posit.byui.edu](https://posit.byui.edu) using ``r params$pins_name``. | ||
|
||
This data is available to __`r params$available`__. | ||
|
||
### Variable description | ||
|
||
- __name__ Carwash name | ||
- __type__ Carwash type | ||
- __time__ (YYYY-MM-DD HR:MIN:SEC) | ||
- __amount__ Sales (dollars) | ||
|
||
### Variable summary | ||
|
||
```{r} | ||
#| echo: false | ||
#| warning: false | ||
skim(dat) |> yank("numeric") | ||
``` | ||
|
||
|
||
```{r} | ||
#| echo: false | ||
#| warning: false | ||
skim(dat) |> yank("character") | ||
``` | ||
|
||
<!-- ```{r} | ||
#| echo: true | ||
#| eval: false | ||
#| code-fold: true | ||
#| code-summary: Explore generating code using R | ||
#| code-line-numbers: true | ||
#| file: ../../../data_scripts/YOURFILEPATH | ||
``` --> | ||
|
||
```{python} | ||
#| echo: true | ||
#| eval: false | ||
#| code-fold: true | ||
#| code-summary: Explore generating code using Python | ||
#| code-line-numbers: true | ||
#| file: ../../../data_scripts/car_wash.R | ||
``` | ||
|
||
|
||
## Access data | ||
|
||
This data is available to __`r params$available`__. | ||
|
||
__Direct Download:__ [`r file_name`](`r url_download`) | ||
|
||
#### __R and Python Download:__ | ||
|
||
##### __URL Connections:__ | ||
|
||
For public data, any user can connect and read the data using `pins::board_connect_url()` in R. | ||
|
||
```r | ||
library(pins) | ||
url_data <- "https://posit.byui.edu/data/`r object_name`/" | ||
board_url <- board_connect_url(c("dat" = url_data)) | ||
dat <- pin_read(board_url, "dat") | ||
``` | ||
|
||
Use this custom function in Python to have the data in a Pandas DataFrame. | ||
|
||
```python | ||
import pandas as pd | ||
import requests | ||
from io import BytesIO | ||
|
||
def read_url_pin(name): | ||
url = "https://posit.byui.edu/data/" + name + "/" + name + ".parquet" | ||
response = requests.get(url) | ||
if response.status_code == 200: | ||
parquet_content = BytesIO(response.content) | ||
pandas_dataframe = pd.read_parquet(parquet_content) | ||
return pandas_dataframe | ||
else: | ||
print(f"Failed to retrieve data. Status code: {response.status_code}") | ||
return None | ||
|
||
# Example usage: | ||
pandas_df = read_url_pin("`r object_name`") | ||
``` | ||
|
||
#### __Authenticated Connection:__ | ||
|
||
Our connect server is [https://posit.byui.edu](https://posit.byui.edu/connect/#/content/listing?filter=min_role:viewer&filter=content_type:all&view_type=compact) which you assign to your `CONNECT_SERVER` environment variable. You must [create an API key](https://docs.posit.co/connect/user/api-keys/#api-keys-creating) and store it in your environment under `CONNECT_API_KEY`. | ||
|
||
_Read more about [environment variables and the pins package](../../environment.qmd) to understand how these environment variables are stored and accessed in R and Python with pins._ | ||
|
||
```r | ||
library(pins) | ||
board <- board_connect(auth = "auto") | ||
dat <- pin_read(board, "`r params$pins_name`") | ||
``` | ||
|
||
|
||
```python | ||
import os | ||
from pins import board_rsconnect | ||
from dotenv import load_dotenv | ||
load_dotenv() | ||
API_KEY = os.getenv('CONNECT_API_KEY') | ||
SERVER = os.getenv('CONNECT_SERVER') | ||
|
||
board = board_rsconnect(server_url=SERVER, api_key=API_KEY) | ||
dat = board.pin_read("`r params$pins_name`") | ||
``` | ||
|
||
[^1]: [`r params$source`](`r params$source`) |