Skip to content

Commit

Permalink
done, with unknown source
Browse files Browse the repository at this point in the history
  • Loading branch information
dustydells committed May 8, 2024
1 parent 5ecaef7 commit 72a2397
Show file tree
Hide file tree
Showing 2 changed files with 192 additions and 0 deletions.
19 changes: 19 additions & 0 deletions data_scripts/car_wash/car_wash.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
pacman::p_load(tidyverse, pins, connectapi)

# Download the file from google drive
sdrive <- shared_drive_find("byuids_data") # This will ask for authentication.
google_file <- drive_ls(sdrive) |>
filter(stringr::str_detect(name, "carwash"))
tempf <- tempfile()
drive_download(google_file, tempf)
car_wash <- read_csv(tempf)

# Publish the data to the server with Bro. Hathaway as the owner.
board <- board_connect()
pin_write(board, car_wash, type = "parquet", access_type = "all")

pin_name <- "car_wash"
meta <- pin_meta(board, paste0("hathawayj/", pin_name))
client <- connect()
my_app <- content_item(client, meta$local$content_id)
set_vanity_url(my_app, paste0("data/", pin_name))
173 changes: 173 additions & 0 deletions dev_posts/car_wash/car_wash.qmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
---
title: "Carwash"
author: "DS 350"
date: "2024-05-08"
categories: [businesses]
description: "Carwash sales over time"
params:
source: ""
pins_name: 'hathawayj/car_wash'
available: all
---


```{r}
#| echo: false
title <- params$title
description <- params$description
source <- params$source
pins_url <- params$pins_url
```

# `r params$title`

`r params$description`

```{python}
#| tags: [parameters]
#| echo: false
title = r.title
description = r.description
source = r.source
pins_url = r.pins_url
```


## Data details

```{r}
#| echo: false
#| warning: false
# https://aeturrell.github.io/skimpy/ python skimr for Python
library(pins)
library(tidyverse)
library(skimr)
library(arrow)
board <- board_connect()
dat <- pin_read(board, params$pins_name)
meta <- pin_meta(board, params$pins_name)
url_path <- meta$local$url
file_name <- meta$file
object_name <- rev(unlist(str_split(meta$name, "/")))[[1]]
url_download <- paste0(
str_remove(
url_path,
paste0(
rev(unlist(str_split(url_path, "/")))[2],
"/")),
file_name)
```

There are __`r prettyNum(nrow(dat), big.mark = ",", scientific = FALSE)` rows__ and __`r ncol(dat)` columns__. The data source[^1] is used to create our data that is stored in our [pins table](`r meta$local$url`). You can access this pin from a connection to [posit.byui.edu](https://posit.byui.edu) using ``r params$pins_name``.

This data is available to __`r params$available`__.

### Variable description

- __name__ Carwash name
- __type__ Carwash type
- __time__ (YYYY-MM-DD HR:MIN:SEC)
- __amount__ Sales (dollars)

### Variable summary

```{r}
#| echo: false
#| warning: false
skim(dat) |> yank("numeric")
```


```{r}
#| echo: false
#| warning: false
skim(dat) |> yank("character")
```

<!-- ```{r}
#| echo: true
#| eval: false
#| code-fold: true
#| code-summary: Explore generating code using R
#| code-line-numbers: true
#| file: ../../../data_scripts/YOURFILEPATH
``` -->

```{python}
#| echo: true
#| eval: false
#| code-fold: true
#| code-summary: Explore generating code using Python
#| code-line-numbers: true
#| file: ../../../data_scripts/car_wash.R
```


## Access data

This data is available to __`r params$available`__.

__Direct Download:__ [`r file_name`](`r url_download`)

#### __R and Python Download:__

##### __URL Connections:__

For public data, any user can connect and read the data using `pins::board_connect_url()` in R.

```r
library(pins)
url_data <- "https://posit.byui.edu/data/`r object_name`/"
board_url <- board_connect_url(c("dat" = url_data))
dat <- pin_read(board_url, "dat")
```

Use this custom function in Python to have the data in a Pandas DataFrame.

```python
import pandas as pd
import requests
from io import BytesIO

def read_url_pin(name):
url = "https://posit.byui.edu/data/" + name + "/" + name + ".parquet"
response = requests.get(url)
if response.status_code == 200:
parquet_content = BytesIO(response.content)
pandas_dataframe = pd.read_parquet(parquet_content)
return pandas_dataframe
else:
print(f"Failed to retrieve data. Status code: {response.status_code}")
return None

# Example usage:
pandas_df = read_url_pin("`r object_name`")
```

#### __Authenticated Connection:__

Our connect server is [https://posit.byui.edu](https://posit.byui.edu/connect/#/content/listing?filter=min_role:viewer&filter=content_type:all&view_type=compact) which you assign to your `CONNECT_SERVER` environment variable. You must [create an API key](https://docs.posit.co/connect/user/api-keys/#api-keys-creating) and store it in your environment under `CONNECT_API_KEY`.

_Read more about [environment variables and the pins package](../../environment.qmd) to understand how these environment variables are stored and accessed in R and Python with pins._

```r
library(pins)
board <- board_connect(auth = "auto")
dat <- pin_read(board, "`r params$pins_name`")
```


```python
import os
from pins import board_rsconnect
from dotenv import load_dotenv
load_dotenv()
API_KEY = os.getenv('CONNECT_API_KEY')
SERVER = os.getenv('CONNECT_SERVER')

board = board_rsconnect(server_url=SERVER, api_key=API_KEY)
dat = board.pin_read("`r params$pins_name`")
```

[^1]: [`r params$source`](`r params$source`)

0 comments on commit 72a2397

Please sign in to comment.