Skip to content
This repository has been archived by the owner on Oct 31, 2019. It is now read-only.

Commit

Permalink
Merge branch 'dev'
Browse files Browse the repository at this point in the history
* dev:
  Bump version number to 0.2.3
  Document deprecated functions. Remove references to deprecated functions.
  Remove azToken from all functions #50 Fix roxygen markdown links #57
  • Loading branch information
andrie committed May 21, 2017
2 parents 3890565 + 4c50cd1 commit b9753ca
Show file tree
Hide file tree
Showing 71 changed files with 543 additions and 608 deletions.
17 changes: 17 additions & 0 deletions AzureSMR.Rproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
Version: 1.0

RestoreWorkspace: Default
SaveWorkspace: Default
AlwaysSaveHistory: Default

EnableCodeIndexing: Yes
UseSpacesForTab: Yes
NumSpacesForTab: 2
Encoding: UTF-8

RnwWeave: Sweave
LaTeX: pdfLaTeX

BuildType: Package
PackageUseDevtools: Yes
PackageInstallArgs: --no-multiarch --with-keep.source
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ Description: Helps users to manage Azure Services and objects from within an
Machines and HDInsight (Spark, Hive). Requires that an Active Directory
application and service principal has been configured to access Azure resources.
Type: Package
Version: 0.2.2
Date: 2016-12-22
Version: 0.2.3
Date: 2017-05-21
Authors@R: c(
person(family="Microsoft Corporation", role="cph"),
person("Alan", "Weaver", role=c("aut", "cre"), email="[email protected]"),
Expand Down
8 changes: 4 additions & 4 deletions R/AzureAuthenticate.R
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
#' @note See \url{https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/} for instructions to set up an Active Directory application
#' @references \url{https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/}
#'
#' @return Retunrs Azure Tokem and sets AzureContext Token
#' @family Resources
#' @return Returns Azure token and sets AzureContext token
#' @family Azure resource functions
#'
#' @importFrom utils URLencode
#' @export
Expand Down Expand Up @@ -65,11 +65,11 @@ azureAuthenticate <- function(azureActiveContext, tenantID, clientID, authKey, v



#' Check the timestamp of a Token and Renew if needed.
#' Check the timestamp of a token and renew if needed.
#'
#' @inheritParams setAzureContext
#' @inheritParams azureAuthenticate
#' @family Resources
#' @family Azure resource functions
#' @export
azureCheckToken <- function(azureActiveContext) {
if (missing(azureActiveContext) || is.null(azureActiveContext)) return(NA)
Expand Down
20 changes: 10 additions & 10 deletions R/AzureBlob.R
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@
#' @inheritParams azureAuthenticate
#' @inheritParams azureSAGetKey
#'
#' @param maxresults Optional. Specifies the maximum number of blobs to return, including all BlobPrefix elements. If the request does not specify maxresults or specifies a value greater than 5,000, the server will return up to 5,000 items. Setting maxresults to a value less than or equal to zero results in error response code 400 (Bad Request).
#' @param maxresults Optional. Specifies the maximum number of blobs to return, including all BlobPrefix elements. If the request does not specify maxresults or specifies a value greater than 5,000, the server will return up to 5,000 items. Setting `maxresults` to a value less than or equal to zero results in error response code 400 (Bad Request).
#' @param prefix Optional. Filters the results to return only blobs whose names begin with the specified prefix.
#' @param delimiter Optional. When the request includes this parameter, the operation returns a BlobPrefix element in the response body that acts as a placeholder for all blobs whose names begin with the same substring up to the appearance of the delimiter character. The delimiter may be a single character or a string.
#' @param marker Optional. A string value that identifies the portion of the list to be returned with the next list operation. The operation returns a marker value within the response body if the list returned was not complete. The marker value may then be used in a subsequent call to request the next set of list items. The marker value is opaque to the client.
#'
#' @return Returns a data frame. This data frame has an attribute called "marker" that can be used with the "marker" argument to return the next set of values.
#' @family blob store functions
#' @return Returns a data frame. This data frame has an attribute called `marker` that can be used with the `marker` argument to return the next set of values.
#' @family Blob store functions
#' @export
azureListStorageBlobs <- function(azureActiveContext, storageAccount, storageKey,
container, resourceGroup, maxresults, prefix, delimiter, marker, verbose = FALSE) {
Expand Down Expand Up @@ -117,7 +117,7 @@ azureListStorageBlobs <- function(azureActiveContext, storageAccount, storageKey
#' @param directory Blob store directory to list for content
#' @param recursive If TRUE, list blob store directories recursively
#'
#' @family blob store functions
#' @family Blob store functions
#' @export
azureBlobLS <- function(azureActiveContext, directory, recursive = FALSE,
storageAccount, storageKey, container, resourceGroup, verbose = FALSE) {
Expand Down Expand Up @@ -275,9 +275,9 @@ azureBlobLS <- function(azureActiveContext, directory, recursive = FALSE,
#' @inheritParams azureSAGetKey
#' @inheritParams azureBlobLS

#' @param type String, either "text" or "raw". Passed to `httr::content`
#' @param type String, either "text" or "raw". Passed to [httr::content()]
#'
#' @family blob store functions
#' @family Blob store functions
#' @export

azureGetBlob <- function(azureActiveContext, blob, directory, type = "text",
Expand Down Expand Up @@ -379,7 +379,7 @@ azureGetBlob <- function(azureActiveContext, blob, directory, type = "text",
#' @param contents - Object or value to store
#' @param file - Local filename to store in Azure blob
#'
#' @family blob store functions
#' @family Blob store functions
#' @export
azurePutBlob <- function(azureActiveContext, blob, contents = "", file = "",
directory, storageAccount, storageKey,
Expand Down Expand Up @@ -487,7 +487,7 @@ azurePutBlob <- function(azureActiveContext, blob, contents = "", file = "",
#' @inheritParams azureSAGetKey
#' @inheritParams azurePutBlob
#'
#' @family blob store functions
#' @family Blob store functions
#' @export
azureBlobFind <- function(azureActiveContext, file, storageAccount, storageKey,
container, resourceGroup, verbose = FALSE) {
Expand Down Expand Up @@ -551,7 +551,7 @@ azureBlobFind <- function(azureActiveContext, file, storageAccount, storageKey,
#' @inheritParams azureBlobLS
#' @inheritParams azurePutBlob
#'
#' @family blob store functions
#' @family Blob store functions
#' @export
azureBlobCD <- function(azureActiveContext, directory, container, file,
storageAccount, storageKey, resourceGroup, verbose = FALSE) {
Expand Down Expand Up @@ -640,7 +640,7 @@ azureBlobCD <- function(azureActiveContext, directory, container, file,
#' @inheritParams azureSAGetKey
#' @inheritParams azureBlobLS
#'
#' @family blob store functions
#' @family Blob store functions
#' @export

azureDeleteBlob <- function(azureActiveContext, blob, directory,
Expand Down
63 changes: 27 additions & 36 deletions R/AzureContainer.R
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,26 @@
#' @inheritParams azureAuthenticate
#' @inheritParams azureSAGetKey

#' @family container functions
#' @family Container functions
#'
#' @export
azureListStorageContainers <- function(azureActiveContext, storageAccount, storageKey,
resourceGroup, azToken, subscriptionID, verbose = FALSE) {
resourceGroup, subscriptionID, verbose = FALSE) {
azureCheckToken(azureActiveContext)

if (missing(subscriptionID)) {
SUBIDI <- azureActiveContext$subscriptionID
} else (SUBIDI <- subscriptionID)
if (missing(azToken)) {
ATI <- azureActiveContext$Token
} else (ATI <- azToken)
subscriptionID <- azureActiveContext$subscriptionID
} else (subscriptionID <- subscriptionID)
if (missing(resourceGroup)) {
RGI <- azureActiveContext$resourceGroup
} else (RGI <- resourceGroup)
resourceGroup <- azureActiveContext$resourceGroup
} else (resourceGroup <- resourceGroup)
if (missing(storageAccount)) {
SAI <- azureActiveContext$storageAccount
} else (SAI <- storageAccount)
verbosity <- if (verbose)
httr::verbose(TRUE) else NULL

if (length(RGI) < 1) {
if (length(resourceGroup) < 1) {
stop("Error: No resourceGroup provided: Use resourceGroup argument or set in AzureContext")
}
if (length(SAI) < 1) {
Expand All @@ -36,7 +33,7 @@ azureListStorageContainers <- function(azureActiveContext, storageAccount, stora
STK <- if (length(azureActiveContext$storageAccountK) < 1 ||
SAI != azureActiveContext$storageAccountK ||
length(azureActiveContext$storageKey) < 1) {
azureSAGetKey(azureActiveContext, resourceGroup = RGI, storageAccount = SAI)
azureSAGetKey(azureActiveContext, resourceGroup = resourceGroup, storageAccount = SAI)
} else {
azureActiveContext$storageKey
}
Expand Down Expand Up @@ -87,7 +84,7 @@ azureListStorageContainers <- function(azureActiveContext, storageAccount, stora
}

azureActiveContext$storageAccount <- SAI
azureActiveContext$resourceGroup <- RGI
azureActiveContext$resourceGroup <- resourceGroup
azureActiveContext$storageKey <- STK

data.frame(
Expand All @@ -109,22 +106,19 @@ azureListStorageContainers <- function(azureActiveContext, storageAccount, stora
#' @inheritParams azureAuthenticate
#' @inheritParams azureSAGetKey

#' @family container functions
#' @family Container functions
#'
#' @export
azureCreateStorageContainer <- function(azureActiveContext, container, storageAccount,
storageKey, resourceGroup, azToken, subscriptionID, verbose = FALSE) {
storageKey, resourceGroup, subscriptionID, verbose = FALSE) {
# azureCheckToken(azureActiveContext)

if (missing(subscriptionID)) {
SUBIDI <- azureActiveContext$subscriptionID
} else (SUBIDI <- subscriptionID)
if (missing(azToken)) {
ATI <- azureActiveContext$Token
} else (ATI <- azToken)
subscriptionID <- azureActiveContext$subscriptionID
} else (subscriptionID <- subscriptionID)
if (missing(resourceGroup)) {
RGI <- azureActiveContext$resourceGroup
} else (RGI <- resourceGroup)
resourceGroup <- azureActiveContext$resourceGroup
} else (resourceGroup <- resourceGroup)
if (missing(storageAccount)) {
SAI <- azureActiveContext$storageAccount
} else (SAI <- storageAccount)
Expand All @@ -137,14 +131,14 @@ azureCreateStorageContainer <- function(azureActiveContext, container, storageAc
verbosity <- if (verbose)
httr::verbose(TRUE) else NULL

if (length(RGI) < 1) {
if (length(resourceGroup) < 1) {
stop("Error: No resourceGroup provided: Use resourceGroup argument or set in AzureContext")
}
if (length(SAI) < 1) {
stop("Error: No storageAccount provided: Use storageAccount argument or set in AzureContext")
}

STK <- refreshStorageKey(azureActiveContext, SAI, RGI)
STK <- refreshStorageKey(azureActiveContext, SAI, resourceGroup)

if (length(STK) < 1) {
stop("Error: No storageKey provided: Use storageKey argument or set in AzureContext")
Expand All @@ -165,7 +159,7 @@ azureCreateStorageContainer <- function(azureActiveContext, container, storageAc

azureActiveContext$container <- container
azureActiveContext$storageAccount <- SAI
azureActiveContext$resourceGroup <- RGI
azureActiveContext$resourceGroup <- resourceGroup

URL <- paste("http://", SAI, ".blob.core.windows.net/", container,
"?restype=container", sep = "")
Expand Down Expand Up @@ -203,22 +197,19 @@ azureCreateStorageContainer <- function(azureActiveContext, container, storageAc
#' @inheritParams azureAuthenticate
#' @inheritParams azureSAGetKey

#' @family container functions
#' @family Container functions
#'
#' @export
azureDeleteStorageContainer <- function(azureActiveContext, container, storageAccount,
storageKey, resourceGroup, azToken, subscriptionID, verbose = FALSE) {
storageKey, resourceGroup, subscriptionID, verbose = FALSE) {
azureCheckToken(azureActiveContext)

if (missing(subscriptionID)) {
SUBIDI <- azureActiveContext$subscriptionID
} else (SUBIDI <- subscriptionID)
if (missing(azToken)) {
ATI <- azureActiveContext$Token
} else (ATI <- azToken)
subscriptionID <- azureActiveContext$subscriptionID
} else (subscriptionID <- subscriptionID)
if (missing(resourceGroup)) {
RGI <- azureActiveContext$resourceGroup
} else (RGI <- resourceGroup)
resourceGroup <- azureActiveContext$resourceGroup
} else (resourceGroup <- resourceGroup)
if (missing(storageAccount)) {
SAI <- azureActiveContext$storageAccount
} else (SAI <- storageAccount)
Expand All @@ -233,14 +224,14 @@ azureDeleteStorageContainer <- function(azureActiveContext, container, storageAc

CNTR <- container

if (length(RGI) < 1) {
if (length(resourceGroup) < 1) {
stop("Error: No resourceGroup provided: Use resourceGroup argument or set in AzureContext")
}
if (length(SAI) < 1) {
stop("Error: No storageAccount provided: Use storageAccount argument or set in AzureContext")
}

STK <- refreshStorageKey(azureActiveContext, SAI, RGI)
STK <- refreshStorageKey(azureActiveContext, SAI, resourceGroup)
if (length(STK) < 1) {
stop("Error: No storageKey provided: Use storageKey argument or set in AzureContext")
}
Expand All @@ -258,7 +249,7 @@ azureDeleteStorageContainer <- function(azureActiveContext, container, storageAc

azureActiveContext$container <- CNTR
azureActiveContext$storageAccount <- SAI
azureActiveContext$resourceGroup <- RGI
azureActiveContext$resourceGroup <- resourceGroup
SIG <- getSig(azureActiveContext, url = URL, verb = "DELETE", key = STK,
storageAccount = SAI,
CMD = paste0(CNTR, "\nrestype:container"), dateSig = D1)
Expand Down
41 changes: 16 additions & 25 deletions R/AzureContextObject.R
Original file line number Diff line number Diff line change
@@ -1,66 +1,57 @@
#' Create an AzureContext.
#'
#' Create a container (azureContextObject) for holding variables used by the AzureSMR package. If the Tenant ID, Client ID and Authenication Key are provided the function will attempt to authenticate the session.
#' Create a container (azureContextObject) for holding variables used by the `AzureSMR` package. If the Tenant ID, Client ID and Authenication Key are provided the function will attempt to authenticate the session.
#'
#' See the Azure documentation (\url{https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/}) for information to configure an Active Directory application.
#'
#' @inheritParams setAzureContext
#' @family azureActiveContext functions
#'
#' @seealso `setAzureContext`
#' @seealso [setAzureContext()]
#' @export
createAzureContext <- function(tenantID, clientID, authKey){
azEnv <- new.env(parent = globalenv())
azEnv <- as.azureActiveContext(azEnv)

if (!missing(tenantID)) azEnv$tenantID <- tenantID else azEnv$tenantID <- "?"
if (!missing(clientID)) azEnv$clientID <- clientID else azEnv$tenantID <- "?"
if (!missing(authKey)) azEnv$authKey <- authKey else azEnv$tenantID <- "?"
if (!missing(tenantID)) azEnv$tenantID <- tenantID else azEnv$tenantID <- "?"
if (!missing(clientID)) azEnv$clientID <- clientID else azEnv$tenantID <- "?"
if (!missing(authKey)) azEnv$authKey <- authKey else azEnv$tenantID <- "?"

if (!missing(tenantID) && !missing(clientID) && !missing(authKey) )
azureAuthenticate(azEnv,tenantID, clientID, authKey)
return(azEnv)
}

#' Dumps out the contents of the AzureContext.
#'
#' @inheritParams setAzureContext
#' @export
dumpAzureContext <- function(azureActiveContext){
.Deprecated("str")
str(azureActiveContext)
}

#' Updates azureActiveContext object.
#'
#' Updates the value of an azureActiveContext object, created by `createAzureContext`
#' Updates the value of an azureActiveContext object, created by [createAzureContext()]
#'
#' @param azureActiveContext A container used for caching variables used by AzureSMR
#' @param tenantID The Tenant ID provided during creation of the Active Directory application / service principal
#' @param clientID The Client ID provided during creation of the Active Directory application / service principal
#' @param authKey The Authentication Key provided during creation of the Active Directory application / service principal
#' @param subscriptionID Set the subscriptionID. This is obtained automatically by `azureAuthenticate` when only a single subscriptionID is available via Active Directory
#' @param azToken Azure authentication token, obtained by `azureAuthenticate`
#' @param subscriptionID Set the subscriptionID. This is obtained automatically by [azureAuthenticate()] when only a single subscriptionID is available via Active Directory
#' @param azToken Azure authentication token, obtained by [azureAuthenticate()]
#' @param resourceGroup Name of the resource group
#' @param vmName Name of the virtual Machine
#' @param storageAccount Name of the azure storage account
#' @param storageKey Storage key associated with storage account
#' @param blob Blob name
#' @param clustername Cluster name, used for HDI and Spark clusters. See `azureCreateHDI`
#' @param sessionID Spark sessionID. See `azureSparkCMD`
#' @param clustername Cluster name, used for HDI and Spark clusters. See [azureCreateHDI()]
#' @param sessionID Spark sessionID. See [azureSparkCMD()]
#' @param hdiAdmin HDInsight admin username
#' @param hdiPassword HDInsight admin password
#' @param container Storage container name. See `azureListStorageContainers`
#' @param container Storage container name. See [azureListStorageContainers()]
#' @param kind HDinsight kind: "hadoop","spark" or "pyspark"
#'
#' @family azureActiveContext functions
#' @export
setAzureContext <- function(azureActiveContext,tenantID, clientID, authKey,azToken,
subscriptionID,resourceGroup,
storageKey,storageAccount,
container,blob,
setAzureContext <- function(azureActiveContext, tenantID, clientID, authKey, azToken,
subscriptionID, resourceGroup,
storageKey, storageAccount,
container, blob,
vmName,
hdiAdmin,hdiPassword ,clustername,kind,sessionID)
hdiAdmin, hdiPassword, clustername, kind, sessionID)
{
if (!missing(tenantID)) azureActiveContext$tenantID <- tenantID
if (!missing(clientID)) azureActiveContext$clientID <- clientID
Expand Down
Loading

0 comments on commit b9753ca

Please sign in to comment.