Skip to content

Commit

Permalink
Merge pull request #44 from sugarcrm/get-builds-from-store
Browse files Browse the repository at this point in the history
Get builds from store
  • Loading branch information
mmarum-sugarcrm authored Apr 10, 2018
2 parents dd5f173 + 15beed6 commit 8cce01c
Show file tree
Hide file tree
Showing 5 changed files with 146 additions and 74 deletions.
6 changes: 3 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
- cd ../scripts
- sudo service mysql stop
script:
- ./SetupEnvAndRunPHPUnitTests.sh $DEV_COMMUNITY_USERNAME $DEV_COMMUNITY_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD workspace/sugardocker
- ./SetupEnvAndRunPHPUnitTests.sh $SUGARCRM_USERNAME $SUGARCRM_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD workspace/sugardocker
- stage: Run PHPUnit
sudo: required
language: php
Expand All @@ -47,7 +47,7 @@ jobs:
- cd ../scripts
- sudo service mysql stop
script:
- ./SetupEnvAndRunPHPUnitTests.sh $DEV_COMMUNITY_USERNAME $DEV_COMMUNITY_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD workspace/sugardocker
- ./SetupEnvAndRunPHPUnitTests.sh $SUGARCRM_USERNAME $SUGARCRM_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD workspace/sugardocker
- stage: Run PHPUnit
sudo: required
language: php
Expand All @@ -63,7 +63,7 @@ jobs:
- cd ../scripts
- sudo service mysql stop
script:
- ./SetupEnvAndRunPHPUnitTests.sh $DEV_COMMUNITY_USERNAME $DEV_COMMUNITY_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD workspace/sugardocker
- ./SetupEnvAndRunPHPUnitTests.sh $SUGARCRM_USERNAME $SUGARCRM_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD workspace/sugardocker
- stage: Build & Post on GitHub
language: php
php: '7.1'
Expand Down
32 changes: 17 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,9 +197,10 @@ You can then click Details to open the build results in Travis CI.
### About the build

The build has four Environment Variables that have been configured in the project settings in Travis CI:
- DEV_COMMUNITY_USERNAME: The username for an account that has access to the
[SugarCRM Developer Builds Space](https://community.sugarcrm.com/community/developer/developer-builds)
- DEV_COMMUNITY_PASSWORD: The password associated with the above account
- SUGARCRM_USERNAME: The username for an account that has access to the
[SugarCRM Developer Builds Space](https://community.sugarcrm.com/community/developer/developer-builds) and the [Sugar
Store](https://store.sugarcrm.com/download)
- SUGARCRM_PASSWORD: The password associated with the above account
- GITHUB_USERNAME: The username for a GitHub account that has access to https://github.com/sugarcrm/unit-tests
- GITHUB_PASSWORD: The password associated with the above account

Expand Down Expand Up @@ -287,13 +288,14 @@ Credentials Binding Plugin.)
1. ID: GITHUB_SUGARCRM_UNIT_TESTS
1. Description: GitHub account that has access to https://github.com/sugarcrm/unit-tests
1. Create new global credentials for the account that has access to the
[SugarCRM Developer Builds Space](https://community.sugarcrm.com/community/developer/developer-builds).
[SugarCRM Developer Builds Space](https://community.sugarcrm.com/community/developer/developer-builds) and the [Sugar
Store](https://store.sugarcrm.com/download).
1. Kind: **Username with password**
1. Scope: **Global**
1. Username: your SugarCRM Developer Builds Space username
1. Password: your SugarCRM Developer Builds Space password
1. ID: SUGARCRM_DEVELOPER_BUILDS_SPACE
1. Description: Account that has access to the SugarCRM Developer Builds Space
1. Username: your SugarCRM username
1. Password: your SugarCRM password
1. ID: SUGARCRM_ACCOUNT
1. Description: SugarCRM Account
1. Create a new global secret for the path to where Sugar Docker will be stored on your host machine (not the path on
Jenkins).
1. Kind: **Secret text**
Expand Down Expand Up @@ -327,10 +329,10 @@ text(s) or file(s)** option.
1. Credentials: **Specific credentials**
1. Select the GITHUB_SUGARCRM_UNIT_TESTS credentials.
1. In the **Bindings** section, select **Add** > **Username and password (separated)**. Then input the following:
1. Username Variable: `DEV_COMMUNITY_USERNAME`
1. Password Variable: `DEV_COMMUNITY_PASSWORD`
1. Username Variable: `SUGARCRM_USERNAME`
1. Password Variable: `SUGARCRM_PASSWORD`
1. Credentials: **Specific credentials**
1. Select the SUGARCRM_DEVELOPER_BUILDS_SPACE credentials.
1. Select the SUGARCRM_ACCOUNT credentials.
1. In the **Bindings** section, select **Add** > **Secret text**. Then input the following:
1. Variable: `PATH_TO_SUGAR_DOCKER_ON_HOST`
1. Credentials: **Specific credentials**
Expand All @@ -351,16 +353,16 @@ text(s) or file(s)** option.
SUGAR_DOCKER_DIRECTORY="workspace/sugardocker"
# This variable is completely optional. If you want to store the Sugar source zips on your local machine instead
# of downloading them from the SugarCRM Developer Builds space, input the path to where the Sugar source zips are
# stored. For example: /var/sugardocker/data/app/sugar_source_zips. Note that your Sugar source zips MUST follow
# this naming convention: Sugar$sugarEdition-$sugarVersion.zip (for example: SugarEnt-7.11.zip).
# of downloading them from the SugarCRM Developer Builds space or Sugar Store, input the path to where the Sugar
# source zips are stored. For example: /var/sugardocker/data/app/sugar_source_zips. Note that your Sugar source zips
# MUST follow this naming convention: Sugar$sugarEdition-$sugarVersion.zip (for example: SugarEnt-7.11.zip).
# SUGAR_SOURCE_ZIPS_DIRECTORY=""
cd scripts
bash -ex RunPackUnitTestsAndBuildProfMPackage.sh $SUGAR_WORKSPACE_PATH
bash SetupEnvAndRunPHPUnitTests.sh $DEV_COMMUNITY_USERNAME $DEV_COMMUNITY_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD $SUGAR_DOCKER_DIRECTORY $SUGAR_SOURCE_ZIPS_DIRECTORY
bash SetupEnvAndRunPHPUnitTests.sh $SUGARCRM_USERNAME $SUGARCRM_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD $SUGAR_DOCKER_DIRECTORY $SUGAR_SOURCE_ZIPS_DIRECTORY
```
Be sure to update the variables appropriately.
1. In the **Post-build Actions** section, click **Add post-build action** and select **Archive the artifacts**.
Expand Down
Binary file modified images/travisenvvars.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
176 changes: 123 additions & 53 deletions scripts/GetCopyOfSugar.sh
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
#!/usr/bin/env bash

# This script gets a copy of Sugar from the designated directory or downloads a Sugar dev build from the SugarCRM
# Developer Builds community.
# This script gets a copy of Sugar from the designated directory or downloads a Sugar dev build from the Sugar Store
# or the SugarCRM Developer Builds community.
#
# Note: you must have access to the SugarCRM Developer Builds community
# (https://community.sugarcrm.com/community/developer/developer-builds) in order for the download to be successful.
# Note: you must have access to the Sugar Store (https://store.sugarcrm.com/download) and/or the SugarCRM Developer
# Builds community (https://community.sugarcrm.com/community/developer/developer-builds) depending on where the build is
# stored in order for the download to be successful.


######################################################################
Expand All @@ -14,7 +15,7 @@
if [[ -z "$1" ]] || [[ -z "$2" ]] || [[ -z "$3" ]] || [[ -z "$4" ]]
then
echo "Not all required command line arguments were set. Please run the script again with the required arguments:
1: Email address associated with your SugarCRM Developer Builds Community account
1: Email address associated with your SugarCRM account
2: Password associated with the above account
3: Sugar name (For example: SugarEnt-7.11)
4. The path to where the Sugar download should be stored
Expand All @@ -26,10 +27,10 @@ then
exit 1
fi

# Email address associated with your SugarCRM developer community account
# Email address associated with your SugarCRM account
email=$1

# Password associated with your SugarCRM developer community account
# Password associated with your SugarCRM account
password=$2

# The Sugar name (For example: SugarEnt-7.11)
Expand Down Expand Up @@ -60,7 +61,7 @@ checkStatusCode(){
statusCode="${BASH_REMATCH[1]}"
if [[ "$statusCode" == $1 ]]
then
echo "Status code is correct: $statusCode"
return
else
echo "Status code is not the expected $1: $statusCode"
echo "$2"
Expand All @@ -87,6 +88,42 @@ getLocationFromResponse(){
fi
}

# Print the value associated with a given key for a JSON response
# $1: The key associated with the JSON value you want to parse
# $2: response from curl command
getJsonValueFromResponse(){
regexJsonValue="$1\":\"([^\"]*)\""

if [[ $2 =~ $regexJsonValue ]]
then
value="${BASH_REMATCH[1]}"
echo "$value"
else
echo "Unable to find the value of $1 in response"
echo $2
exit 1
fi
}

# Print the Sugar Download ID associated with the given Sugar zip file
# $1: The name of the Sugar zip file to search for in the response
# $2: response from curl command
getSugarDownloadIdFromResponse(){
# This regex parses a string similar to
# "id":"download-id-we-are-trying-to-get","name":"SugarEnt-7.9.3.0.zip"
regexJasonValue="\"id\":\"([^\"]*)\",\"name\":\"$1\""

if [[ $2 =~ $regexJasonValue ]]
then
value="${BASH_REMATCH[1]}"
echo "$value"
else
echo "Unable to find the value of $1 in response"
echo $2
exit 1
fi
}

# Print the value associated with a hidden form field
# $1: name of the hidden form field
# $2: response from curl command
Expand Down Expand Up @@ -116,12 +153,47 @@ getHiddenFormFieldValue(){
echo "$2"
exit 1
fi
}

# Authenticate to the Sugar Store and print the URL to download the given Sugar zip
# $1: The name of zip to download (for example: SugarEnt-7.9.3.0.zip)
function authenticateToSugarStoreAndGetDownloadUrl(){

response="$(curl -v -L -c $cookieFile -b $cookieFile 'https://store.sugarcrm.com/download' 2>&1)"
checkStatusCode "200" "$response"
token="$(getHiddenFormFieldValue "_token" "$response")"

response="$(curl -v -L -c $cookieFile -b $cookieFile --data "_token=$token&email=$email&password=$password" https://auth.sugarcrm.com/auth/login 2>&1)"
checkStatusCode "200" "$response"
accountId="$(getJsonValueFromResponse "id" "$response")"

response="$(curl -v -L -c $cookieFile -b $cookieFile "https://store.sugarcrm.com/api/v1/accounts/$accountId/downloads" 2>&1)"
checkStatusCode "200" "$response"
downloadId="$(getSugarDownloadIdFromResponse $1 "$response")"
hash="$(getJsonValueFromResponse "hash" "$response")"

downloadUrl="https://store.sugarcrm.com/download/$downloadId/$hash"
echo $downloadUrl
}

# Authenticate to the Developer Builds Community
function authenticateToDevBuildsCommunity(){

response="$(curl -v -L -c $cookieFile -b $cookieFile 'https://community.sugarcrm.com/login.jspa?ssologin=true&fragment=&referer=%2Fcommunity%2Fdeveloper%2Fdeveloper-builds' 2>&1)"
checkStatusCode "200" "$response"
token="$(getHiddenFormFieldValue "_token" "$response")"

response="$(curl -v -L -c $cookieFile -b $cookieFile --data "_token=$token&email=$email&password=$password" https://auth.sugarcrm.com/saml2/idp/authpage?ReturnTo=https%3A%2F%2Fauth.sugarcrm.com%2Fsaml2%2Fidp%2FSSOService%3Fspentityid%3Dhttps%253A%252F%252Fcommunity.sugarcrm.com%26RelayState%3DL2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw%253D%253D 2>&1)"
checkStatusCode "200" "$response"
samlResponse="$(getHiddenFormFieldValue "SAMLResponse" "$response")"

response="$(curl -v -L -c $cookieFile -b $cookieFile --data-urlencode "SAMLResponse=$samlResponse" --data-urlencode "RelayState=L2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw==" 'https://community.sugarcrm.com/saml/sso' 2>&1)"
checkStatusCode "200" "$response"
}


######################################################################
# Check if we need to download the Sugar source zip
# Check if we have a copy of the Sugar source zip already downloaded
######################################################################

# If we already have a copy of the Sugar source zip, we'll copy it to the Sugar directory and exit the script
Expand Down Expand Up @@ -149,81 +221,79 @@ chmod -R 777 .
sudo chmod -R 777 .


######################################################################
# Authenticate to the community
######################################################################

echo "Authenticating to Developer Builds Community..."

response="$(curl -v -c $cookieFile -b $cookieFile 'https://community.sugarcrm.com/login.jspa?ssologin=true&fragment=&referer=%2Fcommunity%2Fdeveloper%2Fdeveloper-builds' 2>&1)"
checkStatusCode "302" "$response"
location="$(getLocationFromResponse "$response")"

# Location should be something like https://auth.sugarcrm.com/saml2/idp/SSOService?SAMLRequest=fZHNbsIwEIRfxdp7EjuUn1okiBahIlGBSOihNxMMGCV26nVQ%2B%2FZ1IahUlTh6vbPf7sxw9FmV5CQtKqMTYCEFInVhtkrvE1jn02AAo3SIoirjmo8bd9Ar%2BdFIdMQLNfLLTwKN1dwIVMi1qCRyV%2FBs%2FDrncUh5bY0zhSmBjBGldR71bDQ2lbSZtCdVyPVqnsDBuRp5FBWmqhqt3FeIzV7YwlahL0U%2FpAjRAJl4vNLCnVe%2BqoTf7b8gjtS2jrJs0YKATI0t5PmQBHaiRF%2BaTRIQjPXo8aAGne6OdukDUxsmYnHsql7M2KNvwqVAVCf5K0Ns5EyjE9olEFM2CGgcsDhnfU47nPZD1um%2FA1m25z8pfbH1nlebSxPylzxfBstFlgN5u8bjG6ANg5%2Fp9jaF%2B4PF1XpI7xs9jG4Bafv8G376DQ%3D%3D&RelayState=L2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw%3D%3D
response="$(curl -v -c $cookieFile -b $cookieFile $location 2>&1)"
checkStatusCode "302" "$response"
location="$(getLocationFromResponse "$response")"

# Location should be something like https://auth.sugarcrm.com/saml2/idp/authpage?ReturnTo=https%3A%2F%2Fauth.sugarcrm.com%2Fsaml2%2Fidp%2FSSOService%3Fspentityid%3Dhttps%253A%252F%252Fcommunity.sugarcrm.com%26RelayState%3DL2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw%253D%253D
response="$(curl -v -c $cookieFile -b $cookieFile $location 2>&1)"
checkStatusCode "200" "$response"
token="$(getHiddenFormFieldValue "_token" "$response")"

response="$(curl -v -c $cookieFile -b $cookieFile --data "_token=$token&email=$email&password=$password" https://auth.sugarcrm.com/saml2/idp/authpage?ReturnTo=https%3A%2F%2Fauth.sugarcrm.com%2Fsaml2%2Fidp%2FSSOService%3Fspentityid%3Dhttps%253A%252F%252Fcommunity.sugarcrm.com%26RelayState%3DL2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw%253D%253D 2>&1)"
checkStatusCode "302" "$response"
location="$(getLocationFromResponse "$response")"

# Location should be something like 'https://auth.sugarcrm.com/saml2/idp/SSOService?spentityid=https%3A%2F%2Fcommunity.sugarcrm.com&RelayState=L2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw%3D%3D'
response="$(curl -v -c $cookieFile -b $cookieFile $location 2>&1)"
checkStatusCode "200" "$response"
samlResponse="$(getHiddenFormFieldValue "SAMLResponse" "$response")"

response="$(curl -v -c $cookieFile -b $cookieFile --data-urlencode "SAMLResponse=$samlResponse" --data-urlencode "RelayState=L2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw==" 'https://community.sugarcrm.com/saml/sso' 2>&1)"
checkStatusCode "302" "$response"


#######################################################################
## Download the Sugar zip
# Get the URL to download and authenticate to the appropriate location
#######################################################################

sugarVersion_7_10="7.10"
sugarVersion_7_11="7.11"
sugarVersion_8_0="8.0"

sugarEdition_Ult="Ult"
sugarEdition_Ent="Ent"
sugarEdition_Pro="Pro"

# Get the url for the appropriate Sugar version and edition
# Get the url for the appropriate Sugar version and edition as well as
# authenticate to the appropriate location (Sugar Store or Developer Builds Community)
if [[ "$sugarName" == "Sugar$sugarEdition_Ult-$sugarVersion_7_10" ]]
then downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5839-102-1-8005/SugarUlt-7.10.2.0-dev.1.zip"
then
authenticateToDevBuildsCommunity
downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5839-102-1-8005/SugarUlt-7.10.2.0-dev.1.zip"

elif [[ "$sugarName" == "Sugar$sugarEdition_Ent-$sugarVersion_7_10" ]]
then downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5837-102-1-8003/SugarEnt-7.10.2.0-dev.1.zip"
then
authenticateToDevBuildsCommunity
downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5837-102-1-8003/SugarEnt-7.10.2.0-dev.1.zip"

elif [[ "$sugarName" == "Sugar$sugarEdition_Pro-$sugarVersion_7_10" ]]
then downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5838-102-1-8004/SugarPro-7.10.2.0-dev.1.zip"
then
authenticateToDevBuildsCommunity
downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5838-102-1-8004/SugarPro-7.10.2.0-dev.1.zip"

elif [[ "$sugarName" == "Sugar$sugarEdition_Ult-$sugarVersion_7_11" ]]
then downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5958-102-1-8147/SugarUlt-7.11.0.0-dev.1.zip"
then
authenticateToDevBuildsCommunity
downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5958-102-1-8147/SugarUlt-7.11.0.0-dev.1.zip"

elif [[ "$sugarName" == "Sugar$sugarEdition_Ent-$sugarVersion_7_11" ]]
then downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5959-102-1-8148/SugarEnt-7.11.0.0-dev.1.zip"
then
authenticateToDevBuildsCommunity
downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5959-102-1-8148/SugarEnt-7.11.0.0-dev.1.zip"

elif [[ "$sugarName" == "Sugar$sugarEdition_Pro-$sugarVersion_7_11" ]]
then downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5957-102-1-8146/SugarPro-7.11.0.0-dev.1.zip"
then
authenticateToDevBuildsCommunity
downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5957-102-1-8146/SugarPro-7.11.0.0-dev.1.zip"

elif [[ "$sugarName" == "Sugar$sugarEdition_Ult-$sugarVersion_8_0" ]]
then
downloadUrl="$(authenticateToSugarStoreAndGetDownloadUrl "SugarUlt-8.0.0.0.zip")"

elif [[ "$sugarName" == "Sugar$sugarEdition_Ent-$sugarVersion_8_0" ]]
then
downloadUrl="$(authenticateToSugarStoreAndGetDownloadUrl "SugarEnt-8.0.0.0.zip")"

elif [[ "$sugarName" == "Sugar$sugarEdition_Pro-$sugarVersion_8_0" ]]
then
downloadUrl="$(authenticateToSugarStoreAndGetDownloadUrl "SugarPro-8.0.0.0.zip")"

else
echo "Unable to find Sugar download URL for $sugarName"
exit 1
fi

# Download the file

######################################################################
# Download Sugar
######################################################################

echo "Beginning download of $sugarName from $downloadUrl"
response="$(curl -v -c ./mycookie -b ./mycookie -o $sugarName.zip $downloadUrl 2>&1)"
response="$(curl -v -L -c ./mycookie -b ./mycookie -o $sugarName.zip $downloadUrl 2>&1)"
checkStatusCode "200" "$response"
echo "Download complete"

# Check we didn't get an empty zip file
# Verify we didn't get an empty zip file. If the downloadUrl is invalid, we sometimes get a zip file of around 210
# bytes. We've selected 60000000 to ensure we have a sufficiently large file that is likely correct.
fileSize=$(wc -c <"$sugarName.zip")
if [[ $fileSize -lt 60000000 ]]
then
Expand Down
6 changes: 3 additions & 3 deletions scripts/SetupEnvAndRunPHPUnitTests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
if [[ -z "$1" ]] || [[ -z "$2" ]] || [[ -z "$3" ]] || [[ -z "$4" ]] || [[ -z "$5" ]] || [[ -z "$6" ]] || [[ -z "$7" ]]
then
echo "Not all required command line arguments were set. Please run the script again with the required arguments:
1: Email address associated with your SugarCRM Developer Builds Community account
1: Email address associated with your SugarCRM account
2: Password associated with the above account
3: Sugar version (Example: 7.11)
4: Sugar edition (Options: Ult, Ent, Pro)
Expand All @@ -24,10 +24,10 @@ then
exit 1
fi

# Email address associated with your SugarCRM developer community account
# Email address associated with your SugarCRM account
email=$1

# Password associated with your SugarCRM developer community account
# Password associated with your SugarCRM account
password=$2

# The Sugar version to download
Expand Down

0 comments on commit 8cce01c

Please sign in to comment.