Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get builds from store #44

Merged
merged 6 commits into from
Apr 10, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
- cd ../scripts
- sudo service mysql stop
script:
- ./SetupEnvAndRunPHPUnitTests.sh $DEV_COMMUNITY_USERNAME $DEV_COMMUNITY_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD workspace/sugardocker
- ./SetupEnvAndRunPHPUnitTests.sh $SUGARCRM_USERNAME $SUGARCRM_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD workspace/sugardocker
- stage: Run PHPUnit
sudo: required
language: php
Expand All @@ -47,7 +47,7 @@ jobs:
- cd ../scripts
- sudo service mysql stop
script:
- ./SetupEnvAndRunPHPUnitTests.sh $DEV_COMMUNITY_USERNAME $DEV_COMMUNITY_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD workspace/sugardocker
- ./SetupEnvAndRunPHPUnitTests.sh $SUGARCRM_USERNAME $SUGARCRM_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD workspace/sugardocker
- stage: Run PHPUnit
sudo: required
language: php
Expand All @@ -63,7 +63,7 @@ jobs:
- cd ../scripts
- sudo service mysql stop
script:
- ./SetupEnvAndRunPHPUnitTests.sh $DEV_COMMUNITY_USERNAME $DEV_COMMUNITY_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD workspace/sugardocker
- ./SetupEnvAndRunPHPUnitTests.sh $SUGARCRM_USERNAME $SUGARCRM_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD workspace/sugardocker
- stage: Build & Post on GitHub
language: php
php: '7.1'
Expand Down
32 changes: 17 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,9 +197,10 @@ You can then click Details to open the build results in Travis CI.
### About the build

The build has four Environment Variables that have been configured in the project settings in Travis CI:
- DEV_COMMUNITY_USERNAME: The username for an account that has access to the
[SugarCRM Developer Builds Space](https://community.sugarcrm.com/community/developer/developer-builds)
- DEV_COMMUNITY_PASSWORD: The password associated with the above account
- SUGARCRM_USERNAME: The username for an account that has access to the
[SugarCRM Developer Builds Space](https://community.sugarcrm.com/community/developer/developer-builds) and the [Sugar
Store](https://store.sugarcrm.com/download)
- SUGARCRM_PASSWORD: The password associated with the above account
- GITHUB_USERNAME: The username for a GitHub account that has access to https://github.com/sugarcrm/unit-tests
- GITHUB_PASSWORD: The password associated with the above account

Expand Down Expand Up @@ -287,13 +288,14 @@ Credentials Binding Plugin.)
1. ID: GITHUB_SUGARCRM_UNIT_TESTS
1. Description: GitHub account that has access to https://github.com/sugarcrm/unit-tests
1. Create new global credentials for the account that has access to the
[SugarCRM Developer Builds Space](https://community.sugarcrm.com/community/developer/developer-builds).
[SugarCRM Developer Builds Space](https://community.sugarcrm.com/community/developer/developer-builds) and the [Sugar
Store](https://store.sugarcrm.com/download).
1. Kind: **Username with password**
1. Scope: **Global**
1. Username: your SugarCRM Developer Builds Space username
1. Password: your SugarCRM Developer Builds Space password
1. ID: SUGARCRM_DEVELOPER_BUILDS_SPACE
1. Description: Account that has access to the SugarCRM Developer Builds Space
1. Username: your SugarCRM username
1. Password: your SugarCRM password
1. ID: SUGARCRM_ACCOUNT
1. Description: SugarCRM Account
1. Create a new global secret for the path to where Sugar Docker will be stored on your host machine (not the path on
Jenkins).
1. Kind: **Secret text**
Expand Down Expand Up @@ -327,10 +329,10 @@ text(s) or file(s)** option.
1. Credentials: **Specific credentials**
1. Select the GITHUB_SUGARCRM_UNIT_TESTS credentials.
1. In the **Bindings** section, select **Add** > **Username and password (separated)**. Then input the following:
1. Username Variable: `DEV_COMMUNITY_USERNAME`
1. Password Variable: `DEV_COMMUNITY_PASSWORD`
1. Username Variable: `SUGARCRM_USERNAME`
1. Password Variable: `SUGARCRM_PASSWORD`
1. Credentials: **Specific credentials**
1. Select the SUGARCRM_DEVELOPER_BUILDS_SPACE credentials.
1. Select the SUGARCRM_ACCOUNT credentials.
1. In the **Bindings** section, select **Add** > **Secret text**. Then input the following:
1. Variable: `PATH_TO_SUGAR_DOCKER_ON_HOST`
1. Credentials: **Specific credentials**
Expand All @@ -351,16 +353,16 @@ text(s) or file(s)** option.
SUGAR_DOCKER_DIRECTORY="workspace/sugardocker"

# This variable is completely optional. If you want to store the Sugar source zips on your local machine instead
# of downloading them from the SugarCRM Developer Builds space, input the path to where the Sugar source zips are
# stored. For example: /var/sugardocker/data/app/sugar_source_zips. Note that your Sugar source zips MUST follow
# this naming convention: Sugar$sugarEdition-$sugarVersion.zip (for example: SugarEnt-7.11.zip).
# of downloading them from the SugarCRM Developer Builds space or Sugar Store, input the path to where the Sugar
# source zips are stored. For example: /var/sugardocker/data/app/sugar_source_zips. Note that your Sugar source zips
# MUST follow this naming convention: Sugar$sugarEdition-$sugarVersion.zip (for example: SugarEnt-7.11.zip).
# SUGAR_SOURCE_ZIPS_DIRECTORY=""

cd scripts

bash -ex RunPackUnitTestsAndBuildProfMPackage.sh $SUGAR_WORKSPACE_PATH

bash SetupEnvAndRunPHPUnitTests.sh $DEV_COMMUNITY_USERNAME $DEV_COMMUNITY_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD $SUGAR_DOCKER_DIRECTORY $SUGAR_SOURCE_ZIPS_DIRECTORY
bash SetupEnvAndRunPHPUnitTests.sh $SUGARCRM_USERNAME $SUGARCRM_PASSWORD $SUGAR_VERSION $SUGAR_EDITION $GITHUB_USERNAME $GITHUB_PASSWORD $SUGAR_DOCKER_DIRECTORY $SUGAR_SOURCE_ZIPS_DIRECTORY
```
Be sure to update the variables appropriately.
1. In the **Post-build Actions** section, click **Add post-build action** and select **Archive the artifacts**.
Expand Down
Binary file modified images/travisenvvars.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
176 changes: 123 additions & 53 deletions scripts/GetCopyOfSugar.sh
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
#!/usr/bin/env bash

# This script gets a copy of Sugar from the designated directory or downloads a Sugar dev build from the SugarCRM
# Developer Builds community.
# This script gets a copy of Sugar from the designated directory or downloads a Sugar dev build from the Sugar Store
# or the SugarCRM Developer Builds community.
#
# Note: you must have access to the SugarCRM Developer Builds community
# (https://community.sugarcrm.com/community/developer/developer-builds) in order for the download to be successful.
# Note: you must have access to the Sugar Store (https://store.sugarcrm.com/download) and/or the SugarCRM Developer
# Builds community (https://community.sugarcrm.com/community/developer/developer-builds) depending on where the build is
# stored in order for the download to be successful.


######################################################################
Expand All @@ -14,7 +15,7 @@
if [[ -z "$1" ]] || [[ -z "$2" ]] || [[ -z "$3" ]] || [[ -z "$4" ]]
then
echo "Not all required command line arguments were set. Please run the script again with the required arguments:
1: Email address associated with your SugarCRM Developer Builds Community account
1: Email address associated with your SugarCRM account
2: Password associated with the above account
3: Sugar name (For example: SugarEnt-7.11)
4. The path to where the Sugar download should be stored
Expand All @@ -26,10 +27,10 @@ then
exit 1
fi

# Email address associated with your SugarCRM developer community account
# Email address associated with your SugarCRM account
email=$1

# Password associated with your SugarCRM developer community account
# Password associated with your SugarCRM account
password=$2

# The Sugar name (For example: SugarEnt-7.11)
Expand Down Expand Up @@ -60,7 +61,7 @@ checkStatusCode(){
statusCode="${BASH_REMATCH[1]}"
if [[ "$statusCode" == $1 ]]
then
echo "Status code is correct: $statusCode"
return
else
echo "Status code is not the expected $1: $statusCode"
echo "$2"
Expand All @@ -87,6 +88,42 @@ getLocationFromResponse(){
fi
}

# Print the value associated with a given key for a JSON response
# $1: The key associated with the JSON value you want to parse
# $2: response from curl command
getJsonValueFromResponse(){
regexJsonValue="$1\":\"([^\"]*)\""

if [[ $2 =~ $regexJsonValue ]]
then
value="${BASH_REMATCH[1]}"
echo "$value"
else
echo "Unable to find the value of $1 in response"
echo $2
exit 1
fi
}

# Print the Sugar Download ID associated with the given Sugar zip file
# $1: The name of the Sugar zip file to search for in the response
# $2: response from curl command
getSugarDownloadIdFromResponse(){
# This regex parses a string similar to
# "id":"download-id-we-are-trying-to-get","name":"SugarEnt-7.9.3.0.zip"
regexJasonValue="\"id\":\"([^\"]*)\",\"name\":\"$1\""

if [[ $2 =~ $regexJasonValue ]]
then
value="${BASH_REMATCH[1]}"
echo "$value"
else
echo "Unable to find the value of $1 in response"
echo $2
exit 1
fi
}

# Print the value associated with a hidden form field
# $1: name of the hidden form field
# $2: response from curl command
Expand Down Expand Up @@ -116,12 +153,47 @@ getHiddenFormFieldValue(){
echo "$2"
exit 1
fi
}

# Authenticate to the Sugar Store and print the URL to download the given Sugar zip
# $1: The name of zip to download (for example: SugarEnt-7.9.3.0.zip)
function authenticateToSugarStoreAndGetDownloadUrl(){

response="$(curl -v -L -c $cookieFile -b $cookieFile 'https://store.sugarcrm.com/download' 2>&1)"
checkStatusCode "200" "$response"
token="$(getHiddenFormFieldValue "_token" "$response")"

response="$(curl -v -L -c $cookieFile -b $cookieFile --data "_token=$token&email=$email&password=$password" https://auth.sugarcrm.com/auth/login 2>&1)"
checkStatusCode "200" "$response"
accountId="$(getJsonValueFromResponse "id" "$response")"

response="$(curl -v -L -c $cookieFile -b $cookieFile "https://store.sugarcrm.com/api/v1/accounts/$accountId/downloads" 2>&1)"
checkStatusCode "200" "$response"
downloadId="$(getSugarDownloadIdFromResponse $1 "$response")"
hash="$(getJsonValueFromResponse "hash" "$response")"

downloadUrl="https://store.sugarcrm.com/download/$downloadId/$hash"
echo $downloadUrl
}

# Authenticate to the Developer Builds Community
function authenticateToDevBuildsCommunity(){

response="$(curl -v -L -c $cookieFile -b $cookieFile 'https://community.sugarcrm.com/login.jspa?ssologin=true&fragment=&referer=%2Fcommunity%2Fdeveloper%2Fdeveloper-builds' 2>&1)"
checkStatusCode "200" "$response"
token="$(getHiddenFormFieldValue "_token" "$response")"

response="$(curl -v -L -c $cookieFile -b $cookieFile --data "_token=$token&email=$email&password=$password" https://auth.sugarcrm.com/saml2/idp/authpage?ReturnTo=https%3A%2F%2Fauth.sugarcrm.com%2Fsaml2%2Fidp%2FSSOService%3Fspentityid%3Dhttps%253A%252F%252Fcommunity.sugarcrm.com%26RelayState%3DL2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw%253D%253D 2>&1)"
checkStatusCode "200" "$response"
samlResponse="$(getHiddenFormFieldValue "SAMLResponse" "$response")"

response="$(curl -v -L -c $cookieFile -b $cookieFile --data-urlencode "SAMLResponse=$samlResponse" --data-urlencode "RelayState=L2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw==" 'https://community.sugarcrm.com/saml/sso' 2>&1)"
checkStatusCode "200" "$response"
}


######################################################################
# Check if we need to download the Sugar source zip
# Check if we have a copy of the Sugar source zip already downloaded
######################################################################

# If we already have a copy of the Sugar source zip, we'll copy it to the Sugar directory and exit the script
Expand Down Expand Up @@ -149,81 +221,79 @@ chmod -R 777 .
sudo chmod -R 777 .


######################################################################
# Authenticate to the community
######################################################################

echo "Authenticating to Developer Builds Community..."

response="$(curl -v -c $cookieFile -b $cookieFile 'https://community.sugarcrm.com/login.jspa?ssologin=true&fragment=&referer=%2Fcommunity%2Fdeveloper%2Fdeveloper-builds' 2>&1)"
checkStatusCode "302" "$response"
location="$(getLocationFromResponse "$response")"

# Location should be something like https://auth.sugarcrm.com/saml2/idp/SSOService?SAMLRequest=fZHNbsIwEIRfxdp7EjuUn1okiBahIlGBSOihNxMMGCV26nVQ%2B%2FZ1IahUlTh6vbPf7sxw9FmV5CQtKqMTYCEFInVhtkrvE1jn02AAo3SIoirjmo8bd9Ar%2BdFIdMQLNfLLTwKN1dwIVMi1qCRyV%2FBs%2FDrncUh5bY0zhSmBjBGldR71bDQ2lbSZtCdVyPVqnsDBuRp5FBWmqhqt3FeIzV7YwlahL0U%2FpAjRAJl4vNLCnVe%2BqoTf7b8gjtS2jrJs0YKATI0t5PmQBHaiRF%2BaTRIQjPXo8aAGne6OdukDUxsmYnHsql7M2KNvwqVAVCf5K0Ns5EyjE9olEFM2CGgcsDhnfU47nPZD1um%2FA1m25z8pfbH1nlebSxPylzxfBstFlgN5u8bjG6ANg5%2Fp9jaF%2B4PF1XpI7xs9jG4Bafv8G376DQ%3D%3D&RelayState=L2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw%3D%3D
response="$(curl -v -c $cookieFile -b $cookieFile $location 2>&1)"
checkStatusCode "302" "$response"
location="$(getLocationFromResponse "$response")"

# Location should be something like https://auth.sugarcrm.com/saml2/idp/authpage?ReturnTo=https%3A%2F%2Fauth.sugarcrm.com%2Fsaml2%2Fidp%2FSSOService%3Fspentityid%3Dhttps%253A%252F%252Fcommunity.sugarcrm.com%26RelayState%3DL2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw%253D%253D
response="$(curl -v -c $cookieFile -b $cookieFile $location 2>&1)"
checkStatusCode "200" "$response"
token="$(getHiddenFormFieldValue "_token" "$response")"

response="$(curl -v -c $cookieFile -b $cookieFile --data "_token=$token&email=$email&password=$password" https://auth.sugarcrm.com/saml2/idp/authpage?ReturnTo=https%3A%2F%2Fauth.sugarcrm.com%2Fsaml2%2Fidp%2FSSOService%3Fspentityid%3Dhttps%253A%252F%252Fcommunity.sugarcrm.com%26RelayState%3DL2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw%253D%253D 2>&1)"
checkStatusCode "302" "$response"
location="$(getLocationFromResponse "$response")"

# Location should be something like 'https://auth.sugarcrm.com/saml2/idp/SSOService?spentityid=https%3A%2F%2Fcommunity.sugarcrm.com&RelayState=L2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw%3D%3D'
response="$(curl -v -c $cookieFile -b $cookieFile $location 2>&1)"
checkStatusCode "200" "$response"
samlResponse="$(getHiddenFormFieldValue "SAMLResponse" "$response")"

response="$(curl -v -c $cookieFile -b $cookieFile --data-urlencode "SAMLResponse=$samlResponse" --data-urlencode "RelayState=L2NvbW11bml0eS9kZXZlbG9wZXIvZGV2ZWxvcGVyLWJ1aWxkcw==" 'https://community.sugarcrm.com/saml/sso' 2>&1)"
checkStatusCode "302" "$response"


#######################################################################
## Download the Sugar zip
# Get the URL to download and authenticate to the appropriate location
#######################################################################

sugarVersion_7_10="7.10"
sugarVersion_7_11="7.11"
sugarVersion_8_0="8.0"

sugarEdition_Ult="Ult"
sugarEdition_Ent="Ent"
sugarEdition_Pro="Pro"

# Get the url for the appropriate Sugar version and edition
# Get the url for the appropriate Sugar version and edition as well as
# authenticate to the appropriate location (Sugar Store or Developer Builds Community)
if [[ "$sugarName" == "Sugar$sugarEdition_Ult-$sugarVersion_7_10" ]]
then downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5839-102-1-8005/SugarUlt-7.10.2.0-dev.1.zip"
then
authenticateToDevBuildsCommunity
downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5839-102-1-8005/SugarUlt-7.10.2.0-dev.1.zip"

elif [[ "$sugarName" == "Sugar$sugarEdition_Ent-$sugarVersion_7_10" ]]
then downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5837-102-1-8003/SugarEnt-7.10.2.0-dev.1.zip"
then
authenticateToDevBuildsCommunity
downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5837-102-1-8003/SugarEnt-7.10.2.0-dev.1.zip"

elif [[ "$sugarName" == "Sugar$sugarEdition_Pro-$sugarVersion_7_10" ]]
then downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5838-102-1-8004/SugarPro-7.10.2.0-dev.1.zip"
then
authenticateToDevBuildsCommunity
downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5838-102-1-8004/SugarPro-7.10.2.0-dev.1.zip"

elif [[ "$sugarName" == "Sugar$sugarEdition_Ult-$sugarVersion_7_11" ]]
then downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5958-102-1-8147/SugarUlt-7.11.0.0-dev.1.zip"
then
authenticateToDevBuildsCommunity
downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5958-102-1-8147/SugarUlt-7.11.0.0-dev.1.zip"

elif [[ "$sugarName" == "Sugar$sugarEdition_Ent-$sugarVersion_7_11" ]]
then downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5959-102-1-8148/SugarEnt-7.11.0.0-dev.1.zip"
then
authenticateToDevBuildsCommunity
downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5959-102-1-8148/SugarEnt-7.11.0.0-dev.1.zip"

elif [[ "$sugarName" == "Sugar$sugarEdition_Pro-$sugarVersion_7_11" ]]
then downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5957-102-1-8146/SugarPro-7.11.0.0-dev.1.zip"
then
authenticateToDevBuildsCommunity
downloadUrl="https://community.sugarcrm.com/servlet/JiveServlet/downloadBody/5957-102-1-8146/SugarPro-7.11.0.0-dev.1.zip"

elif [[ "$sugarName" == "Sugar$sugarEdition_Ult-$sugarVersion_8_0" ]]
then
downloadUrl="$(authenticateToSugarStoreAndGetDownloadUrl "SugarUlt-8.0.0.0.zip")"

elif [[ "$sugarName" == "Sugar$sugarEdition_Ent-$sugarVersion_8_0" ]]
then
downloadUrl="$(authenticateToSugarStoreAndGetDownloadUrl "SugarEnt-8.0.0.0.zip")"

elif [[ "$sugarName" == "Sugar$sugarEdition_Pro-$sugarVersion_8_0" ]]
then
downloadUrl="$(authenticateToSugarStoreAndGetDownloadUrl "SugarPro-8.0.0.0.zip")"

else
echo "Unable to find Sugar download URL for $sugarName"
exit 1
fi

# Download the file

######################################################################
# Download Sugar
######################################################################

echo "Beginning download of $sugarName from $downloadUrl"
response="$(curl -v -c ./mycookie -b ./mycookie -o $sugarName.zip $downloadUrl 2>&1)"
response="$(curl -v -L -c ./mycookie -b ./mycookie -o $sugarName.zip $downloadUrl 2>&1)"
checkStatusCode "200" "$response"
echo "Download complete"

# Check we didn't get an empty zip file
# Verify we didn't get an empty zip file. If the downloadUrl is invalid, we sometimes get a zip file of around 210
# bytes. We've selected 60000000 to ensure we have a sufficiently large file that is likely correct.
fileSize=$(wc -c <"$sugarName.zip")
if [[ $fileSize -lt 60000000 ]]
then
Expand Down
6 changes: 3 additions & 3 deletions scripts/SetupEnvAndRunPHPUnitTests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
if [[ -z "$1" ]] || [[ -z "$2" ]] || [[ -z "$3" ]] || [[ -z "$4" ]] || [[ -z "$5" ]] || [[ -z "$6" ]] || [[ -z "$7" ]]
then
echo "Not all required command line arguments were set. Please run the script again with the required arguments:
1: Email address associated with your SugarCRM Developer Builds Community account
1: Email address associated with your SugarCRM account
2: Password associated with the above account
3: Sugar version (Example: 7.11)
4: Sugar edition (Options: Ult, Ent, Pro)
Expand All @@ -24,10 +24,10 @@ then
exit 1
fi

# Email address associated with your SugarCRM developer community account
# Email address associated with your SugarCRM account
email=$1

# Password associated with your SugarCRM developer community account
# Password associated with your SugarCRM account
password=$2

# The Sugar version to download
Expand Down