Journey of SunbirdED on GCP

[mahesh-maney]

We have installed Sunbird ED on Google Cloud Platform successfully. In this discussion we are sharing our journey in working towards getting Sunbird-ED work on GCP bed.

We captured;
- The issues that we faced,
- Debugging steps that we went through, and
- Solutions that we arrived at;
to get the Sunbird-ED services up and working.

We faced few issues in our installation journey. We were able to resolve them as and when we faced the challenges on GCP bed.
We have documented these issues to best of our knowledge for any future references (link below)

https://project-sunbird.atlassian.net/wiki/spaces/UPSMF/pages/3277881349/Journey+of+SunbirdED+on+GCP

Our special call out @sharathkashyap for the timely support and suggestions.

[Krishnaj20]

thanks @mahesh-maney for initiating this thread. This is extremely useful information for the entire community. Look forward to how you've resolved the rest of the issues.

@santhosh-tg @SanthoshVasabhaktula @vinukumar-vs @madhucr @alok-os @swayangjit @rajeevsathish @maheshkumargangula

[santhosh-tg]

@gandham-santhosh

[vinukumar-vs]

@mahesh-maney @sharathkashyap Can you please contribute the code fix elated to GCP content schema json files loading as text files.
This is related to content-service fix.

GCP video streaming media service changes are not fully verified by Sunbird team yet(still in-progress). Did your team has done any progress on this?

@Krishnaj20 @gandham-santhosh @pkranga @SanthoshVasabhaktula @vrayulu @pallakartheekreddy @maheshkumargangula

[maheshkumargangula]

@mahesh-maney – As part of Sunbird release verification, we have deployed the GCP related contribution changes. The GCP contribution changes are not working in content-service. The signed URL generation is failing.

After our analysis we understood that, the cloud-storage-sdk changes independently working fine but, having version conflicts in content-service.

The content-service using two google services here, one is storage and another is youtube. The dependencies added for both has version conflict with the core interface dependency (google-api-client) and it is failing to generate the signed URL.

Please help us understand how these changes are tested before the GCP contribution and are we missing any other contributions to handle this?

Cc: @Krishnaj20 @pallakartheekreddy @vrayulu @vinukumar-vs

[Krishnaj20]

@mahesh-maney – As part of Sunbird release verification, we have deployed the GCP related contribution changes. The GCP contribution changes are not working in content-service. The signed URL generation is failing.

After our analysis we understood that, the cloud-storage-sdk changes independently working fine but, having version conflicts in content-service.

The content-service using two google services here, one is storage and another is youtube. The dependencies added for both has version conflict with the core interface dependency (google-api-client) and it is failing to generate the signed URL.

Please help us understand how these changes are tested before the GCP contribution and are we missing any other contributions to handle this?

Cc: @Krishnaj20 @pallakartheekreddy @vrayulu @vinukumar-vs

@mahesh-maney - Appreciate if you can let us know how you have addressed this issue in your ED instance?

[mahesh-maney]

@maheshkumargangula ,
The cloud-storage-sdk uses Apache JCloud library that does not support Google signed URL v4 (which is recommended by google).
Therefore we used google-cloud-storage version 2.16.0 to generate signed URL and not cloud-storage-sdk.

In our content-service(version 5.1.0, forked ) we are using google-cloud-storage version 2.16.0 to generate the signed URL.
We made changes to content-service to use google-cloud-storage version 2.16.0 and hence we were able to build and deploy successfully.

Our case required us to test, file upload using google signed URL. This use case worked after the fix.
However, youtube URL was not in our scope and hence we did not test this use case.

Cc: @Krishnaj20 @pallakartheekreddy @vrayulu @vinukumar-vs @sharathkashyap