This project contains a C library for using the Spectra S3 Deep Storage REST interface.
Join us at our Google Groups forum to ask questions, or see frequently asked questions.
Pull requests are welcome, but first sign the Contributor's Agreement
For Windows we release a binary version of the SDK for your convenience. See win32/README.md for more information about building from source in Windows.
The zip file available on the Releases tab on the github page contains these files:
ds3\bin\ds3.dll
ds3\bin\ds3.dll.manifest
ds3\bin\libglib-2.0-0.dll
ds3\bin\libiconv-2.dll
ds3\bin\libintl-8.dll
ds3\bin\pthreadGC2.dll
ds3\include\ds3.h
ds3\include\ds3_string.h
ds3\include\ds3_string_multimap.h
ds3\lib\ds3.exp
ds3\lib\ds3.lib
To compile against the library:
- Include ds3\include\ds3.h
- Link against ds3\lib\ds3.lib
- Distribute ds3\bin*.dll along with your application.
For Unix/Linux we distribute the SDK as source code. The release tarballs contain a simple build script that should work on most Unix/Linux systems. The build system is currently CMake.
To install CMake on Ubuntu use apt-get and install the following:
$ sudo apt-get install cmake
To install CMake on CentOS use yum and install the following: $ sudo yum install cmake
The SDK depends upon several open source libraries, so you'll need to ensure that you've installed the development header packages for each of them. For example, Linux systems often provide lib*-dev or lib*-devel packages. The Spectra S3 dependencies are:
- libxml2
- libcurl
- libglib-2.0
On Ubuntu you can install them with apt-get:
$ sudo apt-get install libxml2-dev
$ sudo apt-get install libcurl4-openssl-dev
$ sudo apt-get install libglib2.0-dev
On CentOS you can install them with yum:
$ sudo yum install libxml2-devel
$ sudo yum install libcurl-devel
$ sudo yum install glib2-devel
For testing you will need the boost unit test library as well.
On Ubuntu this can be installed with:
$ sudo apt-get install libboost-test-dev
On CentOS this can be installed with:
$ sudo yum install boost-test
Build from cloned source repository:
$ ./rpmbuild.sh
The command will produce both src.rpm and and binary rpm.
Content of the compiled package:
$ rpm -qlp /home/yu/rpmbuild/RPMS/x86_64/ds3_c_sdk-3.2.3-11.el7.centos.x86_64.rpm
/usr/include/ds3/ds3.h
/usr/include/ds3/ds3_connection.h
/usr/include/ds3/ds3_net.h
/usr/include/ds3/ds3_request.h
/usr/include/ds3/ds3_string.h
/usr/include/ds3/ds3_string_multimap.h
/usr/include/ds3/ds3_string_multimap_impl.h
/usr/include/ds3/ds3_utils.h
/usr/lib64/libds3.so
/usr/share/doc/ds3_c_sdk-3.2.3/LICENSE.md
/usr/share/doc/ds3_c_sdk-3.2.3/README.md
/usr/share/doc/ds3_c_sdk-3.2.3/r3x_migration_guide.md
Download the latest tarball from the Releases tab on the github page and open a terminal window.
$ cd directory/containing/release/tarball
$ tar zxf ds3_c_sdk-{version}.tgz
$ cd ds3_c_sdk-{version}
$ cmake .
$ make
$ su
# make install
# ldconfig
# exit
$
The make install command installs the following files on your system:
{prefix}/lib/libds3.la
{prefix}/lib/libds3.a
{prefix}/lib/pkgconfig/libds3.pc
{prefix}/lib/libds3.so.0.0.0
{prefix}/include/ds3.h
To build the SDK from a git clone you must have a relatively recent version of GNU CMake installed. While the release tarball does contain the build scripts that CMake generates, the git repository does not.
Assuming you have CMake installed, you can follow the same instructions as the tarball release.
The SDK provides a pkg-config definition, so you can determine the proper compiler tags using the following commands:
- Compiler flags:
pkg-config --cflags libds3 - Linker flags:
pkg-config --libs libds3 - Both compiler and linker flags:
pkg-config --cflags --libs libds3
The /sample directory has an example project that uses the sdk. It provides a Makefile
to build the SDK and itself.
To build the sample, use the following commands:
$ cd directory/containing/source/tree
$ cd sample
$ make deps # Builds the SDK and installs it into directory/containing/source/tree**/install**
$ make
To run it, first ensure that DS3_ACCESS_KEY, DS3_SECRET_KEY, DS3_ENDPOINT (and optionally http:proxy) are set in environment variables to match the target device. For the simulator, see [Installation Instructions] (https://developer.spectralogic.com/sim-install/)
$ make run-put-bulk # create "books" bucket and put files into it
$ make run-get-service # list all buckets
$ make run-get-bucket # list contents of "books" bucket
$ make run-get-object # get first book and write to temp file
The /test directory contains function tests of the sdk. Ensure that the environment variables DS3_ACCESS_KEY, DS3_SECRET_KEY, DS3_ENDPOINT (and optionally http:proxy) are set. To build and run the tests, use the following commands:
$ cd directory/containing/source/tree
$ cmake .
$ make
$ make install
$ ldconfig
$ cd test
$ cmake .
$ make
$ make test
For the list of API calls in the C SDK please see the documentation here
The following section contains several examples of using the DS3 C SDK. The first example shows how to get a list of all the buckets back from Spectra S3:
#include <stdlib.h>
#include <stdio.h>
// "ds3.h" is the only header that needs to be included to use the DS3 API.
#include "ds3.h"
int main (void) {
// Get Service
ds3_client* client;
ds3_request* request;
ds3_error* error;
ds3_list_all_my_buckets_result_response *response;
uint64_t bucket_index;
// Create a client from environment variables
error = ds3_create_client_from_env(&client);
handle_error(error);
// Create the get service request. All requests to a DS3 appliance start this way.
// All ds3_init_* functions return a ds3_request struct
request = ds3_init_get_service_request();
// This performs the request to a DS3 appliance.
// If there is an error 'error' will not be NULL
// If the request completed successfully then 'error' will be NULL
error = ds3_get_service_request(client, request, &response);
ds3_request_free(request);
handle_error(error);
if(response->num_buckets == 0) {
printf("No buckets returned\n");
ds3_list_all_my_buckets_result_response_free(response);
ds3_creds_free(client->creds);
ds3_client_free(client);
return 0;
}
for (bucket_index = 0; bucket_index < response->num_buckets; bucket_index++) {
ds3_bucket_details_response* bucket = response->buckets[bucket_index];
printf("Bucket: (%s) created on %s\n", bucket->name->value, bucket->creation_date->value);
}
ds3_list_all_my_buckets_result_response_free(response);
ds3_creds_free(client->creds);
ds3_client_free(client);
ds3_cleanup();
return 0;
}
The next demonstrates how to create a new bucket:
#include <stdlib.h>
#include <stdio.h>
// "ds3.h" is the only header that needs to be included to use the DS3 API.
#include "ds3.h"
int main (int args, char * argv[]) {
ds3_creds * creds = ds3_create_creds("cnlhbg==","ZIjGDQAs");
ds3_client * client = ds3_create_client("http://192.168.56.101:8080", creds);
char * bucket = "books";
ds3_request * create_bucket_request = ds3_init_put_bucket_request(bucket_name);
ds3_error* error = ds3_put_bucket_request(client, create_bucket_request);
ds3_request_free(create_bucket_request);
if(error != NULL) {
if(error->error != NULL) {
printf("Failed to create bucket with error (%lu): %s\n", error->error->http_error_code, error->message->value);
printf("Message Body: %s\n", error->error->message->value);
printf("Resource: %s\n", error->error->resource->value);
}
else {
printf("Got a runtime error: %s\n", error->message->value);
}
ds3_free_error(error);
ds3_free_creds(creds);
ds3_free_client(client);
return 1;
}
printf("Successfully created bucket: %s\n", bucket);
ds3_free_creds(creds);
ds3_free_client(client);
ds3_cleanup();
return 0;
}The following is an example of performing a get bucket:
#include <stdlib.h>
#include <stdio.h>
// "ds3.h" is the only header that needs to be included to use the DS3 API.
#include "ds3.h"
int main (int args, char * argv[]) {
ds3_list_bucket_result_response* response;
uint64_t i;
// Setup client credentials and then the actual client itself.
ds3_creds * creds = ds3_create_creds("cnlhbg==","ZIjGDQAs");
ds3_client * client = ds3_create_client("http://192.168.56.101:8080", creds);
// You can optionally set a proxy server that a request should be sent through
//ds3_client_proxy(client, "192.168.56.1:8888");
char * bucket = "books";
// Create the get bucket request.
ds3_request* request = ds3_init_get_bucket(bucket);
// This performs the request to a DS3 appliance.
// If there is an error 'error' will not be NULL
// If the request completed successfully then 'error' will be NULL
ds3_error* error = ds3_get_bucket_request(client, request, &response);
ds3_request_free(request);
// Check that the request completed successfully
if(error != NULL) {
if(error->error != NULL) {
printf("Failed to create bucket with error (%lu): %s\n", error->error->http_error_code, error->message->value);
printf("Message Body: %s\n", error->error->message->value);
printf("Resource: %s\n", error->error->resource->value);
}
else {
printf("Got a runtime error: %s\n", error->message->value);
}
ds3_list_bucket_result_response_free(response);
ds3_error_free(error);
ds3_creds_free(creds);
ds3_client_free(client);
return 1;
}
if (response == NULL) {
printf("Response was null\n");
ds3_creds_free(creds);
ds3_client_free(client);
return 1;
}
if(response->num_objects == 0) {
printf("No objects returned\n");
ds3_list_bucket_result_response_free(response);
ds3_creds_free(creds);
ds3_client_free(client);
return 0;
}
for (i = 0; i < response->num_objects; i++) {
ds3_contents_response* object = response->objects[i];
printf("Object: (%s) created on %s\n", object->name->value, object->last_modified->value);
}
ds3_list_bucket_result_response_free(response);
ds3_creds_free(creds);
ds3_client_free(client);
ds3_cleanup();
return 0;
}The structure of the code is very similar to the previous examples. Setup the client, setup the call, perform the call. Every request follows this same pattern.
Additional examples are available here: samples
Update the version of the SDK before creating a new release. The format is <major>.<minor>.<patch>, where the
<major>.<minor> numbers must match the version of BP. The <patch> is an incrementing number that increments with
each SDK release for a given major/minor release.
The version is specified in CMakeLists.txt.