undefined symbol: cass_session_new when installing with libcassandra

[ciaran-moore]

When I install the extension to use the Datastax CPP-Driver, I receive the error undefined symbol: cass_session_new when trying to use the extension.

The installs are finding the correct headers as I can see the version info in the cassandra.h file as:

#define CASS_VERSION_MAJOR 2
#define CASS_VERSION_MINOR 17
#define CASS_VERSION_PATCH 1

And the extension build shows the version found as:

-- Checking for module 'cassandra'
--   Found cassandra, version 2.17.1

I can also see the symbol in the header being called and assigned to a CassSession*

/**
 * Creates a new session.
 *
 * @public @memberof CassSession
 *
 * @return Returns a session that must be freed.
 *
 * @see cass_session_free()
 */
CASS_EXPORT CassSession*
cass_session_new();

Install Steps:

# Install Datastax CPP Driver
## Clone Repo
git clone https://github.com/datastax/cpp-driver.git --recursive

## make build directory and enter
mkdir cpp-driver/build && cd cpp-driver/build

## Cmake the project
cmake -DCASS_CPP_STANDARD=17 -DCASS_BUILD_STATIC=ON -DCASS_BUILD_SHARED=ON -DCASS_USE_STD_ATOMIC=ON -DCASS_USE_TIMERFD=ON -DCASS_USE_LIBSSH2=ON -DCASS_USE_ZLIB=ON CMAKE_C_FLAGS="-fPIC" -DCMAKE_CXX_FLAGS="-fPIC -Wno-error=redundant-move" -DCMAKE_BUILD_TYPE="RelWithInfo" -G Ninja .. 
## Install
sudo ninja install

# Install PHP Driver
## Clone the Repo and enter
git clone https://github.com/he4rt/scylladb-php-driver.git --recursive && cd scylladb-php-driver

## Cmake the project
cmake --preset ReleaseLibCassandra -DPHP_SCYLLADB_USE_LIBCASSANDRA=ON -DPHP_SCYLLADB_LIBUV_STATIC=ON && cd out/ReleaseLibCassandra
## Install
sudo ninja install

Both build compile successfully, and cassandra module is loaded within php -m with no warnings

ciaran@test-server:/home/ciaran/scylladb-php-driver/out/Release$ php -m
[PHP Modules]
bcmath
brotli
bz2
calendar
cassandra

PHP Test Script

<?php

$cluster = [
  'name'      => 'test_cluster',
  'username'    => 'cassandra',
  'password'    => 'cassandra',
  'hosts'        => ['cassandra-n1', 'cassandra-n2'],
  'keyspace'    => 'system',
  'port'      => 9042,
];

$_cluster   = Cassandra::cluster([
        'default' => true,
        'name' => $cluster['name'],
        'hosts' => $cluster['hosts'],
        'username' => $cluster['username'],
        'password' => $cluster['password'],
        'port' => $cluster['port'],
        'keyspace' => $cluster['keyspace'],
        'datacenter' => 'dc1',
        'timeout' => 100,
        'connectTimeout' => 100,
        'requestTimeout' => 100
        ])                 // connects to localhost by default
         ->build();

$keyspace  = 'system';
$session   = $_cluster->connect($keyspace);        // create session, optionally scoped to a keyspace
$statement = new Cassandra\SimpleStatement(       // also supports prepared and batch statements
    'SELECT keyspace_name, columnfamily_name FROM schema_columnfamilies'
);
$querySent = $session->execute($statement);
$result    = $querySent->get();                      // wait for the result, with an optional timeout

foreach ($result as $row) {                       // results and rows implement Iterator, Countable and ArrayAccess
    printf("The keyspace %s has a table called %s\n", $row['keyspace_name'], $row['columnfamily_name']);
}

Run test script

ciaran@test-server:/home/ciaran$ php cassandra_test.php
php: symbol lookup error: /usr/lib/php/20230831/cassandra.so: undefined symbol: cass_session_new

Any ideas why this would be happening?

[CodeLieutenant]

This looks like to me it cannot find libcassandra.so the library that you have built from Datastax.
can you please do ld cassandra.so, and paste the output here if you need any help.

cassandra.so -> PHP Driver and not libcassandra.so (the datastax driver)

Few suggestions to fix the issue

• Maybe you Linker Loader does not have it in path, try adding path to libcassandra.so to LD_LIBRARY_PATH
• Library is never installed in required path, copy it to the /usr/lib or some equivalent directory

[ciaran-moore]

Apologies for the delay in responding. I have been caught out by some BAU work.

I trashed the previous box and done a fresh install of everything. Still on Deb12 with a PHP8.2 install. This time copying libcassandra.so before building.

These are the steps performed after installing the system pre-requisites / recommended packages:

# Manually Install Libuv
git clone https://github.com/libuv/libuv.git --recursive && cd libuv
sh autogen.sh
./configure
make
sudo make install

# Manually Install libcassandra
git clone https://github.com/datastax/cpp-driver.git --recursive
mkdir cpp-driver/build && cd cpp-driver/build
cmake ..
make
sudo make install

# Copy libcassandra library to lib from x86_64 folder
sudo cp /usr/local/lib/x86_64-linux-gnu/libcassandra.so.2.17.1 /usr/local/lib/libcassandra.so.2.17.1; \
sudo ln -s /usr/local/lib/libcassandra.so.2.17.1 /usr/local/lib/libcassandra.so.2; \
sudo ln -s /usr/local/lib/libcassandra.so.2 /usr/local/lib/libcassandra.so

# Build php-driver
git clone https://github.com/he4rt/scylladb-php-driver.git --recursive;

cmake --preset ReleaseLibCassandra -DPHP_SCYLLADB_USE_LIBCASSANDRA=ON -DPHP_SCYLLADB_LIBUV_STATIC=ON && cd out/ReleaseLibCassandra && sudo ninja install

# Extension not showing after install. Found INI file hadn't been set.
echo "extension=cassandra.so" | sudo tee -a $(php-config --ini-dir)/cassandra.ini

php -m shows cassandra extension correctly.

Switching users was still providing the PHP Startup Warning

PHP Warning:  PHP Startup: Unable to load dynamic library 'cassandra.so' (tried: /usr/lib/php/20220829/cassandra.so (libcassandra.so.2: cannot open shared object file: No such file or directory), /usr/lib/php/20220829/cassandra.so.so (/usr/lib/php/20220829/cassandra.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0

However, expecting this was a linking issue. Doing sudo ldconfig on additional users cleared this warning and showed the extension loaded correctly.

Unfortunately for me, fixing this issue has led to finding another. I am now receiving this error when attempting to test:

php: /home/ciaran/cpp-driver/src/ref_counted.hpp:43: void datastax::internal::RefCounted<T>::dec_ref() const [with T = datastax::internal::core::RetryPolicy]: Assertion `new_ref_count >= 1' failed.
Aborted

Not sure why it's defaulted to using the home folder for the header directory. The path which the error is reporting (/home/ciaran/cpp-driver/src/ref_counted.hpp) no longer exists after I installed libcassandra

[ciaran-moore]

Okay, it looks like an issue maybe with Debian 12?
The ref_counted.hpp issue is present in the scylliadb-cpp-driver and the datastax-cpp-driver with the latest versions of the drivers. After adding some debugging to the ref_counted.hpp file, both in the Datastax and ScylliaDB CPP-Driver I can see that the RetryPolicy ref_count decrements down to 0, and the address doesn't seem to be freed?

Decrement to 0 --> dec_ref called on type: N8datastax8internal4core11RetryPolicyE | ref_count before: 1, after: 0 | on thread: 140426676909760
dec_ref called on type: N8datastax8internal4core26SpeculativeExecutionPolicyE | ref_count before: 1, after: 0 | on thread: 140426676909760
dec_ref called on type: N8datastax8internal4core14AddressFactoryE | ref_count before: 1, after: 0 | on thread: 140426676909760
dec_ref called on type: N8datastax8internal4core27RequestProcessorInitializerE | ref_count before: 3, after: 2 | on thread: 140426676909760
dec_ref called on type: N8datastax8internal4core7ClusterE | ref_count before: 3, after: 2 | on thread: 140426676909760
dec_ref called on type: N8datastax8internal4core14AddressFactoryE | ref_count before: 3, after: 2 | on thread: 140426676909760
dec_ref called on type: N8datastax8internal4core26SpeculativeExecutionPolicyE | ref_count before: 4, after: 3 | on thread: 140426676909760
reference still being used --> dec_ref called on type: N8datastax8internal4core11RetryPolicyE | ref_count before: -1390838656, after: -1390838657 | on thread: 140426676909760
php: /home/ciaran/cpp-driver/src/ref_counted.hpp:55: void datastax::internal::RefCounted<T>::dec_ref() const [with T = datastax::internal::core::RetryPolicy]: Assertion `new_ref_count >= 1' failed.

[ciaran-moore]

Okay, so using php8.2/8.3 on Debian 12 this issue with ref_count happens in each version of the php-driver down to v1.3.8.

I'm not sure what has happened, but it seems that there is a SharedRefPtr somewhere which either isn't freed, or there are objects still referencing it after it has been freed?

I used GDB to perform a backtrace on a Query to system.local and got this:

Starting program: /usr/bin/php /home/ciaran/cassandra_test.php
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Querying Cassandra - system
[New Thread 0x7fffee94b6c0 (LWP 199786)]
[New Thread 0x7fffebbff6c0 (LWP 199787)]
[New Thread 0x7fffeb3fe6c0 (LWP 199788)]
[New Thread 0x7fffeabfd6c0 (LWP 199789)]
[New Thread 0x7fffea3fc6c0 (LWP 199790)]
[New Thread 0x7fffe9bfb6c0 (LWP 199792)]
php: /home/ciaran/cpp-driver/src/ref_counted.hpp:43: void datastax::internal::RefCounted<T>::dec_ref() const [with T = datastax::internal::core::RetryPolicy]: Assertion `new_ref_count >= 1' failed.

Thread 2 "php" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffee94b6c0 (LWP 199786)]
__pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
44  ./nptl/pthread_kill.c: No such file or directory.
(gdb) bt 1000
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
#1  0x00007ffff74a9e9f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2  0x00007ffff745afb2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007ffff7445472 in __GI_abort () at ./stdlib/abort.c:79
#4  0x00007ffff7445395 in __assert_fail_base (fmt=0x7ffff75b9a90 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x7fffef1befc4 "new_ref_count >= 1", file=file@entry=0x7fffef1bef98 "/home/ciaran/cpp-driver/src/ref_counted.hpp", line=line@entry=43, 
    function=function@entry=0x7fffef1bf048 "void datastax::internal::RefCounted<T>::dec_ref() const [with T = datastax::internal::core::RetryPolicy]") at ./assert/assert.c:92
#5  0x00007ffff7453eb2 in __GI___assert_fail (assertion=0x7fffef1befc4 "new_ref_count >= 1", file=0x7fffef1bef98 "/home/ciaran/cpp-driver/src/ref_counted.hpp", line=43, function=0x7fffef1bf048 "void datastax::internal::RefCounted<T>::dec_ref() const [with T = datastax::internal::core::RetryPolicy]") at ./assert/assert.c:101
#6  0x00007fffef00af8b in datastax::internal::RefCounted<datastax::internal::core::RetryPolicy>::dec_ref() const () from /usr/local/lib/libcassandra.so.2
#7  0x00007fffef00ad8b in datastax::internal::SharedRefPtr<datastax::internal::core::RetryPolicy>::~SharedRefPtr() () from /usr/local/lib/libcassandra.so.2
#8  0x00007fffef01216e in datastax::internal::core::ExecutionProfile::~ExecutionProfile() () from /usr/local/lib/libcassandra.so.2
#9  0x00007fffef117dd4 in datastax::internal::core::RequestProcessorSettings::~RequestProcessorSettings() () from /usr/local/lib/libcassandra.so.2
#10 0x00007fffef12b1f2 in datastax::internal::core::SessionInitializer::initialize(datastax::internal::SharedRefPtr<datastax::internal::core::Host> const&, datastax::internal::core::ProtocolVersion, datastax::internal::Map<datastax::internal::core::Address, datastax::internal::SharedRefPtr<datastax::internal::core::Host>, std::less<datastax::internal::core::Address> > const&, datastax::internal::SharedRefPtr<datastax::internal::core::TokenMap> const&, std::__cxx11::basic_string<char, std::char_traits<char>, datastax::internal::Allocator<char> > const&) () from /usr/local/lib/libcassandra.so.2
#11 0x00007fffef128f0d in datastax::internal::core::Session::on_connect(datastax::internal::SharedRefPtr<datastax::internal::core::Host> const&, datastax::internal::core::ProtocolVersion, datastax::internal::Map<datastax::internal::core::Address, datastax::internal::SharedRefPtr<datastax::internal::core::Host>, std::less<datastax::internal::core::Address> > const&, datastax::internal::SharedRefPtr<datastax::internal::core::TokenMap> const&, std::__cxx11::basic_string<char, std::char_traits<char>, datastax::internal::Allocator<char> > const&) () from /usr/local/lib/libcassandra.so.2
#12 0x00007fffef12f095 in datastax::internal::core::SessionBase::on_initialize(datastax::internal::core::ClusterConnector*) () from /usr/local/lib/libcassandra.so.2
#13 0x00007fffef1322b7 in datastax::internal::Callback<void, datastax::internal::core::ClusterConnector*>::MemberInvoker<void (datastax::internal::core::SessionBase::*)(datastax::internal::core::ClusterConnector*), datastax::internal::core::SessionBase>::invoke(datastax::internal::core::ClusterConnector* const&) const ()
   from /usr/local/lib/libcassandra.so.2
#14 0x00007fffef052e7e in datastax::internal::Callback<void, datastax::internal::core::ClusterConnector*>::operator()(datastax::internal::core::ClusterConnector* const&) const () from /usr/local/lib/libcassandra.so.2
#15 0x00007fffef050cba in datastax::internal::core::ClusterConnector::finish() () from /usr/local/lib/libcassandra.so.2
#16 0x00007fffef050dae in datastax::internal::core::ClusterConnector::maybe_finish() () from /usr/local/lib/libcassandra.so.2
#17 0x00007fffef0510e4 in datastax::internal::core::ClusterConnector::on_connect(datastax::internal::core::ControlConnector*) () from /usr/local/lib/libcassandra.so.2
#18 0x00007fffef056561 in datastax::internal::Callback<void, datastax::internal::core::ControlConnector*>::MemberInvoker<void (datastax::internal::core::ClusterConnector::*)(datastax::internal::core::ControlConnector*), datastax::internal::core::ClusterConnector>::invoke(datastax::internal::core::ControlConnector* const&) const ()
   from /usr/local/lib/libcassandra.so.2
#19 0x00007fffef09411c in datastax::internal::Callback<void, datastax::internal::core::ControlConnector*>::operator()(datastax::internal::core::ControlConnector* const&) const () from /usr/local/lib/libcassandra.so.2
#20 0x00007fffef091331 in datastax::internal::core::ControlConnector::finish() () from /usr/local/lib/libcassandra.so.2
#21 0x00007fffef091715 in datastax::internal::core::ControlConnector::on_connect(datastax::internal::core::Connector*) () from /usr/local/lib/libcassandra.so.2
#22 0x00007fffef09105d in datastax::internal::Callback<void, datastax::internal::core::Connector*>::MemberInvoker<void (datastax::internal::core::ControlConnector::*)(datastax::internal::core::Connector*), datastax::internal::core::ControlConnector>::invoke(datastax::internal::core::Connector* const&) const () from /usr/local/lib/libcassandra.so.2
#23 0x00007fffef08231c in datastax::internal::Callback<void, datastax::internal::core::Connector*>::operator()(datastax::internal::core::Connector* const&) const () from /usr/local/lib/libcassandra.so.2
#24 0x00007fffef0803d4 in datastax::internal::core::Connector::finish() () from /usr/local/lib/libcassandra.so.2
#25 0x00007fffef0810d1 in datastax::internal::core::Connector::on_close(datastax::internal::core::Connection*) () from /usr/local/lib/libcassandra.so.2
#26 0x00007fffef05f906 in datastax::internal::core::Connection::on_close() () from /usr/local/lib/libcassandra.so.2
#27 0x00007fffef05e59e in datastax::internal::core::ConnectionHandler::on_close() () from /usr/local/lib/libcassandra.so.2
#28 0x00007fffef133b18 in datastax::internal::core::Socket::handle_close() () from /usr/local/lib/libcassandra.so.2
#29 0x00007fffef1339df in datastax::internal::core::Socket::on_close(uv_handle_s*) () from /usr/local/lib/libcassandra.so.2
#30 0x00007fffef548645 in uv__finish_close (handle=0x7fffe400bec8) at /home/ciaran/scylladb-php-driver/out/ReleaseLibCassandra/_deps/libuv-src/src/unix/core.c:351
#31 uv__run_closing_handles (loop=0x555555bcd948) at /home/ciaran/scylladb-php-driver/out/ReleaseLibCassandra/_deps/libuv-src/src/unix/core.c:365
#32 uv_run (loop=0x555555bcd948, mode=UV_RUN_DEFAULT) at /home/ciaran/scylladb-php-driver/out/ReleaseLibCassandra/_deps/libuv-src/src/unix/core.c:463
#33 0x00007fffef0b981e in datastax::internal::core::EventLoop::handle_run() () from /usr/local/lib/libcassandra.so.2
#34 0x00007fffef0b97de in datastax::internal::core::EventLoop::internal_on_run(void*) () from /usr/local/lib/libcassandra.so.2
#35 0x00007ffff74a8144 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#36 0x00007ffff75287dc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

This is consistent for v1.3.9 onwards, but v1.3.8 builds and works as expected?

[CodeLieutenant]

Ok this is really strange, can you provide me full instructions how to reproduce everything in VPS server (looks like to me you are running VPS). If it's something that you dont want to share here, send me an email ([email protected])

[ciaran-moore]

Okay, so step by step detail from building the VPS are as follows.

Instance

Machine configuration

Machine type | n2-custom-4-2048
CPU platform | N2
Minimum CPU platform | Intel Cascade Lake
Architecture | x86/64

Bootable Disk in Google Cloud Console

Single Region, 40GB SSD
Image: debian-12-bookworm-v20241009 (Debian, Debian GNU/Linux, 12 (bookworm), amd64 built on 20241009)
OS: Debian 12 Bookworm

Install System Pre-Requisites

Once disk is created and instance is loaded, copy install script and run with php-version as parameter:

sudo ./install.sh -p 8.3

Debian12 install.sh

#!/bin/bash

# Default value for DEB_VERSION (optional)
PHP_VERSION=""

# Parse command-line arguments
while [[ "$#" -gt 0 ]]; do
    case $1 in
        -p|--php_version)
            PHP_VERSION="$2"
            shift 2
            ;;
        *)
            echo "Usage: $0 -p <php_version> "
            exit 1
            ;;
    esac
done

# Ensure both PHP_VERSION and NODE are provided
if [[ -z "$PHP_VERSION" || -z "$NODE" ]]; then
    echo "Error: Option -p <php_version> is required."
    echo "Usage: $0 -p 8.3"
    exit 1
fi

INSTALL_DIR=/home/ciaran
WORKSPACE_DIR=/mnt/workspace_v2
APP_DIR=/mnt/app

# Output provided values
echo "PHP Version: $PHP_VERSION"

##################################################################################
#               Update sources, and install base packages.                       #
#               We will run `apt upgrade` after the php repository               #
#               is added, this way, the dependancy packages which we             #
#               install here, will also be upgraded to their latest              #
#               versions                                                         #
##################################################################################


mkdir -p $WORKSPACE_MOUNT
mkdir -p $APP_MOUNT

echo "Updating System"
apt update -y > /dev/null 2>&1


# Install base packages
echo
echo "===================================================="
echo "=             Installing Base Packages             =" 
echo "===================================================="
echo

for APP in apt-transport-https apache2 autoconf brotli build-essential ca-certificates cmake curl dirmngr \
fonts-liberation gcc gconf-service git gnupg gzip libappindicator1 libasound2 libassuan-dev libatk1.0-0 \
libbrotli-dev libc-dev libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 \
libgdk-pixbuf2.0-0 libglib2.0-0 libgmp-dev libgpg-error-dev libgpgme11-dev libgtk-3-0 libkrb5-dev libmcrypt-dev \
libstdc++6 libmemcached-dev libnghttp2-dev libnspr4 libnss3 libpango-1.0-0 libpangocairo-1.0-0 librdkafka1 \
libssl-dev libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 \
libxi6 libxml2-dev libxrandr2 libxrender1 libxss1 libxtst6 libz-dev lsb-release make memcached nghttp2 ninja-build \
ntp openssl pkg-config python3 python3-pip rsync software-properties-common sshpass sudo unzip wget xdg-utils \
zlib1g zlib1g-dev; do
    echo "Installing $APP..."
    apt install $APP -y > /dev/null 2>&1
done

AUTOUPDATE=$(which autoupdate) # Required for WDDX Extension


echo "Libraries Installed"

echo
echo "===================================================="
echo "=                  Installing PHP                  =" 
echo "===================================================="
echo


echo 
echo '########################################'
echo '#         Install PHP & Modules        #'
echo '########################################'
echo 

echo "Installing PHP & Modules"
echo

wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list
apt update -y  > /dev/null 2>&1 
apt upgrade -y  > /dev/null 2>&1

echo "Installing PHP"
apt install php$PHP_VERSION php$PHP_VERSION-common php$PHP_VERSION-cli php$PHP_VERSION-dev php-pear -y > /dev/null 2>&1

# Install PHP Extensions
echo "Installing PHP Extensions"
#Available Mods found here '/etc/php/$PHP_VERSION/mods-available'
for APP in php$PHP_VERSION-bcmath php$PHP_VERSION-bz2 php$PHP_VERSION-curl php$PHP_VERSION-common php$PHP_VERSION-dom php$PHP_VERSION-gd php$PHP_VERSION-gmp php$PHP_VERSION-gnupg \
php$PHP_VERSION-igbinary php$PHP_VERSION-intl php$PHP_VERSION-imap php$PHP_VERSION-mbstring php$PHP_VERSION-mcrypt php$PHP_VERSION-memcached php$PHP_VERSION-mongodb \
php$PHP_VERSION-msgpack php$PHP_VERSION-mysql php$PHP_VERSION-redis php$PHP_VERSION-pspell php$PHP_VERSION-protobuf php$PHP_VERSION-rdkafka php$PHP_VERSION-ssh2 php$PHP_VERSION-soap php$PHP_VERSION-xml \
php$PHP_VERSION-xsl php$PHP_VERSION-zip; do
    echo "Installing $APP..."
    apt-get install $APP -y --no-install-recommends  > /dev/null 2>&1
done


# Compile gnupg extension
echo "Compile BROTLI extension"
printf '\n' | pecl install -f brotli > /dev/null 2>&1

echo "Create BROTLI Extension file"
echo "extension=brotli.so" | sudo tee -a /etc/php/$PHP_VERSION/mods-available/brotli.ini

# Link BROTLI Extension in CLI / Apache
ln -s /etc/php/$PHP_VERSION/mods-available/brotli.ini /etc/php/$PHP_VERSION/cli/conf.d/20-brotli.ini
ln -s /etc/php/$PHP_VERSION/mods-available/brotli.ini /etc/php/$PHP_VERSION/apache2/conf.d/20-brotli.ini

echo "Extension files linked"

EXT_DIR=$(php-config --extension-dir)

# Install WDDX extension
echo "Install WDDX extension"
echo "Get WDDX -> GIT"
wget https://github.com/php/pecl-text-wddx/archive/master.zip -O $INSTALL_DIR/wddx.zip > /dev/null 2>&1
echo "Unpacking Wddx Zip"
unzip wddx.zip  > /dev/null 2>&1
cd pecl-text-wddx-master
echo "Build WDDX "
phpize  > /dev/null 2>&1
$AUTOUPDATE
./configure > /dev/null 2>&1
make > /dev/null 2>&1
echo "Installing WDDX from Source"
make install > /dev/null 2>&1

echo "Create WDDX Extension ini"
echo "extension=wddx.so" | sudo tee -a /etc/php/$PHP_VERSION/mods-available/wddx.ini

# Link WDDX Extension in CLI / Apache
ln -s /etc/php/$PHP_VERSION/mods-available/wddx.ini /etc/php/$PHP_VERSION/cli/conf.d/20-wddx.ini
ln -s /etc/php/$PHP_VERSION/mods-available/wddx.ini /etc/php/$PHP_VERSION/apache2/conf.d/20-wddx.ini

echo "Extension files linked"

cd $INSTALL_DIR
rm -Rf $INSTALL_DIR/pecl-text-wddx-master $INSTALL_DIR/wddx.zip


echo 
echo '###########################################'
echo '# ALLOW RSA LOGIN ON NEWER DEBIAN INSTALL #'
echo '###########################################'
echo 
echo "PubkeyAcceptedAlgorithms ssh-rsa,rsa-sha2-512,rsa-sha2-256" | sudo tee -a /etc/ssh/sshd_config
echo "HostkeyAlgorithms ssh-rsa,rsa-sha2-512,rsa-sha2-256" | sudo tee -a /etc/ssh/sshd_config

Install Cassandra Extension

This process is run manually with copy/paste rather than scripted to ensure that each command is running as it should.

Install Lib UV Extension

Manually install the libuv library to ensure library is compiled with -fpic

# Manually Install Libuv
# pkg-config Name: libuv-static
# pkg-config Version: 1.49.1
git clone https://github.com/libuv/libuv.git --recursive && cd libuv
cmake -DBUILD_TESTING=OFF -DLIBUV_BUILD_SHARED=ON CMAKE_C_FLAGS="-fPIC" -DCMAKE_BUILD_TYPE="RelWithInfo" ..

make
sudo make install

Install CPP-Driver

Install CPP-Driver, ensuring that output LIBDIR is specified and PREFIX to prevent copy to /usr/local/lib following install.

# Manually Install static libcassandra
# Currnet pkg-config Name: cassandra
# Currnet pkg-config Version: 2.17.1
# Currnet pkg-config Requires: libuv
# Currnet pkg-config Requires: openssl
git clone https://github.com/datastax/cpp-driver.git --recursive

mkdir cpp-driver/build && cd cpp-driver/build

# The process is the same for scylladb/cpp-driver and datastax/cpp-driver. 
# Cmake will ignore parameters which are not configured.

cmake -DCASS_CPP_STANDARD=17 -DCASS_BUILD_STATIC=ON -DCASS_BUILD_SHARED=ON -DCASS_USE_TIMERFD=ON -DCASS_USE_LIBSSH2=ON -DCASS_USE_ZLIB=ON -DCMAKE_C_FLAGS="-fPIC" -DCMAKE_CXX_FLAGS="-fPIC -Wno-error=redundant-move" -DCMAKE_BUILD_TYPE="RelWithInfo" -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_INSTALL_LIBDIR=lib -DCASS_INSTALL_PKG_CONFIG=ON ..

make # VERBOSE=1 #Optional verbose logging
sudo make install

Install PHP Extension

While debugging, I have set the CMakePreset.json for ReleaseLibCassandra and Release to Unix Makefiles to view the make process verbosely and monitor what is being linked and to where (mainly for the issue of the driver not being linked in the final .so file.), but present also with Ninja

git clone https://github.com/he4rt/scylladb-php-driver.git --recursive;

# Driver dependant commands:
## Scylladb
cmake --preset Release -DPHP_SCYLLADB_LIBUV_STATIC=ON && cd out/Release
## DS
cmake --preset ReleaseLibCassandra -DPHP_SCYLLADB_USE_LIBCASSANDRA=ON -DPHP_SCYLLADB_LIBUV_STATIC=ON && cd out/ReleaseLibCassandra

# Build dependant command:
## Ninja
sudo ninja install

## Unix Makefiles
make # VERBOSE=1 # Optional verbose logging
sudo make install

# Install .ini file

echo "extension=cassandra.so" | sudo tee -a $(php-config --ini-dir)/cassandra.ini

Finally. To ensure that the library is correctly linked afterwards, and that the system has found the installed extension, run sudo ldconfig to ensure paths are refreshed.

Check module is loaded with php -m and check that libcassandra in linked in the cassandra.so extension with sudo ldd $(php-config --extension-dir)/cassandra.so

[ciaran-moore]

A quick update on reproducing the issue.

<?php 

$cluster = [
  'name'      => 'test_cluster',
  'username'    => 'cassandra',
  'password'    => 'cassandra',
  'hosts'        => ['cassandra-n1', 'cassandra-n2'],
  'keyspace'    => 'system',
  'port'      => 9042,
];


$_cluster   = Cassandra::cluster()
->withContactPoints('127.0.0.1')
->withPort(9042)
->withCredentials($cluster['username'],$cluster['password'])
->withConnectTimeout(30)
->withRetryPolicy(new \Cassandra\RetryPolicy\DefaultPolicy())
->build();
$session   = $_cluster->connect('system');


$statement = new Cassandra\SimpleStatement(
'SELECT * FROM system.sstable_activity_v2 LIMIT 30'
);

$querySent = $session->execute($statement);

//var_dump(get_class_methods($_cluster), "Cluster");
//var_dump(get_class_methods($session), "Session");
// exit;

$i = 0;
foreach($querySent as $row)
{
    echo ++$i ."\n";
}

//$i = 0;
//while($i < $querySent->count())
//{
//    echo ++$i . "\n";
//    $querySent->next();
//}

With this script, if you uncomment both of the var_dump lines, it produces the ref_count error message. However, uncommenting and var_dumping one at a time runs fine.

The foreach and while loops however, (without the var_dumps being present) will run, but both end in a segfault

ciaran@test-server:/home/ciaran/test$ php test.php 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Segmentation fault

I have noticed that var_dumps seem to cause a lot of segmentation faults on various attributes while I've been inching my way through the code to find which specific call is causing the ref_count issue. But the issue seems to be on the teardown process once data has finished being returned.

[CodeLieutenant]

Thanks for the scripts and how to reproduce, this is fenomenal description, I dont have time until the weekend this week, I'll have to focus on some other Scylla stuff before I get permission to work on the driver. If you @danielhe4rt can try to reproduce this issue with the example script and send me the results, that would be fantastic, if you are not able, same as me, I'll do it this weekend. Hope @ciaran-moore it's not late for you and a big deal breaking right now

[CodeLieutenant]

I have noticed that var_dumps seem to cause a lot of segmentation faults on various attributes while I've been inching my way through the code to find which specific call is causing the ref_count issue. But the issue seems to be on the teardown process once data has finished being returned.

Yes, this looks like a shutdown issue, and also the overriden get_properties method which is used in var_dump