Add the following line to 'user/config.php' and set the prefix to your preferred character or string:
define('KEYWORD_PREFIX', '!');
You may also want to move the YOURLS folder into a subdirectory, in case you want to run a different site on your domain, so you also have to change the '.htaccess' file from:
RewriteRule ^(.*)$ /yourls-loader.php [L]
to:
RewriteRule ^(.*)$ /path-to-yourls/yourls-loader.php [L]