Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Oracle uri format does not allow connecting by service name #22

Closed
vectro opened this issue Apr 14, 2023 · 30 comments · Fixed by #23 or #25
Closed

Oracle uri format does not allow connecting by service name #22

vectro opened this issue Apr 14, 2023 · 30 comments · Fixed by #23 or #25

Comments

@vectro
Copy link

vectro commented Apr 14, 2023

The DBD::Oracle module supports a few different connection string formats, all of which support specifying either the SID or the service name. However, the URI-db library only support specifying the SID. This unfortunately means that connecting to an Oracle PDB is not possible without enumerating the PDB in a TNSNAMES.ora file.

@theory
Copy link
Collaborator

theory commented Apr 16, 2023

URI::oracle does support SIDs if no host name or port is specified:

$params =~ s/sid=// unless $self->host || $self->_port;

@vectro
Copy link
Author

vectro commented Apr 16, 2023

I agree that it supports connecting by SID, but not by session name. As I wrote earlier,

However, the URI-db library only support specifying the SID.

But, I should have written "Service Name" rather than "Session Name" earlier. I will correct that now.

@vectro vectro changed the title Oracle uri format does not allow connecting by session name Oracle uri format does not allow connecting by service name Apr 16, 2023
@theory
Copy link
Collaborator

theory commented Apr 16, 2023

What would an interface for supporting a service name look like? How would one distinguish it from a SID?

Also note that you can take the URI parts and do what you want with them, you don't have to rely exclusively on URI::db's interpretation of what to pass to DBI->connect.

@vectro
Copy link
Author

vectro commented Apr 16, 2023

What would an interface for supporting a service name look like? How would one distinguish it from a SID?

DBD::Oracle has many different ways of specifying the connection properties (see documentation); however, the one used by URI::db does not support specifying the service name.

Also note that you can take the URI parts and do what you want with them, you don't have to rely exclusively on URI::db's interpretation of what to pass to DBI->connect.

That is true if you control the caller, but not if URI::db is called from another tool (such as Sqitch).

@theory
Copy link
Collaborator

theory commented Apr 16, 2023

Ah, perhaps URI::oracle should not prepend sid=; it seems like DBD::Oracle can handle either a service name or a SID? Or, no, perhaps specify the service name as the host name? IOW, if your service name is my_service, does db:oracle://my_service work?

@vectro
Copy link
Author

vectro commented Apr 17, 2023

IOW, if your service name is my_service, does db:oracle://my_service work?

I think this will do the right thing, but of course that is not compatible with also specifying the hostname or port, which you might want to do sometimes.

I don't have a strong opinion about this but it seems like maybe you want something like:

  1. If there is not a hostname or port, just put the raw SID or service name (this is way 1 in the DBD::Oracle documentation)
  2. If there is a hostname, use the EZCONNECT syntax (way 3 in the DBD::Oracle documentation)
  3. Otherwise (port but no hostname), I'm not sure what to do, maybe this is not a valid state?

@theory
Copy link
Collaborator

theory commented Apr 19, 2023

Okay here's what I think is worth trying:

  1. just as you suggest: If there is not a hostname or port, just put the raw SID or service name (this is way 1 in the DBD::Oracle documentation)
  2. Otherwise, use the semicolon-delimted key=val pairs as currently implemented, which is the DBI standard and demonstrated in the fourth example in the code block of the DBD::Oracle documentation.

@theory
Copy link
Collaborator

theory commented Apr 19, 2023

Actually, that is how it works!

$ perl -MURI -E 'say URI->new($ARGV[0])->dbi_dsn' 'db:oracle:foo'
dbi:Oracle:foo
$ perl -MURI -E 'say URI->new($ARGV[0])->dbi_dsn' 'db:oracle:///foo'
dbi:Oracle:foo

Does that not allow the DBD::Oracle to interpret foo as a SID or service name?

Oh, but if it ha a host name:

$ perl -MURI -E 'say URI->new($ARGV[0])->dbi_dsn' 'db:oracle://db.com/foo' 
dbi:Oracle:host=db.com;sid=foo

So the question is: if there is a host or port, how do we also support a service name. We could use the EZCONNECT URL, which would be:

$ perl -MURI -E 'say URI->new($ARGV[0])->dbi_dsn' 'db:oracle://db.com/foo' 
dbi:Oracle://db.com/foo

In that case, does it know whether foo is a service name or a SID?

Should we perhaps use service_name=foo if there is a host name?

Hrm, looking at the EZCONNECT again, maybe we should just replace db: with dbi: and simply pass anything else directly through…

theory added a commit that referenced this issue Apr 19, 2023
The goal is to get away from hard-coding support for a SID, since we
can't actually tell whether the database name part of the URL should be
a SID or a service name. The hope is that by just returning an EZCONNECT
URL, instead, the DBI or Oracle client library it uses can make the
appropriate determination. Resolves #22.
@theory
Copy link
Collaborator

theory commented Apr 19, 2023

Seems like that should work. So let's try it. Check out #23 and see what you think.

theory added a commit that referenced this issue Apr 19, 2023
The goal is to get away from hard-coding support for a SID, since we
can't actually tell whether the database name part of the URL should be
a SID or a service name. The hope is that by just returning an EZCONNECT
URL, instead, the DBI or Oracle client library it uses can make the
appropriate determination. Resolves #22.
@theory
Copy link
Collaborator

theory commented Apr 19, 2023

Okay, the solution is probably gonna be either 160af71, which simply returns an EZCONNECT URL, but then incorrectly returns nothing from dbi_params, or else 9fd8bfa, which uses service_name instead of sid for the DBI parameter (added in DBD::Oracle 1.17 in 2006), with the expectation that this SO answer is correct that one can use a SID for a service name and it should work, but not, apparently, the other way around.

I think that 9fd8bfa is the correct approach, but let me know if it doesn't work for you.

@theory
Copy link
Collaborator

theory commented Apr 22, 2023

Just to test that using service_name instead of sid works in general, I pushed sqitchers/sqitch@a49d012, which just messes with the DSN, and let the tests run, and it looks good. @vectro can you confirm that it works for your use case?

@vectro
Copy link
Author

vectro commented Apr 26, 2023

Thanks David for looking at this so quickly.

sqitchers/sqitch@a49d012 work for me.

I didn't test either of the changes in this repo (but will, if you like). They both look fine to me, with some caveats:

  • 9fd8bfa makes me nervous that the use of service_name is undocumented behavior. Perhaps we could prevail upon the DBD::Oracle maintainer to document this.
  • 160af71 might not produce the right behavior when no hostname is provided. According to the documentation, hostname is a required parameter. I see the test includes output like dbi:Oracle:///foo or dbi:Oracle://:42/foo' which I guess is invalid. At a minimum if we go this route, we should probably produce a bare service name (no EZCONNECT //) when no hostname or port are specified.

@theory
Copy link
Collaborator

theory commented Apr 30, 2023

Agreed about the documentation for service_name; opened perl5-dbi/DBD-Oracle#164 to request updated docs on DBI-flavored DSN parameters.

hostname is required for EZconnect, but may not be for DBI syntax, hence part of my reasoning for preferring 9fd8bfa over 160af71. Can you confirm whether 9fd8bfa works when there is no host name and the database name is a SID?

@vectro
Copy link
Author

vectro commented May 5, 2023

Can you confirm whether 9fd8bfa works when there is no host name and the database name is a SID?

Confirmed, thanks!

I also tried this with a port number but no host, and that resulted in the "Can't connect using this syntax" error from DBD::Oracle.

@theory
Copy link
Collaborator

theory commented May 6, 2023

Thank you. Presumably the result is the same when using an EZCONNECT URL with a port and no host name, yes?

If so, I think we're good here, since the presence of a port with no host name doesn't appear to be supported by any DSN syntax.

@vectro
Copy link
Author

vectro commented May 7, 2023

I did not double check the behavior when it is missing, but hostname is documented to be required with EZCONNECT.

@theory
Copy link
Collaborator

theory commented May 8, 2023

Yes, I saw that, too. I think we're good to go here, will merge and release.

@theory theory closed this as completed in 9fd8bfa May 9, 2023
@theory
Copy link
Collaborator

theory commented May 9, 2023

v0.21 now on CPAN.

@vectro
Copy link
Author

vectro commented Feb 24, 2024

So, I'm very embarrassed to say this, but I think may have provided incorrect information earlier, and unfortunately I think this is still not working for all cases. Should I open a new issue? Or reopen this one?

Earlier I wrote this:

Can you confirm whether 9fd8bfa works when there is no host name and the database name is a SID?

Confirmed, thanks!

However, I think that was actually wrong, and I'm not able to reproduce this now. Maybe it's possible that I tested the wrong code by mistake?

Looking more closely at the source code for DBD::Oracle, it's not possible to use service_name by itself. The code in DBD::Oracle requires a ; to parse the service_name, and also requires that the host is set.

I think that probably brings us back to this comment? But I could be wrong.

@theory theory reopened this Feb 24, 2024
@theory
Copy link
Collaborator

theory commented Feb 24, 2024

I don't know, and not being an Oracle user, it'd be very helpful to me if someone just explained what the correct behavior should be.

If you're willing and able to try some experiments, you can use this script:

#!/usr/bin/env perl -w

use strict;
use warnings;
use v5.20;
use DBI;

my ($dsn, $user, $pass) = @ARGV;
die "Usage $0 DSN [USERNAME [PASSWORD]]\n" unless $dsn;

my $dbh = DBI->connect($dsn, $user, $pass, {
    RaiseError => 1,
    PrintError => 0,
    AutoCommit => 1,
});

say $dbh->selectcol_arrayref("SELECT 'connected' FROM DUAL")->[0];

Save it and call it like this:

try_connect 'dbi:Oracle:service_name:foo;host=bar'

Be sure to quote the DSN to prevent the shell from interpreting characters like ;.

If it prints "connected" then it connected and successfully ran a query. Pass a username and password as the second and third args if you need them. Try different combinations and figure out what works and what doesn't, with different TNSNAMES.ora settings with SIDs, service_names, and whatever other combination we have to deal with. If you can figure out the proper incantation and report back, that would be super helpful.

Thanks!

@vectro
Copy link
Author

vectro commented Feb 26, 2024

I'm happy to do that. Results are below; let me know if there are other configurations which would be helpful to test.

Use case: Connecting to a SID DEV via tnsnames.ora

Works:

  • dbi:Oracle:DEV
  • dbi:Oracle:sid=DEV

Does not work:

  • dbi:Oracle:sid=DEV; ("Can't connect using this syntax without specifying a HOST")
  • dbi:Oracle:///DEV or dbi:Oracle://DEV ("ORA-12154: TNS:could not resolve the connect identifier specified")
  • dbi:Oracle:service_name=DEV ("ORA-12154: TNS:could not resolve the connect identifier specified")

The DBD::Oracle documentation describes a use case of specifying sid and host together, but I wasn't able to make that work.

Use case: Connecting to a database running on the local host, on port 49152, with the service name XEPDB1

Works:

  • dbi:Oracle:host=localhost;service_name=XEPDB1;port=49152
  • dbi:Oracle://localhost:49152/XEPDB1

Does not work:

  • dbi:Oracle:service_name=XEPDB1;port=49152 ("Can't connect using this syntax without specifying a HOST")
  • dbi:Oracle:host=localhost;sid=XEPDB1;port=49152 ("ORA-12505: TNS:listener does not currently know of SID given in connect descriptor")
  • dbi:Oracle://:49152/XEPDB1 ("ORA-12154: TNS:could not resolve the connect identifier specified")

@theory
Copy link
Collaborator

theory commented Feb 29, 2024

Does the service name XEPDB1 not specify a host name and port? Does this work?

db:Oracle:XEPDB1

@vectro
Copy link
Author

vectro commented Mar 1, 2024

Does this work?

It does not, because in this use case XEPDB1 does not appear in tnsnames.ora. However, we do have these other use cases:

Use case: Connecting to a database running on the local host, on port 1521, with the service name XEPDB1

Works:

  • dbi:Oracle://localhost/XEPDB1
  • dbi:Oracle:service_name=XEPDB1;host=localhost

Does not work:

  • dbi:Oracle:XEPDB1 ("ORA-12154: TNS:could not resolve the connect identifier specified")
    • I am not sure why this doesn't work; the DBD::Oracle documentation says that it should.
  • dbi:Oracle:///XEPDB1 ("ORA-12154: TNS:could not resolve the connect identifier specified")
  • dbi:Oracle:sid=XEPDB1 ("ORA-12154: TNS:could not resolve the connect identifier specified")
  • dbi:Oracle:service_name=XEPDB1 ("ORA-12154: TNS:could not resolve the connect identifier specified")
  • dbi:Oracle:sid=XEPDB1; ("Can't connect using this syntax without specifying a HOST")
  • dbi:Oracle:service_name=XEPDB1; ("Can't connect using this syntax without specifying a HOST")
  • dbi:Oracle:sid=XEPDB1;host=localhost ("ORA-12505: TNS:listener does not currently know of SID given in connect descriptor")
  • dbi:Oracle:service_name=XEPDB1;host=localhost ("ORA-12505: TNS:listener does not currently know of SID given in connect descriptor")

Use case: Connecting to a SID MYSID, wheretnsnames.ora does not specify an address, but the database is running on the local host, on port 1521, with the service name XEPDB1

In this use case we have the following in tnsnames.ora:

MYSID=
  (DESCRIPTION =
    (CONNECT_DATA =
      (SERVICE_NAME = XEPDB1)
    )
  )

Works:

  • dbi:Oracle:SID=MYSID;host=localhost;service_name=xepdb1
  • dbi:Oracle:host=localhost;service_name=xepdb1

Does not work:

  • dbi:Oracle:SID=MYSID;host=localhost ("ORA-12505: TNS:listener does not currently know of SID given in connect descriptor")
    *dbi:Oracle:MYSID ("ORA-12533: TNS:illegal ADDRESS parameters (DBD ERROR: OCIServerAttach)")

@theory
Copy link
Collaborator

theory commented Mar 2, 2024

Well I have no bloody idea what the right solution is for converting URI to the appropriate DBD::Oracle DSN. But am I right in thinking that a service name requires a host name to work, but a SID may not? Uf you have a SID, does service_name=$sid work?

@vectro
Copy link
Author

vectro commented Mar 2, 2024

If you have a SID, does service_name=$sid work?

I think that is the dbi:Oracle:service_name=DEV example above?

Well I have no bloody idea what the right solution is

One path forward may be something similar to (or the same as) what Oracle did for the JDBC URI.

@vectro
Copy link
Author

vectro commented Mar 8, 2024

Also does this earlier suggestion not work?

  1. If there is not a hostname or port, just put the raw SID or service name (this is way 1 in the DBD::Oracle documentation)
  2. If there is a hostname, use the EZCONNECT syntax (way 3 in the DBD::Oracle documentation)

@theory
Copy link
Collaborator

theory commented Mar 11, 2024

Okay, please try this:

diff --git a/lib/URI/oracle.pm b/lib/URI/oracle.pm
index 4520c20..481f7d2 100644
--- a/lib/URI/oracle.pm
+++ b/lib/URI/oracle.pm
@@ -5,13 +5,26 @@ our $VERSION = '0.22';
 sub default_port { 1521 }
 sub dbi_driver   { 'Oracle' }
 
-sub _dbi_param_map {
+sub _dsn_params {
     my $self = shift;
-    return (
-        [ host => scalar $self->host   ],
-        [ port => scalar $self->_port  ],
-        [ service_name => scalar $self->dbname ],
-    );
+    my $name = $self->dbname || '';
+    my $dsn = $self->host || return $name;
+
+    if (my $p = $self->_port) {
+        $dsn .= ":$port";
+    }
+
+    $dsn .= "/$name";
+
+    if (my @p = $self->query_params) {
+        my @kvpairs;
+        while (@p) {
+            push @kvpairs => join '=', shift @p, shift @p;
+        }
+        $dsn .= '?' . join ';' => @kvpairs;
+    }
+
+    return "//$dsn";
 }
 
 1;

It assumes that URL query parameters should be included in the EZCONNECT string. I don't know if that's actually a thin in Oracle, but if not then one is going to use URL parameters anyway, so should, be harmless.

@vectro
Copy link
Author

vectro commented Mar 13, 2024

This patch works for me, at least. I'm able to connect to both (a) a SID and (b) a service name + host name.

It assumes that URL query parameters should be included in the EZCONNECT string.

Looking at the documentation for EZCONNECT, I see that it does support parameter values and also that there is a ton of other complexity there.

    <connect_identifier> can be in the form of Net Service Name
    or Easy Connect.

      @[<net_service_name> | [[//]Host[:Port]/<service_name>] |
	[[[protocol:]//]host1{,host12}[:port1]{,host2:port2}[/service_name]
	 [:server][/instance_name][?[parameter_name=value]
	 {&parameter_name=value}]]]

@theory
Copy link
Collaborator

theory commented Mar 13, 2024

Looks like plain old URL query params, which URI::db an of course provide.

theory added a commit that referenced this issue Mar 31, 2024
Again. It now works as follows:

-   If there is not a hostname or port, just use the raw SID or service
    name as the database name: `dbi:Oracle:$name`. This is way 1 in the
    DBD::Oracle documentation
-   If there is a hostname or port, use the EZCONNECT syntax. This is
    way 3 in the DBD::Oracle documentation.

Use of a port without a host name may not be valid, but it seems most
prudent to build an EZCONNECT that includes the port in this context and
to let Oracle or DBD::Oracle reject it if appropriate. Thanks again to
@vectro for the and diligence, testing, and patience with this issue
(#22).
theory added a commit that referenced this issue Mar 31, 2024
Again. It now works as follows:

-   If there is not a hostname or port, just use the raw SID or service
    name as the database name: `dbi:Oracle:$name`. This is way 1 in the
    DBD::Oracle documentation
-   If there is a hostname or port, use the EZCONNECT syntax. This is
    way 3 in the DBD::Oracle documentation.

Use of a port without a host name may not be valid, but it seems most
prudent to build an EZCONNECT that includes the port in this context and
to let Oracle or DBD::Oracle reject it if appropriate. Thanks again to
@vectro for the and diligence, testing, and patience with this issue
(#22).
@theory
Copy link
Collaborator

theory commented Mar 31, 2024

Okay, see what you think of #25.

theory added a commit that referenced this issue Apr 5, 2024
Again. It now works as follows:

-   If there is not a hostname or port, just use the raw SID or service
    name as the database name: `dbi:Oracle:$name`. This is way 1 in the
    DBD::Oracle documentation
-   If there is a hostname or port, use the EZCONNECT syntax. This is
    way 3 in the DBD::Oracle documentation.
-   If there are query parameters, delimit them by `&` and not `;`.

Use of a port without a host name may not be valid, but it seems most
prudent to build an EZCONNECT that includes the port in this context and
to let Oracle or DBD::Oracle reject it if appropriate. Thanks again to
@vectro for the and diligence, testing, and patience with this issue
(#22).
@theory theory closed this as completed in #25 Apr 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants