Odo Function with encoding error: ValueError: Unsupported string encoding u'SQL_Latin1_General_CP1_CI_AS'

[Rahuketu86]

df = odo(url_source+"::NibrasEBSElementMap", pd.DataFrame, encoding='cp1512')
and
df = odo(url_source+"::NibrasEBSElementMap", pd.DataFrame, encoding='latin1')
results in following error

ValueError Traceback (most recent call last)
in ()
3 url_source = 'mssql+pyodbc:///?odbc_connect={}'.format(quoted)
4
----> 5 df = odo(url_source+"::NibrasEBSElementMap", pd.DataFrame, encoding='cp1512')
6 df

D:\Rahuketu\Anaconda\lib\site-packages\odo\odo.pyc in odo(source, target, *_kwargs)
89 odo.append.append - Add things onto existing things
90 """
---> 91 return into(target, source, *_kwargs)

D:\Rahuketu\Anaconda\lib\site-packages\multipledispatch\dispatcher.pyc in call(self, _args, *_kwargs)
162 self._cache[types] = func
163 try:
--> 164 return func(_args, *_kwargs)
165
166 except MDNotImplementedError:

D:\Rahuketu\Anaconda\lib\site-packages\odo\into.pyc in wrapped(_args, *_kwargs)
41 raise TypeError('dshape argument is not an instance of DataShape')
42 kwargs['dshape'] = dshape
---> 43 return f(_args, *_kwargs)
44 return wrapped
45

D:\Rahuketu\Anaconda\lib\site-packages\odo\into.pyc in into_string_string(a, b, *_kwargs)
147 @Validate
148 def into_string_string(a, b, *_kwargs):
--> 149 return into(a, resource(b, *_kwargs), *_kwargs)
150
151

D:\Rahuketu\Anaconda\lib\site-packages\multipledispatch\dispatcher.pyc in call(self, _args, *_kwargs)
162 self._cache[types] = func
163 try:
--> 164 return func(_args, *_kwargs)
165
166 except MDNotImplementedError:

D:\Rahuketu\Anaconda\lib\site-packages\odo\into.pyc in wrapped(_args, *_kwargs)
41 raise TypeError('dshape argument is not an instance of DataShape')
42 kwargs['dshape'] = dshape
---> 43 return f(_args, *_kwargs)
44 return wrapped
45

D:\Rahuketu\Anaconda\lib\site-packages\odo\into.pyc in into_type(a, b, dshape, *_kwargs)
50 with ignoring(NotImplementedError):
51 if dshape is None:
---> 52 dshape = discover(b)
53 return convert(a, b, dshape=dshape, *_kwargs)
54

D:\Rahuketu\Anaconda\lib\site-packages\multipledispatch\dispatcher.pyc in call(self, _args, *_kwargs)
162 self._cache[types] = func
163 try:
--> 164 return func(_args, *_kwargs)
165
166 except MDNotImplementedError:

D:\Rahuketu\Anaconda\lib\site-packages\odo\backends\sql.pyc in discover_sqlalchemy_selectable(t)
201 def discover_sqlalchemy_selectable(t):
202 ordering = dict((c, i) for i, c in enumerate(c for c in t.columns.keys()))
--> 203 records = list(sum([discover(c).parameters[0] for c in t.columns], ()))
204 fkeys = [discover(fkey, t, parent_measure=Record(records))
205 for fkey in t.foreign_keys]

D:\Rahuketu\Anaconda\lib\site-packages\multipledispatch\dispatcher.pyc in call(self, _args, *_kwargs)
162 self._cache[types] = func
163 try:
--> 164 return func(_args, *_kwargs)
165
166 except MDNotImplementedError:

D:\Rahuketu\Anaconda\lib\site-packages\odo\backends\sql.pyc in discover_sqlalchemy_column(c)
195 def discover_sqlalchemy_column(c):
196 meta = Option if c.nullable else identity
--> 197 return Record([(c.name, meta(discover(c.type)))])
198
199

D:\Rahuketu\Anaconda\lib\site-packages\multipledispatch\dispatcher.pyc in call(self, _args, *_kwargs)
162 self._cache[types] = func
163 try:
--> 164 return func(_args, *_kwargs)
165
166 except MDNotImplementedError:

D:\Rahuketu\Anaconda\lib\site-packages\odo\backends\sql.pyc in discover_typeengine(typ)
173 return datashape.Decimal(precision=typ.precision, scale=typ.scale)
174 if isinstance(typ, (sa.String, sa.Unicode)):
--> 175 return datashape.String(typ.length, typ.collation)
176 else:
177 for k, v in revtypes.items():

D:\Rahuketu\Anaconda\lib\site-packages\datashape\coretypes.pyc in init(self, *args)
397 except KeyError:
398 raise ValueError('Unsupported string encoding %s' %
--> 399 repr(encoding))
400
401 self.encoding = encoding

ValueError: Unsupported string encoding u'SQL_Latin1_General_CP1_CI_AS'

[daefresh]

Hi,
Has this been fixed by anyone? We're having the same problem. It's prohibiting us from using it in production for SQL Server. Does anyone have a suggestion? I can't change the collation order of my database.
Thanks

[dhirschfeld]

Having the same issue here with a SQL Server db - ValueError: Unsupported string encoding 'Latin1_General_CI_AS'

[KhaoticMind]

+1 With this same error.
In my case, I can change the collation, but I don't see what I can set it to make it work.

The code that breaks is on datashape\coretype.py, looking for the collation on the following dictionary:

_canonical_string_encodings = {
    u'A': u'A',
    u'ascii': u'A',
    u'U8': u'U8',
    u'utf-8': u'U8',
    u'utf_8': u'U8',
    u'utf8': u'U8',
    u'U16': u'U16',
    u'utf-16': u'U16',
    u'utf_16': u'U16',
    u'utf16': u'U16',
    u'U32': u'U32',
    u'utf-32': u'U32',
    u'utf_32': u'U32',
    u'utf32': u'U32'
}

Since SqlServer doesn't support UTF I can't see what I can set the collation to to make it work...

[daefresh]

Are there any updates on this? We're still having an issue with this and we would like to use odo but this is prohibiting us from using it. It would be fantastic if this were prioritized and resolved. Any chances that can happen?

[dhirschfeld]

See also blaze/datashape#176

[thequackdaddy]

For me, the fix in #430 fixed this.

[llllllllll]

Good to close?

[dhirschfeld]

I've been running a locally patched version but thought I'd try to update to the latest to see if this is fixed.

It appears that #430 does fix this issue so I'm happy that this can be closed now that's merged. I'll just note that there's no release yet which incorporates the fix in #430 so I had to keep running odo from source to verify.