Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1479655: VECTOR datatype is not supported #975

Closed
myshon opened this issue Jun 13, 2024 · 2 comments
Closed

SNOW-1479655: VECTOR datatype is not supported #975

myshon opened this issue Jun 13, 2024 · 2 comments
Assignees
Labels
invalid status-triage_done Initial triage done, will be further handled by the driver team

Comments

@myshon
Copy link

myshon commented Jun 13, 2024

  1. What version of .NET driver are you using?

3.1.0

  1. What operating system and processor architecture are you using?

MAC OS / M2.

  1. What version of .NET framework are you using?

net 8.0

  1. What did you do?

I created table

create or replace TABLE RAG.CLUSTERS (
	OBJECT_ID VARCHAR(36) NOT NULL,
	TEXT VARCHAR(16777216) NOT NULL,
	EMBEDDINGS VECTOR(FLOAT, 1024) NOT NULL,
);

I retrieve data from the table using follow code

public class ClusterDto
{
    public required string ObjectId { get; set; }
    public required string Text { get; set; }
    public required string Embeddings { get; set; }
}

public interface ISemanticService
{
    Task<IReadOnlyCollection<ClusterDto>> GetClustersAsync();
}

public class SemanticSnowflakeService : ISemanticService
{
    private readonly SnowflakeDbConnectionProvider _connectionProvider;

    public SemanticSnowflakeService(SnowflakeDbConnectionProvider connectionProvider)
    {
        _connectionProvider = connectionProvider;
    }

    public async Task<IReadOnlyCollection<ClusterDto>> GetClustersAsync()
    {
        await using var conn = await _connectionProvider.GetConnection();
        await conn.OpenAsync();
        await using var cmd = conn.CreateCommand();
        cmd.CommandText = @$"SELECT OBJECT_ID, TEXT, EMBEDDINGS FROM ACME_DB.RAG.CLUSTERS limit 10;";

        // Error: Snowflake Internal Error: Unknow column type: vector
        await using var reader = await cmd.ExecuteReaderAsync();
        var result = new List<ClusterDto>();
        while (await reader.ReadAsync())
        {
            result.Add(new ClusterDto()
            {
                ObjectId = reader.GetString(0),
                Text = reader.GetString(1),
                Embeddings = reader.GetString(2)
            });
        }

        await conn.CloseAsync();
        return result;
    }
}

Reader throws error with message ❌
// Error: Snowflake Internal Error: Unknow column type: vector

  1. What did you expect to see?

I need to retrieve vector as string/array/json or whatever.

❓ Is there any workaround?

@myshon myshon added the bug label Jun 13, 2024
@github-actions github-actions bot changed the title VECTOR datatype is not supported SNOW-1479655: VECTOR datatype is not supported Jun 13, 2024
@sfc-gh-dszmolka sfc-gh-dszmolka self-assigned this Jun 17, 2024
@sfc-gh-dszmolka sfc-gh-dszmolka added invalid status-triage_done Initial triage done, will be further handled by the driver team and removed bug labels Jun 17, 2024
@sfc-gh-dszmolka
Copy link
Contributor

hello - for the moment, unfortunately the error message is correct: https://docs.snowflake.com/en/sql-reference/data-types-vector#data-types

Note
The VECTOR data type is only supported in SQL, the Python connector and the Snowpark Python library. No other languages are supported.

we'll announce in the driver changelogs once it becomes supported. Thank you for bearing with us !

@sfc-gh-dszmolka
Copy link
Contributor

also for the ones interested in this support, besides the changelogs , PR #1002 can be followed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
invalid status-triage_done Initial triage done, will be further handled by the driver team
Projects
None yet
Development

No branches or pull requests

2 participants