Skip to content
This repository has been archived by the owner on Aug 31, 2021. It is now read-only.

In python following readme example but unable to print out any data #84

Open
ahsan-n opened this issue Nov 23, 2020 · 3 comments
Open

Comments

@ahsan-n
Copy link

ahsan-n commented Nov 23, 2020

version: spark-dynamodb_2.12:1.1.0
dynamoDf = spark.read.option("tableName", "mytable").option("region","eu-west-1") \ .format("dynamodb") \ .load() dynamoDf.show(100)

this gives me empty table

++ || ++ || || ++

but when i use load().count() it showed me the right count of total entries in table.

@Kaifee-Mohammad
Copy link

yeah, it happened to me as well, trying passing the schema.

@angeluzdomini
Copy link

@ahsan-n can you tell how you are importing the module? Getting error while importing. Was able to download it as mentioned in readme.

@programmeravi
Copy link

programmeravi commented May 11, 2021

I have the same error as above, get empty table in return

Have given the schema and got it working

from pyspark.sql.types import *

fields = [StructField("email", StringType(), True)
          , StructField("first_name", StringType(), True)
          , StructField("last_name", StringType(), True)]

schema = StructType(fields)

# Load data from dynamo into spark dataframe
dynamoDf = spark.read.option("tableName","db_dev_users") \
                     .option("region",region) \
                     .schema(schema) \
                     .format("dynamodb") \
                     .load() 

Have been using this in the databricks environment, and installed the packages from Maven repo
version 1.1.2, and it works

Thanks @kaifee

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Development

No branches or pull requests

4 participants