You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Aug 31, 2021. It is now read-only.
sc = SparkContext(appName="reading csv")
# creating the context
sqlContext = SQLContext(sc)
dynamoDf = sparkSession.read.option("tableName", "event-poc").format("dynamodb").load()
# Scan the table for the first 100 items (the order is arbitrary) and print them.
dynamoDf.show(10)
# write to some other table overwriting existing item with same keys
dynamoDf.write.option("tableName", "emr-test").format("dynamodb").save()
The text was updated successfully, but these errors were encountered:
angeluzdomini
changed the title
Ho to import the module in pyspark?
How to import the module in pyspark?
Feb 9, 2021
I am trying to import in pyspark in EMR and not able to find the module. It is getting downloaded though:
Arguments :spark-submit --deploy-mode cluster --packages com.audienceproject:spark-dynamodb_2.12:1.1.1 s3://filepath
import com.audienceproject.spark.dynamodb_
if name == 'main':
The text was updated successfully, but these errors were encountered: