GitHub - CameronBeebe/Monte_Carlo: This is a short reference example for a common parallelization exercise using MPI and Pyspark. It is a Monte Carlo estimation of Pi.

CameronBeebe / Monte_Carlo Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

This is a short reference example for a common parallelization exercise using MPI and Pyspark. It is a Monte Carlo estimation of Pi.

MIT license

0 stars 0 forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 34 Commits
.gitattributes		.gitattributes
LICENSE.txt		LICENSE.txt
Monte_Carlo_Pi.ipynb		Monte_Carlo_Pi.ipynb
Pi_MPI.py		Pi_MPI.py
Pi_Pyspark.py		Pi_Pyspark.py
README.txt		README.txt
mpi_pyspark.yaml		mpi_pyspark.yaml
requirements.txt		requirements.txt

Repository files navigation

This is a short reference example for a common parallelization exercise.  It is a Monte Carlo estimation of Pi.  One implementation is in MPI, the other is in Pyspark to compare.  The scripts should be run in the environment created from the .yaml or requirements.txt files, or another suitable environment.  

Code is not optimal, but tries to be clear and demonstrative (can be improved still).  

Pi_MPI.py can be run on the command line in the environment outlined in mpi_min.yaml using commands of the form "mpirun -n 4 python Pi_MPI.py -pts 1000".

Pi_Pyspark.py can be run on the command line using commands of the form "python Pi_Pyspark.py -pts 1000 -procs 4".

TO DO:

1.  Figure out how to have Pyspark ranks/workers spit out confirmations of local work being done like in MPI.

2.  Compare.