TutorTask86_Spring2025_Real_Time_Bitcoin_Price_Analysis_Using_Amazon_EMR_2

[RithikaBaskaran]

This PR includes the main scripts and notebooks for the project titled "Real-Time Bitcoin Price Analysis Using Amazon EMR", under TutorTask86_Spring2025.

Files added:

• bitcoin_ingest.API.py: Pulls real-time Bitcoin price data from an API
• bitcoin_processing.example.py: Handles transformation and aggregation of raw price data
• bitcoin_streaming.example.py: Implements real-time processing using PySpark
• bitcoin_timeseries.example.py: Performs basic time-series analytics
• spark_test.py: A lightweight Spark job to validate setup
• template.API.ipynb: Exploratory notebook with annotated logic and tests

Path:
DATA605/Spring2025/projects/TutorTask86_Spring2025_Real_Time_Bitcoin_Price_Analysis_Using_Amazon_EMR/

The README.md file will be added in a follow-up commit.

[Prahar08modi]

I would like you to make the following changes:

• Write all your utility/wrapper functions in utility file (for eg., emr_utils.py) like fetch_bitcoin_price, save_record, etc.
• Convert all your .py files to namely 2 jupyter notebooks ( for eg., bitcoin_emr.API.ipynb and bitcoin_emr.example.ipynb).
• The notebooks should only call the functions written in emr_utils.py file.
• Apart from these changes, you should also add markdown files for your project.

Note: Please got through the required files for submission here

[RithikaBaskaran]

Hi, I wanted to let you know that I've completed all the requested updates for PR #252 ( TutorTask86_Spring2025_Real_Time_Bitcoin_Price_Analysis_Using_Amazon_EMR_2). The project now fully reflects the real-time pipeline, and includes: - Refactored utility functions in bitcoin_emr_utils.py - Both Jupyter notebooks: bitcoin_emr.API.ipynb and bitcoin_emr.example.ipynb - Completed markdown files: bitcoin_emr.API.md and bitcoin_emr.example.md - Updated README to reflect the real-time architecture Everything has been tested, and the data is successfully writing to S3. Please let me know if anything else is needed. I’d appreciate your review whenever convenient. Thank you! Best regards, Rithika Baskaran

…

On Wed, May 14, 2025 at 7:04 PM Prahar Modi ***@***.***> wrote: ***@***.**** requested changes on this pull request. I would like you to make the following changes: - Write all your utility/wrapper functions in utility file (for eg., emr_utils.py) like fetch_bitcoin_price, save_record, etc. - Convert all your .py files to namely 2 jupyter notebooks ( for eg., bitcoin_emr.API.ipynb and bitcoin_emr.example.ipynb). - The notebooks should only call the functions written in emr_utils.py file. - Apart from these changes, you should also add markdown files for your project. *Note*: Please got through the required files for submission here <https://github.com/causify-ai/tutorials/tree/master/DATA605#submission> — Reply to this email directly, view it on GitHub <#252 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AUAFARKN463BNUA3LA6LGB326PDW3AVCNFSM6AAAAAB3P2PBN6VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDQNBRG4YDEMJWGM> . You are receiving this because you were assigned.Message ID: ***@***.***>

[RithikaBaskaran]

This PR includes the final working version of my project: Real-Time Bitcoin Price Analysis Using Amazon EMR.

Key Updates:

• Real-time pipeline using CoinGecko API → S3 → Spark on EMR
• Producer and consumer scripts refactored and tested
• All reusable logic moved to bitcoin_emr_utils.py
• Updated notebooks, markdowns, and final README.md
• Removed older/placeholder files from early commits

📁 Final Files:

• bitcoin_kafka/bitcoin_producer.py
• bitcoin_kafka/bitcoin_streaming_consumer_emr_debug.py
• bitcoin_kafka/bitcoin_emr_utils.py
• bitcoin_emr.API.ipynb, bitcoin_emr.example.ipynb
• bitcoin_emr.API.md, bitcoin_emr.example.md
• README.md

[RithikaBaskaran]

Final Submission: Real-Time Bitcoin Price Analysis Using Amazon EMR

This PR includes the final, tested version of my project including:

• Real-time pipeline from CoinGecko API → S3 → Spark on EMR
• Updated notebooks:
  • bitcoin_emr.API.ipynb (utility functions with fallback)
  • bitcoin_emr.example.ipynb (simulated full pipeline)
• Final README.md with:
  • Docker setup
  • AWS credential handling
  • EMR cluster instructions
• Docker setup files:
  • Dockerfile, run_jupyter.sh, docker_build.sh, docker_bash.sh

All code runs end-to-end in Docker. AWS interactions gracefully fallback when credentials are missing.

Ready for grading.

Merging