Skip to content

Commit 87ea9fc

Browse files
NE7KNE7K
authored
Revise project overview and training results in README
1 parent 8fd05b8 commit 87ea9fc

1 file changed

Lines changed: 98 additions & 84 deletions

File tree

โ€ŽREADME.mdโ€Ž

Lines changed: 98 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,49 @@
11
<img src="https://github.com/user-attachments/assets/ca28df82-b7fe-467c-a86e-113321e0b5a9" width="100%" height="100%"> </img>
22

3-
## ๐Ÿ“– Project Overview
3+
## ๐Ÿ“– Overview
44

5-
Analysis and Summary of US Stock Market Sentiment Indicators and Market Data
5+
๋ฏธ๊ตญ ์ฆ์‹œ๋Š” ์†Œ์…œ ๋ฏธ๋””์–ด์˜ ๋ฐœ๋‹ฌ๋กœ ์ธํ•ด ์ •๋ณด ํ™•์‚ฐ ์†๋„๊ฐ€ ๊ทน๋„๋กœ ๋นจ๋ผ์กŒ์œผ๋ฉฐ, ๋‹จ์ผ ๋‰ด์Šค ๊ธฐ์‚ฌ๋‚˜ ํŠธ์œ—์ด ๋‹จ๊ธฐ์ ์ธ ๋ณ€๋™์„ฑ์„ ์œ ๋ฐœํ•˜๋Š” ์‚ฌ๋ก€๊ฐ€ ๋นˆ๋ฒˆํ•˜๊ฒŒ ๋ฐœ์ƒํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ์ด์— ๋ณธ ํ”„๋กœ์ ํŠธ๋Š” ๋‰ด์Šค, ๋ธ”๋กœ๊ทธ, ๋Œ“๊ธ€ ๋“ฑ๊ณผ ๊ฐ™์€ ๋น„์ •ํ˜• ๋ฐ์ดํ„ฐ๋ฅผ ์ž์—ฐ์–ด ์ฒ˜๋ฆฌ ๊ธฐ๋ฐ˜ ์ธ๊ณต์ง€๋Šฅ ๋ชจ๋ธ์„ ํ™œ์šฉํ•ด ๋ถ„์„ํ•จ์œผ๋กœ์จ, ์‹œ์žฅ ์ฐธ์—ฌ์ž๋“ค์˜ ์‹ฌ๋ฆฌ์  ๋ฐ˜์‘์„ ์ •๋Ÿ‰์ ์œผ๋กœ ํ‰๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
66

7-
## ๐Ÿ–ฅ๏ธ Training environment
7+
๋” ๋‚˜์•„๊ฐ€, ๋‹จ์ˆœํ•œ ๊ฐ์„ฑ ๋ถ„์„์„ ๋„˜์–ด ์‹œ์žฅ ๊ณ ํ‰๊ฐ€/์ €ํ‰๊ฐ€ ์—ฌ๋ถ€๋ฅผ ๋ฐ˜์˜ํ•˜๋Š” VIX ๋ณ€๋™์„ฑ ์ง€ํ‘œ, ๊ณผ๋งค์ˆ˜ยท๊ณผ๋งค๋„ ํŒ๋‹จ์„ ์œ„ํ•œ RSI ๋ณด์กฐ์ง€ํ‘œ, ์ด๋™ํ‰๊ท ์„  ๊ธฐ๋ฐ˜ ์ถ”์„ธ ์ง€ํ‘œ ๋“ฑ์„ ํ†ตํ•ฉ์ ์œผ๋กœ ๊ณ ๋ คํ•ฉ๋‹ˆ๋‹ค. ์ด๋ฅผ ํ†ตํ•ด ์‹œ์žฅ ์‹ฌ๋ฆฌ๋ฅผ 0์—์„œ 100 ์‚ฌ์ด์˜ ๊ฐ’์œผ๋กœ ํ™˜์‚ฐํ•œ Market Sentiment Index๋ฅผ ์‚ฐ์ถœํ•ฉ๋‹ˆ๋‹ค.
88

9-
```
10-
Tensorflow 2.10
9+
๋ณธ ์‹œ์Šคํ…œ์€ ์ „๋ฌธ ํˆฌ์ž ์ง€์‹์ด ๋ถ€์กฑํ•œ ๊ฐœ์ธ ํˆฌ์ž์ž๋ฟ ์•„๋‹ˆ๋ผ, ์‹œ๊ฐ„ ์ œ์•ฝ์œผ๋กœ ์ธํ•ด ์‹œ์žฅ ๋ชจ๋‹ˆํ„ฐ๋ง์ด ์–ด๋ ค์šด ํˆฌ์ž์ž๋“ค์—๊ฒŒ๋„ ์‹ค์งˆ์ ์ธ ์˜์‚ฌ๊ฒฐ์ • ๋ณด์กฐ ๋„๊ตฌ๋กœ ๊ธฐ๋Šฅํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
10+
11+
The U.S. stock market has become highly sensitive to the rapid spread of information driven by the rise of social media, where a single news article or tweet can frequently trigger short-term volatility. To address this, the project leverages a natural language processing (NLP)โ€“based artificial intelligence model to analyze unstructured data such as news articles, blogs, and comments, thereby quantifying market participantsโ€™ psychological responses.
12+
13+
Beyond sentiment analysis alone, the system also incorporates key market indicators, including the VIX volatility index to capture overvaluation and undervaluation, the RSI oscillator to identify overbought and oversold conditions, and moving averages to assess price trends. These elements are integrated to calculate a Market Sentiment Index, expressed as a numerical value ranging from 0 to 100.
14+
15+
This system is designed not only for individual investors with limited expertise in financial indicators but also for those constrained by time, providing a practical decision-support tool to better understand the prevailing psychological state of the U.S. stock market.
16+
17+
## Project file structure
1118

12-
Cpu : AMD 5600
13-
Ram : 32GB
14-
Gpu : RTX 4060 8GB
1519
```
16-
## ๐Ÿ“ Data Set 1 : Dataset used in the sentiment model
20+
upstock-python/
21+
โ”‚โ”€โ”€ upstock/
22+
โ”‚ โ”‚
23+
โ”‚ โ”œโ”€โ”€ builders/
24+
โ”‚ โ”‚ โ”œโ”€โ”€ pipeline.py # pipline builder
25+
โ”‚ โ”‚
26+
โ”‚ โ”œโ”€โ”€ indicators/
27+
โ”‚ โ”‚ โ”œโ”€โ”€ core.py # calculation logic
28+
โ”‚ โ”‚ โ”œโ”€โ”€ indexer.py # Indicator indexing/management
29+
โ”‚ โ”‚
30+
โ”‚ โ”œโ”€โ”€ models/
31+
โ”‚ โ”‚ โ”œโ”€โ”€ artifacts.py # Model artifact management
32+
โ”‚ โ”‚
33+
โ”‚ โ”œโ”€โ”€ nodes/
34+
โ”‚ โ”‚ โ”œโ”€โ”€ predict.py # Predict node -> News predict part, market predict part
35+
โ”‚ โ”‚ โ”œโ”€โ”€ train.py # Train node
36+
โ”‚ โ”‚
37+
โ”‚ โ”œโ”€โ”€ storage/
38+
โ”‚ โ”‚ โ”œโ”€โ”€ downloader.py # Data download management
39+
โ”‚ โ”‚ โ”œโ”€โ”€ market_data.py # Stock Data Processing
40+
โ”‚ โ”‚
41+
โ”‚ โ”œโ”€โ”€ config.py # Environment
42+
โ”‚
43+
โ”‚โ”€โ”€ main.py # Main entry point
44+
```
45+
46+
## ๐Ÿ“ Data Set : Dataset used in the sentiment model
1747

1848
| Text | Label |
1949
|--------------------------------------------------------------------------------------------------------------------------|-------|
@@ -29,98 +59,82 @@ Gpu : RTX 4060 8GB
2959
doi: [10.34740/kaggle/dsv/1217821](https://doi.org/10.34740/kaggle/dsv/1217821)
3060
License: Data files ยฉ Original Authors
3161

32-
## ๐Ÿ“ Data Set 2
62+
## Sentiment Model Training result and history
3363

34-
```
35-
```
64+
| Epoch | Loss | Accuracy | Val Loss | Val Accuracy |
65+
|-------|--------|----------|----------|--------------|
66+
| 1 | 0.6352 | 0.6516 | 0.5148 | 0.7498 |
67+
| 2 | 0.3841 | 0.8314 | 0.4343 | 0.8007 |
68+
| 3 | 0.1724 | 0.9359 | 0.5599 | 0.7869 |
69+
| 4 | 0.0806 | 0.9743 | 0.7665 | 0.7826 |
70+
| 5 | 0.0433 | 0.9853 | 0.9666 | 0.7627 |
3671

37-
## Sentiment Model Training result and layers status
72+
**Early Stopping:** Best epoch โ†’ **2**
3873

39-
```
40-
Epoch 1/20
41-
loss: 0.6352 - accuracy: 0.6516 - val_loss: 0.5148 - val_accuracy: 0.7498
74+
## Sentiment Model summary()
4275

43-
Epoch 2/20
44-
loss: 0.3841 - accuracy: 0.8314 - val_loss: 0.4343 - val_accuracy: 0.8007
76+
| Layer (type) | Output Shape | Param # |
77+
|------------------------|----------------|---------|
78+
| InputLayer | (None, 141) | 0 |
79+
| Embedding | (None, 141,128)| 1,304,192 |
80+
| Bidirectional(LSTM) | (None, 141,128)| 98,816 |
81+
| GlobalMaxPooling1D | (None, 128) | 0 |
82+
| Dense | (None, 64) | 8,256 |
83+
| Dropout | (None, 64) | 0 |
84+
| Dense | (None, 32) | 2,080 |
85+
| Dense | (None, 1) | 33 |
86+
87+
**Total params:** 1,413,377 (5.39 MB)
88+
**Trainable params:** 1,413,377
89+
**Non-trainable params:** 0
4590

46-
Epoch 3/20
47-
loss: 0.1724 - accuracy: 0.9359 - val_loss: 0.5599 - val_accuracy: 0.7869
91+
## System configuration diagram
4892

49-
Epoch 4/20
50-
loss: 0.0806 - accuracy: 0.9743 - val_loss: 0.7665 - val_accuracy: 0.7826
93+
<img width="100%" alt="system diagram" src="https://github.com/user-attachments/assets/1fe24cfe-0da4-4d8e-bc22-7ada93908529" />
5194

52-
Epoch 5/20
53-
loss: 0.0433 - accuracy: 0.9853 - val_loss: 0.9666 - val_accuracy: 0.7627
95+
## Predict result saved in Supabase Table
5496

55-
loss: 0.0433 - accuracy: 0.9852
56-
Restoring model weights from the end of the best epoch: 2.
57-
Epoch 5: early stopping
58-
```
97+
์˜ˆ์ธก ๊ฒฐ๊ณผ๋Š” News Sentiment์™€ Market Sentiment Index ํ…Œ์ด๋ธ”๋กœ ๋‚˜๋ˆ„์–ด์„œ ์ €์žฅ๋˜๊ณ  ๋ณด๊ด€๋˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.
5998

60-
## Sentiment Model summary()
99+
### News Sentiment Table
61100

62-
```
63-
_________________________________________________________________
64-
Layer (type) Output Shape Param #
65-
=================================================================
66-
model_input (InputLayer) [(None, 141)] 0
67-
68-
embedding (Embedding) (None, 141, 128) 1304192
69-
70-
bidirectional (Bidirection (None, 141, 128) 98816
71-
al)
72-
73-
global_max_pooling1d (Glob (None, 128) 0
74-
alMaxPooling1D)
75-
76-
dense (Dense) (None, 64) 8256
77-
78-
dropout (Dropout) (None, 64) 0
79-
80-
dense_1 (Dense) (None, 32) 2080
81-
82-
dense_2 (Dense) (None, 1) 33
83-
84-
=================================================================
85-
Total params: 1413377 (5.39 MB)
86-
Trainable params: 1413377 (5.39 MB)
87-
Non-trainable params: 0 (0.00 Byte)
88-
_________________________________________________________________
89-
```
101+
```SQL
102+
SELECT text, percent, label FROM news_sentiment LIMIT 10
90103

91-
## Sentiment Model Predict
92104
```
93-
1/1 [==============================] - 0s 281ms/step
94-
[negative] EM portfolios funnel near $45 billion in August but cracks are showing, IIF says
95-
: 0.52
96-
97-
[positive] Stocks' Bull Market Nears 3-Year Anniversary. It Likely Has More Room to Run.
98-
: 0.87
99-
100-
[negative] Stock Market Today: Dow Slides As Oracle Soars; Medicare News Hits Health Leader
101-
: 0.19
102-
103-
[negative] Stock Market Today: Dow and Nasdaq fall, S&P 500 loses momentum ahead of August consumer-price index on Thursday; Oracle share surge highlights technology spending
104-
: 0.67
105-
106-
[positive] Oracle stock booms 35%, on pace for best day since 1992
107-
: 0.86
108-
105+
| text | percent | label |
106+
| ------------------------------------------------------------------------------------------------------------------------------- | --------- | -------- |
107+
| Dow jumps 400 points to record as August inflation increase likely won't derail Fed rate cut | 0.0548475 | negative |
108+
| Dow jumps and S&P touches all-time high while Treasury yields fall as Wall Street reacts to rising inflation and jobless claims | 0.25916 | negative |
109+
| The Fedโ€™s 2% inflation target might not be hit for years, says Janus portfolio manager โ€” unless thereโ€™s a recession | 0.295348 | negative |
110+
| Dow, S&P 500 and Nasdaq push deeper into record territory | 0.88824 | positive |
111+
| Stocks, Bonds Rise as Data Seal September Fed Cut: Markets Wrap | 0.813471 | positive |
112+
| Mexico hikes China auto tariff, S. Korea warns on US investment | 0.141901 | negative |
113+
| Stocks, Gold Set New Records in Run-Up to Fed Meet: Markets Wrap | 0.821933 | positive |
114+
| Vietnam Urges US to Rethink Seafood Ban as Trade Talks Grind On | 0.212265 | negative |
115+
| Stocks climb toward record closes | 0.813072 | positive |
116+
| Shell LNG Plant Wins Place on Carneyโ€™s List of Favored Projects | 0.924645 | positive |
117+
118+
### Market Sentiment Index Table
119+
120+
```SQL
121+
SELECT date_utc, score, zone, rsi, vix, macd_val FROM market_sentiment_index LIMIT 10
109122
```
123+
| date_utc | score | zone | rsi | vix | macd_val |
124+
| ---------- | ----- | ------- | --------------- | ---------------- | ---------------- |
125+
| 2025-09-18 | 58 | Neutral | 67.892115296438 | 15.5600004196167 | 6.48942949393609 |
110126

127+
## ๐Ÿ–ฅ๏ธ Training environment
111128

112-
## System configuration diagram
113-
114-
<img width="100%" alt="system diagram" src="https://github.com/user-attachments/assets/1fe24cfe-0da4-4d8e-bc22-7ada93908529" />
115-
116-
## System Detail Diagram
117-
118-
<img width="100%" alt="UserFlow" src="https://github.com/user-attachments/assets/ed7e5638-796c-4820-8bde-9148ea4b2da3" />
119-
120-
## ๐Ÿ” WBS
129+
์ดˆ๊ธฐ๋ชจ๋ธ ํ•™์Šต ์‹œ๊ฐ„์€ ๋Œ€๋žต 4์‹œ๊ฐ„, ๋‘ ๋ฒˆ์งธ ๋ชจ๋ธ์€ ํ•™์Šต ์‹œ๊ฐ„ ๋Œ€๋žต 1์‹œ๊ฐ„, ํ˜„์žฌ ๋ชจ๋ธ ํ•™์Šต ์‹œ๊ฐ„ 1๋ถ„
121130

122-
-
131+
```
132+
Tensorflow 2.10
123133
134+
Cpu : AMD 5600
135+
Ram : 32GB
136+
Gpu : RTX 4060 8GB
137+
```
124138

125139
## ๐Ÿ“ง Contact
126140

0 commit comments

Comments
ย (0)