Skip to content

Commit 898d08c

Browse files
committed
Merge #278: Refactor bdk-cli structure
63f769a revert parallel request wallet parameter (Vihiga Tyonum) 8dc0503 drop verbose flag (Vihiga Tyonum) 5da7660 feat(handlers): mv feats into subdirs in handlers (Vihiga Tyonum) 5951c17 feat(dns-payment): port dns payment instruction (Vihiga Tyonum) b3fe860 feat(sp): fix command_requires_db fn match arm (Vihiga Tyonum) c2ca2dd ref(payjoin): add payjoin persistence (Vihiga Tyonum) 249948e ref(compile):compile tr desc with rand unspend key (Vihiga Tyonum) 6a6390f feat(WalletEvents): Add wallet events to clients (Vihiga Tyonum) cb9634f ref(create_tx): enforce single recp for sendall (Vihiga Tyonum) 188e68d refactor(runtime): Add address and createtx to db (Vihiga Tyonum) 75b5141 refactor(sp-payment): Rebase silent payment feat (Vihiga Tyonum) aa8876d refactor(main): add runtime wallet module (Vihiga Tyonum) 63162be refactor(handlers): Apply app context (Vihiga Tyonum) f3deed8 Refactor(handlers): Define app context states (Vihiga Tyonum) 5ced5aa refactor(output): apply generics to output mod (Vihiga Tyonum) 1b05c03 ref(handlers): fix offline, online and desc mod (Vihiga Tyonum) dcaeb55 ref(handlers): refactor config, descr and key mods (Vihiga Tyonum) f77ddcc ref(main): add run to handle routing (Vihiga Tyonum) 528956d ref(pretty): remove `--pretty` flag (Vihiga Tyonum) cf9d13b ref(persister): collapse wallet subdir to persister (Vihiga Tyonum) 7a0e68b update namespace and update types (Vihiga Tyonum) eaa4e11 revert mod names for command and clients (Vihiga Tyonum) ada3485 ref(utils): use types in desc output in utils (Vihiga Tyonum) 90017cd ref(handlers): rebase bip322 feature (Vihiga Tyonum) 990f30a ref(types): add simple table helper (Vihiga Tyonum) 055e1fd ref(handlers): add types for desc, key & wallets (Vihiga Tyonum) a973273 refactor(handlers): Add types for outputting data (Vihiga Tyonum) 3e2eec8 ref(main): update main entry point (Vihiga Tyonum) 0bf3e66 ref(handlers): split handler fns into repl (Vihiga Tyonum) bf0551b ref(handlers):mv handler fns into offline & others (Vihiga Tyonum) 03307c9 ref(handlers): split handlers into config, key (Vihiga Tyonum) eb3a6aa ref(persister): mv persister into wallet subdir (Vihiga Tyonum) 2482044 ref(utils): refactor utils into utils subdir (Vihiga Tyonum) 739dec3 ref(config): move config into config subdir (Vihiga Tyonum) 5c286d6 ref(commands): mv commands & clients into subdirs (Vihiga Tyonum) Pull request description: <!-- You can erase any parts of this template not applicable to your Pull Request. --> ### Description This PR restructures the bdk-cli codebase to move away from a monolithic handling model towards a modular architecture. The primary goal is to improve code readability, simplify the addition of new features, and decouple core logic into smaller modules. Fixes #273 <!-- Describe the purpose of this PR, what's being adding and/or fixed --> ### Notes to the reviewers While trying to achieve the above-stated goal, I took maximum care to avoid introducing new bugs. As a result, the existing execution logic was preserved as much as possible. <!-- In this section you can include notes directed to the reviewers, like explaining why some parts of the PR were done in a specific way --> This PR introduces two new subdirectories under `src`: - handlers: This contains all the subcommands' execution logic grouped logically according to the top-level commands. Modules include the `config`, `descriptor`, `key`, `offline`, `online` and `repl` modules. They each contain structures for the subcommand, execution logic, and implementation of the top-level enum. - utils: This subdirectory contains helper functions used across the app. They are also grouped logically according to related functionality. So far, we have the descriptor-related module, the output that formats responses, and the `types` module that defines response structures. **Other notable changes** - main module is retained with added functionality of routing requests to the right module for execution. - All client-related functionalities have been moved from various modules into the `client` module, and it defines the enum that holds all the available clients, as well as implements the functionalities such as broadcasting transactions and syncing operations. - The `AppCommand` trait that defines how each subcommand should be structured and the `execute` method they should implement. - The `AppContext` struct that holds the universal parameters that subcommands may need. Each subcommand's `execute` method accepts this struct and uses the params for its execution. - The output module in the util subdirectory attempts to present a uniform presentation layer by having an output trait, the `FormatOutput` trait, which is implemented by all `types` presenting data response to the user. - Collapsed the `wallet` subdirectory into the `persister` module that defines all structures and functionalities related to persistence. - Introduced a type state pattern for the AppContext: To minimise runtime errors, the `Init`, `OfflineOperations` and `OnlineOperations` state structs were introduced, and AppContext is generic over these states. This moved the validation from runtime to compile time. - Introduced `FormatOutput::write_out` and made it generic over the Write trait to allow for flexibility in testing - Extracted all initialisation logic (database, config loading and persistence initialisation) into `runtime` module. Also, added the `RuntimeWallet` enum to return options of a persisted wallet and a standard non-persisted wallet. The WalletRuntime struct is a wrapper that determines the type of wallet needed and initialises it and handles loading of a wallet from config. **Request-Response Cycle** The following outlines how requests are handled and responses are generated after introducing changes to the structure: 1. When a user enters a set of commands, those commands are received and parsed by the Clap library into Rust structures, such as `CliOpts`, `WalletOpts`, and subcommands. This process takes place in the `commands` module. 2. The parsed request is then passed to the `main` module. The main module uses the `runtime` module to interpret the request and initialize any necessary resources (such as wallets, databases, and clients) needed to fulfill the request. It then routes the request to the appropriate handler for execution. 3. Once a particular handler receives the request along with all the required resources, it processes the specific command using the `AppContext` and prepares a response. 4. After processing is complete, the handler calls the `output` module to format the result, serialize it into JSON, and return it to the terminal. ## Changelog notice - In the `key derive` subcommand, when printing help message, the clap arg for env=“PATH” often pull all system and displays all system paths. This has been changed to derivation_path I.e. env=“DERIVATION_PATH” to prevent clap from picking the system paths and rendering it - Moved the wallet subdirectory that served as persister into the persister module - Moved DatabaseType enum from commands module into persister module - Removed the —pretty flag <!-- Notice the release manager should include in the release tag message changelog --> <!-- See https://keepachangelog.com/en/1.0.0/ for examples --> ### Checklists #### All Submissions: * [x] I've signed all my commits * [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk-cli/blob/master/CONTRIBUTING.md) * [x] I ran `cargo fmt` and `cargo clippy` before committing Top commit has no ACKs. Tree-SHA512: e63ed0693e2580fe80fca278e0ad1f7feacccda32f3e7262d57cff1bdf4e9850e2463dfeee1cbcfad4ce1502fd19d3861d5012705e051d19831def8832b6780a
2 parents d8e1801 + 63f769a commit 898d08c

31 files changed

Lines changed: 4070 additions & 3816 deletions

Cargo.lock

Lines changed: 141 additions & 187 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,6 @@ tracing = "0.1.44"
2626
tracing-subscriber = "0.3.20"
2727
toml = "1.1.0"
2828
serde= {version = "1.0", features = ["derive"]}
29-
shlex = "1.3.0"
3029
tap = "1.0.1"
3130

3231
# Optional dependencies
@@ -36,6 +35,7 @@ bdk_esplora = { version = "0.22.2", features = ["async-https", "tokio"], optiona
3635
bdk_kyoto = { version = "0.15.4", optional = true }
3736
bdk_redb = { version = "0.1.1", optional = true }
3837
bdk_sp = { version = "0.1.0", optional = true, git = "https://github.com/bitcoindevkit/bdk-sp", tag = "v0.1.0" }
38+
shlex = { version = "1.3.0", optional = true }
3939
payjoin = { version = "0.25.0", features = ["v1", "v2", "io", "_test-utils"], optional = true}
4040
reqwest = { version = "0.13.2", default-features = false, features = ["rustls"], optional = true }
4141
url = { version = "2.5.8", optional = true }
@@ -46,7 +46,7 @@ bitcoin-payment-instructions = { version = "0.7.0", optional = true}
4646
default = ["repl", "sqlite"]
4747

4848
# To use the app in a REPL mode
49-
repl = []
49+
repl = ["shlex"]
5050

5151
# Available database options
5252
sqlite = ["bdk_wallet/rusqlite"]

README.md

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -282,15 +282,6 @@ Note: You can modify the `Justfile` to reflect your nodes' configuration values.
282282
cargo run --features rpc -- wallet -w regtest1 balance
283283
```
284284

285-
## Formatting Responses using `--pretty` flag
286-
287-
You can optionally return outputs of commands in human-readable, tabular format instead of `JSON`. To enable this option, simply add the `--pretty` flag as a top level flag. For instance, you wallet's balance in a pretty format, you can run:
288-
289-
```shell
290-
cargo run -- --pretty -n signet wallet -w {wallet_name} balance
291-
```
292-
This is available for wallet, key, repl and compile features. When ommitted, outputs default to `JSON`.
293-
294285
## Shell Completions
295286

296287
`bdk-cli` supports generating shell completions for Bash, Zsh, Fish, Elvish, and PowerShell. For setup instructions, run:

src/client.rs

Lines changed: 330 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,330 @@
1+
#[cfg(feature = "esplora")]
2+
use bdk_esplora::EsploraAsyncExt;
3+
#[cfg(any(
4+
feature = "electrum",
5+
feature = "esplora",
6+
feature = "rpc",
7+
feature = "cbf"
8+
))]
9+
use {
10+
crate::commands::WalletOpts,
11+
crate::error::BDKCliError as Error,
12+
bdk_wallet::{
13+
Wallet,
14+
bitcoin::{Transaction, Txid},
15+
},
16+
clap::ValueEnum,
17+
std::path::PathBuf,
18+
};
19+
#[cfg(feature = "rpc")]
20+
use {
21+
bdk_bitcoind_rpc::{Emitter, bitcoincore_rpc::RpcApi},
22+
bdk_wallet::chain::CanonicalizationParams,
23+
};
24+
25+
#[cfg(feature = "cbf")]
26+
use {
27+
crate::utils::trace_logger,
28+
bdk_kyoto::{BuilderExt, LightClient},
29+
};
30+
31+
#[cfg(any(
32+
feature = "electrum",
33+
feature = "esplora",
34+
feature = "rpc",
35+
feature = "cbf"
36+
))]
37+
#[derive(Clone, ValueEnum, Debug, Eq, PartialEq)]
38+
pub enum ClientType {
39+
#[cfg(feature = "electrum")]
40+
Electrum,
41+
#[cfg(feature = "esplora")]
42+
Esplora,
43+
#[cfg(feature = "rpc")]
44+
Rpc,
45+
#[cfg(feature = "cbf")]
46+
Cbf,
47+
}
48+
49+
#[cfg(any(
50+
feature = "electrum",
51+
feature = "esplora",
52+
feature = "rpc",
53+
feature = "cbf"
54+
))]
55+
pub(crate) enum BlockchainClient {
56+
#[cfg(feature = "electrum")]
57+
Electrum {
58+
client: Box<bdk_electrum::BdkElectrumClient<bdk_electrum::electrum_client::Client>>,
59+
batch_size: usize,
60+
},
61+
#[cfg(feature = "esplora")]
62+
Esplora {
63+
client: Box<bdk_esplora::esplora_client::AsyncClient>,
64+
parallel_requests: usize,
65+
},
66+
#[cfg(feature = "rpc")]
67+
RpcClient {
68+
client: Box<bdk_bitcoind_rpc::bitcoincore_rpc::Client>,
69+
},
70+
71+
#[cfg(feature = "cbf")]
72+
KyotoClient { client: Box<KyotoClientHandle> },
73+
}
74+
75+
#[cfg(any(
76+
feature = "electrum",
77+
feature = "esplora",
78+
feature = "rpc",
79+
feature = "cbf"
80+
))]
81+
impl BlockchainClient {
82+
pub async fn broadcast(&self, tx: Transaction) -> Result<Txid, Error> {
83+
match self {
84+
#[cfg(feature = "electrum")]
85+
Self::Electrum { client, .. } => client
86+
.transaction_broadcast(&tx)
87+
.map_err(|e| Error::Generic(e.to_string())),
88+
89+
#[cfg(feature = "esplora")]
90+
Self::Esplora { client, .. } => client
91+
.broadcast(&tx)
92+
.await
93+
.map(|()| tx.compute_txid())
94+
.map_err(|e| Error::Generic(e.to_string())),
95+
96+
#[cfg(feature = "rpc")]
97+
Self::RpcClient { client } => client
98+
.send_raw_transaction(&tx)
99+
.map_err(|e| Error::Generic(e.to_string())),
100+
101+
#[cfg(feature = "cbf")]
102+
Self::KyotoClient { client } => {
103+
let txid = tx.compute_txid();
104+
let wtxid = client
105+
.requester
106+
.broadcast_random(tx.clone())
107+
.await
108+
.map_err(|_| {
109+
tracing::warn!("Broadcast was unsuccessful");
110+
Error::Generic("Transaction broadcast timed out after 30 seconds".into())
111+
})?;
112+
tracing::info!("Successfully broadcast WTXID: {wtxid}");
113+
Ok(txid)
114+
}
115+
}
116+
}
117+
118+
pub async fn sync_wallet(&self, wallet: &mut Wallet) -> Result<(), Error> {
119+
#[cfg(any(feature = "electrum", feature = "esplora"))]
120+
let request = wallet
121+
.start_sync_with_revealed_spks()
122+
.inspect(|item, progress| {
123+
let pc = (100 * progress.consumed()) as f32 / progress.total() as f32;
124+
eprintln!("[ SCANNING {pc:03.0}% ] {item}");
125+
});
126+
match self {
127+
#[cfg(feature = "electrum")]
128+
Self::Electrum { client, batch_size } => {
129+
// Populate the electrum client's transaction cache so it doesn't re-download transaction we
130+
// already have.
131+
client.populate_tx_cache(wallet.tx_graph().full_txs().map(|tx_node| tx_node.tx));
132+
133+
let update = client.sync(request, *batch_size, false)?;
134+
wallet
135+
.apply_update(update)
136+
.map_err(|e| Error::Generic(e.to_string()))
137+
}
138+
#[cfg(feature = "esplora")]
139+
Self::Esplora {
140+
client,
141+
parallel_requests,
142+
} => {
143+
let update = client
144+
.sync(request, *parallel_requests)
145+
.await
146+
.map_err(|e| *e)?;
147+
wallet
148+
.apply_update(update)
149+
.map_err(|e| Error::Generic(e.to_string()))
150+
}
151+
#[cfg(feature = "rpc")]
152+
Self::RpcClient { client } => {
153+
let blockchain_info = client.get_blockchain_info()?;
154+
let wallet_cp = wallet.latest_checkpoint();
155+
156+
// reload the last 200 blocks in case of a reorg
157+
let emitter_height = wallet_cp.height().saturating_sub(200);
158+
let mut emitter = Emitter::new(
159+
client.as_ref(),
160+
wallet_cp,
161+
emitter_height,
162+
wallet
163+
.tx_graph()
164+
.list_canonical_txs(
165+
wallet.local_chain(),
166+
wallet.local_chain().tip().block_id(),
167+
CanonicalizationParams::default(),
168+
)
169+
.filter(|tx| tx.chain_position.is_unconfirmed()),
170+
);
171+
172+
while let Some(block_event) = emitter.next_block()? {
173+
if block_event.block_height() % 10_000 == 0 {
174+
let percent_done = f64::from(block_event.block_height())
175+
/ f64::from(blockchain_info.headers as u32)
176+
* 100f64;
177+
println!(
178+
"Applying block at height: {}, {:.2}% done.",
179+
block_event.block_height(),
180+
percent_done
181+
);
182+
}
183+
184+
wallet.apply_block_connected_to(
185+
&block_event.block,
186+
block_event.block_height(),
187+
block_event.connected_to(),
188+
)?;
189+
}
190+
191+
let mempool_txs = emitter.mempool()?;
192+
wallet.apply_unconfirmed_txs(mempool_txs.update);
193+
Ok(())
194+
}
195+
#[cfg(feature = "cbf")]
196+
Self::KyotoClient { client } => sync_kyoto_client(wallet, client)
197+
.await
198+
.map_err(|e| Error::Generic(e.to_string())),
199+
}
200+
}
201+
}
202+
203+
/// Handle for the Kyoto client after the node has been started.
204+
/// Contains only the components needed for sync and broadcast operations.
205+
#[cfg(feature = "cbf")]
206+
pub struct KyotoClientHandle {
207+
pub requester: bdk_kyoto::Requester,
208+
pub update_subscriber: tokio::sync::Mutex<bdk_kyoto::UpdateSubscriber>,
209+
}
210+
211+
#[cfg(any(
212+
feature = "electrum",
213+
feature = "esplora",
214+
feature = "rpc",
215+
feature = "cbf",
216+
))]
217+
/// Create a new blockchain from the wallet configuration options.
218+
pub(crate) fn new_blockchain_client(
219+
wallet_opts: &WalletOpts,
220+
_wallet: &Wallet,
221+
_datadir: PathBuf,
222+
) -> Result<BlockchainClient, Error> {
223+
#[cfg(any(feature = "electrum", feature = "esplora", feature = "rpc"))]
224+
let url = &wallet_opts.url;
225+
let client = match wallet_opts.client_type {
226+
#[cfg(feature = "electrum")]
227+
ClientType::Electrum => {
228+
let client = bdk_electrum::electrum_client::Client::new(url)
229+
.map(bdk_electrum::BdkElectrumClient::new)?;
230+
BlockchainClient::Electrum {
231+
client: Box::new(client),
232+
batch_size: wallet_opts.batch_size,
233+
}
234+
}
235+
#[cfg(feature = "esplora")]
236+
ClientType::Esplora => {
237+
let client = bdk_esplora::esplora_client::Builder::new(url).build_async()?;
238+
BlockchainClient::Esplora {
239+
client: Box::new(client),
240+
parallel_requests: wallet_opts.parallel_requests,
241+
}
242+
}
243+
244+
#[cfg(feature = "rpc")]
245+
ClientType::Rpc => {
246+
let auth = match &wallet_opts.cookie {
247+
Some(cookie) => bdk_bitcoind_rpc::bitcoincore_rpc::Auth::CookieFile(cookie.into()),
248+
None => bdk_bitcoind_rpc::bitcoincore_rpc::Auth::UserPass(
249+
wallet_opts.basic_auth.0.clone(),
250+
wallet_opts.basic_auth.1.clone(),
251+
),
252+
};
253+
let client = bdk_bitcoind_rpc::bitcoincore_rpc::Client::new(url, auth)
254+
.map_err(|e| Error::Generic(e.to_string()))?;
255+
BlockchainClient::RpcClient {
256+
client: Box::new(client),
257+
}
258+
}
259+
260+
#[cfg(feature = "cbf")]
261+
ClientType::Cbf => {
262+
let scan_type = bdk_kyoto::ScanType::Sync;
263+
let builder = bdk_kyoto::builder::Builder::new(_wallet.network());
264+
265+
let light_client = builder
266+
.required_peers(wallet_opts.compactfilter_opts.conn_count)
267+
.data_dir(&_datadir)
268+
.build_with_wallet(_wallet, scan_type)?;
269+
270+
let LightClient {
271+
requester,
272+
info_subscriber,
273+
warning_subscriber,
274+
update_subscriber,
275+
node,
276+
} = light_client;
277+
278+
let subscriber = tracing_subscriber::FmtSubscriber::new();
279+
let _ = tracing::subscriber::set_global_default(subscriber);
280+
281+
tokio::task::spawn(async move { node.run().await });
282+
tokio::task::spawn(
283+
async move { trace_logger(info_subscriber, warning_subscriber).await },
284+
);
285+
286+
BlockchainClient::KyotoClient {
287+
client: Box::new(KyotoClientHandle {
288+
requester,
289+
update_subscriber: tokio::sync::Mutex::new(update_subscriber),
290+
}),
291+
}
292+
}
293+
};
294+
Ok(client)
295+
}
296+
297+
// Handle Kyoto Client sync
298+
#[cfg(feature = "cbf")]
299+
pub async fn sync_kyoto_client(
300+
wallet: &mut Wallet,
301+
handle: &KyotoClientHandle,
302+
) -> Result<(), Error> {
303+
if !handle.requester.is_running() {
304+
tracing::error!("Kyoto node is not running");
305+
return Err(Error::Generic("Kyoto node failed to start".to_string()));
306+
}
307+
tracing::info!("Kyoto node is running");
308+
309+
let update = handle.update_subscriber.lock().await.update().await?;
310+
tracing::info!("Received update: applying to wallet");
311+
let events = wallet
312+
.apply_update_events(update)
313+
.map_err(|e| Error::Generic(format!("Failed to apply update: {e}")))?;
314+
crate::utils::print_wallet_events(&events);
315+
316+
tracing::info!(
317+
"Chain tip: {}, Transactions: {}, Balance: {}",
318+
wallet.local_chain().tip().height(),
319+
wallet.transactions().count(),
320+
wallet.balance().total().to_sat()
321+
);
322+
323+
tracing::info!(
324+
"Sync completed: tx_count={}, balance={}",
325+
wallet.transactions().count(),
326+
wallet.balance().total().to_sat()
327+
);
328+
329+
Ok(())
330+
}

0 commit comments

Comments
 (0)