various improvements/refactors of the decoder and its errors#845
Merged
Conversation
This is a little less space efficient but it gets rid of a lifetime parameter on the lexer token, which is annoying because it prevents us from putting it into error types. (Currently we convert it to a string before putting it into an error.) It also eliminates a whole bunch of panics where we convert slices to hashes and .expect() on the length being right, when we know it's right.
Also clean up the names and format messages for them.
…onstructors We have infallible constructors for all the terminals except multi/multia. We should use them and eliminate a bunch of error paths.
The word "parse" is redundant at best, and wrong at worst (we use "decode" when converting a script and "parse" when converting a string). The variants were also overly verbose and the formatting text lost information. Clean all this up.
We should backport this one to 12.x (with a deprecation). Would also be open to doing a deprecation here in master. But this has been bugging me for years, and also it's much less common to decode miniscript from script than we imagined back in 2018. Provides symmetry with encode() and consistency with the docs (though this PR does not fix up the docs).
apoelstra
force-pushed
the
2025-07/decode
branch
from
5cca410 to
cdb36c1
Compare
heap-coder
added a commit
to heap-coder/rust-miniscript
that referenced
this pull request
Sep 27, 2025
…s of the decoder and its errors
cdb36c19b536ef8d8716dd1423e7e6b96433101e miniscript: rename parse_* functions to decode_* (Andrew Poelstra)
5fea2b3cee70f02b18f1d6fbba7b777ab329e7b1 miniscript: rename and cleanup decode::KeyParseError (Andrew Poelstra)
a57fab3ca26a20140a90b8ca66408c9246fdf9cd miniscript: replace many calls to reduce0 in decode with infallible constructors (Andrew Poelstra)
5b736bdede89b15866659c4d38383db4af4240cd miniscript: move lexer errors into their own type (Andrew Poelstra)
ceadd0d4d98bf8f7bc2a80f1727769cb9e442f9a miniscript: use byte arrays in lexer (Andrew Poelstra)
Pull request description:
The last commit, which renames `parse_with_ext` to `decode_with_ext`, I think we should backport with deprecation to 12.x. This is a pretty uncommonly used function and is unlikely to bother anybody, but the current name is inconsistent with our general use of "parse" for strings and "decode" for scripts.
ACKs for top commit:
sanket1729:
ACK cdb36c19b536ef8d8716dd1423e7e6b96433101e
Tree-SHA512: 8a3563dd23e49cf9331b299e237337a3d1097c15d5fe143ec9aa3f686d681fda077f019b25f0d66fccdd9a88ec489a84e144fcbb28410a671eb113f38e0fe58d
apoelstra
added a commit
to apoelstra/rust-miniscript
that referenced
this pull request
Oct 20, 2025
…nsensus Currently we have decode() and decode_insane() (which were both called parse_* before PR rust-bitcoin#845). In practice though, throughout the codebase we see that when decoding from script we want to relax more than sanity. In particular, we frequently with ExtParams::allow_all, which beyond relaxing the sanity rules, also allows raw pubkeyhashes. We do this in the script interpreter and in PSBT, on the basis that once we're working with a script, we should just deal with whatever the library is capable of dealing with. (Is this the right decision? IMO yes. It could be argued. But regardless, it's the decision we've made for the past many versions of rust-miniscript.) I also propose to backport this with deprecation, telling users that if they really want parse_insane, they now need to call decode_with_ext and specify ExtParams::insane. But if their goal was "parse anything that might be allowed", they actually want decode_consensus. There is now an asymmetry between parsing from string and parsing from script: with strings we have parse() and parse_insane(). With script we have decode() and decode_consensus(). I think this is correct, and reflects the fact that somebody "breaking the rules" with a string is likely trying to use syntactically valid Miniscripts without the library whining at him, while somebody "breaking the rules" with a Script is probably trying to deal with some immutable on-chain thing and wants the library to work if at-all possible. Right now the distinction is simply "do we support raw pkh or not" but I think we could accept more-or-less arbitrary extensions to Miniscript in this library that worked the same way (allowed when decoded from script but disallowed from strings since there's no serialization).
apoelstra
added a commit
to apoelstra/rust-miniscript
that referenced
this pull request
Oct 26, 2025
…nsensus Currently we have decode() and decode_insane() (which were both called parse_* before PR rust-bitcoin#845). In practice though, throughout the codebase we see that when decoding from script we want to relax more than sanity. In particular, we frequently with ExtParams::allow_all, which beyond relaxing the sanity rules, also allows raw pubkeyhashes. We do this in the script interpreter and in PSBT, on the basis that once we're working with a script, we should just deal with whatever the library is capable of dealing with. (Is this the right decision? IMO yes. It could be argued. But regardless, it's the decision we've made for the past many versions of rust-miniscript.) I also propose to backport this with deprecation, telling users that if they really want parse_insane, they now need to call decode_with_ext and specify ExtParams::insane. But if their goal was "parse anything that might be allowed", they actually want decode_consensus. There is now an asymmetry between parsing from string and parsing from script: with strings we have parse() and parse_insane(). With script we have decode() and decode_consensus(). I think this is correct, and reflects the fact that somebody "breaking the rules" with a string is likely trying to use syntactically valid Miniscripts without the library whining at him, while somebody "breaking the rules" with a Script is probably trying to deal with some immutable on-chain thing and wants the library to work if at-all possible. Right now the distinction is simply "do we support raw pkh or not" but I think we could accept more-or-less arbitrary extensions to Miniscript in this library that worked the same way (allowed when decoded from script but disallowed from strings since there's no serialization).
apoelstra
added a commit
to apoelstra/rust-miniscript
that referenced
this pull request
Oct 26, 2025
…nsensus Currently we have decode() and decode_insane() (which were both called parse_* before PR rust-bitcoin#845). In practice though, throughout the codebase we see that when decoding from script we want to relax more than sanity. In particular, we frequently with ExtParams::allow_all, which beyond relaxing the sanity rules, also allows raw pubkeyhashes. We do this in the script interpreter and in PSBT, on the basis that once we're working with a script, we should just deal with whatever the library is capable of dealing with. (Is this the right decision? IMO yes. It could be argued. But regardless, it's the decision we've made for the past many versions of rust-miniscript.) I also propose to backport this with deprecation, telling users that if they really want parse_insane, they now need to call decode_with_ext and specify ExtParams::insane. But if their goal was "parse anything that might be allowed", they actually want decode_consensus. There is now an asymmetry between parsing from string and parsing from script: with strings we have parse() and parse_insane(). With script we have decode() and decode_consensus(). I think this is correct, and reflects the fact that somebody "breaking the rules" with a string is likely trying to use syntactically valid Miniscripts without the library whining at him, while somebody "breaking the rules" with a Script is probably trying to deal with some immutable on-chain thing and wants the library to work if at-all possible. Right now the distinction is simply "do we support raw pkh or not" but I think we could accept more-or-less arbitrary extensions to Miniscript in this library that worked the same way (allowed when decoded from script but disallowed from strings since there's no serialization).
nyonson
pushed a commit
to rust-bitcoin/rust-psbt
that referenced
this pull request
Jun 4, 2026
* build: bump miniscript to 13.0.0 | * ci: allow hex-conservative duplicates | * miniscript: replace decode_with_ext(., Ext::allow_all) with decode_consensus | | Currently we have decode() and decode_insane() (which were both called | parse_* before PR #845). In practice though, throughout the codebase we | see that when decoding from script we want to relax more than sanity. | | In particular, we frequently with ExtParams::allow_all, which beyond | relaxing the sanity rules, also allows raw pubkeyhashes. We do this in | the script interpreter and in PSBT, on the basis that once we're working | with a script, we should just deal with whatever the library is capable | of dealing with. (Is this the right decision? IMO yes. It could be | argued. But regardless, it's the decision we've made for the past many | versions of rust-miniscript.) | | I also propose to backport this with deprecation, telling users that if | they really want parse_insane, they now need to call decode_with_ext and | specify ExtParams::insane. But if their goal was "parse anything that | might be allowed", they actually want decode_consensus. | | There is now an asymmetry between parsing from string and parsing from | script: with strings we have parse() and parse_insane(). With script we | have decode() and decode_consensus(). I think this is correct, and | reflects the fact that somebody "breaking the rules" with a string is | likely trying to use syntactically valid Miniscripts without the library | whining at him, while somebody "breaking the rules" with a Script is | probably trying to deal with some immutable on-chain thing and wants the | library to work if at-all possible. | | Right now the distinction is simply "do we support raw pkh or not" but I | think we could accept more-or-less arbitrary extensions to Miniscript in | this library that worked the same way (allowed when decoded from script | but disallowed from strings since there's no serialization). | | Port from rust-bitcoin/rust-miniscript#871 | * miniscript: rename parse_* functions to decode_* | | We should backport this one to 12.x (with a deprecation). Would also be | open to doing a deprecation here in master. But this has been bugging me | for years, and also it's much less common to decode miniscript from | script than we imagined back in 2018. | | Provides symmetry with encode() and consistency with the docs (though | this PR does not fix up the docs). | | Port from rust-bitcoin/rust-miniscript#845 | * miniscript: use hex-conservative 1.0 everywhere for decoding | | It turns out we only ever use do decoding of fixed test vectors. It | would be nice to have a hex! macro for this. Eventually. | | In some cases we were able to use inherent methods on bitcoin types -- | for example ScriptBuf::from_hex or deserialize_hex. | | Port from rust-bitcoin/rust-miniscript#822 | * key(miniscript): replace ConversionError with NonDefiniteKeyError | | The new error type is now more descriptive and a strict superset | of the old one. Furthermore, the new name is more appropriate since | this error typically arises during construction of a DefiniteDescriptorKey | rather than during conversion between different key types. | | Port from rust-bitcoin/rust-miniscript#839 | | keys(miniscript): remove impossible unwrap()s | | Now that we are consistent about the rules that a DerivedDescriptor must | obey, we no longer return a Result from derived_key(), which in turn | eliminates a bunch of other unwraps. | | Port from rust-bitcoin/rust-miniscript#839 | * macros(miniscript): make target pk and error type optional for translate_* macros | | There is no reason callers need to provide these types explicitly. Allow | them to do so anyway for backward compatibility. | | Port from rust-bitcoin/rust-miniscript#839 | * tr(miniscript): add conversion from TrSpendInfo to bitcoin::TapTree | | Moves a bit of ugly logic out of the PSBT module into the spendinfo | module so that it's available for other users. We can convert from a | TrSpendInfo to a bitcoin::TapTree but we can't do the opposite | conversion since TrSpendInfo expects to have a Miniscript for each leaf. | | Port from rust-bitcoin/rust-miniscript#815 | * tr(miniscript): replace `bitcoin::TaprootSpendInfo` with `TrSpendInfo`, update psbt | | This commit stops using the rust-bitcoin `TaprootSpendInfo` in favor of | our new `TrSpendInfo` structure. This one allows us to iterate over all | the leaves of the Taptree, easily accessing their leaf hashes and | control blocks in order, which simplifies satisfaction logic. | | Port from rust-bitcoin/rust-miniscript#815 | * psbt(miniscript): untangle some logic in `update_item_with_descriptor_helper` | | We have the function `update_item_with_descriptor_helper` which does a | few things: it derives a descriptor (replacing all the xpubs with actual | public keys) and updates the appropriate input or output map to map the | derived keys to their keysources. | | It treats Tr outputs differently from other kinds of outputs, because | the relevant maps are different. However, in doing so, it duplicates a | bunch of work in ways that are hard to follow. | | Essentially, the algorithm does three things: (a) derives all the keys | (and the descriptor), (b) optionally checks that the resulting | scriptpubkey is what we expect, and (c) updates the maps. The existing | code handles (a) separately for Tr and non-Tr descriptors. | | In the Tr case, we derive all the keys using | Descriptor::<DescriptorPublicKey>::derived_descriptor which derives all | the keys and throws away the conversion. Then separately it keeps around | the un-derived descriptor, iterates through the keys, and populates the | `tap_key_origins` map by re-computing the derivation. | | In the non-Tr case, we derive all the keys using the `KeySourceLookUp` | object, which does exactly the same thing as `derived_descriptor` except | that it stores its work in a BTreeMap, which is directly added to the | PSBT's `item.bip32_derivation` field. | | This commit pulls out (a) into common code; it then copies all the data | out of the key map into `item.tap_key_origins` along with an empty | vector of tapleaves. It then goes through all the leaves, and for each | key that appears in each leaf, appends that leaf's hash to the vector of | tapleaves. This is still a little ineffecient but will be much cleaner | after a later commit when we improve the Taproot SpendInfo structure. | | The original code dates to Lloyd's 2022 PR #339 which introduces logic to | populate these maps. That algorithm underwent significant refactoring in | response to review comments and I suspect that the duplicated logic went | unnoticed after all the refactorings. | | Port from rust-bitcoin/rust-miniscript#815 | * satisfy(miniscript): change `lookup_tap_key_spend_sig` to take the public key | | There is only one public key possible, but this makes the API more | consistent between keyspends and script-spends, and makes it much | easier for users who have a key->sig lookup table to implement | satisfiers. | | Fixes rust-bitcoin/rust-miniscript#825 | | Port from rust-bitcoin/rust-miniscript#827 | * tr(miniscript): rename `iter_scripts` to `leaves` | | The name `iter_scripts` is dumb and undiscoverable. More importantly, | it's misleading -- this iterator does not yield scripts. It yields | Miniscripts, which can be converted to scripts if you have ToPublicKey | and are willing to pay a cost. | | Port from rust-bitcoin/rust-miniscript#807 | * tr(miniscript): encapsulate object yielded by TapTreeIter | | All this does is wrap the existing (depth, script) return value in a | structure that has a pile of accessors on it. The accessors include | a couple of expensive computations which are currently done manually. | | Port from rust-bitcoin/rust-miniscript#807 | * translator(miniscript): remove `TranslatePk` trait and clean up `Translator` trait | | This is an annoying breaking change for users of the Translator trait | but I think it greatly improves the ergonomics of using the trait. | Rather than having it be parameterized over 3 types, it is now | parameterized over just one (the "source pk type"). | | This matches how this trait is used in practice -- you typically have a | miniscript/policy/whatever with a keytype Pk, and you want to use a | translator from Pk to "whatever the translator maps to" with "whatever | error the translator yields". So the only type parameter you really need | to type is Pk; the others are irrelevant, and making the user name and | type them is annoying. | | Since this eliminates the need to explicitly write out the error types | except when actually implementing the trait, this also changes a ton of | error types from () to Infallible, which is more efficient and correct. | | Port from rust-bitcoin/rust-miniscript#733
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The last commit, which renames
parse_with_exttodecode_with_ext, I think we should backport with deprecation to 12.x. This is a pretty uncommonly used function and is unlikely to bother anybody, but the current name is inconsistent with our general use of "parse" for strings and "decode" for scripts.