A number of people who have been learning about bitcoin have recently asked me what is meant by the term ‘Segregated Witness‘ (abbreviated as SegWit).
Therefore, I decided to dedicate this article to the question “What is Segregated Witness (SegWit)?” Segregated Witness is a change in the structure of bitcoin transactions, whereby signatures (‘witness‘ data) are separated from the list of inputs and are attached in a separate data structure. This allows for the signatures not be computed as part of the transaction hash (i.e. the transaction ID).
In the below article, I will further elaborate on the this answer.
In order for you to fully understand what is set forth below, I will start out by briefly explaining (1) the basic structure of a bitcoin transaction and (2) the basic structure of a bitcoin block. I will then explore several commonly asked questions regarding Segregated Witness (SegWit), such as “What changes are introduced by Segregated Witness (SegWit)?”, “What is the purpose of Segregated Witness (SegWit)?”, “How was Segregated Witness (SegWit) introduced?” Finally, I will answer some related questions, such as “Are Segregated Witness (SegWit) bitcoin addresses different from regular bitcoin addresses?” and “How would you know that you are using Segregated Witness (SegWit) in a transaction?”.
How is a bitcoin transaction structured?
A bitcoin transaction consists of (1) one or more inputs and (2) one or more outputs.
Transaction input
A transaction input is (1) an unspent transaction output (UTXO) and (2) an unlocking script that satisfies the conditions for spending the relevant UTXO.
UTXO
An UTXO that is used as input for a bitcoin transaction is referenced by:
(1) the transaction ID: a 32-byte double SHA-256 hash of the bitcoin transaction where the relevant UTXO is recorded on the bitcoin blockchain; and
(2) the vout index: a 4-byte unsigned integer, indicating the position of the relevant UTXO within the transaction where that UTXO is recorded on the bitcoin blockchain.
Unlocking script
The unlocking script is a script that solves the spending conditions of the UTXO that is used as an input for the bitcoin transaction (i.e. it ‘unlocks’ the UTXO, so that it can be spent). This will usually be a signature by the private key of the bitcoin address from which the bitcoins are sent, proving ownership of the UTXO.
Transaction output
The result of a bitcoin transaction are the transaction outputs and consist of newly created and encumbered UTXOs, registered to the receiving bitcoin address. These newly created UTXOs can only be spend by the new owner, i.e. the person that is able to sign a transaction with the private key of the bitcoin address to which the UTXOs are newly registered.
How is a bitcoin block structured?
A bitcoin block is a data structure that groups any number of bitcoin transactions together, that are to be included in the bitcoin blockchain.
In essence, a bitcoin block is built out of 2 main parts:
(1) the block header: The block header contains 3 types of data: (1) a reference to the block header hash of the block that was last added to the bitcoin blockchain (i.e. ‘mined‘), (2) the mining difficulty, the timestamp and the nonce, and (3) the merkle tree root that summarizes all the transactions that are included in the bitcoin block; and
(2) the transactions that are recorded in the block, represented by the transaction IDs. These individual transactions are summarized in the bitcoin block through the use of the double SHA-256 and are organized in a merkle tree data structure.
Each transaction is first hashed, by applying the double SHA-256, to obtain the transaction ID. The transaction hashes (i.e. H A, H B, H C and H D) are subsequently grouped in pairs and hashed again (i.e. H AB and H CD). This process is repeated, resulting in a final hash, called the merkle tree root (i.e. H ABCD)
The merkle tree root that is included in the block header of each bitcoin block is in essence a digital fingerprint (i.e. a hash value), which summarizes all the transactions that are included in the bitcoin block, and is the result of applying repeatedly the double SHA-256 to pairs of bitcoin transactions that are included in the bitcoin block.
A merkle tree root allows for bitcoin nodes to quickly identify transactions that are included in a block, by only downloading the block header (80 bytes of data), instead of having to download the entire block (1 MB, but see the below Section “What is the purpose of Segregated Witness?”). This mechanism is often used by nodes that are not full nodes.
What changes are introduced by Segregated Witness (SegWit)?
Segregation of transaction ‘witness‘ data
Segregated Witness is an optional, backwards compatible, update to the consensus rules of the bitcoin protocol (i.e. ‘soft fork‘), whereby the signature data and unlocking scripts (also called ‘witness‘ data) is segregated from the other input data of a bitcoin transaction, and attached in a separate data structure.
In effect, this means that the transaction data is split in 2 different parts:
(1) the part containing the data determining the effects of the transaction; and
(2) the part containing the signatures and unlocking scripts (scriptSig), which is the data that is required to verify the validity of a transaction. This so-called witness data is separated and attached at the end (as opposed to the middle of the transaction).
The result is that the transaction signatures are no longer computed as part of the transaction hashes (also known as the ‘transaction ID‘) in the merkle tree.
This in turn means that the transaction signatures do no longer affect the merkle tree root that is included in the block header of each bitcoin block.
The transaction signatures are placed in a separate tree in the bitcoin block, commonly referred to as the ‘witness merkle tree‘.
New unit of measurement: transaction ‘weight‘
An additional change that was included in the Segregated Witness (SegWit) protocol update is the introduction of a new way of measuring block transaction capacity, through a new unit of measurement. This new unit of measurement is expressed in ‘weight‘ (calculated per transaction).
Under the new SegWit consensus rules, bitcoin blocks need to comply with a maximum block weight of 4,000,000, while still respecting the maximum block size of 1,000,000 bytes.
The weight of a transaction is calculated in accordance with the following formula:
(transaction size – the witness data) x 3 + transaction size = weight
A non-SegWit transaction of 1,000 bytes results therefore in a weight of 4,000:
(1,000 – 0) x 3 + 1,000 = 4,000
A SegWit transaction of 1,000 bytes can however result in multiple different weights, depending on the size of the witness data.
Example:
A SegWit transaction of 1,000 bytes and witness data of 100 bytes, result in a weight of 3,700
(1,000 – 100) x 3 + 1,000 = 3,700
A SegWit transaction of 1,000 bytes and witness data of 500 bytes, result in a weight of 2,500
(1,000 – 500) x 3 + 1,000 = 2,500
What is the purpose of Segregated Witness (SegWit)?
The main purpose of introducing Segregated Witness (SegWit) is twofold:
- Protect bitcoin against transaction malleability.
- Increase bitcoin’s block capacity through a soft fork upgrade of the bitcoin protocol.
SegWit ultimately makes the Lightning Network safe to deploy on the bitcoin network, which promises to vastly improve bitcoin’s capacity to scale (i.e. increasing transaction throughput, while lowering transaction fees), bringing bitcoin one step closer to potential worldwide mass adoption.
1. SegWit protects bitcoin against transaction malleability.
Problem
Prior to the implementation of SegWit, the signature of a transaction was included in (i.e. hashed as part of) the transaction ID (i.e. the hash of the bitcoin transaction, see above Section “How is a bitcoin transaction structured?“).
The transaction signature itself is the only part of a transaction that is not signed. This represented a problem because this gave rise to the possibility for anyone to copy a pending but unconfirmed transaction (e.g. an unconfirmed transaction in the mempool) and modify the transaction signature in such a way that the transaction ID would be changed but without affecting the validity of that transaction. In other words, it was possible to produce 2 identical valid but unconfirmed transactions (i.e. same spendable amounts, same destination addresses, etc.), each with different transaction IDs.
More in particular, it is possible to change the way a transaction signature is written without changing the mathematical value of that transaction signature. Indeed, a signature is in essence a binary number (i.e. a number composed out of 0s and 1s). Therefore, one can imagine multiple different ways of writing the same mathematical value in a binary number format (e.g. adding a zero in front of a binary number does not change the decimal value of that number. As an example, the binary numbers 01 and 001 represent the same decimal number 1).
This means that although the (decimal) value of the signature would not be changed in such case, and thus any bitcoin node would verify and accept both transactions as being valid against the public key of the sending bitcoin address, each of these 2 valid transactions would yield a different transaction hash (i.e. transaction ID) because the hashing algorithm (SHA-256) would be applied to 2 different signatures. (i.e. 2 different binary numbers).
Such transaction malleability is problematic from a transaction security perspective, as this opens the possibility for a valid but unconfirmed transaction to get ultimately rejected by nodes of the bitcoin network, in the event another identical transaction but with a different transaction ID gets confirmed (i.e. included in a bitcoin block) first. This also means that any other transactions that built further upon the originally valid but now rejected transaction would be invalidated. Fixing this flaw in the bitcoin protocol was a key factor of making the Lightning Network possible (See below Section “Why does SegWit make the Lightning Network possible?” for a detailed explanation).
Solution
With Segregated Witness (SegWit) the signature data and unlocking scripts (also called ‘witness‘ data) is segregated from the other input data of a bitcoin transaction, and attached in a separate data structure.
As a result, the transaction signatures are no longer hashed as part of the transaction ID. Therefore, a change in the signature will no longer affect the transaction ID. This makes it impossible to create 2 identical validly signed but unconfirmed transactions (i.e. same spendable amounts, same destination addresses, etc.) having different transaction IDs.
2. SegWit increases bitcoin’s block capacity through a soft fork upgrade of the bitcoin protocol.
Block size restriction of 1,000,000 bytes
In accordance with the consensus rules of the bitcoin protocol, the maximum block size is 1,000,000 bytes. This means that nodes of the bitcoin network will reject any block with a size that is larger than 1,000,000 bytes.
SegWit increases the block capacity while still respecting the existing block size limit of 1,000,000 bytes
By segregating the transaction witness data in a separate data structure, SegWit in effect increases the capacity of a SegWit block, within the existing maximum block size limit of 1,000,000 bytes (i.e. more transactions can be included before reaching the maximum limit of 1,000,000 bytes).
However, SegWit blocks need to adhere to an additional new restriction, namely the maximum block weight of 4,000,000 (as explained above).
Hence, the SegWit upgrade in effect further restricts the existing consensus rules of the bitcoin protocol by adding mentioned maximum weight rule, making the upgrade optional and backwards compatible (‘soft fork‘) for legacy nodes; i.e. legacy nodes that do not upgrade to SegWit will still accept the new type of SegWit blocks as valid, because such blocks still comply with the maximum block size of 1,000,000 bytes.
3. Why does SegWit make the Lightning Network possible?
The implementation of SegWit makes the Lightning Network safe to deploy on top of the bitcoin network.
What is the Lightning Network?
If you’re unfamiliar with the Lightning Network, I highly recommend you first watch this short educational video (5 minutes), explaining how it works in simple terms:
In short, the Lightning Network is designed for making small & micro bitcoin payments off-chain (i.e. not broadcasted on the bitcoin blockchain).
Small payments are impractical and expensive to perform on the bitcoin blockchain, because:
(1) bitcoin blocks containing transactions are only added to the bitcoin blockchain on average every 10 minutes, making transaction confirmation times relatively slow; and
(2) the amount of bitcoin fees make it ineffective to make small & micro payments on the bitcoin blockchain.
The Lightning Network solves this scalability issue, by offering the possibility of instant and micro payments (down to 0.00000001 BTC).
How does the Lightning Network work?
Parties that are transacting with each other over the Lightning Network begin by opening a payment channel between them. In essence, this means that the interacting parties begin by sending amounts of bitcoin to a 2-of-2 multi-signature bitcoin address (where it requires consent from both parties to create spends), which is governed by a smart contract (called the ‘Funding Transaction‘).
The funding amounts will act as a balance between the two parties, as long as the payment channel remains open. Both parties can therefore spend from the payment channel to transact with each other, within the limits of their respective bitcoin balance in the payment channel.
When the parties are making use of the payment channel to make payments to each other, they are essentially exchanging signed bitcoin transactions (commonly referred to as ‘commitments‘) which are not broadcasted to the bitcoin network. These signed bitcoin transactions remain therefore unconfirmed (i.e. they are not included in a bitcoin block).
Whenever a payment is made over the Lightning Network, both transacting parties need to sign the transaction (i.e. multi-signature) and the most recent signed transaction represents the current balance between the two transacting parties. Only if and when either party decides to end the payment channel will the latest balance be broadcasted to the bitcoin network. In essence, the transacting parties keep an open balance between each other, exchanging off-chain bitcoin transactions, and only settle the net balance, whenever either party decides to close the payment channel, by broadcasting it to the bitcoin network.
Therefore, parties who are transacting which each other over the Lightning Network are relying on a series of unconfirmed off-chain bitcoin transactions, which in turn build further upon the initial on-chain Funding Transaction. When the payment channel is closed the updated Funding Transaction is signed and broadcasted to the bitcoin network and eventually confirmed in a bitcoin block.
Why is SegWit important for the security of the Lightning Network?
I already established above that parties who are transacting which each other over the Lightning Network are relying on a series of off-chain bitcoin transactions, which in turn build further upon the initial on-chain Funding Transaction, as long as and until one of the parties decide to close the payment channel.
This means that the unconfirmed off-chain payments made over a Lightning Network payment channel refer back to the transaction ID of the Funding Transaction. The updated Funding Transaction will only be broadcast to and confirmed on the bitcoin network at the very end, i.e. when a party decides to close the payment channel.
Transaction malleability (see above, ‘1. SegWit protects bitcoin against transaction malleability‘) could be exploited in order to invalidate the Funding Transaction (and thereby invalidating all subsequent off-chain bitcoin transactions that have taken place through the payment channel), by creating an identical copy of the Funding Transaction but with a different transaction ID and get it confirmed on the bitcoin network before the original Funding Transaction.
With SegWit it is no longer possible to produce identical valid but unconfirmed transactions having different transaction IDs. This reduces the risk of a Funding Transaction ultimately getting rejected by the bitcoin network and invalidating the off-chain payments that took place in the payment channel. Hence, by eliminating transaction malleability, SegWit enhances the integrity and security of the Lightning Network.
How was Segregated Witness (SegWit) introduced?
Segregated Witness (SegWit) was originally proposed in December 2015 by bitcoin core developers Eric Lombrozo, Johnson Lau and Pieter Wuille through BIP 141 (BIP stands for Bitcoin Improvement Proposal). It was activated on August 23, 2017 by way of a user activated soft fork (abbreviated as UASF).
A ‘user activated soft fork‘ (UASF) is a mechanism for activating a backwards compatible change to the consensus rules of the bitcoin protocol, through bitcoin full nodes signalling support by upgrading to the new version of the bitcoin protocol (as opposed to ‘miner activated soft forks‘, requiring mining nodes to signal support for the upgrade).
In effect, UASFs give a bigger say to the full nodes of the bitcoin network when it comes to making changes to the bitcoin protocol, reducing the influence of mining nodes in this regard. Indeed, mining nodes are incentivised to upgrade to the newest version of the bitcoin protocol, otherwise they risk producing bitcoin blocks which are invalid according to the new consensus rules applied by bitcoin full nodes that have upgraded to the latest version of the bitcoin protocol.
Related Questions
Are Segregated Witness (SegWit) bitcoin addresses different from legacy bitcoin addresses?
SegWit addresses are different from legacy bitcoin addresses.
There are two different formats for SegWit addresses:
- SegWit addresses wrapped inside P2SH; and
- Native SegWit addresses.
1. SegWit addresses wrapped inside P2SH
SegWit addresses wrapped inside P2SH, are non-native SegWit addresses.
These type of SegWit addresses will start with the number ‘3‘ and look like the legacy multi-sig bitcoin addresses.
SegWit addresses wrapped inside P2SH are backwards compatible. This means that that this type of SegWit addresses can be used by bitcoin wallets that have not upgraded to SegWit, and therefore do not understand native SegWit addresses, to nevertheless send bitcoin to a SegWit bitcoin wallet.
2. Native SegWit addresses
Native SegWit addresses use a new checksummed base32 format called “Bech32”, as defined in BIP 173 (as opposed to the base58 format used for legacy bitcoin addresses).
You can easily identify native SegWit addresses by the first three characters of such type of addresses, namely the characters ‘bc1‘.
The Bech32 format has a number of advantages, one of which is that mixed case addresses (i.e. addresses with both lower and upper case characters) are no longer allowed.
If you want to learn more about the advantages of introducing the Bech32 format for SegWit addresses, I highly recommend watching the below presentation by bitcoin core developer Pieter Wuille, who is one of the co-authors together with Greg Maxwell of BIP 173:
How would you know that you are using Segregated Witness (SegWit) in a transaction?
When sending or receiving bitcoin, you can recognize a SegWit address by their first characters, i.e. either ‘bc1‘ or the number ‘3‘ (However, in case of an address starting with the number 3, it could also mean a legacy multi-sig address).
Bitcoin addresses starting with the number ‘1‘ are definitely not SegWit addresses.
Attention! Do you store your cryptocurrencies on an online platform? Please note, in that case you are not the actual owner of your cryptocurrencies!
In particular, you run the risk of losing all your cryptocurrencies, without any recourse, in the event that the online platform or your personal account falls victim to hacking or in the event of an unexpected closure (e.g. insolvency) of the online platform.
Protect yourself against hacking and take real ownership of your cryptocurrencies by storing your cryptocurrencies offline on your very own Trezor hardware wallet. Don’t wait before it’s too late and take immediate action now!
Click on the ‘Buy Now’ button below to buy a Trezor wallet from the official Trezor website.
