Implemented Script Verification for P2PKH

guidelines.md

@@ -1,53 +1,147 @@
-# Guidelines and Restrictions
-
-## Guidelines
-
-1. Implement the transaction validation logic manually without using Bitcoin-specific libraries or frameworks.
-2. Implement the mining algorithm yourself, including creating the block header, calculating the hash, and finding a valid nonce that meets the difficulty target.
-3. Use standard cryptographic libraries (e.g., secp256k1 for elliptic curve cryptography, SHA-256 for hashing) as needed.
-4. Document your work in the `SOLUTION.md` file, explaining your design approach, implementation details, results, and conclusions.
-5. Publish your solution in the designated private repository, and do not share it publicly or with peers.
-
-## Restrictions
-
-1. Do not use any Bitcoin-specific libraries or frameworks that automate transaction validation processes.
-2. Do not use existing implementations of the mining algorithm or block construction logic.
-3. Do not plagiarize or copy solutions from others (solutions will be checked for plagiarism).
-4. While you can use AI tools like ChatGPT to gather information and explore alternative approaches, do not rely solely on AI for complete solutions. Verify and validate any insights obtained, and maintain a balance between AI assistance and independent problem-solving.
-
-The main objective of these guidelines and restrictions is to deepen your understanding of Bitcoin fundamentals by implementing the transaction validation and block mining processes from scratch, without relying on existing libraries or implementations. This exercise is designed to test your problem-solving skills and your ability to implement complex algorithms independently.
-
-
-># Asynchronous Transaction Validation and Mining
-
-In this assignment, you should treat the transaction validation and mining process as an asynchronous operation. Here's how you can approach it:
-## 1. Fetch transactions from mempool asynchronously
-
-- Create a function or method that reads all the JSON files in the `mempool` directory asynchronously.
-- Use Python's built-in `asyncio` module or a third-party library like `aiofiles` to read the files concurrently.
-- Deserialize each JSON transaction into your `Transaction` data structure.
-
-## 2. Validate transactions asynchronously
-
-- Create a function or method that takes a list of `Transaction` objects and validates them asynchronously.
-- Use Python's `asyncio` module and its `gather` function to run the validation tasks concurrently.
-- For each transaction, call your transaction validation functions or methods (implemented in `validation.py`) to check if it's valid or not.
-
-## 3. Store valid transactions in a dictionary
-
-- Create an empty dictionary (or any suitable data structure) to store the valid transactions.
-- After the asynchronous validation is complete, iterate through the list of validated transactions and add the valid ones to the dictionary.
-- Use the transaction ID (`txid`) as the key and the `Transaction` object as the value in the dictionary.
-
-## 4. Mine the block
-
-- In your mining logic (implemented in `mining.py`), iterate through the dictionary of valid transactions and construct the block accordingly.
-- Create the coinbase transaction and include it in the block.
-- Serialize the block header and coinbase transaction, and write them to the `output.txt` file as per the specified format.
-- Write the transaction IDs (`txid`) of the valid transactions to the `output.txt` file, following the coinbase transaction.
-
-## 5. Implement concurrency control
-
-- Since multiple tasks are running concurrently, you may need to implement proper synchronization mechanisms to avoid race conditions or data corruption.
-- Use Python's `asyncio` locks, semaphores, or other synchronization primitives to control access to shared resources, such as the dictionary of valid transactions.
-
+# Guidelines on the project solutions 
+
+## Validation a transaction, algorithm thinking process
+1. Check syntactic correctness
+2. Make sure neither in or out lists are empty
+3. Size in bytes < MAX_BLOCK_SIZE
+4. Each output value, as well as the total, must be in legal money range
+5. Make sure none of the inputs have hash=0, n=-1 (coinbase transactions)
+6. Check that nLockTime <= INT_MAX[1], size in bytes >= 100[2], and sig opcount <= 2[3]
+7. Reject "nonstandard" transactions: scriptSig doing anything other than pushing numbers on the stack, or scriptPubkey not matching the two usual forms[4]
+8. Reject if we already have matching tx in the pool, or in a block in the main branch
+9. For each input, if the referenced output exists in any other tx in the pool, reject this transaction.[5]
+10. For each input, look in the main branch and the transaction pool to find the referenced output transaction. If the output transaction is missing for any input, this will be an orphan transaction. Add to the orphan transactions, if a matching transaction is not in there already.
+11. For each input, if the referenced output transaction is coinbase (i.e. only 1 input, with hash=0, n=-1), it must have at least COINBASE_MATURITY (100) confirmations; else reject this transaction
+12. For each input, if the referenced output does not exist (e.g. never existed or has already been spent), reject this transaction[6]
+13. Using the referenced output transactions to get input values, check that each input value, as well as the sum, are in legal money range
+14. Reject if the sum of input values < sum of output values
+15. Reject if transaction fee (defined as sum of input values minus sum of output values) would be too low to get into an empty block
+16. Verify the script PubKey accepts for each input; reject if any are bad
+17. Add to transaction pool[7]
+18. "Add to wallet if mine"
+19. Relay transaction to peers
+20. For each orphan transaction that uses this one as one of its inputs, run all these steps (including this one) recursively on that orphan
+
+## Mining
+1. Check syntactic correctness
+2. Reject if duplicate of block we have in any of the three categories
+3. Transaction list must be non-empty
+4. Block hash must satisfy claimed nBits proof of work
+5. Block timestamp must not be more than two hours in the future
+6. First transaction must be coinbase (i.e. only 1 input, with hash=0, n=-1), the rest must not be
+7. For each transaction, apply "tx" checks 2-4
+8. For the coinbase (first) transaction, scriptSig length must be 2-100
+9. Reject if sum of transaction sig opcounts > MAX_BLOCK_SIGOPS
+10. Verify Merkle hash
+11. Check if prev block (matching prev hash) is in main branch or side branches. If not, add this to orphan blocks, then query peer we got this from for 1st missing orphan block in prev chain; done with block
+12. Check that nBits value matches the difficulty rules
+13. Reject if timestamp is the median time of the last 11 blocks or before
+14. For certain old blocks (i.e. on initial block download) check that hash matches known values
+15. Add block into the tree. There are three cases: 1. block further extends the main branch; 2. block extends a side branch but does not add enough difficulty to make it become the new main branch; 3. block extends a side branch and makes it the new main branch.
+16. For case 1, adding to main branch:
+    1. For all but the coinbase transaction, apply the following:
+       1. For each input, look in the main branch to find the referenced output transaction. Reject if the output transaction is missing for any input.
+       2. For each input, if we are using the nth output of the earlier transaction, but it has fewer than n+1 outputs, reject.
+       3. For each input, if the referenced output transaction is coinbase (i.e. only 1 input, with hash=0, n=-1), it must have at least COINBASE_MATURITY (100) confirmations; else reject.
+       4. Verify crypto signatures for each input; reject if any are bad
+       5. For each input, if the referenced output has already been spent by a transaction in the main branch, reject
+       6. Using the referenced output transactions to get input values, check that each input value, as well as the sum, are in legal money range
+       7. Reject if the sum of input values < sum of output values
+    2. Reject if coinbase value > sum of block creation fee and transaction fees
+    3. (If we have not rejected):
+    4. For each transaction, "Add to wallet if mine"
+    5. For each transaction in the block, delete any matching transaction from the transaction pool
+    6. Relay block to our peers
+    7. If we rejected, the block is not counted as part of the main branch
+17. For case 2, adding to a side branch, we don't do anything.
+18. For case 3, a side branch becoming the main branch:
+    1. Find the fork block on the main branch which this side branch forks off of
+    2. Redefine the main branch to only go up to this fork block
+    3. For each block on the side branch, from the child of the fork block to the leaf, add to the main branch:
+       1. Do "branch" checks 3-11
+       2. For all but the coinbase transaction, apply the following:
+          1. For each input, look in the main branch to find the referenced output transaction. Reject if the output transaction is missing for any input.
+          2. For each input, if we are using the nth output of the earlier transaction, but it has fewer than n+1 outputs, reject.
+          3. For each input, if the referenced output transaction is coinbase (i.e. only 1 input, with hash=0, n=-1), it must have at least COINBASE_MATURITY (100) confirmations; else reject.
+          4. Verify crypto signatures for each input; reject if any are bad
+          5. For each input, if the referenced output has already been spent by a transaction in the main branch, reject
+          6. Using the referenced output transactions to get input values, check that each input value, as well as the sum, are in legal money range
+          7. Reject if the sum of input values < sum of output values
+       3. Reject if coinbase value > sum of block creation fee and transaction fees
+       4. (If we have not rejected):
+       5. For each transaction, "Add to wallet if mine"
+    4. If we reject at any point, leave the main branch as what it was originally, done with block
+    5. For each block in the old main branch, from the leaf down to the child of the fork block:
+       1. For each non-coinbase transaction in the block:
+          1. Apply "tx" checks 2-9, except in step 8, only look in the transaction pool for duplicates, not the main branch
+          2. Add to transaction pool if accepted, else go on to next transaction
+    6. For each block in the new main branch, from the child of the fork node to the leaf:
+       1. For each transaction in the block, delete any matching transaction from the transaction pool
+    7. Relay block to our peers
+19. For each orphan block for which this block is its prev, run all these steps (including this one) recursively on that orphan
+
+## Creating Merkel Root
+* Hash 11 TXIDs together.
+
+## Proper way to comment python codes
+### Classes
+```python
+class MyClass:
+    """
+    Brief summary of the class.
+
+    Attributes:
+        attribute1 (type): Description of attribute1.
+        attribute2 (type): Description of attribute2.
+
+    Methods:
+        method1(arg1, arg2): Description of method1.
+        method2(arg1): Description of method2.
+    """
+    # Class implementation goes here
+```
+
+### Methods
+```python
+class MyClass:
+    def my_method(self, arg1, arg2=None):
+        """
+        Brief summary of my_method.
+
+        Args:
+            arg1 (type): Description of arg1.
+            arg2 (type, optional): Description of arg2. Defaults to None.
+
+        Returns:
+            type: Description of the return value.
+
+        Raises:
+            Exception1: Description of Exception1.
+            Exception2: Description of Exception2.
+        """
+        # Method implementation goes here
+```
+
+## Copied Todo List:
+        # Todo: Not necessary step 1 : check if transaction is unspent
+
+        # Todo: step 2 : Validate the input's scriptSig (or witness data for SegWit inputs) against the referenced output's scriptPubKey.
+
+        # Todo: step 3 : P2PKH
+        if not await self.validate_p2pkh_script(tx):
+            print("Transaction is invalid")
+            return False, "The p2pkh script validation failed"
+
+        # Todo: step 4 :  Ensure that the sum of output values does not exceed the sum of input values (minus any transaction fees).
+
+        # Todo: step 5 : Check that no output value is negative or too large (e.g., above the maximum allowed value).
+
+        # Todo: step 6 : Ensure that each output script is a valid script format (e.g., P2PKH, P2WPKH, P2SH, etc.) and follows the correct script structure.
+
+        # Todo: step 7 : Calculate the transaction fee by subtracting the sum of output values from the sum of input values.
+
+        # Todo: step 8 : Ensure that the transaction fee meets the minimum required fee rate (fees per byte) for the network.
+
+        # Todo: step 9 : Ensure that there are no duplicate inputs (i.e., no double-spending) within the transaction.
+
+        # Todo: step 10 : Ensure that there are no duplicate outputs (which would be considered non-standard).

main.py

@@ -2,50 +2,34 @@
 import asyncio
 # TODO: replace json with a serializer
 import json
-from src.transaction import Transaction, TransactionSchema
 from src.validation import ValidateTransaction
-from marshmallow import ValidationError
 from src.mine import mine_block
 from collections import defaultdict
 
-TRANSACTION_BY_ID = defaultdict(dict)
-
-# TODO: Remove
 async def main():
+    """
+    This function is the entry point of the entire project
+    the overview of the functionality is commented in steps
+    with an appropriate implementation of the concept of separation of concern
+    in the form of modules.
+    :return:
+    """
     # Step 1: Read and deserialize transactions from the mempool directory
     transactions = []
     for filename in os.listdir("mempool"):
         filepath = os.path.join("mempool", filename)
         with open(filepath, "r") as file:
             json_data = file.read()
-            transaction_schema = TransactionSchema()
             try:
                 loaded_data = json.loads(json_data)
-                transaction = transaction_schema.load(loaded_data)
-
-                for tx_input in transaction.vin:
-                    txid = tx_input["txid"]
-                    if txid:
-                        TRANSACTION_BY_ID[txid] = loaded_data
-
                 transactions.append(loaded_data)
-                # print(f"Deserialized transaction from {filename}")
-            except ValidationError as e:
-                # errors = errors + 1
+            except:
                 print(f"Error deserializing transaction {filename}:")
-                print(e.messages)
-
-    # print(f"Total transactions deserialized: {len(transactions)}")
-    # print(f"Total failed transactions:{errors}")
 
     # Step 2: Validate transactions asynchronously
-    # valid_transactions = await validate_transaction.validate_transactions(transactions)
-
-    # implement an initial transaction validation process.
-
-    # print(valid_transactions)
+    validate_transaction = ValidateTransaction()
+    valid_transactions = await validate_transaction.validate_transactions(transactions)
 
-    # print(valid_transactions)
     # Step 3: Mine the block
     block_data = mine_block(valid_transactions)
 

src/scripts.py

@@ -0,0 +1,65 @@
+import asyncio
+from typing import Any, List, Tuple, Union
+from ecdsa import VerifyingKey, BadDigestError, SECP256k1
+import hashlib
+from .utility import Utility
+
+class Scripts:
+    """
+    Summary: This class contains the various scripts for validating a bitcoin transaction, the likes of
+    P2PKH : Pay Too Public Key Hash
+    P2WPKH : Pay Too Witness Public Key Hash
+
+
+    Attributes:
+
+    Methods:
+        validate_p2pkh_script(self, tx: dict) : validates the transaction with Pay Too Public Key Hash(P2PKH)
+    """
+
+    async def validate_p2pkh_script(self, tx: dict) -> bool:
+        """
+        Adds to the transaction validation process by validating the scriptpubkey
+        Args:
+            tx (dict) : A dictionary representation of the json transaction format from mempool
+        Returns:
+            bool: False if the transaction is not valid
+        """
+        for tx_input in tx["vin"]:
+            # list_of_script_types = ['v1_p2tr', 'v0_p2wpkh', 'p2sh', 'p2pkh', 'v0_p2wsh']
+            referenced_output = tx_input["prevout"]
+            scriptpubkey_type = referenced_output["scriptpubkey_type"]
+
+            if scriptpubkey_type != "p2pkh" and scriptpubkey_type != "v0_p2wpkh":
+                continue
+
+            #
+            if scriptpubkey_type == "p2pkh":
+                utility = Utility()
+                sig, pubkey = utility.parse_signature_script(tx_input)
+                pubkey_script = referenced_output["scriptpubkey"]
+                pubkey_hash = utility.extract_pubkey_hash(pubkey_script)
+
+                stack = []
+                stack.append(sig)
+                stack.append(pubkey)
+                for opcode in pubkey_script:
+                    if opcode == "OP_DUP":
+                        stack.append(stack[-1])
+                    elif opcode == "OP_HASH160":
+                        pubkey_hash_from_stack = utility.hash160(stack.pop())
+                        stack.append(pubkey_hash_from_stack)
+                    elif opcode == "OP_EQUALVERIFY":
+                        top1, top2 = stack.pop(), stack.pop()
+                        if top1 != top2:
+                            return False
+                    elif opcode == "OP_CHECKSIG":
+                        pubkey = stack.pop()
+                        sig = stack.pop()
+                        if not utility.verify_signature(sig, pubkey, tx):
+                            print("Faild P2PKH")
+                            return False
+                print("Passed P2PKH")
+                return True
+
+        return False