Introducing Ethereum Script 2.0

This station volition supply the groundwork for a large rework of the Ethereum scripting language, which volition substantially modify the mode ES works though inactive keeping galore of the halfway components moving successful the nonstop aforesaid way. The rework is indispensable arsenic a effect of aggregate concerns which person been raised astir the mode the connection is presently designed, chiefly successful the areas of simplicity, optimization, ratio and future-compatibility, though it does besides person immoderate side-benefits specified arsenic improved relation support. This is not the past iteration of ES2; determination volition apt beryllium galore incremental structural improvements that tin beryllium made to the spec, but it does service arsenic a beardown starting point.

As an important clarification, this rework volition person small effect connected the Ethereum CLL, the stripped-down-Python-like connection successful which you tin constitute Namecoin successful 5 lines of code. The CLL volition inactive enactment the aforesaid arsenic it is now. We volition request to marque updates to the compiler (an alpha mentation of which is present disposable successful Python astatine http://github.com/ethereum/compiler oregon arsenic a affable web interface astatine http://162.218.208.138:3000) successful bid to marque definite the CLL continues to compile to caller versions of ES, but you arsenic an Ethereum declaration developer moving successful E-CLL should not request to spot immoderate changes astatine all.

Problems with ES1

Over the past period of moving with ES1, respective problems with the language’s plan person go apparent. In nary peculiar order, they are arsenic follows:

• Too galore opcodes – looking astatine the specification as it appears today, ES1 present has precisely 50 opcodes – little than the 80 opcodes recovered successful Bitcoin Script, but inactive acold much than the theoretically minimal 4-7 opcodes needed to person a functional Turing-complete scripting language. Some of those opcodes are indispensable due to the fact that we privation the scripting connection to person entree to a batch of information – for example, the transaction value, the transaction source, the transaction data, the erstwhile artifact hash, etc; similar it oregon not, determination needs to beryllium a definite grade of complexity successful the connection explanation to supply each of these hooks. Other opcodes, however, are excessive, and complex; arsenic an example, see the existent explanation of SHA256 oregon ECVERIFY. With the mode the connection is designed close now, that is indispensable for efficiency; otherwise, 1 would person to constitute SHA256 successful Ethereum publication by hand, which mightiness instrumentality galore thousands of BASEFEEs. But ideally, determination should beryllium immoderate mode of eliminating overmuch of the bloat.
• Not future-compatible – the beingness of the peculiar crypto opcodes does marque ES1 overmuch much businesslike for definite specialized applications; acknowledgment to them, computing SHA3 takes lone 40x BASEFEE alternatively of the galore thousands of basefees that it would instrumentality if SHA3 was implemented successful ES directly; aforesaid with SHA256, RIPEMD160 and secp256k1 elliptic curve operations. However, it is perfectly not future-compatible. Even though these existing crypto operations volition lone instrumentality 40x BASEFEE, SHA4 volition instrumentality respective 1000 BASEFEEs, arsenic volition ed25519 signatures, the quantum-proofNTRU, SCIP and Zerocoin math, and immoderate different constructs that volition look implicit the coming years. There should beryllium immoderate earthy mechanics for folding specified innovations successful implicit time.
• Not deduplication-friendly – the Ethereum blockchain is apt to go highly bloated implicit time, particularly with each declaration penning its ain codification adjacent erstwhile the bulk of the codification volition apt beryllium thousands of radical trying to bash the nonstop aforesaid thing. Ideally, each instances wherever codification is written doubly should walk done immoderate process of deduplication, wherever the codification is lone stored erstwhile and lone a pointer to the codification is stored twice. In theory, Ethereum’s Patricia trees bash this already. In practice, however, codification needs to beryllium successful precisely the aforesaid spot successful bid for this to happen, and the beingness of jumps means that it is often hard to abitrarily copy/paste codification without making due modifications. Furthermore, determination is nary incentivization mechanics to person radical to reuse existing code.
• Not optimization-friendly – this is simply a precise akin criterion to future-compatibility and deduplication-friendliness successful immoderate ways. However, present optimization refers to a much automatic process of detecting bits of codification that are reused galore times, and replacing them with memoized oregon compiled instrumentality codification versions.

Beginnings of a Solution: Deduplication

The archetypal contented that we tin grip is that of deduplication. As described above, Ethereum Patricia trees supply deduplication already, but the occupation is that achieving the afloat benefits of the deduplication requires the codification to beryllium formatted successful a precise peculiar way. For example, if the codification successful declaration A from scale 0 to scale 15 is the aforesaid arsenic the codification successful declaration B from scale 48 to scale 63, past deduplication happens. However, if the codification successful declaration B is offset astatine each modulo 16 (eg. from scale 49 to scale 64), past nary deduplication takes spot astatine all. In bid to remedy this, determination is 1 comparatively elemental solution: determination from a dumb hexary Patricia histrion to a much semantically oriented information structure. That is, the histrion represented successful the database should reflector the abstract syntax histrion of the code.

To recognize what I americium saying here, see immoderate existing ES1 code:

TXVALUE PUSH 25 PUSH 10 PUSH 18 EXP MUL LT NOT PUSH 14 JMPI STOP PUSH 0 TXDATA SLOAD NOT PUSH 0 TXDATA PUSH 1000 LT NOT MUL NOT NOT PUSH 32 JMPI STOP PUSH 1 TXDATA PUSH 0 TXDATA SSTORE

In the Patricia tree, it looks similar this:

( (TXVALUE PUSH 25 PUSH 10 PUSH 18 EXP MUL LT NOT PUSH 14 JMPI STOP PUSH) (0 TXDATA SLOAD NOT PUSH 0 TXDATA PUSH 1000 LT NOT MUL NOT NOT PUSH 32) (JMPI STOP PUSH 1 TXDATA PUSH 0 TXDATA SSTORE) )

And present is what the codification looks similar structurally. This is easiest to amusement by simply giving the E-CLL it was compiled from:

if tx.value < 25 * 10^18: stop if contract.storage[tx.data[0]] oregon tx.data[0] < 1000: stop contract.storage[tx.data[0]] = tx.data[1]

No narration astatine all. Thus, if different declaration wanted to usage immoderate semantic sub-component of this code, it would astir surely person to re-implement the full thing. However, if the histrion operation looked somewhat much similar this:

( ( IF (TXVALUE PUSH 25 PUSH 10 PUSH 18 EXP MUL LT NOT) (STOP) ) ( IF (PUSH 0 TXDATA SLOAD NOT PUSH 0 TXDATA PUSH 1000 LT NOT MUL NOT) (STOP) ) ( PUSH 1 TXDATA PUSH 0 TXDATA SSTORE ) )

Then if idiosyncratic wanted to reuse immoderate peculiar portion of codification they easy could. Note that this is conscionable an illustrative example; successful this peculiar lawsuit it astir apt does not marque consciousness to deduplicate since pointers request to beryllium astatine slightest 20 bytes agelong to beryllium cryptographically secure, but successful the lawsuit of larger scripts wherever an interior clause mightiness incorporate a fewer 1000 opcodes it makes cleanable sense.

Immutability and Purely Functional Code

Another modification is that codification should beryllium immutable, and frankincense abstracted from data; if aggregate contracts trust connected the aforesaid code, the declaration that primitively controls that codification should not person the quality to sneak successful changes aboriginal on. The pointer to which codification a moving declaration should commencement with, however, should beryllium mutable.

A 3rd communal optimization-friendly method is the marque a programming connection purely functional, truthful functions cannot person immoderate broadside effects extracurricular of themselves with the objection of instrumentality values. For example, the pursuing is simply a axenic function:

def factorial(n): prod = 1 for one successful range(1,n+1): prod *= i return prod

However, this is not:

x = 0 def next_integer(): x += 1 return x

And this astir surely is not:

import os def happy_fluffy_function(): bal = float(os.popen('bitcoind getbalance').read()) os.popen('bitcoind sendtoaddress 1JwSSubhmg6iPtRjtyqhUYYH7bZg3Lfy1T %.8f' % (bal - 0.0001)) os.popen('rm -rf ~')

Ethereum cannot beryllium purely functional, since Ethereum contracts bash needfully person authorities – a declaration tin modify its semipermanent retention and it tin nonstop transactions. However, Ethereum publication is simply a unsocial concern due to the fact that Ethereum is not conscionable a scripting situation – it is an incentivized scripting environment. Thus, we tin let applications similar modifying retention and sending transactions, but discourage them with fees, and frankincense guarantee that astir publication components are purely functional simply to chopped costs, adjacent portion allowing non-purity successful those situations wherever it makes sense.

What is absorbing is that these 2 changes enactment together. The immutability of codification besides makes it easier to conception a restricted subset of the scripting connection which is functional, and past specified functional codification could beryllium deduplicated and optimized astatine will.

Ethereum Script 2.0

So, what’s going to change? First of all, the basal stack-machine conception is going to astir enactment the same. The main information operation of the strategy volition proceed to beryllium the stack, and astir of your beloved opcodes volition not alteration significantly. The lone differences successful the stack instrumentality are the following:

1. Crypto opcodes are removed. Instead, we volition person to person idiosyncratic constitute SHA256, RIPEMD160, SHA3 and ECC successful ES arsenic a formality, and we tin person our interpreters see an optimization replacing it with bully old-fashioned machine-code hashes and sigs close from the start.
2. Memory is removed. Instead, we are bringing backmost DUPN (grabs the adjacent worth successful the code, accidental N, and pushes a transcript of the point N items down the stack to the apical of the stack) and SWAPN (swaps the apical point and the nth item).
3. JMP and JMPI are removed.
4. RUN, IF, WHILE and SETROOT are added (see beneath for further definition)

Another alteration is successful however transactions are serialized. Now, transactions look arsenic follows:

• SEND: [ 0, nonce, to, value, [ data0 ... datan ], v, r, s ]
• MKCODE: [ 1, nonce, [ data0 ... datan ], v, r, s ]
• MKCONTRACT: [ 2, nonce, coderoot, v, r, s ]

The code of a declaration is defined by the past 20 bytes of the hash of the transaction that produced it, arsenic before. Additionally, the nonce nary longer needs to beryllium adjacent to the nonce stored successful the relationship equilibrium representation; it lone needs to beryllium adjacent to oregon greater than that value.

Now, accidental that you wanted to marque a elemental declaration that conscionable keeps way of however overmuch ether it received from assorted addresses. In E-CLL that’s:

contract.storage[tx.sender] = tx.value

In ES2, instantiating this declaration present takes 2 transactions:

[ 1, 0, [ TXVALUE TXSENDER SSTORE ], v, r, s]

[ 2, 1, 761fd7f977e42780e893ea44484c4b64492d8383, v, r, s ]

What happens present is that the archetypal transaction instantiates a codification node successful the Patricia tree. The hash sha3(rlp.encode([ TXVALUE TXSENDER SSTORE ]))[12:] is 761fd7f977e42780e893ea44484c4b64492d8383, truthful that is the “address” wherever the codification node is stored. The 2nd transaction fundamentally says to initialize a declaration whose codification is located astatine that codification node. Thus, erstwhile a transaction gets sent to the contract, that is the codification that volition run.

Now, we travel to the absorbing part: the definitions of IF and RUN. The mentation is simple: IF loads the adjacent 2 values successful the code, past pops the apical point from the stack. If the apical point is nonzero, past it runs the codification point astatine the archetypal codification value. Otherwise, it runs the codification point astatine the 2nd codification value. WHILE is similar, but alternatively loads lone 1 codification worth and keeps moving the codification portion the apical point connected the stack is nonzero. Finally, RUN conscionable takes 1 codification worth and runs the codification without asking for anything. And that’s each you request to know. Here is 1 mode to bash a Namecoin declaration successful caller Ethereum script:

A: [ TXVALUE PUSH 25 PUSH 10 PUSH 18 EXP MUL LT ] B: [ PUSH 0 TXDATA SLOAD NOT PUSH 0 TXDATA PUSH 100 LT NOT MUL NOT ] Z: [ STOP ] Y: [ ] C: [ PUSH 1 TXDATA PUSH 0 TXDATA SSTORE ] M: [ RUN A IF Z Y RUN B IF Z Y RUN C ]

The declaration would past person its basal beryllium M. But wait, you mightiness say, this makes the interpreter recursive. As it turns out, however, it does not – you tin simulate the recursion utilizing a information operation called a “continuation stack”. Here’s what the afloat stack hint of that codification mightiness look like, assuming the transaction is [ X, Y ] sending V wherever X > 100, V > 10^18 * 25and contract.storage[X] is not set:

{ stack: [], cstack: [[M, 0]], op: RUN } { stack: [], cstack: [[M, 2], [A, 0]], op: TXVALUE } { stack: [V], cstack: [[M, 2], [A, 1]], op: PUSH } { stack: [V, 25], cstack: [[M, 2], [A, 3]], op: PUSH } { stack: [V, 25, 10], cstack: [[M, 2], [A, 5]], op: PUSH } { stack: [V, 25, 10, 18], cstack: [[M, 2], [A, 7]], op: EXP } { stack: [V, 25, 10^18], cstack: [[M, 2], [A, 8]], op: MUL } { stack: [V, 25*10^18], cstack: [[M, 2], [A, 9]], op: LT } { stack: [0], cstack: [[M, 2], [A, 10]], op: NULL } { stack: [0], cstack: [[M, 2]], op: IF } { stack: [0], cstack: [[M, 5], [Y, 0]], op: NULL }

{ stack: [0], cstack: [[M, 5]], op: RUN } { stack: [], cstack: [[M, 7], [B, 0]], op: PUSH } { stack: [0], cstack: [[M, 7], [B, 2]], op: TXDATA } { stack: [X], cstack: [[M, 7], [B, 3]], op: SLOAD } { stack: [0], cstack: [[M, 7], [B, 4]], op: NOT } { stack: [1], cstack: [[M, 7], [B, 5]], op: PUSH } { stack: [1, 0], cstack: [[M, 7], [B, 7]], op: TXDATA } { stack: [1, X], cstack: [[M, 7], [B, 8]], op: PUSH } { stack: [1, X, 100], cstack: [[M, 7], [B, 10]], op: LT } { stack: [1, 0], cstack: [[M, 7], [B, 11]], op: NOT } { stack: [1, 1], cstack: [[M, 7], [B, 12]], op: MUL } { stack: [1], cstack: [[M, 7], [B, 13]], op: NOT } { stack: [1], cstack: [[M, 7], [B, 14]], op: NULL } { stack: [0], cstack: [[M, 7]], op: IF } { stack: [0], cstack: [[M, 9], [Y, 0]], op: NULL }

{ stack: [], cstack: [[M, 10]], op: RUN } { stack: [], cstack: [[M, 12], [C, 0]], op: PUSH } { stack: [1], cstack: [[M, 12], [C, 2]], op: TXDATA } { stack: [Y], cstack: [[M, 12], [C, 3]], op: PUSH } { stack: [Y,0], cstack: [[M, 12], [C, 5]], op: TXDATA } { stack: [Y,X], cstack: [[M, 12], [C, 6]], op: SSTORE } { stack: [], cstack: [[M, 12], [C, 7]], op: NULL } { stack: [], cstack: [[M, 12]], op: NULL } { stack: [], cstack: [], op: NULL }

And that’s each determination is to it. Cumbersome to read, but really rather casual to instrumentality successful immoderate statically oregon dynamically types programming connection oregon possibly adjacent yet successful an ASIC.

Optimizations

In the supra design, determination is inactive 1 large country wherever optimizations tin beryllium made: making the references compact. What the wide and elemental benignant of the supra declaration hid is that those pointers to A, B, C, M and Z aren’t conscionable compact azygous letters; they are 20-byte hashes. From an ratio standpoint, what we conscionable did is frankincense really substantially worse than what we had before, astatine slightest from the constituent of presumption of peculiar cases wherever codification is not nearly-duplicated millions of times. Also, determination is inactive nary inducement for radical penning contracts to constitute their codification successful specified a mode that different programmers aboriginal connected tin optimize; if I wanted to codification the supra successful a mode that would minimize fees, I would conscionable enactment A, B and C into the declaration straight alternatively than separating them retired into functions. There are 2 imaginable solutions:

1. Instead of utilizing H(x) = SHA3(rlp.encode(x))[12:], usage H(x) = SHA3(rlp.encode(x))[12:] if len(rlp.encode(x)) >= 20 other x. To summarize, if thing is little than 20 bytes long, we see it directly.
2. A conception of “libraries”. The thought down libraries is that a radical of a fewer scripts tin beryllium published together, successful a format [ [ ... codification ... ], [ ... codification ... ], ... ], and these scripts tin internally notation to each different with their indices successful the database alone. This wholly alleviates the problem, but astatine immoderate outgo of harming deduplication, since sub-codes whitethorn request to beryllium stored twice. Some intelligent thought into precisely however to amended connected this conception to supply some deduplication and notation ratio volition beryllium required; possibly 1 solution would beryllium for the room to store a database of hashes, and past for the continuation stack to store [ lib, libIndex, codeIndex ] alternatively of [ hash, scale ].

Other optimizations are apt possible. For example, 1 important weakness of the plan described supra is that it does not enactment recursion, offering lone portion loops to supply Turing-completeness. It mightiness look to, since you tin telephone immoderate function, but if you effort to really effort to instrumentality recursion successful ES2 arsenic described supra you soon announcement that implementing recursion would necessitate uncovering the fixed constituent of an iterated hash (ie. uncovering x specified that H(a + H( c + ... H(x) ... + d) + b) = x), a occupation which is mostly assumed to beryllium cryptographically impossible. The “library” conception described supra does really hole this astatine slightest internally to 1 library; ideally, a much cleanable solution would exist, though it is not necessary. Finally, immoderate probe should spell into the question of making functions first-class; this fundamentally means changing the IF and RUNopcode to propulsion the destination from the stack alternatively than from fixed code. This whitethorn beryllium a large usability improvement, since you tin past codification higher-order functions that instrumentality functions arsenic arguments similar map, but it whitethorn besides beryllium harmful from an optimization standpoint since codification becomes harder to analyse and find whether oregon not a fixed computation is purely functional.

Fees

Finally, determination is 1 past question to beryllium resolved. The superior purposes of ES2 arsenic described supra are twofold: deduplication and optimization. However, optimizations by themselves are not enough; successful bid for radical to really payment from the optimizations, and to beryllium incentivized to codification successful patterns that are optimization-friendly, we request to person a interest operation that supports this. From a deduplication perspective, we already person this; if you are the 2nd idiosyncratic to make a Namecoin-like contract, and you privation to usage A, you tin conscionable nexus to A without paying the interest to instantiate it yourself. However, from an optimization perspective, we are acold from done. If we make SHA3 successful ES, and past person the interpreter intelligently regenerate it with a contract, past the interpreter does get overmuch faster, but the idiosyncratic utilizing SHA3 inactive needs to wage thousands of BASEFEEs. Thus, we request a mechanics for reducing the interest of circumstantial computations that person been heavy optimized.

Our existent strategy with fees is to person miners oregon ether holders ballot connected the basefee, and successful mentation this strategy tin easy beryllium expanded to see the enactment to ballot connected reduced fees for circumstantial scripts. However, this does request to beryllium done intelligently. For example, EXP tin beryllium replaced with a declaration of the pursuing form:

PUSH 1 SWAPN 3 SWAP WHILE ( DUP PUSH 2 MOD IF ( DUPN 2 ) ( PUSH 1 ) DUPN 4 MUL SWAPN 4 POP 2 DIV SWAP DUP MUL SWAP ) POP

However, the runtime of this declaration depends connected the exponent – with an exponent successful the scope [4,7] the portion loop runs 3 times, successful the scope [1024, 2047] the portion loop runs eleven times, and successful the scope [2^255, 2^256-1] it runs 256 times. Thus, it would beryllium highly unsafe to person a mechanics which tin beryllium utilized to simply acceptable a fixed interest for immoderate contract, since that tin beryllium exploited to, say, enforce a fixed interest for a declaration computing the Ackermann function (a relation notorious successful the satellite of mathematics due to the fact that the outgo of computing oregon penning down its output grows truthful accelerated that with inputs arsenic debased arsenic 5 it becomes larger than the size of the universe). Thus, a percent discount system, wherever immoderate contracts tin bask fractional arsenic ample a basefee, whitethorn marque much sense. Ultimately, however, a declaration cannot beryllium optimized down to beneath the outgo of calling the optimized code, truthful we whitethorn privation to person a fixed interest component. A compromise attack mightiness beryllium to person a discount system, but combined with a regularisation that nary declaration tin person its interest reduced beneath 20x the BASEFEE.

So however would interest voting work? One attack would beryllium to store the discount of a codification point on broadside that codification item’s code, arsenic a fig from 1 to 232, wherever 232 represents nary discount astatine each and 1 represents the highest discounting level of 4294967296x (it whitethorn beryllium prudent to acceptable the maximum astatine 65536x alternatively for safety). Miners would beryllium authorized to marque peculiar “discount transactions” changing the discounting fig of immoderate codification point by a maximum of 1/65536x of its erstwhile value. With specified a system, it would instrumentality astir 40000 blocks oregon astir 1 period to halve the interest of immoderate fixed script, a capable level of friction to forestall mining attacks and springiness everyone a accidental to upgrade to caller clients with much precocious optimizers portion inactive making it imaginable to update fees arsenic required to guarantee future-compatibility.

Note that the supra statement is not clean, and is inactive precise overmuch not fleshed out; a batch of attraction volition request to beryllium made successful making it maximally elegant and casual to implement. An important constituent is that optimizers volition apt extremity up replacing full swaths of ES2 codification blocks with much businesslike instrumentality code, but nether the strategy described supra volition inactive request to wage attraction to ES2 codification blocks successful bid to find what the interest is. One solution is to person a miner argumentation offering discounts lone to contracts which support precisely the aforesaid interest erstwhile tally careless of their input; possibly different solutions beryllium arsenic well. However, 1 happening is clear: the occupation is not an casual one.