Update: taking this down for now until some questions about the deterministic
builds are answered; currently I do not believe the Zcash trusted setup should
be called a multi-party computation, making my involvement pointless.
1. The Zcash protocol needs a special trusted setup phase for technical reasons.
2. The secret key ("cryptographic toxic waste") generated during this phase can
be used to steal money - currently in the form of creating counterfeit coins,
stealing from everyone collectively.
3. For the secret key to leak, all six participants need to collude *or be
compromised*. Emphasis on the latter: there are a *lot* of ways that the
participants could be compromised against their will.
As one of those participants, here's my account of what I did to ensure that
secret was destroyed.
First of all, I want to make it clear that I am not receiving any compensation
for my participation in the ceremony; the Zcash Electric Coin Company is paying
my expenses. Once the bulk of my participation is complete, I would be willing
to accept donations for this work, although have not yet.
Secondly I want to make it clear that my participation in the Zcash trusted
setup ceremony shouldn't be taken as endorsement of Zcash. But since many
people have taken it that way anyway, I'm going to start with a summary of some
of my major concerns about Zcash and the trusted setup ceremony.
**Nothing you will read below changes the fact that you're trusting me and five
other participants not to collude. Full stop. End of story. It is IMPOSSIBLE
for myself and the other participants to prove to a third party that we did not
collude to keep the secret key. If you do not believe you can trust me, you
should stop reading now.**
The purpose of this writeup is for education, entertainment, and peer review.
If you are not reading this writeup for one of those reasons, STOP NOW.
### Single Point of Failure
As of writing, I'm not aware of any efforts to independently audit the
deterministic build process used to create the compute node DVDs that every
participant in the trusted setup used. This means there's a massive single
point of failure in the whole process that completely undermines the value of
the multi-party computation.
**Until the software and deterministic builds are audited, the entire ceremony is a bunch of crypto hocus pocus that means nothing.**
You may have heard of the [DNSSEC Root Signing Ceremony](https://www.cloudflare.com/dns/dnssec/root-signing-ceremony/).
It's a rare example of a in-production key signing ceremony; its participants
are trusted with the integrity of the DNSSEC system that (is supposed to)
protect the integrity of the DNS system.
Arguably the Zcash trusted setup makes the DNSSEC root signing ceremony look
like childs play. By that, I don't mean to say that the DNSSEC Root Signing
Ceremony isn't done in a professional manner - it is - but rather than that the
security requirements for the Zcash trusted setup are much higher.
The reason why is incentives: if you have a backdoor to DNSSEC, it's hard to
turn that backdoor into profit. If I had a DNSSEC backdoor what could I do with
it? I guess I could try to MITM some online banking... But I'm not a criminal;
I don't have the contacts necessary to do that on a big enough scale; I don't
know how to launder money; I don't know how to find trustworthy
Or maybe I'd try selling that DNSSEC backdoor to the nearest
Chinese/Russian/United States embassy. But how do I know they'd take me
seriously? How do I know they wouldn't turn me over to the local law
Even if I had a backdoor to DNSSEC, I don't have any easy way to turn that
backdoor into something that'll personally benefit me. On the other hand, if I
had a Zcash backdoor, it'd go like this:
1. Acquire Zcash secret keys.
2. Print money at will.
3. There is no step three.
Even a thirty-something kinda-overweight from-the-suburbs nerd like myself can
pull off that!
The cryptography behind Zcash is both highly experimental, and relatively
weak. Fact is, if zk-SNARKS turned out to be totally broken, unlike more
mainstream crypto, it just wouldn't be all that surprising:
> ...and now you may kiss the...
> RSA IS BROKEN!
> Sorry Steve, I'll be back in a half hour! Gonna factor some primes.
Meanwhile, if the pairing crypto behind zk-SNARKS gets broken:
> Mind reviewing this? I think I broke pairings.
> Kinda busy this week; how about you put it up on arXiv?
> You know Zcash uses them right?
> On second thought, close the door...
On top of that, there appears to be uncertainty about the strength of the
actual parameters chosen for Zcash's crypto. The threat here is that an
attacker may be able to create fake zk-SNARK proofs by breaking the crypto
directly, even without having access to the trusted setup backdoor.
I've had some experts tell me they thought the security level was 2^80
operations (*very* weak), while others (including Zooko himself) thought it was
[more like 2^96](https://moderncrypto.org/mail-archive/curves/2016/000742.html). I'm not
sure which figure is right, but the fact that there's disagreement is a bad
### Scale and Scalability
The current Zcash protocol inherits the same O(n^2) scalability problem that
Bitcoin has, and then makes the problem significantly worse. First of all,
unlike Bitcoin, Zcash can't be pruned: every private Zcash transaction results
in one or more revealed nonces that must be stored indefinitely by all Zcash
More importantly, verifying Zcash private transactions is orders of magnitude
slower than verifying Bitcoin transactions: tens of milliseconds compared to a
few microseconds. This is so slow that if Zcash users used private transactions
frequently, even without attacks mining Zcash could become unprofitable for all
but the largest mining pools even due to slow block propagation effects; if
small miners were not forced out of business the Zcash network could even have
difficulty maintaining consensus.
In my opinion the Zcash team has been irresponsible, maybe even dishonest, in
their choice of block interval and size parameters.
### Equihash Proof-of-Work
Zcash uses the Equihash PoW in an attempt to make hashing more decentralized.
The hope here is that Equihash will be "memory-hard", with the costs to hash
dominated by the cost to acquire and operate commodity DRAM rather than
specialize hardware. Equihash hashing is comprised of basically two main
1. Fill a fixed-size memory buffer with the BLAKE2b hashing algorithm.
2. Repeatedly sort that memory while doing a trivial computation.
Since BLAKE2b is fairly efficient, the cost is supposed to be dominated by
the repeated sorting; for Zcash the fixed-size memory buffer was expected to be
about 1GiB. However, as of writing Tromp has managed to get memory requirements
down to just 144MiB! The Equihash paper predicted that kind of optimization
would incur a multiple orders of magnitude penalty... not a good sign.
More generally, the Equihash paper never actually properly analysed the actual
*cost* of implementing Equihash on real hardware. Instead they assumed that
implementations will use commodity DRAM, and that hashing cost will be
dominated by the capital expense of acquiring that DRAM.
That's a big assumption. For example, Tromp has reported good equihash
performance on NVIDIA GTX980 GPU, which as of writing has a retail cost of
about $500 USD and comes with 4GiB of GDDR5 DRAM; if Equihash really works as
intended, the cost of that RAM should dominate hashing costs.
So what is the cost of that RAM? $10-$20, depending on how dodgy of a supplier
you're willing to use. That's a lot of room for improvement with custom
hardware. For instance, it's easy to imagine a $5 ASIC being developed to act
as a highly specialized memory controller; Zcash hashing hardware would simply
be that ASIC surrounded by RAM chips.
The situation gets even worse though when you take energy costs into account.
Fully utilized GDDR5 DRAM needs about 100mW/GiB/s of bandwidth; a GTX980 has
about 225GiB/s of bandwidth, so we'd expect it's GDDR5 DRAM to consume around
22W. At $0.05/kWh that's roughly $10/year minimum - probably more like $20/year
once inefficiencies are taken into account. You'd want to do a more in-depth
analysis than my hand-waving, but the point is, this strongly suggests that over
the lifetime of Equihash hashing equipment, energy costs are higher than
the capital costs of aquiring DRAM.
So can we reduce those energy costs? The Equihash paper states that:
> To the best of our knowledge, the highest reported bandwidth in commercial
> products does not exceed 200 GB/s (Playstation 4 has 172 GB/s bandwidth),
> whereas the desktop DDR3 can have as high as 17 GB/s bandwidth.
> Even if it was the case, the highest reported bandwidth applies to large (a few
> GB) memory chips, for which it is not so hard to place sufficiently many
> reading pins to get the bandwidth. For smaller chips (700 MB) we presume it to
> be much harder.
Their information is very out of date. The current state of the art in both
bandwidth and power consumption is to stack memory dies directly on top of
logic dies and interconnect them with microscopic thru-silicon vias:
![High Bandwidth Memory Diagram](/assets/2016-11-14/hbm.png)
This technique has even been standardized by JEDEC as the [High Bandwidth
Memory (HBM)](https://en.wikipedia.org/wiki/High_Bandwidth_Memory) interface;
Xilinx uses this technology in their UltraScale+ FPGA line, offering 4GiB and
480GiB/s in a single package. A major part of the energy used
by DRAM is consumed by driving long wires at high speed to move bits to compute
units and back; the shorter wires enabled by HBM can reduce IO power
consumption by 10x or more[^hbm-over-gddr5].
We haven't even discussed more exotic possibilities like custom DRAM ASICs, but
I think I've made my point: it's quite likely that custom Equihash hardware
will be developed if Zcash catches on, and that hardware will reduce costs at
least one or two orders of magnitude.
This by itself isn't worse than Bitcoin's PoW. But there's a big potential
problem: because this hardware may be significantly more expensive to develop
than SHA256 ASICs, we may end up with a winner-take-all situation where the
first firm to develop a Equihash ASIC has a massive head start over its
competitors. Exactly what Equihash was trying to avoid.
### Founders Reward
20% of coins for the first four years are allocated to a "founders reward",
intended to compensate the investors and developers who created Zcash. However
20% is extremely high - for starters it gives 51% attackers a significant
advantage. Secondly, because the implementation of the reward sends those funds
to a single multisig address that's rotated periodically, the address itself is
a major single-point-of-failure for the currencies monetary supply: a
compromise of that address could easily lead to the price of Zcash crashing,
for instance by a thief selling their coins. Equally, governments could force
the Zcash Electric Coin Company that controls that address to dump coins on the
market in an attempt to hurt Zcash. This should be a significant concern for
anyone thinking about investing in Zcash.
Secondly, much of the communication around the founders reward has been at best
borderline dishonest, by stating the founders reward is 10%. That figure is
true when you consider the entire lifetime of the coin supply, but given this
is an investment, I believe Zcash should err on the side of caution and stress
the 20% figure that is relevant to investors today - Zcash may not even be
around in four years.
Finally, the fact that the Zcash team has a very obvious and very direct link
between their income and the price of Zcash is legally risky, which in turn is
a risk to the whole Zcash community due to the highly centralized way Zcash
protocol development is organized, particularly with the Zcash trademark
(discussed below). After all, there are obvious similarities with Zcash and
highly regulated constructs like stock offerings.
### Zcash™ Trademark
The Zcash Electric Coin Company has both trademarked "Zcash", and threated
others with lawsuits for using that term. This is a centralization risk: it'd
be easy for control of that trademark to be used to bully alternate development
teams from improving the protocol and/or implementations of it. For example,
the US Government could order the Zcash company to add AML/KYC features to the
Zcash protocol - as was done with the Ripple protocol - and then use the
trademark to prevent other teams from promoting alternate directions for
protocol development, or even simply promoting that the protocol remain
unchanged. Consider the legal risks faced by an exchange who simply wants to
ignore the changes that the wider, decentralized community has rejected, but
now has to contend with the fact that continuing to call Zcash, "Zcash",
exposes them to lawsuits from owners of the trademark.
Note too that Zcash has been specifically designed in a way that makes adding
AML/KYC to it possible because it is possible to prove the origin of funds by
revealing a view key.
### Software Quality
I personally ran into a bug that caused the Zcash wallet to crash the daemon
with an assertion error within the first 30 minutes or so of using Zcash; I
wasn't trying to do anything clever, nor was I looking for bugs. Along with the
other bugs that had to be fixed right after the 1.0.0 launch, I think we have
strong reasons to wonder if the current software quality is at the level needed
in a crypto-currency.
Additionally, there are concerning signs that the team doesn't yet have a
solid crypto-currency background, such as (still open) [issue
713](https://github.com/zcash/zcash/issues/713); Zcash doesn't have any staff
or advisors with significant architectural design and implementation experience
on successful, stable, crypto-currency systems used in production (Ethereum is
neither used in production for its intended purpose, nor has it yet proven to
For investors, a potential concern here is that the Zcash trademark may make it
structurally difficult for other teams of developers to participate and improve
the protocol that the economic majority is using. By contrast in the Bitcoin
space we have multiple competing teams with quite different visions for the
Bitcoin protocol, yet they all make use of the term "Bitcoin" and are all
promoting their efforts to the wider economic community, who in turn has the
ability to choose between them.
### Disclaimer Disclaimer
Keep in mind that my job here isn't to promote Zcash, so the above (and below)
emphasises negatives and doesn't talk about positives; I'm erring on the side
of caution in making sure that my readers leave with a broad understanding of
what problems exist.
Also, all these problems can be fixed, some more easily than others.
## Initial Contact
Zooko initially asked me on Sept 26th via unencrypted Twitter DM if I was
interested in participating in the 2016 Zcash parameter generation ceremony as
a "Human Witness". I asked him to move the conversation to Signal, noting that:
> [W]hile I don't care that much... for the record keep in mind that the moment you ask someone to participate in this, you potentially expose them to pretty big threats. At minimum, if I were in your position I would have asked over Signal, not easily wire-tapped twitter DM's.
Later that day Zooko contacted me via Signal. I asked him to PGP sign the
Signal safety numbers so I could be sure I was actually talking to him, which
he didn't do.
Two weeks later on Oct 14th Zooko contacted me again via Signal, saying he
"Still [hadn't] gotten around PGP-signing my Signal fingerprint" and asking if
I could be a part of either the Oct 15-16th ceremony, or Oct 22-23rd ceremony.
I ignored the message until later that day Zooko finally sent me a PGP-signed
email confirming the Signal safety numbers (and for that matter, his phone
number!). tl;dr: I agreed to be a part of the Oct 22-23rd ceremony.
Zooko reached out again on Oct 18th outlining what would actually happen on the
22-23rd weekend. At the time I had just arrived in San Francisco for the
[Rebooting the Web-of-Trust Conference](http://www.weboftrust.info/). We
discussed me potentially teaming up with Michael Perklin, however he had
already bought the offline hardware for the ceremony, so I rejected that
suggestion as I wouldn't be adding any value as a witness - if the hardware may
be backdoored it's game-over already. As Zooko put it:
> [So] for you to feel like you're providing full redundancy, you wouldn't want part of your explanation to be "and then I relied on Perklin's claims about what he had done..."
I also asked a security expert and developer I knew in SF if they'd be
interested in helping me with the trusted setup. But I didn't get a response
back in time, so I decided to do it alone.
## Threat Modeling
At the time I didn't actually know much about how the trusted setup actually
worked; I hadn't even been given a copy of the instruction sheet. But I knew
the basic architecture:
At one level my job was very simple: run the software on the compute node,
shuffle data to/from the network machine over the air-gap as required, and
ensure the secret is securely deleted when everything is done.
But how was I going to make sure the secret didn't get leaked? Let's look at
the threats I faced:
### Backdoored Software
As mentioned above, the software used by every compute node was identical and
thus a single point of failure that could be backdoored; I actually raised this
as a issue publicly with Zooko a few weeks prior to the ceremony on Twitter.
There's also a second problem: what if my laptop was already compromised? I
knew I'd need to burn a DVD with the compute node software, and since I was
travelling at the time I only had my travel laptop available to do it with.
It'd be possible for an attacker who had backdoored my laptop to cause it to
burn a DVD containing something other than the correct compute node software.
I knew I couldn't actually solve either of those problems directly at the time.
So instead I decided to rely on the fact that the ceremony instructions were to
use a (hopefully!) write-once boot DVD, which could later be audited to ensure
that the correct software was actually used.
Of course, this raises an important question: Is it really impossible to modify
a DVD-R once written?
### Backdoored Hardware
Obviously, if the hardware used has a backdoor, all bets would be off. Here in
the Bitcoin space we even have a suspected example of this in action: it's
plausible that the way Craig Wright fooled Gavin Andresen into thinking he was
Satoshi was via a backdoored laptop.
Commercially available computers are filled with firmware that can undetectably
backdoor your system. For example, academics have demonstrated hard-drive
firmware backdoors that steal
Kaspersky has reported[^equation-the-death-star-of-malware-galaxy] these
exploits in the wild, and the Snowden leaks have
confirmed[^s3285-intern-projects] the NSA has hardware level exploit
capability. I personally was told by a friend of mine who owns a
security-related hardware company that a former NSA employee once applied for a
job there, and when asked about qualifications, they eventually admitted while
at the NSA they had worked on "consumer firmware"; my friend told me that guy
Like every other ceremony participant, my primary defense against hardware
backdoors was to acquire hopefully clean hardware by purchasing brand new
hardware from a randomly chosen store in person with cash. The assumption here
is that it will be infeasible for an adversary to actively backdoor the
hardware purchases because the large number of possible sources of it.
### Physical Attacks
A physically present attacker has a lot of options. It's safe to assume that if
an attacker got direct access to the compute node, even for just long enough to
briefly plug in a USB cable, it'd be game over: it's quite plausible that an
adversary could exploit a firmware-level bug in a USB controller and from that
take over the whole machine. So obviously I'd need ensure that any such
attacker would be detected, e.g. by being caught on video.
Physical attacks was one of the reasons why I was inclined to limit the number
of people I performed the ceremony with. As I said to Zooko:
> I suspect [having multiple observers present] just introduces ways to exploit, rather than removes them. There's fuck all you can do against slight of hand with a skilled magician.
#### Wireless Attacks
It's quite plausible that an attacker could compromise a machine via an attack
over WiFi or Bluetooth; some computers even have remote management capability
at the chipset level via wireless, e.g. Intel's Active Management Technology
(AMT). Equally an attacker could use a wireless interface as a way to get
undetectably exfiltrate data from the compute node, bypassing the DVD audit
Computers have other wireless interfaces too: microphones and accelerometers
are also wireless data inputs. Fortunately I think it's *highly* unlikely that
there exist any ways of compromising a not-previously-backdoored computer via a
microphone or accelerometer, and I say that as a former analog electronics
#### Electromagnetic and Acoustic Leakage
Computers leak electromagnetic interference that often corresponds to the
state of the computation being performed. This is a very real and very
plausible attack: in fact one of the members of the Zcash team, Eran Trommer,
has demonstrated real-world[^cve-2015-3591] attacks against GnuPG that steal secret keys via
both electromagnetic[^stealing-keys-from-pcs-using-a-radio] and more recently
even *acoustic* leaks![^rsa-key-extraction-via-acoustic-cryptanalysis]
[^cve-2015-3591]: Sufficiently real world that they even resulted in multiple CVE's, like [CVE-2014-3591](http://people.canonical.com/~ubuntu-security/cve/2014/CVE-2014-3591.html)
There's multiple different ways these attacks can happen:
1. High Bandwidth --- The state of the computation is directly measured by
the attacker via leaked radio waves. Fortunately modern computers are so fast -
GHz - that the radio waves emitted are both very difficult to record with
anything but highly-specialized equipment, and are relatively easily shielded.
2. Low Bandwidth --- The state of the computation is *indirectly* measured by
changes in power consumption. The lower frequency makes it much less likely for
the leak to happen via radio waves - a computer usually doesn't have any metal
long enough to act as a good antenna - however it does make it possible for
leakage to happen via electric field, magnetic fields, and sound. In
particular, the switch-mode power supplies that are a ubiquitous part of modern
electronics are usually designed with planar magnetics that emits a relatively
large magnetic field that can easily couple to other things in the vicinity.
Secondly, sound is generated directly by switch mode power supplies, both due
to the magnetic components acting as speakers, and due to fact that most
ceramic capacitors are made from piezoelectric materials that physically
vibrate as the voltage across them changes.
A big problem with the low-bandwidth attacks is that the signals emitted can be
sufficiently low frequency to be picked up via audio inputs such as
microphones, either directly in the case of acoustic leakage, or indirectly
via magnetic fields coupling to the analog input circuit. The latter is of
course a very common problem, and is often seen in the form of unwanted 50/60Hz
hum in audio recordings.
This means that an attacker could make use of "found" equipment to perform the
attack. For instance, a cell phone left next to the compute node could
potentially be backdoored in an over the air attack against the baseband
firmware, then the microphone in that cell phone used to pick up an acoustic
or magnetically coupled leak. Tromer's team have even demonstrated this exact
attack against GnuPG, stealing a secret using a plain mobile phone placed next
to a laptop.
Secondly, an attack could happen after the fact. Any audio recording such as a
surveillance video or video taken by a journalist could potentially have
unintentionally recorded leaked data, which an attacker can later extract.
Fortunately low-bandwidth attacks can be prevented in software by using
cryptographic algorithms that use constant power. I have no idea whether or not
the compute node software has this property; it would make for a good research
Now that we've discussed what I was up against, let's look at what I actually
did to attempt to defeat these threats:
### Thursday 20th - Pre-Ceremony
As mentioned previously, I happened to be in San Francisco prior to the
ceremony. I had been thinking all week about a plan, and had finally decided:
my station would be mobile. As I said to Zooko that morning:
> I'm going to do a bit of travel for it. My assumption here is if there is a threat, someone is going to have to physically follow me to get close enough to tamper with the process, so I'm going to optimise to detect such threats.So I plan on sending any such agents on a bit of a chase, and I'd[sic] they exist, hopefully catch them on camera.
What do I mean by mobile? I didn't actually tell Zooko this at the time, but I
had decided to rent a car and do the actual computation in it, spending as much
time as (safely) possible driving and using a cell phone data connection to
Now imagine your an agent assigned to compromising my station. If there aren't
any location independent remote attacks, what do you have left? Physical
attacks. For example, if you knew that Zooko was camped out in a hotel, you
could rent the room next to him and spend the whole weekend with a bunch of
radio gear doing a EMI attack. But how do you do that against a station hurtling
down the highway? You've got a whole bunch of problems:
1. How do you find them?
2. Once you find them, how do you get close enough, long enough, for your fancy
gear to work?
3. How do you avoid being seen while doing this?
Of course, that's not to say this is impossible, or that my solution was
necessarily better than hiding out in a remote forest cabin all weekend, but at
least it'd be a very *different* threat model than the other stations.
My second decision was where to do this. I knew at that point that at least one
or two stations were going to be in the US, so I decided that I'd do the
ceremony in Canada to provide some jurisdictional diversity. Being a Canadian,
I also wanted to be subject to the Canadian legal system rather than US legal
system in case anything went wrong.
More specifically, I decided to go to British Columbia on the west side of
Canada for a few reasons:
1. It's much closer to SF, and in the same timezone; had I gone back home to Toronto I would have had much less time in the day to prepare.
2. I have family living there who could help out if needed.
3. I know the area better. While I've lived in Toronto almost all my life, I've
spent much more "outdoors" time in BC and know the highways and towns in the
area much better than I do Ontario.
And of course, who doesn't want to do a roadtrip in the rockies!? I mean, just
look at it!
![My Favourite Rock](/assets/2016-11-14/my-favourite-rock.jpg)
So with that in mind, I booked a 10am next-day flight on CheapAir (with
Bitcoin!) from SFO to an airport close to my family in BC. Under my name of
course - it's impossible to fly to Canada on commercial flights without showing
ID to the airline, and I'm definitely not going to break the law by using a
fake ID (and I don't even know where to get one!).
Later that day I left the web-of-trust conference I was at an hour early, and
bought two GoPro cameras at Target, a cheap still camera, and all the SD Cards
they had (not many!) and went back to my hotel. Before I went to bed I made one
last phone call to someone I trusted via Signal and told them my plan: head to
where my parents are and do the trusted setup there. After that, I put my phone
in airplane mode.
## Friday 21st - Prep Day
Wait, did you really think I'd buy a plane ticket in the clear? Fuck no. Even
if the attacker wasn't government, god only knows how many people have legit,
and illegit, access to the passenger lists kept by airlines.
Actually, after that call I did one last thing: I checked the flight schedules
to Vancouver for the following morning over Tor. The first flight of the day
was at 6:15am, so I woke up at early and took the hotel shuttle bus to SFO
airport. I got there about an hour prior to departure and bought my ticket
right at the counter.
My rational for the dummy ticket was simple: Vancouver is a few hundred KM
away from the dummy ticket's destination, so if anyone was in fact trying to follow me I'd buy at least a few hours
while they travelled to my new location.
Just prior to my flight I bought a new Android phone and backup SIM
card from one of the BestBuy vending machines; the SIM card was one of those
sketchy and overpriced $0.30/MB things, but I figured it'd be good to have a
On the plane I wrote up a plan - and shopping list - on my (paper!) notebook.
Upon arrival at Vancouver I realised I had forgotten to download the
OpenStreetmaps map for BC, so I turned on WiFi on my phone one last time to do
that via the airport WiFi.
Next task was renting a car. Similar to the random purchase concept with the
computer, I picked one of the half dozen rental companies at random to maximize
my chances of getting a "clean" car.
Car in hand I drove to Vancouver city center. I needed cash though, so I went
to the first bank I found and withdrew $3k CAD; it's _way_ too easy to be
tracked via credit cards! This was the first time I used the two GoPros: I
positioned them on the dashboard to get a view of the interior of the car
looking backwards, and a view looking forwards. The idea here is I wanted to
make it difficult to do things like plant bugs on the car without being caught
While I had a map, unfortunately OpenStreetmaps doesn't have shop information;
I hadn't had enough time the night before to plan out where I'd be going in
detail. And of course, I had my phone in airplane mode the whole time. So I was
mostly driving at random and stopping whenever I found a store selling
something I needed. In order:
1. Home Depot --- Screwdriver set
2. Safeway --- Food, drinks, aluminum foil, aluminum pans
3. Staples --- The compute laptop! Zooko had told me I needed a fast Intel i7,
so I would up buying a brand-new MSI GE62 gamer laptop; there's not many
choices when you want a brand new laptop with a DVD burner. I also bought the
DVD-R media I used later here.
4. Random used laptop store --- I bought a second laptop to use as the network
node; the disk image only had ethernet drivers for networking, so I planned on
using *three* laptops total, with my existing personal laptop acting as a
bridge between the phone and ethernet.
After this I headed out of Vancouver on the Trans-Canada Highway, turning my
phone off entirely and wrapping it in aluminum foil around Langley. The idea
here is if airplane mode wasn't actually working, I wanted to put some distance
between my last known location and where I turned on the never-turned-on burner
Of course, I also wanted to get a better SIM card for it than the dodgy one I
picked up at the airport... Unlike the US, getting SIM cards in Canada without
giving up your identity isn't trivial, but suffice to say, I was able to pull
that off. It was getting late, so I decided to finally break radio silence and
gave Zooko and the team a status report at Abbotsford - not as far from
Vancouver as I had wanted. Also in Abbotsford I finally was able to buy a
decent 120W inverter to actually power the laptops and other gear at a Canadian
### Surveillance Strategy
Now, if you've been paying attention, you might ask what did I do with the
laptop during all this shopping? I relied on the GoPros to watch them for me,
setting them up to record video every time I left the car. On the one hand, I
was the only station operator who ever let anything out of their site; I recall
Zooko(?) mentioned how he put his compute node in the bed while he slept.
My rational for this was in part pragmatic: carrying around three laptops and a
bunch of other equipment like DVDs, ethernet cables, etc. is a huge pain,
and awkward too; I might end up breaking something, or attract attention from
all the gear.
There's also a more devious reason: I *wanted* to tempt any adversaries into
trying to tamper with the equipment. Regardless, they'd be taking a big risk
for the simple fact that they'd have to break into the car to do it.
I also didn't have a choice anyway: once the ceremony had started, the
laptops needed to be left running constantly, and I didn't want to risk messing
with them. I'd probably have to leave them alone on occasion anyway to do
things like pay for gas, so might as well get over it and trust the
Of course, the downside is now I have a pile of surveillance footage that needs
to be carefully reviewed. I've done a preliminary check of all of it, but
another review is a good idea. I was also taking quite a big risk: if a camera
failed for any reason, I could end up with a gap in the chain of custody. For
example, when I got dinner later that night, the lightning was poor enough that
the interior camera didn't get a great view. I happened to be able to see the
car clearly where I parked it, but that's not as convincing as it could be.
I needed a place to stay for the night. Sure, I could have tried to
sleep in the car, but doing the ceremony while tired didn't sound like a good
idea, particularly given that I was driving. Also, I needed a table to do tech
However I didn't want to give away my location. So I found a
rather odd independent motel in Hope and paid with cash. I *did* check-in under
my real name - trying to do otherwise was just going to attract attention - but
being an independent the whole process was done on paper, so no risk there!
I had been hoping to be able to park the car in such a way as to be able to
take surveillance footage of it from the room window, but wasn't able too; I
considered finding another motel, but in the end decided not to as it was
still raining hard and I was getting pretty tired.
### Ceremony Tech Prep
That night at the motel I finally was able to burn the network and compute
DVDs required for the ceremony. I did this in with
[Tails](https://tails.boum.org/), and made (labeled) primary and backup DVDs
A tricky part of this was I didn't want to turn the WiFi adapter on my personal
laptop on; it was a Thinkpad T520, so I used the "airplane mode" switch on the
side. The concern here is that it'd be easy to leak where I was via the MAC
address of that laptop; here in Canada the Snowden leaks revealed that our
equivalent of the NSA, CSEC, was illegally using MAC addresses to [track people at airports](http://www.cbc.ca/news/politics/csec-used-airport-wi-fi-to-track-canadian-travellers-edward-snowden-documents-1.2517881).
So I used a copy of Tails that I already had on the laptop, and did the actual
download of the network and compute node iso's on the newly purchased T530.
Next task was modifying the compute node. Without ever booting it up, I took it
apart and removed the WiFi/Bluetooth adapter, along with the SSD and HD
storage. I probably should have removed the microphone at this point, but
didn't think of it.
After putting it back together the compute node was booted up for the first
time. Note how the fact that I powered it up for the first time only after the
wireless interfaces were removed, to avoid it being compromised wirelessly.
Finally, I booted the compute node DVD and ran the DVD burn test, leaving the
compute node [here](https://github.com/zcash/mpc/blob/2f88b419fb0784d1c0b2357bf477474199c6fceb/src/compute.rs#L70]);
I didn't shutdown the compute node after this point until the end of the
I didn't bother dragging all that equipment into bed with me for the night, but
I did take one precaution against any ninja's attempting to sneak in:
No, I'm not trying to hold the door closed; I'm trying to make sure that if the
door is opened, I'll get woken up.
## Saturday 22nd
### Shielding the Compute Node
The next morning I did one last bit of preparation, and built a faraday cage
for the compute node from the box it came by lining it with aluminum foil.
Believe it or not, this is actually a totally legit way of both shielding
electronics from outside interference, and preventing EMI from leaking; I used
to use it myself on occasion in my previous line of work as an electronics
First of all, you might ask how did I ground it in a car with rubber tires? The
answer is simple: you don't have too. I'm only concerned about relatively high
frequency electromagnetic signals getting into and out of the compute node;
grounding a faraday cage is only relevant if you are trying to shield something
from a static DC field.
Secondly, the way that a faraday cage works - and RF shielding in general - is
basically that the metal in the shield "shorts out" the electromagnetic wave.
Specifically the magnetic component of the electromagnetic wave causes the
electrons in the metal to move, creating what's known as an eddy current.
However, that current also creates a magnetic field too, and as it happens,
that magnetic field will be opposite the first field, cancelling them out.
The better the conductivity of the shield is, the more closely the two fields
cancel out. The problem is, at high frequencies like radio waves, conductors
don't act like conductors anymore; if a high frequency current in a good
conductor is forced to go in a loop, the current will act as though it's going
through a poor conductor instead.
So I made sure to buy the widest aluminum foil available, and covered the inside
of the box with foil in big, continuous strips running from top to bottom in
one go. Secondly, I made sure that at the edges the top and bottom halves of
the clamshell box overlapped as much as possible with a snug fit to give as
many possible paths for current to flow between the edges. A quick check with a
cell phone confirmed that the box did at least successfully shield both cellular
### Setting up the Car
Next I moved everything to the car. This was somewhat tricky as I needed to
maintain surveillance of both the room and the car, so I left a GoPro running
in the room while I setup the second GoPro in the car, then did a few trips to
move all the electronics equipment over with the laptops going last. Note that
the compute node was powered on at this point, running off battery.
Here's a photo of the final setup. Front seat:
While still in the motel parking lot, since I had WiFi available in case I had
any networking issues, I decided to do the first part of the ceremony and hit
enter. Doing that resulted in a [prompt to add additional entropy](https://github.com/zcash/mpc/blob/2f88b419fb0784d1c0b2357bf477474199c6fceb/src/compute.rs#L70):
Please type a random string of text and then press [ENTER] to provide additional entropy.
That's the point where this is getting serious. While I hadn't seen the source
code for the MPC process at that point, it was obvious that I was probably
about to generate the secret. And I was right.
A potential risk here is the secret getting leaked via the cameras or some
other device with a microphone, so I turned off the cameras, stashed away the
cellphone, and took a look around to see if the parking lot was empty.
Satisfied, I proceeded to fill the screen up with random "typing" - mostly just
mashing the keyboard randomly to make the sound as unlike normal typing and as
confusing as possible.
That [generated a commitment](https://github.com/zcash/mpc/blob/2f88b419fb0784d1c0b2357bf477474199c6fceb/src/compute.rs#L78),
which I then entered into the network node as instructed, "locking in" my role
in the MPC.
Interestingly, as I was leaving the motel I noticed that someone else had
entered the lot with a truck and horse trailer and parked across from me; I
circled back to see who they were. They looked like locals, so I left.
They also left soon after me and went in the opposite direction.
Having committed, I was actually done for quite a few hours; I was scheduled
to be the last person to do the next step in the process, disc 'A'. So I
starting driving north up highway 97 towards Merit and did some sightseeing.
But first, gas; it's a 120km without any service stops:
![Filling up in Hope](/assets/2016-11-14/filling-up-in-hope.jpg)
Along the way I got some breakfast:
Not a bad view!
When I got there, I took the opportunity to buy some SD cards... This was
actually a persistent problem: I hadn't actually done the math on how much
space I'd need for all the GoPro footage, having never used them. Turns out
it's a lot! I wound up buying every SD card this store had, something I'd end
up repeating a few times:
![Micro SD Card Receipt](/assets/2016-11-14/merritt-sd-cards-receipt.jpg)
### Highland Valley Copper Mine
After Merritt I headed north to Ashcroft via Logan Lake. Along the way is the
currently the largest open pit mine in Canada, and one of the largest in the
world. They have a rest stop on the hill overlooking the mine:
It has a bunch of cheery displays talking about the environment:
![Rest Stop Displays](/assets/2016-11-14/highland-valley-copper-rest-stop-info-panels.jpg)
Speaking of the environment, a few minutes up the road gives you a much more
interesting view: the tailings pond.
Don't let the pretty blue color deceive you: that's a 10km long, 2km wide, 150m
deep former valley that's now filled with 1.3 billion tons of waste rock from
the mine. The dam behind me is 3km long, so I thought a FUCK YEAH ENGINEERING
selfie was appropriate:
![Tailings Pond Selfie](/assets/2016-11-14/highland-valley-copper-tailings-pond-selfie.jpg)
The funny thing is, I thought the place looked kinda familiar... and I
remembered visiting a big open pit mine when I was a kid. So I asked my
parents, and sure enough it was the same mine.
Notice how I'm getting a signal, even though I'm in the middle of nowhere next
to a tailings pond?
Actually, I lied earlier: this isn't going to be a roadtrip through the
beautiful Rocky Mountains. This is going to be a roadtrip through the British
Columbia interior. While the rockies are beautiful, they have two big problems:
tourists and cell phone coverage.
What I actually want for this trip is to be somewhere as empty as possible, to
make any attackers trying to follow me stand out, while still having good cell
phone coverage. The interior is perfect for this because it's a giant plateau
that's frankly kinda boring. This means there's little if any tourist traffic,
especially in late October, the middle of the off season, and not many people
voluntarily move there.
However, because there aren't annoying mountains in the way, there's a lot of
industry, logging, mining, farming, etc. And since truck drivers - and their
employers - like to be able to stay in touch, coverage is actually surprisingly
Coming into Ashcroft, you can see things are getting quite a bit flatter than
you'd normally think of when you think of British Columbia:
But it's still kinda pretty, so I stopped at a A&W for lunch. Or to be exact, I
went to their drive-thru and ate in the scenic parking lot:
### The Chasm
I continued north on Highway 97 and got one last bit of sightseeing in at the
aptly named [Chasm Provincial Park](http://www.env.gov.bc.ca/bcparks/explore/parkpgs/chasm/):
Apparently by the time I finally managed to get myself and the valley into the
frame at the same time I was getting a bit annoyed...
### ~~70 Mile House~~ ~~93 Mile House~~ ~~100 Mile House~~ 150 Mile House
After the chasm I don't really have very many photos. Here's why:
See, that part of BC is a place so boring they named all the towns after mile
![X Mile House map](/assets/2016-11-14/x-mile-house-map.png)
It's also where I came up with the name "Cypherpunk Desert Bus"
On the bright side, my plan was working just fine: the whole drive I had cell
### Disk A
Finally my turn rolled around, well after dark. So I pulled off at the next
rest stop somewhere near Australian, BC and downloaded disc A, right in front
of an outhouse:
When that was done I opened up the compute node's box, put the DVD in,
pressed enter to start the computation, and quickly closed the lid again. I had
about 30 minutes before the computation would be done, so I started driving
down the highway with cameras running.
Now, I gotta admit, it was a somewhat weird feeling to be finally doing the
ceremony for real. I don't know exactly how the math works, but for all I knew,
in the unlikely event someone was actually trying to steal the keys I would
have been in the middle of nowhere, after dark, and being followed.
So I decided to take the next side road I found... and ran right into Spectra
Energy Transmission's CS No. 5 Australian facility. I'm not quite sure what
that facility actually does - something to do with a natural gas pipeline I
think - but regardless, it continuously emitted a loud turbine-like screeching
noise. Now I don't know about you, but I figured hanging around a natural gas
facility at night with two GoPros and three laptops might be
misinterpreted, so I found another side road nearby to it and drove down it
That also turned out to be not the greatest idea. Not only did my side-road
turn out to run right past the facility, it hadn't been used in years and at
one point I had to drive around a fallen tree:
While the road did have the redeeming feature that I was probably not
trespassing, I decided I wasn't doing my credibility as a non-terrorist any
good, so I headed back to the highway. On the bright side, around this time the
computation finished, which meant I was able to start burning disc B.
After one more jaunt down a side road - this time past a few residential houses -
so I pulled over yet again to upload disc B. And promptly ran into a problem:
I'd run out of credit on my SIM card.
Actually, I knew this was a risk and had tried to refill it hours previously
with [Bitrefill](https://www.bitrefill.com/). Which would have worked just
fine, except that almost all my BTC were on my laptop... which was temporarily
booted into Tails and acting as a bridge between the burner phone and the
ethernet-only network node. It had taken a bit of fiddling to get that setup,
so rather than reboot it I had been trying to get someone else to loan me the
BTC. And of course, no-one in the group had any handy either!
At that point I was getting pretty tired - and potentially about to delay
everyone else to boot - so I decided to cut my losses and use my normal phone
instead. Which of course would have immediately given away my location to any
attacker with access to cell phone tower records.
A nuance here was that I still didn't want to give up my identity to everyone
else on the team. We were using Signal group chat to co-ordinate, and I had
used my burner SIM to register. So to be able to keep using that I ended up
having to also enable the WiFi hotspot on my regular phone; usually Signal
works fine with your original number if you change SIM cards, but I didn't want
to risk messing things up.
In any case, once the upload was complete I told Zooko that I'd be going radio
silent until the morning and turned off both phones to go find a motel without
Unfortunately I didn't really have good options. I could either stay near Quesnel -
only a few minutes away - or drive about 120km north to Prince George, or an
equal distance south to Williams Lake. And when you think about it, all the
options are basically the same anyway: there's only about a dozen places to
stay in a 300km radius of Quesnel, so any attacker who knew what car I was
driving could check every possible place in a few hours.
So I decided to head to Quesnel, and wound up finding a motel on the side of
the highway about a few km before Quesnel city center. Like the previous night,
the check-in process was 100% paper based; they hadn't even configured their
cash register to print their name on the receipt:
![Quesnel Motel Receipt](/assets/2016-11-14/quesnel-motel-receipt.jpg)
I carefully brought all the equipment inside for the night, leaving it on the
bed. Curiously, the room was massive, quite possibly both the largest room I've
ever stayed in, and the cheapest:
![Quesnel Motel Inside](/assets/2016-11-14/quesnel-motel-inside.jpg)
I also setup my anti-ninja alarm:
Unlike the previous night, I was able to park right in front of the room,
allowing me to setup a GoPro in timelapse mode with a view of the car:
![Car Cam Setup](/assets/2016-11-14/quesnel-motel-car-cam.jpg)
I ended up going to bed at around 9:30pm; pretty early, so I didn't bother
setting my alarm.
## Sunday 23rd
...and sure enough, I slept for almost 12 hours; I should have known that'd
happen given I had gotten hardly any sleep the past three nights. In any case,
I quickly moved everything back to the car, returned the room keys, and started
### Disc C
Once I got to Quesnel itself I found a residential road and turned on my cell
phone finally, and found out that the not only was it my turn, everyone had
been wondering where I was for the past hour and a half. So I got everything
reconnected while idling next to a church with a horse out the front:
Once it had finished downloading, and the DVD was burning, I started driving
again. I wound up stopping at what looked to be some kind of water truck refill
station next to a highway interchange:
![Water Refill Station](/assets/2016-11-14/water-refill-station.jpg)
Not such a bad place to be actually: I was basically in the middle of a big
field, and sure no-one was near the compute node. This is where I actually
moved disc C to the compute node and started the computation.
Disc C was expected to take 90 minutes to compute - the longest of the three -
so once I got the compute node running on it I started driving to Prince
George. Though I made a quick detour to the Cariboo Pulp & Paper mill nearby
![Cariboo Pulp and Paper](/assets/2016-11-14/cariboo-pulp-and-paper.jpg)
I also did a few big loops through the rural roads north of Quesnel, including
a drive to the end of the road near Ten Mile Lake (yup, yet another X Mile
name...). When the computation was finally done I pulled off to a rest stop in
the middle of nowhere by Hush Lake to upload the DVD... and was complemented
on how fast it went. Here's the sat view of Hush Lake:
Completely empty other than the rocks, trees... and a big cell tower a
kilometer north of me. Presumably, it was so fast because I was the only person
With disc D uploaded I had a few hours before the next and final round. So I
made my way to Prince George and did my favourite thing: industrial
sightseeing. Behind me is three pulp mills, an oil refinery, a chemical plant,
and a big railway yard, among other things.
![Prince George Selfie](/assets/2016-11-14/prince-george-selfie.jpg)
I had a couple hours to kill, so I decided to figure out how to get some good
photos of it all. I couldn't find a good angle that really showed the massive
scale of the industrial plants in the city. But I did figure out one thing,
it's much easier if you don't stand in front of the frame:
![Prince George Mill](/assets/2016-11-14/prince-george-mill.jpg)
### Last Disc
Eventually my turn finally came near. So I headed slightly north of Prince
George, to the rural area just outside the city core. I burned the DVDs next
to a big bank of mailboxes:
As usual, once the computation was started, I drove off. This disc was expected
to be just 15 minutes, so I soon stopped again by someone's house to burn the
final disc, disk F, and upload it... and then my internet connection stopped
working. Which was especially strange, as my phone wasn't showing any signal
issues, and it had been working perfectly well until the moment it quit.
So I drove back to the mailboxes I had just been at - a place where I had been
using the internet connection just fine - and tried it again... and it worked
briefly, only to stop working again a minute or two later.
Now, keep in mind that I was uploading the very last disc for the entire
ceremony; once I finished it was done. As Zooko said while I was trying to
> Where the hell are you and what kind of security precautions did you take?
Eventually I gave up and drove back to Price George city center - I figured at
worst I'd use the WiFi at the Tim Hortons. In the end I pulled into the parking
lot of the visitors center, the internet started working again, and the upload
> SUCCESS! CEREMONY COMPLETE!
After confirming everything one last time I finally turned off the compute
node, (hopefully!) destroying the "cryptographic toxic waste" forever.
Of course, I was far from finished; the ceremony wasn't worth much if I lost
the surveillance footage sitting on a big pile of micro SD cards. So I found
the nearest Staples and bought two external harddrives to start making copies.
I also needed to drive back to Vancouver, and I was sick of plateaus. With the
sun coming down I spent the rest of the day driving to McBride via the much
more scenic highway 16:
![Drive to McBride](/assets/2016-11-14/drive-to-mcbride.jpg)
## Monday 24th
So an interesting question is what should you do with the compute node
hardware after the ceremony is over?
If everything worked as planned, powering off the compute node should have
destroyed the secrets forever; the secrets should have been kept in only in RAM
and never written to any kind of non-volatile storage.
On the other hand, if things didn't work as planned, there could be a secret
still stored in some kind of non-volatile memory. It's possible, though
unlikely, that an attacker was able to backdoor the system in some way, but
wasn't able to exfiltrate the data. For example, an attacker who was able to
compromise the compute node via an exploit after the initial commitments may
have no way of exfiltrating the secret other than recovering the actual
So I argued that *some* of the stations should fully destroy their compute
nodes, while other stations should preserve them as evidence for later forensic
evaluation. I also argued that the stations with the highest chance of
compromise should be the ones that kept their nodes, to maximize the chances of
an attack being detected.
With all the precautions I took, I thought a successful attack on my station was
less likely than others, so I opted to destroy the hardware thoroughly.
### Destroying the Compute Node
Unfortunately, that's not very easy. The problem is essentially that bits are
incredibly small: cutting up a flash memory chip may be guaranteed to render it
unusable, but doing so won't actually destroy the data on it; in theory an
advanced attacker could still read the data off the chip by reattaching wires
in the right places. Sure, that's going to be incredibly difficult and
expensive, but we're talking about a secret that could potentially be worth
billions of dollars in the future.
EEPROM and Flash memory both work by trapping electrons; if you heat up the
silicon enough it stops acting as a semiconductor and conductivity increases,
allowing the trapped electrons to escape (among other things). Or at least,
that's my understanding of it: I'm no semiconductor engineer!
So that means I just chucked the laptop into a campfire right? Well no. For
starters, I knew this would all end up being public, so I'd be smart to figure
out what the rules and regulations were, and follow them:
I checked the BC Government's [website](http://www2.gov.bc.ca/gov/content/safety/wildfire-status/fire-bans-and-restrictions) to determine if any fire bans were
in effect where I was. I also reviewed their "[Backyard & Industrial Burning](http://www2.gov.bc.ca/assets/gov/public-safety-and-emergency-services/wildfire-status/fire-bans-and-restrictions/bcws_backyardburning.pdf)"
pamphlet and "[Ministry of the Environment Burning Requirements](http://www2.gov.bc.ca/assets/gov/environment/waste-management/garbage/fs_burning_req.pdf)" on that
same site. Given the latter, I decided to not "burn" the laptop per say, but
rather heat the relevant components of it to the point where all information
would be unrecoverable.
I checked that the laptop was [RoHS compliant](https://en.wikipedia.org/wiki/Restriction_of_Hazardous_Substances_Directive). This means that no lead or
other toxic materials were used in its construction.
I bought a propane torch, a respirator, a fire extinguisher, and a large
I drove to a clear-cut (lots of those in British Columbia!) and
found a part of that clear-cut where the topsoil had totally been removed
leaving just a big sandy pit. I think they had been using it to setup their
camp or something - there was a bunch of trash in a pile there as well, and
signs of previous fires:
I completely took apart the laptop and separated all electronic and
non-electronic components (in particular, that means all the plastic components
were removed). While technically "electronic", I removed the actual lithium
battery cells from the battery control electronics in the battery pack - I had
no desire to start a lithium fire!
With respirator on and cameras running, the electronic components were all put
in that big metal pan, and the blowtorch was used to methodically heat the
electronics completely piece by piece until everything was blackened. This was
done in entirely in the metal pan, against a rather absurd backdrop:
![Compute Node Remains](/assets/2016-11-14/compute-node-remains.jpg)
If anyone can recover data off these RAM chips, I'll be very impressed:
![Compute Node Ram Remains](/assets/2016-11-14/compute-node-ram-remains.jpg)
Finally, all of the electronics were removed from the site. Afterwords I bagged
them all up to preserve them as evidence; other than the lithium battery cells
I haven't actually thrown any part of the laptop away yet.
So what's next? A lot more work!
For me personally, after I finally destroyed the compute node I sealed up all
the evidence left over - the DVDs, the electronics used, the shielded box for
the compute node, etc. - for future forensics. I can think of a ton of
questions to ask, and I'm sure others have questions I haven't thought of.
I've also got hours worth of video footage and photos that I need to decide if
and how to release with potential privacy and maybe even legal issues to think
There's also lessons to be learned: I went into the ceremony knowing very
little about it, and having never done anything quite like it. If people make
use of zk-SNARKs, for the foreseeable future we're going to end up doing more of
these crazy ceremonies, so hopefully we can do a better job next time.
After all, next time, rather than there being *theoretically* hundreds of
millions of dollars on the line, next time their might *actually* be hundreds
of millions of dollars on the line; we're going to need to up our game.