Bitcoin Unlimited voting system

[awemany]

Is anyone working on this yet?

If not, I have a couple of ideas. Most importantly, I think it would be nice if all the voting could happen through a very simple command line tool and all BUIPs being in a git as the plumbing layer, of course with a web-frontend on top for those of us with a dislike for command line interaction.

This would make the whole evolution of the BU org a nicely documented and easy-to-verify process - and, IMO very importantly, impossible to hide behind arcane server setups and complexity.

@theZerg as current benevolent dictator and president would produce an initial text file of members (pubkey, nick, maybe-full-name).

And a couple scripts could then be used to vote and query and submit votes. Submission of a vote would be pull request to the BU master on github or so.

If no one knows a good tool that does that, I could start hacking a couple python scripts.

Django (I am somewhat familiar with that) or cmdline calling on the server side from JS/Meteor/whatever could then be used to implement the 'BU voting server'. Members could log in, sign the text to vote on their clients, drag/drop it into some text box and be done. The server could validate first by having a valid Bitcoin signature (that matches with the member list) and only then further submit an (automated) pull request.

Thoughts?

 

[TrevinHofmann]

I'm working on a refactor of the website now, and it should be ready by the end of the weekend. My goal is to include voting functionality. Users would be able to submit and vote on BUIPs using their Bitcoin or PGP keys, and the site would automatically validate the signatures.

 

[freetrader]

@awemany 's suggestion of a simple command line interface appeals to my UNIX side.

I would like to know where this idea/system went since then, it sounded interesting:
http://www.coindesk.com/bitcoin-foundation-blockchain-voting-system-controversy/

 

[awemany]

Announcing BUV voting software prototype

(Bitcoin Unlimited Voting)

Code here: https://github.com/awemany/buv

Ok, so I scratched an itch and implemented what I think should be the voting system for BU.

@theZerg's very valid complaint that some of the votes for BUIP0001 where of the unclear, messy variant 'maybe yes, but' and similar, as well as the potential problems of having all vote data stored on a centralized web server made me think that we IMO really want an automated way to record voting history for our organisation and make it easily available.

Data structures

From a bird's eye view, my idea is the following: Have a bunch of text files that refer to each other using SHA256 hashes which determine who is a member, who vote what where and also how our organisation progresses in terms of agreements.

I propose to (for now) define the following five fundamental data types:

Vote, Election, ProposalMetadata, ProposalText and MemberDict (membership list).

ProposalText is currently just the UTF-8 text of a proposal, such as a BUIP, plus a special tag 'META' that links (through a SHA256) to meta data on this proposal.

A MemberDict is a special variant/subclass of a ProposalText that contains a list of members of BU to be voted on.

The ProposalMetadata, linked by a ProposalText, contains things such as the voting method to be used on the proposal, as well as ordering information (such as invalidated, superseded proposals) or information on past and agreed-upon elections. It also refers to the member list that is eligible to vote on the proposal (or null for the 'genesis membership proposal' . The ProposalMetaData are the main nodes linking the rest of the structures together into a DAG.

A Vote is a Bitcoin-signed record saying that a member (identified by a handle and a Bitcoin address) voted with a certain ballot option (such as FOR, AGAINST) on a certain proposal, identified by its SHA256.

An Election is a collection of votes. The idea of the Election type is used to 'more permanently' record the result of an election on a specific proposal into the 'buv chain'. By voting on a BUIP that refers to meta data that contains a past Elections in its 'CONFIRMS' list, the voter is assumed to implicitely agree on the results of that particular election. (checks not yet implemented)

All data structures are represented as human readable text files on disk. In the linked git, please all have a look at the files in the testexample1/ directory. This is how I see we could formally store the results of our votes.

Of course, as files are just referring to each other by their hashes, any fancy directory structure could later be used to properly organize the data, without changing past votes/elections.

I also propose to put the votes and results into a git. (A further extension of buv might go through all history of a git to collect votes, which would make it easy to 'delete' failed BUIPs etc. in HEAD
(create a clean: 'this is our current state' directory), while also keeping all data available, but that's just an idea I pondered about ...

Software

The software currently reads all text files, determines their content type, and builds just one big data structure containing everything in memory. Then a couple of filters checking for validity are run, followed by tallying up the votes on all proposals. No work has been put into optimizing this yet.

And the set of tests and filters (and also the set of voting methods) is still very incomplete.

The software uses the fine pybitcointools library from V. Buterin et al., so 'pip install pybitcointools' is a requirement. Other than that, only the included python batteries are required. Python 2.7, as pybitcointools seems to be python2 only.

This is of course early test or prototype code (noting this should hopefully keep @Gavin Andresen off my back for a while . It likely contain bugs, lots of them.

If there is general agreement (should I write a BUIP?) that this is the way forward, I will extend the code so that we can do simple votes and make 'buv' a simple command line tool to create meta data, register proposals, and create vote texts (but I'd refrain from putting in the signing code and dealing with private keys, I think that's best left to whatever one wants to use for that).

This buv tool could then be used as the basic plumbing layer for all BU-specific documents and agreements and also for a more fancy voting website.

Of course, a more 'infrastructure' like mode of a JSON-exporting server or WSGI code for some framework or similar could also be used, and maybe those who are currently running the BU website might want to chime in. Or a webserver/static site generator could be made that replaces all SHA256s with links, for easier navigation.

Many things possible!

I welcome all kinds of constructive criticism. Also, note that English is not my native language, so if you think some stuff is named inadequately, please inform me.

And yes, help and patches are very much welcome, of course!

 

[TrevinHofmann]

That sounds like over-engineering to me, but I won't stop you if you'd like to set it up

 

[theZerg]

I think that this is great! While it might be overkill for BU, BU can serve as a laboratory to test these techniques for adoption by much larger organizations. And it would make our process very clean and simple. Go for it!

 

[TrevinHofmann]

@theZerg - After discussing it further with @awemany, we've decided to implement a common library for managing the organization. It will build a chain of atomic "transactions" such as BUIP submissions and votes. Each transaction will include a hash of the previous transaction and a signature by the user's PGP or Bitcoin key. I've started work on it here: https://github.com/thofmann/buip-lib

The plan is that I would then use that library on the website's server and make a simple interface for the website which allows members to submit BUIPs, vote, etc. @awemany would use the library to build a tool for members to independently verify the signatures using the chain of transactions which will be downloadable from the website.

Could you look into this pull request for the website? https://github.com/gandrewstone/BitcoinUnlimitedWeb/pulls

I'd like to know if the changes so far seem alright to you before continuing work on it.

Also, are the elections still planned for the 15th? It might be possible, but having the voting software fully implemented and tested by then will be a challenge.

 

[theZerg]

@TrevinHofmann -- I don't have time to look at the changes in detail right now, so I don't want to pull it into the web site in case something goes wrong. The changes are large enough that I really can't look at it from a diff perspective. I really need to just pull it into a cloned repo and work with it as a whole.

But I think that the approach is great, so please continue to work on it! If I have changes they will simply be to solve issues that may come up.

 

[TrevinHofmann]

Yes, the diff won't be very helpful. I'm not sure if there's a single file that wasn't touched. The actual contents of the site have changed very little, though. No problem - I understand that you're quite busy at the moment.

I'll try to get the voting system implemented and ready by the 15th.

 

[awemany]

@TrevinHofmann :

So I did have a look at your code now. Notes and questions:

- As far as I can see, this is still lacking all the vote counting and sig verification. Do you have the matching js libs for that?

- I think it would be nice if all JSON serializations contain a type specifier
of some sort, so that you can look up in the outer dict what it is.

- Where does the member join timestamp originate from? Do we want it/need it?
Who specifies it?

- I said that in the slack chat already, but I am uncertain about the status
of this:

Is serializing JSON in JS somehow specified to yield just one certain,
reproducible serialization?

Because I think that if it is not, we want to save the actual text of the JSON
and not the structure in some JSON DB. Plus its signature of course. Will the
sig be in JSON or just plain text? Plain text would be the output from the
user's signing tool.

For example, a signed vote using this nested string approach would look like this:


'{"data": "{\\"proposal_meta_hash\\": \\"1234abcd\\", \\"type\\": \\"Vote\\", \\"ballot_option\\": \\"FOR\\"}", "type": "SignedString", "addrsigs": {"12qeUAwuR8XjXRisS1PpJp7BW3RDxGWZqN": "HGXOkCPxT2+JUT/r5JyGhMWSm2n/49kiDxjD9yzyuxTgl5Hi9FxVsVQScs5iKo5VAYGX4mjKSM4AyWedgv+OjiA="}}'

Due to the necessary escaping, I consider that pretty ugly and unreadable, so maybe you have a suggestion on how to store this in a better way. Or someone else might have an idea?

I do like the idea of proposals having titles.

Are BUIPs going to be our only proposal types? In my code, I suggested member lists as another kind of proposal, and that would give one the automatic eligibility checks.
[doublepost=1452794855,1452793865][/doublepost]I think an o.k. option would be to use my vote format as the actual string that is signed, instead of (possibly ambiguously) serialized JSON. What is your opinion on doing that?

 

[awemany]

Ok, so here are my ideas on how the JSON could look like:

Variant1, Vote JSON is escaped into ASCII and then signed (votes are unreadably ugly):
https://github.com/awemany/buv/tree/json-examples/json-out-variant1

Variant2, Vote string that is signed is assembled by a schema specified in the code (not as transparent and easy-to-check as simply signing a string, but JSON otherwise readable):
https://github.com/awemany/buv/tree/json-examples/json-out-variant2

I like input from anyone on how we want to format votes.

Does that make sense? Could you parse that?

My JSON is still missing a title entry for the meta data (and the sigs and hashes are off, this is just for discussion. It is mostly a direct translation of the datastructures in testexample1/)

I also added a version number, in addition to a type specifier.

 

[awemany]

Thinking about this, it is probably best if votes explicitly mention both the proposal as well the proposal meta data (as hashes, at least).

 

[TrevinHofmann]

As far as I can see, this is still lacking all the vote counting and sig verification. Do you have the matching js libs for that?

I will be adding this. I only had time to add a few starting models so far, and even those are likely still missing a few properties.

I think it would be nice if all JSON serializations contain a type specifier
of some sort, so that you can look up in the outer dict what it is.

When you call a function such as `alice.toJSON()` where `alice` is an instance of Member, you will get back a JSON object containing all of Alice's attributes. There is no need to add a type field, because this should always be able to be inferred from context.

Actually, `toJSON` and `fromJSON` aren't even going to be useful for those types of objects. They will always be internally stored as instances in the program. The only thing that will need to have `toJSON` and `fromJSON` will be the `Transaction` class.

Where does the member join timestamp originate from? Do we want it/need it?
Who specifies it?

I suppose it is not necessary, but it seemed like a useful piece of metadata to have. It would probably come from the time of the new member BUIP proposal.

Is serializing JSON in JS somehow specified to yield just one certain,
reproducible serialization?

Because I think that if it is not, we want to save the actual text of the JSON
and not the structure in some JSON DB. Plus its signature of course. Will the
sig be in JSON or just plain text? Plain text would be the output from the
user's signing tool.

For example, a signed vote using this nested string approach would look like this:


'{"data": "{\\"proposal_meta_hash\\": \\"1234abcd\\", \\"type\\": \\"Vote\\", \\"ballot_option\\": \\"FOR\\"}", "type": "SignedString", "addrsigs": {"12qeUAwuR8XjXRisS1PpJp7BW3RDxGWZqN": "HGXOkCPxT2+JUT/r5JyGhMWSm2n/49kiDxjD9yzyuxTgl5Hi9FxVsVQScs5iKo5VAYGX4mjKSM4AyWedgv+OjiA="}}'

Due to the necessary escaping, I consider that pretty ugly and unreadable, so maybe you have a suggestion on how to store this in a better way. Or someone else might have an idea?

I don't quite understand the problem that you are outlining here.

JSON files look like this: https://github.com/thofmann/BitcoinUnlimitedWeb/blob/hofmann-develop/package.json

Users will not look at JSON. It's an internal representation of the data. A program parses the JSON and displays the values in an appropriate format for the user.

Are BUIPs going to be our only proposal types?

There will be a variety of BUIP types: election BUIPs, new member BUIPs, etc. I have not had time to write them all yet.

I think an o.k. option would be to use my vote format as the actual string that is signed, instead of (possibly ambiguously) serialized JSON. What is your opinion on doing that?

I've been thinking about this. I am not sure exactly what format it would look like yet, but it's going to be the "message" field of the `Transaction` object.

Ok, so here are my ideas on how the JSON could look like:

Variant1, Vote JSON is escaped into ASCII and then signed (votes are unreadably ugly):
https://github.com/awemany/buv/tree/json-examples/json-out-variant1

Variant2, Vote string that is signed is assembled by a schema specified in the code (not as transparent and easy-to-check as simply signing a string, but JSON otherwise readable):
https://github.com/awemany/buv/tree/json-examples/json-out-variant2

The messages do not need to be a single line. I think for readability, we can either pretty-print the JSON or use a different format entirely:

{
    "buipId": 5,
    "choice": "reject",
    "timestamp": 1452803871
}

----- BEGIN BITCOIN UNLIMITED BUIP VOTE -----
BUIP ID: 5
Choice: Reject
Timestamp: 1452803871
----- END BITCOIN UNLIMITED BUIP VOTE -----

I don't think the format matters much.

 

[awemany]

When you call a function such as `alice.toJSON()` where `alice` is an instance of Member, you will get back a JSON object containing all of Alice's attributes. There is no need to add a type field, because this should always be able to be inferred from context.

I don't see how it harms, though. And it would make the object self-contained.

I don't quite understand the problem that you are outlining here.

Ok. The problem is that signing a bunch of text (stream of octets) is a well-defined operation. There is no equivalent for signing a JSON in-memory tree. There is, of course, a way to sign serialized JSON (interpreting it as an octet stream and signing it). However, if you want transparency, you want something human-readable. And you definitely want to take out the details of the serializing algorithm out of the equation!
I reworked my code (see below) to be all JSON based now, and I simply sign the vote string in a predetermined, fixed 'serialization' format and put that into the JSON.

JSON files look like this: https://github.com/thofmann/BitcoinUnlimitedWeb/blob/hofmann-develop/package.json

Users will not look at JSON. It's an internal representation of the data. A program parses the JSON and displays the values in an appropriate format for the user.

I disagree to some extend. If you want more transparency, having a text based-format is certainly an advantage. However, I do agree that tools (website/buv) should be used to interact with the data, mostly.

There will be a variety of BUIP types: election BUIPs, new member BUIPs, etc. I have not had time to write them all yet.

Have a look at mine. They're done and they work.

I don't think the format matters much.

I agree, and that's I agreed to switch to JSON as discussed on slack. But, again, I think it should be human readable.

Having said all that, I just pushed my updated code which now reads and validates JSON files and does validation of votes, meta data, BUIPs and membership proposals.

I think this code is complete enough that it could (in principle) be used now do do an election. I'd like some code (and, more importantly, data structure) review, though.

The only caveat is that I have no tooling yet with a nicer way to produce the meta data, so that file would need to be written manually. I volunteer to do that for the first election. I'd again like everyone's opinion on this.

 

[Peter R]

@awemany, @TrevinHofmann: awesome to see you guys drive this project forward! I think it will be quite useful for many applications beyond just BU.

How close is it to being ready to use?

 

[awemany]

@Peter R.:
So it looks like we'll vote @theZerg into position of dev (or not) on this forum, with signed text votes, correct?

My code is good enough so that I could set up a git repo and then post the initial membership proposal + @theZerg's proposal there, with my proposed 'block-chain-like voting', but with the caveat that I still consider this a somewhat experimental mode. If at least some people start telling me that sounds good here in this thread, I'll go ahead and do that!

My code has a command line tool to do the vote counting and data structure validation (but additional validation checks should be implemented eventually). That can be done later, though.

My data structures are a little bit fancier than what @TrevinHofmann originally had in mind and the above post is my latest feedback for getting an agreement on how we do this.

I went and fully JSONified all of my structures now, so I hope he might use them directly on the web server.

If you want, you can have a look at my repo, the folder 'testexample1_template' contains the following:

Alice, Bob and Charlie agree to form a team and sign their initial membership list
Alice, Bob and Charlie vote on 'a BUIP1', with 2/3 against it
Alice, Bob and Charlie vote in new members Debbie and Eddie

I wrote code to 'compile' this template code into test code and then validate the voting results from this test example. That works.

Similar to the Bitcoin protocol, I consider data structures and protocol and agreement on them as more important than a particular implementation.

 

[TrevinHofmann]

@awemany - No, it seems we're voting without requiring signatures. I guess that's one part of the Articles already out the window

 

[Peter R]

Perhaps because @theZerg is acting as "benevolent dictator" for now, and because the voting system wasn't ready, signatures on votes for the election aren't yet required?

I think it is VERY IMPORTANT that we get a voting system working (one that's very easy to use!).
[doublepost=1452896341,1452895527][/doublepost]OK, I'm pasting all of my SIGNED votes here.

-----BEGIN BITCOIN SIGNED MESSAGE-----
YES (Andrew Stone for developer)
-----BEGIN SIGNATURE-----
1BWZe6XkGLcf6DWC3TFXiEtZmcyAoNq5BW
H5qDVm+4iU866taczm5fzlrtu+1SApUvVtBASl6cU9oaTlfvF+jZCvzwxlhIgpxKHowCdiuHj466uDyMGYe+Mvk=
-----END BITCOIN SIGNED MESSAGE-----

-----BEGIN BITCOIN SIGNED MESSAGE-----
Andrew Clifford (solex): VOTE
-----BEGIN SIGNATURE-----
1BWZe6XkGLcf6DWC3TFXiEtZmcyAoNq5BW
H6BdMG198vAC0/Cd151h6oSkef4iV61NitZfQHzZCiJzkh7E9QV83zD/0Zn5SUq53v2Nov7Cuq1B27/U24Ihzw0=
-----END BITCOIN SIGNED MESSAGE-----

-----BEGIN BITCOIN SIGNED MESSAGE-----
YES (Andrew Quentson for secretary)
-----BEGIN SIGNATURE-----
1BWZe6XkGLcf6DWC3TFXiEtZmcyAoNq5BW
H+wclJqaTdLzaj1A1Weu//AHgZunQltcXb2qunOgUeWegZGZxLNvJZVD/xUr1tKJzBS/J0PiT5u6VomBH6jg+lc=
-----END BITCOIN SIGNED MESSAGE-----

 

[theZerg]

Sigs are extremely important but I can't do everything at once. I'm expecting that the new secretary will make this a top priority...

 

[Lee Adams]

I blame the benevolent dictator for this. The actual clause is/was??:

"Members shall supply a Bitcoin public key when applying for membership and sign all formal communications with the corresponding private key."​

Opening the voting was the very definition of 'formal communication'.

I would also point out that there is no harm in signing your votes, even if that clause is suspended by dictact.