Various linux distribution support

[Jonathan Silverblood]

I've been wondering what level of linux distribution support one should expect from a project like BU for a while, but have felt a bit too selfish to request support for my little corner of the world - but given that I actually am one of the people that is using BU on a daily basis I've come back to this question a couple of times now and thought I'd guage the community and see what they think.

My question basically boils down to "should we spend BU resources to actively maintain various linux distributions, and if so - which ones?".

Before I go ahead and make some serious and fancy-looking BUIP about this I'd like to know what distributions people who run BU actually use, how well it works for them today and how much cost it would be to actually maintain such support.

I've started off by asking one of the people that is currently maintaining such support for the distribution of my choice here: https://gitlab.com/chaoslab/chaoslab-overlay/issues/83 - but if you are a linux user and run BU, could you share your experience as well? What works? what is quirky? what lengths do you think is good to go to ensure high-quality, well-maintained installation support for various distros?

 

[Griffith]

right now we only officially support deb. rpm has been requested but im unsure if we actually agreed to start supporting that without going and looking it up.

the most popular linux distros use one of those two package types, if u pick something more obscure it should probably be your responsibility to get the code to work for you

 

[Jonathan Silverblood]

I used to agree with this, but there's been two independent overlays with build instructions for BU on gentoo and out of ~700 public nodes I know of at least two of my own that is run on gentoo. if I assume worst-case scenario then there should be a minimum of 4 such nodes which makes us about 0.5% of the users.

It's low, yes, incredibly so, but what if the reasoning for these numbers are **precisely** because you only support deb?

I guess what I'm trying to say here is: "I think BU is important, and I don't want poorly maintained installation support to be the reason people go elsewhere"

And if I am to be completely honest, that was me some years ago, running the "official" bitcoind because I just couldn't get bitcoin unlimited to compile properly...

 

[Griffith]

I see your point. In this case the expansion of additional package types should probably come in order of their general popularity.

did you have a particular set in mind that you personally think should all be supported?

 

[Jonathan Silverblood]

I'm hoping to find out more by talking with people about what the costs involved are. I'm not actively engeged in the various deb/rpm/flatpack/whatever distributions though so I started close to home by researching what the costs for gentoo support would be.

If the cost is high, I'll drop the subject and move on, but I'm hoping that something as simple as having an official overlay repository to which the currently independent repo maintainers can contribute to might make the already-reasonably-good situation on gentoo a more pleasent experience.

If there's people here that say they've running ABC or XYZ and have a horrible experience though, then I think we should focus on the real pain points first, and your statement of going by general popularity is very much valid - it takes into account the people that **arent** users (yet).

 

[Griffith]

I can give you the general layout of what the process would have to be then,

currently all the developers (if they are using linux) are using ubuntu 18. if we are only concerned with being able to compile for additional package types we would probably just need to add more OS configs to travis so we can test the builds. This would involve probably some minor travis configuration adjustments as well as a PR or two which would iron out any nuances in the configure and build process for the new package. so probably not too much work

if we also plan to support those new package types with our precompiled binaries then in addition to the things previously mentioned we would have to adjust gitian. @sickpig would have to tell you more on how much work it is to adjust gitian, he is usually the one that does that stuff

 

[theZerg]

If a PR were to appear that provides support to create an installation package for some distro, I'd be inclined to merge it. Especially if that person were active here or on github. The RPM PR was kind of a "drive by" submission, so there's nobody to actively maintain or test it which is why it is languishing.

So probably no BUIP necessary, somebody just needs to do the work. Can you do it?

 

[Jonathan Silverblood]

> So probably no BUIP necessary, somebody just needs to do the work. Can you do it?

The ebuilds needed for gentoo exist in the chaoslab overlay repository, and I've submited and issue to ask them about how they'd feel about moving their BU related contributions to a BU controlled official repository instead.

I am here, and I'd be testing every release and can do basic maintainance (copy-pasting the buildfile and change version numbers, filename and checksum references) - but when things break I might not always have the time to dig down and try to debug the compilation issues.

Let me have some more time to talk with the chaoslab people - he/they already supports BU in practice by doing the work and has been keeping it in a decent state the last couple of years.

All this said - I am still interested in hearing others experiences - both success and failure, with trying to run BU on linux.

 

[Griffith]

@Jonathan Silverblood

i added rpm build instructions to the docs based on compiling using CentOS 7.4
when I have the time I will add a single rpm build to travis (no test suite, just compilation testing) as well.

 

[imaginary_username]

As people are bringing up package maintenance: Since there is already a Ubuntu PPA, how much work is there to make sure it's compatible with Debian (and perhaps add instructions for it)? I'd imagine it's not very much work, but I'm a newbie in packaging.

Paging @sickpig

 

[sickpig]

@imaginary_username packaging for Ubuntu PPA repo is pretty straight forward:

- clone the repo
- prepare the debian directory (changelog, rules, control files)
- tar it
- debuild -sa
- dput to launchpad (which will build and deploy the deb files for you)

Setting up a debian repo is quite more difficult:

- you need to install a lot more tools on the system where you are going to build the deb
- you are the one that is going to actually build the the binaries and build the debs
- you have to set up a sever where to run a http deamon used to serve the repo the repo
- you have to properly configure the files system and the web server to serve the file
- you need to set up the pgp conf to sign various packages and repo indexes.

Regardless I think this is a valuable goal because we could have just one repo to serve vboth ubuntu and debian.

Now that I think about it, @dagurval should have done it for XT in the past. He even shared with mt some doc on how to set it up...

 

[imaginary_username]

@dagurval can you chime in? If instructions are relatively straightforward I can even attempt myself, but of course it'll be more efficient to hand to already experienced folks...

 

[Griffith]

@Jonathan Silverblood it would be possible to add gentoo to the travis tests if you think that would help.
i added an rpm build on centos7 to them today. for reference: https://github.com/BitcoinUnlimited/BitcoinUnlimited/pull/1637

 

[Jonathan Silverblood]

@Griffith I'm sure the people that currently make build instructions for gentoo would appreciate if travis cought build errors early, so if that is a low-cost items I'd be happy if it was done.

 

[Griffith]

@Jonathan Silverblood I will add it to my todo list.

 

[Griffith]

@Jonathan Silverblood out of pure curiosity is there a specific reason you use gentoo?

 

[Jonathan Silverblood]

No, not gentoo per-se, that's just because it kindof works for me.

I will never resort to a binary distribution though, and the package manager in gentoo allows me proper control of the dependency graph by letting me control the compile-time configuration options which results in optional dependencies.

The motto of gentoo that "everything is the users choice", resonates with me.
My hardware - my choice.

I am encouraged to file a bug report to gentoo if it every does something for me that I didn't want it to, and it's fairly neutral so it's one of few distros today that don't bundle systemd for example.

I am also in love with the ability to start an installation of a package, pause it at-will, modify the source code, then let it continue. I've lost count on how many times I've cheated poor packaging / build instructions or statically removed suppport for things I don't like..

 

[Griffith]

thanks for the explanation, that actually sounds pretty desirable

 

[dagurval]

@dagurval can you chime in? If instructions are relatively straightforward I can even attempt myself, but of course it'll be more efficient to hand to already experienced folks...

There are lots of straightforward instructions that take you nowhere . Took me a while to figure out how to get working. If you want to work on setting up a debian repo I'll share my notes and try to help you problem solve any issues that you may run into.

 

[Griffith]

@Jonathan Silverblood the amount of time it takes to get from a fresh gentoo docker container to a "ready to compile bitcoin" state is too long to run every time it is needed in travis. if you are willing to build/maintain a docker container with the required deps intalled so the BU travis can simply pull it from docker hub, clone the repo and run the build/tests then I can add it to the travis suite.

If it should have only taken me a minute or two to get all the deps installed it is entirely possible that i went a roundabout way of doing everything since this was my first time using gentoo and i might need some help on that side of things